SEARCH · Engineering Papers
Results for “Data integrity”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Low Yield Nuclear Monitoring Physics Experiment 1 – Integrated Data Acquisition System Design and Initial Observations
The report documents the design of the Integrated Data AcQuisition (IDAQ) system and observations recorded during the first in a series of underground chemical explosions conducted on the Nevada National Security Site (NNSS) in southern Nevada. Experiments are funded as part of Low Yield Nuclear Monitoring (LYNM) research and development within the United States National Nuclear Security Administration NA-22 nuclear non-proliferation program. The series is part of the broader Physical Experiment 1 (PE1) being conducted in and around the P-tunnel facility on the NNSS. Each explosive experiment utilizes several tons of comp-B to generate signals recorded by a broad suite of instrumentation. The IDAQ serves as the backbone for all subsurface instrumentation providing precise time synchronization, remote control, data exfiltration and backup, along with recording several sensing modalities throughout the underground complex that includes ground motion, environmental conditions, and electromagnetic signals.
Bayesian Evaluation Framework for Imperfect Differential and Integral Data or Models [Slides]
This presentation is on Bayesian evaluation framework for imperfect differential and integral data or models. This presentation includes a particular focus on common assumptions used in Bayesian ND evaluations. Additionally, this presentation highlights the benefits of a generalized form of the Bayes' theorem.
Development of a Framework for Data Integration, Assimilation, and Learning for Geological Carbon Sequestration (DIAL-GCS) (Final Report)
This project aimed to develop and demonstrate a Data Integration, Assimilation, and Learning framework for geologic carbon sequestration projects (DIAL-GCS). DIAL-GCS is an intelligence monitoring system (IMS) for automating GCS closed-loop management by leveraging recent developments in machine learning technologies, complex event processing (CEP), and reduced-order modeling. The safe and efficient operation of GCS repositories requires integrated monitoring to track the injected CO¬2 as it moves within a storage reservoir. GCS projects are data intensive, as a result of proliferation of digital instrumentation and smart-sensing technologies. GCS projects are also resource intensive, often requiring multidisciplinary teams performing different monitoring, verification, accounting (MVA) tasks throughout the lifecycle of a project to ensure secure containment of injected CO2. The success of GCS thus depends in a large part on our ability to access, assimilate, and analyze heterogeneous data and information sources in a timely manner. This project included a number of meaningful and necessary tasks to transform the human domain knowledge into machine-interpretable rules for automating knowledge extraction and discovery in GCS. The specific technical objectives of the proposed DIAL-GCS project were to develop an ontology-driven GCS data management module for storing, querying, and exchanging GCS data (both historic and live sensor data) from multiple sources and in heterogeneous formats. Incorporate a CEP engine for detecting abnormal situations by seamlessly combining expert knowledge, rule-based reasoning, and machine learning. Enable uncertainty quantification and predictive analytics using a combination of coupled-process modeling, AI/ML methods, and reduced-order modeling, and integrate and demonstrate the system’s capabilities with both real and simulated data. As far as we know, this is one of the first projects aimed to develop intelligent monitoring systems (IMS) targeting the GCS. Under this project, the team had developed a large number of web applications and scientific algorithms that contribute the main theme of intelligent monitoring. The team has published more than a dozen peer reviewed papers and disseminated the research results at multiple technical meetings.
Data Integration Methods for Bayesian Inverse Problems
Master's Thesis for an MA in Mathematics at Arizona State University titled "Data Integration Methods for Bayesian Inverse Problems."
A data integration framework of additive manufacturing based on FAIR principles
Abstract Laser-powder bed fusion (L-PBF) is a popular additive manufacturing (AM) process with rich data sets coming from both in situ and ex situ sources. Data derived from multiple measurement modalities in an AM process capture unique features but often have different encoding methods; the challenge of data registration is not directly intuitive. In this work, we address the challenge of data registration between multiple modalities. Large data spaces must be organized in a machine-compatible method to maximize scientific output. FAIR (findable, accessible, interoperable, and reusable) principles are required to overcome challenges associated with data at various scales. FAIRified data enables a standardized format allowing for opportunities to generate automated extraction methods and scalability. We establish a framework that captures and integrates data from a L-PBF study such as radiography and high-speed camera video, linking these data sets cohesively allowing for future exploration. Graphical abstract
Toward Integrating Data Quality Assessments and Radiometer Uncertainty for Determining the Expanded Uncertainty of Three-Component Solar Radiation Measurements
This presentation provides an overview on integrating data quality assessments and radiometer uncertainty for determining the expanded uncertainty of three-component solar radiation measurements.
LSTM-Based Data Integration to Improve Snow Water Equivalent Prediction and Diagnose Error Sources
Accurate prediction of snow water equivalent (SWE) can be valuable for water resource managers. Recently, deep learning methods such as long short-term memory (LSTM) have exhibited high accuracy in simulating hydrologic variables and can integrate lagged observations to improve prediction, but their benefits were not clear for SWE simulations. Here we tested an LSTM network with data integration (DI) for SWE in the western United States to integrate 30-day-lagged or 7-day-lagged observations of either SWE or satellite-observed snow cover fraction (SCF) to improve future predictions. SCF proved beneficial only for shallow-snow sites during snowmelt, while lagged SWE integration significantly improved prediction accuracy for both shallow- and deep-snow sites. The median Nash–Sutcliffe model efficiency coefficient (NSE) in temporal testing improved from 0.92 to 0.97 with 30-day-lagged SWE integration, and root-mean-square error (RMSE) and the difference between estimated and observed peak SWE values d max were reduced by 41% and 57%, respectively. DI effectively mitigated accumulated model and forcing errors that would otherwise be persistent. Moreover, by applying DI to different observations (30-day-lagged, 7-day-lagged), we revealed the spatial distribution of errors with different persistent lengths. For example, integrating 30-day-lagged SWE was ineffective for ephemeral snow sites in the southwestern United States, but significantly reduced monthly-scale biases for regions with stable seasonal snowpack such as high-elevation sites in California. These biases are likely attributable to large interannual variability in snowfall or site-specific snow redistribution patterns that can accumulate to impactful levels over time for nonephemeral sites. These results set up benchmark levels and provide guidance for future model improvement strategies.
Computational tools and data integration to accelerate vaccine development: challenges, opportunities, and future directions
The development of effective vaccines is crucial for combating current and emerging pathogens. Despite significant advances in the field of vaccine development there remain numerous challenges including the lack of standardized data reporting and curation practices, making it difficult to determine correlates of protection from experimental and clinical studies. Significant gaps in data and knowledge integration can hinder vaccine development which relies on a comprehensive understanding of the interplay between pathogens and the host immune system. In this review, we explore the current landscape of vaccine development, highlighting the computational challenges, limitations, and opportunities associated with integrating diverse data types for leveraging artificial intelligence (AI) and machine learning (ML) techniques in vaccine design. We discuss the role of natural language processing, semantic integration, and causal inference in extracting valuable insights from published literature and unstructured data sources, as well as the computational modeling of immune responses. Furthermore, we highlight specific challenges associated with uncertainty quantification in vaccine development and emphasize the importance of establishing standardized data formats and ontologies to facilitate the integration and analysis of heterogeneous data. Through data harmonization and integration, the development of safe and effective vaccines can be accelerated to improve public health outcomes. Looking to the future, we highlight the need for collaborative efforts among researchers, data scientists, and public health experts to realize the full potential of AI-assisted vaccine design and streamline the vaccine development process.
Enhancing Streamflow Forecast and Extracting Insights Using Long-Short Term Memory Networks With Data Integration at Continental Scales
Recent observations with varied schedules and types (moving average, snapshot, or regularly spaced) can help to improve streamflow forecasts, but it is challenging to integrate them effectively. Based on a long short-term memory (LSTM) streamflow model, we tested multiple versions of a flexible procedure we call data integration (DI) to leverage recent discharge measurements to improve forecasts. DI accepts lagged inputs either directly or through a convolutional neural network unit. DI ubiquitously elevated streamflow forecast performance to unseen levels, reaching a record continental-scale median Nash-Sutcliffe Efficiency coefficient value of 0.86. Integrating moving-average discharge, discharge from the last few days, or even average discharge from the previous calendar month could all improve daily forecasts. Directly using lagged observations as inputs was comparable in performance to using the convolutional neural network unit. Importantly, we obtained valuable insights regarding hydrologic processes impacting LSTM and DI performance. Before applying DI, the base LSTM model worked well in mountainous or snow-dominated regions, but less well in regions with low discharge volumes (due to either low precipitation or high precipitation-energy synchronicity) and large interannual storage variability. DI was most beneficial in regions with high flow autocorrelation: it greatly reduced baseflow bias in groundwater-dominated western basins and also improved peak prediction for basins with dynamical surface water storage, such as the Prairie Potholes or Great Lakes regions. However, even DI cannot elevate performance in high-aridity basins with 1-day flash peaks. Despite this limitation, there is much promise for a deep-learning-based forecast paradigm due to its performance, automation, efficiency, and flexibility.
The Role of Snowmelt and Subsurface Heterogeneity in Headwater Hydrology of a Mountainous Catchment in Colorado: A Model‐Data Integration Approach
Mountainous headwater streams are sustained by both snowmelt‐driven streamflow and groundwater discharge in the Upper Colorado River Basin. However, predicting headwater stream discharge magnitude and peak flow timing is challenging in mountainous terrains, where snowmelt rates vary with vegetation type and elevation, and heterogeneous subsurface physical properties influence groundwater storage and its release. We used a model‐data integration approach to investigate the roles of snowmelt and subsurface structure in stream discharge and groundwater level. We ran an ensemble of 100 integrated surface‐subsurface hydrologic models for a mountainous headwater catchment near Crested Butte, Colorado, USA. We also evaluated and calibrated these models against observed data sets, including snow depth measurements using distributed temperature probes, stream discharge, and groundwater levels. Calibration with multiple data sources using neural density estimators has further constrained uncertainty in subsurface properties and snowmelt rates. Results indicated that observed slower snowmelt rates in evergreen forests delayed the peak flow and baseflow onset. In upstream areas with lower subsurface permeability, water was stored within the subsurface but was not released as interflow or shallow groundwater flow, and thereby not contributing to downstream streamflow during recession limb periods. Double peaks in groundwater occurred in areas with spatial subsurface heterogeneity, in our case due to the contrast between granodiorite and Mancos shale. These process‐based insights into groundwater and snowmelt dynamics in mountainous headwaters will help improve predictions of headwater hydrology.
Data Integration and Visualization for Enhanced Resilience and Sustainability in Hydropower (DIVERS-H)
U.S. hydropower plants face potential threats from shrinking water supply, rising demands, and warmer stream temperatures from various causes. Power plant owners, operators, and regulators require new tools to take advantage of and interpret the diverse range of scientific data being produced by both observational methods (for example, satellite, radar, stream gauges) and computer modeling methods that evaluate and predict how earth's dynamic systems (atmosphere, oceans, land surface, and sea ice) are changing and interacting. Combining datasets such as these with AI-based analyses introduces a novel decision support system to help users anticipate and address potential impacts on power generation stations. This new technology has been named DIVERS-H for "Data Integration and Visualization for Enhanced Resilience and Sustainability in Hydropower." In Phase I, technical feasibility was established with the development and demonstration of all the new technologies that are required. Most notably, DIVERS-H will use new artificial intelligence (AI) methods to capture the complex dynamics of water availability, demand, and environmental changes. In addition, new data management software was developed, and a prototype user interface was implemented as the precursor to a full scale decision support system. With technical research complete, the project focus now shifts to development of a commercial software product to provide users with actionable insight into water availability and the risk/resilience of critical systems at their locations of interest. Although DIVER-H was originally conceived as a tool for hydroelectric power applications, the same underlying technology can be readily applied to other water-consuming systems including coal, natural gas, oil, and nuclear power plants.
Accelerating nuclear-integrated data center pursuits in the USA: SWOT analysis, power-thermal management strategies and demonstration plan
Here, this study explores the increasing interest in leveraging nuclear power to meet the escalating energy demands of data centers in the United States (U.S.) by focusing on key factors that contribute to accelerated deployment. The study highlights the importance of N+1/N+2 power supplies (where N is the required number of units), outlines research and innovations in nuclear-integrated data center thermal management and demonstration plan. It also provides updates about status and costing of various reactor system designs. A summarized strengths, weaknesses, opportunities, and threats (SWOT) analysis shows the potential options for grid connectivity, reactors, and site selection. Suitable site discussions consider land and water availability, grid access, and optical fiber connectivity, and the study presents graded prospects for Department of Energy (DOE) sites with a specific example. Community engagement and partnerships are emphasized, particularly the roles of local government, federal agencies, utilities, and data center industry partners, which are crucial for accelerating deployment, business outreach, and approvals. The study provides actionable insights for stakeholders to accelerate the deployment of nuclear-powered data centers.
Machine learning-enabled model-data integration for predicting subsurface water storage
Subsurface water storage (SWS) is a key variable of the climate system and a storage component for precipitation and radiation anomalies, inducing persistence in the climate system. It plays a critical role in climate-change projections and can mitigate the impacts of climate change on ecosystems. However, because of the difficult accessibility of the underground, hydrologic properties and dynamics of SWS are poorly known. Direct observations of SWS are limited, and accurate incorporation of SWS dynamics into Earth system land models remains challenging. We propose a machine learning-enabled model-data integration framework to improve the SWS prediction at local to conus scales in a changing climate by leveraging all the available observation and simulation resources, as well as to inform the model development and guide the observation collection. The accurate prediction will enable an optimal decision of water management and land use and improve the ecosystem's resilience to the climate change.
A CIM Based Data Integration Framework for Distribution Utilities
With the proliferation of distributed energy resources and advanced metering, modern electric power distribution systems are data rich and include advanced capabilities for distribution automation. Distribution utilities need applications for planning and operations that can integrate all the available data from the enterprise applications and may incorporate distributed approaches to operate and control. This paper describes a framework for standardizing and integrating the data available at different vendor specific applications in a utility utilizing the Common Information Model (CIM). Theframework incorporates the conversion of these the standardized CIM models into models compatible with GridLAB-D, an open source distribution system simulator, for developing advanced planning and operation strategies.
Battery data integrity and usability: Navigating datasets and equipment limitations for efficient and accurate research into battery aging
A tremendous commitment of resources is needed to acquire, understand and apply battery data in terms of performance and aging behavior. There are many state of performance (SOP) and state of health (SOH) metrics that are useful to guide alignment of batteries to end-use, yet how these metrics are measured or extracted can make the difference between usable, valuable datasets versus data that lacks the necessary integrity to meet baseline confidence levels for SOP/SOH quantification. This work will speak to 1) types of data that support SOP and SOH evaluations on mechanistic terms, 2) measurement conditions needed to assure high data integrity, 3) equipment limitations that can compromise data high fidelity, and 4) the impact of cell polarization on data quality. A common goal in battery research and field use is to work from a data platform that supports economical paths of data capture while minimizing down-time for battery diagnostics. An ideal situation would be to utilize data obtained during normal daily use (“pulses or cycles of convenience”) without stopping the daily duty cycles to perform dedicated SOP/SOH diagnostic routines. However, difficulties arise in trying to make use of daily duty cycle data (denoted as cycle-by-cycle, CBC) that underscores the need for standardization of conditions: temperature and duty cycles can vary over the course of a day and throughout a week, month and year; polarization can develop within an immediate cycle and throughout successive cycles as a hysteresis. If CBC data is envisioned as a data source to determine performance and aging trends, it should be recognized that polarization is a frequent consequence of CBC and thus makes it difficult to separate reversible and irreversible components to metrics such as capacity loss and resistance increase over aging. Since CBC conditions can have a major impact on data usability, we will devote part of this paper to CBC data conditioning and management. Differential analyses will also be discussed as a means to detect changing trends in data quality. Our target cell chemistries will be lithium-ion types NMC/graphite and LMO/LTO.
sciCAN: single-cell chromatin accessibility and gene expression data integration via cycle-consistent adversarial network
The boom in single-cell technologies has brought a surge of high dimensional data that come from different sources and represent cellular systems from different views. With advances in these single-cell technologies, integrating single-cell data across modalities arises as a new computational challenge. Here, we present an adversarial approach, sciCAN, to integrate single-cell chromatin accessibility and gene expression data in an unsupervised manner. We benchmarked sciCAN with 5 existing methods in 5 scATAC-seq/scRNA-seq datasets, and we demonstrated that our method dealt with data integration with consistent performance across datasets and better balance of mutual transferring between modalities than the other 5 existing methods. We further applied sciCAN to 10X Multiome data and confirmed that the integrated representation preserves biological relationships within the hematopoietic hierarchy. Finally, we investigated CRISPR-perturbed single-cell K562 ATAC-seq and RNA-seq data to identify cells with related responses to different perturbations in these different modalities.
Integrating data types to estimate spatial patterns of avian migration across the Western Hemisphere
For many avian species, spatial migration patterns remain largely undescribed, especially across hemispheric extents. Recent advancements in tracking technologies and high-resolution species distribution models (i.e., eBird Status and Trends products) provide new insights into migratory bird movements and offer a promising opportunity for integrating independent data sources to describe avian migration. Here, we present a three-stage modeling framework for estimating spatial patterns of avian migration. First, we integrate tracking and band re-encounter data to quantify migratory connectivity, defined as the relative proportions of individuals migrating between breeding and nonbreeding regions. Next, we use estimated connectivity proportions along with eBird occurrence probabilities to produce probabilistic least-cost path (LCP) indices. In a final step, we use generalized additive mixed models (GAMMs) both to evaluate the ability of LCP indices to accurately predict (i.e., as a covariate) observed locations derived from tracking and band re-encounter data sets versus pseudo-absence locations during migratory periods and to create a fully integrated (i.e., eBird occurrence, LCP, and tracking/band re-encounter data) spatial prediction index for mapping species-specific seasonal migrations. To illustrate this approach, we apply this framework to describe seasonal migrations of 12 bird species across the Western Hemisphere during pre- and postbreeding migratory periods (i.e., spring and fall, respectively). We found that including LCP indices with eBird occurrence in GAMMs generally improved the ability to accurately predict observed migratory locations compared to models with eBird occurrence alone. Using three performance metrics, the eBird + LCP model demonstrated equivalent or superior fit relative to the eBird-only model for 22 of 24 species–season GAMMs. In particular, the integrated index filled in spatial gaps for species with over-water movements and those that migrated over land where there were few eBird sightings and, thus, low predictive ability of eBird occurrence probabilities (e.g., Amazonian rainforest in South America). This methodology of combining individual-based seasonal movement data with temporally dynamic species distribution models provides a comprehensive approach to integrating multiple data types to describe broad-scale spatial patterns of animal movement. Further development and customization of this approach will continue to advance knowledge about the full annual cycle and conservation of migratory birds.