Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Staging”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

An overview of data tools for representing and managing building information and performance data

Building information modeling (BIM) has been widely adopted for representing and exchanging building data across disciplines during building design and construction. However, BIM's use in the building operation phase is limited. With the increasing deployment of low-cost sensors and meters, as well as affordable digital storage and computing technologies, growing volumes of data have been collected from buildings, their energy services systems, and occupants. Such data are crucial to help decision makers understand what, how, and when energy is consumed in buildings—a critical step to improving building performance for energy efficiency, demand flexibility, and resilience. However, practical analyses and use of the collected data are very limited due to various reasons, including poor data quality, ad-hoc representation of data, and lack of data science skills. To unlock value from building data, there is a strong need for a toolchain to curate and represent building information and performance data in common standardized terminologies and schemas, to enable interoperability between tools and applications. This study selected and reviewed 24 data tools based on common use cases of data across the building life cycle, from design to construction, commissioning, operation, and retrofits. The selected data tools are grouped into three categories: (1) data dictionary or terminology, (2) data ontology and schemas, and (3) data platforms. The data are grouped into ten typologies covering most types of data collected in buildings. This study resulted in five main findings: (1) most data representation tools can represent their intended data typologies well, such as Green Button for smart meter data and Brick schema for metadata of sensors in buildings and HVAC systems, but none of the tools cover all ten types of data; (2) there is a need for data schemas to represent the basis of design data and metadata of occupant data; (3) standard terminologies such as those defined in BEDES are only adopted in a few data tools; (4) integrating data across various stages in the building life cycle remains a challenge; and (5) most data tools were developed and maintained by different parties for different purposes, their flexibility and interoperability can be improved to support broader use cases. Finally, recommendations for future research on building data tools are provided for the data and buildings community based on the FAIR principles to make data Findable, Accessible, Interoperable, and Reusable.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2025

Knowledge graph-aided Bayesian active learning for top- K genetic interaction discovery

In silico methods for predicting the effects of multi-gene perturbations hold great promise for advancing functional genomics, computational drug discovery, and disease modeling. However, the development of these predictive algorithms for mammalian systems has been hampered by limited datasets and high experimental costs. In this study, we present a Bayesian active learning framework designed to discover pairwise host gene knockdowns that effectively inhibit viral proliferation in an in vitro HIV-1 infection model. Our method leverages a biological knowledge graph as side information and employs a computationally efficient batch diversification approach. We evaluated this framework using a dataset of viral load measurements obtained from multi-day dual-gene depletion experiments, encompassing all possible pairwise knockdowns of over 350 host genes associated with HIV infection. We demonstrate that our framework rapidly identifies the most effective gene knockdown pairs for reducing viral load. Furthermore, we show that incorporating side information enhances performance during the early stages of active learning (low data regime), while our batch diversification strategy significantly boosts performance in later stages (high data regime). This framework is general and can be adapted to explore gene interactions in other contexts, such as synthetic lethality prediction and mapping epistatic effects across quantitative trait loci.

Computational biology and bioinformatics↗
DOE OSTI2021

SQL and NoSQL Databases for Cyber Physical Production Systems in Internet of Things for Manufacturing (IoTfM)

Abstract In this paper, the design and performance differences between Relational Database Management Systems (RDBMS) and NoSQL Database Systems are examined, with attention to their applicability for real-world Internet of Things for manufacturing (IoTfM) data. While previous work has extensively compared SQL and NoSQL for both generalized and IoT uses, this work specifically examines the tradeoffs and performance differences for manufacturing applications by using a high-fidelity data set collected from a large US manufacturing firm. Growing an IoT system beyond the pilot stage requires scalable data storage; this work seeks to determine the impact of selected database systems on data write performance at scale. Payload size and message frequency were used as the primary characteristics to maintain model fidelity in simulated clients. As the number of simulated asset clients grow, the data write latency was calculated to determine how both database systems’ performance were affected. To isolate the RDBMS and NoSQL differences, a cloud environment was created using Amazon Web Services (AWS) with two identical data ingestion pipelines: writing data to an RDMBS (1) using AWS Aurora MySQL, and (2) using AWS DynamoDB NoSQL. The findings may provide guidance for further experimentation in large-scale manufacturing IoT implementations.

Gamero, David↗
DOE OSTI2020

ATLAS data quality operations and performance for 2015–2018 data-taking

The ATLAS detector at the Large Hadron Collider reads out particle collision data from over 100 million electronic channels at a rate of approximately $100$ kHz, with a recording rate for physics events of approximately 1 kHz. Before being certified for physics analysis at computer centres worldwide, the data must be scrutinised to ensure they are clean from any hardware or software related issues that may compromise their integrity. Prompt identification of these issues permits fast action to investigate, correct and potentially prevent future such problems that could render the data unusable. This is achieved through the monitoring of detector-level quantities and reconstructed collision event characteristics at key stages of the data processing chain. This paper presents the monitoring and assessment procedures in place at ATLAS during 2015-2018 data-taking. Through the continuous improvement of operational procedures, ATLAS achieved a high data quality efficiency, with 95.6% of the recorded proton-proton collision data collected at $\sqrt{s}=13$ TeV certified for physics analysis.

43 PARTICLE ACCELERATORS↗
DOE OSTI2020

White-light interferometry for early-stage metal oxide growth characterization

Oxidation of metals can provide a functional oxide film or an oxide layer that can lead to material degradation causing performance limiting concerns. Because of the technological and economic impacts of oxidation, understanding early-stage oxidation growth and its mechanisms is of high importance. Unfortunately, poor empirical data during early-stage oxidation has left our current understanding of oxidation incomplete and woefully inadequate. Measurement techniques previously used for oxidation kinetics characterization include microbalance, x-ray diffraction, Fourier-transform infrared spectroscopy, and spectroscopic ellipsometry. However, these techniques lack the necessary time-resolution or vertical resolution in addition to not providing any spatial data, which is essential to early-stage oxidation studies to understand both oxidation kinetics and mechanisms. Here, a novel rapid and surface-sensitive data collection technique that provides three-dimensional spatial data of a growing oxide layer with sub-nanometer thickness resolution using white-light interferometry has been developed. This technique will be the first technique capable of providing real-time spatial results of early-stage oxidation growth. The effectiveness of this characterization tool is shown by observing the early-stage oxidation kinetics and film cracking of uranium heated to 90 °C under 500 torr of pure O 2 . Initially a parabolic oxidation growth is observed. Once the oxide layer reaches ≈ 30 nm, the epitaxial stresses between the uranium substrate and oxide film layer causes localized cracking in addition to the observed transition of the parabolic growth rate to a linear growth rate. The results of the white-light interferometry are verified by glancing incidence x-ray diffraction.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

An integrated approach to understanding RF vacuum arcs

Although used in the design and costing of large projects such as linear colliders and fusion tokamaks, the theory of vacuum arcs and gradient limits is not well understood. Almost 120 years after the isolation of vacuum arcs, the exact mechanisms of the arcs and the damage they produce are still being debated. We describe our simple and general model of the vacuum arc that can incorporate all active mechanisms and aims to explain all relevant data. Our four stage model, is based on experiments done at 805 MHz with a variety of cavity geometries, magnetic fields, and experimental techniques as well as data from Atom Probe Tomography and failure analysis of microelectronics. The model considers the trigger, plasma formation, plasma evolution and surface damage phases of the RF arc. This paper also examines how known mechanisms can explain the observed sharp field dependence, fast breakdown times and observed surface damage. We update the model and discuss new features while also pointing out where new data would be useful in extending the model to a wider range of frequencies.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

The Dynamic Networks Experiments: Virtual Experiments to Quantify Gains in Nuclear Explosion Monitoring

We describe an ongoing series of virtual experiments conducted collaboratively by four United States National Laboratories: Sandia National Laboratories, Los Alamos National Laboratory, Lawrence Livermore National Laboratory, and Pacific Northwest National Laboratory. These Dynamic Network Experiments (DNEs) provide an experimental framework to evaluate the potential impact of new research tools on nuclear explosion monitoring. The second DNE (DNE2), completed in 2024, exploited waveform data (seismic, infrasound, and electromagnetic) that was recorded by multi-modal sensors within and near the Nevada National Security Site and synthetic radionuclide signatures over multiple time periods. During the execution of DNE2, we processed and analyzed data through a multi-stage event processing pipeline that ingested raw data, performed quality control, detected signals, built events from these signals, located these events, and characterized the events’ source types and sizes. For each stage and over the entire event processing pipeline, we evaluated performance changes by comparing the performance of new data processing methods, models, and algorithms against a baseline. We also performed an additional execution phase to assess event processing pipeline function, speed, and efficiency against that of an expert analyst, including computational and manual efforts. Finally, we assessed the impact and effort of modern computing infrastructure on the monitoring pipeline. This paper describes key elements of the DNEs, from formulation through execution, as demonstrated in DNE2. The DNEs introduce several novel concepts to quantitatively measure the potential impact of new methods on explosion monitoring, including the collaborative design of multi-modal datasets, performance and logistical metrics, and integrated analyses.

42 ENGINEERING↗
DOE OSTI2026

From Raw to Curated Data: A Lakehouse Approach for Scientific Workflows

This report provides a technical overview of how to go from raw to curated data in three stages using a lakehouse approach. We focus on the application of open source tools in scientific use cases (while noting parallels to enterprise and commercial alternatives). Our goal is to provide scientific data managers and infrastructure providers with a common frame of reference for understanding and applying modern lakehouse technologies and approaches.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2022

Simurgh: A Framework for Cad-Driven Deep Learning Based X-Ray CT Reconstruction

High-resolution X-ray computed tomography (XCT) is an important technique for the inspection of additively manufactured (AM) parts. While XCT is typically used off-line to inspect a subset of manufactured parts, significantly accelerating measurement speed while retaining accuracy would enable use of XCT for in-line inspection to rapidly identify defects in each part as it is manufactured. Here, we propose a deep learning (DL) based approach that uses computer aided design (CAD) models of the AM parts and physics-based information to rapidly produce high-quality reconstructions from sparse XCT measurements without high quality ground truth data. Our approach uses a generative adversarial neural network (GAN) to produced realistic training data from the CAD-based simulations and a deep neural network that is trained using data from the first stage to produce accurate 3D reconstructions. Using experimental XCT data of metal parts, we demonstrate enhanced defect detection capabilities while dramatically reducing the scan time.

Ziabari, Amir↗
DOE OSTI2022

A Bayesian Approach for Quantifying Data Scarcity when Modeling Human Behavior via Inverse Reinforcement Learning

Computational models that formalize complex human behaviors enable study and understanding of such behaviors. However, collecting behavior data required to estimate the parameters of such models is often tedious and resource intensive. Thus, estimating dataset size as part of data collection planning (also known as Sample Size Determination) is important to reduce the time and effort of behavior data collection while maintaining an accurate estimate of model parameters. In this paper, we present a sample size determination method based on Uncertainty Quantification (UQ) for a specific Inverse Reinforcement Learning (IRL) model of human behavior, in two cases: 1) pre-hoc experiment design—conducted in the planning stage before any data is collected, to guide the estimation of how many samples to collect; and 2) post-hoc dataset analysis—performed after data is collected, to decide if the existing dataset has sufficient samples and whether more data is needed. Here, we validate our approach in experiments with a realistic model of behaviors of people with Multiple Sclerosis (MS) and illustrate how to pick a reasonable sample size target. Our work enables model designers to perform a deeper, principled investigation of effects of dataset size on IRL.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

In situ probing of interfacial roughness and transient phases during ceramic cold sintering process

The ceramic cold sintering process (CSP) offers an eco-friendly approach to producing fully dense ceramics at low temperatures. However, an incomplete mechanistic understanding hinders its optimization and widespread adoption. In this study, we analyze the microstructural and structural changes in ZnO, a model CSP system, using in situ synchrotron-based high-energy small-angle X-ray scattering and X-ray diffraction techniques. Our results reveal the time evolution of ZnO particles' surface area and roughness, reflecting the dissolution and re precipitation processes that enable densification. The in situ measurements supply valuable kinetic data for these stages of CSP. Alongside microstructural changes and densification, we observed the evolution of secondary phases representing reaction products between ZnO and acetic acid, the solvent used. The initial ZnO/solvent mixture's dominant secondary phase is attributed to zinc acetate, which is gradually replaced by a zinc soap-type structure during CSP. This structure has a large (≈ 21 Å) lattice parameter and is assumed to have a layered nature. The formation of this soap phase, which is retained in the sintered product as an intergranular component, appears to be a signature of successful cold sintering as it facilitates mass transport, leading to densification. Here, our study underscores the potential of in situ synchrotron characterization for revealing microstructural and phase-evolution details during CSP. These findings, which would be challenging to obtain through ex situ measurements, provide crucial data to guide and validate theoretical models, ultimately enhancing CSP's effectiveness and adoption.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Dual Channel Dual Staging: Hierarchical and Portable Staging for GPU-Based In-Situ Workflow

In-situ workflows have emerged as an attractive approach for addressing data movement challenges at very large scales. Since GPU-based architectures dominate the HPC landscapes, porting these in-situ workflows, and, specifically, the inter-application data exchange, to GPU-based systems can be challenging. Technologies such as GPUDirect RDMA (GDR), which is typically used for I/O in GPU applications as an optimization that circumvents the CPU overhead, can be leveraged to support bulk data exchanges between GPU applications. However, current GDR design often lacks performance portability across HPC clusters built with different hardware configurations. Furthermore, the local CPU may also be effectively used as an auxiliary communication mechanism to offload data exchanges. In this paper, we present a dual channel dual staging approach for efficient, scalable, and performance-portable inter-application data exchange for in-situ workflows. This approach exploits the data access pattern within in-situ workflows along with the inherent execution asynchrony to accelerate data exchanges and, at the same time, improve performance portability. Specifically, the dual channel dual staging method leverages both the local CPU and the remote data staging server to build a hierarchical joint staging area and uses this staging area to transform blocking inter-application bulk data exchanges into best-effort local data movements between GPU and CPU. The dual channel dual staging is implemented as a portability extension of the Dataspaces-GPU staging framework. We present an experimental evaluation of its performance, portability, and scalability using this implementation on three leadership GPU clusters. The evaluation results demonstrate that the dual channel dual staging method saves up to 75% in data-exchange time compared to host-based, GDR, and alternate portable designs, while maintaining scalability (up to 512 GPUs) and performance portability across the three platforms.

Zhang, Bo [University of Utah]↗
DOE OSTI2022

Influence of design and operating parameters for additively manufactured intensified packing devices on CO 2 -Absorption column cooling and capture efficiency

Solvent-based CO 2 absorption is technologically a matured CO 2 capture pathway but suffers from: high regeneration energy demand, and solvent temperature rise and decreased capture efficiency caused by the heat of reaction. While research has focused on developing non-aqueous and low-aqueous solvents for decreasing the regeneration energy, the temperature bulge due to exothermic absorption is typically dealt with by cooling the solvent with an external inter-stage heat-exchanger. This approach may increase the overall process footprint, as well as capital and operating costs. Here, the current study explores a process intensification approach by incorporating inside the column an additively manufactured intensified packing device that consists of corrugated plates and internal coolant channels. The corrugated plates provide surface area for mass transfer between gas and liquid, while cooling fluid inside the internal channels removes heat from the exothermic reactive system. Two different intensified devices with specific surface areas of 266 m 2 /m 3 and 359 m 2 /m 3 were designed, manufactured, and tested inside a 2.06-m long and 0.203-m diameter column packed with commercial Mellapak 250Y packing. The device with a lower surface area showed up to 27 % reduction in cooling performance. A steady-state heat transfer model provides good agreement with the experimental column temperature and intra-stage heat removal data. Although a decrease in heat transfer performance was observed for the device with lower surface area, CO 2 capture experiments performed with simulated flue gas and low-aqueous solvent demonstrated that both intensified devices lead to a similar improvement of 12 % in capture efficiency. This study provides an understanding on simplifying the intensified packing device geometry and decreasing the device fabrication cost by 25 % without compromising with the CO 2 absorption performance of the packed column.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Biomass char gasification kinetic rates compared to data, including ash effects

This investigation provides both experimental and theoretical analyses of biomass char gasification kinetics over 1150–1350 °C in the environment of CO 2 , H 2 O and combinations of these reactants. The experimental data include continuous measurements of particle mass, external and internal temperatures, shape, and size with periodic but not continuous measurements of char porosity, pore size distribution, and pore surface area. The model describes heterogeneous char gasification for poplar wood, corn stover and switchgrass in different shapes and industrially relevant sizes (12.5 mm and smaller). Nonlinear least-squares regression produces optimized power-law model parameters for global kinetics that describe biomass char gasification with respect to both CO 2 and H 2 O separately and in combination. A single set of kinetic parameters describes reaction rates for all three fuels. Model simulations agree with measured data at all stages of char conversion even though the observed mass loss rates change significantly with conversion. In conclusion, the change in rate with conversion depends strongly on ash content, and this investigation provides a simple and theoretically based reaction rate expression that includes the effect of inert ash occupying an increasing portion of the reactive surface with increased conversion.

09 BIOMASS FUELS↗
DOE OSTI2021

Unsupervised learning for identifying events in active target experiments

This article presents novel applications of unsupervised machine learning methods to the problem of event separation in an active target detector, the Active-Target Time Projection Chamber (AT-TPC). The overarching goal is to group similar events in the early stages of the data analysis, thereby improving efficiency by limiting the computationally expensive processing of unnecessary events. The application of unsupervised clustering algorithms to the analysis of two-dimensional projections of particle tracks from a resonant proton scattering experiment on 46 Ar is introduced. We explore the performance of autoencoder neural networks and a pre-trained VGG16 Simonyan and Zisserman (2015) convolutional neural network. We study clustering performance on both data from a simulated 46 Ar experiment, and real events from the AT-TPC detector. We find that a -means algorithm applied to simulated data in the VGG16 latent space forms almost perfect clusters. Additionally, the VGG16+-means approach finds high purity clusters of proton events for real experimental data. Here, we also explore the application of clustering the latent space of autoencoder neural networks for event separation. While these networks show strong performance, they suffer from high variability in their results.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗