Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Staging”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Artificial Intelligence for Data Center Operations (AIOps): Cooperative Research and Development (Final Report)

High performance computing data centers will increasingly need to rely on automation to keep pace with exascale growth in compute capability and to manage and optimize the data center environment and facility resources. Artificial intelligence and machine learning approaches provide the means to improve HPC data center operational efficiency, by learning historical trends and training models to operate on real-time data collected from both IT and facilities sources. NREL has developed methods of real-time collection, aggregation and streaming of these data in the ESIF HPC Data Center and has collected a significant dataset of relevant metrics across computer systems, racks, environmental, building and utility sources for research into various predictive analytics problems. HPE's Advanced Technology Group (ATG) is doing comprehensive research into exascale monitoring and management for High Performance Computing (HPC) systems (hereinafter HPE's Data Monitoring/ Management Technology). NREL and HPE will collaborate to add Artificial Intelligence (AI) to NREL's real-time data collection/ aggregation/ streaming system and HPE's Data Monitoring/ Management System, with the goal of improving the operational efficiency of NREL's Energy Systems Integration Facility (ESIF) HPC Data Center through data analytics on both historical and real-time data from IT systems and facilities operations. This collaboration will consist of efforts in Data Management, Data Analytics, and AI/ML Optimization for both manual and autonomous intervention in data center operations. This will be a multi-year, multi-staged effort with a goal towards building capabilities for an Advanced Smart Facility, and demonstration of these techniques in the NREL ESIF HPC Data Center.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

A systematic method for selecting molecular descriptors as features when training models for predicting physiochemical properties

Machine learning has proven to be a powerful tool for accelerating biofuel development. Although numerous models are available to predict a range of properties using chemical descriptors, there is a trade-off between interpretability and performance. Neural networks provide predictive models with high accuracy at the expense of some interpretability, while simpler models such as linear regression often lack in accuracy. In addition to model architecture, feature selection is also critical for developing interpretable and accurate predictive models. We present a method for systematically selecting molecular descriptor features and developing interpretable machine learning models without sacrificing accuracy. Our method simplifies the process of selecting features by reducing feature multicollinearity and enables discoveries of new relationships between global properties and molecular descriptors. To demonstrate our approach, we developed models for predicting melting point, boiling point, flash point, yield sooting index, and net heat of combustion with the help of the Tree-based Pipeline Optimization Tool (TPOT). For training, we used publicly available experimental data for up to 8351 molecules. Our models accurately predict various molecular properties for organic molecules (mean absolute percent error (MAPE) ranges from 3.3% to 10.5%) and provide a set of features that are well-correlated to the property. This method enables researchers to explore sets of features that significantly contribute to the prediction of the property, offering new scientific insights. To help accelerate early stage biofuel research and development, we also integrated the data and models into a open-source, interactive web tool.

09 BIOMASS FUELS↗
DOE OSTI2024

Integrated edge-to-exascale workflow for real-time steering in neutron scattering experiments

We introduce a computational framework that integrates artificial intelligence (AI), machine learning, and high-performance computing to enable real-time steering of neutron scattering experiments using an edge-to-exascale workflow. Focusing on time-of-flight neutron event data at the Spallation Neutron Source, our approach combines temporal processing of four-dimensional neutron event data with predictive modeling for multidimensional crystallography. At the core of this workflow is the Temporal Fusion Transformer model, which provides voxel-level precision in predicting 3D neutron scattering patterns. The system incorporates edge computing for rapid data preprocessing and exascale computing via the Frontier supercomputer for large-scale AI model training, enabling adaptive, data-driven decisions during experiments. This framework optimizes neutron beam time, improves experimental accuracy, and lays the foundation for automation in neutron scattering. Although real-time experiment steering is still in the proof-of-concept stage, the demonstrated potential of this system offers a substantial reduction in data processing time from hours to minutes via distributed training, and significant improvements in model accuracy, setting the stage for widespread adoption across neutron scattering facilities and more efficient exploration of complex material systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Passive and active peer effects in the spatial diffusion of residential solar panels: A case study of the Las Vegas Valley

This research analyzes the role of peer influences on the adoption of residential rooftop solar photovoltaic panels (PV) within the context of the Diffusion of Innovation Theory. PV literature indicates that adopters are influenced by word of mouth (WOM) information exchange with peers, i.e., active peer effects, while other studies suggest that living near households with visible rooftop PV installations influences adoption, i.e., passive peer effects. We bridge the gap in this literature by conducting a mixed methods analysis. We develop and administer a survey to Las Vegas Valley (LVV) residents to identify current and potential PV adopters' perceptions of peer-effects and consumer intention variables. We conduct a spatial analysis of Google's Project Sunroof data to identify LVV neighborhoods in the later stages of the PV diffusion process, i.e., those with the highest PV adoption rates. Key results show that current PV adopters living in early diffusion areas report significantly higher active and passive peer effects compared to adopters in later diffusion areas. Potential adopters in later diffusion areas report higher passive peer effects than those in early diffusion areas. Overall, because LVV has a low PV adoption rate (<3%), short term strategies aimed at increasing PV adoption should emphasize WOM active peer effects. Here, we caution against long-term green marketing strategies focusing solely on peer-effects as the PV market matures.

14 SOLAR ENERGY↗
DOE OSTI2025

Tracking seasonal variability in plant traits from spaceborne PRISMA and NEON AOP across forest types and ecoregions

Plant traits serve as critical indicators of how plants adapt to environmental changes and influence ecosystem functions. While airborne hyperspectral remote sensing effectively maps plant traits through detailed reflectance properties, it is limited by cost and scale, making large-scale and temporal studies challenging. The recently launched spaceborne hyperspectral imager, PRecursore IperSpettrale della Missione Applicativa (PRISMA), offers frequent, large scale and high-fidelity observations on a spatial resolution of 30 m and a revisit time of around 29 days, making it suitable for large-scale seasonal trait mapping. However, their potential remains largely unexplored. This study developed a multi-stage framework by leveraging the PRISMA spaceborne hyperspectral data and National Ecological Observatory Network (NEON) Airborne Observation Platform (AOP) hyperspectral data to investigate the seasonal dynamics of four key plant traits — chlorophyll content, carotenoid content, equivalent water thickness, and nitrogen content — across eleven NEON sites representing diverse forest types and ecoregions in the contiguous U.S. Our results demonstrated that PRISMA hyperspectral data can reliably track seasonal variability in plant traits, achieving overall R 2 values ranging from 0.78 to 0.88 and normalized root mean square error (NRMSE) values ranging from 5.4% to 8.4% for the four traits. Seasonal patterns revealed bell-shaped trajectories for chlorophyll and carotenoids, while equivalent water thickness decreased steadily across most sites, driven by structural changes during leaf maturation and senescence. Nitrogen content exhibited less pronounced seasonal variation but followed expected nutrient resorption patterns. Analysis of environmental drivers showed that seasonal variability is primarily controlled by solar radiation and day length in northern sites, vapor pressure in semi-arid regions, and temperature in mid-southeastern sites. Spatial variability, meanwhile, was primarily driven by soil properties, particularly during the peak growing season. However, the influence of soil variables slightly declines toward the end of the season at several sites, as climatic factors become more prominent. This study highlights the capability of PRISMA, and potentially other similar spaceborne hyperspectral data for large-scale, time-series plant trait mapping and provides valuable insights into the interactions between plant traits and environmental factors. In conclusion, these findings contribute to advancing our understanding of plant functional ecology and improving predictions of ecosystem responses to environmental changes.

Environmental drivers↗
DOE OSTI2026

Life Cycle Analysis of Natural Gas Supply Chain and End Use Applications in the United States

Natural gas (NG) plays a crucial role in current and future energy systems in the United States due to its abundance and affordability. In this study a life cycle analysis of the NG supply chain in the United States was conducted using Argonne's R&D GREET model, examining stages from recovery to distribution using reported field data processed and documented by National Energy Technology Laboratory. Supply chain emissions were evaluated across multiple spatial scales, including national average, overall regional production, region-to-region, and basin-to-region scenarios. The GHG intensity of the U.S. average NG supply chain was estimated at 10.3 kg CO 2 e/MMBtu (lower heating value), with a range across regions from 7.8 kg CO 2 e/MMBtu (Northeast) to 15.1 kg CO 2 e/MMBtu (Pacific). The analysis further assessed how upstream NG emissions influence the life cycle GHG emissions of key end-use applications, including electricity generation (0.044–0.086 kg CO 2 e/kWh from upstream NG in combined cycle facilities), hydrogen production (1.04–2.20 kg CO 2 e/kg H 2 for steam methane reforming [SMR] and 1.06–2.23 kg CO 2 e/kg for autothermal reforming [ATR]), and transit bus operation utilizing compressed natural gas fuel (0.19–0.37 kg CO 2 e/mile) and hydrogen fuel (0.12–0.25 kg CO 2 e/mile for hydrogen produced in SMR and ATR).

compression↗
DOE OSTI2024

An updated technique to obtain explosive kinetics data on microsecond timescales

There are few techniques available for chemists to obtain time-to-explosion data with known temperature inputs at the early stages of the design and synthesis of new explosives. In the 1960s, a technique was developed to rapidly heat milligram-quantities of confined explosives to ~1000 K on microsecond timescales. Wenograd loaded explosives inside stainless steel hypodermic needles, connected them to a fireset and rapidly discharged a capacitor through the steel. He obtained the temperature by measuring the needle resistance in a Wheatstone bridge arrangement and the time to explosion from a needle rupture. However, owing to the narrow-gauge needles used in the original research, the experiment was only possible with melt-castable explosives; it was never replicated, and modern diagnostics are now available with advances beyond the 1960s. Here, we report the development of the High Explosives Initiation Time (HEIT) test, which utilizes a 250 J pulsed power system to heat the needles. This work extends the Wenograd approach by using optical diagnostics, computational modeling, and advanced techniques to measure needle resistance and needle rupture. Preliminary rate information for pentaerythritol tetranitrate (PETN) will be presented.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2021

Real-Time Dispatch With Secondary Frequency Regulation: A Pathway to Consider Intra-Interval Fluctuations

Real-time dispatch balances the power demand with minimized operating costs. For the current dispatch model, the demand is assumed to be constant within a time interval, while the intra-interval power balance is left to frequency regulation. Based on practical experience and simulations, this behavior may lead to insufficient frequency regulation and uneconomic regulation costs considering the increase in intra-interval fluctuations caused by renewables. Here, a real-time dispatch method with secondary frequency regulation behaviors is proposed. Without changing the interval of the real-time dispatch command, the system regulation mileage and intra-interval generation adjustment are explicitly formulated in a mixed-integer optimization model. To reduce the computational burden, an efficient two-stage calculation method is proposed. With practical utility data, this article finds that the proposed method can effectively improve the system frequency performance with the subminute net load forecasting curve obtained by an off-the-shelf forecasting approach. Time-domain simulations in IEEE and practical utility systems validate the effectiveness of the proposed method in terms of the frequency performance and total operating costs.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Bayesian And Human Reliability Analysis (hra)-aided Method For The Reliability Analysis Of Software (bahamas)

The purpose of the BAHAMAS code is to provide a simplified process for performing quantitative evaluations of software reliability. The Bayesian and Human Reliability Analysis (HRA)-Aided method for the Reliability Analysis of software (BAHAMAS) was developed specifically to perform quantification under limited data conditions, i.e., when limited testing or operational data are available, such as during early development stages. BAHAMAS essentially examines the quality of a software development life cycle to determine the probability of specific types of software failure. BAHAMAS will have modules to support user input for detailed and simplified analyses. The user interface will also support software common cause failure analysis.

Wang, Congjian (0000000207789927)↗
DOE OSTI2025

The Impact of Demographic Lifecycle States on Time to Vehicle Purchase: Insights from the Panel Study of Income Dynamics

This study examines the impact of demographic lifecycle stages on the timing of vehicle purchases, using data from the Panel Study of Income Dynamics from 1999 to 2021. Survival analysis was employed to model the duration until households purchase vehicles, incorporating key lifecycle variables such as age, employment status, marital status, childbirth, home ownership, and the presence of school-going children. The life table results indicate that early adulthood (ages 20–35) is the prime period for vehicle acquisition, with significant peaks around ages 25 to 30. Additionally, the instantaneous hazard of purchasing a vehicle is highest in the late 40s and early 50s. According to the Cox proportional hazards model, employment, marital status, and home ownership significantly increase the likelihood of purchasing a vehicle, while living in multi-unit dwellings decreases it. Interaction effects reveal that married individuals with employed spouses are substantially more likely to purchase vehicles. In conclusion, this study serves as a steppingstone toward integrating demographic lifecycle analysis into car ownership modeling that better reflects real-world scenarios and increases the accuracy of policy and strategic planning.

Car ownership↗
DOE OSTI2024

The microbiome of a Pacific moon jellyfish Aurelia coerulea

The impact of microbiome in animal physiology is well appreciated, but characterization of animal-microbe symbiosis in marine environments remains a growing need. This study characterizes the microbial communities associated with the moon jellyfish Aurelia coerulea, first isolated from the East Pacific Ocean and has since been utilized as an experimental system. We find that the microbiome of this Pacific Aurelia culture is dominated by two taxa, a Mollicutes and Rickettsiales. The microbiome is stable across life stages, although composition varies. Mining the host sequencing data, we assembled the bacterial metagenome-assembled genomes (MAGs). The bacterial MAGs are highly reduced, and predict a high metabolic dependence on the host. Analysis using multiple metrics suggest that both bacteria are likely new species. We therefore propose the names Ca. Mariplasma lunae (Mollicutes) and Ca. Marinirickettsia aquamalans (Rickettsiales). Finally, comparison with studies of Aurelia from other geographical populations suggests the association with Ca. Mariplasma lunae occurs in Aurelia from multiple geographical locations. The low-diversity microbiome of Aurelia provides a relatively simple system to study host-microbe interactions.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Algorithms and Statistical Models for Scientific Discovery in the Petabyte Era

The field of astronomy has arrived at a turning point in terms of size and complexity of both datasets and scientific collaboration. Commensurately, algorithms and statistical models have begun to adapt --- e.g., via the onset of artificial intelligence --- which itself presents new challenges and opportunities for growth. This white paper aims to offer guidance and ideas for how we can evolve our technical and collaborative frameworks to promote efficient algorithmic development and take advantage of opportunities for scientific discovery in the petabyte era. We discuss challenges for discovery in large and complex data sets; challenges and requirements for the next stage of development of statistical methodologies and algorithmic tool sets; how we might change our paradigms of collaboration and education; and the ethical implications of scientists' contributions to widely applicable algorithms and computational modeling. We start with six distinct recommendations that are supported by the commentary following them. This white paper is related to a larger corpus of effort that has taken place within and around the Petabytes to Science Workshops (https://petabytestoscience.github.io/).

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

DeepDiagnostics: A Software Package for Streamlined Posterior Evaluation

Automated prediction techniques like simulation-based inference (SBI) are important tasks for science experiments that produce large amounts of complex, raw data. However, their development remains in its early stages because the uncertainties of these techniques lack sufficient trustworthiness and interpretability. Packages for SBI provide a growing set of diagnostics; however, the software requirements are substantial, as they are tied to the inference technology itself, and the APIs lack adaptability. We introduce the DeepDiagnostics package for diagnosing posteriors from analytic likelihood-based methods and SBI methods, such as neural posterior estimation. DeepDiagnostics produces a comprehensive set of high-quality visualizations and metrics in a highly accessible, easy-to-use, and flexible package. We address all of these goals by providing a command-line inference tool and a Python API that is controlled through a configuration file. The package includes common diagnostics, such as parity plots, corner (covariance) plots, simulation-based calibration (SBC) diagnostics (including posterior coverage and rank histograms), Lemos et al. s PQMass and TARP, Masserano et al. s WALDO, Linhart et al. s LC2ST, as well as credible region diagnostics developed by our group.

Voetberg, Maggie [Fermilab]↗
DOE OSTI2022

microTrait: A Toolset for a Trait-Based Representation of Microbial Genomes

Remote sensing approaches have revolutionized the study of macroorganisms, allowing theories of population and community ecology to be tested across increasingly larger scales without much compromise in resolution of biological complexity. In microbial ecology, our remote window into the ecology of microorganisms is through the lens of genome sequencing. For microbial organisms, recent evidence from genomes recovered from metagenomic samples corroborate a highly complex view of their metabolic diversity and other associated traits which map into high physiological complexity. Regardless, during the first decades of this omics era, microbial ecological research has primarily focused on taxa and functional genes as ecological units, favoring breadth of coverage over resolution of biological complexity manifested as physiological diversity. Recently, the rate at which provisional draft genomes are generated has increased substantially, giving new insights into ecological processes and interactions. From a genotype perspective, the wide availability of genome-centric data requires new data synthesis approaches that place organismal genomes center stage in the study of environmental roles and functional performance. Extraction of ecologically relevant traits from microbial genomes will be essential to the future of microbial ecological research. Here, we present microTrait , a computational pipeline that infers and distills ecologically relevant traits from microbial genome sequences. microTrait maps a genome sequence into a trait space, including discrete and continuous traits, as well as simple and composite. Traits are inferred from genes and pathways representing energetic, resource acquisition, and stress tolerance mechanisms, while genome-wide signatures are used to infer composite, or life history, traits of microorganisms. This approach is extensible to any microbial habitat, although we provide initial examples of this approach with reference to soil microbiomes.

Karaoz, Ulas↗
DOE OSTI2022

A behavior and physiology-based decision support tool to predict thermal comfort and stress in non-pregnant, mid-gestation, and late-gestation sows

Background: Although thermal indices have been proposed for swine, none to our knowledge differentiate by reproductive stage or predict thermal comfort using behavioral and physiological data. The study objective was to develop a behavior and physiology-based decision support tool to predict thermal comfort and stress in multiparous (3.28 ± 0.81) non-pregnant (n = 11), mid-gestation (n = 13), and late-gestation (n = 12) sows. Results: Regression analyses were performed using PROC MIXED in SAS 9.4 to determine the optimal environmental indicator [dry bulb temperature (T DB ) and dew point] of heat stress (HS) in non-pregnant, mid-gestation, and late-gestation sows with respiration rate (RR) and body temperature (T B ) successively used as the dependent variable in a cubic function. A linear relationship was observed for skin temperature (T S ) indicating that TDB rather than the sow HS response impacted TS and so TS was excluded from further analyses. Reproductive stage was significant for all analyses (P < 0.05). Heat stress thresholds for each reproductive stage were calculated using the inflections points of RR for mild HS and T B for moderate and severe HS. Mild HS inflection points differed for non-pregnant, mid-gestation, and late gestation sows and occurred at 25.5, 25.1, and 24.0 °C, respectively. Moderate HS inflection points differed for non-pregnant, mid-gestation, and late gestation sows and occurred at 28.1, 27.8, and 25.5 °C, respectively. Severe HS inflection points were similar for non-pregnant and mid-gestation sows (32.9 °C) but differed for late-gestation sows (30.8 °C). These data were integrated with previously collected behavioral thermal preference data to estimate the T DB that non-pregnant, mid-gestation, and late-gestation sows found to be cool (T DB < T DB preference range), comfortable (T DB = T DB preference range), and warm (T DB preference range < T DB < mild HS). Conclusions: The results of this study provide valuable information about thermal comfort and thermal stress thresholds in sows at three reproductive stages. The development of a behavior and physiology-based decision support tool to predict thermal comfort and stress in non-pregnant, mid-gestation, and late-gestation sows is expected to provide swine producers with a more accurate means of managing sow environments.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Decision-Making Under Uncertainty for a Digital Thread-Enabled Design Process

Abstract Digital thread is a data-driven architecture that links together information from all stages of the product lifecycle. Despite increasing application in manufacturing, maintenance/operations, and design related tasks, a principled formulation of analyzing the decision-making problem under uncertainty for the digital thread remains absent. The contribution of this article is to present a formulation using Bayesian statistics and decision theory. First, we address how uncertainty propagates in the product lifecycle and how the digital thread evolves based on the decisions we make and the data we collect. By using these mechanics, we explore designing over multiple product generations or iterations and provide an algorithm to solve the underlying multistage decision problem. We illustrate our method on an example structural design problem where our method can quantify and optimize different types and sequences of decisions, ranging from experimentation, manufacturing, and sensor placement/selection, to minimize total accrued costs.

Engineering↗
DOE OSTI2020

National Cancer Institute Think-Tank Meeting Report on Proteomic Cartography and Biomarkers at the Single-Cell Level: Interrogation of Premalignant Lesions

A Think-Tank Meeting was convened by the National Cancer Institute (NCI) to solicit experts’ opinion on the development and application of multi-omic single-cell analyses, and especially single-cell proteomics, to improve the development of a new generation of biomarkers for cancer risk, early detection, diagnosis, prognosis as well as to discuss the discovery of new targets for prevention and therapy. It is anticipated that such markers and targets will be based on cellular, subcellular, molecular and functional aberrations within the lesion and within individual cells. Additionally, single-cell proteomic data will be essential for the establishment of new tools that with searchable and scalable features that include spatial and temporal cartographies of premalignant and malignant lesions. Challenges and potential solutions that were discussed included: • The best way/s to analyze single-cells from fresh and preserved tissue • Detection and analysis of secreted molecules and from single cells, especially from a tissue slice • Detection of new, previously undocumented cell type/s in the premalignant and early stage cancer tissue microenvironment • Multi-omic integration of data to support and inform proteomic measurements • Subcellular organelles – identifying abnormal structure, function, distribution and location within individual premalignant and malignant cells • How to improve the dynamic range of single-cell proteomic measurements for discovery of differentially expressed proteins and their post translational modifications (PTM) • The depth of coverage measured concurrently using single-cell techniques • Quantitation - absolute or semiquantitative? • Single methodology or multiplexed combinations? • Application of analytical methods for identification of biologically significant subsets • Data visualization of N-dimensional datasets • How to construct intercellular signaling networks in individual cells within premalignant tumor microenvironments (TME) • Associations between intrinsic cellular processes and extrinsic stimuli • How to predict cellular responses to stress inducing stimuli • Identification of new markers for prediction of progression from precursor, benign and localized lesions to invasive cancer, based on spatial and temporal changes within individual cells • Identification of new targets for immunoprevention or immunotherapy – identification of neoantigens and surfactome of individual cells within a lesion

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

AFUE Analysis Software Tool

The Annual Fuel Utilization Efficiency (AFUE) analysis of residential and light commercial furnaces follows ANSI/ASHRAE Standard 103-2017 (i.e., Method of Testing for Annual Fuel Utilization Efficiency of Residential Central Furnaces and Boilers) . The analysis is a complex, comprehensive method based on furnace configuration and specific components equipped, requiring detailed furnace testing and measurement data. For this reason, an AFUE Analysis Tool using Microsoft Excel enabled with Visual Basic for Applications (VBA) was developed with a user-friendly interface and comprehensive coverage. The tool consists of three worksheets: unit and configuration selection, geometry and measurement data input, and AFUE plus key results. This tool can be used to estimate the AFUE of both condensing and noncondensing furnaces with single-stage, two-stage, and step-modulating functions. The tool was validated with experimental data from Oak Ridge National Laboratory’s natural gas furnace projects that are commercially available. The results indicate the tool is reasonably accurate in the evaluation of a new R&D modified furnace unit.

Gao, Zhiming [Oak Ridge National Laboratory (ORNL)↗