Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “datasets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Assessing data change in scientific datasets

Summary Scientific datasets are growing rapidly and becoming critical to next‐generation scientific discoveries. The validity of scientific results relies on the quality of data used and data are often subject to change, for example, due to observation additions, quality assessments, or processing software updates. The effects of data change are not well understood and difficult to predict. Datasets are often repeatedly updated and recomputing derived data products quickly becomes time consuming and resource intensive and may in some cases not even be necessary, thus delaying scientific advance. Despite its importance, there is a lack of systematic approaches for best comparing data versions to quantify the changes, and ad‐hoc or manual processes are commonly used. In this article, we propose a novel hierarchical approach for analyzing data changes, including real‐time (online) and offline analyses. We employ a variety of fast‐to‐compute numerical analyses, graphical data change representations, and more resource‐intensive recomputations of a subset of the data product. We illustrate the application of our approach using three scientific diverse use cases, namely, satellite, cosmological, and x‐ray data. The results show that a variety of data change metrics should be employed to enable a comprehensive representation and qualitative evaluation of data changes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

A Dataset of CFD Simulated Industrial Furnace Images for Conditional Automatic Generation with GANs

The steel industry is constantly looking for ways to automate processes and improve efficiency. A standard practice in industry is to simulate how complex systems will operate before they are actually used. Some complex systems, including steel industry processes such as blast furnaces, require complex physics-based simulations utilizing computational fluid dynamics (CFD). These CFD physics-based simulations are very accurate but can take significant time and computational resources to process, resulting in challenges for the implementation of the models in real-world operational environments. In recent years, deep learning (DL) has been considered as a substitute for these CFD models. DL models can be trained on validated CFD simulation data and then used for industrial process inference. Previous DL-based solutions have made great contributions for industrial automation but are currently missing the additional visualization component that CFD simulations also provide. In this paper, we propose a dataset for simple DL generative approaches that can help to address this issue. The dataset and methodology under development to approach this prediction are discussed in this work.

Calix, Ricardo↗
DOE OSTI2025

Enhancing Interpretability in Generative Modeling: Statistically Disentangled Latent Spaces Guided by Generative Factors in Scientific Datasets

This study addresses the challenge of statistically extracting generative factors from complex, high-dimensional datasets in unsupervised or semi-supervised settings. We investigate encoder-decoder-based generative models for nonlinear dimensionality reduction, focusing on disentangling low-dimensional latent variables corresponding to independent physical factors. Introducing Aux-VAE, a novel architecture within the classical Variational Autoencoder framework, we achieve disentanglement with minimal modifications to the standard VAE loss function by leveraging prior statistical knowledge through auxiliary variables. These variables guide the shaping of the latent space by aligning latent factors with learned auxiliary variables. We validate the efficacy of Aux-VAE through comparative assessments on multiple datasets, including astronomical simulations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

An Open Combinatorial Diffraction Dataset Including Consensus Human and Machine Learning Labels with Quantified Uncertainty for Training New Machine Learning Models

Modern machine learning and autonomous experimentation schemes in materials science rely on accurate analysis of the data ingested by these models. Unfortunately, accurate analysis of the underlying data can be difficult, even for domain experts, complicating the training of the models intended to drive experiments. This is especially true when the goal is to identify the presence of weak signatures in diffraction or spectroscopic datasets. In this work, we examine a set of as-obtained diffraction data that track the phase transition from monoclinic to tetragonal in a Nb-doped VO2 film as a function of temperature and dopant concentration. We then task a set of domain experts and a set of machine learning experts with identifying which phase is present in each diffraction pattern manually and algorithmically, respectively; in both cases, the labels can vary dramatically, especially at the phase boundaries. We use the mode of the labels and the Shannon entropy as a method to capture, preserve and propagate consensus labels and their variance. Further we use the expert labels as a benchmark and demonstrate the use of Shannon entropy weighted scoring to test the performance of machine learning generated labels. Finally, we propose a material data challenge centered around generating improved labeling algorithms. This real-world dataset curated with expert labels can act as test bed for new algorithms. The raw data, annotations and code used in this study are all available online at data.gov and the interested reader is encouraged to replicate and improve the existing models

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Segmentation of tomography datasets using 3D convolutional neural networks

Dendritic microstructures are ubiquitous in nature and are the primary solidification morphologies in metallic materials. Techniques such as X-ray computed tomography (XCT) have provided new insights into dendritic phase transformation phenomena. However, manual identification of dendritic morphologies in microscopy data can be both labor intensive and potentially ambiguous. The analysis of 3D datasets is particularly challenging due to their large sizes (terabytes) and the presence of artifacts scattered within the imaged volumes. Here, in this study, we trained 3D convolutional neural networks (CNNs) to segment 3D datasets. Three CNN architectures were investigated, including a new version of FCDenseNet which we extended to 3D. We show that using hyperparameter optimization (HPO) and fine-tuning techniques, both 2D and 3D CNN architectures outperform the previous state of the art. The 3D U-Net architecture trained in this study produced the best segmentations according to quantitative metrics (intersection-over-union of 95.56% and a boundary displacement error of 0.58 pixels), while 3D FCDense produced the smoothest boundaries and best segmentations according to visual inspection. The trained 3D CNNs are able to segment entire 852 × 852 × 250 voxel 3D volumes in only ~60 s, thus hastening the progress towards a deeper understanding of phase transformation phenomena such as dendritic solidification.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Areal interpolation of population projections consistent with different SSPs from 1-km resolution to block level based on USA Structures dataset

Population data are normally collected at various census administrative levels, and areal interpolation of population is often required to transform population to the desired spatial resolution. Building footprint datasets, such as Microsoft building footprints, have proven to be useful in estimating population distribution and can therefore be used for areal interpolation of population. In addition to Microsoft building footprints, the recently released USA Structures dataset provides additional information such as building type and building height for some regions, which may provide valuable information for a better depiction of population distribution and improved population areal interpolation accuracy. In this study, we have conducted areal interpolation of population projections consistent with three different Shared Socioeconomic Pathways (SSP2, SSP3, and SSP5) from 1-km grid cells to block level in Washington state for every ten years from 2020 to 2040 based on USA Structures. We assessed USA Structures-based population downscaling accuracy using U.S. decennial survey data in 2020 under three different downscaling schemes, including population downscaling from census tracts to block groups, from census tracts to blocks, and from block groups to blocks. The resulting accuracies were compared with those based on Microsoft building footprints. The comparison showed that USA Structures achieved higher accuracies across different population density regions and areas with different urbanization extent within our study area.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2022

AIMSim : An accessible cheminformatics platform for similarity operations on chemicals datasets

The recent advances in deep learning, generative modeling, and statistical learning have ushered in a renewed interest in traditional cheminformatics tools and methods. Quantifying molecular similarity is essential in molecular generative modeling, exploratory molecular synthesis campaigns, and drug-discovery applications to assess how new molecules differ from existing ones. Further, most tools target advanced users and lack general implementations accessible to the larger community. In this work, we introduce Artificial Intelligence Molecular Similarity (AIMSim), an accessible cheminformatics platform for performing similarity operations on collections of molecules called molecular datasets. AIMSim provides a unified platform to perform similarity-based tasks on molecular datasets, such as diversity quantification, outlier and novelty analysis, clustering, dimensionality reduction, and inter-molecular comparisons. AIMSim implements all major binary similarity metrics and molecular fingerprints and is provided as a Python package that includes support for command-line use as well as a Graphical User Interface for code-free utilization with fully interactive plots.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Dataset of simulated vibrational density of states and X-ray diffraction profiles of mechanically deformed and disordered atomic structures in Gold, Iron, Magnesium, and Silicon

This dataset is comprised of a library of atomistic structure files and corresponding X-ray diffraction (XRD) profiles and vibrational density of states (VDoS) profiles for bulk single crystal silicon (Si), gold (Au), magnesium (Mg), and iron (Fe) with and without disorder introduced into the atomic structure and with and without mechanical loading. Included with the atomistic structure files are descriptor files that measure the stress state, phase fractions, and dislocation content of the microstructures. All data was generated via molecular dynamics or molecular statics simulations using the Large-scale Atomic/Molecular Massively Parallel Simulator (LAMMPS) code. This dataset can inform the understanding of how local or global changes to a materials microstructure can alter their spectroscopic and diffraction behavior across a variety of initial structure types (cubic diamond, face-centered cubic (FCC), hexagonal close-packed (HCP), and body-centered cubic (BCC) for Si, Au, Mg, and Fe, respectively) and overlapping changes to the microstructure (i.e., both disorder insertion and mechanical loading).

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2020

A multi-dimensional dataset for two-phase instability in low pressure natural circulation based on direct transient local measurement

Thermal hydraulic instability can challenge the control and the safety of nuclear reactors. This experimental research captures and investigates the instability in a low-pressure (1–10 atmospheres) natural circulation loop with an adiabatic chimney installed above the heated section. Direct transient local measurement of two-phase flow is performed at multiple axial locations along the flow channel with thermocouples and conductivity probes traversed by customized high-precision mechanisms. Based on the periodicity of oscillations, a post-processing method is proposed to align the data captured non-simultaneously to reconstruct the instability behavior in one representative period. The averaging operation over multiple oscillation cycles is performed to reduce the random errors. Cyclic phenomena are clearly demonstrated by the post-processed data, and rapid void fraction change induced by the flashing is observed to be significant in the driving mechanism for low pressure oscillations. Parametric effects of inlet subcooling, heat flux, inlet flow restriction and system pressure are studied within the experiment range, and qualitative results are explained and compared with existing literature. The ratio between the oscillation period and the travelling time of the fluid particle through the test section is compared with the theoretical value for the density wave oscillation. The new dataset contains directly measured transient multi-dimensional two-phase data along the flow channel for natural circulation under density wave oscillations. The generated dataset is valuable for the validation of system analysis codes and CFD codes in the simulation of natural circulation phenomena and dynamic two-phase flow.

42 ENGINEERING↗
DOE OSTI2021

Class imbalance in out-of-distribution datasets: Improving the robustness of the TextCNN for the classification of rare cancer types

In the last decade, the widespread adoption of electronic health record documentation has created huge opportunities for information mining. Natural language processing (NLP) techniques using machine and deep learning are becoming increasingly widespread for information extraction tasks from unstructured clinical notes. Disparities in performance when deploying machine learning models in the real world have recently received considerable attention. In the clinical NLP domain, the robustness of convolutional neural networks (CNNs) for classifying cancer pathology reports under natural distribution shifts remains understudied. In this research, we aim to quantify and improve the performance of the CNN for text classification on out-of-distribution (OOD) datasets resulting from the natural evolution of clinical text in pathology reports. We identified class imbalance due to different prevalence of cancer types as one of the sources of performance drop and analyzed the impact of previous methods for addressing class imbalance when deploying models in real-world domains. Our results show that our novel class-specialized ensemble technique outperforms other methods for the classification of rare cancer types in terms of macro F1 scores. We also found that traditional ensemble methods perform better in top classes, leading to higher micro F1 scores. Based on our findings, we formulate a series of recommendations for other ML practitioners on how to build robust models with extremely imbalanced datasets in biomedical NLP applications.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2026

Re-evaluating probable maximum precipitation estimates: sensitivity to transposition domains and storm rotation using modern datasets

This study examines the sensitivity of Probable Maximum Precipitation (PMP) estimates to key methodological decisions embedded in the legacy approach adopted in the U.S. National Weather Service Hydrometeorological Reports No. 51 and No. 52. Although widely used for infrastructure design and risk regulation, fundamental aspects of PMP estimation—such as storm sample size, transposition domain, maximization procedures, and storm rotation—remain poorly constrained and lack formal guidance. Using the Red Rock watershed in Iowa as a case study, and leveraging the 2002–2023 NOAA Analysis of Record for Calibration (AORC) precipitation dataset, we systematically evaluate how each methodological choice, individually and in combination, influences PMP estimates. Our findings demonstrate that PMP is not a fixed physical upper bound but rather a modeling construct shaped heavily by user-defined assumptions. Notably, PMP values derived from modern gridded rainfall datasets can be substantially higher than the legacy estimate used in the original spillway design for Red Rock Dam. Decisions regarding storm sample size, domain extent, climatological window, and particularly storm rotation all contributed to higher PMP estimates. Storm rotation alone—a loosely constrained element in the current PMP practice—can amplify PMP by more than 25%. These results reveal the lack of standardized bounds in current PMP workflows and the need for systematic sensitivity and uncertainty analysis. As PMP estimation shifts toward probabilistic approaches, incorporating physically meaningful storm attributes will be key to developing more transparent, defensible methods for dam safety and climate-resilient infrastructure.

Probable maximum precipitation↗
DOE OSTI2022

Disentangling error structures of precipitation datasets using decision trees

Characterizing error structures in precipitation products not only facilitates their proper applications for scientific and practical purposes but also helps improve their retrieval algorithms and processing methods. Despite the fact that multiple precipitation products have been assessed in the literature, factors that affect their error structures remain inadequately addressed. By interpreting 60 binary decision trees, this study disentangles the error characteristics of precipitation products in terms of their spatiotemporal patterns and geographical factors. Three independent precipitation products - two satellite-based and one reanalysis datasets: the Integrated Multi-satellitE Retrievals for GPM (Global Precipitation Measurement) late run (IMERG-L), Soil Moisture to Rain-Advanced SCATterometer (SM2RAIN-ASCAT), and the Modern-Era Retrospective analysis for Research and Applications, Version 2 uncorrected precipitation output (MERRA2-UC), are evaluated across the contiguous United States from 2010 to 2019. Here, the ground-based Stage IV precipitation dataset is used as the ground truth. Results indicate that the MERRA2-UC outperforms the IMERG-L and SM2RAIN-ASCAT with higher accuracy and more stable interannual patterns for the analysis period. Decision trees cross-assess three spatiotemporal factors and find that the underestimation of MERRA2-UC occurs in the east of the Rocky Mountains, and SM2RAIN-ASCAT underestimates precipitation over high latitudes, especially in winter. Additionally, the decision tree method ascribes system errors to nine different geographical characteristics, of which the distance to the coast, soil type, and DEM are the three dominant features. On the other hand, the land cover type, topography position index, and aspect are three relatively weak factors.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Analysis of alternative climate datasets and evapotranspiration methods for the Upper Mississippi River Basin using SWAT within HAWQS

This study reports the application of Soil andWater Assessment Tool (SWAT) within the Hydrologic andWater Quality System(HAWQS) on-line platform, for the UpperMississippi River Basin (UMRB). TheUMRB is an important ecosystem located in the north central U.S. that is experiencing a range of ecological stresses. Specifically, testing of SWAT was performed for: (1) Hargreaves (HG) and Penman-Monteith (PM) PET methods, and (2) Livneh, National Climatic Data Center (NCDC) and Parameter-elevation Regressions on Independent Slopes Model (PRISM) climate datasets. The Livneh-PM combination resulted in the highest average annual water yield of 380.6 mm versus the lowest estimated water yield of 193.9 mm for the Livneh-HG combination, in response to 23-year uncalibrated simulations. Higher annual ET and PET values were predicted with HG method versus the PM method for all three weather datasets in response to the uncalibrated simulations, due primarily to higher HG-based estimates during the growing season. Based on these results, it was found that the HG method is the preferred PET option for the UMRB. Initial calibration of SWAT was performed using the Livneh data and HGmethod for threeMississippi Rivermain stem gauge sites, which was followed by spatial validation at 10 other gauge sites located within the UMRB streamnetwork. Overall satisfactory results were found for the calibration and validation gauge sites, with the majority of R2 values ranging between 0.61 and 0.82, Nash- Sutcliffe modeling efficiency (NSE) values ranging between 0.50 and 0.79, and Kling-Gupta efficiency (KGE) values ranging between 0.61 and 0.84. The results of an additional experimental suite of six scenarios,which represented different combinations of climate data sets and calibrated parameters, revealed that suggested statistical criteria were again satisfied by the different scenario combinations. Overall, the PRISM data exhibited the strongest reliability for the UMRB.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Constructing a spatiotemporally coherent long-term PM2.5 concentration dataset over China during 1980–2019 using a machine learning approach

The lack of long-term observations and satellite retrievals of health-damaging fine particulate matter in China has demanded the estimates of historical PM 2.5 (particulate matter less than 2.5 µm in diameter) concentrations. This study constructs a gridded near-surface PM 2.5 concentration dataset across China covering 1980–2019 using the space-time random forest model with atmospheric visibility observations and other auxiliary data. The modeled daily PM2.5 concentrations are in excellent agreement with ground measurements during 2015–2019, with a coefficient of determination of 0.95 and mean relative error of 12%. Besides the atmospheric visibility which explains 30% of total importance of variables in the model, both emissions and meteorological conditions are also essential factors affecting PM2.5 predictions. From 1980 to 2014, the model-predicted PM2.5 concentrations increased constantly with the maximum growth rate of 5–10 µg/m3/decade over eastern China. Due to the clean air actions, PM 2.5 concentrations have decreased effectively at a rate over 50 µg/m 3 /decade in the North China Plain and 20–50 µg/m 3 /decade over eastern China during 2014–2019. The newly generated dataset of 1-degree gridded PM 2.5 concentrations for the past 40 years across China provides a useful means for investigating interannual and decadal environmental and climate impacts related to aerosols.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Antiviral discovery using sparse datasets by integrating experiments, molecular simulations, and machine learning

Computational methods have demonstrated success in identifying virucidal agents, effectively contributing to the discovery of novel virucidal molecules. In this study, we developed a machine learning (ML) model, trained on a small dataset, to predict inhibitors of human enterovirus 71 (EV71), a pathological agent that causes severe disease in children and immunocompromised adults. Despite the dataset’s limitation, comprising of only 36 compounds tested, our ML framework demonstrated significant predictive capability. Notably, experimental validation revealed that five out of the eight compounds predicted by our model from the Chinese cosmetic material list exhibited virucidal activity. The inhibitor effects displayed by the main active compounds were further confirmed by molecular dynamics simulation. This underscores the potential of our AI-driven approach to bypass data constraints in identifying active molecules against viral pathogens.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Long-term trends in urban NO 2 concentrations and associated paediatric asthma incidence: estimates from global datasets

Combustion-related nitrogen dioxide (NO 2 ) air pollution is associated with paediatric asthma incidence. We aimed to estimate global surface NO 2 concentrations consistent with the Global Burden of Disease study for 1990-2019 at a 1 km resolution, and the concentrations and attributable paediatric asthma incidence trends in 13 189 cities from 2000 to 2019. We scaled an existing annual average NO 2 concentration dataset for 2010-12 from a land use regression model (based on 5220 NO 2 monitors in 58 countries and land use variables) to other years using NO 2 column densities from satellite and reanalysis datasets. We applied these concentrations in an epidemiologically derived concentration-response function with population and baseline asthma rates to estimate NO 2 -attributable paediatric asthma incidence. We estimated that 1.85 million (95% uncertainty interval [UI] 0.93-2.80 million) new paediatric asthma cases were attributable to NO 2 globally in 2019, two thirds of which occurred in urban areas (1.22 million cases; 95% UI 0.60-1.8 million). The proportion of paediatric asthma incidence that is attributable to NO 2 in urban areas declined from 19.8% (1.22 million attributable cases of 6.14 million total cases) in 2000 to 16.0% (1.24 million attributable cases of 7.73 million total cases) in 2019. Urban attributable fractions dropped in high-income countries (-41%), Latin America and the Caribbean (-16%), central Europe, eastern Europe, and central Asia (-13%), and southeast Asia, east Asia, and Oceania (-6%), and rose in south Asia (+23%), sub-Saharan Africa (+11%), and north Africa and the Middle East (+5%). The contribution of NO 2 concentrations, paediatric population size, and asthma incidence rates to the change in NO 2 -attributable paediatric asthma incidence differed regionally. Despite improvements in some regions, combustion-related NO 2 pollution continues to be an important contributor to paediatric asthma incidence globally, particularly in cities. Mitigating air pollution should be a crucial element of public health strategies for children.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2020

Constructing Self-Labeled Materials Imaging Datasets from Open Access Scientific Journals with EXSCLAIM!

Due to recent improvements in image resolution and acquisition speeds, materials microscopy is experiencing an explosion in imaging data. Yet, despite the volume of images generated, the overall accessibility landscape is highly fragmented, as researchers who do release images to the public, often only do so as snapshots of their larger private dataset in context of scientific journal publications. The effort to automatically consolidate images and descriptive information from web-based platforms has garnered broad attention from the computer vision, language technologies, and chemistry/materials informatics communities. However, these methods are problematic for scientific figures because over 30% of figures are compound in nature, and it is the individual images themselves, paired with relevant context, that are necessary for construction a proper labeled dataset. To this end, we outline in this paper the design of a software pipeline for the automatic EXtraction, Separation, and Caption-based natural Language Annotation of IMages from scientific figures (EXSCLAIM!). Successful consolidation of materials imaging across literature sources will enhance navigation and searchability of materials microscopy images for both novice and experienced researchers, as well as establish the framework necessary for users to search by images, text, or some combination of both.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Insights From Dayflow: A Historical Streamflow Reanalysis Dataset for the Conterminous United States

Abstract Reconstructed historical streamflow time series can supplement limited streamflow gauge observations. However, there are common challenges of typical modeling approaches: process‐based hydrologic models can be data/computation‐intensive, and statistics‐based models can be region/stream‐specific. Here we present a nationally scalable modeling framework integrating the simulated runoff from the Variable Infiltration Capacity (VIC) model with the Routing Application for Parallel computatIon of Discharge (RAPID) routing model leveraging high‐performance computing. We demonstrate an efficient method of assimilating streamflow at US Geological Survey (USGS) streamflow monitoring sites using a simple hierarchical approach in the VIC‐RAPID framework. The result is a reconstructed 36‐year (1980–2015) daily and monthly streamflow dataset (Dayflow) at ∼2.7 million NHDPlusV2 stream reaches in the conterminous US (CONUS). We perform a comprehensive evaluation at 7,526 USGS sites and characterize their error statistics. The results demonstrate that 49% of the USGS sites demonstrate Kling–Gupta Efficiency (KGE) > 0.5 and 58% of the sites show percentage bias within ±20% for the daily naturalized streamflow. Streamflow data assimilation across CONUS shows an overall improvement over naturalized streamflow, notably in the western semiarid‐to‐arid regions. Comparison to other national and global streamflow reanalysis datasets such as the National Water Model and Global Reach‐scale A priori Discharge Estimates for SWOT demonstrates improved KGE, reduced bias, and directions for Dayflow improvements. Investigations of error statistics with key hydrologic, hydroclimatic, and geomorphologic basin characteristics reveal region‐specific patterns which may help improve future framework applications. Overall, Dayflow may enable a better understanding of hydrologic conditions in a changing environment, especially in locations currently not represented by streamflow monitoring networks.

54 ENVIRONMENTAL SCIENCES↗