Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “dataset”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29
All sourcesNASA NTRSDOE OSTI
NASA NTRS2016

Outcomes of a NASA Workshop to Develop a Portfolio of Low Latency Datasets for Time-Sensitive Applications

It is widely accepted that time-sensitive remote sensing data serve the needs of decision makers in the applications communities and yet to date, a comprehensive portfolio of NASA low latency datasets has not been available. This paper will describe the NASA low latency, or Near-Real Time (NRT), portfolio, how it was developed and plans to make it available online through a portal that leverages the existing EOSDIS capabilities such as the Earthdata Search Client (https:search.earthdata.nasa.gov), the Common Metadata Repository (CMR) and the Global Imagery Browse Service (GIBS). This paper will report on the outcomes of a NASA Workshop to Develop a Portfolio of Low Latency Datasets for Time-Sensitive Applications (27-29 September 2016 at NASA Langley Research Center, Hampton VA). The paper will also summarize findings and recommendations from the meeting outlining perceived shortfalls and opportunities for low latency research and application science.

remote sensing↗
DOE OSTI2021

Assessing data change in scientific datasets

Summary Scientific datasets are growing rapidly and becoming critical to next‐generation scientific discoveries. The validity of scientific results relies on the quality of data used and data are often subject to change, for example, due to observation additions, quality assessments, or processing software updates. The effects of data change are not well understood and difficult to predict. Datasets are often repeatedly updated and recomputing derived data products quickly becomes time consuming and resource intensive and may in some cases not even be necessary, thus delaying scientific advance. Despite its importance, there is a lack of systematic approaches for best comparing data versions to quantify the changes, and ad‐hoc or manual processes are commonly used. In this article, we propose a novel hierarchical approach for analyzing data changes, including real‐time (online) and offline analyses. We employ a variety of fast‐to‐compute numerical analyses, graphical data change representations, and more resource‐intensive recomputations of a subset of the data product. We illustrate the application of our approach using three scientific diverse use cases, namely, satellite, cosmological, and x‐ray data. The results show that a variety of data change metrics should be employed to enable a comprehensive representation and qualitative evaluation of data changes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2013

Updated high-resolution grids of monthly climatic observations – the CRU TS3.10 Dataset

This paper describes the construction of an updated gridded climate dataset (referred to as CRU TS3.10) from monthly observations at meteorological stations across the world's land areas. Station anomalies (from 1961 to 1990 means) were interpolated into 0.5° latitude/longitude grid cells covering the global land surface (excluding Antarctica), and combined with an existing climatology to obtain absolute monthly values. The dataset includes six mostly independent climate variables (mean temperature, diurnal temperature range, precipitation, wet‐day frequency, vapor pressure and cloud cover). Maximum and minimum temperatures have been arithmetically derived from these. Secondary variables (frost day frequency and potential evapotranspiration) have been estimated from the six primary variables using well‐known formulae. Time series for hemispheric averages and 20 large sub‐continental scale regions were calculated (for mean, maximum and minimum temperature and precipitation totals) and compared to a number of similar gridded products. This gridded product will be publicly available, including the input station series (http://www.cru.uea.ac.uk and http://badc.nerc.ac.uk/data/cru/).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2014

Signatures of naturally induced variability in the atmosphere using multiple reanalysis datasets

A multiple linear regression analysis of nine different reanalysis datasets has been performed to test the robustness of variability associated with volcanic eruptions, the El Niño Southern Oscillation, the Quasi-Biennial Oscillation and with a specific focus on the 11-year solar cycle. The analysis covers both the stratosphere and troposphere and extends over the period 1979–2009. The characteristic signals of all four sources of variability are remarkably consistent between the datasets and confirm the responses seen in previous analyses. In general, the solar signatures reported are primarily due to the assimilation of observations, rather than the underlying forecast model used in the reanalysis system. Analysis of the 11-year solar response in the lower stratosphere confirms the existence of the equatorial temperature maximum, although there is less consistency in the upper stratosphere, probably reflecting the reduced level of assimilated data there. The solar modulation of the polar jet oscillation is also evident, but only significant during February. In the troposphere, vertically banded anomalies in zonal mean zonal winds are seen in all the reanalyses, with easterly anomalies at 30°N and 30°S suggesting a weaker and possibly broader Hadley circulation under solar maximum conditions. This structure is present in the annual signal and is particularly evident in NH wintertime. As well as the ‘top-down’ solar contribution to Northern Annular Mode variability, we show the potential contribution from the surface conditions allowing for a ‘bottom-up’ pathway. Finally, the reanalyses are compared with both observed global-mean temperatures from the Stratospheric Sounding Unit (SSU) and from the latest general circulation models from CMIP-5. The SSU samples the stratosphere over three different altitudes, and the 11-year solar cycle fingerprint is identified in these observations using detection and attribution techniques.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

A Dataset of CFD Simulated Industrial Furnace Images for Conditional Automatic Generation with GANs

The steel industry is constantly looking for ways to automate processes and improve efficiency. A standard practice in industry is to simulate how complex systems will operate before they are actually used. Some complex systems, including steel industry processes such as blast furnaces, require complex physics-based simulations utilizing computational fluid dynamics (CFD). These CFD physics-based simulations are very accurate but can take significant time and computational resources to process, resulting in challenges for the implementation of the models in real-world operational environments. In recent years, deep learning (DL) has been considered as a substitute for these CFD models. DL models can be trained on validated CFD simulation data and then used for industrial process inference. Previous DL-based solutions have made great contributions for industrial automation but are currently missing the additional visualization component that CFD simulations also provide. In this paper, we propose a dataset for simple DL generative approaches that can help to address this issue. The dataset and methodology under development to approach this prediction are discussed in this work.

Calix, Ricardo↗
DOE OSTI2013

Quantifying and relating land-surface and subsurface variability in permafrost environments using lidar and surface geophsical datasets

The complexity of permafrost dynamics and its critical impact on climate feedbacks warrant continued development of advanced high-latitude terrestrial ecosystem characterization and monitoring approaches. In this study, we explore the value of remote sensing and surface geophysical data for characterizing land surface and subsurface properties and their linkages in an Alaskan Coastal Plain ecosystem. We base our study on data collected at the end of the 2011 growing season in the Barrow Environmental Observatory, where a nested suite of measurements were collected within a polygon-dominated region including: surface ground penetrating radar, electromagnetic, and electrical resistance tomography data; thaw depth, soil temperature and moisture content, soil texture, soil carbon and nitrogen content, and major and trace cations. Previously-collected lidar data were also available for the study. Analysis of the datasets, individually and in combination, revealed the utility of the methods for characterizing critical land-surface and subsurface properties and associated spatial zonation. Lidar analysis was performed to extract geomorphic metrics (such as slope, curvature, and directed distance of polygons), which potentially indicate drainage potential and permafrost deformation state. Cluster analysis of these lidar-obtained attributes suggested that the land surface can be grouped into three spatially coherent zones, each having a dominant geomorphic expression including: a high centered polygon zone, a low centered polygon zone and a transitional zone. Comparison of the geophysical attributes from radar, electrical resistance tomography, and electromagnetic data with point measurements suggests that the surface geophysical data can provide very high-resolution information about subsurface properties that affect ecosystem feedbacks to climate, such as thaw depth and moisture content. Cluster analysis suggested that the geophysical attributes also varied spatially in a systematic way, suggesting the presence of three laterally distinct subsurface zones. Analysis of zone-based subsurface point measurements suggests that the geophysically-defined zones have unique distributions of hydrological, thermal, and geochemical properties and that the subsurface (geophysically-based) and land-surface (lidar-based) zonation is consistent. Although the close linkage between land surface (polygonal geomorphology) and subsurface (active layer) variability revealed through our study is not surprising, to our knowledge this is the first study to document such relationships using high resolution and non-invasive approaches. This study suggests the potential of using coincident lidar and surface geophysical measurements to quantify land surface and subsurface properties (respectively) and their linkages, which are likely to play a role in terrestrial ecosystem evolution and feedbacks to climate. These findings open the way for future research focused on using combined geophysical and remote sensing datasets to estimate subsurface and land-surface properties in high resolution and over large regions as is needed for process understanding and numerical model initialization in high latitude terrestrial ecosystems.

Climate↗
DOE OSTI2025

Enhancing Interpretability in Generative Modeling: Statistically Disentangled Latent Spaces Guided by Generative Factors in Scientific Datasets

This study addresses the challenge of statistically extracting generative factors from complex, high-dimensional datasets in unsupervised or semi-supervised settings. We investigate encoder-decoder-based generative models for nonlinear dimensionality reduction, focusing on disentangling low-dimensional latent variables corresponding to independent physical factors. Introducing Aux-VAE, a novel architecture within the classical Variational Autoencoder framework, we achieve disentanglement with minimal modifications to the standard VAE loss function by leveraging prior statistical knowledge through auxiliary variables. These variables guide the shaping of the latent space by aligning latent factors with learned auxiliary variables. We validate the efficacy of Aux-VAE through comparative assessments on multiple datasets, including astronomical simulations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

An Open Combinatorial Diffraction Dataset Including Consensus Human and Machine Learning Labels with Quantified Uncertainty for Training New Machine Learning Models

Modern machine learning and autonomous experimentation schemes in materials science rely on accurate analysis of the data ingested by these models. Unfortunately, accurate analysis of the underlying data can be difficult, even for domain experts, complicating the training of the models intended to drive experiments. This is especially true when the goal is to identify the presence of weak signatures in diffraction or spectroscopic datasets. In this work, we examine a set of as-obtained diffraction data that track the phase transition from monoclinic to tetragonal in a Nb-doped VO2 film as a function of temperature and dopant concentration. We then task a set of domain experts and a set of machine learning experts with identifying which phase is present in each diffraction pattern manually and algorithmically, respectively; in both cases, the labels can vary dramatically, especially at the phase boundaries. We use the mode of the labels and the Shannon entropy as a method to capture, preserve and propagate consensus labels and their variance. Further we use the expert labels as a benchmark and demonstrate the use of Shannon entropy weighted scoring to test the performance of machine learning generated labels. Finally, we propose a material data challenge centered around generating improved labeling algorithms. This real-world dataset curated with expert labels can act as test bed for new algorithms. The raw data, annotations and code used in this study are all available online at data.gov and the interested reader is encouraged to replicate and improve the existing models

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Segmentation of tomography datasets using 3D convolutional neural networks

Dendritic microstructures are ubiquitous in nature and are the primary solidification morphologies in metallic materials. Techniques such as X-ray computed tomography (XCT) have provided new insights into dendritic phase transformation phenomena. However, manual identification of dendritic morphologies in microscopy data can be both labor intensive and potentially ambiguous. The analysis of 3D datasets is particularly challenging due to their large sizes (terabytes) and the presence of artifacts scattered within the imaged volumes. Here, in this study, we trained 3D convolutional neural networks (CNNs) to segment 3D datasets. Three CNN architectures were investigated, including a new version of FCDenseNet which we extended to 3D. We show that using hyperparameter optimization (HPO) and fine-tuning techniques, both 2D and 3D CNN architectures outperform the previous state of the art. The 3D U-Net architecture trained in this study produced the best segmentations according to quantitative metrics (intersection-over-union of 95.56% and a boundary displacement error of 0.58 pixels), while 3D FCDense produced the smoothest boundaries and best segmentations according to visual inspection. The trained 3D CNNs are able to segment entire 852 × 852 × 250 voxel 3D volumes in only ~60 s, thus hastening the progress towards a deeper understanding of phase transformation phenomena such as dendritic solidification.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Areal interpolation of population projections consistent with different SSPs from 1-km resolution to block level based on USA Structures dataset

Population data are normally collected at various census administrative levels, and areal interpolation of population is often required to transform population to the desired spatial resolution. Building footprint datasets, such as Microsoft building footprints, have proven to be useful in estimating population distribution and can therefore be used for areal interpolation of population. In addition to Microsoft building footprints, the recently released USA Structures dataset provides additional information such as building type and building height for some regions, which may provide valuable information for a better depiction of population distribution and improved population areal interpolation accuracy. In this study, we have conducted areal interpolation of population projections consistent with three different Shared Socioeconomic Pathways (SSP2, SSP3, and SSP5) from 1-km grid cells to block level in Washington state for every ten years from 2020 to 2040 based on USA Structures. We assessed USA Structures-based population downscaling accuracy using U.S. decennial survey data in 2020 under three different downscaling schemes, including population downscaling from census tracts to block groups, from census tracts to blocks, and from block groups to blocks. The resulting accuracies were compared with those based on Microsoft building footprints. The comparison showed that USA Structures achieved higher accuracies across different population density regions and areas with different urbanization extent within our study area.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2022

AIMSim : An accessible cheminformatics platform for similarity operations on chemicals datasets

The recent advances in deep learning, generative modeling, and statistical learning have ushered in a renewed interest in traditional cheminformatics tools and methods. Quantifying molecular similarity is essential in molecular generative modeling, exploratory molecular synthesis campaigns, and drug-discovery applications to assess how new molecules differ from existing ones. Further, most tools target advanced users and lack general implementations accessible to the larger community. In this work, we introduce Artificial Intelligence Molecular Similarity (AIMSim), an accessible cheminformatics platform for performing similarity operations on collections of molecules called molecular datasets. AIMSim provides a unified platform to perform similarity-based tasks on molecular datasets, such as diversity quantification, outlier and novelty analysis, clustering, dimensionality reduction, and inter-molecular comparisons. AIMSim implements all major binary similarity metrics and molecular fingerprints and is provided as a Python package that includes support for command-line use as well as a Graphical User Interface for code-free utilization with fully interactive plots.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Dataset of simulated vibrational density of states and X-ray diffraction profiles of mechanically deformed and disordered atomic structures in Gold, Iron, Magnesium, and Silicon

This dataset is comprised of a library of atomistic structure files and corresponding X-ray diffraction (XRD) profiles and vibrational density of states (VDoS) profiles for bulk single crystal silicon (Si), gold (Au), magnesium (Mg), and iron (Fe) with and without disorder introduced into the atomic structure and with and without mechanical loading. Included with the atomistic structure files are descriptor files that measure the stress state, phase fractions, and dislocation content of the microstructures. All data was generated via molecular dynamics or molecular statics simulations using the Large-scale Atomic/Molecular Massively Parallel Simulator (LAMMPS) code. This dataset can inform the understanding of how local or global changes to a materials microstructure can alter their spectroscopic and diffraction behavior across a variety of initial structure types (cubic diamond, face-centered cubic (FCC), hexagonal close-packed (HCP), and body-centered cubic (BCC) for Si, Au, Mg, and Fe, respectively) and overlapping changes to the microstructure (i.e., both disorder insertion and mechanical loading).

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2017

Play fairway analysis of geothermal resources across the State of Hawaii: 1. Geological, geophysical, and geochemical datasets

Phase 1 of a Play Fairway Analysis (PFA) of geothermal resource potential across the State of Hawaii was recently completed. The final products of this work include a statewide geothermal resource probability map, a map of confidence in this probability, an assessment of the viability of development in areas of interest, and a prioritized list of recommended future exploration activities. The intersection of subsurface heat (H), permeability (P), and fluid (F) is necessary for an optimum geothermal play. This study: (1) identified and compiled all legacy and current data relevant to Hawaii’s geothermal resource; (2) ranked these datasets in terms of their relevance to subsurface H, P, F; (3) developed a Bayesian statistical method to incorporate the data and their rankings, and produce a statewide resource probability map; (4) developed a method to assess confidence in the probability values; and (5) assessed what we term ‘development viability’ in resulting areas of interest across the state. This paper details the basic project workflow, and activities (1) and (2). It describes the rationale for including this study’s datasets: surface geologic mapping data (calderas, rift zones, volcanic vents, dikes, faults), groundwater data (temperature, Chloride:Magnesium, SiO 2 ) and geophysical data (gravity, magnetotelluric, seismic, geodetic strain), and justifies their relative rankings in terms of H, P, and F. A second paper describes activities (3) and (4) related to the statistical methodology, and a third paper describes activity (5), the development viability criteria, as well as our suggested roadmap for future exploration activities. Overall, we find that the likelihood of abundant of geothermal resources is highest on the youngest island of Hawaii, but groundwater indicators suggest there may also be resources on the other, older islands. Furthermore, the higher demand for renewable energy on the more populated islands of Maui and Oahu, as well as a high viability of development on Lanai, also motivate further exploration on these islands.

15 GEOTHERMAL ENERGY↗
DOE OSTI2020

A multi-dimensional dataset for two-phase instability in low pressure natural circulation based on direct transient local measurement

Thermal hydraulic instability can challenge the control and the safety of nuclear reactors. This experimental research captures and investigates the instability in a low-pressure (1–10 atmospheres) natural circulation loop with an adiabatic chimney installed above the heated section. Direct transient local measurement of two-phase flow is performed at multiple axial locations along the flow channel with thermocouples and conductivity probes traversed by customized high-precision mechanisms. Based on the periodicity of oscillations, a post-processing method is proposed to align the data captured non-simultaneously to reconstruct the instability behavior in one representative period. The averaging operation over multiple oscillation cycles is performed to reduce the random errors. Cyclic phenomena are clearly demonstrated by the post-processed data, and rapid void fraction change induced by the flashing is observed to be significant in the driving mechanism for low pressure oscillations. Parametric effects of inlet subcooling, heat flux, inlet flow restriction and system pressure are studied within the experiment range, and qualitative results are explained and compared with existing literature. The ratio between the oscillation period and the travelling time of the fluid particle through the test section is compared with the theoretical value for the density wave oscillation. The new dataset contains directly measured transient multi-dimensional two-phase data along the flow channel for natural circulation under density wave oscillations. The generated dataset is valuable for the validation of system analysis codes and CFD codes in the simulation of natural circulation phenomena and dynamic two-phase flow.

42 ENGINEERING↗
DOE OSTI2021

Class imbalance in out-of-distribution datasets: Improving the robustness of the TextCNN for the classification of rare cancer types

In the last decade, the widespread adoption of electronic health record documentation has created huge opportunities for information mining. Natural language processing (NLP) techniques using machine and deep learning are becoming increasingly widespread for information extraction tasks from unstructured clinical notes. Disparities in performance when deploying machine learning models in the real world have recently received considerable attention. In the clinical NLP domain, the robustness of convolutional neural networks (CNNs) for classifying cancer pathology reports under natural distribution shifts remains understudied. In this research, we aim to quantify and improve the performance of the CNN for text classification on out-of-distribution (OOD) datasets resulting from the natural evolution of clinical text in pathology reports. We identified class imbalance due to different prevalence of cancer types as one of the sources of performance drop and analyzed the impact of previous methods for addressing class imbalance when deploying models in real-world domains. Our results show that our novel class-specialized ensemble technique outperforms other methods for the classification of rare cancer types in terms of macro F1 scores. We also found that traditional ensemble methods perform better in top classes, leading to higher micro F1 scores. Based on our findings, we formulate a series of recommendations for other ML practitioners on how to build robust models with extremely imbalanced datasets in biomedical NLP applications.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2012

Analysis and compression of six-dimensional gyrokinetic datasets using higher order singular value decomposition

Higher order singular value decomposition (HOSVD) is explored as a tool for analyzing and compressing gyrokinetic data. An efficient numerical implementation of an HOSVD algorithm is described. HOSVD is used to analyze the full six-dimensional (three spatial, two velocity space, and time dimensions) gyrocenter distribution function from gyrokinetic simulations of ion temperature gradient, electron temperature gradient, and trapped electron mode driven turbulence. The HOSVD eigenvalues for the velocity space coordinates decay very rapidly, indicating that only a few structures in velocity space can capture the most important dynamics. In almost all of the cases studied, HOSVD extracts parallel velocity space structures which are very similar to orthogonal polynomials. HOSVD is also used to compress gyrokinetic datasets, an application in which it is shown to significantly outperform the more commonly used singular value decomposition. It is shown that the effectiveness of the HOSVD compression improves as the dimensionality of the dataset increases. (C) 2012 Elsevier Inc. All rights reserved.

equation↗
DOE OSTI2026

Re-evaluating probable maximum precipitation estimates: sensitivity to transposition domains and storm rotation using modern datasets

This study examines the sensitivity of Probable Maximum Precipitation (PMP) estimates to key methodological decisions embedded in the legacy approach adopted in the U.S. National Weather Service Hydrometeorological Reports No. 51 and No. 52. Although widely used for infrastructure design and risk regulation, fundamental aspects of PMP estimation—such as storm sample size, transposition domain, maximization procedures, and storm rotation—remain poorly constrained and lack formal guidance. Using the Red Rock watershed in Iowa as a case study, and leveraging the 2002–2023 NOAA Analysis of Record for Calibration (AORC) precipitation dataset, we systematically evaluate how each methodological choice, individually and in combination, influences PMP estimates. Our findings demonstrate that PMP is not a fixed physical upper bound but rather a modeling construct shaped heavily by user-defined assumptions. Notably, PMP values derived from modern gridded rainfall datasets can be substantially higher than the legacy estimate used in the original spillway design for Red Rock Dam. Decisions regarding storm sample size, domain extent, climatological window, and particularly storm rotation all contributed to higher PMP estimates. Storm rotation alone—a loosely constrained element in the current PMP practice—can amplify PMP by more than 25%. These results reveal the lack of standardized bounds in current PMP workflows and the need for systematic sensitivity and uncertainty analysis. As PMP estimation shifts toward probabilistic approaches, incorporating physically meaningful storm attributes will be key to developing more transparent, defensible methods for dam safety and climate-resilient infrastructure.

Probable maximum precipitation↗