Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “datasets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23

Dataset for "Machine Learning Ensembles Can Enhance Hydrologic Predictions and Uncertainty Quantification" Willard et al. (2025).

This data release provides all data and code used in the paper " "Machine Learning Ensembles Can Enhance Hydrologic Predictions and Uncertainty Quantifications" Willard et al. (2025)" to model stream temperature, evaluate, and assess results. The associated manuscript explores the effect of different ensemble construction techniques across different common machine learning (ML) architectures for predictions in unmonitored basins. Modeling was done using long short-term memory (LSTM), gated recurrent unit (GRU), temporal convolution network (TCN), and extreme gradient boosting (XGBoost) models, and stream site coverage spans 1362 locations across the conterminous United States. The ensemble construction techniques investigated include ensemble by random weight initialization, differing hyperparameters, different random subsets of training data, different subselections of input features, different architectures, and Monte Carlo Dropout. The data is organized into these items items:Code repository and data for the paper " "Machine Learning Ensembles Can Enhance Hydrologic Predictions and Uncertainty Quantifications" Willard et al. (2025).Code: stream_temp_ml_regionalization.zip contains the code repositoryData to run the code:- data_dir.zip -- contains all files that should be moved to the "DATA_DIR" variable defined in the "set_env_vars.sh" script in the code repository- metadata_dir.zip -- contains all files that should be moved to the "METADATA_DIR" variable defined in the "set_env_vars.sh" script in the code repositoryData produced by the code and used in the paper:- outputs_dir.zip - contains model output and results (outputs_dir/results), model weights (outputs_dir/models), and all other outputs used for the paper including feature importances.To cite this code, please use the following BibTeX or MLA entries:bibtex:@misc{willard2025streamensembles,author = {Jared Willard and Charuleka Varadharajan},title = {Dataset for "Machine Learning Ensembles Can Enhance Hydrologic Predictions and Uncertainty Quantification"},year = {2024},doi = {10.15485/2527393},publisher = {ESS-DIVE Repository},url = {https://data.ess-dive.lbl.gov/datasets/doi:10.15485/2527393}}MLA: Willard, Jared, et al. Dataset for "Machine Learning Ensembles Can Enhance Hydrologic Predictions and Uncertainty Quantification". 2025. ESS-DIVE Repository, doi:10.15485/2448016.

54 ENVIRONMENTAL SCIENCES↗

CROCUS Air Quality Dataset from the University of Illinois Chicago (UIC), July 2024

This dataset was collected by the measurement system in the Atmosphere, Climate, and Ecosystems (ACE) Lab at the University of Illinois Chicago (UIC) from July 12 to July 31, 2024, as part of the Community Research on Climate and Urban Science (CROCUS) Urban Integrated Field Laboratory (UIFL) project, led by Argonne National Laboratory.To enhance understanding of urban air quality dynamics in Chicago, and as part of the CROCUS 2024 Urban Canyon Intensive Observation Period (IOP), several instruments were set up to provide continuous measurements of air quality parameters in Chicago during July 2024. These measurements cover both aerosols and gas-phase species. It focuses on particle size distribution (2.5–478 nm) measured by two Scanning Mobility Particle Sizers (SMPS) at a 4-min resolution, total particle number concentrations at a 1-s resolution, and chemical composition from a High-Resolution Time-of-Flight Aerosol Mass Spectrometer (AMS) at a 1-min resolution. Key gas-phase species, including NO, NO₂, SO₂, and O₃, are measured at a 1-min resolution, along with high-resolution NO and dimethyl sulfide (DMS) data from a Chemical Ionization Mass Spectrometer (CIMS). Volatile organic compound (VOC) data for toluene, isoprene, and benzene are provided by a GC-PID with a time resolution of 25 minutes.The data are formatted as NetCDF (.nc) files, making them easily accessible using common software such as MATLAB, R, and Python. Each parameter is stored in an individual dataset, which includes detailed instrument information in the header, as well as the corresponding sample start time and concentration/distribution data for each sample.

54 ENVIRONMENTAL SCIENCES↗

Comprehensive GOM Federal Waters Platform, Incident, Metocean, and Geohazard Dataset

The dataset contains integrated data from an array of disparate data sources, all spatially and temporally linked to platforms in the federal waters of the Gulf of Mexico (platform data from BSEE, 2020). Integrated data includes past reported incidents dating back to 1956 (BSEE, BOEM, MMS), metocean data (see Nelson et al. in review for source information), and geohazard data (see Nelson et al. in review for source information). Proprietary well production information was redacted from this dataset, but was used in resulting analytics.

Full System↗

U.S. Offshore Pipeline and Reported Incident Datasets

The U.S. Offshore Pipeline and Reported Incident Datasets provide a compilation of data from a variety of credible resources, spatially-temporally integrated into multivariate resources. This spatial resource includes more than 80,000 points along existing and abandoned pipelines in the Gulf with matched incidents based on similar lease blocks and temporal timelines (e.g., the incident date occurs within reported pipeline lifespan), structural characteristics, geologic and seafloor data, and meteorological, oceanographic, and biochemical statistics spatially and temporally matched to each point. This is provided as both a feature class in a file geodatabase, as well as a CSV file for ease of use. The pipeline incidents table is a CSV file containing more than 900 reported incidents from 1986 to 2021, including incident date, area (Outer Continental Shelf (OCS) lease block and area code), reported causes, reported incident information, and results (i.e., cost, repairs, inspections), along with quantitative severity metrics. Field dictionaries are included for both the pipeline locations and incidents datasets, which detail field definitions. The pipeline locations field dictionary includes original resource reference information.

Advanced Infrastructure Integrity Model↗

deeplenstronomy: A dataset simulation package for strong gravitational lensing

Automated searches for strong gravitational lensing in optical imaging survey datasets often employ machine learning and deep learning approaches. These techniques require more example systems to train the algorithms than have presently been discovered, which creates a need for simulated images as training dataset supplements. This work introduces and summarizes deeplenstronomy, an open-source Python package that enables efficient, large-scale, and reproducible simulation of images of astronomical systems. A full suite of unit tests, documentation, and example notebooks are available at https://deepskies.github.io/deeplenstronomy/ .

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Uncertainties associated with GAN-generated datasets in high energy physics

Recently, Generative Adversarial Networks (GANs) trained on samples of traditionally simulated collider events have been proposed as a way of generating larger simulated datasets at a reduced computational cost. In this paper we point out that data generated by a GAN cannot statistically be better than the data it was trained on, and critically examine the applicability of GANs in various situations, including a) for replacing the entire Monte Carlo pipeline or parts of it, and b) to produce datasets for usage in highly sensitive analyses or sub-optimal ones. We present our arguments using information theoretic demonstrations, a toy example, as well as in the form of a formal statement, and identify some potential valid uses of GANs in collider simulations.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

A Publicly Available, Annotated Dataset for Naturalistic Driving Study and Computer Vision Algorithm Development

Oak Ridge National Laboratory developed and implemented a data collection effort to create a dataset for use in evaluating and testing algorithms for analyzing driver behavior under controlled settings for support of the Federal Highway Administration’s Exploratory Advanced Research Program. This collection is called the ORNL Naturalistic Driving Study Sample (ONDSS). The dataset is designed to emulate aspects of the Second Strategic Highway Research Project (SHRP2), which contained a massive naturalistic driving study (NDS) with over 3000 drivers between 2010 and 2013 using their personal vehicles, with over 4300 person-years of data collected [HANKEY].

42 ENGINEERING↗

A library of AI-assisted FAIR water cycle and related disturbance datasets to enable model training, parameterization and validation

This whitepaper is responsive to focal area Data acquisition and assimilation enabled by machine learning, AI, and advanced methods. Here we describe how FAIR (Findable, Accessible, Reusable, Interoperable) datasets related to water cycle extremes are essential for successful implementation of ML in Earth System and other models. We also describe how AI can be used to acquire and integrate water cycle data related to extreme events to create a library of FAIR datasets for training and evaluating algorithms.

58 GEOSCIENCES↗

Global normal spectral irradiance in Albuquerque: a one-year open dataset for PV research

This report describes the creation process and final content of a spectral irradiance dataset for Albuquerque NM. The spectral irradiance measurements were made using a dual-axis tracker; therefore, they represent global normal irradiance. The dataset combines spectroradiometer and weather measurements from a two-year period into a continuous calendar year. The data files are accompanied by extensive metadata as well as example calculations and graphs to demonstrate the potential uses of this database.

14 SOLAR ENERGY↗

Evaluation of Data Catalog Software for Hanford Site Environmental Datasets

Environmental information and data underpin achievement of the U.S. Department of Energy (DOE) Office of Environmental Management (EM) mission at the Hanford Site. The Hanford Environmental Data Management (HEDM) Program is the DOE Richland Operations Office (RL) approach to develop and implement a formal program for managing environmental data and the associated records, materials, and systems at the Hanford Site. The current project, contract, organization, and contractor-specific efforts at managing environmental data sets are insufficient to provide orderly, long-term, site-wide access. A vital element to be created within the HEDM program plan is a catalog of data sources, called the Hanford Environmental Information and Data Index (HEIDI), that will enable long-term access and retrievability for the multiple independent sources of data that might otherwise be difficult to discover. This report compares leading open source and commercial data catalog platforms using criteria to assess the functionality needed to develop the HEIDI catalog of Hanford data sources that connects and exchanges data with established Hanford Local Area Network (HLAN) enterprise information technology systems. Proprietary platforms evaluated included ArcGIS Enterprise Sites, Junar, OpenDataSoft, and Socrata, and non-proprietary platforms included Energy Data eXchange (EDX), Comprehensive Knowledge Archive Network (CKAN), and DKAN (a Drupal-based open data portal based on CKAN). Capabilities supporting data discoverability, retrieval, and archival, as well as metadata standard requirements and integration into the HLAN were rated as either failing to meet requirements (F), meeting requirements (M), or exceeding requirements by delivering additional desired features (E). The lowest rating for any capability area was assigned as the overall rating for the platform. These findings enable DOE-RL and the contractors implementing the HEDM plan to focus on candidate tools likely to meet the requirements for implementing HEIDI. All of the platforms receiving an overall rating of ‘F’ were unable to be deployed on Hanford infrastructure or within dedicated cloud resources. A propriety software-as-a-service (SaaS) model of delivering a data catalog (e.g., found in software such as Junar and OpenDataSoft) favors consistency across customers at the expense of customization and configurable roles that are needed for Hanford work. Hosting data on a shared commercial platform places limits on dataset size (maximum of 240 Mb for OpenDataSoft), a significant limitation for HEIDI implementation. EDX, a government data catalog based on CKAN, received the ‘F’ rating due to an inability to incorporate authentication from HLAN into the system. Among platforms rated ‘M’ or ‘E’, only the Socrata platform had a SaaS delivery model. In contrast to other SaaS platforms, Socrata provided custom roles and gateways that allow local datasets to be incorporated into an online catalog. Socrata also complies with the Federal Risk and Authorization Management Program, a significant benefit for cloud-based management of Hanford data. The other platforms rated ‘M’ or ‘E’, ArcGIS Enterprise Sites, CKAN, and DKAN, provide fully self-hosted options, allowing for greater control and flexibility with the HEIDI catalog. These widely used tools have supportive communities of practice, extensive customization options, and demonstrated deployments that provide evidence that they can meet requirements, often deliver additional desired features, and work well with federal government systems. Completely customized alternatives built on a collection of applications were not evaluated because achieving similar performance to CKAN or DKAN requires substantial resources, especially in the absence of the active communities that have grown to support these tools. ArcGIS Enterprise Sites, Socrata, CKAN, and DKAN were evaluated as strong candidates for successful implementation with HEIDI.

54 ENVIRONMENTAL SCIENCES↗

Surface 3D Electrical Resistivity Tomography Inversion of 2005 BC Cribs and Trenches Datasets

Hydrogeophysics, Inc. (HGI) conducted an electrical resistivity tomography (ERT) dataset at the BC Cribs and Trenches site, located in the Central Plateau of the Hanford Site. The 20 trenches and 6 cribs received large volumes of liquid inorganic waste in the 1950s, resulting in a large inventory of contaminants in the vadose zone. The objective of the ERT survey was to map plume extents resulting from the legacy discharges. The HGI interpretation of the resistivity data was performed using geometric inversion to interpolate 2D lines into a 3D image. To demonstrate a newly developed geophysical code capability (E4D), the resistivity data were re-processed to fit a full 3D model of the bulk electrical conductivity. This proof-of-concept model inversion was executed in calendar year 2011, as the large dataset was well-suited for the use of high-performance computing. The 3D re-processing of the BC Cribs and Trenches ERT data conducted in 2011 resolved the true bulk electrical conductivity. This means that all of the resistivity data were fit to a single model of the bulk electrical conductivity, with true horizontal and vertical dimensions. This differed from the HGI data interpretation approach that used geometric inversion to process 2D lines independently, which were then interpolated into a 3D image. Both the full 3D re-processing and the 2D interpolation to a 3D image demonstrated a higher electrical conductivity observed immediately beneath the trenches and cribs. The electrical conductivity is strongly correlated with nitrate concentrations, indicating the presence of nitrate and other co-located contaminants.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

A High-Throughput Computing Infrastructure to Generate Custom, Open Community Geothermal Datasets

The most significant challenge facing geothermal research, development, and deployment is a lack of comprehensive datasets describing the geological and economical properties of North America. Automated knowledge base construction, the process of designing algorithms to analyze text and images to programmatically build new datasets, is one possible solution to this problem. The xDD library of full-text scientific articles (https://xdd.wisc.edu) is one of the largest collections of open and controlled-access scientific documents available for knowledge base construction in the world, but it has been underutilized by experts in geothermal research. The xDD development team attributed the lack of engagement by software developers and geothermal researchers to two perceived shortcomings of the system. First, the workflow for obtaining data from xDD for local development and testing of data mining applications was unnecessarily abstruse and required significant manual intervention by xDD systems administrators. Second, although xDD already held articles from a broad cross-section of scientific literature with an emphasis on the geosciences, it did not have an explicit set of geothermal research documents that could serve as the nucleus of a geothermal data mining application. To address these issues, the Automated Data Extraction PlaTform (ADEPT) was proposed to extend the data distribution capabilities of the xDD system. The ADEPT extension added the following four key features to xDD: 1) integration of National Geothermal Data System (NGDS) documents into the xDD library to provide an explicitly geothermally-themed collection; 2) improved RESTful (i.e., https-protocol driven) web services for external partners to access xDD data for machine learning application development; 3) a web platform for end-users and xDD administrators to coordinate the development of data mining applications from the initial step of browsing available documents to the final stage of deploying a production-quality machine learning application on high-throughput computing infrastructure; and 4) the development of demonstration data mining applications to illustrate the new workflow to potential collaborators. A total of 21,674 geothermal documents from NGDS were fully ingested into the xDD library and the associated metadata is publicly available through the xDD web services; furthermore, the ADEPT web platform is now publicly accessible and fully live at https://xdd.wisc.edu/adept/.

15 GEOTHERMAL ENERGY↗

End-Use Savings Shapes: Public Dataset Release for Residential Round 1 [Slides]

The End-Use Load Profiles project created a public database of 900,000 individual building end-use load profiles. Load profiles were modeled to represent the U.S. building stock as it was in 2018, as nearly as possible based on the best available data. The End-Use Savings Shapes follow-on project adds measure impact profiles for energy efficiency and electrification packages to the public dataset. This presentation details the public dataset release on September 20, 2022.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Developing a simulator-based satellite dataset for using machine learning techniques to derive aerosol-cloud-precipitation interactions in models and observations in a consistent framework

Aerosol-cloud-precipitation interactions (ACPI) remain a major uncertainty in understanding the Earth’s radiation budget and water cycle (including extremes). After decades of active research, various observationally based metrics have been developed to constrain ACPI in Earth System Models (ESMs), but direct comparison of model and data estimates can confound scientific understanding because limitations and uncertainties in sampling and retrieval procedures may combine with model deficiencies in process representations of ACPI to obstruct understanding. Furthermore, conventional ACPI metrics often vary from one regime to another, and the ACPI process representation in ESMs is also typically derived based on only a limited area/regime (even though the parameterization applies globally). To bridge the gap between models and data and to correctly describe ACPI in all regimes, we propose to construct a new CALIPSO-CloudSat merged dataset that is produced by the same algorithms used in satellite simulators in ESMs, and to use machine learning techniques to derive new ACPI metrics that can be accurately estimated by satellites and can provide meaningful constraints on cloud microphysical process representations in ESMs. The dataset will include measured and retrieved variables for aerosol, cloud, and precipitation from CALIPSO and CloudSat, and environmental variables from meteorological reanalysis. The data will be used to train a neural network to construct the ACPI metrics as a function of environmental conditions. The new ACPI formula will be used to constrain the ACPI in the Energy Exascale Earth System Model (E3SM), and to augment/reformulate the ACPI process representation in the E3SM to improve the simulation of the evolution of the atmosphere under different environmental conditions.

54 ENVIRONMENTAL SCIENCES↗

PV module operating conditions and temperature measurements: an open dataset for PV research

This report describes the structure and content of an open dataset created for the purpose of testing and validating PV module temperature prediction models and their parameters. The dataset contains the main environmental parameters that affect temperature: irradiance, ambient temperature, wind speed and down-welling infrared radiation, as well as measured back-of-module temperature.

14 SOLAR ENERGY↗

Global horizontal spectral irradiance and module spectral response measurements: an open dataset for PV research

This report describes the creation process and final content of a spectral irradiance dataset for Albuquerque, New Mexico accompanied by a set of spectral response measurements for modules deployed at the same location. The spectral irradiance measurements were made using horizontally mounted spectroradiometers; therefore, they represent global horizontal irradiance. The dataset combines non-continuous spectroradiometer and weather measurements from a two-year period into a single calendar year. The data files are accompanied by extensive metadata as well as example calculations and graphs to demonstrate the potential uses of this database. The spectral response measurements were carried out by the National Renewable Energy Laboratory using 12 commercial silicon modules types that are undergoing long-term evaluation at Sandia National Laboratories in Albuquerque.

14 SOLAR ENERGY↗

Accelerating engineered microbe optimization through machine learning and multi-omics datasets

This project demonstrated the use of a combination of multi-omics data with deep learning and a high-throughput Design- Build-Test-Learn (DBTL) cycle to improve the production of malonic acid, a versatile product with a large market. The project leveraged the unique capabilities of both Lygos and the Agile BioFoundry (ABF): Lygos provided its expertise efficiently designing, building, and cultivating P. kudriavzevii strains; LBNL, PNNL, and NTESS provided multi-omics analysis in the Test phase, LBNL provided machine learning techniques in the Learn phase to analyze the -omics datasets and make recommendations so as to increase malonic acid production in the next DBTL cycle. This project is the first to use large amounts of multi-omics time-series data to feed deep learning models, creating around 80,000 data points in a single DBTL cycle. This project has 1) demonstrated the utility of combining deep learning and multi-omics data sets by improving the production of malonic acid two fold, 2) created a large time-series datasets to be released publicly for external development of new machine learning algorithms, and 3) shown that supply chain problems, strain building bottlenecks, and adaptation times for new ML approaches are key obstacles for fast DBTL cycle times.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗