Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluation datasets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30
All sourcesNASA NTRSDOE OSTI
NASA NTRS2019

Evaluation of Classifier Complexity for Delay Tolerant Network Routing

The growing popularity of small cost effective satellites (SmallSats, CubeSats, etc.) creates the potential for a variety of new science applications involving multiple nodes functioning together or independently to achieve a task, such as swarms and constellations. As this technology develops and is deployed for missions in Low Earth Orbit and beyond, the use of delay tolerant networking (DTN) techniques may improve communication capabilities within the network. In this paper, a network hierarchy is developed from heterogeneous networks of SmallSats, surface vehicles, relay satellites and ground stations which form an integrated network. There is a tradeoff between complexity, flexibility, and scalability of user defined schedules versus autonomous routing as the number of nodes in the network increases. To address these issues, this work proposes a machine learning classifier based on DTN routing metrics. A framework is developed which will allow for the use of several categories of machine learning algorithms (decision tree, random forest and deep learning) to be applied to a dataset of historical network statistics, which allows for the evaluation of algorithm complexity versus performance to be explored. We develop the emulation of a hierarchical network, consisting of tens of nodes which form a cognitive network architecture. CORE (Common Open Research Emulator) is used to emulate the network using bundle protocol and DTN IP neighbor discovery.

Dudukovich, Rachel↗
DOE OSTI2021

An Evaluation of the Spectral Irradiance Data from the NSRDB

A recent effort by the National Renewable Energy Laboratory (NREL) has led to a new radiative transfer model, the Fast All-sky Radiation Model for Solar applications with Narrowband Irradiances on Tilted surfaces (FARMS-NIT), to efficiently compute spectral irradiances in the plane-of-array (POA). This model has been implemented in the National Solar Radiation Data Base (NSRDB) to provide photovoltaic (PV) resource in both narrowband and broadband wavelengths. This study conducts an evaluation of the spectral irradiances from the PV Resource dataset using surface-based observations at NREL’s Solar Radiation Research Laboratory (SRRL) and University of Oregon (UO). The results demonstrate that the PV resource has a generally good agreement with the long-term observations in both clear-sky and cloudy-sky conditions. Further research is needed to reduce the overestimation of visible irradiances in clear sky conditions and underestimation of near-infrared irradiances in cloudy-sky conditions.

14 SOLAR ENERGY↗
DOE OSTI2022

Evaluation of a scientific data search infrastructure

The ability to search over large scientific datasets has become crucial to next-generation scientific discoveries as data generated from scientific facilities grow dramatically. In previous work, we developed and deployed ScienceSearch, a search infrastructure for scientific data which uses machine learning to automate metadata creation. Our current deployment is deployed atop a container based platform at a HPC center. In this article, we present an evaluation and discuss our experiences with the ScienceSearch infrastructure. Specifically, we present a performance evaluation of ScienceSearch's infrastructure focusing on scalability trends. The obtained results show that ScienceSearch is able to serve up to 130 queries/min with latency under 3 s. We discuss our infrastructure setup and evaluation results to provide our experiences and a perspective on opportunities and challenges of our search infrastructure.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2026

Data for Yield from Iowa’s first commercial miscanthus fields: implications of spatial variability for productivity and sustainability beyond research plots

This dataset contains biomass yield measurements and associated vegetation index data collected from commercial Miscanthus × giganteus fields in eastern Iowa during the 2022–2023 growing seasons. The data support the analyses presented in the article: “Yield From Iowa's First Commercial Miscanthus Fields: Implications of Spatial Variability for Productivity and Sustainability Beyond Research Plots.” We collected 105 ground-truth biomass samples from four mature commercial fields (>4 years old) covering 92.81 ha. Samples were taken from 3 m² quadrats that were hand-harvested in alignment with commercial harvest timing. Stem biomass (excluding leaves) was weighed, moisture-corrected, and converted to dry-matter yield expressed in Mg DM ha⁻¹. Sampling locations were selected to capture spatial variability visible in aerial imagery and were recorded using RTK GPS. Each biomass observation was paired with vegetation indices derived from high-resolution PlanetScope satellite imagery (3 m resolution). Images were acquired throughout the growing season, and indices were calculated to evaluate their ability to predict end-of-season biomass yield. Statistical and machine learning approaches were used to identify key predictors, and a linear regression model based on end-of-July Green Normalized Difference Vegetation Index (GNDVI) was developed and evaluated. This repository includes the data used in that modeling workflow. Management practices, economic data, full imagery time series, and additional methodological details are described in the associated publication and are not included here. The dataset consists of three comma-separated value (CSV) files: 1. Combine_Groundtruth_Yield_VI_22_23.csv This file contains ground-truth biomass yield measurements and associated key vegetation index values collected during the 2022 and 2023 growing seasons. Rows: 105 observations Columns: Year — Year of observation (2022 or 2023) Field — Field location identifier Sample_number — Unique sample identifier GNDVI_End_Jul — Green Normalized Difference Vegetation Index calculated at end of July GNDVI_End_Aug — Green Normalized Difference Vegetation Index calculated at end of August NDRE_End_Aug — Normalized Difference Red Edge index calculated at end of August Biomass_Stem_Yield_MgDM/ha — Measured stem biomass yield (megagrams dry matter per hectare) 2. trainData_GNDVI.csv This file contains the subset of observations used to train the predictive relationship between July GNDVI and biomass yield. Rows: 76 observations Columns: Unnamed: 0 — Row index retained from the original data processing workflow GNDVI_End_Jul — GNDVI at end of July Stem_Yield_MgDM/ha — Observed stem biomass yield (Mg DM ha⁻¹) 3. testData_GNDVI.csv This file contains the test dataset used to evaluate model performance. Rows: 29 observations Columns: Unnamed: 0 — Row index retained from the original data processing workflow GNDVI_End_Jul — GNDVI at end of July Predicted_Yield_MgDM/ha — Model-predicted stem biomass yield (Mg DM ha⁻¹) Observed_Yield_MgDM/ha — Measured stem biomass yield (Mg DM ha⁻¹)

Potential yield, yield gap, in-field management, y↗
DOE OSTI2025

Towards philosophical reasoning with agentic LLMs: Socratic method for scientific assistance

As large language models (LLMs) become central tools in science, improving their reasoning capabilities is critical for meaningful and trustworthy applications. We introduce a Socratic agent for scientific reasoning, implemented through a structured system prompt that guides LLMs via classical principles of inquiry. Unlike typical prompt engineering or retrieval-based methods, our approach leverages definition, analogy, hypothesis elimination, and other Socratic techniques to generate more coherent, critical, and domain-aware responses. We evaluate the agent across diverse scientific domains and benchmark it on the abstraction and reasoning corpus challenge dataset, achieving 97.15% under a fixed prompting protocol and without fine-tuning or external tools. Expert evaluation shows improved reasoning depth, clarity, and adaptability over conventional LLM outputs, suggesting that structured prompting rooted in philosophical reasoning can improve the scientific utility of language models.

LLM reasoning↗
NASA NTRS2007

Evaluation of Radiation Belt Space Weather Forecasts for Internal Charging Analyses

A variety of static electron radiation belt models, space weather prediction tools, and energetic electron datasets are used by spacecraft designers and operations support personnel as internal charging code inputs to evaluate electrostatic discharge risks in space systems due to exposure to relativistic electron environments. Evaluating the environment inputs is often accomplished by comparing whether the data set or forecast tool reliability predicts measured electron flux (or fluence over a given period) for some chosen period. While this technique is useful as a model metric, it does not provide the information necessary to evaluate whether short term deviances of the predicted flux is important in the charging evaluations. In this paper, we use a 1-D internal charging model to compute electric fields generated in insulating materials as a function of time when exposed to relativistic electrons in the Earth's magnetosphere. The resulting fields are assumed to represent the "true" electric fields and are compared with electric field values computed from relativistic electron environments derived from a variety of space environment and forecast tools. Deviances in predicted fields compared to the "true" fields which depend on insulator charging time constants will be evaluated as a potential metric for determining the importance of predicted and measured relativistic electron flux deviations over a range of time scales.

Minow, Joseph I.↗
NASA NTRS2019

Approach for Propagating Radiometric Data Uncertainties Through NASA Ocean Color Algorithms

Spectroradiometric satellite observations of the ocean are commonly referred to as “ocean color” remote sensing. NASA has continuously collected, processed, and distributed ocean color datasets since the launch of the Sea-viewing Wide-field-of-view Sensor (SeaWiFS) in 1997. While numerous ocean color algorithms have been developed in the past two decades that derive geophysical data products from sensor-observed radiometry, few papers have clearly demonstrated how to estimate measurement uncertainty in derived data products. As the uptake of ocean color data products continues to grow with the launch of new and advanced sensors, it is critical that pixel-by-pixel data product uncertainties are estimated during routine data processing. Knowledge of uncertainties can be used when studying long-term climate records, or to assist in the development and performance appraisal of bio-optical algorithms. In this method paper we provide a comprehensive overview of how to formulate first-order first-moment (FOFM) calculus for propagating radiometric uncertainties through a selection of bio-optical models. We demonstrate FOFM uncertainty formulations for the following NASA ocean color data products: chlorophyll-a pigment concentration (Chl), the diffuse attenuation coefficient at 490 nm (K(sub d,490)), particulate organic carbon (POC), normalized fluorescent line height (nflh), and inherent optical properties (IOPs). Using a quality-controlled in situ hyperspectral remote sensing reflectance (R(sub rs,i)) dataset, we show how computationally inexpensive, yet algebraically complex, FOFM calculations may be evaluated for correctness using the more computationally expensive Monte Carlo approach. We compare bio-optical product uncertainties derived using our test R(sub rs) dataset assuming spectrally-flat, uncorrelated relative uncertainties of 1, 5, and 10%. We also consider spectrally dependent, uncorrelated relative uncertainties in R(sub rs). The importance of considering spectral covariances in R(sub rs), where practicable, in the FOFM methodology is highlighted with an example SeaWiFS image. We also present a brief case study of two POC algorithms to illustrate how FOFM formulations may be used to construct measurement uncertainty budgets for ecologically-relevant data products. Such knowledge, even if rudimentary, may provide useful information to end-users when selecting data products or when developing their own algorithms.

Bio-optics↗
DOE OSTI2023

Evaluation of Multi-Fidelity Soil Moisture Products Across the Continental United States

We have aggregated the most recent soil moisture datasets from a diverse range of sources, encompassing the Continental United States (CONUS). These sources encompass gridded data from remote sensing products, reanalysis products, machine learning-based projects, and land surface modeling products. Additionally, we have obtained and processed in-situ soil moisture observations from the International Soil Moisture Network. The collected datasets exhibit variations in both temporal and spatial resolutions. Among the 20 datasets, six are available at a spatial resolution of 0.25 degrees, while three are at a coarser spatial resolution of 25 km. To minimize spatial interpolation, we conducted data uncertainty evaluations at the 0.25-degree spatial resolution. For our data evaluations, we maintained a monthly temporal resolution, which effectively captures soil moisture seasonality and interannual variability. Our data processing strategy preserves the raw data and interpolated data at their original temporal resolutions. Datasets with higher temporal resolutions, including daily, three-hourly, and hourly datasets, are set aside for subsequent analyses. These analyses will delve into topics such as soil moisture changes and recovery during extreme weather events. Furthermore, we have processed auxiliary data to enhance our evaluation, leveraging tools such as Google Earth Engine. This includes incorporating topography data, land use land cover data, Köppen-Geiger climate classification, and more to provide a comprehensive assessment from multiple sources.

Li, Lingcheng↗
DOE OSTI2025

Compilation of Experimental Yield Data for Spontaneous Fission of 252 Cf

We present a comprehensive compilation and curation of experimental fission yield (FY) data for the spontaneous fission of 252 Cf, extracted from the EXFOR database. The compilation follows a structured methodology developed for prior compilations of neutron-induced fission yields, and incorporates both independent (IFY) and cumulative (CFY) yields. A total of 62 datasets were reviewed, with entries spanning from 1955 to 2021. A significant portion of the literature reports pre-neutron emission yields, which were excluded from the present compilation due to limitations in format compatibility. Each accepted dataset was processed into a standardized JSON format, including metadata, uncertainties, and bibliographic references. Where available, decay radiation information was used to update the FY data using the latest ENSDF evaluations; 237 data points were corrected accordingly. These corrections are fully traceable and preserve original values. The result is a curated dataset suitable for use in nuclear data evaluations. This work is part of an ongoing effort to modernize the handling of FY data and provide evaluators with high-quality, machine-readable experimental inputs

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2026

Data for Zheng et al. (2025), "AquaMEND: Reconciling multiple impacts of salinization on soil carbon biogeochemistry"

Soil salinization, exacerbated by climate change, poses a global threat to coastal ecosystems and soil function. Salinity affects soil carbon cycling by directly impacting microbial activity and indirectly altering soil physicochemical properties, but current models inadequately represent these complexities. This dataset contains the observational and modeling data from Zheng et al. (2025), which described a process-based modeling framework that couples soil solution chemistry with microbial carbon cycling reactions to study the impacts of soil salinization. This conceptual model is implemented numerically into the open-source geochemical program PHREEQC 3.0 (Parkhurst and Appelo, 2013). This dataset consists of: - Figure2_AquaMEND_salinity_buffer: Contains model simulation outputs to assess the impact of three different cation exchange and surface complexation processes on salinity buffering (Fig. 2 from Zheng et al. 2025). - Figure3_Salinity_function: Contains salinity function fitting for literature data (Fig. 3 from Zheng et al. 2025). - Figure4_AquaMEND_microbial_mechanisms: Contains model simulation outputs for testing various microbial process-based hypotheses related to soil salinization, including microbial mortality, carbon use efficiency (CUE), extracellular enzyme activity, and other microbial mechanisms (Fig. 4 from Zheng et al. 2025). - Figure5_AquaMEND_Redox: Contains on model simulation outputs to evaluate shifts among key redox processes, such as aerobic respiration, sulfate reduction, and methanogenesis (Fig.5 from Zheng et al. 2025). - Figure6_AquaMEND_sorption: Contains on model simulation outputs for investigating the effects of salinity on dissolved organic matter (DOM) sorption and desorption processes (Fig. 6 from Zheng et al. 2025). - Figure7_AquaMEND_process_couple: Contains on model simulation outputs for exploring coupled biotic-abiotic processes and their interactions (Fig. 7 from Zheng et al. 2025). - data: Includes datasets used to develop salinity response functions and evaluate salinity buffering capacity. Datasets for MEND model calibration. - database: Contains the `.dat` file required by PHREEQC for model execution. - README.md: A Markdown plain text file describing the computational tools and directories. Files are a mixture of plain text CSV (comma-separated value) and plain text *.dat files written by the model; no special software is required to read them.

EARTH SCIENCE > AGRICULTURE > SOILS > SOIL SALINIT↗
DOE OSTI2022

Development of Training Materials for Pathologists to Provide Machine Learning Validation Data of Tumor-Infiltrating Lymphocytes in Breast Cancer

The High Throughput Truthing project aims to develop a dataset for validating artificial intelligence and machine learning models (AI/ML) fit for regulatory purposes. The context of this AI/ML validation dataset is the reporting of stromal tumor-infiltrating lymphocytes (sTILs) density evaluations in hematoxylin and eosin-stained invasive breast cancer biopsy specimens. After completing the pilot study, we found notable variability in the sTILs estimates as well as inconsistencies and gaps in the provided training to pathologists. Using the pilot study data and an expert panel, we created custom training materials to improve pathologist annotation quality for the pivotal study. We categorized regions of interest (ROIs) based on their mean sTILs density and selected ROIs with the highest and lowest sTILs variability. In a series of eight one-hour sessions, the expert panel reviewed each ROI and provided verbal density estimates and comments on features that confounded the sTILs evaluation. We aggregated and shaped the comments to identify pitfalls and instructions to improve our training materials. From these selected ROIs, we created a training set and proficiency test set to improve pathologist training with the goal to improve data collection for the pivotal study. We are not exploring AI/ML performance in this paper. Instead, we are creating materials that will train crowd-sourced pathologists to be the reference standard in a pivotal study to create an AI/ML model validation dataset. The issues discussed here are also important for clinicians to understand about the evaluation of sTILs in clinical practice and can provide insight to developers of AI/ML models.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2026

RADAI: A Large-Scale Realistic Dataset for Radiation Detection Algorithm Development

Open, realistic datasets are essential for developing and benchmarking radiation detection algorithms, yet they remain scarce. The Radiological Anomaly Detection and Identification (RADAI) project was develop to create datasets that meet the training and testing needs for sophisticated radiation detection algorithms. The RADAI dataset is a large-scale synthetic resource that integrates high-fidelity Monte Carlo simulations with realistic urban scenarios to capture both background variability and source signatures. RADAI models construction-material NORM, people and vehicles, urban clutter, and dynamic environmental effects such as cosmic-ray and rain-induced transients, and they provide list-mode detector data with motion and response modeling suitable for algorithm training and evaluation. The RADAI project resulted in three publicly-released complementary datasets together with an online scoring portal for standardized performance assessment and an open software toolkit that supports data access, augmentation, model development, and evaluation. These resources enable reproducible comparisons across methods and promote rigorous studies at the scale required by contemporary machine learning. By grounding algorithm development in realistic, well-documented conditions, RADAI supports progress toward more robust detection, identification, and localization in complex urban environments.

Ghawaly, James M. [Division of Computer Science an↗
NASA NTRSDate not supplied

A Comprehensive Forward Model for Spaceborne Radar Instruments

We present the development and validation of a comprehensive forward model designed to enhance remote sensing capabilities of spaceborne radar instruments. To overcome limitations in existing models, we integrated a Discrete Dipole Approximation (DDA) cloud scattering database into our Radiative Transfer Model (RTM), focusing on microwave frequencies. By simulating the optical properties of non-spherical frozen hydrometeors, the DDA technique effectively reduced discrepancies between simulated and observed values, surpassing traditional Mie tables. The evaluation of DDA lookup tables involved comparisons with a collocated dataset comprising short-term forecasts and satellite microwave data, providing evidence of their superiority. Additionally, we address the challenges of assimilating active radar measurements, which offer vertically resolved insights into clouds and precipitation. We explored the assimilation of spaceborne radar measurements in Numerical Weather Prediction (NWP) models by integrating a forward radar model, along with its adjoint and tangent linear, into the data assimilation system. Evaluation using CloudSat measurements demonstrated promising agreement between simulations and observations, particularly when the input hydrometeor profiles aligned with the measured reflectivity profiles, showcasing the potential of the developed forward radar model. Moreover, we discuss other challenges in radar measurement assimilation within NWP models, including potential observation errors and biases.

Isaac Moradi↗
NASA NTRSDate not supplied

An Intercomparison of High Spectral Resolution Lidar and Satellite Ocean Color Backscatter Retrievals

We present an intercomparison of satellite ocean color and high spectral resolution lidar (HSRL) retrievals of upper ocean particulate backscattering (bbp) and diffuse attenuation coefficients (Kd). Ocean retrievals of bbp and Kd have been performed over a wide variety of optical and ecological domains using the NASA Langley Research Center HSRL-1 (532 nm) and the recently upgraded HSRL-2 (532 & 355 nm). These datasets provide a unique opportunity to perform a critical assessment of the HSRL technique and to explore the utility of HSRL measurements for evaluating the quality of ocean color remote sensing retrievals on regional scales. A matchup dataset of co-located and high-quality HSRL and ocean color remote sensing measurements was created using data from research flights conducted over the western North Atlantic Ocean during the NASA sponsored SABOR, NAAMES, and ACTIVATE campaigns. Overall, comparisons showed a strong agreement between HSRL and ocean color retrievals of bbp and Kd, providing confidence in our ability to retrieve upper ocean optical properties using oceanographic lidar. However, comparisons from individual flights can exhibit deviations that were not attributed to shifts in water column optical domains. In one of these cases, the simultaneous HSRL atmosphere-ocean retrievals from two consecutive days and within the same region were used to identify ocean color atmospheric correction errors resulting from the presence of absorbing aerosols. These results highlight the utility of airborne HSRL as a standalone ocean observing technology and as an independent and calibrated technique for assessing the quality of ocean color remote sensing retrievals.

Brian Collister↗
DOE OSTI2024

Unpaired image translation to mitigate domain shift in liquid argon time projection chamber detector responses

Deep learning algorithms often are developed and trained on a training dataset and deployed on test datasets. Any systematic difference between the training and a test dataset may severely degrade the final algorithm performance on the test dataset—what is known as the domain shift problem . This issue is prevalent in many scientific domains where algorithms are trained on simulated data but applied to real-world datasets. Typically, the domain shift problem is solved through various domain adaptation (DA) methods. However, these methods are often tailored for a specific downstream task, such as classification or semantic segmentation, and may not easily generalize to different tasks. This work explores the feasibility of using an alternative way to solve the domain shift problem that is not specific to any downstream algorithm. The proposed approach relies on modern Unpaired Image-to-Image (UI2I) translation techniques, designed to find translations between different image domains in a fully unsupervised fashion. In this study, the approach is applied to a domain shift problem commonly encountered in Liquid Argon Time Projection Chamber (LArTPC) detector research when seeking a way to translate samples between two differently distributed LArTPC detector datasets deterministically. This translation allows for mapping real-world data into the simulated data domain where the downstream algorithms can be run with much less domain-shift-related performance degradation. Conversely, using the translation from the simulated data to a real-world domain can increase the realism of the simulated dataset and reduce the magnitude of any systematic uncertainties. To evaluate the quality of the translations, we use both pixel-wise metrics and a downstream task to measure the effectiveness of UI2I methods for mitigating the domain shift problem. We adapted several popular UI2I translation algorithms to work on scientific data and demonstrated the viability of these techniques for solving the domain shift problem with LArTPC detector data. To facilitate further development of DA techniques for scientific datasets, the ‘Simple Liquid-Argon Track Samples’ dataset used in this study is also published.

97 MATHEMATICS AND COMPUTING↗