Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluation datasets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Evaluation of CMIP6 models in simulating the statistics of extreme precipitation over Eastern Africa

We report the Eastern Africa region experiences frequent extreme precipitation events that can cause destruction of property and environment, and loss of lives. Thus, there is a need to understand how these events may change in the future and how well the global climate models that are used to make projections can simulate precipitation extremes in this region before they can be used in downscaling or flood and drought impact assessment studies. In this work, we evaluated the ability of sixteen Coupled Model Intercomparison Project Phase 6 (CMIP6) models to simulate present-day precipitation extremes over the Eastern Africa region during the two rainy seasons (March–May and September–November). We used nine extreme precipitation indices (including seven (one) indices of wet (dry) extremes) defined by the Expert Team on Climate Change Detection and Indices. The CMIP6 models were evaluated against two gridded observation datasets: Global Precipitation Climatology Project One-Degree Daily Dataset and Tropical Rainfall Measuring Mission Multi-satellite Precipitation Analysis 3B42. Three model performance metrics (percentage bias, normalized root-mean-square error, and pattern correlation coefficient) were employed to further assess the strengths and weakness of the models. Our results show that the multi-model ensemble mean generally provides a better representation of observed precipitation and related extremes compared to individual models when considering all metrics and seasons. Several consistent biases are evident across CMIP6 models, which tend to overestimate the total-wet day precipitation and consecutive wet days, and underestimate very wet days and maximum 5-day precipitation in both seasons. Furthermore, no single model consistently performs best, model performance varies with the season and index under consideration and is generally independent of horizontal resolution.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

Generative large language models for predictive maintenance planning

Maintenance planning and the generation of necessary components for tasks can prove time-consuming and complex. Automating the creation of recurring or similar tasks by leveraging previous planning packages and data, while uncovering insights to automate planning package generation, presents an opportunity to conserve valuable time and resources. This work aims to harness the textual and probabilistic capabilities of large language models (LLMs) to automate the generation of planning packages. Utilizing diverse data sources ranging from raw data to handwritten text, both singular and collaborative LLMs are trained and tested. Results demonstrate their capability to generate essential planning package components, effectively replicating the statistical patterns in the data. This demonstrates the use of these tools inside a digital asset for automated planning. This work outlines a methodology for constructing datasets, a training suite, and evaluation methods for LLM-based textual and conversational planning tools utilized in an asset digital twin. Results indicate that the fine-tuned models generate estimated planning information within the statistical ranges observed in real maintenance data. The models achieve high accuracy (>90%) in document question-answering and instruction generation tasks. Furthermore, the conversational retrieval-augmented generation (RAG) assistant system achieves 100% document retrieval accuracy, while conversational information capture exceeds 98% across the majority of work-package assistant modules.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Quality control and crop characterization framework for multi-temporal UAV LiDAR data over mechanized agricultural fields

Recent developments in remote sensing are enabling automatic, high resolution, and non-destructive survey of agriculture fields, providing the key basis for advancing plant breeding. Among the used remote sensing modalities, LiDAR has attracted wide attention for its ability to directly provide accurate 3D information. Despite the increasing utilization of LiDAR technology in phenotyping, there is still a lack of effective quality control strategies, in particular, quality control of LiDAR data collected on a multi-temporal basis. This study proposes a targetless framework for multi-temporal LiDAR data quality control and crop characterization in mechanized agricultural fields. Features extracted from the fields – terrain patches and row/alley locations – are utilized for evaluating the vertical and planimetric relative accuracy of the point clouds. Row/alley locations in the field are automatically identified from the point clouds based on the assumption that higher point density and/or higher elevation correspond to plant locations. The performance of the proposed quality control strategies is evaluated using multi-temporal datasets collected in agricultural fields of different sizes, orientation, crops, and growth stages. The result shows that the net vertical and planimetric discrepancies between multi-temporal point clouds are ±3 cm and ±8 cm, respectively. While the former reflects the actual accuracy of the point clouds, the latter is a combined effect of the LiDAR point cloud accuracy, rasterization artifacts, crop type, growth pattern, and wind condition during data acquisition. In terms of row and alley detection, the result shows that the proposed strategy achieves high performance and can deal with different planting orientation, crop types, growth stages, canopy cover, and planting density. In conclusion, this study presents a quality control framework for multi-temporal LiDAR data. Finally, the row and alley detection leads to automated extraction of plots, and hence facilitates the use of remotely sensed data for automated phenotyping.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Converting tabular data into images for deep learning with convolutional neural networks

Abstract Convolutional neural networks (CNNs) have been successfully used in many applications where important information about data is embedded in the order of features, such as speech and imaging. However, most tabular data do not assume a spatial relationship between features, and thus are unsuitable for modeling using CNNs. To meet this challenge, we develop a novel algorithm, image generator for tabular data (IGTD), to transform tabular data into images by assigning features to pixel positions so that similar features are close to each other in the image. The algorithm searches for an optimized assignment by minimizing the difference between the ranking of distances between features and the ranking of distances between their assigned pixels in the image. We apply IGTD to transform gene expression profiles of cancer cell lines (CCLs) and molecular descriptors of drugs into their respective image representations. Compared with existing transformation methods, IGTD generates compact image representations with better preservation of feature neighborhood structure. Evaluated on benchmark drug screening datasets, CNNs trained on IGTD image representations of CCLs and drugs exhibit a better performance of predicting anti-cancer drug response than both CNNs trained on alternative image representations and prediction models trained on the original tabular data.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Graph-Based Prediction of Spatio-Temporal Vaccine Hesitancy From Insurance Claims Data

Growing vaccine hesitancy is contributing to the decline in immunization rates for highly contagious, vaccine-preventable childhood diseases. Therefore, there has been a significant interest in understanding how hesitancy is spreading at higher spatio-temporal resolutions, enabling more targeted interventions. Motivated by this, we study the problem of prediction of vaccine hesitancy at the ZIP Code level, referred to as the VaxHesitancy problem. A significant challenge for this problem is the lack of high-resolution data that indicates hesitancy. Here, we develop a hybrid VaxHesSTL framework that combines a Graph Neural Network (GNN) and a Recurrent Neural Network (RNN) to address the VaxHesitancy problem. The GNN uses a ZIP Code-level network to capture spatial signals from neighboring areas, while the RNN models the temporal dynamics present in the data. We train and evaluate VaxHesSTL using a large dataset, namely the All-Payer Claims Databases (APCD), for Virginia, consisting of insurance claims from over five million individuals for six years. We find that an aggregated contact network or graph, developed from a detailed activity-based population network, plays an important role in the performance of VaxHesSTL, compared to graph models based solely on spatial proximity. Experiments demonstrate that VaxHesSTL outperforms a range of state-of-the-art baselines, which rely solely on historical time series data without accounting for spatial relationships. Since hesitancy data at higher spatial resolution is often unavailable or hard to get, we incorporate an active learning approach with our VaxHesSTL framework to optimize the training set without compromising the prediction performance. We find that hesitancy data for only 18% of ZIP Codes selected by active learning allows us to forecast hesitancy for all the ZIP Codes in the Virginia.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2026

DP-TwoLevel: two-stage gradient subspace learning for differentially private federated learning

Federated learning (FL) enables collaborative model training across distributed data sources without sharing raw data, but faces fundamental challenges in communication efficiency and privacy. Differentially private (DP) training mitigates information leakage but introduces noise that degrades model performance, especially in high-dimensional settings. We propose DP-TwoLevel, a hierarchical gradient projection method that improves utility under fixed DP constraints by exploiting low-dimensional structure in model updates. Our approach learns a two-level PCA-based representation of gradients and applies DP noise in a reduced-dimensional subspace, thereby lowering the effective noise magnitude while preserving dominant signal components. We evaluate the method across three datasets (MNIST, Fashion-MNIST, CIFAR-10) and three privacy regimes (ϵ∈0.5, 1.0, 2.0). Across nine experimental settings, DP-TwoLevel consistently outperforms DP-FedAvg, achieving an average accuracy improvement of 9.44%, with larger gains observed in lower ϵ(higher-noise) regimes (up to +22.31%). We further analyze scalability across models ranging from 100K to 1.49M parameters and identify a variance-based success criterion: performance remains strong when the projection preserves more than 75% of gradient variance, degrades in a marginal regime (65–75%), and fails below this threshold. Our results demonstrate that structure-aware dimensionality reduction can significantly improve the privacy–utility tradeoff in FL without modifying formal privacy guarantees. We also provide empirical evidence of scaling limitations for global projections and motivate per-layer extensions for larger models.

Kotevska, Olivera [ORNL] (ORCID:0000000316772243)↗
DOE OSTI2021

Graphene U-Net v1

Graphene U-Net is a library that provides a simplified platform for training deep neural networks for the task of microscopy image segmentation. It contains functions and classes that make the process of training a neural network simple such that non-ML experts can train and evaluate models on their own datasets. It uses Pytorch as a backend and can run on both CUDA-enable GPUs and CPUs. It contains the main library file Microscopy_Unet.py as well as unet.py which contains the UNET model used for this software. This can be replaced with any other fully convolutional deep learning architecture with relative ease.

Sadre, Robbie↗
DOE OSTI2021

Explainable Neural Architecture Search (XNAS)

Code for the paper Learning Interpretable Models Through Multi-Objective Neural Architecture Search by Zachariah Carmichael, Tim Moon, and Sam Ade Jacobs. Monumental advances in deep learning have led to unprecedented achievements across a multitude of domains. While the performance of deep neural networks is indubitable, the architectural design and interpretability of such models are nontrivial. Research has been introduced to automate the design of neural network architectures through neural architecture search (NAS). Recent progress has made these methods more pragmatic by exploiting distributed computation and novel optimization algorithms. However, there is little work in optimizing architectures for interpretability. To this end, we propose a multiobjective distributed NAS framework that optimizes for both task performance and introspection. We leverage the non-dominated sorting genetic algorithm (NSGA-II) and explainable AI (XAI) techniques to reward architectures that can be better comprehended by humans. The framework is evaluated on several image classification datasets. We demonstrate that jointly optimizing for introspection ability and task error leads to more disentangled architectures that perform within tolerable error.

Carmichael, ZachariahJ↗
DOE OSTI2023

CLIP Representation Enhanced Predicate Estimation

This software provides functions to perform visual predicate estimation by leveraging the pre-trained vision-language model CLIP. In addition to training the prediction estimator with a labeled dataset, it also performs performance evaluation and visualization of results.

Jayaraman Thiagarajan, Jayaraman↗
DOE OSTI2023

Spectral Peak Enhancement by Combining Trusted Response Elements via Machine Learning (SPECTRE-ML) v0.8.0

SPECTRE-ML (Spectral Peak Enhancement by Combining Trusted Response Elements via Machine Learning) is a machine learning based program for finding optimal clusters of radiation detector segments (i.e., pixels or voxels) in order to improve spectral performance. It provides facilities for pre-processing and analyzing training datasets, running ML algorithms, and evaluating and visualizing outputs. SPECTRE-ML outperforms simpler ad-hoc segmentation methods such as uniform depth clusters, learning detector performance trends such as dead layers, edge effects, and gain shifts. Although extensible to arbitrary highly-segmented spectroscopic radiation detectors, SPECTRE-ML currently focuses on improving spectral performance in highly-segmented CdZnTe (CZT) detectors for International Atomic Energy Agency (IAEA) non-destructive assay (NDA) safeguards tasks.

Vavrek, Jayson↗
DOE OSTI2025

Spatio-temporal Fourier Transformer for Long-term Dynamics Prediction (StFT) v1.0

We propose a novel machine learning model spatio-temporal Fourier transformer (StFT) to emulate long-term dynamics of multi-scale and multi-physics systems. Our method StFT overcomes the limitations of rapid error accumulation, particularly in long-term forecasting of systems characterized by complex and coupled dynamics. StFT achieves outstanding accuracy and computational efficiency by effectively capturing multi-scale interactions, and quantify the uncertainties inherent in the predictions. Our model leverages a structured hierarchy of StFT blocks, and explicitly captures dynamics across both macro- and micro- spatial scales. Evaluations conducted on three benchmark datasets (plasma, fluid, and atmospheric dynamics) demonstrate the advantages of our approach over state-of-the-art ML methods.

Bai, Zhe [Lawrence Berkeley National Laboratory (L↗
DOE OSTI2026

PRIME: An evaluation framework for protein representation inference and generalization in viral mutation space

Background Protein language models (PLMs) have revolutionized protein fitness prediction, yet their application to rapidly evolving viral pathogens is often confounded by extreme sequence homology. This homology leads to “data leakage” in standard random validation splits, yielding inflated performance metrics that fail to translate into real-world biosurveillance utility. Results We present Protein Representation Inference for Mutation Evaluation (PRIME), a framework that integrates domain-specific fine-tuning with a rigorous position-stratified validation protocol to evaluate viral threats. Using a dataset of 347,432 SARS-CoV-2 receptor binding domain (RBD) sequences, we demonstrate that while random training data split yields deceptive R 2 values (> 0.90), they fail to generalize to novel mutational sites. By benchmarking models up to 650 M parameters, we show that domain-specific fine-tuning of the ESM-C 600 M model with correctly stratified data provides an initial demonstration of predictive signal for binding affinity and expression at unseen mutational sites of binding affinity and expression on unseen sites (R 2 ~0.23), a significant advancement over base foundation models which exhibit no predictive power (R 2 <0). PRIME’s embedding-based clustering identified 3.03% of bat coronavirus sequences as candidates for further experimental prioritization based on their functional similarity to human-infective strains in embedding space, offering a perspective complementary to traditional phylogenetic methods. Conclusion PRIME establishes a new benchmark for the application of PLMs in pathogen surveillance. Our findings demonstrate that state-of-the-art models and fine-tuning, when paired with stratified validation, provide biologically meaningful insights into pathogen evolution and zoonotic risk.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

XCal: model-based approach to X-ray CT spectral calibration

Transmission X-ray computed tomography (CT) is widely used to quantitatively reconstruct 3D objects composed of multiple materials. However, accurate CT reconstruction requires the system to be calibrated to account for the effective X-ray spectrum. Unfortunately, measurement of the effective spectrum is ill-posed, and existing calibration methods require that the system be recalibrated when the system parameters are changed. In this paper, we propose XCal, a multi-energy model-based spectral calibration approach for X-ray CT. The XCal approach models the effective spectrum using a separable physics-based model of the CT system. The model parameters are then estimated by fitting calibration data with known objects at multiple energies. An important advantage of XCal is that it allows the user to change scanner settings, such as the source voltage or X-ray filters, without the need for recalibration. Evaluations on simulated and measured datasets demonstrate that XCal significantly improves the accuracy of the estimated spectrum as compared to existing calibration methods.

Li, Wenrui [Purdue Univ., West Lafayette, IN (Unit↗
DOE OSTI2022

Patterns and controls on island-wide aboveground biomass accumulation in second-growth forests of Puerto Rico

This dataset includes two products from Martinuzzi et al. (2022): "biomass.tif" is a 26-m resolution forest biomass (AGB) map for Puerto Rico derived from NASA G-LiHT lidar data and forest inventory data (FIA plots), in raster format. "input_multivariate_v2.shp" is a point shapefile with information on forest age, substrate, past land use, topographic wetness, slope, and precipitation, for each forest pixel. These two datasets can be used to evaluate spatial patterns of AGB in second-growth forests across transects of lidar data in humid forests of Puerto Rico, and to analyze relationship(s) between AGB and environmental variables. Additional information on these products can be found on the supporting file called "Readme.txt" included within the data archive, as well as in the original manuscript by Martinuzzi et al (2022).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Development of a Comprehensive Two-Phase Flow Database for the Validation of NEK-2P

Three-dimensional (3-D) two-phase Computational Fluid Dynamics (CFD) codes are emerging as a powerful and potentially practical tool for applications in which detailed local flow information is needed. However, two-phase flow models and associated closure relations are not well established for CFD applications, which is partly due to the lack of high-quality validation data. The main objective of this work is to develop a comprehensive database of two-phase flows that can be used to validate two-phase CFD codes such as NEK-2P. In this project, four advanced local measurement systems, including Particle Image Velocimetry and Planar Laser-Induced Fluorescence (PIV-PLIF), high-speed imaging, x-ray densitometry, and multi-sensor conductivity probe are employed to measure the local two-phase flow parameters of both gas and liquid phases. By combining these techniques, the local void fraction, bubble velocity, interfacial area concentration, bubble frequency, liquid velocity, turbulence intensity, etc., in various two-phase flow regimes can be obtained. These local measurement techniques are first used in a 25.4 mm circular pipe test section. Seven air-water two-phase flow conditions spanning the bubbly, slug, churn-turbulent, and annular flow regimes are measured in this facility. The obtained database contains the radial profiles of both gas- and liquid-phase parameters at three axial locations along the test section. The second facility used in this work contains a 30 mm × 10 mm rectangular test section, in which three two-phase flow and two single-phase flow conditions are measured. Two-dimensional distributions of local two-phase flow parameters in the cross-sectional plane are measured in this facility at three axial locations as well. A facility featuring a 3×3 electrically heated rod bundle is also designed and being constructed in this project. This facility is specially designed for optical measurements and is expected to provide high-quality boiling data in the future. Preliminary analyses have been performed for the data taken in the 25.4 mm circular pipe. Both center-peaked and wall-peaked void fraction profiles have been observed in the data depending on the two-phase flow conditions and/or developing lengths. The 1-D drift-flux model was evaluated with the newly obtained datasets, in which both gas- and liquid-phases data were directly measured. The distribution parameter model has been optimized based on a new void-profile classification method proposed in this study. The optimized drift-flux model shows a significant improvement in predicting both gas velocity and void fraction. The measured liquid-phase turbulence was used to benchmark Sato’s turbulence model considering the bubble-induced shear stress for the three tested bubbly flows. The benchmark results showed good agreement between the PIV measurements and model predictions. In the bubbly flows tested that have low void fractions less than 3%, the effect of the bubble-induced turbulence was found not significant. However, the bubble-induced shear stress becomes important with the increase of the void fraction. The Conjugate Heat Transfer (CHT) model was developed and implemented in NEK-2P. This model allows the coupled simulation of the solid domain and two-phase fluid domain, allowing the specification of realistic boundary conditions. The CHT implementation was verified first with non-boiling simulations. The predicted temperatures in the fluid domain were shown to be identical in simulations with or without the CHT model. The CHT model was then validated through simulations of three Becker benchmark CHF tests performed under both Dryout (DO) and Departure from Nucleate Boiling (DNB) conditions. Reasonably good agreement was obtained between calculated wall temperatures and corresponding experimental data.

42 ENGINEERING↗
NASA NTRS2005

Standardization Process for Space Radiation Models Used for Space System Design

The space system design community has three concerns related to models of the radiation belts and plasma: 1) AP-8 and AE-8 models are not adequate for modern applications; 2) Data that have become available since the creation of AP-8 and AE-8 are not being fully exploited for modeling purposes; 3) When new models are produced, there is no authorizing organization identified to evaluate the models or their datasets for accuracy and robustness. This viewgraph presentation provided an overview of the roadmap adopted by the Working Group Meeting on New Standard Radiation Belt and Space Plasma Models.

Barth, Janet↗
NASA NTRS2007

Assessment of Turbulent Shock-Boundary Layer Interaction Computations Using the OVERFLOW Code

The performance of two popular turbulence models, the Spalart-Allmaras model and Menter s SST model, and one relatively new model, Olsen & Coakley s Lag model, are evaluated using the OVERFLOWcode. Turbulent shock-boundary layer interaction predictions are evaluated with three different experimental datasets: a series of 2D compression ramps at Mach 2.87, a series of 2D compression ramps at Mach 2.94, and an axisymmetric coneflare at Mach 11. The experimental datasets include flows with no separation, moderate separation, and significant separation, and use several different experimental measurement techniques (including laser doppler velocimetry (LDV), pitot-probe measurement, inclined hot-wire probe measurement, preston tube skin friction measurement, and surface pressure measurement). Additionally, the OVERFLOW solutions are compared to the solutions of a second CFD code, DPLR. The predictions for weak shock-boundary layer interactions are in reasonable agreement with the experimental data. For strong shock-boundary layer interactions, all of the turbulence models overpredict the separation size and fail to predict the correct skin friction recovery distribution. In most cases, surface pressure predictions show too much upstream influence, however including the tunnel side-wall boundary layers in the computation improves the separation predictions.

Oliver, A. B.↗
NASA NTRS2017

Satellite Monitoring of Cyanobacterial Harmful Algal Bloom Frequency in Recreational Waters and Drinking Water Sources

Cyanobacterial harmful algal blooms (cyanoHAB) cause extensive problems in lakes worldwide, including human and ecological health risks, anoxia and sh kills, and taste and odor problems. CyanoHABs are a particular concern in both recreational waters and drinking water sources because of their dense biomass and the risk of exposure to toxins. Successful cyanoHAB assessment using satellites may provide an indicator for human and ecological health protection. In this study, methods were developed to assess the utility of satellite technology for detecting cyanoHAB frequency of occurrence at locations of potential management interest. The European Space Agency's MEdium Resolution Imaging Spectrometer (MERIS) was evaluated to prepare for the equivalent series of Sentinel-3 Ocean and Land Colour Imagers (OLCI) launched in 2016 as part of the Copernicus program. Based on the 2012 National Lakes Assessment site evaluation guidelines and National Hydrography Dataset, the continental United States contains 275,897 lakes and reservoirs greater than 1 ha in area. Results from this study show that 5.6% of waterbodies were resolvable by satellites with 300 m single-pixel resolution and 0.7% of waterbodies were resolvable when a three by three pixel (3 x 3-pixel) array was applied based on minimum Euclidian distance from shore. Satellite data were spatially joined to U.S. public water surface intake (PWSI) locations, where single-pixel resolution resolved 57% of the PWSI locations and a 3 x 3-pixel array resolved 33% of the PWSI locations. Recreational and drinking water sources in Florida and Ohio were ranked from 2008 through 2011 by cyanoHAB frequency above the World Health Organizations (WHO) high threshold for risk of 100,000 cells m/L. The ranking identified waterbodies with values above the WHO high threshold, where Lake Apopka, FL (99.1%) and Grand Lake St. Marys, OH (83%) had the highest observed bloom frequencies per region. The method presented here may indicate locations with high exposure to cyanoHABs and therefore can be used to assist in prioritizing management resources and actions for recreational and drinking water sources.

Cyanobacteria↗