Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluation datasets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 631 records · Page 35
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Evaluating multi-slice ptychography tomography for X-ray imaging

X-ray ptychography typically relies on single-slice reconstruction approaches that assume minimal beam propagation through the specimen. However, this approximation breaks down for thick samples or when using high numerical aperture optics, leading to reconstruction artifacts due to multiple scattering. Multi-slice ptychographic tomography (MSPT) addresses this limitation by explicitly modeling beam propagation through the specimen while reducing angular sampling requirements for tomography, but its performance depends critically on reconstruction parameters and experimental conditions. Here, we systematically evaluate MSPT using numerical simulations and experimental X-ray datasets to understand how factors such as object thickness, optical parameters, and initialization strategies affect reconstruction accuracy. We demonstrate that informed object initialization significantly improves depth resolution and enables accurate projection extension, reducing the angular sampling requirements for tomographic reconstruction. Experimental demonstration confirms that MSPT produces reconstructions with reduced artifacts, establishing it as a robust approach for nanoscale imaging at modern synchrotron facilities.

Luktuke, Amey [Argonne National Laboratory (ANL), ↗
DOE OSTI2024

Understanding spatial and temporal drivers of variation in tree hydraulic processes and their consequences for climate feedbacks (Final Technical Report)

This is the final technical report from the first phase of a project that changed institutions. The grant was titled “Understanding spatial and temporal drivers of variation in tree hydraulic processes and their consequences for climate feedbacks.” The overall objectives of this project were to (1) provide model‐compatible datasets of key plant hydraulic traits and status for model evaluation, parameterization and validation and (2) use these data to pinpoint ecosystem responses to a changing hydroclimate by addressing both long‐term climatic drying and episodic extreme droughts. We planned to address the objectives with three research activities to quantify plant responses to chronic water stress and episodic drought: (1) generate high frequency observations of soil and plant hydraulic data across different landscape positions at multiple sites, (2) quantify plant hydraulic trait plasticity in response to experimental soil moisture reduction in situ in two central hardwood forests, and (3) simulate the carbon consequences of incorporating plant hydrodynamics and plant acclimation to water stress in the DOE‐sponsored plant hydrodynamics model FATES‐HYDRO. As of the transfer of this project to another institution, we had made substantial progress on activities 1 and 2, and started activity 3.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

TrojAI Alternate Analysis

In this portion of the TrojAI evaluation, we focus on the cyber-network-c2-mar2024 dataset. Recall that in this round ResNet18 and ResNet34 neural networks (NN) were trained on the USTC-TFC2016 dataset with the aim of distinguishing between benign versus botnet command and control (c2) packets. A range of bytes from each packet was reformatted into a 28x28 pixel image, and the collection of reformatted packets served as the training (and testing) data for the two ResNet models. For some of the data a trigger watermark was strategically placed to affect various inputs to the NNs. This watermarked, or poisoned, data in turn created a poisoned, or trojaned NN. The data were poisoned in different ways ultimately creating different trojaned NNs. This collection of trojaned NNs was combined with various versions of not trojaned NNs and served as the training and testing data for the performers. The performers’ task was to construct a classifier to distinguish between the trojaned and not trojaned models. It was previously noted that the performers struggled with the cyber-network-c2-mar2024 dataset, motivating this investigation of potential reasons the performers experienced challenges.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Model Card for WaveDenoiser

This study used STEAD to train and evaluate this model because STEAD is among the best benchmark datasets available for local to regional data. STEAD is a global dataset with over 1 million 60 s long seismic waveforms that originated from approximately 450,000 earthquakes and background noise captured by more than 2,500 seismic stations. Each waveform in STEAD was attached with metadata such as earthquake locations, station locations, and signal arrival times when available

58 GEOSCIENCES↗
DOE OSTI2024

Efficient Distributed Sequence Parallelism for Transformer-Based Image Segmentation

We introduce an efficient distributed sequence parallel approach for training transformer-based deep learning image segmentation models. The neural network models are comprised of a combination of a Vision Transformer encoder with a convolutional decoder to provide image segmentation mappings. The utility of the distributed sequence parallel approach is especially useful in cases where the tokenized embedding representation of image data are too large to fit into standard computing hardware memory. To demonstrate the performance and characteristics of our models trained in sequence parallel fashion compared to standard models, we evaluate our approach using a 3D MRI brain tumor segmentation dataset. We show that training with a sequence parallel approach can match standard sequential model training in terms of convergence. Furthermore, we show that our sequence parallel approach has the capability to support training of models that would not be possible on standard computing resources.

Lyngaas, Isaac↗
DOE OSTI2023

Data Validation Experiments with a Computer-Generated Imagery Dataset for International Nuclear Safeguards

Computer vision models have great potential as tools for international nuclear safeguards verification activities, but off-the-shelf models require fine-tuning through transfer learning to detect relevant objects. Because open-source examples of safeguards-relevant objects are rare, and to evaluate the potential of synthetic training data for computer vision, we present the Limbo dataset. Limbo includes both real and computer-generated images of uranium hexafluoride containers for training computer vision models. Here, we generated these images iteratively based on results from data validation experiments that are detailed here. The findings from these experiments are applicable both for the safeguards community and the broader community of computer vision research using synthetic data.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2022

Evaluation of obstacle modelling approaches for resource assessment and small wind turbine siting: case study in the northern Netherlands

Abstract. Growth in adoption of distributed wind turbines for energy generation is significantly impacted by challenges associated with siting and accurate estimation of the wind resource. Small turbines, at hub heights of 40 m or less, are greatly impacted by terrestrial obstacles such as built structures and vegetation that can cause complex wake effects. While some progress in high-fidelity complex fluid dynamics (CFD) models has increased the potential accuracy for modelling the impacts of obstacles on turbulent wind flow, these models are too computationally expensive for practical siting and resource assessment applications. To understand the efficacy of available models in situ, this study evaluates classic and commonly used methods alongside new state-of-the-art lower-order models derived from CFD simulations and machine learning approaches. This evaluation is conducted using a subset of an extensive original dataset of measurements from more than 300 operational wind turbines in the northern Netherlands. The results show that data-driven methods (e.g. machine learning and statistical modelling) are most effective at predicting production at real sites with an average error in annual energy production of 2.5 %. When sufficient data may not be available de novo to support these data-driven approaches, models derived from high-fidelity simulations show promise and reliably outperform classic methods. On average these models have 6.3 %–11.5 % error compared with 26 % for classic methods and 27 % baseline error for reanalysis data without obstacle correction. While more performant on average, these methods are also sensitive to the quality of obstacle descriptions and reanalysis inputs.

17 WIND ENERGY↗
NASA NTRS2018

The Importance of User Feedback in Sustaining Trusted Repositories

The NASA Earth Observation System Data and Information System (EOSDIS) has been operating since 1994 and is serving a global user community with well-managed Earth science data in a variety of scientific disciplines. EOSDIS processes, archives and distributes data and information products resulting from spaceborne and airborne instruments as well as in situ measurements from field campaigns. The Earth Science Data and Information System (ESDIS) Project at the NASA Goddard Space Flight Center manages EOSDIS with its 12 Distributed Active Archive Centers (DAACs) located across the United States. During the entire life of EOSDIS, the ESDIS Project and the DAACs have deployed many different mechanisms for user feedback, which have proven extremely valuable to their evolution and performance in their service to user communities. Some of the inputs from our user groups have resulted in fundamental changes in the architecture, design and operations of EOSDIS, while others have provided novel ideas for incremental changes. The EOSDIS DAACs have User Working Groups (UWGs) that represent broad user communities in the Earth science disciplines served by the DAACs. The UWGs meet periodically to assess and provide feedback on dataset and service priorities. As regular users of the data and services of the DAAC and experts in the scientific disciplines, the UWG members provide valuable inputs for planning and prioritizing the services, as well as addition of new datasets, for the benefit of the community. The EOSDIS is evaluated annually through an independently administered survey of its users resulting in the American Customer Satisfaction Index (ACSI). The survey provides an ACSI score as well as free-text suggestions from users, which are also helpful in making specific system improvements. In addition, each of the DAACs has a user services group that address on-going requests for help and other comments from users. The ESDIS Project has established a mechanism through the "earthdata" website (http://earthdata.nasa.gov) for users to provide feedback of any kind and these questions/comments are routed to the appropriate individuals in the Project or the DAACs. These various forms of receiving user feedback and responding to them continue to be extremely valuable in evolving and sustaining our Earth Science repository.

Behnke, Jeanne↗
NASA NTRS2021

Semantic Segmentation of High-Resolution Satellite Imagery using Generative Adversarial Networks with Progressive Growing

With increase in urbanization and Earth Sciences research into urban areas, the need to quickly and accurately segment urban rooftop maps has never been greater. Cur-rent machine learning techniques struggle to produce high accuracy maps in dense urban zones where there is high image noise and foot print overlap. In this paper, we evaluate a training methodology for pixel-wise segmentation for high resolution satellite imagery using progressive growing of generative adversarial networks as a solution. We apply our model to segmenting building rooftops and compare these results to conventional methods for rooftop segmentation. We evaluate our approach using the SpaceNet version 2 and xView datasets. Our experiments show that for SpaceNet, progressive Generative Adversarial Network (GAN) training achieved a test accuracy of 93% compared to 89% for traditional GAN training and 87% for U-Net architecture, while for xView, we achieved 71% accuracy using progressive GAN training compared to 69% through traditional GAN training and 65% using U-Net.

Semantic↗
NASA NTRS2022

Re-Evaluation of Low Cloud Amount Relationships With Lower-Tropospheric Stability and Estimated Inversion Strength

Lower-tropospheric stability (LTS) and estimated inversion strength (EIS) have a widely accepted relationship with low cloud amount and are key observational foundations for understanding and modeling low-level stratiform clouds. Using the updated surface-based and satellite cloud data, we find that low cloud amount is not as strongly correlated with LTS, and not as sensitive to LTS, as established in the past. EIS does not provide a stronger correlation with low cloud amount than LTS over all eight regions (including the midlatitudes). Further analyzing the relationships between LTS and EIS with different types of low clouds, we find that there is a strong correlation of LTS and EIS with stratocumulus only. This explains the weaker correlation of low cloud fraction (including cumulus, stratocumulus, and stratus) to both LTS and EIS. These results also suggest the need to re-evaluate these relationships in Earth system models. Plain Language Summary: Low-level clouds play an important role in the global radiation budget. Low cloud amount has two established linear relationships with lower-tropospheric stability (LTS) and estimated inversion strength (ETS). These two relationships are re-evaluated here using three separate observational cloud and two reanalysis datasets for three distinct but overlapping time periods. The analysis shows that the strength of the LTS and ETS relationships with low cloud amount varies, depending on the cloud data used. They have a strong relationship with stratocumulus cloud amount only.

Lauren Cutler↗
NASA NTRSDate not supplied

The Next Frontier for Remote Sensing of Freshwater HABS: Utilizing Imaging Spectroscopy Data to Identify Cyanobacteria Bloom Types

Cyanobacteria Harmful Algal Blooms (HABs) and their associated toxicity are a concern for inland waters. Due to the extensive spatial coverage and frequent data availability of satellite-based sensors, multi-spectral remote sensing tools have demonstrated utility for monitoring, understanding, and managing these blooms. The next frontier is utilizing high spectral resolution imaging spectroscopy data, such as from NASA’s upcoming PACE, GLIMR, and SBG missions planned for this decade, which allow for the development of more sophisticated cyanobacteria detection algorithms. We evaluate the performance of cyanobacteria genera differentiation algorithms using precursor datasets in support of these upcoming hyperspectral missions. In situ measurements of lake optical, biological, chemical, and physical properties are used to characterize cyanobacteria blooms in hypereutrophic Clear Lake, CA, USA, which supports large, diverse algal and cyanobacteria populations. Data collection occurred during 12 field events in 2021-2022 across all seasons. Three field events were conducted concurrently with hyperspectral data acquisitions from the DESIS sensor on the International Space Station. The results of this study will support the development of future tools utilizing satellite-based imaging spectroscopy data to identify the cyanobacteria genera present in a bloom, and thus the potential for cyanotoxin production. This outcome – the ability to quickly characterize a cyanobacteria bloom at a low cost – will have enormous benefits for public health.

remote sensing↗
NASA NTRSDate not supplied

Quantitative Assessment of the HLS Surface Reflectance Consistency

The Harmonized Landsat and Sentinel-2 (HLS) project produces compatible surface reflectance (SR) from observations acquired by Landsat-8/9 OLI and Sentinel-2A/2B MSI. The HLS harmonization procedures include atmosphere correction, cloud masking, view angle normalization, and bandpass adjustment. The objective of this study is to quantitatively assess the reflectance consistency between Landsat and Sentinel-2 within the Version 2 HLS data. We collected 545 pairs of same-day Landsat/Sentinel-2 images across the globe to represent a wide range of vegetation types and climate regimes. The mean absolute difference (MAD) in reflectance between Landat and Sentinel-2 was calculated as a consistency indicator for each harmonization step. The MAD generally increased after the atmosphere correction, and then greatly decreased after the BRDF and bandpass adjustments, to smaller than the top-of-atmosphere MAD values. The MAD ranged from 0.0048 to 0.0093 for the six common spectral bands (blue, green, red, NIR, SWIR1, and SWIR2) in the final products, only slightly greater than the difference between Landsat and Sentinel-2 calibrations.. An evaluation on a few commonly used vegetation indices also showed good agreement between Landsat and Sentinel-2 reflectance. All these evaluations demonstrate that the HLS project produces a consistent SR dataset from Landsat-OLI and Sentinel-MSI, which will be a valuable resource for a wide range of remote sensing applications.

Qiang Zhou↗
DOE OSTI2021

An analysis of gamma-ray data collected at traffic intersections in Northern Virginia

Gamma-ray spectral data were collected from sensors mounted to traffic signals around Northern Virginia. The data were collected over a span of approximately fifteen months. A subset of the data were analyzed manually and subsequently used to train machine-learning models to facilitate the evaluation of the remaining 50k anomalous events identified in the dataset. We describe the analysis approach used here and discuss the results in terms of radioisotope classes and frequency patterns over day-of-week and time-of-day spans. Data from this work has been archived and is available for future and ongoing research applications.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2025

Low-Temperature Geothermal Play Fairway Analysis for the Denver Basin

This project is part of a national initiative to showcase the benefits of incorporating low-temperature geothermal resource assessment into the deployment of geothermal heating, combined heat and power (CHP), and geothermal direct-use (GDU) technologies. The initiative was established to accelerate the country's decarbonization efforts by identifying potential for low-temperature geothermal resource utilization (<150 degrees C, e.g., CHP and GDU) in selected sedimentary basins with numerous population centers. The play fairway analysis (PFA) methodologies in this study were adapted from previous PFA investigations of sedimentary basin geothermal play types (SBGPTs) that evaluated the potential for low-temperature resources (<150 degrees C). Workflows, relevant datasets, a new Python library, and common and composite geological criteria maps are utilized to develop low-temperature geothermal resource favorability maps for the Denver Basin, a sedimentary basin spanning Colorado, Nebraska, and Wyoming. The replication of these methodologies in other SBGPTs can evaluate potential for low-temperature resources. To facilitate future assessment of low-temperature geothermal resources in SBGPTs, this project provides PFA workflows, data, tools, and favorability maps that will ultimately support the utilization of low-temperature geothermal resources in sedimentary basins.

combined heat and power↗
DOE OSTI2020

The ARM Data-oriented Metrics and Diagnostics Package for Climate Models - A New Tool for Evaluating Climate Models with Field Data

The U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) program User Facility produces ground-based long-term continuous unique measurements for atmospheric state, precipitation, turbulent fluxes, radiation, aerosol, cloud and the land surface, which are collected at multiple sites. These comprehensive datasets have been widely used to calibrate climate models and are proven to be invaluable for climate model development and improvement. This article introduces an evaluation package to facilitate the use of ground-based ARM measurements in climate model evaluation. The ARM data-oriented metrics and diagnostics package (ARM-DIAGS) includes both ARM observational datasets and a Python-based analysis toolkit for computation and visualization. The observational datasets are compiled from multiple ARM data products and specifically tailored for use in climate model evaluation. In addition, ARM-DIAGS also includes simulation data from models participating the Coupled Model Inter-comparison Project (CMIP), which will allow climate-modeling groups to compare a new, candidate version of their model to existing CMIP models. The analysis toolkit is designed to make the metrics and diagnostics quickly available to the model developers.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

AviSense: A Real-time System for Detection, Classification, and Analysis of Aviation Signals

Wireless systems are an integral part of aviation. Apart from their apparent use in air-to-ground communication, wireless systems play a crucial role in avionic functions including navigation and landing. An interference-free wireless environment is therefore critical for the uninterrupted operation and safety of an aircraft. Hence, there is an urgency for airport facilities to acquire the capability to continuously monitor aviation frequency bands for real-time detection of interference and anomalies. To meet this critical need, we design and build AviSense, an SDR-based real-time, versatile system for monitoring aviation bands. AviSense detects and characterizes signal activities to enable practical and effective anomaly detection. We identify and tackle the challenges posed by a diverse set of critical aviation bands and technologies. We evaluate our methodology with real-world aviation signal measurements and two custom datasets of anomalous signals. We find that our signal classification capability achieves a true positive rate of ~99%, with few exceptions, and a false positive rate of less than 4%. We also demonstrate that AviSense can effectively distinguish between different types of anomalies. We build and evaluate a prototype implementation of AviSense that supports distributed monitoring.

42 ENGINEERING↗
DOE OSTI2025

CGSim: A Simulation Framework for Large Scale Distributed Computing Environment

Large-scale distributed computing infrastructures such as the Worldwide LHC Computing Grid (WLCG) require comprehensive simulation tools for evaluating performance, testing new algorithms, and optimizing resource allocation strategies. However, existing simulators suffer from limited scalability, hardwired algorithms, lack of real-time monitoring, and inability to generate datasets suitable for modern machine learning approaches. We present CGSim, a simulation framework for large-scale distributed computing environments that addresses these limitations. Built upon the validated SimGrid simulation framework, CGSim provides high-level abstractions for modeling heterogeneous grid environments while maintaining accuracy and scalability. Key features include a modular plugin mechanism for testing custom workflow scheduling and data movement policies, interactive real-time visualization dashboards, and automatic generation of event-level datasets suitable for AI-assisted performance modeling. We demonstrate CGSim’s capabilities through a comprehensive evaluation using production ATLAS PanDA workloads, showing significant calibration accuracy improvements across WLCG computing sites. Scalability experiments show near-linear scaling for multi-site simulations, with distributed workloads achieving 6 × better performance compared to single-site execution. The framework enables researchers to simulate WLCG-scale infrastructures with hundreds of sites and thousands of concurrent jobs within practical time budget constraints on commodity hardware.

Vatsavai, Sairam Sri [Brookhaven National Laborato↗