Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluation datasets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 505 records · Page 28
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Application of unsupervised deep learning to image segmentation and in-situ contact angle measurements in a CO 2 -water-rock system

Rock surface wettability is a critical property that regulates multiphase flows in porous media, which can be quantified using the surface contact angle (CA). X-ray micro-computed tomography (μCT) provides an effective approach to in-situ measurements of surface CAs. However, the CA measurement accuracy depends significantly on the quality of CT image segmentation, which is the clustering of CT pixels into separate phases. Inspired by this, we developed a deep learning (DL)-based CA measurement workflow. Motivated by the recent tremendous progress in unsupervised learning techniques and aiming to avoid expensive manual data annotations, an unsupervised DL pipeline for CT image segmentation was proposed and implemented, which includes unsupervised model training and post-processing. The unsupervised model training was driven by a novel loss function constrained with feature similarity and spatial continuity and implemented by iterative forward and backward paths; the former clustered the pixel-wise feature vectors extracted by convolution neural networks, whereas the latter updated the parameters using gradient descent. An over-segmentation strategy was adopted for model training. The post-processing steps based on agglomerative hierarchical clustering (AHC) were implemented to further merge the over-segmented model output to the desired cluster number, which is intended to improve the efficiency of image segmentation. The developed unsupervised DL pipeline was compared with other commonly-used image segmentation methods using pixel-wise and physics-based evaluation metrics on a synthetic raw-image dataset, which had a known ground truth. The unsupervised DL pipeline showed the best performance. Next, the segmented images were input to an automatic CA measurement tool, and the results were validated by comparisons with manual measurements. The CA values from the manual and automatic measurements showed similar distributions and statistical properties. The automatic measurement demonstrated a wider spectrum because of the much larger number of measurement data points. The primary novelty of the unsupervised DL pipeline developed in this study lies in the novel loss function and the over-segmentation strategy associated with AHC post-processing. Finally, the workflow has been proven an efficient tool for pore-scale wettability characterization, which has a wide range of applications in fundamental studies of multiphase flows in natural porous media, which have critical implications to geological carbon sequestration, hydrocarbon energy recovery, and contaminant transport in groundwater.

42 ENGINEERING↗
DOE OSTI2024

Development of typical solar years and typical wind years for efficient assessment of renewable energy systems across the U.S.

Weather data plays a critical role in renewable energy analysis. Compared to using multiple Actual Meteorological Years, simulations using a single typical year require significantly fewer computational resources. Previous efforts to create typical weather datasets for renewable energy analysis either lack justified or optimized strategies for selecting and weighting different weather parameters or are limited to a few specific locations. Here, in this study, we developed a dataset comprising Typical Solar Years (TSYs) and Typical Wind Years (TWYs) for over 2000 locations across the U.S., based on data from NASA's POWER project. The strategies for creating TSYs and TWYs were optimized based on the simulated outputs of various PV systems and wind turbines in 16 representative cities. This dataset provides an efficient means for the rapid evaluation and optimization of renewable energy systems throughout the entire U.S. Additionally, the optimal strategies identified in this study can be directly applied to create near-optimal TSYs and TWYs for most locations worldwide. However, readers can also employ the optimization approach presented in this work to develop optimal strategies tailored for particular regions.

NASA POWER↗
DOE OSTI2024

Learning From User Behavior: A Survey-Assist Algorithm for Longitudinal Mobility Data Collection

GPS-based travel surveys are widely used in mobility studies to gather crucial qualitative data, like purpose, transportation mode and replaced mode. However, survey response still poses a burden to users, especially in long-term mobility studies, leading to response fatigue. We explore a survey-assist strategy to ease this burden by a novel, user-level modeling approach that leverages past responses from each user to predict responses for new trips, without relying on external data sources like GIS data. We investigate three main algorithms for predicting responses: (i) clustering trips and extrapolating responses for similar trips, (ii) using random forest classification, and (iii) clustering that uses a hybrid algorithm to determine spatial structure, which is then fed as input to a classic random forest classifier. The clustering approach can flexibly predict responses for even complex qualitative survey questions; it achieved F-scores of 65%. The random forest pipeline uses architecture that restricts it to predicting three predetermined survey questions: trip purpose, mode, and replaced mode. However, it achieved F-scores of 78%. While the survey-assist approach has been implemented by several proprietary systems, to our knowledge, this is the first exploration in the academic literature. It follows that this is also the first rigorous evaluation of multiple algorithms that can implement the approach. The evaluation uses a large scale, publicly available, longitudinal dataset consisting of ~ 92k trips from 235 users over a period of roughly one and a half years. With this approach, travel surveys can be pre-filled with the predicted responses for each trip, thus streamlining the survey process for users. Combined with an active learning system that requests user input on low-confidence predictions, models can be updated and improved over time to better support the long-term collection of longitudinal qualitative data.

clustering↗
DOE OSTI2022

An End-To-End Earthquake Detection Method for Joint Phase Picking and Association Using Deep Learning

Earthquake monitoring by seismic networks typically involves a workflow consisting of phase detection/picking, association, and location tasks. In recent years, the accuracy of these individual stages has been improved through the use of machine learning techniques. Here, in this study, we introduce a new end-to-end approach that improves overall earthquake detection accuracy by jointly optimizing each stage of the detection pipeline. We propose a neural network architecture for the task of multi-station processing of seismic waveforms recorded over a seismic network. This end-to-end architecture consists of three sub-networks: a backbone network that extracts features from raw waveforms, a phase picking sub-network that picks P- and S-wave arrivals based on these features, and an event detection sub-network that aggregates the features from multiple stations to associate and detect earthquakes across a seismic network. We use these sub-networks together with a shift-and-stack module based on back-projection that introduces kinematic constraints on arrival times, allowing the neural network model to generalize to different velocity models and to variable station geometry in seismic networks. We evaluate our proposed method on the STanford EArthquake Dataset (STEAD) and on the 2019 Ridgecrest, CA earthquake sequence. The results demonstrate that our end-to-end approach can effectively pick P- and S-wave arrivals and achieve earthquake detection accuracy rivaling that of other state-of-the-art approaches. Because our approach preserves information across tasks in the detection pipeline, it has the potential to outperform approaches that do not.

58 GEOSCIENCES↗
DOE OSTI2023

Global Nuclear Explosion Discrimination Using a Convolutional Neural Network

Using P-wave seismograms, we trained a seismic source classifier using a Convolutional Neural Network. We trained for three classes: earthquake P-wave, underground nuclear explosion (UNE) P-wave, and noise. With the current absence of nuclear testing by countries that have signed the Comprehensive Test Ban Treaty, high quality seismic data from UNEs is limited. Even with limited training data, our model can accurately characterize most events recorded at regional and teleseismic distances, finding over 95% signals in the validation set. We applied the model on holdout datasets of the North Korean test explosions to evaluate the performance on unique region and station-source pairs, with promising results. Additionally, we tested on the Source Physics Experiment events to investigate the potential for chemical explosions to act as a surrogate for nuclear explosions. We anticipate that machine-learning models like our classifier system can have broad application for other seismic signals including volcanic and non-volcanic tremor, anomalous earthquakes, ice-quakes or landslide-quakes.

58 GEOSCIENCES↗
DOE OSTI2022

Disentangling multiple scattering with deep learning: application to strain mapping from electron diffraction patterns

Abstract A fast, robust pipeline for strain mapping of crystalline materials is important for many technological applications. Scanning electron nanodiffraction allows us to calculate strain maps with high accuracy and spatial resolutions, but this technique is limited when the electron beam undergoes multiple scattering. Deep-learning methods have the potential to invert these complex signals, but require a large number of training examples. We implement a Fourier space, complex-valued deep-neural network, FCU-Net, to invert highly nonlinear electron diffraction patterns into the corresponding quantitative structure factor images. FCU-Net was trained using over 200,000 unique simulated dynamical diffraction patterns from different combinations of crystal structures, orientations, thicknesses, and microscope parameters, which are augmented with experimental artifacts. We evaluated FCU-Net against simulated and experimental datasets, where it substantially outperforms conventional analysis methods. Our code, models, and training library are open-source and may be adapted to different diffraction measurement problems.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Using convolutional neural networks to detect edge localized modes in DIII-D from Doppler backscattering measurements

In H-mode tokamak plasmas, the plasma is sometimes ejected beyond the edge transport barrier. These events are known as edge localized modes (ELMs). ELMs cause a loss of energy and damage the vessel walls. Understanding the physics of ELMs, and by extension, how to detect and mitigate them, is an important challenge. In this paper, we focus on two diagnostic methods—deuterium-alpha (D α ) spectroscopy and Doppler backscattering (DBS). The former detects ELMs by measuring Balmer alpha emission, while the latter uses microwave radiation to probe the plasma. DBS has the advantages of having a higher temporal resolution and robustness to damage. These advantages of DBS diagnostic may be beneficial for future operational tokamaks, and thus, data processing techniques for DBS should be developed in preparation. In sight of this, we explore the training of neural networks to detect ELMs from DBS data, using D α data as the ground truth. With shots found in the DIII-D database, the model is trained to classify each time step based on the occurrence of an ELM event. The results are promising. When tested on shots similar to those used for training, the model is capable of consistently achieving a high f1-score of 0.93. Furthermore, this score is a performance metric for imbalanced datasets that ranges between 0 and 1. We evaluate the performance of our neural network on a variety of ELMs in different high confinement regimes (grassy ELM, RMP mitigated, and wide-pedestal), finding broad applicability. Beyond ELMs, our work demonstrates the wider feasibility of applying neural networks to data from DBS diagnostic.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

A comparative analysis of YOLOv8 and U-Net image segmentation approaches for transmission electron micrographs of polycrystalline thin films

Metallic thin films offer a platform to experimentally study the dynamics of microstructural evolution, but the required transmission electron microscopy (TEM)-based imaging generates complex images that are challenging to segment and quantify. This work provides a comparative analysis of a new YOLOv8 model and an established U-Net model for bright-field TEM images of polycrystals, employing a framework leveraging physical observables to evaluate performance against two hand-traced benchmark datasets. This methodology obviates the comparison of large, diversely structured, and manually labeled datasets that are required to assess performance on a per-image/per-pixel basis. It is found that the YOLOv8 model, adapted for real-time instance segmentation, has up to 43× faster inferencing (NVIDIA GeForce RTX 4090) compared to U-Net and reconstructs hand-traced grain size distributions (GSDs) with excellent fidelity, finding mean diameter within 3% for grains near an optimal magnification; for grains that deviate from the optimal pixel-diameter, the size of small- (large)-diameter grains is systematically over- (under)-estimated. This is partially mitigated by including scale-aware augmentations during training. Moreover, when the bias is corrected post-inference by a rigid shift in distribution, the YOLOv8 model reproduces ground truth GSDs with exceptional fidelity, with statistical tests indicating <5% probability that the distributions are distinct. Based on ground truth data, calibration curves pertaining to this shift can be constructed for a given model. This issue is not present in the U-Net model’s results, indicating that for quantitative measurements where the true size of objects is of interest, special procedures must be implemented for YOLO-based models.

36 MATERIALS SCIENCE↗
DOE OSTI2020

LyaCoLoRe : synthetic datasets for current and future Lyman-α forest BAO surveys

The statistical power of Lyman-α forest Baryon Acoustic Oscillation (BAO) measurements is set to increase significantly in the coming years as new instruments such as the Dark Energy Spectroscopic Instrument deliver progressively more constraining data. Generating mock datasets for such measurements will be important for validating analysis pipelines and evaluating the effects of systematics. With such studies in mind, here we present LyaCoLoRe: a package for producing synthetic Lyman-α forest survey datasets for BAO analyses. LyaCoLoRe transforms initial Gaussian random field skewers into skewers of transmitted flux fraction via a number of fast approximations. In this work we explain the methods of producing mock datasets used in LyaCoLoRe, and then measure correlation functions on a suite of realisations of such data. We demonstrate that we are able to recover the correct BAO signal, as well as large-scale bias parameters similar to literature values. Finally, we briefly describe methods to add further astrophysical effects to our skewers — high column density systems and metal absorbers — which act as potential complications for BAO analyses.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Multilevel atomic structural model for interstratified opal materials

The structure of opal has long fascinated scientists. It occurs in a number of structural states, ranging from amorphous to exhibiting features of stacking disorder. Opal-CT, where C and T signify cristobalite- and tridymite-like interstratification, represents an important link in the length scales between amorphous and crystalline states. However, details about local atomic (dis)order and arrangements extending to long-range stacking faults in opal polymorphs remain incompletely understood. Here, a multilevel modeling approach is reported that considers stacking states in correlation with the abundance of C and T segments as a high-level structural parameter (i.e. not each atom). Optimization accounting for inter-tetrahedral bond lengths and angles and the regularity of the silicate tetrahedra is included as lower levels of structural parameters. Together, a set of parameters with both coarse-grained and atomistic features for different levels of structural details is refined. Structural disorder at the ~10–100 Å distance scale is evaluated using experimental pair distribution function and diffraction datasets, comparing peak intensities, widths and asymmetry. Here this work presents a complete multilevel structural description of natural opal-CT and explains many of the unusual features observed in X-ray powder diffraction patterns. This modeling approach can be adopted generally for analyzing layered materials and their assembly into 3D structures.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Efficient Client Selection in Federated Learning

Federated Learning (FL) enables decentralized machine learning while preserving data privacy. This paper proposes a novel client selection framework that integrates differential privacy and fault tolerance. The adaptive client selection adjusts the number of clients based on performance and system constraints, with noise added to protect privacy. Evaluated on the UNSW-NB15 and ROAD datasets for network anomaly detection, the method improves accuracy by 7% and reduces training time by 25 % compared to baselines. Fault tolerance enhances robustness with minimal performance trade-offs.

Marfo, William [University of Texas at El Paso,Dep↗
DOE OSTI2020

Autonomous Selective Parts-Based Tracking

Object tracking from videos is still a challenging task due to various changes throughout a video sequence including occlusions, motion blur, scale and other deformation changes. In this paper, we propose a selective parts-based approach, using correlation filters, that makes choices based on a consensus of the parts and global tracking. Moreover, we further enhance our parts-based approach by introducing a segmentation-assisted parts initialization. In addition, we present a genetic algorithm-based method to autonomously select various parameters of the tracking algorithm, as opposed to the common practice of manually tuning those parameters. In contrast to existing part-based methods, the proposed method does not dilute accurate tracking by averaging results over multiple parts at every frame. Instead, we take a selective approach based on the relative weight of the responses across parts. Moreover, we only make location corrections when a part diverges, and rely on these location corrections to maintain an accurate appearance model. In the case of occlusions, which are among the main reasons for using a parts-based approach, our proposed approach consistently achieves the best performance. It is due to the ability to handle occlusion and not dilute decisions with incorrect parts, that our proposed approach enables state-of-the-art performance. The approach we propose was evaluated on videos from three different challenging benchmark datasets. Our approach has resulted in better overall precision and success rates for three different base tracking approaches.

42 ENGINEERING↗
DOE OSTI2024

A Data-Driven Framework for Power System Event Type Identification via Safe Semi-Supervised Techniques

Herein this paper investigates the use of phasor measurement unit (PMU) data with deep learning techniques to construct real-time event identification models for transmission networks. Increasing penetration of distributed energy resources represents a great opportunity to achieve decarbonization, as well as challenges in systematic situational awareness. When high-resolution PMU data and sufficient manually recorded event labels are available, the power event identification problem is defined as a statistical classification problem that can be solved by numerous cutting-edge classifiers. However, in real grids, collecting tremendous high-quality event labels is quite expensive. Utilities frequently have a large number of event records without in-depth details (i.e., unlabeled events). To bridge this gap, we propose a novel semi-supervised learning-based method to improve the performance of event classifiers trained with a limited number of labeled events by exploiting the information from massive unlabeled events. In other words, compared to existing data-driven methods, our method requires only a small portion of labeled data to achieve a similar level of accuracy. Meanwhile, this work discusses and addresses the performance degradation caused by class distribution mismatch between the training set and the real applications. Based on the proposed safe learning mechanism, our model does not directly use all unlabeled events during model training, but selectively uses them through a comprehensive evaluation procedure. Numerical studies on a sizable PMU dataset have been used to validate the performance of the proposed method.

42 ENGINEERING↗
DOE OSTI2025

SigTime: Learning and Visually Explaining Time Series Signatures

Understanding and distinguishing temporal patterns in time series data is essential for scientific discovery and decision-making. For example, in biomedical research, uncovering meaningful patterns in physiological signals can improve diagnosis, risk assessment, and patient outcomes. However, existing methods for time series pattern discovery face major challenges, including high computational complexity, limited interpretability, and difficulty in capturing meaningful temporal structures. Here, to address these gaps, we introduce a novel learning framework that jointly trains two Transformer models using complementary time series representations: shapelet-based representations to capture localized temporal structures and traditional feature engineering to encode statistical properties. The learned shapelets serve as interpretable signatures that differentiate time series across classification labels. Additionally, we develop a visual analytics system—SigTime—with coordinated views to facilitate exploration of time series signatures from multiple perspectives, aiding in useful insights generation. We quantitatively evaluate our learning framework on eight publicly available datasets and one proprietary clinical dataset. Additionally, we demonstrate the effectiveness of our system through two usage scenarios along with the domain experts: one involving public ECG data and the other focused on preterm labor analysis.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Simulation-Based Inference for Neutrino Parameter Tuning

This code trains a simulation-based inference (SBI) model for neutrino interaction parameter tuning and subsequently evaluates its performance on MicroBooNE, T2K, and NuWro datasets.

Tame-Narvaez, Karla [Fermi National Accelerator La↗
DOE OSTI2020

Improving Predictability of Mixed-Phase Clouds and Aerosol Interactions in the Community Earth System Model (CESM) with ARM Measurements

The objective of this project is to improve the simulation and predictability of mixed-phase clouds and aerosol interactions in the Community Earth System Model (CESM) through comparisons with the DOE ARM observations. There are four major goals of the proposed study: (1) Improve the representation of ice microphysical processes in mixed-phase clouds; (2) Develop a long-term multi-sensor mixed-phase cloud dataset; (3) Test the performance of improved ice microphysics in CESM-CAM5 with the ARM data; and (4) Examine mixed-phase cloud microphysics-aerosol-dynamics-radiation interactions in CESM-CAM5. In this project, we have (1) Improved the representation of ice microphysical processes in mixed-phase clouds that include ice nucleation, ice depositional growth through the Wegener–Bergeron–Findeisen (WBF) process, feedbacks between cloud microphysics, dynamics and radiation, and provides links to aerosol properties in the Community Earth System Model (CESM)-Community Atmosphere Model version 5 (CAM5); (2) Developed a long-term multi-sensor mixed-phase cloud dataset. The mixed-phase cloud retrieval algorithms are improved by refining the phase determinations and ice number concentration retrievals. A multi-year mixed-phase cloud observation dataset (1999-2004; October 2013-February 2017) is regenerated at the ARM Barrow site with the improved algorithms. The dataset has been used in the global model evaluation to improve the model parameterizations; (3) Tested the performance of improved ice microphysics in CESM-CAM5 with the ARM data. CESM-CAM5 is run in the DOE Cloud-Associated Parameterizations Testbed (CAPT) and in the single-column model (SCM) mode to facilitate comparison with ARM observations. Multi-year CAPT simulations of seasonal variations of cloud microphysical properties, cloud liquid water path (LWP) and ice water path (IWP), cloud longwave and shortwave forcings, cloud occurrence frequency are evaluated against the ARM multi-sensor mixed-phase cloud retrievals; and (4) Examined mixed-phase cloud microphysics-aerosol-dynamics-radiation interactions in CESM-CAM5 that include the impacts of different ice nucleation mechanisms, sensitivity to dust IN concentration, treatment of the WBF process on mixed-phase cloud properties. Aerosol effect on mixed-phase clouds through the liquid phase (droplet activation) and ice phase processes (e.g., the glaciation indirect effect) is investigated.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Surface and Buried Thermal, and RGB Unexploded Ordnance Data Collection

This document provides a description of a data collection campaign of unexploded ordnance (UXOI) set. The dataset captures a controlled UAV imaging campaign designed to support detection of UXO across varied environmental conditions. Data were collected during three campaigns in Norris and Northeast Knoxville, Tennessee, using RGB, and thermal sensors mounted on Parrot UKR. In total, the dataset contains 9925 images, 26 full-motion video, and approximately 81.99 GB of data, collected across late spring/summer conditions, every hour during sunlight, and multiple surface contexts, including tall grass, short grass, gravel, as well as buried in sand, and other gravel mixtures. The collection was designed to capture thermal and visual variability relevant to UXO detection in agricultural land, bare earth, and subsurface. Review of the imagery showed that ordnance was most detectable during periods of changing solar input, especially approximately 10-60 minutes after sunrise, approximately 20-60 minutes after sunset, and 2-3 min after cloud cover interrupted prolonged solar heating. These conditions increased thermal contrast because many ordnance items retained or released heat differently than the surrounding vegetation and ground surface. This dataset provides a useful resource for developing and evaluating airborne UXO detection methods under realistic field conditions. All ordnance used in the study was inert, and thermal behavior may differ from that of live ordnance. In addition, variation in ordnance type, composition, and placement introduced differences in thermal response that should be considered when interpreting results.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2023

Characterizing Families of Spectral Similarity Scores and Their Use Cases for Gas Chromatography–Mass Spectrometry Small Molecule Identification

Metabolomics provides a unique snapshot into the world of small molecules and the complex biological processes that govern the human, animal, plant, and environmental ecosystems encapsulated by the One Health modeling framework. However, this “molecular snapshot” is only as informative as the number of metabolites confidently identified within it. The spectral similarity (SS) score is traditionally used to identify compound(s) in mass spectrometry approaches to metabolomics, where spectra are matched to reference libraries of candidate spectra. Unfortunately, there is little consensus on which of the dozens of available SS metrics should be used. This lack of standard SS score creates analytic uncertainty and potentially leads to issues in reproducibility, especially as these data are integrated across other domains. In this work, we use metabolomic spectral similarity as a case study to showcase the challenges in consistency within just one piece of the One Health framework that must be addressed to enable data science approaches for One Health problems. Here, using a large cohort of datasets comprising both standard and complex datasets with expert-verified truth annotations, we evaluated the effectiveness of 66 similarity metrics to delineate between correct matches (true positives) and incorrect matches (true negatives). We additionally characterize the families of these metrics to make informed recommendations for their use. Our results indicate that specific families of metrics (the Inner Product, Correlative, and Intersection families of scores) tend to perform better than others, with no single similarity metric performing optimally for all queried spectra. This work and its findings provide an empirically-based resource for researchers to use in their selection of similarity metrics for GC-MS identification, increasing scientific reproducibility through taking steps towards standardizing identification workflows.

59 BASIC BIOLOGICAL SCIENCES↗