Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “validation dataset”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

EMT data generation

The integration of inverter-based resources (IBRs) in power systems is accelerating, bringing with it significant benefits such as reduced greenhouse gas emissions, improved grid resilience, and increased energy independence. Despite these advantages, the widespread adoption of IBRs introduces several challenges, including issues related to grid stability, increased operational complexity, and the need for updated regulatory frameworks. To address these challenges, IEEE released Standard 2800 in 2022, which sets forth the necessary interconnection capabilities and performance criteria for IBRs connected to transmission and sub-transmission systems. This standard outlines the performance requirements to ensure the reliable integration of IBRs into the bulk power system. Furthermore, in 2023, the North American Electric Reliability Corporation (NERC) published a reliability guideline for electromagnetic transient (EMT) modeling of BPS-connected IBRs. This guideline provides recommendations for developing EMT model requirements, performing model quality checks, and implementing verification practices specifically for EMT models representing BPS-connected inverter-based resources in reliability studies conducted by transmission planners and planning coordinators. These standards and guidelines have a profound impact on EMT studies for transmission networks, influencing system stability analyses, grid recovery and resynchronization processes, fault ride-through evaluations, protection and coordination strategies, advanced control methodologies, and the inclusion of IBRs in transient models of transmission networks. As a result, the generation of EMT data is crucial for conducting various transient-based studies to understand the impact of IBRs. EMT data generation use cases serve as the basis for scenarios in event detection and identification use cases, providing comprehensive details about EMT data generation for transmission grids with inverter-based resources. These use cases supply sufficient training and validation datasets for subsequent EMT analysis algorithms.

Xia, Qianxue↗
DOE OSTI2022

Classification of Cloud Particle Imagery from Aircraft Platforms Using Convolutional Neural Networks

Abstract A vast amount of ice crystal imagery exists from a variety of field campaign initiatives that can be utilized for cloud microphysical research. Here, nine convolutional neural networks are used to classify particles into nine regimes on over 10 million images from the Cloud Particle Imager probe, including liquid and frozen states and particles with evidence of riming. A transfer learning approach proves that the Visual Geometry Group (VGG-16) network best classifies imagery with respect to multiple performance metrics. Classification accuracies on a validation dataset reach 97% and surpass traditional automated classification. Furthermore, after initial model training and preprocessing, 10 000 images can be classified in approximately 35 s using 20 central processing unit cores and two graphics processing units, which reaches real-time classification capabilities. Statistical analysis of the classified images indicates that a large portion (57%) of the dataset is unusable, meaning the images are too blurry or represent indistinguishable small fragments. In addition, 19% of the dataset is classified as liquid drops. After removal of fragments, blurry images, and cloud drops, 38% of the remaining ice particles are largely intersecting the image border (≥10% cutoff) and therefore are considered unusable because of the inability to properly classify and dimensionalize. After this filtering, an unprecedented database of 1 560 364 images across all campaigns is available for parameter extraction and bulk statistics on specific particle types in a wide variety of storm systems, which can act to improve the current state of microphysical parameterizations.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Power System Waveform Classification Using Time-Frequency and CNN

Many modern reclosers and circuit breakers have microprocessor relays that record waveforms of system events. In some cases, utilities may record a half-a-dozen event captures for every event. This is thousands of events per year. The industry needs faster, more automated, more conclusive, and easy-to-use systems that can process massive amounts of event recordings without extensive input/support from power system engineers. To address the need for a commercially viable solution that can classify waveform data, energies were directed to develop a universal neural network (NN) structure (deep learning algorithm) that works for a wide variety of system event types. The structure that showed the most promise was one that included the use of spectrograms. The technique has shown positive results in audio engineering, particularly with respect to speech recognition. A waveform signature could be treated as a spoken word like audio waveforms for specific things such as “YES” or “UP”. No two people produce the exact same waveform when speaking each of these words, but audio processing algorithms based on spectrograms and convolutional neural networks (CNN) can still distinguish the word regardless of the speaker. No two circuits produce the exact same waveform for a given event, but the NN can be trained to classify the event type regardless of the circuit or location on the circuit. A Power System Neural Network (PSNN) has been developed to use a CNN to classify events within waveform data for power systems. The waveform is converted to an array of values by way of spectrograms and interpreted as an image. This image is passed into the CNN. The test results on independent simulated test and validation datasets show greater than 99% accuracy. While the results thus far are based on simulated data, the performance of the PSNN is very promising and should work for a wide variety of power system conditions of interest. Ultimately, much of the custom code and tools used today and much of the manual effort expended today may be automated using this PSNN.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

The 2021 Blind PVPMC Modeling Intercomparison

This document provides the instructions for participating in the 2021 blind photovoltaic (PV) modeling intercomparison organized by the PV Performance Modeling Collaborative (PVPMC). It describes the system configurations, metadata, and other information necessary for the modeling exercise. The practical details of the validation datasets are also described. The datasets were published online in open access in April 2023, after completing the analysis of the results.

14 SOLAR ENERGY↗
DOE OSTI2023

Seasonal forecasting skill for the High Mountain Asia region in the Goddard Earth Observing System

Seasonal variability of the global hydrologic cycle directly impacts human activities, including hazard assessment and mitigation, agricultural decisions, and water resources management. This is particularly true across the High Mountain Asia (HMA) region, where availability of water resources can change depending on local seasonality of the hydrologic cycle. Forecasting the atmospheric states and surface conditions, including hydrometeorologically relevant variables, at subseasonal-to-seasonal (S2S) lead times of weeks to months is an area of active research and development. NASA's Goddard Earth Observing System (GEOS) S2S prediction system has been developed with this research goal in mind. Here, we benchmark the forecast skill of GEOS-S2S (version 2) hydrometeorological forecasts at 1–3-month lead times in the HMA region, including a portion of the Indian subcontinent, during the retrospective forecast period, 1981–2016. To assess forecast skill, we evaluate 2 m air temperature, total precipitation, fractional snow cover, snow water equivalent, surface soil moisture, and terrestrial water storage forecasts against the Modern-Era Retrospective analysis for Research and Applications, Version 2 (MERRA-2) and independent reanalysis data, satellite observations, and data fusion products. Anomaly correlation is highest when the forecasts are evaluated against MERRA-2 and particularly in variables with long memory in the climate system, likely due to the similar initial conditions and model architecture used in GEOS-S2S and MERRA-2. When compared to MERRA-2, results for the 1-month forecast skill range from an anomaly correlation of R anom =0.18 for precipitation to R anom =0.62 for soil moisture. Anomaly correlations are consistently lower when forecasts are evaluated against independent observations; results for the 1-month forecast skill range from R anom =0.13 for snow water equivalent to R anom =0.24 for fractional snow cover. We find that, generally, hydrometeorological forecast skill is dependent on the forecast lead time, the memory of the variable within the physical system, and the validation dataset used. Overall, these results benchmark the GEOS-S2S system's ability to forecast HMA hydrometeorology.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Integration of GOES Data for Solar Resource Assessment of the Contiguous United States

The National Solar Radiation Database (NSRDB), produced by the National Laboratory of the Rockies (NLR), provides high-resolution solar resource data for the contiguous United States (CONUS) using Geostationary Operational Environmental Satellite (GOES) East and West observations. This study evaluates the integration of multi-satellite data within the GOES-East/West overlap regions, where conventional longitude-based selection methods often produce an artificial boundary seam. Our results demonstrate that an advanced blending algorithm, which incorporates sun-satellite scattering angles and satellite viewing zenith angles, improves NSRDB accuracy and creates a spatially continuous dataset. Validation against ground-based irradiance measurements reveals reductions in both percentage error (PE) and normalized Root Mean Square Error (nRMSE), particularly in the central United States. The dynamical integration of multi-satellite data provides a robust foundation for more precise modeling of solar resource and improved spatiotemporal analysis of solar ramp across the CONUS.

14 SOLAR ENERGY↗
DOE OSTI2022

Development of Training Materials for Pathologists to Provide Machine Learning Validation Data of Tumor-Infiltrating Lymphocytes in Breast Cancer

The High Throughput Truthing project aims to develop a dataset for validating artificial intelligence and machine learning models (AI/ML) fit for regulatory purposes. The context of this AI/ML validation dataset is the reporting of stromal tumor-infiltrating lymphocytes (sTILs) density evaluations in hematoxylin and eosin-stained invasive breast cancer biopsy specimens. After completing the pilot study, we found notable variability in the sTILs estimates as well as inconsistencies and gaps in the provided training to pathologists. Using the pilot study data and an expert panel, we created custom training materials to improve pathologist annotation quality for the pivotal study. We categorized regions of interest (ROIs) based on their mean sTILs density and selected ROIs with the highest and lowest sTILs variability. In a series of eight one-hour sessions, the expert panel reviewed each ROI and provided verbal density estimates and comments on features that confounded the sTILs evaluation. We aggregated and shaped the comments to identify pitfalls and instructions to improve our training materials. From these selected ROIs, we created a training set and proficiency test set to improve pathologist training with the goal to improve data collection for the pivotal study. We are not exploring AI/ML performance in this paper. Instead, we are creating materials that will train crowd-sourced pathologists to be the reference standard in a pivotal study to create an AI/ML model validation dataset. The issues discussed here are also important for clinicians to understand about the evaluation of sTILs in clinical practice and can provide insight to developers of AI/ML models.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2023

The executive disruption model of tinnitus distress: Model validation in two independent datasets using factor score regression

This study presents the executive disruption model (EDM) of tinnitus distress and subsequently validates it statistically using two independent datasets (the Construction Dataset: n = 96 and the Validation Dataset: n = 200). The conceptual EDM was first operationalised as a structural causal model (construction phase). Then multiple regression was used to examine the effect of executive functioning on tinnitus-related distress (validation phase), adjusting for the additional contributions of hearing threshold and psychological distress. For both datasets, executive functioning negatively predicted tinnitus distress score by a similar amount (the Construction Dataset: β = −3.50, p = 0.13 and the Validation Dataset: β = −3.71, p = 0.02). Theoretical implications and applications of the EDM are subsequently discussed; these include the predictive nature of executive functioning in the development of distressing tinnitus, and the clinical utility of the EDM.

Clarke, Nathan A.↗
DOE OSTI2021

Highly Accurate Experimental Heave Decay Tests with a Floating Sphere: A Public Benchmark Dataset for Model Validation of Fluid–Structure Interaction

Highly accurate and precise heave decay tests on a sphere with a diameter of 300 mm were completed in a meticulously designed test setup in the wave basin in the Ocean and Coastal Engineering Laboratory at Aalborg University, Denmark. The tests were dedicated to providing a rigorous benchmark dataset for numerical model validation. The sphere was ballasted to half submergence, thereby floating with the waterline at the equator when at rest in calm water. Heave decay tests were conducted, wherein the sphere was held stationary and dropped from three drop heights: a small drop height, which can be considered a linear case, a moderately nonlinear case, and a highly nonlinear case with a drop height from a position where the whole sphere was initially above the water. The precision of the heave decay time series was calculated from random and systematic standard uncertainties. At a 95% confidence level, uncertainties were found to be very low—on average only about 0.3% of the respective drop heights. Physical parameters of the test setup and associated uncertainties were quantified. A test case was formulated that closely represents the physical tests, enabling the reader to do his/her own numerical tests. The paper includes a comparison of the physical test results to the results from several independent numerical models based on linear potential flow, fully nonlinear potential flow, and the Reynolds-averaged Navier–Stokes (RANS) equations. A high correlation between physical and numerical test results is shown. The physical test results are very suitable for numerical model validation and are public as a benchmark dataset.

42 ENGINEERING↗
DOE OSTI2026

Maximizing efficiency of dataset compression for machine learning potentials with information theory

Machine learning interatomic potentials (MLIPs) balance high accuracy and lower costs compared to density functional theory calculations, but their performance often depends on the size and diversity of training datasets. Large datasets improve model accuracy and generalization but are computationally expensive to produce and train on, while smaller datasets risk discarding rare but important atomic environments and compromising MLIP accuracy/reliability. Here, we develop an information-theoretical framework to quantify the efficiency of dataset compression methods and propose an algorithm that maximizes this efficiency. By framing atomistic dataset compression as an instance of the minimum set cover (MSC) problem over atom-centered environments, our method identifies the smallest subset of structures that contains as much information as possible from the original dataset while pruning redundant information. The approach is extensively demonstrated on the GAP-20 and TM23 datasets and validated on 64 varied datasets from the ColabFit repository. Across all cases, MSC consistently retains outliers, preserves dataset diversity, and reproduces the long-tail distributions of forces even at high compression rates, outperforming other subsampling methods. Furthermore, MLIPs trained on MSC-compressed datasets exhibit reduced error for out-of-distribution data even in low-data regimes. We explain these results using an outlier analysis and show that such quantitative conclusions could not be achieved with conventional dimensionality reduction methods. The algorithm is implemented in the open-source QUESTS package and can be used for several tasks in atomistic modeling, from data subsampling, outlier detection, and training improved MLIPs at a lower cost.

36 MATERIALS SCIENCE↗
DOE OSTI2021

dsgrid (Demand-Side Grid Toolkit) [SWR-21-52]

The demand-side grid model harnesses decades of sector-specific energy modeling expertise to understand current and future U.S. electricity load for power systems analyses. The API provides tools for submitting datasets to dsgrid projects, validating individual datasets and project data as a whole, querying, analysis, and visualization.

Hale, Elaine↗
DOE OSTI2024

Using pile-up collisions as an abundant source of low-energy hadronic physics processes in ATLAS and an extraction of the jet energy resolution

During the 2015–2018 data-taking period, the Large Hadron Collider delivered proton-proton bunch crossings at a centre-of-mass energy of 13 TeV to the ATLAS experiment at a rate of roughly 30 MHz, where each bunch crossing contained an average of 34 independent inelastic proton-proton collisions. The ATLAS trigger system selected roughly 1 kHz of these bunch crossings to be recorded to disk. Offline algorithms then identify one of the recorded collisions as the collision of interest for subsequent data analysis, and the remaining collisions are referred to as pile-up. Pile-up collisions represent a trigger-unbiased dataset, which is evaluated to have an integrated luminosity of 1.33 pb -1 in 2015–2018. This is small compared with the normal trigger-based ATLAS dataset, but when combined with vertex-by-vertex jet reconstruction it provides up to 50 times more dijet events than the conventional single-jet-trigger-based approach, and does so without adding any additional cost or requirements on the trigger system, readout, or storage. The pile-up dataset is validated through comparisons with a special trigger-unbiased dataset recorded by ATLAS, and its utility is demonstrated by means of a measurement of the jet energy resolution in dijet events, where the statistical uncertainty is significantly reduced for jet transverse momenta below 65 GeV.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2026

MARIAH PCAP data for Validation Demonstration

This dataset holds simulated PCAP (packet capture) data from the SCEPTRE validation demonstration model as a set of pairwise communications between devices via specific protocols. All connections should be assumed to be symmetric, as this data is an aggregation of the true PCAP. A mapping is also provided associating each IP address with its true device type.

cyber-physical system↗
DOE OSTI2025

A network-enabled pipeline for gene discovery and validation in non-model plant species

Identifying key regulators of important genes in non-model crop species is challenging due to limited multi-omics resources. To address this, we introduce the network-enabled gene discovery pipeline NEEDLE, a user-friendly tool that systematically generates coexpression gene network modules, measures gene connectivity, and establishes network hierarchy to pinpoint key transcriptional regulators from dynamic transcriptome datasets. After validating its accuracy with two independent datasets, we applied NEEDLE to identify transcription factors (TFs) regulating the expression of cellulose synthase-like F6 ( CSLF6 ), a crucial cell wall biosynthetic gene, in Brachypodium and sorghum. Our analyses uncover regulators of CSLF6 and also shed light on the evolutionary conservation or divergence of gene regulatory elements among grass species. These results highlight NEEDLE’s capability to provide biologically relevant TF predictions and demonstrate its value for non-model plant species with dynamic transcriptome datasets.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Very small-scale, segregating-fluidized-bed experiments: A dataset for CFD-DEM validation and uncertainty quantification

IWe report fluidization experiments were conducted on a small scale and with a rapid response (short duration) to enable corresponding simulations at low-computational cost. Rise times are reported for four or fewer polyethylene particles (intruders) in an air-fluidized bed of ~5000 group D glass beads. Experimental inputs were completely characterized—particle properties, system dimensions and operating conditions—which is necessary for validating computational fluid mechanics (CFD)-discrete element method (DEM) including a comprehensive uncertainty quantification (UQ) analysis. Input uncertainties are reported as bounds or cumulative distribution functions of measured values. The staggering number of simulations required to complete a UQ analysis (~O[10 4 ] simulations corresponding to ~5 uncertain inputs) motivates this study. These segregating-bed experiments are designed to permit analogous CFD-DEM simulations to complete in less than a day on a single (~2.5 GHz) computational processor unit (CPU). Segregation times are reported for several operating conditions, intruder sizes, and initial configurations, providing a rich dataset for numerical model testing, validation and UQ.

42 ENGINEERING↗
DOE OSTI2021

Quantifying the Solar Energy Resource for Puerto Rico

After Hurricane Maria, multiple U.S. Department of Energy laboratories studied the state of the electric grid in Puerto Rico and analyzed grid resilience and grid integration of renewable energy. As part of the work done at the National Renewable Energy Laboratory, researchers created new solar resource data, conducted a technical potential and supply curve analysis, and studied the interannual variability of the solar resource. A new methodology was developed to downscale solar resource data from the National Solar Radiation Data Base (NSRDB) from a 4-km x 4-km spatial and 30-minute temporal resolution to a 2 km x 2 km and 5-minute resolution. This methodology primarily used simple physical principles to develop high-resolution cloud properties which were then used to compute solar radiation. The high-resolution datasets were validated against ground measurements and the error metrics were found to be similar to the original lower resolution dataset. Using 20 years of downscaled data from the NSRDB multi-year capacity factors for photovoltaics (PV) were developed for both single-axis tracking and fixed latitude-tilt configurations. Use of the multi-year data provides the ability to understand variability in capacity factors due to variability in weather over a long period of time. For Puerto Rico the coastal regions were found to have significant higher capacity factors than inland. Using land-use and terrain information a technical potential analysis was conducted for Puerto Rico. This analysis restricted single PV plant development to a maximum of 100 MW nameplate capacity. The nameplate capacity for each municipality were then determined. Based on our assumptions, 56 of the 78 total municipalities of Puerto Rico contain some level of solar capacity. Most of the interior municipalities did not have any capacity because of the geographic exclusions used in this study. The lowest capacity for a PV plant observed in a municipality was 10 MW. The maximum capacity within a county was 2,000 MW. Further a supply curve analysis was conducted by taking the results of the technical potential and quantifying system and transmission costs. The levelized cost of energy (LCOE) was calculated for each theoretical PV plant site, and the levelized cost of transmission was added to the LCOE to produce a total cost estimate for each site. The results of the supply curve analysis allow for a relative comparison of the cost for integrating new PV capacity into the grid. This analysis indicates that cheaper total LCOE sites tend to be larger in capacity. The total capacity in this study was found to be far beyond the maximum peak load for the island. However, this study does not consider the economic and market potential for development. The cumulative capacity presented in this study assumes that the best locations are developed first and ignores the complex decision paths for new power plant development. Therefore, this analysis can only be treated as illustrative. Finally this study investigates the impact of inter-annual variability of resource using a variety of metrices including probability of exceedance and variation in capacity factor and LCOE. This study demonstrates that the capacity factor or LCOE could vary by over 10% year to year. This clearly indicates the risks involved in using any particular year of data and clearly points to the use of multi-year data to reduce some of the risks related to variability in weather.

14 SOLAR ENERGY↗
DOE OSTI2025

Dark Energy Survey: Modeling strategy for multiprobe cluster cosmology and validation for the Full Six-year Dataset

We introduce an updated To&Krause2021 model for joint analyses of cluster abundances and large-scale two-point correlations of weak lensing and galaxy and cluster clustering (termed CL+3x2pt analysis) and validate that this model meets the systematic accuracy requirements of analyses with the statistical precision of the final Dark Energy Survey (DES) Year 6 (Y6) dataset. The validation program consists of two distinct approaches, (1) identification of modeling and parameterization choices and impact studies using simulated analyses with each possible model misspecification (2) end-to-end validation using mock catalogs from customized Cardinal simulations that incorporate realistic galaxy populations and DES-Y6-specific galaxy and cluster selection and photometric redshift modeling, which are the key observational systematics. In combination, these validation tests indicate that the model presented here meets the accuracy requirements of DES-Y6 for CL+3x2pt based on a large list of tests for known systematics. In addition, we also validate that the model is sufficient for several other data combinations: the CL+GC subset of this data vector (excluding galaxy--galaxy lensing and cosmic shear two-point statistics) and the CL+3x2pt+BAO+SN (combination of CL+3x2pt with the previously published Y6 DES baryonic acoustic oscillation and Y5 supernovae data).

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗