Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,171 records · Page 65

Cloud micro- and macrophysical properties from ground-based remote sensing during the MOSAiC drift experiment

In the framework of the Multidisciplinary drifting Observatory for the Study of Arctic Climate Polarstern expedition, the Leibniz Institute for Tropospheric Research, Leipzig, Germany, operated the shipborne OCEANET-Atmosphere facility for cloud and aerosol observations throughout the whole year. OCEANET-Atmosphere comprises, amongst others, a multiwavelength Raman lidar, a microwave radiometer, and an optical disdrometer. A cloud radar was operated aboard Polarstern by the US Atmospheric Radiation Measurement program. These measurements were processed by applying the so-called Cloudnet methodology to derive cloud properties. To gain a comprehensive view of the clouds, lidar and cloud radar capabilities for low- and high-altitude observations were combined. Cloudnet offers a variety of products with a spatiotemporal resolution of 30 s and 30 m, such as the target classification, and liquid and ice microphysical properties. Additionally, a lidar-based low-level stratus retrieval was applied for cloud detection below the lowest range gate of the cloud radar. Based on the presented dataset, e.g., studies on cloud formation processes and their radiative impact, and model evaluation studies can be conducted.

54 ENVIRONMENTAL SCIENCES↗

A unified ensemble soil moisture dataset across the continental United States

Abstract A unified ensemble soil moisture (SM) package has been developed over the Continental United States (CONUS). The data package includes 19 products from land surface models, remote sensing, reanalysis, and machine learning models. All datasets are unified to a 0.25-degree and monthly spatiotemporal resolution, providing a comprehensive view of surface SM dynamics. The statistical analysis of the datasets leverages the Koppen-Geiger Climate Classification to explore surface SM’s spatiotemporal variabilities. The extracted SM characteristics highlight distinct patterns, with the western CONUS showing larger coefficient of variation values and the eastern CONUS exhibiting higher SM values. Remote sensing datasets tend to be drier, while reanalysis products present wetter conditions. In-situ SM observations serve as the basis for wavelet power spectrum analyses to explain discrepancies in temporal scales across datasets facilitating daily SM records. This study provides a comprehensive soil moisture data package and an analysis framework that can be used for Earth system model evaluations and uncertainty quantification, quantifying drought impacts and land–atmosphere interactions and making recommendations for drought response planning.

54 ENVIRONMENTAL SCIENCES↗

Efficient learning of accurate surrogates for simulations of complex systems

Machine learning methods are increasingly deployed to construct surrogate models for complex physical systems at a reduced computational cost. However, the predictive capability of these surrogates degrades in the presence of noisy, sparse or dynamic data. Here, we introduce an online learning method empowered by optimizer-driven sampling that has two advantages over current approaches: it ensures that all local extrema (including endpoints) of the model response surface are included in the training data, and it employs a continuous validation and update process in which surrogates undergo retraining when their performance falls below a validity threshold. We find, using benchmark functions, that optimizer-directed sampling generally outperforms traditional sampling methods in terms of accuracy around local extrema even when the scoring metric is biased towards assessing overall accuracy. Finally, the application to dense nuclear matter demonstrates that highly accurate surrogates for a nuclear equation-of-state model can be reliably autogenerated from expensive calculations using few model evaluations.

79 ASTRONOMY AND ASTROPHYSICS↗

WPEC SG50: Developing an Automatically Readable, Comprehensive and Curated Experimental Nuclear Reaction Database

The Organisation for Economic Co-operation and Development (OECD) Nuclear Energy Agency (NEA) Working Party on International Nuclear Data Evaluation Co-operation (WPEC) subgroup (SG) 50 was formed in 2020 to develop an automatically readable, comprehensive and curated experimental nuclear reaction database. This database is called MEDUSAL (Machine-readable Experimental Data User Application & Library), and will draw from EXFOR. The EXFOR database preserves experimental nuclear reaction data true to its original documentation and information from the authors of the data. MEDUSAL will deviate from EXFOR by storing additional information from users of the data for their fields of work (evaluation, model development, validation, etc.). This includes expert judgment on the data sets, identification of data points as outliers, renormalization of the data to the newest monitor reactions, and estimations of missing uncertainty sources. The format for MEDUSAL is being developed to enable easy automatic parsing of large amounts of data. Here, we will summarize the use cases, high-level requirements and first steps towards developing the database MEDUSAL and the API to access it.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Image-driven discriminative and generative machine learning algorithms for establishing microstructure–processing relationships

We investigate methods of microstructure representation for the purpose of predicting processing condition from microstructure image data. A binary alloy that is currently under development as a nuclear fuel was studied for the purpose of developing an improved machine learning approach to image recognition, characterization, and building predictive capabilities linking microstructure to processing conditions. Here, we test different microstructure representations and evaluate model performance based on classification accuracy. A classification accuracy of 95.8% was achieved fordistinguishing between micrographs corresponding to ten different thermo-mechanical material processing conditions.We find that our newly developed microstructure representation describes image data well, and the traditional approachof utilizing area fractions of different phases is insufficient for distinguishing between multiple classes using a relativelysmall, imbalanced original data set of 272 images. To explore the applicability of generative methods for supplementing such limited data sets, generative adversarial networks were trained to generate artificial microstructure images. Two different generative networks were trained and tested to assess performance. Challenges and best practices associated with applying machine learning to limited microstructure image data sets is also discussed. Our work has implications for quantitative microstructure analysis, and development of microstructure-processing relationships in limited data sets typical of metallurgical process design studies.

36 MATERIALS SCIENCE↗

Interconnection Assessment Methodology and Cost Benefit Analysis for High-Penetration PV Deployment in the Arizona Public Service System

In an effort to better understand the impacts of high penetrations of photovoltaic (PV) generators on distribution systems, Arizona Public Service and its partners completed a multi-year project to develop the tools and knowledge base needed to safely and reliably integrate high penetrations of utility- and residential-scale PV. Building upon the APS Community Power Project-Flagstaff Pilot, this project investigates the impact of PV on a representative feeder in northeast Flagstaff. To quantify and catalog the effects of the estimated 1.3 MW of PV that will be installed on the feeder (both smaller units at homes and large, centrally located systems), high-speed weather and electrical data acquisition systems and digital 'smart' meters were designed and installed to facilitate monitoring and to build and validate comprehensive, high-resolution models of the distribution system. These models are being developed to analyze the impacts of PV on distribution circuit protection systems (including coordination and anti-islanding), predict voltage regulation and phase balance issues, and develop volt/VAr control schemes. This paper continues from a paper presented at the 2014 IEEE PVSC conference that described feeder model evaluation and high penetration advanced scenario analysis, specifically feeder reconfiguration. This paper presents results from Phase 5 of the project. Specifically, the paper discusses tool automation; interconnection assessment methodology and cost benefit analysis.

14 SOLAR ENERGY↗

The Cost and Benefit of Enhancing Cybersecurity for Hybrid AC/DC Grids

As critical interfaces of AC grids and DC grids inside a hybrid AC/DC grid, the voltage-sourced-converter (VSC) has been demonstrated to be vulnerable to false data injection (FDI) cyber-attacks. As a result, the cyber-attack-induced AC grid frequency deviations and DC grid voltage deviations threaten the secure operation. Here, to enhance cybersecurity in a not only feasible but also cost-effective manner, this paper proposes a cost-benefit-based cyber-defense strategy for a hybrid AC/DC grid. First, this paper establishes a spatial-temporal dual cyber-attack evaluation model, in which the cyber-attack-induced frequency and voltage deviations are modelled in both a spatially and temporally dual manner. Then, the proposed cost-benefit-based cyber-defense strategy is modelled as a VSC commitment problem to achieve the trade-off between maximizing the cyber-defense benefits and minimizing the cyber-defense costs. The VSC commitment problem is then mathematically convexified into a mixed-integer second-order cone programming (MISOCP) problem, which could be efficiently solved in an event-triggered manner against unfolding cyber-attack events. Simulation results on a test hybrid AC/DC grid verified the feasibility and the cost-effectiveness of the proposed cyber-defense strategy.

cost-benefit analysis↗

Climate Extremes and Variability Surrounding Chesapeake Bay: Past, Present, and Future

Measures of extreme climate variability around Chesapeake Bay have changed over the past century (1895–2014), resulting in the need to establish new baselines for understanding future change. Here, observed climate variability is compared with Coupled Model Intercomparison Project fifth assessment climate models to evaluate ensemble model skill in this region. Observed trends include annual and seasonal declines in the percentage of cold days. Similarly, increases in the annual and seasonal percentages of warm days occurred in all seasons of the North Chesapeake (>38.2°N) but were only significant in spring and summer in the South (<38.2°N) demonstrating regional differences even at local scales. Precipitation intensity increased over the past century. Models using emissions scenarios RCP4.5 and 8.5 project these trends will continue, though they have little skill in precipitation extremes. Pacific and Atlantic climate modes are generally more correlated with climate extremes than mean temperature and precipitation suggesting potential predictability in the synoptic patterns underlying the extremes. However, they also drive the need for time series at least 60-years long for establishing climate trends. This local analysis differs from regional, state level, and local 2.5° × 3.75° grid analyses, highlighting the importance of local climate assessments that consider topographic and regional weather patterns.

54 ENVIRONMENTAL SCIENCES↗

Carbon dynamics of mature and regrowth tropical forests derived from a pantropical database ( T rop F or C ‐db)

Abstract Tropical forests play a critical role in the global carbon (C) cycle, storing ~45% of terrestrial C and constituting the largest component of the terrestrial C sink. Despite their central importance to the global C cycle, their ecosystem‐level C cycles are not as well‐characterized as those of extra‐tropical forests, and knowledge gaps hamper efforts to quantify C budgets across the tropics and to model tropical forest‐climate interactions. To advance understanding of C dynamics of pantropical forests, we compiled a new database, the Tropical Forest C database (TropForC‐db), which contains data on ground‐based measurements of ecosystem‐level C stocks and annual fluxes along with disturbance history. This database currently contains 3568 records from 845 plots in 178 geographically distinct areas, making it the largest and most comprehensive database of its type. Using TropForC‐db, we characterized C stocks and fluxes for young, intermediate‐aged, and mature forests. Relative to existing C budgets of extra‐tropical forests, mature tropical broadleaf evergreen forests had substantially higher gross primary productivity ( GPP ) and ecosystem respiration (R eco ), their autotropic respiration (R a ) consumed a larger proportion (~67%) of GPP , and their woody stem growth ( ANPP stem ) represented a smaller proportion of net primary productivity ( NPP , ~32%) or GPP (~9%). In regrowth stands, aboveground biomass increased rapidly during the first 20 years following stand‐clearing disturbance, with slower accumulation following agriculture and in deciduous forests, and continued to accumulate at a slower pace in forests aged 20–100 years. Most other C stocks likewise increased with stand age, while potential to describe age trends in C fluxes was generally data‐limited. We expect that TropForC‐db will prove useful for model evaluation and for quantifying the contribution of forests to the global C cycle. The database version associated with this publication is archived in Dryad ( DOI : 10.5061/dryad.t516f ) and a dynamic version is maintained at https://github.com/forc-db .

Anderson‐Teixeira, Kristina J.↗

MPAS-Analysis

MPAS-Analysis is a software package designed to process and analyze output data of the Model for Prediction Across Scales (MPAS) Ocean, Sea-Ice and Land-Ice model components. It is built primarily on the python language, with remapping of model unstructured-grid fields onto regular grids performed through the ESMF package, and with the computation of climatologies performed through the NCO package. The main purpose of MPAS-Analysis is to analyze the MPAS simulations in terms of ocean and ice metrics, in comparison with available observations, and therefore provide an accurate model evaluation and simulation assessment.

Veneziani, Milena↗

Graphene U-Net v1

Graphene U-Net is a library that provides a simplified platform for training deep neural networks for the task of microscopy image segmentation. It contains functions and classes that make the process of training a neural network simple such that non-ML experts can train and evaluate models on their own datasets. It uses Pytorch as a backend and can run on both CUDA-enable GPUs and CPUs. It contains the main library file Microscopy_Unet.py as well as unet.py which contains the UNET model used for this software. This can be replaced with any other fully convolutional deep learning architecture with relative ease.

Sadre, Robbie↗

PV DMFA [SWR-21-105]

The Photovoltaic Dynamic Material Flow Assessment (PV DMFA) model (also referred to here as “The model”) is a computational framework written in Python based on utility-scale PV electricity generation to quantify time-series stocks and flows of PV materials primarily in crystalline silicon PV technologies. The model evaluates cradle-to-cradle life cycle of utility-scale solar PV systems in the United States in the period 2000-2100. PV DMFA serves as a sustainability analysis tool to assess the impacts of different material circularity practices (i.e., reduce, reuse/refurbish, remanufacture, and recycle), PV module design shifts and sensitivity of material processing and technology related parameters to material installations, waste creation and raw material depletion in PV material supply chains. This tool enables advanced planning for future material needs and informs sustainable pathways for PV material management in the circular economy. This tool could be helpful to a wide range of stakeholders; Particularly, researchers and manufacturers looking for technoeconomic and/or environmental life cycle analysis (LCA) feedback for renewable energy (RE) systems.

Khalifa, SherifA.↗

Wetlands Geoprocessor (v 1.0)

Wetlands have often been described as the “Earth’s kidneys” because wetlands have the ability filter pollutants and purify water. Based upon wetland storage capacity, average daily flow, and inflow nutrient loading this model estimates potential load reductions of nitrate and orthophosphate) from a constructed surface flow wetland (CW). The model relies upon equations, coefficients, and models provided by Cheng et al. 2020 and Cheng & Basu 2017. Nitrogen (N) and phosphorus are important nutrients to ecosystems, but high concentrations can lead to water quality issues. Major sources of these nutrients come from agricultural areas where fertilizer is applied in large quantitates and/or animal waste is present. Excess nitrate and phosphorus may result in overgrowth of algae, which can decrease the dissolved oxygen in water, thereby harming and killing fish and other aquatic species. This model evaluates water purification results and estimates cost to install a surface flow constructed wetlands.

Davis, Cleve↗

Toadstool Deep Learning Framework

SAND2025-11741O Toadstool is a deep learning framework and support library that provides PyTorch boiler plate training and testing loops. This enables the user to remember parts and customize a callback interface. Toadstool also provides useful callbacks and other methods for deep learning experimentation. The framework also implements publicly available temperature and calibration methods, model initialization methods, learning rate schedulers, and model evaluation methods. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Heidbrink, Scott↗

The DOE ARM Aerial Facility

The Department of Energy Atmospheric Radiation Measurement (ARM) Program is a climate research user facility operating stationary ground sites that provide long-term measurements of climate relevant properties, mobile ground- and ship-based facilities to conduct shorter field campaigns (6-12 months), and the ARM Aerial Facility (AAF). The airborne observations acquired by the AAF enhance the surface-based ARM measurements by providing high-resolution in-situ measurements for process understanding, retrieval-algorithm development, and model evaluation that are not possible using ground- or satellite-based techniques. Several ARM aerial efforts were consolidated into the AAF in 2006. With the exception of a small aircraft used for routine measurements of aerosols and carbon cycle gases, AAF at the time had no dedicated aircraft and only a small number of instruments at its disposal. In this "virtual hangar" mode, AAF successfully carried out several missions contracting with organizations and investigators who provided their research aircraft and instrumentation. In 2009, AAF started managing operations of the Battelle-owned Gulfstream I (G-1) large twin-turboprop research aircraft. Furthermore, the American Recovery and Reinvestment Act of 2009 provided funding for the procurement of over twenty new instruments to be used aboard the G-1 and other AAF virtual-hangar aircraft. AAF now executes missions in the virtual- and real-hangar mode producing freely available datasets for studying aerosol, cloud, and radiative processes in the atmosphere. AAF is also engaged in the maturation and testing of newly developed airborne sensors to help foster the next generation of airborne instruments.

aerial↗

Are Atmospheric Models Too Cold in the Mountains? The State of Science and Insights from the SAIL Field Campaign

Mountains play an outsized role in water resource availability, and the amount and timing of water they provide depend strongly on temperature. To that end, we ask the question: How well are atmospheric models capturing mountain temperatures? We synthesize results showing that high-resolution, regionally relevant climate models produce 2-m air temperature (T2m) measurements colder than what is observed (a “cold bias”), particularly in snow-covered midlatitude mountain ranges during winter. We find common cold biases in 44 studies across global mountain ranges, including single-model and multimodel ensembles. We explore the factors driving these biases and examine the physical mechanisms, data limitations, and observational uncertainties behind T2m. Our analysis suggests that the biases are genuine and not due to observation sparsity or resolution mismatches. Cold biases occur primarily on mountain peaks and ridges, whereas valleys are often warm biased. Our literature review suggests that increasing model resolution does not clearly mitigate the bias. By analyzing data from the Surface Atmosphere Integrated Field Laboratory (SAIL) field campaign in the Colorado Rocky Mountains, we test various hypotheses related to cold biases and find that local wind circulations, longwave (LW) radiation, and surface-layer parameterizations contribute to the T2m biases in this particular location. We conclude by emphasizing the value of coordinated model evaluation and development efforts in heavily instrumented mountain locations for addressing the root cause(s) of T2m biases and improving predictive understanding of mountain climates.

54 ENVIRONMENTAL SCIENCES↗

Turbulence and Cloudiness in Cumulus-Topped Marine Boundary Layers

Turbulence and cloudiness in cumulus-topped marine boundary layers are studied using data from the Atmospheric Radiation Measurement (ARM) Eastern North Atlantic (ENA) site. The analysis includes eight periods of nonprecipitating shallow cumulus clouds, spanning 141 h and encompassing 603 individual clouds. On average, the cumulus had bases at 558 m, were 99 m thick, had a chord length of 500 m, and exhibited an hourly base-layer cloudiness of 12%. Changes in cloud fraction were primarily driven by variations in cloud number not chord length. High-resolution Doppler lidar and cloud radar observations were used to estimate updraft and downdraft mass fluxes in the cumulus base layer under both clear and cloudy regions. Cloudy updrafts contributed only ∼23% of the total updraft mass flux at cloud base, indicating that most upward mass flux originated from clear-air eddies. Updraft strength, rather than updraft fraction, was found to predominantly control both clear and cloudy updraft mass fluxes. Cloud-base cloudiness strongly correlated with cloudy updraft mass flux but showed negligible correlation with clear-air updraft mass flux. Clear-air downdraft mass flux exhibited a strong relationship with the ratio of surface buoyancy to inversion strength. Mesoscale analysis revealed that moist patches had greater cloudiness, updraft mass flux, and vertical velocity variance compared to dry patches. Additionally, mesoscale base-layer cloud fraction was highly and significantly correlated with cloudy updraft mass flux in both dry and moist environments. Results presented herein have implications for cumulus parameterization development along with climatological and model evaluation studies conducted at the ENA site.

cumulus clouds↗

Climatological Characteristics of Typical Daily Precipitation

Precipitation is often quantified by the amount that falls over a given period of time but not the rate at which most of it falls or the rate associated with the most frequent events. Here, three metrics are introduced to distill salient characteristics of typical daily precipitation accumulation based on the full distribution of rainfall: rain amount peak (the rain rate at which the most rain falls), rain frequency peak (the most frequent nonzero rain rate), and rain amount width (a measure of the variability of typical precipitation accumulation). These metrics are applied to two observational datasets to describe the climatology of typical daily precipitation accumulation: GPCP 1° daily (October 1996–October 2015) and TMPA 3B42 (January 1998–October 2015). Results show that the rain frequency peak is similar to total rainfall in terms of geographical pattern and seasonal cycle and varies inversely with rain amount width. In contrast, the rain amount peak varies distinctly, reaching maxima on the outer edges of the regions of high total precipitation, and with less seasonal variation. Despite that GPCP and TMPA 3B42 are both merged satellite–gauge precipitation products, they show substantial differences. In particular, the rain amount peak and rain amount width are uniformly greater in TMPA 3B42 compared to GPCP, and there are large discrepancies in their rain frequency distributions (peak and width). Issues relating to model evaluation are highlighted using CESM1 as an illustrative example and underscore the need for observational datasets incorporating measurements of light rain.

Meteorology & Atmospheric Sciences↗