Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model benchmarking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 937 records · Page 52

Neutronic-Thermal Hydraulic Benchmark using the SKINATH Point Kinetics Model

The SKINATH point kinetics program, whose name comes from: A Computer Program for Solving the Reactor Point Kinetics Equations with Simple Thermal Hydraulic-Feedback, had its origins at ORNL. The authors were interested in investigating low-level criticality events such as might occur in a waste repository of 55 gallon drums. We assume a relatively realistic thermal feedback for fissionable material in a container (called a reactor here), either critical or subcritical, with the potential to undergo a further excursion. The system is a homogeneous fissile fluid (mostly water), increasing in temperature as it absorbs all heat from the excursion. An ideal gas eternally cools the reactor at constant temperature T{sub c}. The SKINATH numerical concept in 1984 was to use an off-the-shelf Livermore ODE solver LSODE to solve the kinetic equations, much as one does today with MATLAB{sup TM}. Here, we develop a novel solver that uses the simplest of numerical methods including convergence acceleration. This work represents the initiation of a Multiphysics benchmark effort at INL. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Tracking vegetation phenology across diverse North American biomes using PhenoCam imagery

Vegetation phenology controls the seasonality of many ecosystem processes, as well as numerous biosphere-atmosphere feedbacks. Phenology is also highly sensitive to climate change and variability. Here we present a series of datasets, together consisting of almost 750 years of observations, characterizing vegetation phenology in diverse ecosystems across North America. Our data are derived from conventional, visible-wavelength, automated digital camera imagery collected through the PhenoCam network. For each archived image, we extracted RGB (red, green, blue) colour channel information, with means and other statistics calculated across a region-of-interest (ROI) delineating a specific vegetation type. From the high-frequency (typically, 30 min) imagery, we derived time series characterizing vegetation colour, including "canopy greenness", processed to 1- and 3-day intervals. For ecosystems with one or more annual cycles of vegetation activity, we provide estimates, with uncertainties, for the start of the "greenness rising" and end of the "greenness falling" stages. Lastly, the database can be used for phenological model validation and development, evaluation of satellite remote sensing data products, benchmarking earth system models, and studies of climate change impacts on terrestrial ecosystems.

54 ENVIRONMENTAL SCIENCES↗

PowerModelsGAT-AI: Physics-Informed Graph Attention for Multi-System Power Flow With Continual Learning

Solving the alternating current power flow equations in real time is essential for secure grid operation, yet classical Newton–Raphson solvers can be slow under stressed conditions. Existing graph neural networks for power flow are typically trained on a single system and often degrade on different systems. We present PowerModelsGAT-AI, a physics-informed graph attention network that predicts bus voltages and generator injections. The model uses bus-type-aware masking to handle different bus types and balances multiple loss terms, including a power-mismatch penalty, using learned weights. We evaluate the model on 14 benchmark systems (4 to 6,470 buses) and train a unified model on 13 of these under contingency conditions with up to two branch outages, achieving an average normalized mean absolute error of 0.89% for voltage magnitudes and R 2 >0.99 for voltage angles. We also show continual learning: when adapting a base model to a new 1,354-bus system, standard fine-tuning causes severe forgetting with error increases exceeding 1000% on base systems, while our experience replay and elastic weight consolidation strategy keeps error increases below 2% and in some cases improves base-system performance. Interpretability analysis shows that learned attention weights correlate with physical branch parameters (susceptance: r=0.38 ; thermal limits: r=0.22 ), and feature importance analysis supports that the model captures established power flow relationships.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Evaluating and Quantifying the Climate-Driven Interannual Variability in Global Inventory Modeling and Mapping Studies (GIMMS) Normalized Difference Vegetation Index (NDVI3g) at Global Scales

Satellite observations of surface reflected solar radiation contain informationabout variability in the absorption of solar radiation by vegetation. Understanding thecauses of variability is important for models that use these data to drive land surface fluxesor for benchmarking prognostic vegetation models. Here we evaluated the interannualvariability in the new 30.5-year long global satellite-derived surface reflectance index data,Global Inventory Modeling and Mapping Studies normalized difference vegetation index(GIMMS NDVI3g). Pearsons correlation and multiple linear stepwise regression analyseswere applied to quantify the NDVI interannual variability driven by climate anomalies, andto evaluate the effects of potential interference (snow, aerosols and clouds) on the NDVIsignal. We found ecologically plausible strong controls on NDVI variability by antecedent precipitation and current monthly temperature with distinct spatial patterns. Precipitation correlations were strongest for temperate to tropical water limited herbaceous systemswhere in some regions and seasons 40 of the NDVI variance could be explained byprecipitation anomalies. Temperature correlations were strongest in northern mid- to-high-latitudes in the spring and early summer where up to 70 of the NDVI variance was explained by temperature anomalies. We find that, in western and central North America,winter-spring precipitation determines early summer growth while more recent precipitation controls NDVI variability in late summer. In contrast, current or prior wetseason precipitation anomalies were correlated with all months of NDVI in sub-tropical herbaceous vegetation. Snow, aerosols and clouds as well as unexplained phenomena still account for part of the NDVI variance despite corrections. Nevertheless, this study demonstrates that GIMMS NDVI3g represents real responses of vegetation to climate variability that are useful for global models.

interference↗

Using Hybrid Model OpenSHMEM + CUDA to Implement the SHOC Benchmark Suite

This work describes the process of porting the Scalable HeterOgeneous Computing (SHOC) benchmark suite from the hybrid MPI + CUDA implementation to OpenSHMEM + CUDA. SHOC includes a wide variety of benchmark kernels used to measure accelerator performance in both single node and cluster configurations. The hybrid model implementation attempts to place all major computation on accelerator devices, and uses MPI to synchronize and aggregate results. In some cases, MPI Groups are used to gradually reduce the number of accelerators used for computation as the problem size drops. Porting this behavior to OpenSHMEM required implementing several synchronizing collective operations, and using SHMEM teams to replace MPI Group functionality. Benchmark results on a Cray XK7 system with one GPU per compute node show that SHMEM performance is equal to MPI performance in these hybrid tasks. These results and porting experience show that using OpenSHMEM for accelerator devices benefits from adding functionality for synchronization and teams, and would further benefit from adding support for communication within accelerator kernels.

97 MATHEMATICS AND COMPUTING↗

Using Information Theory to Evaluate Directional Precipitation Interactions Over the West Sahel Region in Observations and Models

Water availability has historically been one of the most significant threats to African regional social and economic well-being. Over the Sahel region, a megadrought during the 1960s and 1970s induced by an abrupt and substantial rainfall reduction caused widespread famine and death. The postdrought recovery, which is still ongoing, has been characterized by gradual increases in rainfall, but with dramatic fluctuations. The large negative human impacts, slow recovery, and variability raise important questions of why and how rainfall dynamics evolve and interact with other components of the regional climate system. Here we provide an observational assessment of rainfall interactions mechanisms (informed by directional transfer of information entropy) that regulate Sahel rainfall. We quantitatively demonstrate that (1) sea surface temperature over the Gulf of Guinea controls moisture advection and transport to the West Sahel region and (2) strong bidirectional interactions exist between local vegetation dynamics and rainfall patterns. Then we assess the directional interaction patterns from nine state-of-the-art Earth System Models (ESMs). We find that most ESMs are able to represent either the unidirectional control of sea surface temperature on precipitation or the bidirectional interaction between vegetation and precipitation. However, none of the ESMs represents both interactive patterns. The GFDL and IPSL-CM5A-LR models successfully reproduced observed patterns over ~50% of the West Sahel region but were not accurate in reproducing observed precipitation regional trends or interannual variation. We propose that the directional information transfer is a powerful mechanistic benchmark to assess model fidelity at the process level.

54 ENVIRONMENTAL SCIENCES↗

Deforestation reshapes land-surface energy-flux partitioning

Land-use and land-cover change significantly modify local land-surface characteristics and water/energy exchanges, which can lead to atmospheric circulation and regional climate changes. In particular, deforestation accounts for a large portion of global land-use changes, which transforms forests into other land cover types, such as croplands and grazing lands. Many previous efforts have focused on observing and modeling land–atmosphere–water/energy fluxes to investigate land–atmosphere coupling induced by deforestation. However, interpreting land–atmosphere–water/energy-flux responses to deforestation is often complicated by the concurrent impacts from shifts in land-surface properties versus background atmospheric forcings. In this study, we used 29 paired FLUXNET sites, to improve understanding of how deforested land surfaces drive changes in surface-energy-flux partitioning. Each paired sites included an intact forested and non-forested site that had similar background climate. We employed transfer entropy, a method based on information theory, to diagnose directional controls between coupling variables, and identify nonlinear cause–effect relationships. Transfer entropy is a powerful tool to detective causal relationships in nonlinear and asynchronous systems. The paired eddy covariance flux measurements showed consistent and strong information flows from vegetation activity (gross primary productivity (GPP)) and physical climate (e.g. shortwave radiation, air temperature) to evaporative fraction (EF) over both non-forested and forested land surfaces. More importantly, the information transfers from radiation, precipitation, and GPP to EF were significantly reduced at non-forested sites, compared to forested sites. We then applied these observationally constrained metrics as benchmarks to evaluate the Energy Exascale Earth System Model (E3SM) land model (ELM). ELM predicted vegetation controls on EF relatively well, but underpredicted climate factors on EF, indicating model deficiencies in describing the relationships between atmospheric state and surface fluxes. Moreover, changes in controls on surface energy flux partitioning due to deforestation were not detected in the model. We highlight the need for benchmarking model simulated surface-energy fluxes and the corresponding causal relationships against those of observations, to improve our understanding of model predictability on how deforestation reshapes land surface energy fluxes.

54 ENVIRONMENTAL SCIENCES↗

Demography, dynamics and data: building confidence for simulating changes in the world's forests

Vegetation demographic models (VDMs) are advanced tools for simulating forest responses to climate and land-use changes, and are essential for projecting carbon cycling and large-scale forest management strategies. Despite their increasing incorporation into Earth System Models, VDMs differ in their demographic assumptions, with no prior quantitative comparison of their performance. We benchmarked nine VDMs against observational data from boreal, temperate and tropical sites, assessing their accuracy in predicting tree growth, carbon turnover, biomass stocks and size distributions. Models were simulated under consistent climate conditions with postdisturbance recovery monitored for at least 420 yr. Postdisturbance carbon recovery trajectories showed significant variability while remaining within observational ranges. Initial regrowth rates varied substantially (0.03-0.60, 0.18-0.70 and 0.35-1.10 kgCm-2 yr-1 for boreal, temperate and tropical sites, respectively), influenced by each model's initial forest state. Models captured mature forest carbon content but showed compensating effects between overestimated growth and underestimated mortality rates. This first multi-model benchmarking identifies growth and mortality rates as critical calibration targets and highlights the need to refine postdisturbance establishment conditions for model development. We outline specific benchmarking variables needed to improve predictions of forest responses to environmental change.

demographic vegetation model benchmarking↗

Utilizing ensemble learning for performance and power modeling and improvement of parallel cancer deep learning CANDLE benchmarks

Abstract Machine learning (ML) continues to grow in importance across nearly all domains in modeling to learn from data. Often a tradeoff exists between a model's ability to minimize bias and variance. In this article, we utilize ensemble learning to combine linear, nonlinear, and tree‐/rule‐based ML methods to cope with the bias‐variance tradeoff and result in more accurate models. We use the datasets collected for two parallel cancer deep learning CANDLE benchmarks, NT3 and P1B2, to build performance and power models based on hardware performance counters using single‐object and multiple‐objects ensemble learning to identify the most important counters for improvement on the Cray XC40 Theta at Argonne National Laboratory. Based on the insights from these models, we improve the performance and energy of P1B2 and NT3 by optimizing the deep learning environments TensorFlow, Keras, Horovod, and Python under the huge page size of 8 MB. Experimental results show that ensemble learning not only produces more accurate models but also provides more robust performance counter ranking. We achieve up to 61.15% performance improvement and up to 62.58% energy saving for P1B2 and up to 55.81% performance improvement and up to 52.60% energy saving for NT3 on up to 24,576 cores.

Wu, Xingfu↗

PERFORMANCE ANALYSIS OF MECHANICAL DRAFT COOLING TOWER

Industrial processes use mechanical draft cooling towers (MDCT's) to dissipate waste heat by transferring heat from water to air via evaporative cooling, which causes air humidification. The Savannah River Site (SRS) has cross-flow and counter-current MDCT's consisting of four independent compartments called cells. Each cell has its own fan to help maximize heat transfer between ambient air and circulated water. The primary objective of the work is to simulate the cooling tower performance for the counter-current cooling tower and to conduct a parametric study under different fan speeds and ambient air conditions. The Savannah River National Laboratory (SRNL) developed a computational fluid dynamics (CFD) model and performed the benchmarking analysis against the integral measurement results to accomplish the objective. The model uses three-dimensional steady-state momentum, continuity equations, air-vapor species balance equation, and two-equation turbulence as the basic governing equations. It was assumed that vapor phase is always transported by the continuous air phase with no slip velocity. In this case, water droplet component was considered as discrete phase for the interfacial heat and mass transfer via Lagrangian approach. Thus, the air-vapor mixture model with discrete water droplet phase is used for the analysis. A series of parametric calculations was performed to investigate the impact of wind speeds and ambient conditions on the thermal performance of the cooling tower when fans were operating and when they were turned off. The model was also benchmarked against the literature data and the SRS integral test results for key parameters such as air temperature and humidity at the tower exit and water temperature for given ambient conditions. Detailed results will be published here.

42 ENGINEERING↗

Neutron Capture Gamma-Ray Libraries for Nuclear Applications

The neutron capture reaction is useful in identifying and analyzing the gamma-ray spectrum from an unknown assembly as it gives unambiguous information on its composition. This can be done passively or actively where an external neutron source is used to probe an unknown assembly. There are known capture gamma-ray data gaps in the ENDF libraries used by transport codes for various nuclear applications. The Evaluated Gamma-ray Activation file (EGAF) is a new thermal neutron capture database of discrete line spectra and cross sections for over 260 isotopes that was developed as part of an IAEA Coordinated Research Project. EGAF is being used to improve the capture gamma production in ENDF libraries. For medium to heavy nuclei the quasi continuum contribution to the gamma cascades is not experimentally resolved. The continuum contains up to 90% of all the decay energy and is modeled here with the statistical nuclear structure code DICEBOX. This code also provides a consistency check of the level scheme nuclear structure evaluation. The calculated continuum is of sufficient accuracy to include in the ENDF libraries. This analysis also determines new total thermal capture cross sections and provides an improved RIPL database. For higher energy neutron capture there is less experimental data available making benchmarking of the modeling codes more difficult. We are investigating the capture spectra from higher energy neutrons experimentally using surrogate reactions and modeling this with Hauser-Feshbach codes. This can then be used to benchmark CASINO, a version of DICEBOX modified for neutron capture at higher energy. This can be used to simulate spectra from neutron capture at incident neutron energies up to 20 MeV to improve the gamma-ray spectrum in neutron data libraries used for transport modeling of unknown assemblies.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

A Code to Code Benchmark for HTGR Fuel Element Depletion

The current lack of well-qualified experimental measurements for spent HGTR fuel elements limits the validation of advanced doubly heterogeneous depletion methods. Because of this shortage of data, a benchmark was developed as the first, simplest phase in a planned series of increasingly complex set of code-to-code benchmarks. The intent of this benchmark was to encourage submission of a wide range of computational results for depletion calculations in a set of basic fuel cell models. Comparison of results using independent methods and data should provide insight into potential limitations in various modeling approximations. The benchmark seeks to provide the simplest possible models, in order to minimize the effect of competing and potentially offsetting phenomena that might mask weaknesses in given methods. Twenty-one sets of results were submitted by twelve participants internationally. Various approaches were used for the treatment of microscopic grains dispersed in fuel elements in the second and third parts of the benchmark, including explicit treatment of grains distributed within a regular lattice, a randomized distribution of explicit grains, reactivity-equivalent physical transformation and a two-pass homogenization with disadvantage factors. The broad agreement appears to validate each of the double heterogeneity treatments.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Search for high-mass resonances decaying into τ-lepton pairs in pp collisions at s = 7 TeV

A search for high-mass resonances decaying into tau-lepton pairs is performed using a data sample of pp collisions at $ \sqrt{s}=7 $ TeV. The data were collected with the CMS detector at the LHC and correspond to an integrated luminosity of 4.9 inverse femtobarns. The number of observed events is in agreement with the standard model prediction. An upper limit on the product of the resonance cross section and branching fraction into tau-lepton pairs is calculated as a function of the resonance mass. Using the sequential standard model resonance Z'(SSM) and the superstring-inspired E(6) model with resonance Z'(psi) as benchmarks, resonances with standard model couplings with masses below 1.4 and 1.1 TeV, respectively, are excluded at 95% confidence level.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Machine Learning for Materials Scientists: An Introductory Guide toward Best Practices

This Methods/Protocols article is intended for materials scientists interested in performing machine learning-centered research. Herein, we cover broad guidelines and best practices regarding the obtaining and treatment of data, feature engineering, model training, validation, evaluation and comparison, popular repositories for materials data and benchmarking data sets, model and architecture sharing, and finally publication. In addition, we include interactive Jupyter notebooks with example Python code to demonstrate some of the concepts, workflows, and best practices discussed. Overall, the data-driven methods and machine learning workflows and considerations are presented in a simple way, allowing interested readers to more intelligently guide their machine learning research using the suggested references, best practices, and their own materials domain expertise.

36 MATERIALS SCIENCE↗

Benchmarking Simulated Precipitation in Earth System Models

Earth system models (ESMs) bridge observationally based and theoretical understanding of the Earth system. They are among the most often used tools to study a variety of questions related to variability and changes in Earth’s climate. For many applications, ESMs must realistically simulate observed large-scale precipitation patterns and seasonal cycles that have a multitude of societal and national security implications. Despite steady improvement in the simulation of precipitation, model errors in many aspects of precipitation characteristics limit the use of ESMs both in understanding Earth system variability and change and for decision-making.

54 ENVIRONMENTAL SCIENCES↗

Handbook of LHC Higgs Cross Sections: 4. Deciphering the Nature of the Higgs Sector

This Report summarizes the results of the activities of the LHC Higgs Cross Section Working Group in the period 2014-2016. The main goal of the working group was to present the state-of-the-art of Higgs physics at the LHC, integrating all new results that have appeared in the last few years. The first part compiles the most up-to-date predictions of Higgs boson production cross sections and decay branching ratios, parton distribution functions, and off-shell Higgs boson production and interference effects. The second part discusses the recent progress in Higgs effective field theory predictions, followed by the third part on pseudo-observables, simplified template cross section and fiducial cross section measurements, which give the baseline framework for Higgs boson property measurements. The fourth part deals with the beyond the Standard Model predictions of various benchmark scenarios of Minimal Supersymmetric Standard Model, extended scalar sector, Next-to-Minimal Supersymmetric Standard Model and exotic Higgs boson decays. This report follows three previous working-group reports: Handbook of LHC Higgs Cross Sections: 1. Inclusive Observables (CERN-2011-002), Handbook of LHC Higgs Cross Sections: 2. Differential Distributions (CERN-2012-002), and Handbook of LHC Higgs Cross Sections: 3. Higgs properties (CERN-2013-004). The current report serves as the baseline reference for Higgs physics in LHC Run 2 and beyond.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗