Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model benchmarking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 847 records · Page 47

WE-G-BRD-09: Novel MRI Compatible Electron Accelerator for MRI-Linac Radiotherapy

Purpose: MRI guided radiotherapy is a rapidly growing field; however current linacs are not designed to operate in MRI fringe fields. As such, current MRI- Linac systems require magnetic shielding, impairing MR image quality and system flexibility. Here, we present a bespoke electron accelerator concept with robust operation in in-line magnetic fields. Methods: For in-line MRI-Linac systems, electron gun performance is the major constraint on accelerator performance. To overcome this, we propose placing a cathode directly within the first accelerating cavity. Such a configuration is used extensively in high energy particle physics, but not previously for radiotherapy. Benchmarked computational modelling (CST, Darmstadt, Germany) was employed to design and assess a 5.5 cell side coupled accelerator with a temperature limited thermionic cathode in the first accelerating cell. This simulation was coupled to magnetic fields from a 1T MRI model to assess robustness in magnetic fields for Source to Isocenter Distance between 1 and 2 meters. Performance was compared to a conventional electron gun based system in the same magnetic field. Results: A temperature limited cathode (work function 1.8eV, temperature 1245K, emission constant 60A/K/cm{sup 2}) will emit a mean current density of 24mA/mm{sup 2} (Richardson’s Law). We modeled a circular cathode with radius 2mm and mean current 300mA. Capture efficiency of the device was 43%, resulting in target current of 130 mA. The electron beam had a FWHM of 0.2mm, and mean energy of 5.9MeV (interquartile spread of 0.1MeV). Such an electron beam is suitable for radiotherapy, comparing favourably to conventional systems. This model was robust to operation the MRI fringe field, with a maximum current loss of 6% compared to 85% for the conventional system. Conclusion: The bespoke electron accelerator is robust to operation in in-line magnetic fields. This will enable MRI-Linacs with no accelerator magnetic shielding, and minimise painstaking optimisation of the MRI fringe field. This work was supported by US (NIH) and Australian (NHMRC & Cancer Institute NSW) government research funding. In addition, I would like to thank cancer institute NSW and the Ingham Institute for scholarship support.

60 APPLIED LIFE SCIENCES↗

Semileptonic (lepton, neutrino and jets) WW/WZ resonances searches at √s = 8 and 13 TeV with the ATLAS detector at the LHC

This talk presents the analyses results of the diboson (WW or WZ ) resonances production search in pp collisions at √s = 8 and 13 TeV with the ATLAS detector at the LHC with the semileptonic final state. As benchmark signal models Randall-Sundrum bulk model for KK G* → WW and Heavy Vector Triplet model for W′ → WZ and Z′ → WW are used. No significant excess for diboson resonances production is observed and upper limits on the production cross section times branching fraction of G*, W′ and Z′ are determined at 95% CL.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Benchmarking Parametric and Machine Learning Models for Genomic Prediction of Complex Traits

The usefulness of genomic prediction in crop and livestock breeding programs has prompted efforts to develop new and improved genomic prediction algorithms, such as artificial neural networks and gradient tree boosting. However, the performance of these algorithms has not been compared in a systematic manner using a wide range of datasets and models. Using data of 18 traits across six plant species with different marker densities and training population sizes, we compared the performance of six linear and six non-linear algorithms. First, we found that hyperparameter selection was necessary for all non-linear algorithms and that feature selection prior to model training was critical for artificial neural networks when the markers greatly outnumbered the number of training lines. Across all species and trait combinations, no one algorithm performed best, however predictions based on a combination of results from multiple algorithms (i.e., ensemble predictions) performed consistently well. While linear and non-linear algorithms performed best for a similar number of traits, the performance of non-linear algorithms vary more between traits. Although artificial neural networks did not perform best for any trait, we identified strategies (i.e., feature selection, seeded starting weights) that boosted their performance to near the level of other algorithms. Our results highlight the importance of algorithm selection for the prediction of trait values.

59 BASIC BIOLOGICAL SCIENCES↗

Evaluation of Seismic Artificial Intelligence with Uncertainty

Artificial intelligence has transformed the seismic community with deep learning models (DLMs) that are trained to complete specific tasks within workflows. However, there is still a lack of robust evaluation frameworks for evaluating and comparing DLMs. Here, we address this gap by designing an evaluation framework that jointly incorporates two crucial aspects: performance uncertainty and learning efficiency. To target these aspects, we meticulously construct the training, validation, and test splits using a clustering method tailored to seismic data and enact an expansive training design to segregate performance uncertainty arising from stochastic training processes and random data sampling. The framework’s ability to guard against misleading declarations of model superiority is demonstrated through the evaluation of PhaseNet (Zhu and Beroza, 2018), a popular seismic phase picking DLM, under three training approaches. Our framework helps practitioners choose the best model for their problem and set performance expectations by explicitly analyzing model performance with uncertainty at varying budgets of training data.

58 GEOSCIENCES↗

Semi-local Density Functional Approximations for Bulk, Surface, and Confinement Physics

Due to its balance of accuracy and computational cost, density functional theory has become the method of choice for computing the electronic structure and related properties of materials. However, present-day semi-local approximations to the exchange-correlation energy of density functional theory break down for materials containing d and f electrons. In this report we summarize the results of our research efforts within the LDRD 200202 titled "Making density functional theory work for all materials" in addressing this issue. Our efforts are grouped into two research thrusts. In the first thrust, we develop an exchange-correlation functional (BSC functional) within the subsystem functional formalism. It enables us to capture bulk, surface, and confinement physics with a single, semi-local exchange-correlation functional in density functional theory calculations. We present the analytical properties of the BSC functional and demonstrate that the BSC functional is able to capture confinement physics more accurately than standard semi-local exchange-correlation functionals. The second research thrust focusses on developing a database for transition metal binary compounds. The database consists of materials properties (formation energies, ground-state energies, lattice constants, and elastic constants) of 26 transition metal elements and 89 transition metal alloys. It serves as a reference for benchmarking computational models (such as lower-level modeling methods and exchange-correlation functionals). We expect that our database will significantly impact the materials science community. We conclude with a brief discussion on the future research directions and impact of our results.

36 MATERIALS SCIENCE↗

ECP Report: Update on Proxy Applications and Vendor Interactions

The ExaLearn miniGAN team (Ellis and Rajamanickam) have released miniGAN, a generative adversarial network(GAN) proxy application, through the ECP proxy application suite. miniGAN is the first machine learning proxy application in the suite (note: the ECP CANDLE project did previously release some benchmarks) and models the performance for training generator and discriminator networks. The GAN's generator and discriminator generate plausible 2D/3D maps and identify fake maps, respectively. miniGAN aims to be a proxy application for related applications in cosmology (CosmoFlow, ExaGAN) and wind energy (ExaWind). miniGAN has been developed so that optimized mathematical kernels (e.g., kernels provided by Kokkos Kernels) can be plugged into to the proxy application to explore potential performance improvements. miniGAN has been released as open source software and is available through the ECP proxy application website (https://proxyapps.exascaleproject.ordecp-proxy-appssuite/) and on GitHub (https://github.com/SandiaMLMiniApps/miniGAN). As part of this release, a generator is provided to generate a data set (series of images) that are inputs to the proxy application.

97 MATHEMATICS AND COMPUTING↗

Development of Gamma Background Radiation Digital Twin with Machine Learning Algorithms: Application of Unsupervised Machine Learning to Detection of Anomalies and Nuisances in Gamma Background Radiation Environmental Screening Data

Environmental screening of gamma radiation consists of detecting weak nuisance and anomaly signal in the presence of strong and highly varying background. In a typical scenario, a mobile detector-spectrometer continuously measures gamma radiation spectra in short, e.g., one-second, signal acquisition intervals. The measurement data is a 2D matrix, where one dimension is gamma ray energy, and the other dimension is the number of measurements or total time. In principle, gamma radiation sources can be detected and identified from the measured data by their unique spectral lines. Detecting sources from data measured in a search scenario is difficult due to the highly varying background because of naturally occurring radioactive material (NORM), and low signal-to-noise ratio (S/N) of spectral signal measured during one-second acquisition intervals. The objective of this work is to explore unsupervised machine learning (ML) algorithms for development of a digital twin of gamma radiation background, and for detection and identification of weak nuisances and anomalies events in the presence of highly fluctuating background. In one segment of work, we developed a gamma background estimation model using a Longshort term memory (LSTM) network for one-step CPS time series prediction. The LSTM model was validated with two data sets of measurements from two independent NaI detectors positioned on a mobile platform. The data sets contained background radiation only and no orphan isotope sources. The LSTM model was constructed and tested using data from one of the detectors. Performance of the LSTM model was validate through one-step prediction of CPS time series of another NaI detector without re-training. This approach allows to create a digital twin for nuclear background estimation. Using LSTM, it could be possible to detect a source through subtraction of the estimated counts from the measured background. In another segment of work, we investigated detection of gamma emitting sources in the presence of complex background using unsupervised machine learning. Spectral lines of isotopes are difficult to observe in one-second measurements. Averaging over the entire measurement campaign data set reveals spectral lines of most common background isotopes. Spectral lines of orphan sources, which might appear only in a few measurements during the campaign, will be washed out if averaging is performed over the entire measurement data set. The approach we have explored consists of extracting one-second measurements containing weak spectral features through data clustering. Averaging one-second spectra in a cluster should reveal the presence of anomaly sources. We created two ML models using K-means clustering and Neural Network Self-organizing Map (SOM). Performance of these ML models was benchmarked using search data. One data set contained 137 Cs source, and another dataset contained 131 I source.

54 ENVIRONMENTAL SCIENCES↗

Closures and Simulation for Thermal Radiation Transport in Stochastic Media with Nonlinear Temperature Dependence

Because of the practical challenge of rendering very complex realistic spatial structures for numerical work, it is common practice to resort to characterizing such media as stochastic mixtures of materials, ideally parametrized with low order statistics such as the mean, variance, and correlation functions of the now random material properties. This enables realizations of the medium to be repeatedly generated and radiation transport computations to, in principle, be performed for a large ensemble of these realizations to obtain a statistically well-characterized radiation field. Statistical post-processing yields desired quantities such as conditional and unconditional mean radiation flux and probability distributions of transmitted radiation. However, such computations prove expensive for all but the simplest stochastic geometries and are most suited for benchmarking approximate models. The most common approximations lead to homogenized media so that transport computations are required only on a single medium realization but by construct provide only limited statistical information on the radiation field. Almost all approximate approaches to this problem attempt to develop equations for low order moments of the radiation intensity (mean, second moment, correlation function) but inevitably encounter a closure problem: the equation for any statistical moment will contain terms depending on unknown higher-order moments. Thus, the challenge shifts to one of developing closure relations that relate the unknown moments to the lower order moments. Under very special conditions, an exact closure can be derived but in general closures are heuristically stated constitutive relations. Also, closure approaches depend on whether the mixing statistics are spatially and/or temporally continuous as in fluctuating turbulent fields, or discontinuous as in randomly mixed solid chunks of material. Thus, unconditional averaging is generally applied in the former case but conditional averaging is more appropriate when the mixing is discontinuous. In this work, the emphasis is on binary statistical mixtures of immiscible fluids as as such the quantities of interest are averages (flux, temperature) conditioned on the material type.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Detection of Anomalies in Gamma Background Radiation Data with K-Means and Self-Organizing Map Clustering Algorithms (Consortium on Nuclear Security Technologies (CONNECT) Q1 Report)

Environmental screening of gamma radiation consists of detecting weak nuisance and anomaly signal in the presence of strong and highly varying background. In a typical scenario, a mobile detector-spectrometer continuously measures gamma radiation spectra in short, e.g., one-second, signal acquisition intervals. The measurement data is a 2D matrix, where one dimension is gamma ray energy, and the other dimension is the number of measurements or total time. In principle, gamma radiation sources can be detected and identified from the measured data by their unique spectral lines. Detecting sources from data measured in a search scenario is difficult due to the highly varying background because of naturally occurring radioactive material (NORM), and low signal-to-noise ratio (S/N) of spectral signal measured during one-second acquisition intervals. The objective of this work is to explore unsupervised machine learning (ML) algorithms for detection and identification of weak nuisances and anomalies events in the presence of highly fluctuating background. The challenge is that spectral lines of isotopes are difficult to observe in one-second measurements. Averaging over the entire measurement campaign data set reveals spectral lines of most common background isotopes. Spectral lines of orphan sources, which might appear only in a few measurements during the campaign, will be washed out if averaging is performed over the entire measurement data set. The approach we have explored consists of extracting one-second measurements containing weak spectral features through data clustering. Averaging one-second spectra in a cluster should reveal the presence of anomaly sources. We created two ML models using K-means clustering and Neural Network Self-organizing Map (SOM). Performance of these ML models was benchmarked using search data. One data set contained 137 Cs source, and another dataset contained 131 I source.

61 RADIATION PROTECTION AND DOSIMETRY↗

Griffin/Pronghorn PBMR-400 Benchmark Results [Slides]

Content Overview: 1) Pronghorn & Griffin, MOOSE based applications overview: Codes capabilities description & MOOSE ecosystem overiew; Current research activities and synergies with other projects; Different multiphysics coupling scheme description. 2) PBMR-400 Benchmark: OECD NEA Benchmark and Pronghorn/Griffin models description; Standalone NK Ex1 and TH Ex2 Results description; Coupled Steady state Ex3; Results analysis for Transiet Ex4a (100-40-100 Load Following) & Ex3 (Protected PLOFC). 3) Conclusions $ Future steps.

97 MATHEMATICS AND COMPUTING↗

Brine Availability Test in Salt (BATS) FY24 Update

This report summarizes fiscal year 2024 (FY24) activities centered around a series of field tests in bedded salt at the Waste Isolation Pilot Plant (WIPP) funded by the Office of Spent Fuel and Waste Science and Technology in the Spent Fuel and Waste Disposition (SFWD) program of the US Department of Energy’s Office of Nuclear Energy (DOE-NE). High-level Purpose of Experiments: The Brine Availability Test in Salt (BATS) field tests are revealing both brine occurrence (i.e., where, and how much) and brine migration (i.e., how easily it moves) in the excavation damaged zone (EDZ). This understanding is foundational to develop a safety case for a future heat-generating waste repository in salt, and to starting up a generic repository program in salt to buy down risk. BATS seeks to predict how much brine can flow into both ambient and heated excavations (e.g., boreholes or rooms) in salt. This work is educating and empowering new repository scientists on two fronts: “design and execution of field tests” and “prediction and modeling of coupled processes.” DOE-NE capabilities in salt have grown and been tested through international modeling and benchmarking exercises (e.g., DECOVALEX, RANGERS, KOMPASS, and MEASURES; see Mills et al., 2024). The hands-on expertise we are building is a necessary step towards large-scale disposal demonstrations and eventual implementation.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DECOVALEX-2023: Task G, SAFENET Final Report

DECOVALEX Task G deals with fracture mechanics at several scales using a combined approach of experimental work, related model development, benchmarking and experimental analysis. Fig. 0.1 provides a graphical abstract for Task G. The experimental basis for the related mechanical (M), hydro-mechanical (HM), and thermo-mechanical (TM) processes comes from the rock mechanics laboratories of Universities of Freiberg (TUBAF) and Edinburgh, as well as the Korea Institute of Civil Engineering and Building Technology (KICT). The experimental work in the rock laboratories is closely linked to the underground research laboratories (URLs) Reiche Zeche (Germany), KURT (Korea), and Mont Terri (Switzerland). The scientific key questions are related to fracture permeability evolution under THM conditions, anisotropy effects on fracturing processes, and thermal fracture slip, which are being addressed in specific steps of the task. A large variety of numerical methods have been developed and applied for experimental analysis, ranging from continuum to discontinuum approaches. A detailed comparison of mechanical and hydro-mechanical processes is given in the section 3 via the benchmarking exercises. As a result of DECOVALEX-2023 Task G we further improved our understanding and predictability of fracturing processes under THM conditions. This was based on robust numerical simulation methods and in-depth experimental analysis.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Numerical Benchmark for High-Reynolds-Number Supercritical Flows with Large Density Gradients

Because of the extreme complexity of physical phenomena at high pressure, only limited data are available for solver validation at device-relevant conditions such as liquid rocket engines, gas turbines, or diesel engines. In the present study, a two-dimensional direct numerical simulation is used to establish a benchmark for supercritical flow at a high Reynolds number and high-density ratio at conditions typically encountered in liquid rocket engines. Emphasis has been placed on maintaining the flow characteristics of actual systems with simple boundary conditions, grid spacing, and geometry. Results from two different state-of-the-art codes, with markedly different numerical formalisms, are compared using this benchmark. The strong similarity between the two numerical predictions lends confidence to the physical accuracy of the results. The established database can be used for solver benchmarking and model development at conditions relevant to many propulsion and power systems.

Engineering↗

The Fluid Dynamics Uncertainty Quantification Challenge Problem: XFOIL vs. MFOIL

Uncertainty quantification (UQ) has become more critical in aerospace engineering due to the growing dependence on computational tools for design optimization and performance analyses of aerospace vehicles. Even though the significance of UQ in assessing the credibility of computational analyses is well recognized, its costs and complexity impede its integration into standard practices, particularly in computational fluid dynamics (CFD) and other fluid analyses. This paper presents a UQ study for low-fidelity computational aerodynamics analyses with XFOIL and mfoil (i.e., the MATLAB version of XFOIL with several implementation modifications); these tools are utilized widely in both research and education. The main contributions of this paper are as follows: 1) improved precision in quantifying the uncertainty of the baseline Monte Carlo results used to benchmark surrogate modeling techniques for UQ, 2) quantification of the effect of the implementation differences between XFOIL and mfoil on solution quantities of interest (QoIs), such as lift and pitching moment coefficients, and 3) development of an open-source UQ library for use with XFOIL and mfoil, which has educational values and helps promote UQ for fluid analyses with aerospace applications. Results and discussions revolve around cases 1-4 of the challenge problem posed by the AIAA Fluid Dynamics Technical Committee’s Uncertainty Quantification Discussion Group (UQDG). In case 3, this work employs CFDverify, an open-source solution verification software, to quantify the discretization error and evaluate the extrapolated QoIs based on the grid convergence index (GCI). This UQ study differentiates itself from previous studies in the rigor of handling baseline Monte Carlo uncertainty and in including mfoil, which is a more accessible alternative to XFOIL. Finally, despite the growing computing power, low-fidelity computational tools remain valuable, such as for aerodynamic shape optimization at Mach numbers below 0.65 and low-to-mid Reynolds numbers.

Lay, Aidan S [University of Tennessee, Knoxville (↗

AlphaBuilding - Synthetic Buildings Operation Dataset

This is a synthetic building operation dataset which includes HVAC, lighting, miscellaneous electric loads (MELs) system operating conditions, occupant counts, environmental parameters, end-use and whole-building energy consumptions at 10-minute intervals. The data is created with 1395 annual simulations using the U.S. DOE detailed medium-sized reference office building, and 30 years' historical weather data in three typical climates including Miami, San Francisco, and Chicago. Three energy efficiency levels of the building and systems are considered. Assumptions regarding occupant movements, occupants' diverse temperature preferences, lighting, and MELs are adopted to reflect realistic building operations. A semantic building metadata schema - BRICK, is used to store the building metadata. The dataset is saved in a 1.2 TB of compressed HDF5 file. This dataset can be used in various applications, including building energy and load shape benchmarking, energy model calibration, evaluation of occupant and weather variability and their influences on building performance, algorithm development and testing for thermal and energy load prediction, model predictive control, policy development for reinforcement learning based building controls.

AlphaBuilding↗

Challenging a Global Land Surface Model in a Local Socio-Environmental System

Land surface models (LSMs) predict how terrestrial fluxes of carbon, water, and energy change with abiotic drivers to inform the other components of Earth system models. Here, we focus on a single human-dominated watershed in southwestern Michigan, USA. We compare multiple processes in a commonly used LSM, the Community Land Model (CLM), to observational data at the single grid cell scale. For model inputs, we show correlations (Pearson’s R) ranging from 0.46 to 0.81 for annual temperature and precipitation, but a substantial mismatch between land cover distributions and their changes over time, with CLM correctly representing total agricultural area, but assuming large areas of natural grasslands where forests grow in reality. For CLM processes (outputs), seasonal changes in leaf area index (LAI; phenology) do not track satellite estimates well, and peak LAI in CLM is nearly double the satellite record (5.1 versus 2.8). Estimates of greenness and productivity, however, are more similar between CLM and observations. Summer soil moisture tracks in timing but not magnitude. Land surface reflectance (albedo) shows significant positive correlations in the winter, but not in the summer. Looking forward, key areas for model improvement include land cover distribution estimates, phenology algorithms, summertime radiative transfer modelling, and plant stress responses.

54 ENVIRONMENTAL SCIENCES↗

Benchmarking parametric and machine learning models for genomic prediction of complex traits

The usefulness of genomic prediction in crop and livestock breeding programs has prompted efforts to develop new and improved genomic prediction algorithms, such as artificial neural networks and gradient tree boosting. However, the performance of these algorithms has not been compared in a systematic manner using a wide range of datasets and models. Using data of 18 traits across six plant species with different marker densities and training population sizes, we compared the performance of six linear and six non-linear algorithms. First, we found that hyperparameter selection was necessary for all non-linear algorithms and that feature selection prior to model training was critical for artificial neural networks when the markers greatly outnumbered the number of training lines. Across all species and trait combinations, no one algorithm performed best, however predictions based on a combination of results from multiple algorithms (i.e. ensemble predictions) performed consistently well. While linear and non-linear algorithms performed best for a similar number of traits, the performance of non-linear algorithms vary more between traits. Although artificial neural networks did not perform best for any trait, we identified strategies (i.e. feature selection, seeded starting weights) that boosted their performance to near the level of other algorithms. Our results highlight the importance of algorithm selection for the prediction of trait values.

59 BASIC BIOLOGICAL SCIENCES↗

Design of inertial fusion implosions reaching the burning plasma regime

One of the last remaining milestones in fusion research before reaching ignition is creating a burning plasma state, where alpha particles from deuterium-tritium (DT) fusion reactions redeposit their energy as the dominant source of heating in the plasma. The indirect-drive inertial confinement fusion approach at the National Ignition Facility (NIF) uses a laser-generated radiation cavity (hohlraum) to spherically implode DT fuel to high temperatures and densities in a central ”hot spot”. Here, we deliver more energy to the hot spot than ever before, while maintaining the extreme pressures required for inertial confinement, by increasing the size of the implosion compared to previous experiments. We develop more efficient hohlraums, to drive these larger implosions within NIF’s current laser energy and power capability and control symmetry by moving energy between laser beams and by changing the shape of the hohlraum. These designs resulted in record fusion powers of 1.5 petawatts, greater than the input power of the laser, and 170 kJ of fusion energy. Radiation hydrodynamics simulations show alpha particle heating as the dominant term in the hot spot energy balance, e.g. a burning plasma state. This work is expected to motivate future studies of burning plasmas and improve predictive capability by providing a benchmark for modeling used to understand the proximity to ignition.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗