Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Benchmark testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,135 records · Page 63

A general spatial-temporal framework for short-term building temperature forecasting at arbitrary locations with crowdsourcing weather data

Weather forecasting has been a critical component to predict and control building energy consumption for better building energy management. Without accessibility to other data sources, the onsite observed temperatures or the airport temperatures are used in forecast models. In this paper, we present a novel approach by utilizing the crowdsourcing weather data from neighboring personal weather stations (PWS) to improve the weather forecast accuracy around buildings using a general spatial-temporal modeling framework. The final forecast is based on the ensemble of local forecasts for the target location using neighboring PWSs. Our approach is distinguished from existing literature in various aspects. First, we leverage the crowdsourcing weather data from PWS in addition to public data sources. In this way, the data is at much finer time resolution (e.g., at 5-minute frequency) and spatial resolution (e.g., arbitrary location vs grid). Second, our proposed model incorporates spatial-temporal correlation information of weather variables between the target building and a set of neighboring PWSs so that underlying correlations can be effectively captured to improve forecasting performance. Here, we demonstrate the performance of the proposed framework by comparing to the benchmark models on temperature forecasting for a building located at an arbitrary location at San Antonio, Texas, USA. In general, the proposed model framework equipped with machine learning technique such as Random Forest can improve forecasting by 50% compares with persistent model and has 90% chance to outperform airport forecast in short-term forecasting. In a real-time setting, the proposed model framework can provide more accurate temperature forecasting results compared with using airport temperature forecast for most forecast horizon. Moreover, we analyze the sensitivity of model parameters to gain insights on how crowdsourcing data from the neighboring personal weather stations impacts forecasting performance. Finally, we implement our model in other cities such as Syracuse and Chicago to test the model's performance in different landforms and climate types.

54 ENVIRONMENTAL SCIENCES↗

Consistency testing for robust phase estimation

In this work, we present an extension to the robust phase estimation protocol, which can identify incorrect results that would otherwise lie outside the expected statistical range. Robust phase estimation is increasingly a method of choice for applications such as estimating the effective process parameters of noisy hardware, but its robustness is dependent on the noise satisfying certain threshold assumptions. We provide consistency checks that can indicate when those thresholds have been violated, which can be difficult or impossible to test directly. We test these consistency checks for several common noise models, and identify two possible checks with high accuracy in locating the point in a robust phase estimation run at which further estimates should not be trusted. One of these checks may be chosen based on resource availability, or they can be used together in order to provide additional verification.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

If We Build Them, They Will Run: Automated HPC Apps Deployment and Profiling with eBPF in Cloud

The high performance computing (HPC) community is in a period of transition. The rise of AI/ML coupled with a changing landscape of resources deems portability a new metric of performance, and methods to move between on-premises and cloud environments and assess compatibility are paramount. Here we design and test a strategy for bridging the gap between traditional HPC and Kubernetes environments – first containerizing applications, providing automated orchestration to run studies, and packaging the setup with automated means to assess performance using low overhead eXtended Berkeley Packet Filter (eBPF) programs. We first assess different designs for eBPF collection, demonstrating a tradeoff between number of programs deployed on a node and overhead added. We develop 5 low overhead eBPF programs that combine with streaming ML models to assess CPU, futex, TCP, shared memory, and file access across four different builds of an HPC application for CPU and GPU. We use eBPF data to generate insights into the possible underlying etiology of scaling issues. We then assess compatibility of a well-known benchmark, HPCG, across matrices of micro-architectures and optimization levels (217 containers across 24 instance types and over 7500 runs). We provide to the community 30 applications to deploy in our automated setup and perform a scaling study from 4 to a maximum of 256 nodes for both CPU and GPU applications. Finally, we use our gained knowledge about performance to generate compatibility artifacts that are used by a newly developed Kubernetes controller to intelligently select instance type based on optimizing a figure of merit. Along with insights to scaling in this environment with a collection of applications and templates to work from, we provide an overall strategy for approaching HPC application deployment and image selection based on compatibility in cloud.

Computer science↗

Robust historical evapotranspiration trends across climate regimes

Evapotranspiration (ET) links the hydrological, energy and carbon cycles on the land surface. Quantifying ET and its spatio-temporal changes is also key to understanding climate extremes such as droughts, heatwaves and flooding. Regional ET estimates require reliable observation-based gridded ET datasets, and while many have been developed using physically based, empirically based and hybrid techniques, their efficacy, and particularly the efficacy of their uncertainty estimates, is difficult to verify. In this work, we extend the methodology used in Hobeichi et al. (2018) to derive two new versions of the Derived Optimal Linear Combination Evapotranspiration (DOLCE) product, with observationally constrained spatio-temporally varying uncertainty estimates, higher spatial resolution, more constituent products and extended temporal coverage (1980–2018). After demonstrating the efficacy of these uncertainty estimates with out-of-sample testing, we derive novel ET climatology clusters for the land surface, based on the magnitude and variability of ET at each location on land. The new clusters include three wet and three dry regimes and provide an approximation of Köppen–Geiger climate classes. The verified uncertainty estimates and extended time period then allow us to examine the robustness of historical trends spatially and in each of these six ET climatology clusters. We find that despite robust decreasing ET trends in some regions these do not correlate with behavioural ET clusters. Each cluster, and the majority of the Earth's surface, shows clear robust increases in ET over the recent historical period. The new datasets DOLCE V2.1 and DOLCE V3 can be used for benchmarking global ET estimates and for examining ET trends respectively.

54 ENVIRONMENTAL SCIENCES↗

Physics vs structure: A systematic benchmark of learning strategies for multi-zone building thermal dynamics

Recent advances in physics-informed and data-driven machine learning promise improved thermal models for advanced building control, yet there is limited quantitative evidence on when added physics structure and architectural complexity are beneficial. Here, this work presents a systematic benchmark of five representative system identification methods for modeling multi-zone building thermal dynamics: linear state-space models, multi-layer perceptrons, neural state-space models, neural ordinary differential equations, and physically-consistent neural networks. The methods are evaluated across multiple data regimes and zone coupling strategies. Using a high-fidelity multi-zone commercial building emulator, we examine short-term and long-term prediction accuracy, computational efficiency, and ease of development. Our results reveal critical trade-offs between prediction performance, model complexity, and physical consistency. We demonstrate that decoupled, nonlinear black-box models consistently outperform coupled physics-constrained architectures in both predictive accuracy and out-of-distribution robustness in majority of the test cases for the building type considered in the study. Our findings quantify the cost of complexity in building thermal modeling and provide concrete, actionable, scenario-based guidelines for selecting model classes for control-oriented applications.

Building thermal modeling↗

Modeling of fluid injection-induced fault reactivation using coupled fluid flow and mechanical interface model

The present study is aimed at developing a numerical model to reproduce coupled hydro-mechanical processes associated with fault reactivation by fluid injection in low permeability rock, as part of the DECOVALEX-2019 project Task B. We proposed a modeling approach for simulating the processes using the TOUGH-FLAC simulator, and modeled a fault reactivation experiment conducted at Mont Terri Rock Laboratory in Switzerland. The first step of the study involved benchmark calculations considering a simplified fault plane and geometry. Fluid flow along a fault was modeled using elements of aperture-sized thickness on the basis of Darcy's law and the cubic law in TOUGH2, whereas the mechanical behavior of a single fault was represented by zero-thickness interface elements in FLAC3D upon which a slip and/or separation is allowed. A methodology to connect a TOUGH2 volume element to a FLAC3D interface element was developed for handling the hydro-mechanical interactions on the fault during fluid injection. Two different fault models for describing the evolutions of hydraulic aperture by elastic fracture opening and failure-induced aperture increase were considered in the benchmark calculations. In the coupling process, the changes in geometrical features and hydrological properties induced by mechanical deformation were continuously updated. The transient responses of the fault and host rock to stepwise pressurization were examined during the simulation. The hydro-mechanical behavior, including the injection flow rate, pressure distribution around the borehole, stress conditions, and displacements in normal and shear directions were monitored in the surrounding rock and along the fault. The results of benchmark calculations suggest that the developed model reasonably represents the hydro-mechanical behavior of a fault and the surrounding rock. This modeling approach was applied to the fault reactivation experiment of the Mont Terri Rock Laboratory. In this interpretive modeling, a parametric study was conducted to examine the effects of input parameters regarding in situ stress and fault properties on the hydro-mechanical responses of the fault to water injection. Then, an optimal parameter set to reproduce the field experiment results was chosen by trial-and-error. The injection flow rate and pressure response during fault reactivation closely matched those obtained at the site, which indicates the capability of the model to appropriately capture the progressive pathway evolution during fault reactivation tests at the site. Finally, the anchor displacements were overestimated by the model, but a fair agreement was obtained in terms of the order of magnitude and the variation tendency.

58 GEOSCIENCES↗

The status of the Japanese material properties handbook and the challenge to facilitate structural design criteria for DEMO in-vessel components

This work summarizes the current status of the material properties handbook for a structural design using Japanese reduced-activation ferritic/martensitic steel F82H. Specifically, the key structural parameters, e.g. time-independent/dependent design stresses and fatigue design curves, were determined by following the French structural design code RCC-MRx. Moreover, under the Japan–U.S. collaboration, tensile data were newly added to the benchmark heavy irradiation data up to 80 dpa, as critical input information in the intermediate check and review in Japan. Furthermore, the status of structural material data and the near-term and long-term issues were clarified by the evaluation using the attribute guides. In parallel, the structural design approaches, which were newly introduced and extended to cope with the structural design issues under the complex environmental conditions peculiar to the DEMO reactor, were noted with the initial R&D results. Of the many design issues, the multi-axial loading conditions due to the complexity of the DEMO reactor as well as the coolant compatibility and the irradiation effect are mentioned. For example, in the paper, multi-axial fatigue–creep testing and evaluation using the modified universal slope method and brittle/ductile fracture testing and evaluation using the local approach are explained toward DEMO.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

KGML-ag: a modeling framework of knowledge-guided machine learning to simulate agroecosystems: a case study of estimating N<sub>2</sub>O emission using data from mesocosm experiments

Abstract. Agricultural nitrous oxide (N2O) emission accounts for a non-trivial fraction of global greenhouse gas (GHG) budget. To date, estimating N2O fluxes from cropland remains a challenging task because the related microbial processes (e.g., nitrification and denitrification) are controlled by complex interactions among climate, soil, plant and human activities. Existing approaches such as process-based (PB) models have well-known limitations due to insufficient representations of the processes or uncertainties of model parameters, and due to leverage recent advances in machine learning (ML) a new method is needed to unlock the “black box” to overcome its limitations such as low interpretability, out-of-sample failure and massive data demand. In this study, we developed a first-of-its-kind knowledge-guided machine learning model for agroecosystems (KGML-ag) by incorporating biogeophysical and chemical domain knowledge from an advanced PB model, ecosys, and tested it by comparing simulating daily N2O fluxes with real observed data from mesocosm experiments. The gated recurrent unit (GRU) was used as the basis to build the model structure. To optimize the model performance, we have investigated a range of ideas, including (1) using initial values of intermediate variables (IMVs) instead of time series as model input to reduce data demand; (2) building hierarchical structures to explicitly estimate IMVs for further N2O prediction; (3) using multi-task learning to balance the simultaneous training on multiple variables; and (4) pre-training with millions of synthetic data generated from ecosys and fine-tuning with mesocosm observations. Six other pure ML models were developed using the same mesocosm data to serve as the benchmark for the KGML-ag model. Results show that KGML-ag did an excellent job in reproducing the mesocosm N2O fluxes (overall r2=0.81, and RMSE=3.6 mgNm-2d-1 from cross validation). Importantly, KGML-ag always outperforms the PB model and ML models in predicting N2O fluxes, especially for complex temporal dynamics and emission peaks. Besides, KGML-ag goes beyond the pure ML models by providing more interpretable predictions as well as pinpointing desired new knowledge and data to further empower the current KGML-ag. We believe the KGML-ag development in this study will stimulate a new body of research on interpretable ML for biogeochemistry and other related geoscience processes.

54 ENVIRONMENTAL SCIENCES↗

RELAP5-3D Modeling of the OECD-NEA HTTF Benchmark

HTTF at Oregon State University (OSU) Reference: General Atomics’ modular high-temperature gas-cooled reactor Helium cooled, electrically heated (2.2 MW) Prismatic graphite blocks in the core and reflectors Alumina ceramic blocks are used to simulate the core and top and bottom reflectors One-fourth scale in length and diameter Most of the coolant channels in the core are full scale Lower pressure compared to the prototype reactor (0.7 MPa) Over 500 instruments Designed primarily to investigate depressurized (DCC) and pressurized (PCC) conduction cooldown transients

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Benchmark Exercise Report for Experimental Study of Bubble Scrubbing in Sodium Pool

Mechanistic source term (MST) analyses are likely to be an important part of advanced reactor licensing applications. For the purpose, an MST analysis code applicable to newly introduced advanced reactors, called SRT (Simplified Radionuclide Transport) code, has been developed by Argonne National Laboratory. SRT can track overall behaviors of radionuclides especially in metal fuel-based sodium fast reactors (SFRs) and microreactors. Throughout the simulation, migration inside fuel pins before failure, interaction with coolant (for SFR), removal/leakage in cover gas and containment (or confinement), and environmental dose impacts are considered alongside radioactive decay for short-lived nuclides. Among the postulated process, pool scrubbing phenomenon, especially under sodium pool condition, has been identified as high importance with limited supportive data. The phenomenon plays a crucial role in assessing the degree of radiological impacts as radioactive aerosols or vapors are efficiently and effectively removed during the process. To provide validation basis for SRT in assessing pool scrubbing performance inside sodium pools, the University of Wisconsin-Madison performed tests including extensive parametric effects. Separate effect tests were conducted to directly evaluate the SRT models and to estimate degree of contribution by each contributing factor. Specifically, bubble size, aerosol size, aerosol density, aerosol concentration, pool depth, system temperature, and bubble swarm effects were considered. According to the parametric effects, decontamination performance enhances with decreasing bubble size, large density, and deeper pool height. Aerosol concentration provides no effect for the whole range of interest, and pool temperature variation shows minor effects under the considered temperature condition. When multiple bubbles are injected generating a bubble swarm condition, DF performance further enhances by bubble interactions and turbulence characteristics. The measurement shows the exceptional importance of aerosol size range considered, with the lowest decontamination, where most radionuclides are expected to escape.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Laser Driven Hydrothermal Processing (LDHP) for Rapid Dissolution of Glassy Nuclear Debris

Discovered in 2009 on the SLAB laser system at LLNL, Laser-Driven Hydrothermal Processing (LDHP) both spalls and transiently dissolves a wetted surface (e.g., rock, concrete, paint) when that surface is struck by a laser beam pulse under specific conditions. Previous projects have explored the application of LDHP to expediting sample dissolution for post-detonation nuclear forensics. Early results were very promising, indicating that this method could decrease material dissolution time by a factor of 10 or more. However, the large and complicated lasers previously used are impractical for this application. Under the R^3 venture we aimed to benchmark the LDHP capabilities of turn-key commercial off-the-shelf (COTS) lasers, and to establish how this process could be integrated into an analytical workflow for nuclear forensic analysis. We demonstrated that geological rock standards could be quickly processed into fine powder by LDHP followed by complete acid digestion in 15 minutes, without elemental fractionation that would skew analytical results. We determined that the COTS laser can be used for LDHP that could potentially reduce dissolution time for solid nuclear debris samples in post detonation nuclear forensics. One downside to this approach, however, is that the laser interactions are different with every material, so extensive method validation would be required for all likely material compositions. The LDHP processing rates of several materials were tested and determined to be favorable in the beginning with significant slowdown due to particulate scattering. This could be mitigated either by stopping and centrifuging the sample at specific intervals, or through design of a continuous solvent flow through mechanism.

36 MATERIALS SCIENCE↗

High-Precision Spectroscopy of 20 O Benchmarking Ab Initio Calculations in Light Nuclei

The excited states of unstable 20 O were investigated via γ-ray spectroscopy following the 19 O(d, p) ⁢20 O reaction at 8 A⁢ MeV. By exploiting the Doppler shift attenuation method, the lifetimes of the $2^+_2$ and $3^+_1$ states were firmly established. From the γ-ray branching and E⁢2/M⁢1 mixing ratios for transitions deexciting the $2^+_2$ and $3^+_1$ states, the B⁡(E⁢2) and B⁡(M⁢1) were determined. Various chiral effective field theory Hamiltonians, describing the nuclear properties beyond ground states, along with a standard USDB interaction, were compared with the experimentally obtained data. Such a comparison for a large set of γ-ray transition probabilities with the valence space in medium similarity renormalization group ab initio calculations was performed for the first time in a nucleus far from stability. It was shown that the ab initio approaches using chiral effective field theory forces are challenged by detailed high-precision spectroscopic properties of nuclei. Finally, the reduced transition probabilities were found to be a very constraining test of the performance of the ab initio models.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

FY20 Proxy App Suite Release: Report for ECP Proxy App Project Milestone ADCD-504-10

Version 4.0 of the ECP Proxy App Suite is practically unchanged from the previous release. The current set of proxies has proven useful for many aspects of benchmarking and co-design and we see little reason to alter the suite. Although there have been few changes to the ECP suite, the team has been hard at work in other areas. In the area of Machine Learning (ML) we have now created a separate proxy suite dedicated to this scientific applications of ML. The suite includes: miniGAN (Generative Adversarial Networks), miniRL (Reinforcement Learning), CRADL (inline inference), Cosmoflow-Benchmark (Convolutional Neural Network), and MLPerf-DeepCam (Climate Segmentation Benchmark). Section 3 contains more information about these proxies as well as the principles that are guiding the development of the suite. We have surveyed available proxies for several application domains including Computational Fluid Dynamics, Quantum Chemistry, Quantum Computing Simulation, Molecular Dynamics, Monte Carlo Transport, and Density Functional Theory to identify gaps in proxy coverage. Several new proxy apps are either already available or will be released soon to fill these gaps. Section 4 provides full details. Finally, section 5 reports on our continued collaboration with the ECP Continuous Integration (CI) effort to use proxy apps to help identify problems and roadblocks to cross-lab CI. We have also assisted the El Capitan Center of Excellence (COE) to stand up a CI system that will be used to test releases of HPE and AMD software stacks. We hope that the COE effort can serve as a model for ECP by showing how the Proxy App Team can work with AD and ST teams identify critical features, kernels, patterns, etc. and incorporate them into a CI system that will help ensure that Frontier and Aurora will provide those needed capabilities.

97 MATHEMATICS AND COMPUTING↗

Numerical Treatment of Shock-induced Nuclear Burning in Double Detonation Type Ia Supernovae

We present a benchmark problem to assess the treatment of shock-induced nuclear burning in the context of double detonation Type Ia supernovae. In a stratified white dwarf model, we implement a shock-detection criterion that suppresses burning in zones characterized by compression and significant pressure gradients, controlled by a tunable parameter, f shock . One-dimensional simulations, using the open-source Castro suite, were conducted across three treatments—burning fully enabled, and burning suppressed with f shock = 2/3 and f shock = 1—across three spatial resolutions (5.0, 2.5, and 0.3125 km). At the finest resolution, the burning-enabled and f shock = 1 models converge, while the f shock = 2/3 front continues to show slight offset behavior. Since most simulations are carried out at much lower resolutions, our tests support the idea that burning in shocks should always be disabled in practice. We also observe that the behavior of lower-resolution simulations remains extremely sensitive to the choice of f shock .

GPU computing↗

Design and commissioning of an e-beam irradiation beamline at the Upgraded Injector Test Facility at Jefferson Lab

We report the Upgraded Injector Test Facility (UITF) at Jefferson Lab is a continuous-wave superconducting linear accelerator capable of providing an electron beam with energy up to 10 MeV. A beamline for electron-beam irradiation has been designed, installed and successfully commissioned at this facility, aimed at the degradation study of 1,4-dioxane and per- and polyfluoroalkyl substances (PFAS) in wastewater treatment. A solenoid with a peak axial magnetic field of up to 0.28 T and a set of raster coils were used to obtain a Gaussian beam profile with a transverse standard deviation of ~ 15.0 mm at the target location. Monte-Carlo simulations using FLUKA were carried out to calculate the total absorbed dose and the dose distribution in the sample volume inside the target cell. The simulations were benchmarked experimentally by dosimetry mapping using optichromic dosimeters. The results of the irradiation experiments showed a ~ 95% reduction of 1,4-dioxane in ultra-pure water for a dose of 1 kGy, demonstrating the potential of electron-beam irradiation towards addressing growing challenges in environmental remediation.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Fuzzy Simplicial Networks: A Topology-Inspired Model to Improve Task Generalization in Few-shot Learning

Deep learning has shown great success in settings with massive amounts of data but has struggled when data is limited. Few-shot learning algorithms, which seek to address this limitation, are designed to generalize well to new tasks with limited data. Typically, models are evaluated on unseen classes and datasets that are defined by the same fundamental task as they are trained for (e.g. category membership). One can also ask how well a model can generalize to fundamentally different tasks within a fixed dataset (for example: moving from category membership to tasks that involve detecting object orientation or quantity). To formalize this kind of shift we define a notion of “independence of tasks” and identify three new sets of labels for established computer vision datasets that test a model's ability to generalize to tasks which draw on orthogonal attributes in the data. We use these datasets to investigate the failure modes of metric-based few-shot models. Based on our findings, we introduce a new few-shot model called Fuzzy Simplicial Networks (FSN) which leverages a construction from topology to more flexibly represent each class from limited data. In particular, FSN models can not only form multiple representations for a given class but can also begin to capture the low-dimensional structure which characterizes class manifolds in the encoded space of deep networks. We show that FSN outperforms state-of-the-art models on the challenging tasks we introduce in this paper while remaining competitive on standard few-shot benchmarks.

deep learning↗

Code Validation of SAM Using Forced and Natural Circulation Data from NACIE-UP Benchmark

Heavy liquid metals (HLMs) are promising candidates as coolants of Generation IV fast reactors due to their thermophysical properties. In the last decade, experimental work has been proposed as part of research and development efforts to develop such systems. In this context, researchers from the Brasimone Research Center have conducted many experiments using the Natural Circulation Experiment Upgrade (NACIE-UP) facility to study the thermofluid dynamic behavior of HLMs in rod bundle configurations with or without wire wrappers. This facility consists of a rectangular loop operated with lead-bismuth eutectic. Sensors across the loop monitor relevant parameters, i.e. temperatures, heat transfer, and flow conditions. Here, in the present work, we carefully select published data from NACIE-UP to validate the System Analysis Module (SAM), a modern system analysis code developed at Argonne National Laboratory. We developed one SAM model using specifications of the facility geometry and materials existing in relevant papers and reports. On top of that, these references provided the boundary conditions for simulating natural circulation and forced convection experiments in either steady or transient conditions. The SAM model simulates five test cases with diverse operating conditions. Ultimately, the code is proven to predict temperatures and mass flow rates that closely match the experiments. The discrepancies between numerical predictions and diverse transients are limited to a few degrees Celsius, showcasing that SAM is well suited for analyzing nuclear systems relying on HLM coolants.

advanced reactors↗

BUTTER - Empirical Deep Learning Dataset

The BUTTER Empirical Deep Learning Dataset represents an empirical study of the deep learning phenomena on dense fully connected networks, scanning across thirteen datasets, eight network shapes, fourteen depths, twenty-three network sizes (number of trainable parameters), four learning rates, six minibatch sizes, four levels of label noise, and fourteen levels of L1 and L2 regularization each. Multiple repetitions (typically 30, sometimes 10) of each combination of hyperparameters were preformed, and statistics including training and test loss (using a 80% / 20% shuffled train-test split) are recorded at the end of each training epoch. In total, this dataset covers 178 thousand distinct hyperparameter settings ("experiments"), 3.55 million individual training runs (an average of 20 repetitions of each experiments), and a total of 13.3 billion training epochs (three thousand epochs were covered by most runs). Accumulating this dataset consumed 5,448.4 CPU core-years, 17.8 GPU-years, and 111.2 node-years.

Array↗