Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Performance benchmark”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,243 records · Page 69
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

DOE FAIR Surrogate Benchmarks Supporting AI and Simulation Research (SBI Surrogate Benchmark Initiative) (Final Report)

Computational Science is being revolutionized by integrating AI and simulation and, in particular, by deep learning surrogate models that can replace all or part of traditional large‐scale HPC computations. Such surrogates can achieve remarkable performance improvements, as much as several orders of magnitude, and save both compute time and energy. The Surrogate Benchmark Initiative (SBI) project creates a community repository and FAIR (Findable, Accessible, Interoperable, and Reusable) data ecosystem for HPC application surrogate benchmarks. The SBI team comes from Argonne National Laboratory (ANL), Indiana University (IU), Rutgers University, the University of Tennessee, Knoxville (UTK), and the University of Virginia(UVA). SBI repositories include data, code, and all relevant collateral artifacts, that the science and engineering community needs to use and reuse these data sets and surrogates. SBI repositories generate active research from both participants in SBI and the broader AI and domain science communities. This project develops surrogates that use several different neural nets to learn and quickly infer the results of simulations and data systems and capture them as surrogate benchmarks with a rich set of metadata, covering. Data; Model; Metrics specification; Machine specification; Science, Speed, Power Results, We research FAIR metadata for these benchmarks. We develop application surrogate examples as benchmarks across many fields (ANL, UTK, IU, UVA). We also study non Surrogate benchmarks that have many common features and similar issues regarding FAIRness. We work with MLCommons (UVA, UTK), which is a major machine learning benchmarking activity where we get metadata ontologies, software, and benchmarks, benchmarks have datasets, models, and metadata, and they need a technical framework developed by UTK and Rutgers and deployed by UVA. We study features of Surrogates, including performance, training set size, and uncertainty quantification (Rutgers, UVA and IU).

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1991

Collected notes from the Benchmarks and Metrics Workshop

In recent years there has been a proliferation of proposals in the artificial intelligence (AI) literature for integrated agent architectures. Each architecture offers an approach to the general problem of constructing an integrated agent. Unfortunately, the ways in which one architecture might be considered better than another are not always clear. There has been a growing realization that many of the positive and negative aspects of an architecture become apparent only when experimental evaluation is performed and that to progress as a discipline, we must develop rigorous experimental methods. In addition to the intrinsic intellectual interest of experimentation, rigorous performance evaluation of systems is also a crucial practical concern to our research sponsors. DARPA, NASA, and AFOSR (among others) are actively searching for better ways of experimentally evaluating alternative approaches to building intelligent agents. One tool for experimental evaluation involves testing systems on benchmark tasks in order to assess their relative performance. As part of a joint DARPA and NASA funded project, NASA-Ames and Teleos Research are carrying out a research effort to establish a set of benchmark tasks and evaluation metrics by which the performance of agent architectures may be determined. As part of this project, we held a workshop on Benchmarks and Metrics at the NASA Ames Research Center on June 25, 1990. The objective of the workshop was to foster early discussion on this important topic. We did not achieve a consensus, nor did we expect to. Collected here is some of the information that was exchanged at the workshop. Given here is an outline of the workshop, a list of the participants, notes taken on the white-board during open discussions, position papers/notes from some participants, and copies of slides used in the presentations.

Drummond, Mark E.↗
DOE OSTI2020

CephFS experiments on stria.sandia.gov

This report is an institutional record of experiments conducted to explore performance of a vendor installation of CephFS on the SNL stria cluster. Comparisons between CephFS, the Lustre parallel file system, and NFS were done using the IOR and MDTEST benchmarking tools, a test program which uses the SEACAS/Trilinos IOSS library, and the checkpointing activity performed by the LAMMPS molecular dynamics simulation.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2020

CephFS experiments on stria.sandia.gov

This report is an institutional record of experiments conducted to explore performance of a vendor installation of CephFS on the SNL stria cluster. Comparisons between CephFS, the Lustre parallel file system, and NFS were done using the IOR and MDTEST benchmarking tools, a test program which uses the SEACAS/Trilinos IOSS library, and the checkpointing activity performed by the LAMMPS molecular dynamics simulation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Taming the fixed-node error in diffusion Monte Carlo via range separation

By combining density-functional theory (DFT) and wave function theory via the range separation (RS) of the interelectronic Coulomb operator, we obtain accurate fixed-node diffusion Monte Carlo (FN-DMC) energies with compact multi-determinant trial wave functions. In particular, we combine here short-range exchange-correlation functionals with a flavor of selected configuration interaction known as configuration interaction using a perturbative selection made iteratively (CIPSI), a scheme that we label RS-DFT-CIPSI. One of the take-home messages of the present study is that RS-DFT-CIPSI trial wave functions yield lower fixed-node energies with more compact multi-determinant expansions than CIPSI, especially for small basis sets. Indeed, as the CIPSI component of RS-DFT-CIPSI is relieved from describing the short-range part of the correlation hole around the electron-electron coalescence points, the number of determinants in the trial wave function required to reach a given accuracy is significantly reduced as compared to a conventional CIPSI calculation. Importantly, by performing various numerical experiments, we evidence that the RS-DFT scheme essentially plays the role of a simple Jastrow factor by mimicking short-range correlation effects, hence avoiding the burden of performing a stochastic optimization. Considering the 55 atomization energies of the Gaussian-1 benchmark set of molecules, we show that using a fixed value of mu = 0.5 bohr(-1) provides effective error cancellations as well as compact trial wave functions, making the present method a good candidate for the accurate description of large chemical systems.

Scemama, Anthony↗
DOE OSTI2024

Binder-Coated Carbon Cloth Electrodes for All-Vanadium Redox Flow Batteries

Vanadium redox flow batteries (VRFBs) are a promising solution for integrating intermittent renewable energy sources into the existing power grid. However, enhancing the electrochemical performance of VRFBs is critical for their widespread adoption in grid-scale energy storage. This study investigates the impact of adding a porous binder to a carbon-cloth electrode, with a focus on optimizing thermal activation conditions. The electrochemical performance of the binder-coated electrodes compared to uncoated electrodes is evaluated through electrochemical impedance spectroscopy, polarization curve measurements, and charge-discharge cycling. The surface morphology and structural integrity of the binder-coated electrodes at each activation stage are examined using various material characterization techniques to assess the effects of thermal activation. The results are benchmarked against the experiments using non-coated electrodes to determine the performance improvements offered by the binder coating. Notably, the study reveals that binder-coated electrodes exhibit significantly lower resistance and improved efficiency compared to their uncoated counterparts, with optimal activation conditions enhancing performance metrics crucial for VRFB applications. These findings provide valuable insights for further optimizing electrode design and activation strategies, advancing the development of more efficient VRFB systems for large-scale energy storage.

Caiado, Ashley A.↗
DOE OSTI2023

Invited Paper: Benchmarking and Optimizing Data Movement on Emerging Heterogeneous Architectures

As supercomputers evolve, nodes are continually increasing in complexity. As a result, each generation of parallel systems brings new performance challenges. For instance, on recent systems inter-node communication has outperformed inter-socket, resulting in poor performance of many node-aware communication optimizations. Communication optimizations are critical for the performance and scalability of parallel applications, but are dependent on the parallel architecture, which varies significantly among recent generations of supercomputers. Furthermore, this paper investigates the performance of various paths of data movement on recent generations of systems, and analyzes the increased complexity of communication, particularly on recent heterogeneous systems. The paper also introduces MPI Advance, a communication library that enables optimizations to be created based on benchmark analysis of each emerging system.

benchmarking↗
NASA NTRS2024

PALMO: An OVERFLOW Machine Learning Airfoil Performance Database

The OVERFLOW Machine Learning Airfoil Performance (PALMO) database has been created to enable robust modeling of airfoil performance in a variety of applications. The database uses OVERFLOW simulation data second-order accurate in time and fourth-order accurate in space with Spalart-Allmaras turbulence closure. The foundation of the in-development PALMO database is the airfoil base cube. Each base cube includes simulation data parametrized over a range of Mach numbers, Reynolds numbers, and angles-of-attack. This first release of the database includes the NACA 4-series airfoils, with parametrization in airfoil thickness and camber from an NACA 0006 to an NACA 4424. In total, 52,480 NACA 4-series calculations were run on the NASA High-End Compute Capability (HECC) supercomputer and the corresponding airfoil performance coefficients are embedded in the Appendix of this document for public distribution. This provides high-order-accurate simulation data covering a wide range of aerospace design applications, which enables users to develop OVERFLOW-quality airfoil performance look-up tables without additional high-performance computing. In addition to engineering design and analysis of aerospace vehicles, PALMO is well suited to be a benchmark dataset for the development and testing of machine learning methods in aerospace engineering. Downstream surrogate models enable OVERFLOW- quality airfoil performance predictions for any arbitrary combination of camber, thickness, Mach number, Reynolds number, and angle-of-attack within the bounds of the database.

Database↗
DOE OSTI2023

Extreme γ –Ray Radiation Tolerance of Spectrometer–Grade CsPbBr 3 Perovskite Detectors

The perovskite compound CsPbBr 3 has recently been discovered as a promising room-temperature semiconductor radiation detector, offering an inexpensive and easy-to-manufacture alternative to the current benchmark material Cd 1–x Zn x Te (CZT). The performance of CsPbBr 3 sensors is evaluated under harsh conditions, such as high radiation doses often found in industrial settings and extreme radiation in space. Results show minimal degradation in detector performance after exposure to 1 Mrad of Co-60 gamma radiation, with no significant change to energy resolution or hole mobility and lifetime. Additionally, many of the devices are still functional after being exposed to a 10 Mrad dose over 3 days, and those that do not survive can still be refabricated into working detectors. Notably, these results suggest that the failure mode in these devices is likely related to the interface between the electrode and material and their reaction, or the electrode itself and not the material itself. Overall, the study suggests that CsPbBr 3 has high potential as a reliable and efficient radiation detector in various applications, including those involving extreme fluxes and energies of gamma-ray radiation.

36 MATERIALS SCIENCE↗
DOE OSTI2024

A high-performance intermediate temperature reversible solid oxide cell with a new barrier layer free oxygen electrode

The best solution to address the critical durability issue of solid oxide electrolytic cells (SOECs) for high-efficiency and high-rate H 2 production is to lower the operating temperature without sacrificing the performance. Developing high performance oxygen electrodes (OEs) is a key to capitalizing this solution. Here, in this paper, we report on a highly active OE for intermediate temperature ZrO 2 -based SOECs without a CeO 2 barrier layer. The new barrier-layer-free (BLF) OE is a composite of two materials, (Bi 0.75 Y 0.25 ) 0.93 Ce 0.07 O 1.5±δ (BYC) that exhibits high oxide-ion conductivity and La 0.8 Sr 0.2 MnO 3 (LSM) that possesses a high electronic conductivity to enable fast oxygen reduction/evolution reactions (ORR/OER). Featuring a microscale porous BYC scaffold decorated with high surface area LSM nanoparticles (NPs), the new BLF-OE exhibited a low area specific resistance (ASR) of 0.10 Ω cm 2 at 650 °C in air. With 50%H 2 -50%H 2 O as a feed to hydrogen electrode (HE) and air to OE, the single cell performance achieved 588 mA cm -2 at 0.80 V in the fuel cell mode and 688 mA cm -2 at 1.30 V in the electrolytic mode at 650 °C. Our in-house testing showed that this level of performance was ~3.5x higher than the cell with the benchmark La 0.6 Sr 0.4 Co 0.2 Fe 0.8 O 3-δ -Ce 0.9 Gd 0.1 O 2-δ OE. The long-term durability testing under alternating fuel cell and electrolytic modes showed a low degradation rate of 0.10 mA cm -2 h -1 over 550 hours. These encouraging results showed the great promise of the newly developed BYC-LSM to be an excellent OE candidate for intermediate temperature SOECs.

25 ENERGY STORAGE↗
DOE OSTI2025

Implementation of a Mesh refinement algorithm into the quasi-static PIC code QuickPIC

Plasma-based acceleration (PBA) has emerged as a promising candidate for the accelerator technology used to build a future linear collider and/or an advanced light source. In PBA, a trailing or witness particle beam is accelerated in the plasma wave wakefield (WF) created by a laser or particle beam driver. The WF is often nonlinear and involves the crossing of plasma particle trajectories in real space and thus particle-in-cell methods are used. The distance over which the drive beam evolves is several orders of magnitude larger than the wake wavelength. This large disparity in length scales is amenable to the quasi-static approach. Three-dimensional (3D), quasi-static (QS), particle-in-cell (PIC) codes, e.g., QuickPIC, have been shown to provide high fidelity simulation capability with 2-4 orders of magnitude speedup over 3D fully explicit PIC codes. In PBA, the witness beam needs to be matched to the focusing forces of the WF to reduce the emittance growth. In some linear collider designs, the matched spot size of the witness beam can be 2 to 3 orders of magnitude smaller than the spot size (and wavelength) of the wakefield. Such an additional disparity in length scales is ideal for mesh refinement where the WF within the witness beam is described on a finer mesh than the rest of the WF. A mesh refinement scheme is described that has been implemented into the 3D QS PIC code, QuickPIC. Very fine (high) resolution is used in a small spatial region that includes the witness beam and progressively coarser resolutions in the rest of the simulation domain. A fast multigrid Poisson solver has been implemented for the field solve on the refined meshes and a Fast Fourier Transform (FFT) based Poisson solver is used for the coarse mesh. The code has been parallelized with both MPI and OpenMP, and the parallel scalability has also been improved by using pipelining. A preliminary adaptive mesh refinement technique is described to optimize the computational time for simulations with an evolving witness beam size. Several test problems are used to verify that the mesh refinement algorithm provides accurate results. Additionally, the results are benchmarked against highly resolved simulations exhibiting near-azimuthal symmetry, performed using QPAD—a novel hybrid QS PIC code that uses a PIC description in the coordinates (r, ct – z) and a gridless description in the azimuthal angle, Φ.

Linear collider↗
DOE OSTI2021

Probabilistic Deep Learning Approach to Automate the Interpretation of Multi-phase Diffraction Spectra

Autonomous synthesis and characterization of inorganic materials requires the automatic and accurate analysis of X-ray diffraction spectra. For this task, we designed a probabilistic deep learning algorithm to identify complex multi-phase mixtures. At the core of this algorithm lies an ensemble convolutional neural network trained on simulated diffraction spectra, which are systematically augmented with physics-informed perturbations to account for artifacts that can arise during experimental sample preparation and synthesis. In this work, larger perturbations associated with off-stoichiometry are also captured by supplementing the training set with hypothetical solid solutions. Spectra containing mixtures of materials are analyzed with a newly developed branching algorithm that utilizes the probabilistic nature of the neural network to explore suspected mixtures and identify the set of phases that maximize confidence in the prediction. Our model is benchmarked on simulated and experimentally measured diffraction spectra, showing exceptional performance with accuracies exceeding those given by previously reported methods based on profile matching and deep learning. We envision that the algorithm presented here may be integrated in experimental workflows to facilitate the high-throughput and autonomous discovery of inorganic materials.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Cholesky Decomposition-Based Implementation of Relativistic Two-Component Coupled-Cluster Methods for Medium-Sized Molecules

A Cholesky decomposition (CD)-based implementation of relativistic two-component coupled-cluster (CC) and equation-of-motion CC (EOM-CC) methods using an exact two-component Hamiltonian augmented with atomic-mean-field integrals (the X2CAMF scheme) is reported. Furthermore, the present CD-based implementation of X2CAMF-CC and EOM-CC methods employs atomic-orbital-based algorithms to avoid the construction of two-electron integrals and intermediates involving three and four virtual indices. The CD-based implementation extends the applicability of X2CAMF-CC and EOMCC methods to medium-sized molecules with the correlation of around 1000 spinors. Benchmark calculations for uranium-containing small molecules have been performed to assess the dependence of CC results with respect to the Cholesky threshold. A Cholesky threshold of 10 –4 is shown to maintain chemical accuracy. Example calculations to illustrate the capability of the CD-based relativistic CC methods are reported for the bond dissociation energy of the uranium hexafluoride molecule, UF 6 , with up to quadruple-zeta basis sets and the lowest excitation energy in solvated uranyl ion [UO 2 2+ (H 2 O) 12 ].

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Quantum adiabatic machine learning by zooming into a region of the energy surface

Recent work has shown that quantum annealing for machine learning, referred to as QAML, can perform comparably to state-of-the-art machine learning methods with a specific application to Higgs boson classification. Here, we propose QAML-Z, an algorithm that iteratively zooms in on a region of the energy surface by mapping the problem to a continuous space and sequentially applying quantum annealing to an augmented set of weak classifiers. Results on a programmable quantum annealer show that QAML-Z matches classical deep neural network performance at small training set sizes and reduces the performance margin between QAML and classical deep neural networks by almost 50% at large training set sizes, as measured by area under the receiver operating characteristic curve. The significant improvement of quantum annealing algorithms for machine learning and the use of a discrete quantum algorithm on a continuous optimization problem both opens a class of problems that can be solved by quantum annealers and suggests the approach in performance of near-term quantum machine learning towards classical benchmarks.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Conceptual design of a high reactive-power ferroelectric fast reactive tuner

We present a novel design of a ferroelectric fast reactive tuner (FE-FRT) capable of modulating mega-VAR reactive power on a submicrosecond timescale. The high reactive power capability of our design extends the range of applications of reactive tuners to numerous applications. We present a detailed analytical model of the performance of a megawatt-class reactive power device and benchmark it against finite-element method eigenmode and frequency domain electromagnetic simulations. We introduce new features, including an annulus design for the ferroelectric capacitors and capacitive window coupling to the cavity. We consider thermal design issues and nonlinear effects in the ferroelectric. The model covers several configurations, allowing control of the frequency of superconducting and normal-conducting cavities in a variety of applications and frequencies. We calculate that the FE-FRT designed should be capable of handling around 0.45 MVAR of reactive power with around 3 kW of resistive losses, providing a frequency tuning range of 8 kHz in an example of 400 MHz cavity geometry. Published by the American Physical Society 2024

43 PARTICLE ACCELERATORS↗
DOE OSTI2025

Link Scheduling in Satellite Networks via Machine Learning Over Riemannian Manifolds

Low Earth Orbit (LEO) satellites play a crucial role in enhancing global connectivity, serving a complementary solution to existing terrestrial systems. In wireless networks, scheduling is a vital process that allocates time-frequency resources to users for interference management. However, LEO satellite networks face significant challenges in scheduling their links towards ground users due to the satellites’ mobility and overlapping coverage. This paper addresses the dynamic link scheduling problem in LEO satellite networks by considering spatio-temporal correlations introduced by the satellites’ movements. The first step in the proposed solution involves modeling the network over Riemannian manifolds, thanks to their representation as symmetric positive definite matrices. We introduce two machine learning (ML)-based link scheduling techniques that model the dynamic evolution of satellite positions and link conditions over time and space. To accurately predict satellite link states, we present a recurrent neural network (RNN) over Riemannian manifolds, which captures spatio-temporal characteristics over time. Furthermore, we introduce a separate model, the convolutional neural network (CNN) over Riemannian manifolds, which captures geometric relationships between satellites and users by extracting spatial features from the network topology across all links. Simulation results demonstrate that both RNN and CNN over Riemannian manifolds deliver comparable performance to the fractional programming-based link scheduling (FPLinQ) benchmark. Remarkably, unlike other ML-based models that require extensive training data, both models only need 30 training samples to achieve over 99% of the sum rate while maintaining similar computational complexity relative to the benchmark.

42 ENGINEERING↗
DOE OSTI2023

RX-ADS: Interpretable Anomaly Detection Using Adversarial ML for Electric Vehicle CAN Data

Recent year has brought considerable advancements in Electric Vehicles (EVs) and associated infrastructures/ communications. Intrusion Detection Systems (IDS) are widely deployed for anomaly detection in such critical infrastructures. This paper presents an Interpretable Anomaly Detection System (RX-ADS) for intrusion detection in CAN protocol communication in EVs. Contributions include: 1) window based feature extraction method; 2) deep Autoencoder based anomaly detection method; and 3) adversarial machine learning based explanation generation methodology. The presented approach was tested on two benchmark CAN datasets: OTIDS and Car Hacking. The anomaly detection performance of RX-ADS was compared against the state-of-the-art approaches on these datasets: HIDS and GIDS. The RX-ADS approach presented performance comparable to the HIDS approach (OTIDS dataset) and has outperformed HIDS and GIDS approaches (Car Hacking dataset). Further, the proposed approach was able to generate explanations for detected abnormal behaviors arising from various intrusions. Furthermore, these explanations were later validated by information used by domain experts to detect anomalies. Other advantages of RX-ADS include: 1) the method can be trained on unlabeled data; 2) explanations help experts in understanding anomalies and root course analysis, and also help with AI model debugging and diagnostics, ultimately improving user trust in AI systems.

42 ENGINEERING↗
DOE OSTI2020

Dealing with Measurement Uncertainties as Nuisance Parameters in Bayesian Model Calibration

In the presence of model discrepancy, the calibration of physics-based models for physical parameter inference is a challenging problem. Lack of identifiability between calibration parameters and model discrepancy requires additional identifiability constraints to be placed on the model discrepancy to obtain unique physical parameter estimates. If these assumptions are violated, the inference for the calibration parameters can be systematically biased. In many applications, such as in dynamic material property experiments, many of the calibration inputs refer to measurement uncertainties. Here, we develop a metric for identifying overfitting of these measurement uncertainties, propose a prior capable of reducing this overfitting, and show how this leads to a diagnostic tool for validation of physical parameter inference. The approach is demonstrated for a benchmark example and applied for a material property application to perform inference on the equation of state parameters of tantalum.

97 MATHEMATICS AND COMPUTING↗