Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “building benchmarking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Key Questions for the Quantum Machine Learner to Ask Themselves

Within the last several years quantum machine learning (QML) has begun to mature; however, many open questions remain. Rather than review open questions, in this perspective piece I will discuss my view about how we should approach problems in QML. In particular I will list a series of questions that I think we should ask ourselves when developing quantum algorithms for machine learning. These questions focus on what the definition of quantum ML is, what is the proper quantum analogue of QML algorithms is, how one should compare QML to traditional ML and what fundamental limitations emerge when trying to build QML protocols. As an illustration of this process I also provide information theoretic arguments that show that amplitude encoding can require exponentially more queries to a quantum model to determine membership of a vector in a concept class than classical bit-encodings would require; however, if the correct analogue is chosen then both the quantum and classical complexities become polynomially equivalent. This example underscores the importance of asking ourselves the right questions when developing and benchmarking QML algorithms.

Wiebe, Nathan O.↗
DOE OSTI2024

Trace Adsorptive Removal of PFAS from Water by Optimizing the UiO‐66 MOF Interface

Abstract The confluence of pervasiveness, bioaccumulation, and toxicity in freshwater contaminants presents an environmental threat second to none. Exemplifying this trifecta, per‐ and polyfluoroalkyl substances (PFAS) present an alarming hazard among the emerging contaminants. State‐of‐the‐art PFAS adsorbents used in drinking water treatment, namely, activated carbons and ion‐exchange resins, are handicapped by low adsorption capacity, competitive adsorption, and/or slow kinetics. To overcome these shortcomings, metal–organic frameworks (MOFs) with tailored pore size, surface, and pore chemistry are promising alternatives. Thanks to the compositional modularity of MOFs and polymer–MOF composites, herein we report on a series of water‐stable zirconium carboxylate MOFs and their low‐cost polymer‐grafted composites as C 8 –PFAS adsorbents with benchmark kinetics and “parts per billion” removal efficiencies. Bespoke insights into the structure–function relationships of PFAS adsorbents are obtained by leveraging interfacial design principles on solid sorbents, creating a synergy between the extrinsic particle surfaces and intrinsic molecular building blocks.

Ilić, Nebojša↗
DOE OSTI2024

Comparative Study of Large Language Model Architectures on Frontier

Large language models (LLMs) have garnered significant attention in both the AI community and beyond. Among these, the Generative Pre-trained Transformer (GPT) has emerged as the dominant architecture, spawning numerous variants. However, these variants have undergone pre-training under diverse conditions, including variations in input data, data preprocessing, and training methodologies, resulting in a lack of controlled comparative studies. Here we meticulously examine two prominent open-sourced GPT architectures, GPT-NeoX and LLaMA, leveraging the computational power of Frontier, the world’s first Exascale supercomputer. Employing the same materials science text corpus and a comprehensive end-to-end pipeline, we conduct a comparative analysis of their training and downstream performance. Our efforts culminate in achieving state-of-the-art performance on a challenging materials science benchmark. Furthermore, we investigate the computation and energy efficiency, and propose a computationally efficient method for architecture design. To our knowledge, these pre-trained models represent the largest available for materials science. Our findings provide practical guidance for building LLMs on HPC platforms.

Yin, Junqi↗
NASA NTRS2015

Cloud-Based Numerical Weather Prediction for Near Real-Time Forecasting and Disaster Response

The use of cloud computing resources continues to grow within the public and private sector components of the weather enterprise as users become more familiar with cloud‐computing concepts, and competition among service providers continues to reduce costs and other barriers to entry. Cloud resources can also provide capabilities similar to high‐performance computing environments, supporting multi‐node systems required for near real‐time, regional weather predictions. Referred to as "Infrastructure as a Service", or IaaS, the use of cloud-based computing hardware in an on‐demand payment system allows for rapid deployment of a modeling system in environments lacking access to a large, supercomputing infrastructure. Use of IaaS capabilities to support regional weather prediction may be of particular interest to developing countries that have not yet established large supercomputing resources, but would otherwise benefit from a regional weather forecasting capability. Recently, collaborators from NASA Marshall Space Flight Center and Ames Research Center have developed a scripted, on‐demand capability for launching the NOAA/NWS Science and Training Resource Center (STRC) Environmental Modeling System (EMS), which includes pre‐compiled binaries of the latest version of the Weather Research and Forecasting (WRF) model. The WRF‐EMS provides scripting for downloading appropriate initial and boundary conditions from global models, along with higher‐resolution vegetation, land surface, and sea surface temperature data sets provided by the NASA Short‐term Prediction Research and Transition (SPoRT) Center. This presentation will provide an overview of the modeling system capabilities and benchmarks performed on the Amazon Elastic Compute Cloud (EC2) environment. In addition, the presentation will discuss future opportunities to deploy the system in support of weather prediction in developing countries supported by NASA's SERVIR Project, which provides capacity building activities in environmental monitoring and prediction across a growing number of regional hubs throughout the world. Capacity‐building applications that extend numerical weather prediction to developing countries are intended to provide near real‐time applications to benefit public health, safety, and economic interests, but may have a greater impact during disaster events by providing a source for local predictions of weather‐related hazards, or impacts that local weather events may have during the recovery phase.

Molthan, Andrew↗
DOE OSTI2026

Building Energy Analysis of Manufactured and Multifamily Housing Types in Juneau, Alaska

This report details the results of building energy modeling analysis evaluating the potential energy savings, economic outcomes, and grid-level electricity reduction associated with cold climate air source heat pump (ccASHP) adoption across multifamily and manufactured housing (MMFH) building typologies in the City and Borough of Juneau (CBJ). Three building archetypes were evaluated: multifamily 4-plex apartments, multifamily 8-plex apartments, and manufactured housing units. Building energy models were developed using OpenStudio-HPXML and calibrated to actual utility consumption data and local meteorological data from the Juneau International Airport weather station using an automated calibration tool following the BPI-2400-S-2015 v.2 standard for model calibration. Occupant behaviors present the greatest variability in successful calibrations. Calibrated models were benchmarked against a baseline electric resistance heating condition, with the selected ccASHP modeled as the retrofit condition and typical meteorological year weather data for all results generation.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2020

Stable and radioactive carbon isotope partitioning in soils and saturated systems: a reactive transport modeling benchmark study

This benchmark provides the first rigorous test of a three-isotope system [ 12 C, 13 C, and 14 C] subject to the combined effects of radioactive decay and both stable equilibrium and kinetic fractionation. We present a series of problems building in complexity based on the cycling of carbon in both organic and inorganic forms. The key components implement (1) equilibrium fractionation between multiple coexisting carbon species as a function of pH, (2) radioactive decay of radiocarbon with associated mass-dependent speciation demonstrating appropriate correction of the Δ 14 C value in agreement with reporting convention, and (3) kinetic stable isotope fractionation due to the oxidation of organic carbon to inorganic forms as a function of time and space in an open, through-flowing system. Participating RTM codes are CrunchTope, ToughReact, Hytec, and The Geochemist’s Workbench. Across all problem levels, simulation results from all RTMs demonstrate good agreement.

58 GEOSCIENCES↗
DOE OSTI2026

Bayesian discovery of optimal reduced order models from mechanistic and experimental data: A case study of Pd penetration in TRISO fuels using BISON

TRistructural ISOtropic (TRISO) particles rely on a silicon carbide (SiC) layer as the primary structural material and barrier to metallic fission products (FPs) release. Accurate prediction of palladium (Pd) transport and penetration is therefore critical for qualifying TRISO fuels for advanced reactors. The empirical correlation for Pd penetration in BISON is derived from historical particle-fuel data, but cannot explain the large scatter in the experimental data that arises from varying experimental conditions. To aid fuel qualification, we previously developed a mechanistic reduced order model (ROM) using BISON that resolves these dependencies. Here, in this work we build on that mechanistic ROM and perform validation and quantify its uncertainty using Bayesian uncertainty quantification (UQ). calibration against a suite of in-pile and out-of-pile experiments spanning particle compositions, geometries, and operating conditions, and we benchmark it against the empirical correlation. Bayesian UQ identifies influential parameters, calibrates them to data, and yields predictive intervals. Results show that while the empirical correlation can be tuned to fit a single experiment type, it transfers poorly; the mechanistic ROM sustains accuracy with credible uncertainty across disparate conditions. This demonstrates a practical path—via Bayesian UQ applied to mechanistic ROMs—to leverage single-effect experiments for inferring in-reactor behavior and supporting TRISO fuel qualification.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2026

Performance-Aligned LLMs for Generating Fast HPC Code

Optimizing scientific software is a difficult task because codebases are often large and complex, and performance can depend upon several factors including the algorithm, its implementation, and hardware among others. Causes of poor performance can originate from disparate sources and be difficult to diagnose. Recent years have seen a multitude of work that use large language models (LLMs) to assist in software development tasks. However, these tools are trained to model the distribution of code as text, and are not specifically designed to understand performance aspects of code. In this work, we introduce a reinforcement learning based methodology to align the outputs of code LLMs with performance. This allows us to build upon the current code modeling capabilities of LLMs and extend them to generate better performing code. Here, we demonstrate that our fine-tuned model improves the expected speedup of generated code over base models for a set of benchmark tasks from 0.9 to 1.6 for serial code and 1.9 to 4.5 for OpenMP parallel code.

Computer science↗
DOE OSTI2024

Exploring Li-Ion Transport Properties of Li 3 TiCl 6 : A Machine Learning Molecular Dynamics Study

We performed large-scale molecular dynamics simulations based on a machine-learning force field (MLFF) to investigate the Li-ion transport mechanism in cation-disordered Li 3 TiCl 6 cathode at six different temperatures, ranging from 25°C to 100°C. In this work, deep neural network method and data generated by ab − initio molecular dynamics (AIMD) simulations were deployed to build a high-fidelity MLFF. Radial distribution functions, Li-ion mean square displacements (MSD), diffusion coefficients, ionic conductivity, activation energy, and crystallographic direction-dependent migration barriers were calculated and compared with corresponding AIMD and experimental data to benchmark the accuracy of the MLFF. From MSD analysis, we captured both the self and distinct parts of Li-ion dynamics. The latter reveals that the Li-ions are involved in anti-correlation motion that was rarely reported for solid-state materials. Similarly, the self and distinct parts of Li-ion dynamics were used to determine Haven’s ratio to describe the Li-ion transport mechanism in Li 3 TiCl 6 . Obtained trajectory from molecular dynamics infers that the Li-ion transportation is mainly through interstitial hopping which was confirmed by intra- and inter-layer Li-ion displacement with respect to simulation time. Ionic conductivity (1.06 mS/cm) and activation energy (0.29eV) calculated by our simulation are highly comparable with that of experimental values. Overall, the combination of machine-learning methods and AIMD simulations explains the intricate electrochemical properties of the Li 3 TiCl 6 cathode with remarkably reduced computational time. Thus, our work strongly suggests that the deep neural network-based MLFF could be a promising method for large-scale complex materials.

Selvaraj, Selva Chandrasekaran (ORCID:000000029023↗
DOE OSTI2025

Gerischer Electrochemistry Today

Semiconductor photoelectrochemistry is a dynamic and interdisciplinary field at the forefront of research in solar fuels, energy conversion, and catalysis. Here, this Perspective captures the collective insights from the second Gerischer Electrochemistry Today Symposium, held at Colorado State University in Fort Collins, CO, in August 2024, which convened leading researchers, early-career scientists, and industry partners to define the critical next steps for the field. Through interactive sessions, technical talks, panel discussions, and training initiatives─including a Semiconductor Electrochemistry Bootcamp─the symposium emphasized three pillars of advancement: (i) facilitating the exchange of new ideas in semiconductor electrochemistry and charge separation; (ii) fostering the development of future researchers, research topics, and participation in the semiconductor workforce; and (iii) building community. This Energy Focus distills key themes from the meeting and identifies major knowledge gaps in the following areas: mechanisms of charge separation and recombination, role of defects and disorder, dynamic and operando characterization methods, interfacial chemistry and surface passivation, theoretical and modeling limitations, and standardization and benchmarking. The inclusive and collaborative structure of the symposium enabled the generation of this comprehensive report that will serve as a roadmap for fundamental and applied research in the rapidly evolving field of semiconductor electrochemistry over the next decade.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Phase Composition and Phase Transformation of Additively Manufactured Nickel Alloy 718 AM Bench Artifacts

Additive manufacturing (AM) technologies offer unprecedented design flexibility but are limited by a lack of understanding of the material microstructure formed under their extreme and transient processing conditions and its subsequent transformation during post-build processing. As part of the 2022 AM Bench Challenge, sponsored by the National Institute of Standards and Technology, this study focuses on the phase composition and phase evolution of AM nickel alloy 718, a nickel-based superalloy, to provide benchmark data essential for the validation of computational models for microstructural predictions. Here, we employed high-energy synchrotron X-ray diffraction, in situ synchrotron X-ray scattering, as well as high-resolution transmission electron microscopy for our analyses. The study uncovers critical aspects of the microstructure in its as-built state, its transformation during homogenization, and its phase evolution during subsequent aging heat treatment. Specifically, we identified secondary phases, monitored the dissolution and coarsening of microstructural elements, and observed the formation and stability of γ ’ and γ ” phases. The results provide the rigorous benchmark data required to understand the atomic and microstructural transformations of AM nickel alloy 718, thereby enhancing the reliability and applicability of AM models for predicting phase evolution and mechanical properties.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Machine Learning-enabled Scalable Performance Prediction of Scientific Codes

Hardware architectures become increasingly complex as the compute capabilities grow to exascale. Here, we present the Analytical Memory Model with Pipelines (AMMP) of the Performance Prediction Toolkit (PPT). PPT-AMMP takes high-level source code and hardware architecture parameters as input and predicts runtime of that code on the target hardware platform, which is defined in the input parameters. PPT-AMMP transforms the code to an (architecture-independent) intermediate representation, then (i) analyzes the basic block structure of the code, (ii) processes architecture-independent virtual memory access patterns that it uses to build memory reuse distance distribution models for each basic block, and (iii) runs detailed basic-block level simulations to determine hardware pipeline usage. PPT-AMMP uses machine learning and regression techniques to build the prediction models based on small instances of the input code, then integrates into a higher-order discrete-event simulation model of PPT running on Simian PDES engine. We validate PPT-AMMP on four standard computational physics benchmarks and present a use case of hardware parameter sensitivity analysis to identify bottleneck hardware resources on different code inputs. We further extend PPT-AMMP to predict the performance of a scientific application code, namely, the radiation transport mini-app SNAP. To this end, we analyze multi-variate regression models that accurately predict the reuse profiles and the basic block counts. We validate predicted SNAP runtimes against actual measured times.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Emerging Flexible Designs for Geospatial Multimodal Foundation Models

Foundation models are rapidly transforming Earth observation by enabling scalable pretraining across diverse unlabeled geospatial modalities. However, their architectural diversity—ranging from encoder-only to encoder-decoder and masked autoencoding paradigms—makes it challenging to assess performance trade-offs in a consistent manner. In this work, we present an apples-to-apples comparison of leading FM architectures designed for geospatial multimodal reasoning, with a particular focus on flexibility across varied spectral band configurations. We standardize pretraining using identical self-supervised learning objectives and training datasets, and evaluate all models under consistent parameterization on the GEOBench benchmark across classification and segmentation tasks. Our results offer new insights into the design trade-offs between model flexibility, modality alignment, and downstream task performance. By highlighting architectural strengths and limitations under controlled conditions, this study provides practical guidance for building next-generation geospatial foundation models capable of robust multimodal reasoning.

Ambrozio Dias, Philipe [ORNL] (ORCID:0000000194277↗
DOE OSTI2020

A Framework for the Analysis of Compiler Optimizations

Compilers transform program source code to machine executable code. During this transformation, they perform a number of compiler optimizations to improve the performance of the generated executable code. Importantly, applying those optimizations depends on the source code structure, such as the parallel programming model used to parallelize an algorithm. Often, implementations of the same algorithm with different programming models have vastly different performance because the compiler optimized them differently. We create FAROS, a framework to structure and automate the analysis of compiler optimizations on programs. FAROS automates the building process, execution profiling, and analysis of compiler optimization of programs, through a configuration interface. It outputs compiler optimization reports to show which optimizations applied to which line of source code, leveraging compilation remarks output by the compiler. Also, FAROS supports benchmarking performance of different program versions by collecting execution time results. In this first release of FAROS, we provide a configuration file to analyze compiler optimization differences for sequential vs OpenMP compilation, including 38 programs consisting of HPC proxy/mini/large applications, and NAS and Rodina kernels for analysis.

Georgakoudis, Giorgis↗
DOE OSTI2020

Algorithmic and GPU enhancements for molecular dynamics in Cabana and LAMMPS

The focus of this milestone was to implement and test new algorithms and optimizations for improved GPU performance by molecular dynamics (MD) and related particle codes. The work was done in the CoPA Cabana library, the CabanaMD mini-app which uses Cabana to implement and benchmark prototypical MD models, and the LAMMPS MD code which is used by the ECP EXAALT project as well as for materials modeling generally. Efforts included (a) implementation of new or optimized algorithms for fundamental MD kernels like neighbor-list building and FFTs which are also widely used in other particle methods, (b) performance testing of data structure layouts for both simple pairwise and computationally intense neural network potentials, (c) optimizations aimed at improving GPU performance for models with small atom counts, and (d) optimizations specific to the SNAP interatomic potential due to its importance for the EXAALT project.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Structural Changes in Metal Chalcogenide Nanoclusters Associated with Single Heteroatom Incorporation

Atomically precise nanoclusters (NCs) are promising building blocks for designing materials and interfaces with unique properties. By incorporating heteroatoms into the core, the electronic and magnetic properties of NCs can be precisely tuned. To accurately predict these properties, density functional theory (DFT) is often employed, making the rigorous benchmarking of DFT results particularly important. In this study, we present a benchmarking approach based on metal chalcogenide NCs as a model system. We synthesized a series of bimetallic, iron-cobalt chalcogenide NCs [Co 6-x Fe x S 8 (PEt 3 ) 6 ] + (x = 0-6) (PEt = triethyl phosphine) and investigated the effect of heteroatoms in the octahedral metal chalcogenide core on their size and electronic properties. Using ion mobility-mass spectrometry (IM-MS), we observed a gradual increase in the collision cross section (CCS) with an increase in the number of Fe atoms in the core. DFT calculations combined with trajectory method CCS simulations successfully reproduced this trend, revealing that the increase in cluster size is primarily due to changes in metal-ligand bond lengths, while the electronic properties of the core remain largely unchanged. Moreover, this method allowed us to exclude certain multiplicity states of the NCs, as their CCS values were significantly different from those predicted for the lowest-energy structures. Here, this study demonstrates that gas-phase IM-MS is a powerful technique for detecting subtle size differences in atomically precise NCs, which are often challenging to observe using conventional NC characterization methods. Accurate CCS measurements are established as a benchmark for comparison with theoretical calculations. The excellent correspondence between experimental data and theoretical predictions establishes a robust foundation for investigating structural changes of transition metal NCs of interest to a broad range of applications.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Quantitative assessment of fitting errors associated with streak camera noise in Thomson scattering data analysis

Thomson scattering measurements in high energy density experiments are often recorded using optical streak cameras. In the low-signal regime, noise introduced by the streak camera can become an important and sometimes the dominant source of measurement uncertainty. In this paper, we present a formal method of accounting for the presence of streak camera noise in our measurements. We present a phenomenological description of the noise generation mechanisms and present a statistical model that may be used to construct the covariance matrix associated with a given measurement. This model is benchmarked against simulations of streak camera images. We demonstrate how this covariance may then be used to weight fitting of the data and provide quantitative assessments of the uncertainty in the fitting parameters determined by the best fit to the data and build confidence in the ability to make statistically significant measurements in the low-signal regime, where spatial correlations in the noise become apparent. These methods will have general applicability to other measurements made using optical streak cameras.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

Scalable Predictive Control and Optimization for Grid Integration of Large-Scale Distributed Energy Resources

Integrating a large number of distributed energy resources (DERs) into the power grid needs a scalable power balancing method. We formulate the power balancing problem as a look-ahead optimization problem to be solved sequentially by a power distribution system aggregator based on a model predictive control (MPC) framework. Solving large-scale look-ahead control problems requires proper configuration of the control steps. In this paper, to solve large-scale control problems, we propose a variable time granularity where control time steps nearby the current control step have finer resolutions. The aggregator objective includes maximization of power production revenue and minimization of power purchasing expense, renewable power curtailment, and mileage costs for energy storage and electric vehicle (EV) charging stations while satisfying system capacity and operational constraints. The control problem is formulated as a mixed-integer linear program (MILP) and solved using the XpressMP solver. We perform simulations considering a copper plate representation of a large distribution network consisting of 2507 devices (control-lable DERs), including curtailable photovoltaics (PVs), energy storage batteries, EV charging stations, and buildings with heating, ventilation, and air conditioning units (HVACs). We show the effectiveness of the proposed approach in managing DERs interactively for maximum energy trading profit and local supply-demand power balancing. Finally, we demonstrate that the proposed method outperforms other benchmark controllers regarding computation time without compromising operational performance.

DER↗