Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Benchmarking Software”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

PyApprox: A software package for sensitivity analysis, Bayesian inference, optimal experimental design, and multi-fidelity uncertainty quantification and surrogate modeling

PyApprox is a Python-based one-stop-shop for probabilistic analysis of numerical models such as those used in the earth, environmental and engineering sciences. Easy to use and extendable tools are provided for constructing surrogates, sensitivity analysis, Bayesian inference, experimental design, and forward uncertainty quantification. The algorithms implemented represent a wide range of methods for model analysis developed over the past two decades, including recent advances in multi-fidelity approaches that use multiple model discretizations and/or simplified physics to significantly reduce the computational cost of various types of analyses. An extensive set of Benchmarks from the literature is also provided to facilitate the easy comparison of new or existing algorithms for a wide range of model analyses. Here, this paper introduces PyApprox and its various features, and presents results demonstrating the utility of PyApprox on a benchmark problem modeling the advection of a tracer in groundwater.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2017

Design and Benchmarking of a Network-In-the-Loop Simulation for Use in a Hardware-In-the-Loop System

Distributed engine control (DEC) systems alter aircraft engine design constraints because of fundamental differences in the input and output communication between DEC and centralized control architectures. The change in the way communication is implemented may create new optimum engine-aircraft configurations. This paper continues the exploration of digital network communication by demonstrating a Network-In-the-Loop simulation at the NASA Glenn Research Center. This simulation incorporates a real-time network protocol, the Engine Area Distributed Interconnect Network Lite (EADIN Lite), with the Commercial Modular Aero-Propulsion System Simulation 40k (C-MAPSS40k) software. The objective of this study is to assess digital control network impact to the control system. Performance is evaluated relative to a truth model for large transient maneuvers and a typical flight profile for commercial aircraft. Results show that a decrease in network bandwidth from 250 Kbps (sampling all sensors every time step) to 40 Kbps, resulted in very small differences in control system performance.

Networked Systems↗
NASA NTRS2017

Design and Benchmarking of a Network-In-the-Loop Simulation for Use in a Hardware-In-the-Loop System

Distributed engine control (DEC) systems alter aircraft engine design constraints be- cause of fundamental differences in the input and output communication between DEC and centralized control architectures. The change in the way communication is implemented may create new optimum engine-aircraft configurations. This paper continues the exploration of digital network communication by demonstrating a Network-In-the-Loop simulation at the NASA Glenn Research Center. This simulation incorporates a real-time network protocol, the Engine Area Distributed Interconnect Network Lite (EADIN Lite), with the Commercial Modular Aero-Propulsion System Simulation 40k (C-MAPSS40k) software. The objective of this study is to assess digital control network impact to the control system. Performance is evaluated relative to a truth model for large transient maneuvers and a typical flight profile for commercial aircraft. Results show that a decrease in network bandwidth from 250 Kbps (sampling all sensors every time step) to 40 Kbps, resulted in very small differences in control system performance.

Networked Systems↗
DOE OSTI2023

DeePMD-kit v2: A software package for deep potential models

DeePMD-kit is a powerful open-source software package that facilitates molecular dynamics simulations using machine learning potentials known as Deep Potential (DP) models. This package, which was released in 2017, has been widely used in the fields of physics, chemistry, biology, and material science for studying atomistic systems. The current version of DeePMD-kit offers numerous advanced features, such as DeepPot-SE, attention-based and hybrid descriptors, the ability to fit tensile properties, type embedding, model deviation, DP-range correction, DP long range, graphics processing unit support for customized operators, model compression, non-von Neumann molecular dynamics, and improved usability, including documentation, compiled binary packages, graphical user interfaces, and application programming interfaces. This article presents an overview of the current major version of the DeePMD-kit package, highlighting its features and technical details. Additionally, this article presents a comprehensive procedure for conducting molecular dynamics as a representative application, benchmarks the accuracy and efficiency of different models, and discusses ongoing developments.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Benchmark Dose Analysis of DNA Damage Biomarker Responses Provides Compound Potency and Adverse Outcome Pathway Information for the Topoisomerase II Inhibitor Class of Compounds

Genetic toxicology data have traditionally been utilized for hazard identification to provide a binary call for a compound's risk. Recent advances in the scientific field, especially with the development of high‐throughput methods to quantify DNA damage, have influenced a change of approach in genotoxicity assessment. The in vitro MultiFlow® DNA Damage Assay is one such method which multiplexes γH2AX, p53, phospho‐histone H3 biomarkers into a single‐flow cytometric analysis (Bryce et al., [2016]: Environ Mol Mutagen 57:546–558). This assay was used to study human TK6 cells exposed to each of eight topoisomerase II poisons for 4 and 24 hr. Using PROAST v65.5, the Benchmark Dose approach was applied to the resulting flow cytometric datasets. With “compound” serving as covariate, all eight compounds were combined into a single analysis, per time point and endpoint. The resulting 90% confidence intervals, plotted in Log scale, were considered as the potency rank for the eight compounds. The in vitro MultiFlow data showed a maximum confidence interval span of 1Log, which indicates data of good quality. Patterns observed in the compound potency rank were scrutinized by using the expert rule‐based software program Derek Nexus, developed by Lhasa Limited. Compound sub‐classification and structural alerts were considered contributory to the potencies observed for the topoisomerase II poisons studied herein. The Topo II poison Adverse Outcome Pathway was evaluated with MultiFlow endpoints serving as Key Events. The step‐wise approach described herein can be considered as a foundation for risk assessment of compounds within a specific mode of action of interest. Environ. Mol. Mutagen. 2020. © 2020 Wiley Periodicals, Inc.

Wheeldon, Ryan P.↗
DOE OSTI2021

Building Efficiency Targeting Tool for Energy Retrofits (BETTER) Web Application (BETTER Web App) v1.0

The Building Efficiency Targeting Tool for Energy Retrofits (BETTER) V.1.0 web application identifies cost-saving energy and emissions reductions in buildings and portfolios, without site visits or complex modeling. With minimal data entry, BETTER benchmarks a building's or portfolio's energy use against peers; quantifies energy, cost, and greenhouse gas (GHG) reduction potential; and recommends energy efficiency measures for individual buildings or portfolios, targeting specific energy savings levels. The source code of BETTER's modular, cross-platform analytical engine has previously been disclosed (2019-001) and is available on GitHub and can be adopted, redeveloped, and redistributed freely under an open-source license, allowing users to incorporate BETTER's analytical capabilities into their own software platforms and tools. The BETTER V.1.0 web application, developed with the Django web-framework and the Model-View-Controller (MVC) architecture being disclosed here, provides a graphical user interface (GUI) for any user to view the software tutorials, download and upload a data entry template, configure and run the BETTER analyses, and view the final analytical reports.

Szum, Carolyn↗
DOE OSTI2025

SineKAN: Kolmogorov-Arnold Networks using sinusoidal activation functions

Recent work has established an alternative to traditional multi-layer perceptron neural networks in the form of Kolmogorov-Arnold Networks (KAN). The general KAN framework uses learnable activation functions on the edges of the computational graph followed by summation on nodes. The learnable edge activation functions in the original implementation are basis spline functions (B-Spline). Here, we present a model in which learnable grids of B-Spline activation functions are replaced by grids of re-weighted sine functions (SineKAN). We evaluate numerical performance of our model on a benchmark vision task. We show that our model can perform better than or comparable to B-Spline KAN models and an alternative KAN implementation based on periodic cosine and sine functions representing a Fourier Series. Further, we show that SineKAN has numerical accuracy that could scale comparably to dense neural networks (DNNs). Compared to the two baseline KAN models, SineKAN achieves a substantial speed increase at all hidden layer sizes, batch sizes, and depths. Current advantage of DNNs due to hardware and software optimizations are discussed along with theoretical scaling. Additionally, properties of SineKAN compared to other KAN implementations and current limitations are also discussed.

Reinhardt, Eric↗
DOE OSTI2025

Lessons Learned and Scalability Achieved When Porting Uintah to DOE Exascale Systems

A key challenge faced when preparing codes for Department of Energy (DOE) exascale systems was designing scalable applications for systems featuring hardware and software not yet available at leadership-class scale. With such systems now available, it is important to evaluate scalability of the resulting software solutions on these target systems. One such code designed with the exascale DOE Aurora and DOE Frontier systems in mind is the Uintah Computational Framework, an open-source asynchronous many-task (AMT) runtime system. To prepare for exascale, Uintah adopted a portable MPI+X hybrid parallelism approach using the Kokkos performance portability library (i.e., MPI+Kokkos). This paper complements recent work with additional details and an evaluation of the resulting approach on Aurora and Frontier. Results are shown for a challenging benchmark demonstrating interoperability of 3 portable codes essential to Uintah-related combustion research. These results demonstrate single-source portability across Aurora and Frontier with scaling characteristics shown to 3,072 Aurora nodes and 9,216 Frontier nodes. In addition to showing results run to new scales on new systems, this paper also discusses lessons learned through efforts preparing Uintah for exascale systems.

Holmen, John [ORNL] (ORCID:0000000259342641)↗
DOE OSTI2024

Machine learning in materials research: Developments over the last decade and challenges for the future

The number of studies that apply machine learning (ML) to materials science has been growing at a rate of approximately 1.67 times per year over the past decade. In this review, I examine this growth in various contexts. First, I present an analysis of the most commonly used tools (software, databases, materials science methods, and ML methods) used within papers that apply ML to materials science. The analysis demonstrates that despite the growth of deep learning techniques, the use of classical machine learning is still dominant as a whole. It also demonstrates how new research can effectively build upon past research, particular in the domain of ML models trained on density functional theory calculation data. Next, I present the progression of best scores as a function of time on the matbench materials science benchmark for formation enthalpy prediction. In particular, a dramatic improvement of 7 times reduction in error is obtained when progressing from feature-based methods that use conventional ML (random forest, support vector regression, etc.) to the use of graph neural network techniques. Finally, I provide views on future challenges and opportunities, focusing on data size and complexity, extrapolation, interpretation, access, and relevance.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Berkeley Lab Finite Element Framework (BELFEM) v0.1

The software program, referred to as BELFEM, is a specialized finite element code designed for the magnetodynamic modeling of high-temperature superconducting (HTS) tapes. It incorporates novel mixed finite element formulations, particularly the h-ϕ-formulation with thin-shell simplification, to efficiently simulate larger geometries. This methodology is extremely promising for predicting the electrodynamic performance of HTS tapes used in superconducting cables and magnets, offering the benefit of reduced computational cost. Compared to similar technologies like COMSOL Multiphysics and GetDP, BELFEM's performance benchmarking indicates superior efficiency in its thin-shell implementation. In the future, it will also support features like thermal coupling and inter-tape current sharing, enhancing its utility in research and development, particularly in nuclear fusion applications. The intent is to develop BELFEM as a robust and efficient tool for the HTS community, contributing to the analysis and design of superconducting cables and magnets.

Messe, Christian↗
DOE OSTI2024

PyOED: An Extensible Suite for Data Assimilation and Model-Constrained Optimal Design of Experiments

This article describes PyOED, a highly extensible scientific package that enables developing and testing model-constrained optimal experimental design (OED) for inverse problems. Specifically, PyOED aims to be a comprehensive Python toolkit for model-constrained OED. The package targets scientists and researchers interested in understanding the details of OED formulations and approaches. It is also meant to enable researchers to experiment with standard and innovative OED technologies with a wide range of test problems (e.g., simulation models). OED, inverse problems (e.g., Bayesian inversion), and data assimilation (DA) are closely related research fields, and their formulations overlap significantly. Thus, PyOED is continuously being expanded with a plethora of Bayesian inversion, DA, and OED methods as well as new scientific simulation models, observation error models, and observation operators. These pieces are added such that they can be permuted to enable testing OED methods in various settings of varying complexities. The PyOED core is completely written in Python and utilizes the inherent object-oriented capabilities; however, the current version of PyOED is meant to be extensible rather than scalable. Specifically, PyOED is developed to “enable rapid development and benchmarking of OED methods with minimal coding effort and to maximize code reutilization.” This article provides a brief description of the PyOED layout and philosophy and provides a set of exemplary test cases and tutorials to demonstrate the potential of the package.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

ReactionMechanismSimulator.jl: A modern approach to chemical kinetic mechanism simulation and analysis

Abstract We present ReactionMechanismSimulator.jl (RMS), a modern differentiable software for the simulation and analysis of chemical kinetic mechanisms, including multiphase systems. RMS has already been applied to problems in combustion, pyrolysis, polymers, pharmaceuticals, catalysis, and electrocatalysis. RMS is written in Julia, making it easy to develop and allowing it to take advantage of Julia's extensive numerical computing ecosystem. In addition to its extensive library of optimized analytic Jacobians, RMS can generate and use Jacobians computed using automatic differentiation and symbolically generated analytic Jacobians. RMS is demonstrated to be faster than Cantera and Chemkin in several benchmarks. RMS also implements an extensive set of features for analyzing chemical mechanisms, including a library of easy‐to‐call plotting functions, molecular structure resolved flux diagram generation, crash analysis, traditional sensitivity analysis, transitory sensitivity analysis, and an automatic mechanism analysis toolkit. RMS implements efficient adjoint and parallel forward sensitivity analyses. We also demonstrate the ease of adding new features to RMS.

Johnson, Matthew S.↗
DOE OSTI2021

Thermal loading analysis of the ring injection dump for the Spallation Neutron Source facility

The ring injection dump (RID) is the largest beam dump in the Spallation Neutron Source facility at Oak Ridge National Laboratory and accepts a fraction of the beam from the ring that is not captured in the ring during injection. Thermo-fluid modeling of the RID components was performed during the initial design by using the software ANSYS CFX to characterize the power rating for the RID. From the calculations, the power rating was lowered from 200 kW to 150 kW, at 1.3 GeV beam energy due to concerns regarding the heating of the concrete structure. The Proton Power Upgrade project scope included reevaluating the power limit to the RID in anticipation that increasing it would provide more operational flexibility. The focus was on the shielding because it was the limiting factor, and data collected from thermocouples installed on the structure were used to benchmark the analysis. A three-stage process was adopted in the latest study. First, the already existing steady-state calculations were validated and extended with the latest Monte Carlo N-Particle source term calculations. Second, transient calculations were conducted to capture the dynamic state of the system in response to the energy dumped to the RID during the following operational period of the beam from 2005–2019 (14 years), and sensitivity analyses of crucial parameters were performed to benchmark the model with reasonable accuracy. Finally, an idealized transient cycle analysis was conducted with realistic duty factors to predict the temperature distribution at higher beam powers. Overall, the results confirmed the accuracy of the original steady state model but allowed the development of a new validated transient model with higher accuracy for future analyses.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Evaluating Unified Memory Performance in HIP

Heterogeneous unified memory management between a CPU and a GPU is a major challenge in GPU computing. Recently, unified memory (UM) has been supported by software and hardware components on AMD computing platforms. The support could simplify the complexities of memory management. In this paper, we attempt to have a better understanding of UM by evaluating the performance of UM programs on an AMD MI100 GPU. More specifically, we evaluate data migration using UM against other data transfer techniques for the overall performance of an application, assess the impacts of three commonly used optimization techniques on the kernel execution time of a vector add sample, and compare the performance and productivity of selected benchmarks with and without UM. The performance overhead associated with UM is not trivial, but it can improve programming productivity by reducing lines of code for scientific applications. We aim to present early results and feedback on the UM performance to the vendor.

Jin, Zheming↗
NASA NTRS2000

Automatic Generation of Directive-Based Parallel Programs for Shared Memory Parallel Systems

The shared-memory programming model is a very effective way to achieve parallelism on shared memory parallel computers. As great progress was made in hardware and software technologies, performance of parallel programs with compiler directives has demonstrated large improvement. The introduction of OpenMP directives, the industrial standard for shared-memory programming, has minimized the issue of portability. Due to its ease of programming and its good performance, the technique has become very popular. In this study, we have extended CAPTools, a computer-aided parallelization toolkit, to automatically generate directive-based, OpenMP, parallel programs. We outline techniques used in the implementation of the tool and present test results on the NAS parallel benchmarks and ARC3D, a CFD application. This work demonstrates the great potential of using computer-aided tools to quickly port parallel programs and also achieve good performance.

Jin, Hao-Qiang↗
DOE OSTI2025

Benchmarking Near-Surface Winds in the HRRR Analyses Using Multisource Observations over Complex Terrain in the Southeastern United States

Wind energy plays a crucial role in sustainable power generation, yet its full potential in the southeast United States (SEUS) remains underexplored. This study advances wind resource assessment in the SEUS using existing regional modeling and multisource observations. We find that terrain complexity, influenced by orography and forest canopy, significantly impacts the accuracy of modeled wind speed. While the High-Resolution Rapid Refresh (HRRR) model effectively simulates wind profiles over flat, nonforested terrain, larger errors are produced above forest canopies, and particularly for those stands in hilly and mountainous terrain where there are no observations aloft that could be used by HRRR’s data assimilation software. Seasonal variations, e.g., changes in the leaf area index, further complicate the relationship between terrain complexity and simulation errors. Here, our findings emphasize the critical need for comprehensive wind profile measurements, extending from below the canopy to height above the canopy and through the lower planetary boundary layer, to fully quantify model performance over and near forests, and for improving wind resource assessment, planning, and management. This study provides valuable insights not only for further model development but also for future field campaign deployment with the goal of further improving our understanding of wind resources in the region.

Complex terrain↗
DOE OSTI2021

Design and Performance of Kokkos Staging Space toward Scalable Resilient Application Couplings

With the growing number of applications designed for heterogeneous HPC devices, application programmers and users are finding it challenging to compose scalable workflows as ensembles of these applications, that are portable, performant and resilient. The Kokkos C++ library has been designed to simplify this cumbersome procedure by providing an intra-application uniform programming model and portable performance. However, assembling multiple Kokkos-enabled applications into a complex workflow is still a challenge. Although Kokkos enables a uniform programming model, the inter-application data exchange still remains a challenge from both performance and software development cost perspectives. In order to address this issue, we propose Kokkos data staging memory space, an extension of Kokkos' data abstraction (memory space) for heterogeneous computing systems. This new abstraction allows to express data on a virtual shared-space for multiple Kokkos applications, thus extending Kokkos to support inter-application data exchange to build an efficient application workflow. Additionally, we study the effectiveness of asynchronous data layout conversions for applications requiring different memory access patterns for the shared data. Our preliminary evaluation with a synthetic benchmark indicate the effectiveness of this conversion adapted to three different scenarios representing access frequency and use patterns of the shared data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Performance-Aligned LLMs for Generating Fast HPC Code

Optimizing scientific software is a difficult task because codebases are often large and complex, and performance can depend upon several factors including the algorithm, its implementation, and hardware among others. Causes of poor performance can originate from disparate sources and be difficult to diagnose. Recent years have seen a multitude of work that use large language models (LLMs) to assist in software development tasks. However, these tools are trained to model the distribution of code as text, and are not specifically designed to understand performance aspects of code. In this work, we introduce a reinforcement learning based methodology to align the outputs of code LLMs with performance. This allows us to build upon the current code modeling capabilities of LLMs and extend them to generate better performing code. Here, we demonstrate that our fine-tuned model improves the expected speedup of generated code over base models for a set of benchmark tasks from 0.9 to 1.6 for serial code and 1.9 to 4.5 for OpenMP parallel code.

Computer science↗