Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Benchmark testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 865 records · Page 48
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Benchmark Models for Classification of Radiation Type Induced in Immune Cells

NASA Biological and Physical Sciences and the Science Mission Directorate have published a benchmark dataset of mouse immune cells subjected to radiation-induced DNA damage. The dataset comprises ML-ready microscopic imagery of said cells, including labels indicating radiation type and dose. The machine learning team at NASA Interagency Implementation and Advanced Concept Team (IMPACT) created multiple benchmark models. Initially, we conducted a preliminary analysis using thresholding. The algorithm used thresholds on average brightness of the available images to classify them into their respective radiation type. We also tested machine learning approaches. Convolutional Neural Networks (CNN) emerged as the best-performing model. This poster presents the benchmark scores obtained by the models.

Vishal Perekadan↗
DOE OSTI2021

Flow Redirection and Induction in Steady State (FLORIS) Wind Plant Power Production Data Sets

This dataset contains turbine- and plant-level power outputs for 252,500 cases of diverse wind plant layouts operating under a wide range of yawing and atmospheric conditions. The power outputs were computed using the Gaussian wake model in NREL's FLOw Redirection and Induction in Steady State (FLORIS) model, version 2.3.0. The 252,500 cases include 500 unique wind plants generated randomly by a specialized Plant Layout Generator (PLayGen) that samples randomized realizations of wind plant layouts from one of four canonical configurations: (i) cluster, (ii) single string, (iii) multiple string, (iv) parallel string. Other wind plant layout parameters were also randomly sampled, including the number of turbines (25-200) and the mean turbine spacing (3D-10D, where D denotes the turbine rotor diameter). For each layout, 500 different sets of atmospheric conditions were randomly sampled. These include wind speed in 0-25 m/s, wind direction in 0 deg.-360 deg., and turbulence intensity chosen from low (6%), medium (8%), and high (10%). For each atmospheric inflow scenario, the individual turbine yaw angles were randomly sampled from a one-sided truncated Gaussian on the interval 0 deg.-30 deg. oriented relative to wind inflow direction. This random data is supplemented with a collection of yaw-optimized samples where FLORIS was used to determine turbine yaw angles that maximize power production for the entire plant. To generate this data, a subset of cases were selected (50 atmospheric conditions from 50 layouts each for a total of additional 2,500 cases) for which FLORIS was re-run with wake steering control optimization. The IEA onshore reference turbine, which has a 130 m rotor diameter, a 110 m hub height, and a rated power capacity of 3.4 MW was used as the turbine for all simulations. The simulations were performed using NREL's Eagle high performance computing system in February 2021 as part of the Spatial Analysis for Wind Technology Development project funded by the U.S. Department of Energy Wind Energy Technologies Office. The data was collected, reformatted, and preprocessed for this OEDI submission in May 2023 under the Foundational AI for Wind Energy project funded by the U.S. Department of Energy Wind Energy Technologies Office. This dataset is intended to serve as a benchmark against which new artificial intelligence (AI) or machine learning (ML) tools may be tested. Baseline AI/ML methods for analyzing this dataset have been implemented, and a link to their repository containing those models has been provided. The .h5 data file structure can be found in the GitHub repository under explore_wind_plant_data_h5.ipynb.

AI↗
NASA NTRS2000

Performance Monitoring of Distributed Data Processing Systems

Test and checkout systems are essential components in ensuring safety and reliability of aircraft and related systems for space missions. A variety of systems, developed over several years, are in use at the NASA/KSC. Many of these systems are configured as distributed data processing systems with the functionality spread over several multiprocessor nodes interconnected through networks. To be cost-effective, a system should take the least amount of resource and perform a given testing task in the least amount of time. There are two aspects of performance evaluation: monitoring and benchmarking. While monitoring is valuable to system administrators in operating and maintaining, benchmarking is important in designing and upgrading computer-based systems. These two aspects of performance evaluation are the foci of this project. This paper first discusses various issues related to software, hardware, and hybrid performance monitoring as applicable to distributed systems, and specifically to the TCMS (Test Control and Monitoring System). Next, a comparison of several probing instructions are made to show that the hybrid monitoring technique developed by the NIST (National Institutes for Standards and Technology) is the least intrusive and takes only one-fourth of the time taken by software monitoring probes. In the rest of the paper, issues related to benchmarking a distributed system have been discussed and finally a prescription for developing a micro-benchmark for the TCMS has been provided.

Ojha, Anand K.↗
DOE OSTI2024

The battery failure databank: Insights from an open-access database of thermal runaway behaviors of Li-ion cells and a resource for benchmarking risks

The thermal response of Li-ion cells can greatly vary for identical cell designs tested under identical conditions, the distribution of which is costly to fully characterize experimentally. The open-source Battery Failure Databank presented here contains robust, high-quality data from hundreds of abuse tests spanning numerous commercial cell designs and testing conditions. Data was gathered using a fractional thermal runaway calorimeter and contains the fractional breakdown of heat and mass that was ejected, as well as high-speed synchrotron radiography of the internal dynamic response of cells during thermal runaway. The distribution of thermal output, mass ejection, and internal response of commercial cells are compared for different abuse-test conditions, which when normalized on a per amp-hour basis show a strong positive correlation between heat output from cells, the fraction of mass ejected from the cells, their energy- and power-density. Ejected mass was shown to contain 10x more heat per gram than non-ejected mass. The causes of 'outlier' thermal and ejection responses i.e., extreme cases, are elucidated by high-speed radiography which showed how occurrences such as vent clogging can create more hazardous conditions. High-speed radiography also demonstrated how the time-resolved interplay of thermal runaway propagation and mass ejection influences the total heat generated.

25 ENERGY STORAGE↗
DOE OSTI2024

GO Competition Challenge 1 Benchmark Approach and Validation Support

At the outset ARPA-e’s grid optimization competition sought to test a hypothesis that modern nonlinear optimization methods had reached a level of performance and reliability necessary to optimize energy grids using nonlinear power flow equations. The results demonstrated by this project and the other participants in the grid optimization competition have demonstrated beyond any shadow of doubt that this hypothesis is true. This result suggest that the mathematical optimization technology now exists to produce a revolution in power grid optimization software, which would improve the efficiency of energy grid operations to the broad benefit of society.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS2011

A Comparison of Methods for Assessing Space Suit Joint Ranges of Motion

Through the Advanced Exploration Systems Program, NASA is attempting to use the vast collection of space suit mobility data from 50 years worth of space suit testing to build predictive analysis tools to aid in early architecture decisions for future missions and exploration programs. However, the design engineers must first understand if and how data generated by different methodologies can be compared directly and used in an essentially interchangeable manner. To address this question, the isolated joint range of motion data from two different test series were compared. Both data sets were generated from participants wearing the Mark III Space Suit Technology Demonstrator (MK-III), Waist Entry I-suit (WEI), and minimal clothing. Additionally the two tests shared a common test subject that allowed for within subject comparisons of the methods that greatly reduced the number of variables in play. The tests varied in their methodologies: the Space Suit Comparative Technologies Evaluation used 2D photogrammetry to analyze isolated ranges of motion while the Constellation space suit benchmarking and requirements development used 3D motion capture to evaluate both isolated and functional joint ranges of motion. The isolated data from both test series were compared graphically, as percent differences, and by simple statistical analysis. The results indicated that while the methods generate results that are statistically the same (significance level p= 0.01), the differences are significant enough in the practical sense to make direct comparisons ill advised. The concluding recommendations propose direction for how to bridge the data gaps and address future mobility data collection to allow for backward compatibility.

Aitchison, Lindsay↗
DOE OSTI2023

A step towards the final frontier: Lessons learned from acceptance testing of the first HPE/Cray EX 3000 system at ORNL

Summary In this article, we summarize the deployment of the Air Force Weather (AFW) HPC11 system at Oak Ridge National Laboratory (ORNL) including the process followed to successfully complete acceptance testing of the system. HPC11 is the first HPE/Cray EX 3000 system that has been successfully released to its user community in a federal facility. HPC11 consists of two identical 800‐node supercomputers, Fawbush and Miller, with access to two independent and identical lustre parallel file systems. HPC11 is equipped with Slingshot 10 interconnect technology and relies on the HPE Performance Cluster Manager software for system configuration. ORNL has a clearly defined acceptance testing process used to ensure that every new system deployed can provide the necessary capabilities to support user workloads. We worked closely with HPE and AFW to develop a set of tests that used the United Kingdom's Meteorological Office's Unified Model and 4‐dimensional variational data assimilation. We also included benchmarks and applications from the Oak Ridge Leadership Computing Facility portfolio to fully exercise the HPE/Cray programming environment and evaluate the functionality and performance of the system. Acceptance testing of HPC11 required parallel execution of each element on Fawbush and Miller. In addition, careful coordination was needed to ensure successful acceptance of the newly deployed lustre file systems alongside the compute resources. In this work, we present test results from specific system components and provide an overview of the issues identified, challenges encountered, and the lessons learned along the way.

Melesse Vergara, Verónica G.↗
DOE OSTI2022

Optimal Gas-Electric Energy System Decarbonization Planning

As energy utilities implement climate change mitigation policies, system planners require strategies for achieving affordable emissions reductions. Coordinated planning of electric power and natural gas systems will allow synergistic investments to address cross-sector operational constraints, competing uses for net-zero emissions fuels, and shifts in energy demands across energy carriers. In this study, we develop a novel optimization program that finds the cost-minimizing mix of infrastructure expansion or reduction across gas and electric systems to satisfy sector-specific emissions constraints. Alongside energy supply resources, our framework allows for central-planning of end-use equipment stocks to allow switching between gas and electric appliances upon failure or premature replacement. The proposed model is used to simulate case study scenarios for a benchmark 24-pipe gas network coupled to a 24-node power system test network. We find that electrification of greater than 80% of core gas demands is a component of the least-cost solution for modeled energy systems. Despite this substitution, the gas system is maintained to service difficult-to-electrify customers and to deliver net-zero emissions gas to electricity generators in times of peak electricity demand. Restricting electrification of gas appliances increases reliance on power- to-gas technologies and increases annual costs by 15% in 2040. Neglecting constraints on pipeline blending of hydrogen can produce a misleading result that relies on hydrogen blend fractions of greater than 50%. In all cases, we find the average costs of delivered gas increase nearly 5-fold across the decarbonization transition, highlighting the importance of future work investigating cost-allocation strategies for ensuring an equitable energy transition.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2023

Simulations and analysis tools for charge-exchange (d, 2 He) reactions in inverse kinematics with the AT-TPC

Charge-exchange (d, 2 He) reactions in inverse kinematics at intermediate energies are a very promising method to investigate the Gamow–Teller transition strength in unstable nuclei. A simulation and analysis software based on the attpcroot package was developed to study this type of reactions with the active-target time projection chamber (AT-TPC). The simulation routines provide a realistic detector response that can be used to understand and benchmark experimental data. Analysis tools and correction routines can be developed and tested from simulations in ATTPCROOT , because they are processed in the same way as the real data. In particular, we study the feasibility of using coincidences with beam-like particles to unambiguously identify the (d, 2 He) reaction channel, and to develop a kinematic fitting routine for future applications. More technically, the impact of space-charge effects in the track reconstruction, and a possible correction method are investigated in detail. Finally, this analysis and simulation package constitutes an essential part of the software development for the fast-beams program with the AT-TPC.

(d,2He)↗
DOE OSTI2021

Observations of modal coupling due to bolted joints in an experimental benchmark structure

The goal of this paper is to present a set of measurements from a benchmark structure containing two bolted joints to support future efforts to predict the damping due to the joints and to model nonlinear coupling between the first two elastic modes. Bolted joints introduce nonlinearities in structures, typically causing a softening in the natural frequency and an increase in damping because of frictional slip between the contact interfaces within the joint. These nonlinearities pose significant challenges when characterizing the response of the structure under a large range of load amplitudes, especially when the modal responses become coupled, causing the effective damping and natural frequency to not only depend on the excitation amplitude of the targeted mode, but also the relative amplitudes of other modes. In this work, two nominally identical benchmark structures, known in some prior works as the S4 beam, are tested to characterize their nonlinear properties for the first two elastic modes. Detailed surface measurements are presented and validated through finite element analysis and reveal distinct contact interactions between the two sets of beams. The free-free test structures are excited with an impact hammer and the transient response is analyzed to extract the damping and frequency backbone curves. A range of impact amplitudes and drive points are used to isolate a single mode or to excite both modes simultaneously. Differences in the nonlinear response correlate with the relative strength of the modes that are excited, allowing one to characterize mode coupling. Each of the beams shows different nonlinear properties for each mode, which is attributed to the different contact pressure distributions between the parts, although the mode coupling relationship is found to be consistent between the two. Lastly, the test data key finding are presented in this paper and the supporting data is available on a public repository for interested researchers.

42 ENGINEERING↗
DOE OSTI2023

A data-driven method for modelling dissipation rates in stratified turbulence

We present a deep probabilistic convolutional neural network (PCNN) model for predicting local values of small-scale mixing properties in stratified turbulent flows, namely the dissipation rates of turbulent kinetic energy and density variance, $\varepsilon$ and $\chi$ . Inputs to the PCNN are vertical columns of velocity and density gradients, motivated by data typically available from microstructure profilers in the ocean. The architecture is designed to enable the model to capture several characteristic features of stratified turbulence, in particular the dependence of small-scale isotropy on the buoyancy Reynolds number $Re_b:=\varepsilon /(\nu N^2)$ , where $\nu$ is the kinematic viscosity and $N$ is the background buoyancy frequency, the correlation between suitably locally averaged density gradients and turbulence intensity and the importance of capturing the tails of the probability distribution functions of values of dissipation. Empirically modified versions of commonly used isotropic models for $\varepsilon$ and $\chi$ that depend only on vertical derivatives of density and velocity are proposed based on the asymptotic regimes $Re_b\ll 1$ and $Re_b\gg 1$ , and serve as an instructive benchmark for comparison with the data-driven approach. When trained and tested on a simulation of stratified decaying turbulence which accesses a range of turbulent regimes (associated with differing values of $Re_b$ ), the PCNN outperforms assumptions of isotropy significantly as $Re_b$ decreases, and additionally demonstrates improvements over the fitted empirical models. A differential sensitivity analysis of the PCNN facilitates a comparison with the theoretical models and provides a physical interpretation of the features enabling it to make improved predictions.

42 ENGINEERING↗
DOE OSTI2026

Inferring performance metrics for laser direct drive experiments on OMEGA

Quantifying performance improvements on the OMEGA laser facility requires robust inference of established no-alpha performance metrics, which requires, at minimum, a model to infer the shocked fuel mass and pressure of the confined fusion plasma. In this work, we describe the methodology used to infer performance metrics on OMEGA and present the current state-of-the art model used to infer these metrics from OMEGA experiments. In particular, since neutron images of cryogenic implosions are not available on OMEGA at present, we present how x-ray sizes are determined on OMEGA using a Gaussian Process regression model and how the neutron production region's size is inferred from them. As a result, we end by benchmarking the model using synthetic data and 1-D LILAC simulations and test its experimental self-consistency across available x-ray diagnostic channels.

Gopalaswamy, V. [Laboratory for Laser Energetics, ↗
DOE OSTI2024

Advanced Signal Decomposition Analysis and Anomaly Detection in Photovoltaic Systems

With the rapid expansion of large-scale photovoltaic (PV) plants, it is paramount for solar stakeholders to understand the reliability and efficiency of their plants to inform maintenance decisions, increase production, and understand the design factors that impact performance. Diagnosing underperformance in PV plants is challenging due to the relatively few monitoring points with respect to the large geographic footprint of the plant. This work introduces a cutting-edge method that transforms the analysis and management of key factors influencing PV plant performance, including performance loss rate (PLR), recoverable soiling, and major system changes. Identifying these factors is critical for deriving actionable insights. Leveraging advanced analytical techniques such as wavelet transformation, robust regression, and extreme point analysis, this approach provides a nuanced understanding of these factors. This method has been tested across two synthetic datasets and one real dataset, consistently surpassing existing benchmarks by achieving a lower median mean absolute error and reduced error variability across all comparable components.

14 SOLAR ENERGY↗
DOE OSTI2022

Novel Real- Time EMT- TS Modeling Architecture for Feeder Blackstart Simulations

This paper presents the development and benchmarking of a novel real-time electromagnetic-transient and transient-stability (EMT-TS) modeling architecture for distribution feeder restoration studies. The work presents for the first time in the literature a real-time EMT-TS testbed in which the grid-forming unit is simulated in EMT domain, operating as the slack bus of the phasor domain while including unbalanced voltage conditions. To evaluate the performance and limitations of the proposed model, an equivalent EMT testbed is developed to serve as a benchmark. First, the co-simulation framework and the domain coupling methodology are introduced. Next, the steady state operation of the EMT-TS and EMT models are matched. Then, tests are conducted to analyze the transient performance of the proposed EMT-TS model when compared to its EMT benchmark. Results reveal that when utilizing a battery energy storage system (BESS) as the grid-forming unit, the EMT-TS testbed can maintain high accuracy for typical load steps.

Paduani, Victor↗
DOE OSTI2020

Performance and Techno-Economic Evaluation of a Three-Phase, 50-kW SiC-Based PV Inverter

The technical capabilities and benefits of silicon carbide (SiC) compared to silicon (Si) based power electronics converters as well as the premium associated with using SiC instead of Si are well understood. This work proves that the benefits provided by SiC, such as increased efficiency, would result in a lower levelized cost of energy (LCOE) compared to both commercially available, state-of-the-art inverters and the benchmark commercial system cost calculated for the U.S. Department of Energy Solar Energy Technologies Office's SunShot program. The LCOE is calculated to be same as the commercial benchmark despite the installed system cost being 8% more than the benchmark. The LCOE calculation is based on both experiential values obtained from testing the SiC photovoltaic (PV) inverter and on the bottoms-up modeled volume cost for the same 50-kW SiC-based inverter. The PV system used for the LCOE model is also explained, and the system cost breakout is presented in this paper. Furthermore, multiple scenarios with variations in the PV system are presented to show their effect on the system LCOE.

14 SOLAR ENERGY↗
DOE OSTI2023

Rethinking the Price Formation Problem–Part 2: Rewarding Flexibility and Managing Price Risk

In this study, part 1 of this two-part paper describes the impact that uncertainty has on the design and analysis of price formation policies in the non-convex auctions conducted by U.S. wholesale electricity market operators. Using first a toy model and then a large-scale test system, Part 2 demonstrates the difference in prices under the idealized benchmark of ex ante convex hull pricing defined in Part 1 versus existing methods, in particular documenting the potential for suppression of volatility and therefore under-compensation of flexibility by existing methods. The examples highlight that inefficient spot price formation can induce inefficient forward commitments of generators, necessitating out-of-market intervention to restore a reliable and efficient operating plan.Given the potential side effects of existing policies for investment and operation, we suggest two elements in a reoriented approach to the price formation problem: first ensuring that prices exhibit full-strength volatility, and second ensuring that risk-averse market participants have sufficient ability to manage this volatility.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

hypredrive: high-level interface for solving linear systems with hypre

This software introduces a high-level interface designed to simplify solving linear systems using hypre, a renowned library for such computational challenges. It is crafted to be accessible and user-friendly, making the powerful capabilities of hypre available to a broader audience without requiring in-depth technical knowledge. The interface is characterized by its use of YAML for input, a format celebrated for its structured yet straightforward readability. This choice ensures that users can easily configure the software to meet their specific needs. Additionally, the software boasts an intuitive API that encapsulates hypre's functionalities, making it easier for users to interact with the process of solving linear systems. It is particularly beneficial for prototyping, offering a quick and efficient means to test various solver and preconditioner configurations. Furthermore, the software allows for the creation of an offline testing framework in which predefined linear systems are read from files and benchmarked with user-defined solution strategies. This makes it an invaluable tool for developers and researchers exploring and validating their computational models. Overall, the software serves as a bridge, bringing the advanced computational capabilities of hypre closer to users who may need more specialized technical expertise, thereby facilitating innovation and exploration in the field of numerical linear algebra.

Paludetto Magri, Victor↗
DOE OSTI2020

Low-Cost High-Power Anodes for Electric Vehicle Batteries

Saratoga Energy has developed a system for making graphite from carbon dioxide. LBNL has experience in benchmarking graphites for Li-ion battery applications. In half cells, we tested graphites for their reversible capacity and their rate capability. They used this data to support a phase II proposal that was funded by DOE to scale up their best material and test in larger cells.

25 ENERGY STORAGE↗