Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Benchmark testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 955 records · Page 53
All sourcesNASA NTRSDOE OSTI
NASA NTRS2020

Durability Characterization of Low Ice Adhesion and Impact Resistance Coatings

Glaze, rime and mixed icing conditions can cause significant variation in aerodynamic performance, fuel efficiency, and flight safety. For commercial aircraft, active icing mitigation strategies are utilized to enable safe flight within icing conditions according to the FAR Part 25/29 Appendix C icing envelope. For general aviation and unmanned aerial vehicles that cannot support active anti-icing technologies, avoidance is the only recourse. Passive approaches have been investigated to reduce weight and energy consumption for active systems and to expand the operational envelope for smaller aircraft. Coatings are one passive method to reduce or mitigate ice accretion on frontal surfaces of commercial aircraft with the greatest region of interest being wing leading edges. Aircraft wing leading edges represent an extreme environment. Durability must be considered for any material to be applied as a coating in this area. Currently, there are no specifications regarding coating durability of a low ice adhesion material applied on the wing leading edge. Therefore, a reasonable starting point is to use the durability specifications put forth for an aircraft external coating. Besides the tests called out in these specifications, others were included such as Taber abrasion to simulate wear and erosion. Beyond durability, performance metrics with regards to adhesion strength of accreted in-flight (i.e., impact) icing are central to determine coating usefulness. Determination of this property though is particularly challenging and only a few facilities have the capability to quantify this. An instrument at NASA Langley Research Center [Adverse Environment Rotary Test Stand Jr., (AERTS Jr.)], based on the AERTS system at The Pennsylvania State University, allows for screening numerous coatings subjected to the icing environment. To establish a benchmark for comparison with research coatings, a state-of-the-art (SOA) commercial aircraft coating was subjected to an array of salient durability and performance experiments. The same tests were conducted using a baseline epoxy (BE) resin formulation that demonstrated some initial promising results for comparison. Results from this initial screening will be discussed herein.

Miranda L Beaudry↗
DOE OSTI2023

A Reproducible Validation of Algorithms for Estimating Array Tilt and Azimuth from Photovoltaic Power Time Series

In this research, we assess the viability of four different, publicly available algorithms for estimating the azimuth and tilt parameters of solar photovoltaic systems using only the associated AC power time series data and site latitude-longitude coordinates. In this work, we curated a benchmarking data set of 44 fixed-tilt systems, comprising 275 measured AC power inverter data streams, with known azimuth and tilt parameters. Additionally, we isolated test cases in the data set with real- world issues, including shading and clipping, to determine how algorithm performance varies based on the presence of these phenomena. Using this data set for benchmarking, we evaluated the estimated vs. actual system characteristics for each algorithm, as well as the associated algorithm execution time using a standardized benchmarking process. The two highest performing algorithms were the Solar Data Tools and the PVWatts 5- based methods, which both achieved a median absolute error of approximately 5 and 1 degrees for azimuth and tilt, respectively. During run time analysis, the SDT method was approximately 5 times faster than the PVWatts 5-based method, with the median execution time for a stream varying between 6 and 8 seconds vs. a median run time of 31 seconds for the PVWatts 5-based method.

azimuth↗
DOE OSTI2024

Linearised Fokker–Planck collision model for gyrokinetic simulations

We introduce a gyrokinetic, linearised Fokker–Planck collision model that satisfies conservation laws and is accurate at arbitrary collisionalities. The differential test-particle component of the operator is exact; the integral field-particle component is approximated using a spherical harmonic and a modified Laguerre polynomial expansion developed by Hirshman and Sigmar (1976 Phys. Fluids 19 1532). The numerical methods of the implementation in the δf-gyrokinetic code stella (Barnes et al 2019 J. Comput. Phys. 391 365–80) are discussed, and conservation properties of the operator are demonstrated. The collision model is then benchmarked against the collision model of the gyrokinetic solver GS2 in the limiting cases of a reduced test-particle collision operator and energy- and momentum-conserving operator. The accuracy of the full collision model is investigated by solving the parallel Spitzer-Härm problem for the transport coefficients. It is shown that retaining collisional energy flux and higher-order terms in the field-particle operator reduces errors in the transport coefficients from 10%–25% for a simple momentum- and energy-conserving model to under 1%.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

A Reproducible Validation of Algorithms for Estimating Array Tilt and Azimuth from Photovoltaic Power Time Series

In this research, we assess the viability of four different, publicly available algorithms for estimating the azimuth and tilt parameters of solar photovoltaic systems using only the associated AC power time series data and site latitude-longitude coordinates. In this work, we curated a benchmarking data set of 44 fixed-tilt systems, comprising 275 measured AC power inverter data streams, with known azimuth and tilt parameters. Additionally, we isolated test cases in the data set with real-world issues, including shading and clipping, to determine how algorithm performance varies based on the presence of these phenomena. Using this data set for benchmarking, we evaluated the estimated vs. actual system characteristics for each algorithm, as well as the associated algorithm execution time using a standardized benchmarking process. The two highest performing algorithms were the Solar Data Tools and the PVWatts 5-based methods, which both achieved a median absolute error of approximately 5 and 1 degrees for azimuth and tilt, respectively. During run time analysis, the SDT method was approximately 5 times faster than the PVWatts 5-based method, with the median execution time for a stream varying between 6 and 8 seconds vs. a median run time of 31 seconds for the PVWatts 5-based method.

algorithm validation↗
DOE OSTI2024

Benchmarking machine learning interatomic potentials via phonon anharmonicity

Abstract Machine learning approaches have recently emerged as powerful tools to probe structure-property relationships in crystals and molecules. Specifically, machine learning interatomic potentials (MLIPs) can accurately reproduce first-principles data at a cost similar to that of conventional interatomic potential approaches. While MLIPs have been extensively tested across various classes of materials and molecules, a clear characterization of the anharmonic terms encoded in the MLIPs is lacking. Here, we benchmark popular MLIPs using the anharmonic vibrational Hamiltonian of ThO 2 in the fluorite crystal structure, which was constructed from density functional theory (DFT) using our highly accurate and efficient irreducible derivative methods. The anharmonic Hamiltonian was used to generate molecular dynamics (MD) trajectories, which were used to train three classes of MLIPs: Gaussian approximation potentials, artificial neural networks (ANN), and graph neural networks (GNN). The results were assessed by directly comparing phonons and their interactions, as well as phonon linewidths, phonon lineshifts, and thermal conductivity. The models were also trained on a DFT MD dataset, demonstrating good agreement up to fifth-order for the ANN and GNN. Our analysis demonstrates that MLIPs have great potential for accurately characterizing anharmonicity in materials systems at a fraction of the cost of conventional first principles-based approaches.

interatomic potentials↗
DOE OSTI2020

Concentration and Recovery of Rare Earth Elements from Eastern US Coal Refuse

Recent studies funded by the US Department of energy have shown that coal and coal byproducts contain elevated contents of Rare Earth Elements (REEs), making them a potential resource for these critical materials. The approach employed in this research focused on the concentration and extraction of REEs from fine coal refuse derived from various preparation plants in the Appalachian coal basin of the United States. Initial efforts in this research focused on the identification and characterization of REEs in various fine coal refuse streams from nine distinct industrial preparation plants in Appalachia. The average REE content in these materials was determined to be approximately 200 ppm, but the REE content showed a strong correlation to the aluminum content, suggesting that the REEs are closely associated with the clay minerals present in the refuse. Given the relatively low REE concentrations, initial efforts sought to concentrate the REEs through decarbonization and dispersive liberation steps. In these tests, high-shear agitation in the presence of a polyelectrolyte, followed by sedimentation was able to isolate the REE-enriched fine clay particles from siliceous gangue minerals. Following the dispersive liberation step, all samples were found to have an REE content greater than 300 ppm, a benchmark used for many initial exploratory studies. In one case, the REE content was increased by more than 125%. Subsequent extraction tests initially utilized a direct ion-exchange leaching approach with ammonium sulfate as lixiviant. In all cases, the simple ion-exchange leaching process failed to recover significant quantities of rare earth elements, ultimately suggesting that the REEs in fine coal waste may be passivated or bound in a colloidal phase. To access this colloidal phase, several alternative approaches were evaluated, including leaching with alternative ion-exchange lixiviants, reductive leaching, gas-purged leaching, and others. The approach that showed the most promise was strong alkaline pretreatment, followed by ion-exchange leaching with ammonium sulfate at pH 4. A combination of strong alkali and high-temperatures treatment successfully liberated the REEs, converting them to a form amenable to ion-exchange leaching. The highest REE recovery achieved with this method was determined to be 39%. Lastly, bench-scale solvent extraction tests were used to further concentrate REEs in the leach solution and demonstrate that mixed rare earth concentrates can be successfully produced from fine coal refuse.

01 COAL, LIGNITE, AND PEAT↗
DOE OSTI2023

Comparative verification of hydro-mechanical fracture behavior: Task G of international research project DECOVALEX–2023

Numerical simulations become a necessity when experimental approaches cannot cover the required physical and time scale of interest. One of such area is a simulation of long-term host rock behaviors for nuclear waste disposal and simulation tools involved in the assessment must go through rigorous validation tests. The DECOVALEX project (Development of COupled models and their VAlidation against EXperiments) is dedicated to this purpose by international participants. This work is part of the ongoing phase DECOVALEX–2023 (D–2023, Task G) particularly aiming to simulate fracture behaviors under various conditions. Here, in this study, we cross-verified a variety of numerical methods including continuous and discontinuous approaches against four benchmark exercises with emphasis on numerical accuracy and parameterization of the various numerical approaches. The systematic inter-comparisons of test cases highlight advantages and disadvantages of the different numerical models. Numerical details on discretization effects (e.g. mesh density and orientation) and domain size were investigated in detail for practical applications. It became evident that meticulous attention to mesh resolution and domain size is imperative for achieving accurate numerical simulations, even for static cracks. Moreover, when comparing numerical methods to closed-form solutions for static cracks, all models successfully reproduced the maximum crack opening but encountered challenges near the crack tips. Finally, the paper discusses how to convert between and therefore compare parameters of various numerical approaches. Our benchmark studies reveal that each model necessitates a distinct number of parameters, even in simple scenarios like static crack aperture benchmarks. It is generally more practical to employ fewer parameters to mitigate model over-parameterization and enhance experimental feasibility.

58 GEOSCIENCES↗
DOE OSTI2022

Time-Based CAN IDS Paper Results Code

Modern vehicles are complex cyber-physical systems made of hundreds of electronic control units (ECUs) that communicate over controller area networks (CANs). This inherited complexity has expanded the CAN attack surface which is vulnerable to message injection attacks. These injections change the overall timing characteristics of messages on the bus, and thus, to detect these malicious messages, time-based intrusion detection systems (IDSs) have been proposed. However, time-based IDSs are usually trained and tested on low-fidelity datasets with unrealistic, labeled attacks. This makes difficult the task of evaluating, comparing, and validating IDSs. Here we detail and benchmark four time-based IDSs against the newly published ROAD dataset, the first open CAN IDS dataset with real (non-simulated) stealthy attacks with physically verified effects. We found that methods that perform hypothesis testing by explicitly estimating message timing distributions have lower performance than methods that seek anomalies in a distribution related statistic. In particular, these “distribution-agnostic” based methods outperform “distribution-based” methods by at least 55% in area under the precision-recall curve (AUC-PR). Our results expand the body of knowledge of CAN time-based IDSs by providing details of these methods and reporting their results when tested on datasets with real advanced attacks. Finally, we develop an after-market plug-in detector using lightweight hardware, which can be used to deploy the best performing IDS method on nearly any vehicle.

Moriano, Pablo [Oak Ridge National Lab. (ORNL), Oa↗
DOE OSTI2021

Propagating neutronic uncertainties for FFTF LOFWOS Test #13

The safety evaluation conducted for licensing a Sodium-cooled Fast Reactor (SFR) may require modeling transients with best-estimate calculation tools that must first be validated against real-world measurements. To provide the community with a valuable benchmarking opportunity for validating SFR analysis tools and methods, the International Atomic Energy Agency (IAEA) initiated a coordinated research project (CRP) in 2018 for the analysis of the Fast Flux Test Facility (FFTF) Loss of Flow Without Scram (LOFWOS) Test #13. The impact of nuclear data uncertainties on neutronics parameters was previously investigated based on the COMMARA-2.0 covariance matrix. Since the transient simulation results are very sensitive to certain reactivity coefficients, it was decided to employ rigorous uncertainty propagation methods to quantify the impact of nuclear data uncertainties on the best-estimate predication of FFTF LOFWOS Test #13. The DAKOTA code is used to propagate neutronic uncertainties through SAS4A/SASSYS-1 transient simulations, while taking into account spatial and reaction-wise correlations within these uncertainties. This study shows that the remaining discrepancies observed between the Argonne National Laboratory (ANL) best-estimate results and the experimental measurements can be partly explained by the uncertainty associated with Gas Expansion Module (GEM) worth, which contributes the majority of the overall nuclear data uncertainty on the output from the FFTF LOFWOS Test #13 transient simulation. This study also confirmed the importance of including spatial and reaction-wise correlations of nuclear data uncertainties on feedback coefficients in the uncertainty propagation to avoid under-estimating their impact during the transient simulations.

42 ENGINEERING↗
DOE OSTI2022

Colloidal synthesis of monodisperse trimetallic Pt–Fe–Ni nanocrystals and their enhanced electrochemical performances

Among the multi-metallic nanocatalysts, Pt-based alloy nanocrystals (NCs) have demonstrated promising performance in fuel cells and water electrolyzers. In this paper, we demonstrate a facile colloidal synthesis of monodisperse trimetallic Pt–Fe–Ni alloy NCs through a co-reduction of metal precursors. The as-synthesized ternary NCs exhibit superior mass and specific activities toward oxygen reduction reaction (ORR), which are ~2.8 and 5.6 times as high as those of the benchmark Pt/C catalyst, respectively. The ORR activity of the carbon-supported Pt–Fe–Ni nanocatalyst is persistently retained after the durability test. Owing to the incorporation of Fe and Ni atoms into the Pt lattice, the as-prepared trimetallic Pt-alloy electrocatalyst also manifestly enhances the electrochemical activity and durability toward the oxygen evolution reaction with a reduced overpotential when compared with that of the benchmark Pt/C (Δη = 0.20 V, at 10 mA cm –2 ). This synthetic strategy paves the way for improving the reactivity for a broad range of electrocatalytic applications.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Testing and Expertise for Marine Energy (TEAMER) Program Support - Numerical Modeling Assistance for iProTech's "PIP" WEC Device: Cooperative Research and Development Final Report, CRADA Number CRD-20-17303

Develop a time-domain, 3 degree of freedom numerical model of IProTech's PIP device, including power take-off (PTO), that can be used to develop control algorithms and establish a baseline technology performance level (TPL). This is important to IProTech because having a good numerical model allows IProTech to optimize design and control algorithms on an ongoing basis. A baseline TPL enables the PIP technology to be benchmarked against other wave energy converter (WEC) concepts, providing (i) justification (or not) for investment in wave tank model testing, (ii) identifying optimum design parameters and control algorithms for the test model, (iii) specifying test conditions and (iv) interpreting wave tank test results. An accredited WEC-Sim model of the PIP device is an essential tool for further development of the technology. 1) The development of numerical models of the PIP concept will enable iProTech to evaluate the potential performance of this concept, and to investigate how sensitive the device is to different variables within the system (e.g. hydraulic components, geometry dimensions, etc.) 2) The numerical models were developed with WEC-Sim and PTO-Sim -- the de-facto industry standard WEC numerical modeling tools. A detailed model of the system's hydraulic circuit was developed with PTO-Sim. However, WEC-Sim is based on linear hydrodynamics and has several limitations; validation against high fidelity models/physical scale models would help to build confidence in the results.

16 TIDAL AND WAVE POWER↗
DOE OSTI2021

Validation of SAS4A/SASSYS-1 for predicting steady-state single-phase natural circulation

The validation of system analysis codes for nuclear reactor systems is required for the development and application of these computational tools. Designed as a comprehensive system analysis code for advanced nuclear reactors, SAS4A/SASSYS-1 requires validation of its physics model for capturing single-phase natural circulation behavior. To support the validation of SAS4A/SASSYS-1, high-precision experiments are performed capturing steady-state single-phase natural circulation on a scaled facility with comprehensive instrumentation. In this work, dedicated tests are performed quantifying the critical modeling parameters, and a single-phase natural circulation benchmark dataset is obtained with well-documented uncertainty and comprehensive facility description. The validation is then performed against the dataset examining the capability of SAS4A/SASSYS-1 in simulating steady-state single-phase natural circulation. The experimental facility is modeled in the candidate code. Solution verification is performed using Richardson-extrapolation-based estimators which quantify and restrict numerical errors from discretization. Input uncertainty provided by the benchmark dataset is forward propagated through the candidate code, quantifying the output uncertainty in a Monte Carlo approach. The composition of the output uncertainty is also quantified through a variance-based sensitivity analysis. With the uncertainty quantified for each individual condition, a detailed comparison between the simulation results and experimental data is performed covering the whole dataset. The results show consistent agreement for all primary parameters. The current validation activity provides a valuable benchmark dataset for the validation of system analysis codes in capturing single-phase natural circulation and demonstrates satisfactory prediction capability of SAS4A/SASSYS1 for steady-state single-phase natural circulation.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2026

Numerical implementation of the MOOSE subchannel module (SCM) algorithm

The subchannel module of moose (SCM), previously referred to as Pronghorn-SC, is a subchannel code designed to resolve single-phase flow fields and calculate the relevant flow variables, in nuclear reactor fuel-pin assemblies. The assemblies it models are: water-cooled, with bare fuel-pins in a square ducted, quadrilateral lattice and liquid-metal/water-cooled, with bare/wire-wrapped fuel pins in a hexagonal ducted, triangular lattice. Previous publications have presented the development, validation and verification of SCM. This work presents an comparative overview of the different solvers implemented within SCM. This includes the solver algorithm description, and a performance comparison. The two test cases chosen to demonstrate the solver performance were taken from the PSBT enthalpy mixing benchmark and the ORNL-19 pin benchmark.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2020

A coupled polygonal DEM-LBM technique based on an immersed boundary method and energy-conserving contact algorithm

Here we present a framework of coupling polygonal discrete elements and the lattice Boltzmann method using a direct forcing immersed boundary scheme. In this technique, an energy-conserving contact algorithm is utilized to handle the interactions between convex and concave polygonal particles. The surface of a polygon is represented by discrete boundary points which includes vertices of polygonal particles and/or points interpolated from vertices. The fluid-particle coupling is obtained through the interactions of the boundary points and the imaginary fluid particles using a direct-forcing immersed boundary method. Validations of the proposed technique are made by single particle and multiple arbitrarily-shaped particle sedimentation tests, and the effect of particle shape is illustrated using a drafting-kissing-tumbling benchmark.

42 ENGINEERING↗
DOE OSTI2021

Enhance Low Level Temperature and Moisture Profiles Through Combining NUCAPS, ABI Observations, and RTMA Analysis

Thermodynamic information from low levels in the atmosphere is crucial for operational weather forecasts and meteorological researchers. The NOAA Unique Combined Atmospheric Processing System (NUCAPS) sounding products have been proven beneficial to fill the data gap between synoptic radiosonde observations (RAOBs). However, compared with the upper troposphere, the accuracy of NUCAPS soundings in the low levels still needs improvement. In this study, a deep neural network (DNN) is applied to fuse multiple data sources to enhance the NUCAPS temperature and moisture profiles in the lower atmosphere. The network is developed by combining satellite observations, including NUCAPS sounding retrievals and high resolution geostationary satellite observations from the Advanced Baseline Imager, and surface analysis from the Real-Time Mesoscale Analysis (RTMA) as inputs, while collocated soundings from ECMWF re-analysis version 5 are used as the benchmark for the training. The performance of the model is evaluated by using the independent testing data set, data from a different year, as well as collocated RAOBs, showing improvement to the temperature and moisture profiles by reducing the root-mean-squared-error (RMSE) by more than 30% in the lower atmosphere (from 700 hPa to surface) in both clear sky and partially cloudy conditions. A convective event from June 18, 2017 is presented to illustrate the application of the enhanced low level soundings on high impact weather events. The enhanced soundings from fused data capture the large surface-based convective available potential energy structures in the preconvection environment, which is very useful for severe storm nowcasting and forecasting applications.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Demonstration of object location, classification, and characterization by developed deep learning dust ablation trail analysis code package using plasma jets

Based on deep learning, a Dust Ablation Trail Analysis (DATA) code package is developed to detect dust ablation trails in tokamaks, which is intended to analyze a large amount data of tokamak dusts. To validate and benchmark the DATA code package, 2440 plasma jet images are exploited for the training and test of the deep learning DATA code package, since plasma jets resemble the shape and size of dust ablation clouds in tokamaks. After being trained by 1920 plasma jet images, the DATA code package is able to locate 100% plasma jets, classify plasma jets with the accuracy of >99.9%, and output image skeleton information for classified plasma jets. The DATA code package trained by the plasma jet images is also used to analyze the dust ablation trails captured in the Experimental Advanced Superconducting (EAST) tokamak with the satisfactory performance, further verifying its applicability in the fusion dust ablation investigation. Based on its excellent performance presented here, it is demonstrated that our DATA code package is able to automatically identify and analyze dust ablation trails in tokamaks, which can be used for further detailed investigations, such as the three-dimensional reconstruction of dusts and their ablation trails.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Novel technique for the study of pileup events in cryogenic bolometers

Precise characterization of detector time resolution is of crucial importance for next-generation cryogenic-bolometer experiments searching for neutrinoless double-beta decay, such as CUPID, in order to reject background due to pile-up of two-neutrino double-beta decay events. In this paper, we describe a technique developed to study the pile-up rejection capability of cryogenic bolometers. Our approach, which consists of producing controlled pile-up events with a programmable waveform generator, has the benefit that we can reliably and reproducibly control the time separation and relative energy of the individual components of the generated pile-up events. The resulting data allow us to optimize and benchmark analysis strategies to discriminate between individual and pile-up pulses. We describe a test of this technique performed with a small array of detectors at the Laboratori Nazionali del Gran Sasso, in Italy; we obtain a 90% rejection efficiency against pulser-generated pile-up events with rise time of ~15 ms down to time separation between the individual events of 2 ms.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

Benchmarking Optimizers for Qumode State Preparation with Variational Quantum Algorithms

Quantum state preparation involves preparing a target state from an initial system, a process integral to applications such as quantum machine learning and solving systems of linear equations. Recently, there has been a growing interest in qumodes due to advancements in the field and their potential applications. However there is a notable gap in the literature specifically addressing this area. This paper aims to bridge this gap by providing performance benchmarks of various optimizers used in state preparation with Variational Quantum Algorithms. We conducted extensive testing across multiple scenarios, including different target states, both ideal and sampling simulations, and varying numbers of basis gate layers. Our evaluations offer insights into the complexity of learning each type of target state and demonstrate that some optimizers perform better than others in this context. Notably, the Powell optimizer was found to be exceptionally robust against sampling errors, making it a preferred choice in scenarios prone to such inaccuracies. Additionally, the Simultaneous Perturbation Stochastic Approximation optimizer was distinguished for its efficiency and ability to handle increased parameter dimensionality effectively.

Kan, Shuwen [Fordham University]↗