Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluations”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Evaluating Efficiency and Security of Connected and Autonomous Vehicle Applications

Evaluating efficiency and security of Connected and Autonomous Vehicles (CAVs) requires an environment that can support applications and measurements under real-world conditions. This work introduces our implementation and evaluation of a Connected and Autonomous Vehicle Research Environment (CAVRE). We implement and evaluate an existing CAV application called Cooperative Adaptive Cruise Control (CACC) using physical Vehicle-to-Vehicle (V2V) communications between a virtual agent and a real autonomous vehicle operating on a steerable dynamometer. CAVRE allows the follower to autonomously control longitudinal behavior on the dynamometer in order to maintain a steady following time gap from the leader. The effects of a wireless jamming attack on CACC and fuel efficiency is also evaluated. By executing attacks in a controlled environment, we learn how compromised communications can degrade CAV applications. We show that jamming V2V communications can impact CACC’s string stability and decrease fuel efficiency by more than 50%.

Taylor, Curtis↗
DOE OSTI2023

DeepThermo: Deep Learning Accelerated Parallel Monte Carlo Sampling for Thermodynamics Evaluation of High Entropy Alloys

Since the introduction of Metropolis Monte Carlo (MC) sampling, it and its variants have become standard tools used for thermodynamics evaluations of physical systems. However, a long-standing problem that hinders the effectiveness and efficiency of MC sampling is the lack of a generic method (a.k.a. MC proposal) to update the system configurations. Consequently, current practices are not scalable. Here we propose a parallel MC sampling framework for thermodynamics evaluation—DeepThermo. By using deep learning–based MC proposals that can globally update the system configurations, we show that DeepThermo can effectively evaluate the phase transition behaviors of high entropy alloys, which have an astronomical configuration space. For the first time, we directly evaluate a density of states expanding over a range of ~e 10,000 for a real material. We also demonstrate DeepThermo’s performance and scalability up to 3,000 GPUs on both NVIDIA V100 and AMD MI250X-based supercomputers.

Yin, Junqi↗
DOE OSTI2023

Distributed PV Hosting Capacity Evaluation Considering Equitable PV Accommodation

Distributed photovoltaic (DPV) hosting capacity is widely used by distribution system operators to evaluate how much DPV the distribution networks can accommodate without violating operational constraints. If the proliferation of DPV deployment is appropriately managed, it offers an excellent opportunity to mitigate the energy inequity issue in electric grids. Hosting capacity improvement plans that neglect energy equity can exacerbate unequal DPV access. To consider energy equity in distribution network planning to enhance DPV accommodation, hosting capacity evaluations should be improved to consider equity constraints endogenously. This paper proposes a DPV hosting capacity evaluation method that considers equitable DPV access among sub-regions of a distribution feeder partitioned by socioeconomic indicators, such as income level. The results demonstrate that considering equity in DPV hosting capacity evaluations can provide a valuable reference in the equitable energy transition. This work provides guidance for future distribution network upgrade planning for equitable DPV deployment.

distributed PV↗
DOE OSTI2020

Remote Hardware-in-the-Loop Approach for Microgrid Controller Evaluation

Utilities have been installing microgrids because of the increased resilience and reliability advantages they may provide to the distribution system. A microgrid controller is a critical component in microgrids. It is of great benefit to derisk the installation of microgrid controllers before field deployment. Hardware-in-the-loop (HIL) testing is used by controller developers and utilities to evaluate the controllers under stressful conditions. In this work, a microgrid control function developed by the Synchrophasor Grid Monitoring and Automation (SyGMA) laboratory at the University of California, San Diego is tested in a remote HIL (RHIL) setup. The digital real-time simulation of the detailed microgrid system was operated at the National Renewable Energy Laboratory's Energy Systems Integration Facility. Under such RHIL setup, successful controller operation is contingent on understanding and characterizing the communications channel and in particular network latencies. The novelty of this paper is the proposed use of a RHIL setup that leverages existing power system communications protocols to evaluate the controller in conjunction with the simulation capabilities of a remote facility. The work presented here will provide the complete setup of the HIL evaluation platform, the details of the communications protocols used by the setup for data transfer between the two organizations, test cases developed to evaluate the controller, and the results from the experiments.

controller hardware-in-the-loop↗
DOE OSTI2021

Evaluation of Centralized Model Based FLISR in a Lab Setup

Utilities are installing advanced distribution management systems (ADMS) around the globe to improve the sensing and control of distribution systems. ADMS is becoming a critical component to improve the resiliency and reliability of distribution systems. These management systems host a multitude of applications that can be used to sense, control, and operate distribution systems. Fault location, isolation, and service restoration (FLISR) is one of the ADMS applications that is critical to improving resilience during fault conditions. FLISR applications use smart, controllable devices that are installed in the distribution system for FLISR operation. These controllable devices include distributed energy resources (DER) and reclosers. Evaluating such ADMS applications before installation in the field can help de-risk the field implementation and avoid costly failures in the field. This paper presents a background on experimental setup that are typically used to evaluate ADMS applications. This is followed by briefly presenting the setup used to evaluate the FLISR application in an off-the-shelf ADMS tool. Finally, results from the evaluation experiments are presented.

94 GMLC - Grid Modernization Laboratory Consortium↗
DOE OSTI2024

An Evaluation of the Effect of Network Cost Optimization for Leadership Class Supercomputers

Dragonfly-based networks are an extensively deployed network topology in large-scale high-performance computing due to their cost-effectiveness and efficiency. The US will soon have three Exascale supercomputers for leadership class workloads deployed using dragonfly networks. Compared to indirect networks of similar scale, the dragonfly network has considerably reduced cable lengths, cable counts, and switch counts, resulting in significant network cost savings for a given system size, however, these cost reductions result in reduced global minimal paths and more challenging routing. Additionally, large scale dragonfly networks often require a taper at the global link level, resulting in less bisection bandwidth than is achievable in other traditional non-blocking topologies of equivalent scale. While dragonfly networks have been extensively studied, they have yet to be fully evaluated in an extreme scale (i.e., exascale) system that targets capability workloads. In this paper, we present the results of the first large scale evaluation of a dragonfly network on an exascale system (Frontier) and compare its behavior to a similar scale fat-tree network on a previous generation TOP500 system (Summit). This evaluation aims to determine the effect of network cost optimizations by measuring a tapered topology’s impact on capability workloads. Our evaluation is based on a collection of synthetic microbenchmarks, mini-apps, and full scale applications. It compares the scaling efficiencies of each benchmark between the dragonfly-based Frontier and the fat-tree-based Summit systems. Our results show that a dragonfly network is $\sim \mathbf{3 0 \%}$ more cost efficient than a fat-tree topology, which amortizes to $\sim 3 \%$ of an exascale system cost. Furthermore, while tapered dragonfly networks impose significant tradeoffs, the impacts are not as broad as initially thought and are mostly seen in applications with global communication patterns, particularly all-to-all (e.g., FFT-based algorithms), but also local communication patterns (e.g., nearest-neighbor algorithms) that are sensitive to network performance variability.

Khan, Awais↗
DOE OSTI2021

Electric Water Heaters for Transactive Systems: Model Evaluations and Performance Quantification

Electric water heaters (EWHs) are opportune appliances for implementing demand-side control. EWH models serve as a fundamental step toward accurately estimating EWH flexibility potential and designing proper control strategies. Existing studies have adapted numerous modeling approaches in evaluating the potential of EWHs for a variety of grid applications. This paper presents an analytical study that evaluates the performance of state-of-art EWH models in terms of accuracy and computational complexity for adaptation in evaluation studies for the transactive system. The work proposes a transactive control strategy that optimally utilizes the thermal inertia of EWHs for providing grid services. Here, the performance of the control strategy and the impact of modeling accuracy is evaluated for device-level and feeder-level use cases using the IEEE 123 node test distribution system appropriately populated with EWHs. The simulation results illustrate the effectiveness of the control strategy in reducing the feeder demand during peak period by 13% and also quantities the impact of using simplified modeling approaches for determining the potential of EWHs for providing grid services.

42 ENGINEERING↗
DOE OSTI2020

Development and evaluation of a Novel RT‐PCR system for reliable and rapid SARS‐CoV ‐2 screening of blood donations

Abstract Background The ongoing outbreak of severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2) has caused great global concerns. In contrast to SARS, some SARS‐CoV‐2–infected people can be asymptomatic or have only mild nonspecific symptoms. Furthermore, there is evidence that SARS‐CoV‐2 may be infectious during an asymptomatic incubation period. With the discovery that SARS‐CoV‐2 can be detected in plasma or serum, blood safety is worthy of consideration. Study Design and Methods We developed a nucleic acid test (NAT) screening system for SARS‐CoV‐2 targeting nucleocapsid protein (N) and open reading frame 1ab (ORF 1ab) gene that could screen 5076 samples every 24 hours. The 2019 novel coronavirus RNA standard was used to evaluate linearity of standard curves. Diagnostic sensitivity and reproducibility were evaluated using artificial SARS‐CoV‐2. Specificity was evaluated with 61 other respiratory pathogens. Diagnostic performance was evaluated by testing two sputum and nine oropharyngeal swab specimens. The reverse transcription polymerase chain reaction (RT‐PCR) assay was used to screen SARS‐CoV‐2 RNA in blood donor specimens collected during the outbreak of SARS‐CoV‐2 in Chengdu. Results Limits of detection of the SARS‐CoV‐2 RT‐PCR assay for N and ORF 1ab gene were 12.5 and 27.58 copies/mL, respectively. Intra‐assay and interassay for the SARS‐CoV‐2 RT‐PCR assay based on cycle threshold were acceptably low. No cross‐reactivity was observed with other respiratory virus and bacterial isolates. The overall agreement value between the SARS‐CoV‐2 RT‐PCR assay and clinical diagnostic results was 100%. A total of 16 287 blood specimens collected from blood donors during SARS‐CoV‐2 surveillance were tested negative. Conclusions A high‐throughput NAT screening system was developed for SARS‐CoV‐2 screening of blood donations during the outbreak of SARS‐CoV‐2.

Li, Meng↗
DOE OSTI2025

HPC Digital Twins for Evaluating Scheduling Policies, Incentive Structures and their Impact on Power and Cooling

Schedulers are critical for optimal resource utilization in high-performance computing. Traditional methods to evaluate sched- ulers are limited to post-deployment analysis, or simulators, which do not model associated infrastructure. In this work, we present the first-of-its-kind integration of scheduling and digital twins in HPC. This enables what-if studies to understand the impact of parameter configurations and scheduling decisions on the physical assets, even before deployment, or regarching changes not easily realizable in production. We (1) provide the first digital twin framework extended with scheduling capabilities, (2) integrate various top-tier HPC systems given their publicly available datasets, (3) implement extensions to integrate external scheduling simulators. Finally, we show how to (4) implement and evaluate incentive structures, as- well-as (5) evaluate machine learning based scheduling, in such novel digital-twin based meta-framework to prototype scheduling. Our work enables what-if scenarios of HPC systems to evaluate sustainability, and the impact on the simulated system.

Maiterth, Matthias [ORNL] (ORCID:000000018698460X)↗
DOE OSTI2025

Graph-based Reversible Evaluation and Tangents Library

GRETL is a C++ library for evaluation, re-evaluation and algorithmic differentiation of functional operations on an arbitrary computational graph with limited memory usage. Similar to popular machine learning frameworks in Python, like PyTorch and JAX, it tracks and stores both operations and output data as functions are evaluated. Once this composition of functions is built up, the entire chain of operations can be back propagated to compute sensitivities of the final result with respect to any number of inputs. In contrast to most machine learning applications, memory usage becomes the bottleneck for back propagation in many physics applications, especially for time-dependent PDEs. Dynamic check pointing becomes essential. An important distinguishing feature of GRETL is its ability to limit the maximum memory usage by automatically dynamic checkpointing the data output for each graph operation (see Wang, Moin, Iaccarino, 2009). During backpropagation, parts of the graph that are no longer in memory are automatically re-evaluated from upstream checkpointed states as needed for derivative sensitivity calculations (or more precisely, for vector-Jacobian products). GRETL is particularly beneficial for applications, such as coupled multi-physics, where deriving adjoint-based sensitivities and managing checkpoint memory across modules becomes onerous. Cases which can be readily handled by the GRETL library include: different time-integration algorithms per physics (e.g., coupled predictor-corrector algorithms, IMEX, etc.), sub-cycling, asynchronous integrators, state dependent timestep sizes, iterative solvers and coupling algorithms, controller algorithms, and more.

Tupek, MichaelR [Lawrence Livermore National Labor↗
DOE OSTI2021

Evaluating Climate Models with the CLIVAR 2020 ENSO Metrics Package

El Niño–Southern Oscillation (ENSO) is the dominant mode of interannual climate variability on the planet, with far-reaching global impacts. It is therefore key to evaluate ENSO simulations in state-of-the-art numerical models used to study past, present, and future climate. Recently, the Pacific Region Panel of the International Climate and Ocean: Variability, Predictability and Change (CLIVAR) Project, as a part of the World Climate Research Programme (WCRP), led a community-wide effort to evaluate the simulation of ENSO variability, teleconnections, and processes in climate models. The new CLIVAR 2020 ENSO metrics package enables model diagnosis, comparison, and evaluation to 1) highlight aspects that need improvement; 2) monitor progress across model generations; 3) help in selecting models that are well suited for particular analyses; 4) reveal links between various model biases, illuminating the impacts of those biases on ENSO and its sensitivity to climate change; and to 5) advance ENSO literacy. By interfacing with existing model evaluation tools, the ENSO metrics package enables rapid analysis of multipetabyte databases of simulations, such as those generated by the Coupled Model Intercomparison Project phases 5 (CMIP5) and 6 (CMIP6). The CMIP6 models are found to significantly outperform those from CMIP5 for 8 out of 24 ENSO-relevant metrics, with most CMIP6 models showing improved tropical Pacific seasonality and ENSO teleconnections. Only one ENSO metric is significantly degraded in CMIP6, namely, the coupling between the ocean surface and subsurface temperature anomalies, while the majority of metrics remain unchanged.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

A Surface Radiation Balance Dataset from Siple Dome in West Antarctica for Atmospheric and Climate Model Evaluation

Abstract A field campaign at Siple Dome in West Antarctica during the austral summer 2019/20 offers an opportunity to evaluate climate model performance, particularly cloud microphysical simulation. Over Antarctic ice sheets and ice shelves, clouds are a major regulator of the surface energy balance, and in the warm season their presence occasionally induces surface melt that can gradually weaken an ice shelf structure. This dataset from Siple Dome, obtained using transportable and solar-powered equipment, includes surface energy balance measurements, meteorology, and cloud remote sensing. To demonstrate how these data can be used to evaluate model performance, comparisons are made with meteorological reanalysis known to give generally good performance over Antarctica (ERA5). Surface albedo measurements show expected variability with observed cloud amount, and can be used to evaluate a model’s snowpack parameterization. One case study discussed involves a squall with northerly winds, during which ERA5 fails to produce cloud cover throughout one of the days. A second case study illustrates how shortwave spectroradiometer measurements that encompass the 1.6- μ m atmospheric window reveal cloud phase transitions associated with cloud life cycle. Here, continuously precipitating mixed-phase clouds become mainly liquid water clouds from local morning through the afternoon, not reproduced by ERA5. We challenge researchers to run their various regional or global models in a manner that has the large-scale meteorology follow the conditions of this field campaign, compare cloud and radiation simulations with this Siple Dome dataset, and potentially investigate why cloud microphysical simulations or other model components might produce discrepancies with these observations. significance statement Antarctica is a critical region for understanding climate change and sea level rise, as the great ice sheets and the ice shelves are subject to increasing risk as global climate warms. Climate models have difficulties over Antarctica, particularly with simulation of cloud properties that regulate snow surface melting or refreezing. Atmospheric and climate-related field work has significant challenges in the Antarctic, due to the small number of research stations that can support state-of-the-art equipment. Here we present new data from a suite of transportable and solar-powered instruments that can be deployed to remote Antarctic sites, including regions where ice shelves are most at risk, and we demonstrate how key components of climate model simulations can be evaluated against these data.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Evaluating uncertainty-based active learning for accelerating the generalization of molecular property prediction

Deep learning models have proven to be a powerful tool for the prediction of molecular properties for applications including drug design and the development of energy storage materials. However, in order to learn accurate and robust structure–property mappings, these models require large amounts of data which can be a challenge to collect given the time and resource-intensive nature of experimental material characterization efforts. Additionally, such models fail to generalize to new types of molecular structures that were not included in the model training data. The acceleration of material development through uncertainty-guided experimental design has the promise to significantly reduce the data requirements and enable faster generalization to new types of materials. To evaluate the potential of such approaches for electrolyte design applications, we perform comprehensive evaluation of existing uncertainty quantification methods on the prediction of two relevant molecular properties - aqueous solubility and redox potential. We develop novel evaluation methods to probe the utility of the uncertainty estimates for both in-domain and out-of-domain data sets. Finally, we leverage selected uncertainty estimation methods for active learning to evaluate their capacity to support experimental design.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Evaluation of Thermal Neutron Scattering Cross Sections for CaH 2

Solid metal hydrides have long been considered viable moderators for nuclear reactor designs due to their moderating ratios, hydrogen densities, high dissociation temperatures, and mechanical properties. CaH 2 is an orthorhombic saline-hydride that has recently been investigated and shows promise for use as a moderator in microreactors. At present, there is no Thermal Scatting Law (TSL) evaluation for CaH 2 in the ENDF/B-VIII.0 database. An evaluation does exist in the JEFF-3.3 database, performed by Serot, however it was limited by the implemented methods and thus both drastically over-predicts the incoherent elastic contribution and completely ignores the coherent elastic contribution to scattering from the metal ions. The thermal neutron scattering cross sections of CaH 2 are evaluated in this work to establish accurate data for use in reactor design. The present evaluation corrects the inaccuracies of the JEFF-3.3 data, and yields three distinct libraries: Ca in CaH 2 , H 1 in CaH 2 , and H 2 in CaH 2 .

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Evaluation of Oak Ridge National Laboratory Health Physics Research Reactor Operation Data for Critical Benchmark Creation

The Oak Ridge National Laboratory (ORNL) Health Physics Research Reactor (HPRR) was a research reactor designed and built at ORNL in 1961. The critical assembly used a highly enriched uranium and molybdenum alloy as the fuel and could be operated in steady-state or burst modes. The HPRR has recently been the object of an investigation to create a criticality benchmark. Such benchmarks are very important, as they are used primarily to show the accuracy of newly developed modeling codes and to help experimental validation and reactor licensing. The evaluated experiments considered in this paper were carried out between 1974 and 1986 from various HPRR activities such as steady-state subcritical, steady-state critical, and burst prompt super-critical operations of the reactor for dosimetry, irradiation, or training purposes. By using the HPRR experimental logbook information and the as-built drawings of the critical assembly, a highly detailed model of the HPRR was created with SCALE 6.2.4/KENO-VI, and a first version of a critical benchmark of the HPRR was developed following the International Criticality Safety Benchmark Evaluation Project (ICSBEP) guidelines for thorough description and uncertainty/sensitivity quantification. Unfortunately, in most of the evaluated experiments, the obtained difference between calculated and experimental k eff is around 1,000 pcm, corresponding to a relative error of approximately 1%, beyond the quality standards of the ICSBEP recommending a relative error below 0.1%. Moreover, the derived experimental uncertainty is high, around 4% relative, mainly due to the U-Mo fuel density uncertainty, but also from numerous other factors. For these reasons, the creation of a valuable critical benchmark from HPRR operation data is thus far compromised. In this paper, the different steps of the experiments’ evaluation are summarized, and the reasons for the experimental/calculation discrepancies and potential ways to solve them are explored. This paper also aims to remind us always to exercise considerable care when performing experimental work, and to record all the data possible for potential future uses.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2020

Evaluating kratom alkaloids using PHASE

Kratom is a botanical substance that is marketed and promoted in the US for pharmaceutical opioid indications despite having no US Food and Drug Administration approved uses. Kratom contains over forty alkaloids including two partial agonists at the mu opioid receptor, mitragynine and 7-hydroxymitragynine, that have been subjected to the FDA’s scientific and medical evaluation. However, pharmacological and toxicological data for the remaining alkaloids are limited. Therefore, we applied the Public Health Assessment via Structural Evaluation (PHASE) protocol to generate in silico binding profiles for 25 kratom alkaloids to facilitate the risk evaluation of kratom. PHASE demonstrates that kratom alkaloids share structural features with controlled opioids, indicates that several alkaloids bind to the opioid, adrenergic, and serotonin receptors, and suggests that mitragynine and 7-hydroxymitragynine are the strongest binders at the mu opioid receptor. Subsequently, the in silico binding profiles of a subset of the alkaloids were experimentally verified at the opioid, adrenergic, and serotonin receptors using radioligand binding assays. The verified binding profiles demonstrate the ability of PHASE to identify potential safety signals and provide a tool for prioritizing experimental evaluation of high-risk compounds.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2023

Laboratory evaluation of open source and commercial electrical conductivity sensor precision and accuracy: How do they compare?

Variation in the electrical conductivity (EC) of water can reveal environmental disturbance and natural dynamics, including factors such as anthropogenic salinization. Broader application of open source (OS) EC sensors could provide an inexpensive method to measure water quality. While studies show that other water quality parameters can be robustly measured with sensors, a similar effort is needed to evaluate the performance of OS EC sensors. To address this need, we evaluated the accuracy (mean error, %) and precision (sample standard deviation) of OS EC sensors in the laboratory via comparison to EC calibration standards using three different OS and OS/commercial-hybrid (OS/C) EC sensors and data logger configurations and two commercial (C) EC sensors and data logger configurations. We also evaluated the effect of cable length (7.5 m and 30 m) and sensor calibration on OS sensor accuracy and precision. We found a significant difference between OS sensor mean accuracy (3.08%) and all other sensors combined (9.23%). Our study also found that EC sensor precision decreased across all sensor configurations with increasing calibration standard EC. There was also a significant difference between OS sensor mean precision (2.85 μS/cm) and the mean precision of all other sensors combined (9.12 μS/cm). Cable length did not affect OS sensor precision. Furthermore, our results suggest that future research should include evaluating how performance is impacted by combining OS sensors with commercial data loggers as this study found significantly decreased performance in OS/commercial-hybrid sensor configurations. To increase confidence in the reliability of OS sensor data, more studies such as ours are needed to further quantify OS sensor performance in terms of accuracy and precision across different settings and OS sensor and data collection platform configurations.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Evaluation of Seismic Artificial Intelligence with Uncertainty

Artificial intelligence has transformed the seismic community with deep learning models (DLMs) that are trained to complete specific tasks within workflows. However, there is still a lack of robust evaluation frameworks for evaluating and comparing DLMs. Here, we address this gap by designing an evaluation framework that jointly incorporates two crucial aspects: performance uncertainty and learning efficiency. To target these aspects, we meticulously construct the training, validation, and test splits using a clustering method tailored to seismic data and enact an expansive training design to segregate performance uncertainty arising from stochastic training processes and random data sampling. The framework’s ability to guard against misleading declarations of model superiority is demonstrated through the evaluation of PhaseNet (Zhu and Beroza, 2018), a popular seismic phase picking DLM, under three training approaches. Our framework helps practitioners choose the best model for their problem and set performance expectations by explicitly analyzing model performance with uncertainty at varying budgets of training data.

58 GEOSCIENCES↗