Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluations”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Controller Hardware-in-the-Loop Evaluation of a Microgrid Controller for a Microgrid System with Multiple Grid-Forming Inverters

This paper presents the laboratory evaluation of a commercial Microgrid Management System (MGMS) implemented in the real-world Bronzeville Microgrid which features a futuristic scenario with high renewable energy integration and the use of multiple Grid-Forming (GFM) inverters. The primary objective of the performance evaluation for the MGMS is to assess the MGMS's capability to dispatch GFM units, including a GFM PV unit and two GFM battery units, to maintain the system stability and ensure economic operation, thus guaranteeing the microgrid's resilience during prolonged outages and dynamic events. The laboratory controller hardware-in-the-loop provides realistic testing environment through detailed electromagnetic transient modeling of the microgrid system, hardware MGMS, and standard communication protocols (DNP3). The CHIL evaluation shows how the MGMS effectively manages the GFM inverters, highlighting its performance in maintaining stability, reliability, and survivability in a microgrid environment with a high penetration of renewable energy sources.

controller hardware-in-the-loop↗
DOE OSTI2024

Long-Range Biometric Identification in Real World Scenarios: A Comprehensive Evaluation Framework Based on Missions

The considerable body of data available for evaluating biometric recognition systems in Research and Development (R&D) environments has contributed to the increasingly common problem of target performance mismatch. Biometric algorithms are frequently tested against data that may not reflect the real world applications they target. From a Testing and Evaluation (T&E) standpoint, this domain mismatch causes difficulty assessing when improvements in State-of-the-Art (SOTA) research actually translate to improved applied outcomes. This problem can be addressed with thoughtful preparation of data and experimental methods to reflect specific use-cases and scenarios.To that end, this paper evaluates research solutions for identifying individuals at ranges and altitudes, which could support various application areas such as counterterrorism, protection of critical infrastructure facilities, military force protection, and border security. We address challenges including image quality issues and reliance on face recognition as the sole biometric modality. By fusing face and body features, we propose developing robust biometric systems for effective long-range identification from both the ground and steep pitch angles. Preliminary results show promising progress in whole-body recognition. This paper presents these early findings and discusses potential future directions for advancing long-range biometric identification systems based on mission-driven metrics.

Aykac, Deniz↗
DOE OSTI2023

Performance Evaluation of Next-Generation Grid Automation and Controls with High PV Penetration

This paper presents a hardware-in-the-loop (BIL) simulation to evaluate the performance of an advanced grid automation architecture, referred to as data-enhanced hierarchical control (DEHC), in achieving voltage regulation and conservation voltage reduction (CVR) in distribution networks with very high photovoltaic (PV) generation. This architecture comprises an advanced distribution management system (ADMS), a distributed energy resource management system (DERMS), and grid-edge devices working synergistically to provide the grid benefits. The HIL setup used for the evaluation includes ADMS, DERMS, and grid-edge devices. The DEHC performance is evaluated in two representative scenarios considering loose and tight constraints of the power factor at the substation. The results show that the DEHC architecture is effective in achieving voltage regulation and CVR and thus enables the grid integration of high levels of PV generation.

ADMS↗
DOE OSTI2022

Evaluation of an LED Simulator for Single- and Multi-Junction PV Cell Performance Testing

We present the evaluation of a solar simulator comprising a Light Emitting Diode (LED) array as its light source. While multiple applications can be envisioned for such a light source, here we focus on its use for performance testing of solar cells at Standard Test Conditions. With this in mind, we present characterization of the spatial uniformity of the irradiance and its temporal stability as well as the spectral range and spectral class. For a multisource simulator, the spectral uniformity across the test plane should also be evaluated and we discuss a method for performing this evaluation that is well suited to a solar cell performance testing application. One major advantage of this simulator is the ability to easily adjust the spectral output as needed in multijunction PV cell measurements and we also discuss this application here.

LED↗
DOE OSTI2023

Evaluating Robust Entanglement on a Trapped Ion Platform

We present proof of principle results for evaluating an entanglement volumetric benchmark [1] on trapped ion platforms. The benchmark quantifies the robustness of multipartite and bipartite entanglement using stabilizer measurements and witness functions. Each n-qubit graph state is prepared and used to evaluate n state-specific stabilizer strings. These stabilizer measurements are used to evaluate entanglement witness functions. The entanglement benchmark defines families of graph states associated with an initial sub-graph of the hardware qubit connectivity - with all-to-all connectivity, trapped ion systems provide a flexibility in the choice of this initial graph, and this will affect the associated family. In this work, we present results targeting several classes of entangled states: 1) 1-D cluster states, 2) n-qubit GHZ states, and 3) cycle graph states. These states have been found in the literature as standard hardware benchmarks, and have connections to many near-term applications.

Hamilton, Kathleen↗
DOE OSTI2024

Advancing Sustainability in Data Centers: Evaluation of Hybrid Air/Liquid Cooling Schemes for IT Payload Using Sea Water

Abstract-The growth in cloud computing, Big Data, AI and high-performance computing (HPC) necessitate the deployment of additional data centers (DC's) with high energy demands. The unprecedented increase in the Thermal Design Power (TDP) of the computing chips will require innovative cooling techniques. Furthermore, DC's are increasingly limited in their ability to add powerful GPU servers by power capacity constraints. As cooling energy use accounts for up to 40% of DC energy consumption, creative cooling solutions are urgently needed to allow deployment of additional servers, enhance sustainability and increase energy efficiency of DC's. The information in this study is provided from Start Campus' Sines facility supported by Alfa Laval for the heat exchanger and CO 2 emission calculations. The study evaluates the performance and sustainability impact of various data center cooling strategies including an air-only deployment and a subsequent hybrid air/water cooling solution all utilizing sea water as the cooling source. Here we evaluate scenarios from 3 MW to 15+1 MW of IT load in 3 MW increments which correspond to the size of heat exchangers used in the Start Campus' modular system design. This study also evaluates the CO 2 emissions compared to a conventional chiller system for all the presented scenarios. Results indicate that the effective use of the sea water cooled system combined with liquid cooled systems improve the efficiency of the DC, plays a role in decreasing the CO 2 emissions and supports in achieving sustainability goals.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Using an Advanced Distribution Management System Test Bed to Evaluate the Impact of Model Quality on Volt/VAR Optimization: Preprint

In this paper, we present a test bed for evaluating existing and future advanced distribution management system (ADMS) applications in a realistic laboratory setting, including other utility management systems and field equipment. We present an example of using it to evaluate the impact of the ADMS network model quality on a Volt/VAR optimization (VVO) application. The test bed integrates a commercial ADMS with a real-time simulation model of a utility distribution feeder. Representative power and controller hardware are integrated through hardware-in-the-loop (HIL) techniques. The performance of the ADMS VVO application is also evaluated for different levels of measurement density. Initial results indicate that a higher model quality achieves the highest possible energy savings while avoiding voltage violations, whereas a lower model quality results in increased energy savings but at the expense of more voltage violations.

ADMS↗
DOE OSTI2020

Hardware-in-the-Loop Test Bed and Test Methodology for Microgrid Controller Evaluation

Increasing penetration levels of distributed energy resources are making the deployment of microgrids more feasible. Controllers that operate such microgrids are key to realizing the objectives of the microgrid owner or operator and there is a need to evaluate microgrid controller performance prior to field deployment. This paper describes a controller hardware-in-the-loop and power hardware-in-the-loop microgrid controller test bed that was designed and constructed to evaluate the capabilities of a microgrid controller for a proposed campus microgrid. This paper also presents a test methodology to evaluate microgrid controller functionality, and it describes how the controller was assessed through the application of different test scenarios. Results from the testing are presented to provide insight into the capabilities of the test bed.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Evaluation of Copper Catalytic Effects in Cable Insulation Polymers

This paper summarizes work performed to evaluate a phenomenon that occurs in electrical cable insulation polymers subjected to accelerated aging while in contact with copper metal. This effect, commonly known as the copper catalytic effect, is a result of chemical reactions that occur when copper ions diffuse into insulation polymers. This diffusion process is observed in various types of polymeric materials exposed to elevated temperatures and happens at the interface between the insulation and metallic components (e.g., conductor, shielding, etc.) in a cable. This polymer-metal interaction has only been observed in cables constructed with copper components (i.e., no interactions observed in cables with aluminum or other metal conductor/shielding) and results in a significant catalytic effect that increases the oxidation rate (e.g., aging) of the material. Under this research, the copper catalytic effects observed in cross-linked polyethylene, cross-linked polyolefin and ethylene propylene rubber insulated cables subjected to thermal accelerated aging were evaluated. These evaluations involved applying infrared spectroscopy, microscopy, and energy dispersive X-ray spectroscopy cross-sectional depth profiling to obtain an in-depth understanding of the aging characteristics of the materials under accelerated conditions. Based on the results of these assessments, the copper catalytic effect can have a significant impact on the mechanical, thermal, chemical, and electrical properties of cable insulation polymers. Here, the results acquired from this research provided the information needed to characterize the copper catalytic effects observed in these polymers, analyze how this phenomenon affected the polymer degradation process, and compare and understand the differences in the properties of the materials.

42 ENGINEERING↗
DOE OSTI2020

Evaluating the potential for bird‐habitat models to support biodiversity‐friendly urban planning

Abstract Urban expansion poses a major threat to wildlife populations. Biodiversity‐friendly urban landscapes could deliver benefits for both wildlife and people, by incorporating conservation and ecosystem services objectives. Well‐designed urban developments could also soften the ecological impacts of urbanization. However, delivering urban landscapes that integrate biodiversity requirements effectively remains challenging. Ecological models, designed to predict wildlife population responses to alternative urban designs, could prove effective in supporting the creation of biodiversity‐friendly urban landscapes. Here, we combine national‐scale bird abundance data with high resolution, spatially explicit habitat data to characterize relationships between bird densities and urban landscape form in Britain. From these analyses and cross validation, we evaluate the potential for well‐parameterized, species‐specific models to be used to predict bird densities in novel or modified urban areas. Our analyses indicate that responses of bird abundance to urban habitat are species‐specific and complex, with few variables consistently affecting a large proportion of species. However, contiguous areas of greenspace within urban sites are preferential for accommodating breeding birds, compared to a more fragmented arrangement of multiple, small greenspace patches. In combination, the bird‐habitat relationships identified could successfully predict observed variation in abundance for most bird species considered. Further evaluation of habitat descriptor variables, spatial scales of species' habitat use and analytical modelling approaches may be needed to improve the predictive ability of bird‐habitat models for certain species, particularly waterbirds and those observed less frequently in urban areas. Synthesis and applications . We modelled breeding bird abundance in built‐up areas with respect to the characteristics and contexts of urban environments. While most variables were important for multiple species, responses overall were species‐specific, so simple assemblage metrics, like diversity, will not describe the variation in bird communities well. However, the results illustrate the potential of an evidence‐based, spatially explicit evaluation of urban development impacts on biodiversity, by predicting the consequences for bird numbers. Subject to verification of predictive ability, practitioners can apply the models to compare, for example, land‐sparing and sharing within developments, or to quantify the biodiversity requirements for effective offsetting. This would be facilitated by incorporation into an online tool allowing user‐determined input scenarios.

Plummer, Kate E.↗
DOE OSTI2022

A New Evaluation Metric for Demand Response-Driven Real-Time Price Prediction Towards Sustainable Manufacturing

Abstract The increasing industry energy demand highlights the urgency of demand response management, while the emerging smart manufacturing technologies pave the way for the implementation of real-time price (RTP)-based demand response management towards sustainable manufacturing. The demand response management requires scheduling of manufacturing systems based on RTP predictions, and thus the prediction quality can directly alter the effectiveness of demand response. However, since the general price prediction algorithms and prediction evaluation metrics are not specifically designed for RTP in demand response problems, a good RTP prediction obtained and evaluated by these algorithms and metrics may not be suitable for demand response scheduling. Therefore, in this study, the relationships between the effectiveness of demand response for manufacturing systems and evaluation results from six commonly used metrics are investigated. Meanwhile, a new metric called k-peak distance (KPD), considering the characteristics of the demand response problem, is proposed and compared with the other six metrics. Furthermore, an encoder-decoder long short-term memory recurrent neural network with KPD is proposed to provide better RTP prediction for manufacturing demand response problems. The case studies indicate that the proposed KPD metric shows a 1.8–3.6 times higher correlation with the demand response effectiveness compared to the other metrics. In addition, the production schedule based on the RTP prediction obtained from the proposed algorithm can improve the effectiveness of demand response by 23.4% on average.

Engineering↗
DOE OSTI2024

Evaluating Model Robustness for Defect Identification and Classification in a Composite Aerostructure Material

Aircraft structures are required to have a high level of quality to satisfy their need for light weight, efficient flight, and withstanding high loads over their lifespan. These aerostructures are typically made from a composite material due to their good tensile strength and resistance to compression. To ensure their structural integrity, the composite material requires inspection for common flaws such as porosity, delaminations, voids, foreign object debris, and other defects. Ultrasonic testing (UT) is a popular non-destructive inspection (NDI) technique used for effectively evaluating the composite material. Current inspection methods rely heavily on human experience and are extremely time consuming. Therefore, there is a need for the development of techniques to reduce the manual inspection time. This work compares the performance of different deep learning-based methods in the identification and classification of defects. Deep learning has shown great promise in numerous fields, and we show its effectiveness in the evaluation of the composite aerostructure material. The methods developed here are both highly reliable with a top recall value of 98.64% as well as extremely efficient requiring an average of 4 s during the inferencing stage to evaluate new composites. Lastly, we investigate model robustness to concept drift by measuring its performance over time.

36 MATERIALS SCIENCE↗
DOE OSTI2023

QASMBench: A Low-Level Quantum Benchmark Suite for NISQ Evaluation and Simulation

The rapid development of quantum computing (QC) in the NISQ era urgently demands a low-level benchmark suite and insightful evaluation metrics for characterizing the properties of prototype NISQ devices, the efficiency of QC programming compilers, schedulers and assemblers, and the capability of quantum system simulators in a classical computer. In this work, we fill this gap by proposing a low-level, easy-to-use benchmark suite called QASMBench based on the OpenQASM assembly representation. It consolidates commonly used quantum routines and kernels from a variety of domains including chemistry, simulation, linear algebra, searching, optimization, arithmetic, machine learning, fault tolerance, cryptography, and so on, trading-off between generality and usability. To analyze these kernels in terms of NISQ device execution, in addition to circuit width and depth, we propose four circuit metrics including gate density, retention lifespan, measurement density, and entanglement variance, to extract more insights about the execution efficiency, the susceptibility to NISQ error, and the potential gain from machine-specific optimizations. Applications in QASMBench can be launched and verified on several NISQ platforms, including IBM-Q, Rigetti, IonQ and Quantinuum. For evaluation, we measure the execution fidelity of a subset of QASMBench applications on 12 IBM-Q machines through density matrix state tomography, comprising 25K circuit evaluations. In addition we also compare the fidelity of executions among the IBM-Q machines, the IonQ QPU and the Rigetti Aspen M-1 system.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Evaluation of Cathode Materials with Lithium-Metal Anodes: Baseline Performance and Protocol Standardization of Coin Cells

The collaborative evaluation of electrode materials across multiple research entities requires standardized electrochemical testing protocols to produce reliable, one-to-one comparisons between different systems of interest. Similar to the work done by Long et al. on protocol standardization for coin-cell testing with graphite anodes [J. Electrochem. Soc., 163, A2999, (2016)], here we introduce two standardized testing protocols designed to quickly evaluate important electrochemical properties of cathode materials using lithium-metal anodes. The two protocols measure kinetic and thermodynamic capacity losses, rate- and voltage-dependent cycling capacities, instabilities at high voltage and high cycling rate, and overpotentials at various states of charge. We then apply these protocols to four commercially available cathode materials to establish benchmark performance metrics that can be used to screen and evaluate new cathode materials.

25 ENERGY STORAGE↗
DOE OSTI2024

MLEC-Sim: A Simulator for Evaluating Multi-Level Erasure Coding

We present MLEC-Sim, a sophisticated simulator for Multi-Level Erasure Coding (MLEC), developed in approximately 13 KLOC. The simulator is engineered to analyze the impact of various system configurations and erasure coding policies on system durability and network overhead. It supports a comprehensive range of parameters including disk capacity, disk I/O bandwidth, failure rates, network bandwidth, and system scale, accommodating various erasure coding approaches such as Single-Level Erasure Coding (SLEC), Multi-Level Erasure Coding (MLEC), and Local Reconstruction Codes (LRC). MLEC-Sim provides support for multiple chunk placement policies, including clustered parity and declustered parity, and encompasses a variety of repair methods like Repair-ALL, Repair-FCO, Repair-HYB, and Repair-MIN. It is capable of simulating disk failures through a variety of means, including distribution-based or trace-based mechanisms, and can handle complex multi-level (de)clustered placements and repair processes. A key feature of MLEC-Sim is its adoption of the splitting simulation method for evaluating system durabilities at extremely high levels, which are challenging to assess with traditional simulation approaches. This feature allows for a detailed evaluation of system resilience under a range of conditions, aiding in the selection of appropriate erasure coding solutions for enhancing system durability. MLEC-Sim contributes to the field of data storage and reliability by providing a tool for the detailed evaluation of the durability and efficiency of erasure coding configurations, intended for use by researchers and practitioners in the design and optimization of storage systems.

Wang, Meng↗
DOE OSTI2021

Analytical Tool to evaluate Heterogeneous Neuromorphic Architectures (ATHENA)

SAND2022-7062 O The Analytical Tool to Evaluate Heterogeneous Neuromorphic Architectures (ATHENA) quickly evaluates performance metrics like energy, area, and latency for an AI/ML network. It currently supports the evaluation of analog neural networks. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Hahn, Andrew↗
DOE OSTI2020

The ARM Data-oriented Metrics and Diagnostics Package for Climate Models - A New Tool for Evaluating Climate Models with Field Data

The U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) program User Facility produces ground-based long-term continuous unique measurements for atmospheric state, precipitation, turbulent fluxes, radiation, aerosol, cloud and the land surface, which are collected at multiple sites. These comprehensive datasets have been widely used to calibrate climate models and are proven to be invaluable for climate model development and improvement. This article introduces an evaluation package to facilitate the use of ground-based ARM measurements in climate model evaluation. The ARM data-oriented metrics and diagnostics package (ARM-DIAGS) includes both ARM observational datasets and a Python-based analysis toolkit for computation and visualization. The observational datasets are compiled from multiple ARM data products and specifically tailored for use in climate model evaluation. In addition, ARM-DIAGS also includes simulation data from models participating the Coupled Model Inter-comparison Project (CMIP), which will allow climate-modeling groups to compare a new, candidate version of their model to existing CMIP models. The analysis toolkit is designed to make the metrics and diagnostics quickly available to the model developers.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Evaluation of Mesoscale Convective Systems in Climate Simulations: Methodological Development and Results from MPAS-CAM over the U.S.

In this study, a process-oriented approach is developed to evaluate warm-season mesoscale convective system (MCS) precipitation and their favorable large-scale meteorological patterns (FLSMPs) over the U.S. This approach features a novel observation-driven MCS-tracking algorithm using infrared brightness temperature and precipitation feature at 12, 25 and 50 km resolution and metrics to evaluate the model large-scale environment favorable for MCS initiation. The tracking algorithm successfully reproduces the observed MCS statistics from a reference 4-km radar MCS database. To demonstrate the utility of the new methodologies in evaluating MCS in climate simulations with mesoscale resolution, the process-oriented approach is applied to two climate simulations produced by the Variable-Resolution Model for Prediction Across Scales coupled to the Community Atmosphere Model physics, with refined horizontal grid spacing at 50 km and 25 km over North America. With the tracking algorithm applied to simulations and observations at equivalent resolutions, the simulated number of MCS and associated precipitation amount, frequency and intensity are found to be consistently underestimated in the Central U.S., particularly from May to August. The simulated MCS precipitation shows little diurnal variation and lasts too long, while MCS precipitation area is too large and intensity is too weak. The model is able to simulate four types of observed FLSMP associated with frontal systems and low-level jets (LLJ) in spring, but the frequencies are underestimated because of low-level dry bias and weaker LLJ. Precipitation simulated under different FLSMPs peak during daytime, in contrast to the observed nocturnal peak. Implications of these findings for future model development and diagnostics are discussed.

54 ENVIRONMENTAL SCIENCES↗