Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluation framework”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Sensor impacts on building and HVAC controls: A critical review for building energy performance

Building operations rely heavily on control systems and sensors. This paper provides a sophisticated literature review on sensor systems in building/HVAC systems, particularly in the context of controls, and their impacts on energy efficiency and thermal comfort. This study aims to understand the previous and current research and identify future research opportunities on this subject. The reviewed sensor systems were used for heuristic rule-based controls, local controls, and advanced optimal controls for existing and emerging technologies. Five major aspects of sensors are reviewed here: control loops for sensors, sensor types, sensor locations, sensor data, and a sensor impact evaluation framework. To augment the literature review, comprehensive standardized interviews were also conducted with relevant industry experts and practitioners. These interviews were designed and performed to (1) identify significant factors for selecting sensor sets and current undergoing issues, (2) identify potential improvements in sensor configuration/deployment, and (3) integrate expert (e.g., researcher, building operation practitioner) knowledge and experiences to develop structured use-case scenarios. Researchers collected and analyzed 31 interview responses for this paper.

Building control↗
DOE OSTI2020

Anode-free, Lean-Electrolyte Lithium-Sulfur Batteries Enabled by Tellurium-Stabilized Lithium Deposition

For realizing practically viable lithium-sulfur (Li-S) batteries, it is imperative to stabilize Li deposition and improve cyclability while reducing excess Li and electrolyte. We have discovered that introducing tellurium (Te) into the Li-S system as a cathode additive significantly improves the reversibility of Li plating and stripping by forming a tellurized and sulfide-rich solid-electrolyte interphase (SEI) layer on the Li surface. A remarkable improvement in cyclability is demonstrated in anode-free full cells with limited Li inventory and large-area Li-S pouch cells under lean electrolyte conditions. Tellurium reacts with polysulfides to generate soluble polytellurosulfides that migrate to the anode side and form stabilizing lithium thiotellurate and lithium telluride in situ as SEI components. A significant reduction in electrolyte decomposition on the Li surface is also engendered. This work demonstrates Te inclusion as a viable strategy for stabilizing Li deposition and establishes a robust evaluation framework for preserving electrochemical performance under limited Li and limited electrolyte conditions.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Uncertainty quantification and reliability assessment for intermodal freight transportation

Intermodal freight optimization models support cost-effective, low-emission, and timely goods movement by coordinating trucks, rail, and barges. These models determine optimal flows, routing, and modal switches while respecting infrastructure and operational constraints. However, their real-world utility is often undermined by pervasive uncertainties-such as fluctuating transportation costs and emissions, variable terminal capacities, and uncertain freight demand-that distort key performance outcomes, including total system cost, carbon footprint, and transit time reliability. This study presents a structured framework for quantifying uncertainty in intermodal freight transportation (IFT) optimization. The framework evaluates how input uncertainty affects system performance and reliability, a critical need for ensuring that model-based decisions remain robust under real-world variability, especially amid volatile fuel prices, shifting demand, and growing disruptions. It integrates three complementary methods: (1) Sobol-based global sensitivity analysis to identify influential parameters affecting cost, emissions, and transit time, (2) Monte Carlo-based capacity perturbation analysis to assess robustness under probabilistic facility disruptions, and (3) Monte Carlo filtering with Bayesian inference to detect threshold-based performance vulnerabilities. The results highlight diesel truck unit cost as the dominant driver of variability. To improve system resilience, planners should prioritize uncertainty in fuel-related parameters when designing intermodal strategies.

Intermodal freight transportation↗
DOE OSTI2023

Resilience assessment and planning in power distribution systems: Past and future considerations

High impact low probability (HILP) events such as hurricanes, heat waves, and floods have instigated widespread power outages and blackouts around the globe in the past decade. With the increasing challenges concerning the threats to the power distribution systems and the growing need to mitigate the impacts of the HILP events, resilience has become a crucial requirement for the power grid infrastructures. Numerous efforts have been made to define, measure, and characterize the resilience of power distribution systems. This study thoroughly reviews the state-of-the-art methods on the existing resilience evaluation framework and metrics. Here, the desirable characteristics of resilience metrics are highlighted, and the challenges associated with formulating, developing, and calculating such metrics are discussed. Next, we detail the state-of-the-art literature on planning solutions to ensure distribution system resilience. This paper aims to extract a deep insight into this challenging and critical research area and envision future opportunities that can guide the power distribution system operators/planners to formulate more effective mitigation strategies to enhance the resilience of power distribution systems.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

Thermomechanical analysis and modeling of a high-temperature light trapping planar cavity solar receiver

The development of durable particle-based high-temperature solar receivers is critical for advancing concentrating solar-thermal (CST) technologies to enable high-efficiency power generation and industrial process heat. Here, this study presents a computational framework to evaluate the thermomechanical performance of a proposed enclosed light-trapping planar cavity receiver designed for particle-based thermal energy systems. The receiver incorporates absorptive cavities and fluidized particle-bed channels to enhance heat capture and reduce thermal losses. Finite element analysis (FEA) is employed to assess stress, strain, and creep-fatigue behavior under concentrated solar flux using realistic thermal boundary conditions derived from coupled system models and experimental assembly parameters. The analysis investigates the influence of particle-to-wall heat transfer coefficients (HTC) ranging from 800 to 1800 W/m 2 .K on the thermomechanical response of six candidate high-temperature alloys: Alloy 740H, Alloy 282, Alloy 617, 316H, Alloy 230, and 800H. Results show that increasing HTC reduces thermal gradients, leading to lower stresses and strains and extended minimum predicted creep life. While all materials satisfy fatigue life requirements under the investigated conditions, significant differences in creep resistance are observed. Alloy 740H consistently exhibits the longest minimum predicted creep life and the most favorable durability margins, followed by Alloy 282, with the remaining materials showing reduced creep resistance under identical loading. The reported creep lives are conservative lower-bound estimates intended for comparative material evaluation. This framework highlights the critical roles of material selection and geometry optimization in improving mechanical durability and reliability of solar-thermal receivers, forming a foundation for future experimental validation and design optimization.

14 SOLAR ENERGY↗
DOE OSTI2021

Opportunities for Treatment and Reuse of Agricultural Drainage in the United States

Irrigation accounts for 42% of the total freshwater withdrawals in the United States. Climate change, the pressure of a growing population, degrading water quality, and increased competition from other sectors could constrain continuous supply to meet future agricultural water demand. This study presents an evaluation framework to assess the potential reuse of agricultural drainage water for crop irrigation. Using a regional approach, we review the current state of agricultural drainage treatment and reuse and the institutional, economic, and other barriers that can influence the reuse decision. In the 31 eastern states, agricultural drainage contains valuable nutrients that can be reused for irrigation with minimal treatment, while the 17 western states struggle with large volumes of saline drainage that can contain constituents of concern (e.g., selenium), preventing reuse without treatment. Using a new decision-support tool called WaterTAP3, a potential treatment train for saline agricultural drainage was analyzed to identify treatment challenges, research needs, and the potential implementation at a larger scale. As demonstrated by our case study, desalination of agricultural drainage is costly and energy intensive and will require sizable investments to fully develop and optimize technologies as well as manage the generated waste and brine.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2026

AutoLabs: cognitive multi-agent systems with self-correction for autonomous chemical experimentation

The automation of chemical research through self-driving laboratories (SDLs) promises to accelerate scientific discovery, yet the reliability and granular performance of the underlying AI agents remain critical, under-examined challenges. In this work, we introduce AutoLabs, a self-correcting, multi-agent architecture designed to autonomously translate natural-language instructions into executable protocols for a high-throughput liquid handler. The system engages users in dialogue, decomposes experimental goals into discrete tasks for specialized agents, performs tool-assisted stoichiometric calculations, and iteratively self-corrects its output before generating a hardware-ready file. We present a comprehensive evaluation framework featuring five benchmark experiments of increasing complexity, from simple sample preparation to multi-plate timed syntheses. Through a systematic ablation study of 20 agent configurations, we assess the impact of reasoning capacity, architectural design (single- vs. multi-agent), tool use, and self-correction mechanisms. Our results demonstrate that agent reasoning capacity is the most critical factor for success, reducing quantitative errors in chemical amounts (nRMSE) by over 85% in complex tasks. When combined with a multi-agent architecture and iterative self-correction, AutoLabs approaches expert-authored reference procedures on the benchmark (F1-score > 0.89) on challenging multi-plate syntheses. These findings establish a clear blueprint for developing robust and trustworthy AI partners for autonomous laboratories, highlighting the synergistic effects of modular design, advanced reasoning, and self-correction to ensure both performance and reliability in high-stakes scientific applications. Code: https://github.com/pnnl/autolabs

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

GPU-based optical simulation of the DARWIN detector

Abstract Understanding propagation of scintillation light is critical for maximizing the discovery potential of next-generation liquid xenon detectors that use dual-phase time projection chamber technology. This work describes a detailed optical simulation of the DARWIN detector implemented using Chroma, a GPU-based photon tracking framework. To evaluate the framework and to explore ways of maximizing efficiency and minimizing the time of light collection, we simulate several variations of the conventional detector design. Results of these selected studies are presented. More generally, we conclude that the approach used in this work allows one to investigate alternative designs faster and in more detail than using conventional Geant4 optical simulations, making it an attractive tool to guide the development of the ultimate liquid xenon observatory.

Instruments & Instrumentation↗
DOE OSTI2023

Evaluating the Risk of Enabling Energy Storage Systems to Provide Multiple Services

Battery energy storage systems (BESS) are emerging as a promising, versatile technology for modern power grids due to their inherent distributed nature, their ability to inject bidirectional power flows and to provide multiple grid services. With the proliferation of intermittent renewable energy resources, the BESS procurement is also expected to increase, thereby creating systematic and strategic opportunities for BESS to various revenue streams for providing multiple grid services. Utilizing a co-simulation architecture, this work presents evaluation framework for enabling Utility-scale BESS with simultaneous provision of multiple services. The proposed framework is implemented on a real-life feeder modeled in GridLAB-D, leveraging multi-service grid support. A Risk-Aware control strategy is also implemented in this work that that takes into account the uncertain nature of some of the stochastic services and ensures the BESS to deliver their service commitments. This promotes overall grid reliability and encourages BESS implementation through the revenue streams from providing multiple grid services.

Theisen, John↗
DOE OSTI2026

A System Level Comparative Study of SecondarySide Rectifiers for UAV Wireless Power Transfer in High-Altitude Airborne Energy Platforms

Wireless power transfer (WPT) systems for unmanned aerial vehicles (UAVs) must satisfy stringent design requirements to ensure reliable operation under constraints related to efficiency, thermal performance, altitude, weight, and reliability, particularly in airborne and high-altitude environments. Thermal management becomes increasingly challenging at higher altitudes due to reduced air density, while reliability is strongly affected by the number and complexity of active components. The weight and compactness of secondary-side rectifiers directly impact UAV flight endurance and system integration, influencing energy consumption and payload allocation. These factors collectively define the practical feasibility of UAV WPT systems and guide rectifier topology selection. This paper presents a comparative and experimentally validated evaluation framework for secondary-side rectifiers under UAV-specific constraints. Three representative rectifier architectures are analyzed using component-based assessment of weight, thermal behavior, and reliability, together with experimental efficiency measurements obtained from a 1 kW WPT prototype. The results show that the single-switch active rectifier achieves the highest peak efficiency of approximately 93.2% and an overall efficiency of 91%, while also offering advantages in weight and reliability. The findings demonstrate how constraint-oriented analysis, combined with experimental validation, enables informed selection of secondary-side rectifiers for UAV wireless power transfer applications, including emerging airborne and high-altitude platforms.

Asa, Erdem [ORNL] (ORCID:0000000190884812)↗
DOE OSTI2021

A Multi-Site Networked Hardware-in-Loop Platform for Evaluation of Interoperability and Distributed Intelligence at Grid-Edge

Electric power systems have experienced large increases in the number of intelligent, connected and controllable devices being deployed, leading to a high degree of distributed intelligence at the grid-edge. These devices, both utility-owned and consumer-owned, include but are not limited to: renewable generation sources, energy storage, remote switches, voltage regulators, and smart controllable loads such as electric vehicles. These new devices provide significant potential for increased operational flexibility that can be leveraged to achieve system reconfiguration, resiliency improvements, power quality improvements, and distribution system automation. However, there are two significant challenges that must be addressed before these assets can be leveraged for operations: interoperability and system level validation prior to deployment. Because of the complexity of distributed control systems, and their interactions with legacy centralized controls, a purely simulations-based approach for pre-deployment validation is not sufficient. It requires hardware-in-loop testing to emulate the operational hardware devices and evaluate their performance. Additionally, securely integrating multiple test facilities at utility operators and vendors might enable rapid scale-up of evaluation platforms, and remove the need for multiple expensive standalone installations. Presented in this paper, is the development of a multi-site evaluation platform that employs Advanced Distribution Management Systems (ADMS), distributed control devices, real-time hardware-in-loop assets, secure communication links, and protocol adapters. This platform uses standards-based approaches and open-source tools, and hence can serve as a template for other researchers and institutions to implement their multi-site evaluation frameworks for pre-deployment testing.

Essakiappan, Somasundaram↗
DOE OSTI2026

Detect the Unobservable: Abnormality Detection in mixed Autonomy for Lane Change Maneuver with Following Vehicles’ Trajectories Only

Highly Automated Vehicles (HAVs) and Advanced Driver-Assistance Systems (ADAS) are transforming modern transportation with enhanced mobility, safety, and efficiency. Despite their advantages, cybersecurity vulnerabilities in these systems can lead to abnormal behavior, posing significant risks to surrounding human-driven vehicles (HDVs) in mixed traffic environments. Here, this article addresses the challenge of detecting abnormal lateral movements of HAVs/ADAS vehicles using only trajectory profiles of following HDVs. Specifically, we propose a novel modeling approach that captures both normal and abnormal lateral behaviors through vehicle kinematics, integrated decision-making processes, vehicle control using symbolic regression for lane change vehicles. Additionally, we introduce an abnormality detection framework that relies on observable HDV data, even in occlusion scenarios. The framework evaluates the sensitivity of various car-following models to detect abnormal behaviors, providing insights into the interaction between HAVs/ADAS and HDVs in mixed autonomy systems.

Connected and Automated vehicles↗
DOE OSTI2023

pnnl/EXPERT2

This software includes the Jupyter notebooks, model pretraining and evaluation code for the EXPERT 2.0 Human-AI Reasoning Engine V0.1. It supports pre-training a Human-AI model for reasoning over multi-layer network representations. It includes prompt-based evaluation framework in a Jupyter notebook for AI reasoning and Jupyter widgets with AI-based techniques for evidence generation and uncertainty quantification to support human-AI reasoning.

Horawalavithana, Sameera↗
DOE OSTI2025

Using ARM Observations to Evaluate Process-Interactions in MCS Simulations Across Scales (Final Progress Report)

This project, funded by DOE Atmospheric System Research (DE-SC0020050), focused on improving the representation of mesoscale convective systems (MCSs) in numerical weather and climate models by leveraging high-resolution observations from the DOE Atmospheric Radiation Measurement (ARM) program. The research aimed to evaluate model sensitivities to grid spacing, microphysics, and planetary boundary layer (PBL) schemes, with a particular emphasis on improving convection parameterization for high-resolution modeling. Findings from this work highlight several key advancements. Model validation against ARM radar wind profiler data from the Southern Great Plains (SGP) and Manaus (MAO) sites revealed systematic biases in simulated convective mass flux profiles, leading to the development of an observationally constrained evaluation framework for diagnosing and improving model performance. Sensitivity analyses demonstrated that the representation of Amazonian MCSs was highly dependent on PBL scheme selection, while mid-latitude MCSs were more strongly influenced by microphysics parameterizations. A series of high-resolution WRF simulations, ranging from 4 km to 125 m grid spacing, provided insight into the behavior of convective drafts across scales. While updraft properties converged at sub-kilometer resolutions, biases in downdraft intensity persisted even at the finest resolution tested, emphasizing the need for further refinements in model physics. Additionally, comparisons of MCS vertical structures between mid-latitude and tropical environments revealed stronger updrafts and larger mass flux in mid-latitude MCSs, providing critical insights for improving climate model representations of storm-scale dynamics. The project’s findings have already contributed to advancing numerical modeling capabilities, particularly in WRF, MPAS, ICON, and DOE’s SCREAM model, by refining how convective processes are represented in high-resolution climate simulations. Results were disseminated through peer-reviewed publications, conference presentations, and ARM/ASR Research Highlights, engaging the broader scientific community. The project also provided valuable training opportunities for two postdoctoral researchers, who played central roles in model development, analysis, and dissemination of results. Their work contributed to several publications and conference presentations, helping prepare them for careers in atmospheric modeling. By improving the simulation of MCSs, this research directly supports the development of next-generation climate models capable of more accurately representing extreme precipitation and convective processes. The insights gained will inform future improvements in convective parameterization and guide the design of high-resolution weather and climate simulations, ultimately enhancing the reliability of climate projections and weather forecasts.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Evaluation of LLM-Generated Kokkos Code Using Compile-Time and Run-Time Testing

Due to the growing use of large language models (LLMs) by developers and researchers, it has become essential to reliably evaluate their ability to generate code that uses specialized libraries. We explore the use of compile-time and run-time evaluation of LLM-generated Kokkos code through extending the methods used by OpenAI with the HumanEval dataset. Our evaluation framework is based on the first 40 prompts from the Kokkos138 dataset. We start by discussing two different forms of LLM prompting, using entirely plain English or providing pseudocode for added context. These two methods are used to generate Kokkos code with the Llama-3.1-8B-Instruct and CodeQwen1.5-7B-Chat models. We found that both forms of prompting led to high failure rates and difficulties with reliably parsing LLM-generated code, while prompts with pseudocode for context generally led to improved results on more complicated tests.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Pull Force Evaluation of CCS Insulated End Caps

Insulated end caps installed on DC pins of Combined Charging System (CCS) inlets have been identified as potential debris sources within electric vehicle supply equipment (EVSE) connectors, increasing the risk of electric shock and fire hazards. Although standards such as IEC 62196-1:2022 specify mechanical pull-force requirements for these end caps, it remains unclear whether these requirements provide adequate robustness under real-world conditions. To address this concern, the National Charging Experience (ChargeX) Consortium's Hardware Task Force conducted evaluations of insulated end caps used in OEM CCS inlets. This study assesses the performance of insulated end caps used in OEM CCS inlets, specifically those installed on DC pins, using the procedures defined in IEC 62196-1:2022 (Section Sign) 26.7 and SAE J3400, supplemented by additional experimental conditioning. To ensure relevance to real-world conditions, NLR collaborated with charging network operators (CNOs) to guide the selection of charging inlet samples. Material analyses of the insulated end cap samples were conducted to confirm that the tested materials reflected the types and properties commonly observed in the field. The selected samples underwent temperature and humidity conditioning followed by pull-force testing to evaluate the end-cap integrity. Although SAE J3400 inlets were not directly tested due to limited diversity in available field data, the underlying end-cap principles between CCS and SAE J3400 are comparable, allowing the study's insights to be relevant to both technologies. Overall, this study provides a structured evaluation framework to inform potential refinements to mechanical pull-force requirements in charging standards.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2026

AI-Ready Data Pilot Project Report

The proliferation of artificial intelligence in scientific research has created an urgent need to define "AI-ready data" for researchers and, more importantly, provide resources to help them produce AI-ready data. At Pacific Northwest National Laboratory, we conducted a pilot study with three data scientists evaluating three CSV datasets from different scientific domains, followed by semi-structured interviews capturing assessment practices. Our findings reveal that AI-readiness evaluation is intuition-based, with practitioners asking "How fast can I go from raw data to my machine learning pipeline?" Data scientists consistently prioritized workflow efficiency, human interpretability, and quality stewardship signals. From these insights, we developed a practical evaluation framework comprising data requirements, metadata standards, and validation tests that provides actionable criteria for producing and curating AI-ready datasets, addressing the gap between theoretical understanding and practical implementation.

97 MATHEMATICS AND COMPUTING↗