Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Evaluation Framework”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21

Exploration of Domain Aware Machine Learning for Grid Analytics: Transfer-Learnt Energy Models to Assist Buildings Control with Sparse Field Data

Buildings are a primary consumer of energy in the United States and are also increasingly being perceived as providers of grid services such as load shifting, shedding and modulation. High fidelity models of building energy consumption are needed to set appropriate baselines for measurement and verification (M&V) of controllers designed for energy efficient operation of buildings and to enable buildings to provide grid services via. participation in demand response programs. State-of-the-art building energy modeling techniques either rely on Physics based models, or extensive instrumentation of the building envelope to gather “big” data to train machine learning based models such as deep neural networks. While Physics based models are often limited by their accuracy, it is not always feasible to gather a significant amount of field data required to train machine learning based models with sufficient accuracy. In this paper, we explore the use of transfer learning-based strategies to address unsatisfactory accuracy of models for estimating building energy consumption when available field data for training is sparse or of unacceptable quality. In particular, we transfer knowledge in the form of data and parameters, from Physics based simulation frameworks to the field to improve the model accuracy, thus resulting in a Physics-informed Machine Learning framework. We evaluated the efficacy of our approach on field data collected from six commercial buildings and our results indicate that the proposed transfer learning based models provide comparative (and in some cases better) accuracy than state-of-the-art machine learning and deep learning solutions, with just one month of field data.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

A Framework for Non-Pipeline Alternatives Analysis and Review of Existing Approaches

States are increasingly interested in gas distribution system planning and the role that demand-side resources can play in maintaining an affordable, reliable, equitable, and low-carbon energy system. In particular, public utility commissions and utilities are exploring the role of non-pipeline alternatives (NPA) – an investment or activity that defers, reduces, or avoids the need to construct or replace a pipeline – in gas distribution system planning. NPAs are an emerging cost and risk mitigation tool that can provide gas utilities with an opportunity to reduce emissions, gas system costs, and customer risk by avoiding unnecessary infrastructure spending. Rather than address system issues with more costly and long-lived traditional capital projects, utilities can leverage demand-side NPA resources such as energy efficiency and electrification as well as supply-side NPA resources to meet system needs. However, there is limited guidance on performing an NPA and even fewer lessons learned. To fill this gap, the U.S. Department of Energy funded two reports on NPAs. The first report is a literature review that focuses on four states' NPA policies. The second report offers a framework for evaluating NPAs that states can adapt to meet their policy needs.

03 NATURAL GAS↗

Arctic Critical Infrastructure: Assessing and Predicting the Risk to Critical Permafrost Infrastructure from Climate Change: A New Thermomechanical Approach

This study presents the development of a computational framework designed to predict the interaction between permafrost and infrastructure, addressing potential failure modes and mitigation strategies in the context of climate change. The framework, rooted in advanced modeling and simulation (mod/sim) techniques, integrates thermomechanical coupling to account for the complex interplay between heat flow, ice content, and mechanical behavior in permafrost. Existing models fail to fully capture these dynamics, particularly as they relate to the effects of ice saturation on structural integrity. Our innovative Arctic Coastal Erosion (ACE) framework fills this gap by coupling thermal and mechanical models to accurately simulate subsidence and deformation in permafrost environments. We applied the ACE framework to a representative runway, demonstrating its capability to predict settlement due to rising temperatures and subsequent permafrost thaw. This proof-of-concept showcases the potential of the framework to evaluate risks to Arctic infrastructure, which supports over four million people and 70% of existing permafrost-based structures. By simulating various infrastructure types and environmental conditions, our research offers insights into failure mechanisms and evaluates structural solutions to mitigate risk. The anticipated deliverables, including a prototype runway exemplar, position this project as a critical advancement in permafrost infrastructure modeling, with applications in national security and resilience planning.

54 ENVIRONMENTAL SCIENCES↗

Application of Cyber-Informed Engineering for Protecting BESS

This white paper synthesizes an array of crucial grid services provided by BESS technology, assesses its architecture and communications, and presents a case study for analysis against the principles introduced by Cyber-Informed Engineering (CIE). Furthermore, in walking through the analysis, this paper presents a framework to evaluate risks and solutions when considering BESS components. Asset owners and buyers could perform this analysis to assess their BESS product implementations, alternative inverter-based resources (IBR), and energy management systems (EMS). Battery systems fulfill various roles contingent on the unique market demands and the specific challenges presented by regional grid infrastructures. These roles also vary due to the differing utility models for ownership and operation, which are adapted to meet regional and local capabilities and requirements. Concerns have been raised regarding the potential for adversaries to exploit knowledge of battery operational patterns to orchestrate decisive attacks. However, the security of operational data for these systems may not be the primary vulnerability, as much of this information is already well-understood within the community. Applying a modest degree of subject matter expertise can often yield valuable predictions regarding how a battery will respond under certain conditions, such as grid emergencies, high or low-temperature days, Public Safety Power Shutoff (PSPS) events, and outages. The operational characteristics of batteries are well-documented, and their capabilities, including the risks associated with misoperation and the resulting consequences, are published and understood within the industry. CIE practices represent the next step in gaining functional assurance and providing an acceptable level of risk, regardless of whether a battery vendor can support a trusted and validated supply chain. While this issue has exacerbated supply chain challenges, it is not an isolated condition. This foreign supply route is the primary source of BESS for the U.S. market. Significant efforts are underway through the Bipartisan Infrastructure Law (BIL) to change that. Still, strategic short-term operational mitigations are needed to ensure the security of our operational technology (OT) systems, which are enhanced by instilling trust and are separate from vendors implementing CIE principles.

25 ENERGY STORAGE↗

Optimal Operation and Impact Assessment of Distributed Wind for Improving Efficiency and Resilience of Rural Electricity Systems

This project aims to empower rural utilities by developing advanced optimization models and algorithms for effectively integrating distributed wind energy alongside battery storage and other distributed energy resources (DERs). The primary objectives are to reduce peak demand, ensure reliable emergency power supply, and regulate voltage and frequency. To address operational challenges, the project introduces innovative mitigation strategies and ultrafast assessment frameworks to evaluate the impacts of distributed wind and DERs on rural grids, offering actionable solutions to potential issues. Economic viability is assessed through cost-benefit analysis using real rural utility data, ensuring the practical application of the project outcomes.

17 WIND ENERGY↗

Water Sorption Study of Various Polymers and Polymeric Composites

In the formulations of many composite materials, polymeric materials are often used as binder materials to enhance processability, functionality, and safety during manufacturing. For instance, poly(ester urethane), such as Estane® 5703, is used in the formulation of PBX 9501, and vinyl copolymer elastomer (VCE) is used in the formulation of VCE/filler compos ites. The stability of these binders directly impacts the overall stability and performance of their composites. Estane and VCE are prone to hydrolytic degradation, with water sorption behavior playing a significant role in their long-term stability and aging behavior. Hence, it is critical to study their water sorption behavior under various humidity and tempera ture conditions. To improve the accuracy and reduce labor-intensive work involved in water sorption experiments, an automated Dynamic Vapor Sorption (DVS) system was employed, which can generate large datasets. To efficiently process and analyze these datasets, Python scripts were developed. These scripts not only streamlined data processing, but also accu rately derived the Arrhenius parameters related to water diffusion and sorption properties for the tested materials. This approach offers a robust framework for evaluating water trans port and sorption processes in polymeric materials and adds valuable capabilities to future water sorption testing efforts.

36 MATERIALS SCIENCE↗

Nine Canyon Long-Duration Energy Storage: A Feasibility Study

The Nine Canyon Long Duration Energy Storage (LDES) Feasibility Study explores the technical and economic viability of deploying advanced energy storage technologies at Energy Northwest's (EN) Nine Canyon (9C) Wind Project site in Benton County, Washington. Supported by the Washington State Department of Commerce and the U.S. Department of Energy’s Office of Electricity under its LDES Voucher Program, the study represents a collaborative effort between EN, Pacific Northwest National Laboratory (PNNL), and ARES North America. At the core of this effort is the development of a generalized techno-economic modeling framework and evaluation tool designed to assess the value proposition of LDES projects across a variety of contexts. The modeling tool is technology-agnostic and accommodates user-defined parameters such as rated power, energy duration, round-trip efficiency, capital and operational costs, and dispatch constraints. It also integrates economic inputs, including market prices, energy revenue structures, and financing parameters to evaluate performance through key metrics. The tool provides utilities with a transparent, adaptable platform to support decision-making, investment prioritization, and portfolio planning for various storage technologies. To guide scenario design and interpretation, the study first surveyed the LDES technology landscape, including lithium-ion batteries, flow batteries, non-hydro gravity storage, and thermo-mechanical systems, comparing cost trajectories, technical performance, safety and hazards, materials sourcing and recyclability, and spatial/siting considerations. This literature-grounded review highlights technology trade-offs and reinforces the need to align technology choice with site characteristics, use cases, and project objectives. A companion chapter examines ownership structures (EN ownership, third-party ownership, shared models) and offtake options (energy marketing, capacity/energy PPAs, time-of-use PPAs, block-delivery PPAs, and tolling), where PPAs (power purchase agreements) represent contractual arrangements for buying and selling electricity. The chapter also highlights implications for risk allocation, capital access, operational control, and revenue certainty. The study also evaluates supervisory control and data acquisition (SCADA) and transmission interconnection pathways, options include upgrading the existing SCADA or deploying a dedicated LDES controller, with attention to protection schemes, data telemetry, cybersecurity, and regulatory coordination with BPA. In addition, an ARES-specific geotechnical and hydrology assessment presented in the appendix screens multiple corridors for slope stability, bearing capacity, cut-and-fill magnitude, and stormwater behavior.

25 ENERGY STORAGE↗

A CHIL Validation of Machine Learning-Assisted Methods for Real-Time Controls of Solar PV for Grid Services

Recent research has highlighted the potential for solar to act as a zero-marginal-cost and zero-emission flexibility resource on the bulk power system when operated with advanced control systems. To increase the performance of these systems, leading technologies, including machine learning (ML) and hierarchical inverter set point allocation, have been proposed; however, these technologies lack comprehensive validation under real-world application scenarios. This paper addresses this gap by designing and developing a controller-hardware-in-the-loop framework to evaluate the performance of different flexible solar technologies in responding to automatic generation control signals in a closed-loop fashion. Simulation results indicate the superior performance of an ML-based approach compared to the conventional reference-control grouping-based approach, showcasing its potential to support grid stability and operational efficiency.

14 SOLAR ENERGY↗

Computational materials reliability assessment of hydrogen fueled gas turbine power generation engines

The use of blended fuel sources in land based gas turbine engines drives variations in the resulting operational profile (temperatures and pressures) which can impact engine reliability. Furthermore, variability in the manufacture of components affects the resulting microstructure which directly impacts material performance and reliability. Currently, data-driven models are typically used for maintaining and inspecting fleets of engines. Without explicitly capturing material and operational sources of variability conservatism must be used in developing component-level reliability models. Therefore, there exists an opportunity to use information from materials-scale physics models to better inform reliability modeling and reduce conservatism; the impact is more cost-efficient operation and maintenance of current and future fleets. Specifically, this work establishes a computational framework for evaluating the probabilistic high temperature creep performance of hot-section Ni-based superalloys where uncertainty comes from both microstructural and operational variability. A novel high-fidelity physics model which phenomenologically captures grain-boundary sensitive phenomena has been established. A probabilistic calibration procedure was used to calibrate the model and capture uncertainty in the parameterized model coefficients. A design of experiments methodology was established for identifying informative microstructural digital representations for suitable for forward model evaluation. Results show that training a machine-learning surrogate using this design criteria outperforms random selection of microstructural representations. Finally, two surrogate models were developed: (1) a deterministic surrogate model which predicts the local field response given microstructure, constitutive model parameters, and operating conditions (stress, temperature) and (2) a probabilistic model, where uncertainty comes from constitutive law uncertainty, built using denoising diffusion probabilistic models which samples responses given (1) microstructure and (2) operating conditions. These surrogate models enable partner Siemens Energy to rapidly perform UQ analysis specific to creep deformation across a range of microstructures and operating conditions. The impact is that these ML and physics codes can be used to establish more advanced reliability models for the inspection, servicing, and maintenance of land based gas turbine engines.

36 MATERIALS SCIENCE↗

Agent-based Modeling for the Multi-objective Optimization of Energy Production Pathways: Integrated Techno-Economics and Life Cycle Assessment

This project developed an integrated modeling framework to evaluate how purpose-grown bioenergy feedstocks could support large-scale sustainable aviation fuel production in the United States. Led by Colorado State University with project partners, the work combined geographically resolved analysis, techno-economic assessment, life cycle assessment, multi-objective optimization, and agent-based modeling to examine feedstock performance, deployment potential, land-use implications, and adoption dynamics. The overall goal was to identify practical and sustainable pathways for producing aviation fuel from feedstocks such as energy crops, woody crops, oilseeds, and algae while helping inform policy, investment, and research needs tied to national SAF goals. In essence, the project provides a systems-level assessment of the opportunities and constraints associated with scaling bioenergy-based aviation fuels in the U.S.

09 BIOMASS FUELS↗

Uncertainty and Sensitivity Analysis Methods and Applications in the GDSA Framework (FY2024)

The Spent Fuel and Waste Science and Technology Campaign (SFWST) of the U.S. Department of Energy (DOE) Office of Nuclear Energy (NE) is conducting research and development on geologic disposal of spent nuclear fuel (SNF) and high-level nuclear waste (HLW). Two priorities for SFWST are design concept development and disposal system modeling. These priorities are directly addressed in the Geologic Disposal Safety Assessment (GDSA) control account, which is charged with developing a geologic repository system modeling and analysis capability, and the associated software, GDSA Framework, for evaluating disposal system performance for nuclear waste in geologic media.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Superstructure Optimization for Brine Valorization from Brackish Water Desalination

This poster presents preliminary results from a superstructure optimization framework developed to identify cost-optimal brine valorization configurations for brackish water desalination plants across diverse U.S. regional feed chemistries. The study uses brackish groundwater compositions from Arizona, California, Florida, New Mexico, and Texas. Using Pyomo Generalized Disjunctive Programming (GDP) within the WaterTAP modeling environment, the optimization framework simultaneously evaluates thousands of candidate treatment configurations, spanning nanofiltration, reverse osmosis, and chemical precipitation, to minimize the levelized cost of water (LCOW) while meeting water recovery targets and product recovery constraints. Results across eight representative feed clusters demonstrate water recovery rates of 57–87% and net LCOW values ranging from -$0.032/m³ (net revenue-positive) to $0.80/m. Notably, no single process configuration was optimal across all feed types, underscoring the necessity of feed-specific optimization. Products targeted include calcium carbonate (CaCO₃) at $0.01/kg and sodium chloride (NaCl) at $0.10/kg, both at 95% purity, with product revenues offsetting treatment costs in several scenarios. The work advances NAWI's process systems engineering capabilities for multi-configuration screening.

58 GEOSCIENCES↗

Flexible AI Models for Grid Resilience

The rapid growth in size and complexity of artificial intelligence (AI) and machine learning (ML) models has led to increased energy demands, posing a threat to the reliability of the existing power grid. This project addresses the challenge of highly intermittent and energy-intensive inference workloads by (1) developing fidelity-adaptive neural networks capable of dynamic response to grid conditions and (2) integrating these networks with power flow simulations to assess their impact on power grid reliability. We will explore both top-down and bottom-up approaches to create hierarchies of submodels that provide a controlled trade-off between power draw and prediction accuracy. The top-down method utilizes NN pruning to reduce a flagship model into progressively smaller, energy-efficient variants. The bottom-up approach employs geometrically principled weight setting strategies to construct depth-efficient models from the ground up. A real-time hardware-in-the-loop (HIL) platform will be developed to simulate a scaled AC power grid, integrating live AI workload power draw and enabling dynamic model switching in response to grid feedback. This work will provide a novel framework for evaluating the impact of flexible AI/ML workloads on grid performance and establish new methodologies for energy-aware computing in data centers. The outcomes will demonstrate that adaptive AI/ML can play a critical role in improving grid stability while advancing NREL's leadership in energy-efficient computing research.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Resilience Metrics for Solar Photovoltaics

This workshop presentation proposes the development of solar photovoltaic (PV) system resilience metrics and a methodology and framework for evaluation of PV resilience metrics. PV resilience metrics are needed to establish a consistent basis for reporting, evaluation, and data collection by industry, evaluate performance of PV systems that have been subject to natural hazards, correlating resilience to system attributes, and predicting resilience for any PV system. PV resilience metrics can guide improved system design, standards, and insurance coverage. Establishing consistent metrics can foster data collection on impacts of natural hazards on PV systems.

14 SOLAR ENERGY↗

The ABCs of On-Demand Transit (ODT)

On-demand mobility - also referred to as on-demand transit (ODT) - is a form of public mobility that is flexible with respect to where and when service is provided, and ODT deployments have increased significantly in recent years. Transit agencies are becoming increasingly interested in ODT, and due to differing definitions and various service design and business model options, it can be difficult to learn about the emerging industry. This work provides an overview of the definitions of ODT, recent trends internationally and in the U.S., ODT's benefits and challenges (particularly compared to fixed-route transit), three primary service design options, system costs and funding considerations, and a metrics framework for evaluating ODT systems to ensure continued successful performance. Identified benefits include increased service areas, short ride and wait times, increased user flexibility, potential to reduce energy consumption and emissions through shared trips and smaller, right-sized vehicles, increased safety and comfort through door-to-door service, and rich data streams including granular spatio-temporal data that can be analyzed to continuously improve the service. Challenges include scaling ODT service up as small increases in ridership require additional supply to keep service quality high, serving peak times including keeping low wait times, the lack of fixed schedule being challenging for commuters, integrating ODT services with nearby transit systems, and equity for riders without smartphones who cannot track the vehicle in a mobile app. Finally, an overview of seven ODT case studies (in Texas, Missouri, New York, and Ontario, Canada) performed by NREL and related analysis of travel time, energy and emissions, costs, and equity are presented. Initial key findings include: ODT can be cost- and energy-effective compared to fixed-route transit, ODT serves more people than other transit options, and ODT system deployments can be followed by rapid growth.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Atmospheric Modeling and Denoising for Millimeter-Wave Line Intensity Mapping

Line-intensity mapping (LIM) offers a promising approach to mapping large-scale cosmic structure, and the greatest obstacle for ground-based observations at millimeter wavelengths is foreground contamination from atmospheric emission. In this work, we present a simulation and denoising framework designed to isolate and subtract atmospheric fluctuations from LIM data, modeled after the instrument parameters of the South Pole Telescope Summertime Line Intensity Mapper (SPT-SLIM). We generate mock observations spanning 125-175 GHz containing cosmic signals, precipitable water vapor screens, ice crystal fluctuations, and photon noise. We then implement a spatial-spectral atmospheric removal pipeline combining per-pixel linear template regression with a two-dimensional Fourier-domain filter. The framework is evaluated under simulated conditions in the South Pole and the Atacama Desert across three key metrics: cosmic signal preservation, foreground subtraction efficiency, and instrument noise injection. Our pipeline achieves atmospheric suppression at large spatial scales, and these results establish a physically grounded foundation for atmosphere removal in ground-based LIM data collection.

Saye, Laney [UC, Berkeley (main)] (ORCID:000900078↗

Utilizing Reinforcement Learning to Continuously Improve a Primitive-Based Motion Planner

We report in this paper describes how the performance of motion primitive-based planning algorithms can be improved using reinforcement learning. Specifically, we describe and evaluate a framework that autonomously improves the performance of a primitive-based motion planner. The improvement process consists of three phases: exploration, extraction, and reward updates. This process can be iterated continuously to provide successive improvement. The exploration step generates new trajectories, and the extraction step identifies new primitives from these trajectories. These primitives are then used to update rewards for continued exploration. This framework required novel shaping rewards, development of a primitive extraction algorithm, and modification of the Hybrid A* algorithm. The framework is tested on a navigation task using a nonlinear F-16 model. The framework autonomously added 91 motion primitives to the primitive library and reduced average path cost by 21.6 seconds, or 35.75% of the original cost. The learned primitives are applied to an obstacle field navigation task, which was not used in training, and reduced path cost by 16.3 seconds, or 24.1%. Additionally, two heuristics for the modified Hybrid A* algorithm are designed to improve effective branching factor.

42 ENGINEERING↗