Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluation framework”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Applying a Multisector Scenario Framework to Evaluate Past and Future Public Surface Water Supply Infrastructure Strategies in Texas

Datasets supporting the index model and scenario analysis used in evaluating surface water supply strategies across different water system types in Texas. These data underpin the scenario development and application of five key indicators: Water Availability Index (WAI), Water Quality Index (WQI), Energy Requirement Index (ERI), Water Treatment Cost (WTC), and Water Infrastructure Cost (WIC). The datasets are organized by system type—stream reaches (flowlines), waterbodies, and reservoirs—and include both raw and standardized index values. The integrated datasets also provide scenario classifications (original and adjusted) based on infrastructure and planning priorities, enabling comparison across Shared Socioeconomic Pathways (SSPs). Additional strategy-level data are included to support evaluation of state-level new reservoir projects in relation to cost and availability tradeoffs. Please refer to the README file provided in Files for more details. Descriptions of the datasets are provided below. Dataset(s) Descriptions Folder: Index_model_database.zip Subfolder: Stream_reach.zip Fl_wf.csv, Fl_wq.csv, Fl_er.csv, Fl_wf_wtcUV.csv, Fl_wf_wtcnoUV.csv, Fl_allfac_wic1.csv, Fl_allfac_wic2.csvDatasets for computing WAI, WQI, ERI, WTC, and WIC for surface water systems classified as stream reaches (flowlines). Subfolder: Waterbody.zip Wb_wf.csv, Wb_wq.csv, Wb_er.csv, Wb_wf_wtcUV.csv, Wb_wf_wtcnoUV.csv, Wb_allfac_wic1.csv, Wb_allfac_wic2.csvEquivalent index model datasets for waterbodies, reflecting hydrologic and infrastructure attributes specific to impounded natural systems. Subfolder: Reservoir.zip Rs_wf.csv, Rs_wq.csv, Rs_er.csv, Rs_wf_wtcUV.csv, Rs_wf_wtcnoUV.csv, Rs_allfac_wic1.csv, Rs_allfac_wic2.csvIndex model datasets specific to regulated reservoir systems, incorporating both resource indicators and cost parameters. Folder: Integrated data.zip combined_merged_data.csv, combined_merged_data_scenario.csvDatasets integrating index model indicators (both raw and scaled) with scenario classifications, including adjustments reflecting SSP-aligned transitions and planning shifts. Folder: Additional data.zip wai_supplystrat_wic_merged.csvCurated dataset capturing proposed major reservoir-based municipal water supply strategies in Texas. Integrates site-level planning data with estimated capital infrastructure costs and water availability scores for comparative assessment.

geospatial↗
DOE OSTI2026

Interregional Transmission Operational Coordination (IRTOC)

This report presents a modeling and evaluation framework developed through the Inter-Regional Transmission Operational Coordination (IRTOC) project to study market-to-market (M2M) congestion management across day-ahead and real-time markets. The framework extends the Sienna platform through Sienna Decomposition, a multi-stage evaluation architecture that enables flexible representation of multiple regions and systematic assessment of alternative market coordination designs. Additional modeling capabilities include reserve deliverability constraints, High-Voltage Direct Current (HVDC) optimization for Alternating Current (AC) congestion management, and several real-time distributed coordination algorithms. Case studies using the RTS-GMLC test system and a large-scale Eastern Interconnection model demonstrate that the framework can evaluate alternative coordination structures and quantify their economic and operational impacts. The proposed framework provides a scalable platform for analyzing inter-regional coordination strategies in large-scale electricity markets.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2025

Challenges in predicting protein-protein interactions of understudied viruses: Arenavirus-human interactions

Understanding protein-protein interactions (PPIs) between viruses and host organisms is crucial for uncovering infection mechanisms and identifying potential therapeutic targets. The ability to generalize PPI predictive models across understudied viruses presents a significant challenge. In this work, we use arenavirus-human PPIs to illustrate the difficulties associated with model generalization, which are compounded by a lack of both positive and negative data. We employ a Transfer Learning approach to investigate arenavirus-human PPIs by utilizing models trained on better-studied virus-human and human-human PPIs. Additionally, we curate and assess four types of negative sampling datasets to evaluate their impact on model performance. Despite the overall high accuracies (93–99 %) and AUPRC scores (0.8–0.9) appearing promising, further analysis indicates that these performance metrics can be misleading due to data leakage, data bias, and overfitting, especially concerning under-represented viral proteins. We reveal these gaps and assess the impact of data imbalance using standard k-fold cross-validation and Independent Blind Testing with a Balanced Dataset, resulting in a drop in accuracy below 50 %. We propose a viral protein-specific evaluation framework that categorizes viral proteins into majority and minority classes based on their representation in the dataset, enabling comparison of model performance across these groups using balanced accuracies. This framework offers a more robust evaluation of model generalizability, addressing biases inherent in standard evaluation techniques and paving the way for more reliable PPI prediction models for understudied viruses.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

FY2021 Progress Report on BISON Metallic Fuel Model Development and V&V Using EBR-II Legacy Data

In this report, the activities and achievements made by Argonne National Laboratory for the Nuclear Energy Advanced Modeling and Simulation (NEAMS) BISON code metallic fuel validation and verification project in FY2021 are summarized. The cladding degradation model based on the FCCI/CCCI wastage calculations has been developed and implemented into BISON. A comprehensive evaluation of the cladding degradation model was performed based on FIPD data of the IFR experiment X447. BISON objects were also developed to enable direct use of time-varying cladding outer surface temperature profile as temperature boundary conditions, which proved to provide more accurate temperature predictions for the metallic fuel pins irradiated in EBR-II. Additionally, a new BISON object was implemented to enable direct comparison between BISON predicted data and FIPD-based post-irradiation examination (PIE) results, which would significantly facilitate BISON metallic fuel verification and validation (V&V) activities. These new BISON-FIPD integration features were used in the establishment of a low-burnup fuel swelling evaluation framework as demonstration. The framework was successfully used to evaluate current fuel swelling models based on the IFR experiment X423.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2026

Evaluation Toolkit for Technical Assistance Programs

Across the United States, every home, business, industry, and government depends on abundant, reliable, and affordable electricity. Energy stakeholders have many options to improve how energy is generated, distributed, and used, but choosing the right path can be complex and challenging. To support more informed decision-making about local electricity systems, the U.S. Department of Energy (DOE) and its national laboratories provide customized technical assistance (TA) through several different programs. These TA programs are designed to support a wide range of stakeholders with a variety of needs. Technical assistance may include brief consultations with subject-matter experts, in-depth technical modeling and analysis, stakeholder engagement, and peer-to-peer exchange. Delivering effective TA is an iterative process that requires evaluation and adaptation. A comprehensive and robust evaluation framework provides the structure needed to ensure that TA programs and practitioners remain effective, responsive to industry trends, and aligned with local needs. This toolkit provides a framework—including a clear, structured process—that TA program staff can adapt to their program's goals to evaluate success.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2020

Advancing Diagnostic Model Evaluation to Better Understand Water Shortage Mechanisms in Institutionally Complex River Basins

Abstract Water resources systems models enable valuable inferences on consequential system stressors by representing both the geophysical processes determining the movement of water and the human elements distributing it to its various competing uses. This study contributes a diagnostic evaluation framework that pairs exploratory modeling with global sensitivity analysis to enhance our ability to make inferences on water scarcity vulnerabilities in institutionally complex river basins. Diagnostic evaluation of models representing institutionally complex river basins with many stakeholders poses significant challenges. First, it needs to exploit a large and diverse suite of simulations to capture important human‐natural system interactions as well as institutionally aware behavioral mechanisms. Second, it needs to have performance metrics that are consequential and draw on decision‐relevant model outputs that adequately capture the multisector concerns that emerge from diverse basin stakeholders. We demonstrate the proposed model diagnostic framework by evaluating how potential interactions between changing hydrologic conditions and human demands influence the frequencies and durations of water shortages of varying magnitudes experienced by hundreds of users in a subbasin of the Colorado River. We show that the dominant factors shaping these effects vary both across users and, for an individual user, across percentiles of shortage magnitude. These differences hold even for users sharing diversion locations, demand levels or water right seniority. Our findings underline the importance of detailed institutional representation for such basins, as institutions strongly shape how dominant factors of stakeholder vulnerabilities propagate through the complex network of users.

Hadjimichael, Antonia↗
DOE OSTI2024

A Model-Based Systems Engineering Approach for Effective Decision Support of Modern Energy Systems Depicted with Clean Hydrogen Production

A holistic approach to decision-making in modern energy systems is vital due to their increase in complexity and interconnectedness. However, decision makers often rely on narrowly-focused strategies, such as economic assessments, for energy system strategy selection. The approach in this paper helps considers various factors such as economic viability, technological feasibility, environmental impact, and social acceptance. By integrating these diverse elements, decision makers can identify more economically feasible, sustainable, and resilient energy strategies. While existing focused approaches are valuable since they provide clear metrics of a potential solution (e.g., an economic measure of profitability), they do not offer the much needed system-as-a-whole understanding. This lack of understanding often leads to selecting suboptimal or unfeasible solutions, which is often discovered much later in the process when a change may not be possible. This paper presents a novel evaluation framework to support holistic decision-making in energy systems. The framework is based on a systems thinking approach, applied through systems engineering principles and model-based systems engineering tools, coupled with a multicriteria decision analysis approach. The systems engineering approach guides the development of feasible solutions for novel energy systems, and the multicriteria decision analysis is used for a systematic evaluation of available strategies and objective selection of the best solution. The proposed framework enables holistic, multidisciplinary, and objective evaluations of solutions and strategies for energy systems, clearly demonstrates the pros and cons of available options, and supports knowledge collection and retention to be used for a different scenario or context. The framework is demonstrated in case study evaluation solutions for a novel energy system of clean hydrogen generation.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2023

Performance assessment for climate intervention (PACI): preliminary application to a stratospheric aerosol injection scenario

As the prospect of exceeding global temperature targets set forth in the Paris Agreement becomes more likely, methods of climate intervention are increasingly being explored. With this increased interest there is a need for an assessment process to understand the range of impacts across different scenarios against a set of performance goals in order to support policy decisions. The methodology and tools developed for Performance Assessment (PA) for nuclear waste repositories shares many similarities with the needs and requirements for a framework for climate intervention. Using PA, we outline and test an evaluation framework for climate intervention, called Performance Assessment for Climate Intervention (PACI) with a focus on Stratospheric Aerosol Injection (SAI). We define a set of key technical components for the example PACI framework which include identifying performance goals, the extent of the system, and identifying which features, events, and processes are relevant and impactful to calculating model output for the system given the performance goals. Having identified a set of performance goals, the performance of the system, including uncertainty, can then be evaluated against these goals. Using the Geoengineering Large Ensemble (GLENS) scenario, we develop a set of performance goals for monthly temperature, precipitation, drought index, soil water, solar flux, and surface runoff. The assessment assumes that targets may be framed in the context of risk-risk via a risk ratio, or the ratio of the risk of exceeding the performance goal for the SAI scenario against the risk of exceeding the performance goal for the emissions scenario. From regional responses, across multiple climate variables, it is then possible to assess which pathway carries lower risk relative to the goals. The assessment is not comprehensive but rather a demonstration of the evaluation of an SAI scenario. Future work is needed to develop a more complete assessment that would provide additional simulations to cover parametric and aleatory uncertainty and enable a deeper understanding of impacts, informed scenario selection, and allow further refinements to the approach.

climate intervention↗
DOE OSTI2026

A Co-Simulation Framework for Steady-State Analyses of Multiple Droop-based MTdc Grids in Continental-Scale Systems

The growing scale and complexity of planning continental hybrid ac and multi-terminal dc (MTdc) systems require scalable steady-state modeling and analysis approaches not currently available in commercial tools. This paper presents a comprehensive multi-fidelity model-conversion framework that enables the efficient transition of MTdc grid models from production cost modeling (PCM) and approximated ac power flow to detailed ac–MTdc power flow for large-scale planning studies. The core of this framework is a scalable co-simulation approach that, for the first time, enables power flow analysis in continental-scale ac–MTdc systems. It seamlessly couples commercial ac solvers with a detailed MTdc grid model that incorporates droop-based control and current-limiting strategies of multiple meshed MTdc grids. Leveraging this capability, an evaluation framework to systematically assess and compare different MTdc power redispatch strategies under ac and dc contingencies is introduced. The proposed framework and algorithm are evaluated using a combined Western and Eastern Interconnection system with 11 MTdc grids of various sizes, showing a coherent transition from PCM to detailed ac–MTdc power flow and improved system performance in voltage regulation and line overload mitigation following typical contingencies.

Nguyen, Quan H.↗
DOE OSTI2022

Evaluation of CMIP6 GCMs Over the CONUS for Downscaling Studies

Despite the necessity of Global Climate Models (GCMs) sub-selection in downscaling studies, an objective approach for their selection is currently lacking. Building on the previously established concepts in GCMs evaluation frameworks, we develop a weighted averaging technique to remove the redundancy in the evaluation criteria and rank 37 GCMs from the sixth phase of the Coupled Models Intercomparison Project over the contiguous United States. GCMs are rated based on their average performance across 66 evaluation measures in the historical period (1981–2014) after each metric is weighted between zero and one, depending on its uniqueness. The robustness of the outcome is tested by repeating the process with the empirical orthogonal function analysis in which each GCM is ranked based on its sum of distances from the reference in the principal component space. The two methodologies work in contrasting ways to remove the metrics redundancy but eventually develop similar GCMs rankings. A disparity in GCMs' behavior related to their sensitivity to the size of the evaluation suite is observed, highlighting the need for comprehensive multi-variable GCMs evaluation at varying timescales for determining their skillfulness over a region. The sub-selection goal is to use a representative set of skillful models over the region of interest without substantial overlap in their future climate responses and modeling errors in representing historical climate. Additional analyses of GCMs' independence and spread in their future projections provide the necessary information to objectively select GCMs while keeping all aspects of necessity in view.

54 ENVIRONMENTAL SCIENCES↗