Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “SCORE”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19
All sourcesNASA NTRSDOE OSTI
NASA NTRS2023

Bhutan Agriculture III: Monitoring Cropland Changes in Bhutan using Remote Sensing to Bolster Food Security and Support Crop Monitoring

The Bhutan Agriculture III team aimed to improve agricultural efficiency in Bhutan. Bhutan is a nation heavily reliant on agriculture, but it faces challenges such as geophysical limitations and lack of scientific agricultural practice. The team partnered with a primary end user, Bhutan’s Department of Agriculture (DoA), and with collaborators; the Bhutan Foundation, National Plant Protection Centre (NPPC), Agricultural Research Department Centre (ARDC), National Statistics Bureau (NSB), and the Ugyen Wangchuck Institute for Conservation and Environment Research (UWICER). Advised by NASA SERVIR, the team developed crop masks and monitored rice distribution from 2015 to 2022 utilizing Earth observations such as Landsat 8 Operational Land Imager (OLI), Landsat 9 OLI-2, Sentinel-1 C-Band Synthetic Aperture Radar (C-SAR), Sentinel-2 MultiSpectral Instrument (MSI) and Shuttle Radar Topography Mission (SRTM). The team gathered 5,000 points from the five dzongkhags that yield the most rice in Bhutan (Paro, Punakha, Samtse, Sarpang and Wangue Phodrang) using Collect Earth Online (CEO). With the data collected, the team split the data into training and validation data on Google Earth Engine (GEE) for a random forest (RF) classifier for rice and non-rice classification. After running the data on the Random Forest (RF) model, the team got an accuracy score of 81.48%, a kappa score of 55.75% and an F1 score of 86.11%. This data supports better agricultural decision-making for the governing body of Bhutan, helps enhance farming efficiency and foster sustainable practices, assists in overcoming data inaccuracy and bolsters food security in the country.

Sonam Seldon Tshering↗
NASA NTRSDate not supplied

Human Systems Risk Network - A Ranking Analysis of Risks

INTRODUCTION The Human Systems Risk Board (HSRB) is responsible for understanding, managing, and mitigating the risks associated with spaceflight. For a particular mission, the HSRB assigns each human system risk a rating on a 5x5 grid assessing its likelihood and consequence, which is ultimately used to compare and rank the risks. The HSRB approaches risk management by primarily establishing the context of each human system risk individually with the understanding that mitigating one risk might affect the likelihood, consequence, and mitigation approaches of another. To support this effort the HSRB, subject matter experts, and risk custodian teams created directed acyclic graphs (DAG), often called a causal graph, for the twenty-nine risks. In this presentation, we propose a new ranking algorithm for the risks which includes the downstream influence of each risk according to the information in the DAGs and provide an application of graph theoretic tools. METHODS In 2014, Mindock and Klaus proposed a taxonomy for human system risk influences which we have adopted to categorize the nodes in each DAG. Analyzing the nodes that correspond to the risks in this taxonomy allows us to analyze and understand how each risk influences the others. We construct an auxiliary network, which we call the Primary Risk Network (PRN), where the nodes are the twenty-nine space flight risks and, a directed edge connects Risk A to Risk B if Risk A has some influence on the likelihood or consequence of Risk B as described in the DAGS. We perform a variety of graph theoretic ranking methods on the nodes (or risks) in the PRN, including Katz centrality. RESULTS We rank the nodes in the PRN using the Katz centrality score. The ten risks with the highest score are pictured in Figure 1, colored (light to dark) according to their score. We analyze other centrality measures like betweenness centrality, eigenvector centrality, and the Estrada index, and provide the meaning of the corresponding rankings in terms of the risks. Future work includes analyzing the other categories in the taxonomy defined by Mindock and Klaus [1]. For example, we are interested in analyzing the nodes that are labeled as countermeasures or capabilities and perform similar analysis to measure their effect on certain medical conditions.

dag↗
NASA NTRSDate not supplied

2024 Nasa Lunabotics University Competition: Site Preparation With Bulk Regolith

Introduction: Lunabotics provides accredited institutions of higher learning students (vocational-technical, college, university) an opportunity to apply the NASA systems engineering process to design and build a prototype robot. This robot would be capable of performing the proposed operations on the Lunar surface in support of future Artemis mission goals. Lunabotics features a systems engineering design challenge to engage students in the next phase of hu-man space exploration supporting the Artemis missions. This two-semester event encourages students to design and build an autonomous or telerobotic robot designed to traverse the simulated Lunar surface and complete the assigned construction tasks. The number of teams accepted into this challenge is not predetermined but is based on the scores and overall quality of the Project Management Plans received and other factors. The culmination of the Lunabotics virtual challenge will be the design, build and operation of a functional prototype Lunar robot. Teams are required to submit the following: (1) Project Management Plan, (2) Systems Engineering Paper, a (3) STEM Engage-ment Report, and a (4) Proof of Life Video. This is an optional item, but to qualify for the grand prize a team must also submit a: (5) Presentation and Demonstration. Background: The NASA Lunabotics University Competition was first held in 2010 as a follow on to the NASA Regolith Excavation Challenge [1]. The high level of interest and participation from over 50 universities each year has led to a sustained annual competi-ion cadence: it has been held every May for the past 14 years [2,3]. Over 6,000 students have participated and been inspired to pursue Science, Technology, En-gineering and Mathematics careers (STEM). 2024 Competition: The necessary lunar surface tasks are evolving to meet the NASA Artemis Mission requirements. In the past Lunabotics challenges we gathered data to support Lunar mining for consuma-bles in the Lunar regolith. Now, in 2024, the task is to gather data on Lunar site preparation and construction by designing and building a robot that will traverse the chaotic Lunar terrain and construct a regolith-based berm. The goal is to build a berm structure which would be useful to the Artemis Mission for blast and ejecta protection during lunar landings and launches, shading cryogenic propellant tank farms, providing radiation protection around a nuclear power plant and other mission critical uses. Lunabotics will consist of three separate events this year. The first event is NASA’s Lunabotics Project Development Challenge, where teams submit various deliverables to be scored by judges. The second event will be the University of Central Florida (UCF) Lunabotics Qualification challenge, in the Exolith laboratory, where teams will put their de-signs to the test. The top ten scoring teams from the Qualification challenge are then invited to the third and final event, NASA’s Lunabotics On-Site Challenge at Kennedy Space Center in Florida. This presentation will summarize the results and lessons learned from the NASA Lunabotics University Competition held in May 2024.

Lunabotics↗
NASA NTRSDate not supplied

Objective Structured Clinical Evaluation (OSCE) of an Artificial Intelligence (AI) Clinical Decision Support System (CDSS) Tool

BACKGROUND Objective Structured Clinical Evaluations (OSCEs) have long been established as a robust methodology for summative assessment of clinical skills and decision-making during medical education. The recent integration of Artificial Intelligence (AI) into clinical decision-making processes has prompted the need for novel evaluation frameworks to assess the efficacy and reliability of AI clinical decision support system (CDSS) tools. This abstract outlines the process of quantitatively evaluating a novel CDSS (“Doc in a Box” Google 2024) trained on curated medical spaceflight data in the psychomotor domain as it interfaces with a human volunteer acting as the crew medical officer (CMO). PURPOSE The AI CDSS under review was developed as part of the Lunar Command and Control Interoperability (LuCCI) project, which is intended to address a gap in how Lunar Surface Systems (LSS) would interoperate across multiple programs, commercial partners, and international partners. The project objective is to define, prototype, integrate, and evaluate an interoperable lunar command, control, data, and software reference architecture to enable autonomy and informatics capability through common standards across LSS. A multi-modal AI-based CDSS compatible with Federated LSS will assist clinicians in diagnosing and managing complex medical conditions by providing evidence-based recommendations through predictive analytics. Given the critical role of decision-support as NASA continues to evolve its Earth-independent medical operations (EIMO), it is imperative to ensure that such AI tools perform reliably and align with clinical standards during progressive lunar and Martian exploration class missions. METHODS The OSCE framework, traditionally used for evaluating human clinicians, was adapted to assess the AI tool's decision-making capabilities in simulated clinical scenarios. In this adapted OSCE, the AI CDSS was tested across a series of structured clinical scenarios designed to mimic real-life spaceflight patient cases. These scenarios included a range of conditions and complexities, allowing for comprehensive assessment of the tool's performance. Key evaluation metrics included accuracy of diagnosis, timeliness of decision-making, and appropriate recommendations for therapies. The OSCE was scored by human physician evaluators who assessed the AI's recommendations in comparison with expert clinicians' medical decision making to ensure alignment with best practices and the standard of care. RESULTS Preliminary results indicate that the AI CDSS demonstrated high accuracy in diagnostic recommendations and decision support across various scenarios. However, certain limitations were noted, such as occasional discrepancies in handling complex or nuanced cases that required a more contextual understanding. Additionally, the tool scored higher on the diagnostic portion of the rubric, with lower scores in the therapeutic recommendations. These findings highlight the importance of continuous refinement and validation of AI tools through rigorous evaluation frameworks like the OSCE. The adaptation of OSCEs for AI tools presents several advantages, including a structured and reproducible approach to evaluation, the ability to test AI systems in diverse clinical scenarios, and the opportunity to benchmark AI performance against established clinical standards to permit charting of future progress as aerospace medicine evolves as a discipline. Remaining challenges include ensuring that these evaluations capture the full spectrum of clinical decision-making scenarios that will be confronted by CMOs during missions and adequately reflecting real-world variability of the austere spaceflight environment. CONCLUSION Employing OSCEs to evaluate AI clinical decision support tools offers a promising approach to validating their clinical utility and efficacy. This methodology not only provides insights into the tool's performance but also fosters ongoing improvement and alignment with standard of care practices. Future research should focus on refining these evaluation processes and addressing limitations to enhance the integration of AI tools in clinical spaceflight settings. REFERENCES Scott S, Hearns V, Barker MA. Testing Clinical Skills: A Look at the OSCE and USMLE Clinical Skills Exams. S D Med. 2019 Oct;72(10):451-453. Majumder MAA, Kumar A, Krishnamurthy K, Ojeh N, Adams OP, Sa B. An evaluative study of objective structured clinical examination (OSCE): students and examiners perspectives. Adv Med Educ Pract. 2019 Jun 5;10:387-397. Karam VY, Park YS, Tekian A, Youssef N. Evaluating the validity evidence of an OSCE: results from a new medical school. BMC Med Educ. 2018 Dec 20;18(1):313.

Ariana M Nelson↗
NASA NTRS2025

Navigating Team Dynamics: Automated Detection of Micro-Behaviors Between Team Members Through Longitudinal Interaction Data

The success in future long term space exploration missions will depend on the cooperation, coordination, and mutual understanding among the crew members. Micro-behaviors are momentary, subtle linguistic and paralinguistic indicators of thinking and feeling toward another member of the team (Cortina et al., 2001; Smith & Griffiths, 2022) that can significantly impact team dynamics and influence the overall team performance (Paromita & Chaspari, 2024). Due to their interactive nature, micro-behaviors have a sender (i.e., the team member expressing the micro-behavior) and a target (the team member impacted by the micro-behavior). Detection of these behaviors can assist in avoiding possible conflict among crew members and promoting the overall team success. Our prior research focused on an initial proof of concept of machine learning (ML) models and natural language processing (NLP) techniques that were used for automatically detect micro-behaviors among crew members of the US National Aeronautics and Space Administration’s (NASA) Human Exploration Research Analog (HERA) Campaigns 4 and 5 missions (Paromita et al., 2023). Results underscored the importance of incorporating contextual information in the ML models in the form of sentiment analysis, type of task, and dyadic interaction among team members. Here, we expand the scope of our prior work in two ways. First, we assess ML/NLP methods on new behavioral annotations coded using an adapted version of Smith & Griffins (2022) theoretical framework in terms of Violation (i.e., presence of valenced behavior, uplifting/positive or discouraging/negative), Intensity (i.e., force of behavior in terms of how uplifting or discouraging is the behavior), and Intent (i.e., motive of the behavior in terms of whether it was deliberate or unintentional). Second, we expand the design of the ML model to preserve information about the role of each team member within the occurrence of the micro-behavior (in contrast to the previous model that only considered the sender and the target without determining the team member role). This allows to consider all team members' contributions in the conversation and model long-term dependencies in the dialogue. Our experiments for this study are conducted on data from 5 teams of the NASA HERA C4 (NASA grant NNX16AQ48G (PI: Bell)). Conversations were extracted from the 1.5 hour Team Interaction Battery (TIB) task that occurred 5 times in-mission per crew. This resulted in a total of 13,058 conversational turns (i.e., 17.8% uplifting, 3.3% discouraging, 75.76% neutral, 3.14% nulls). Our findings with the revised behavioral coding and ML/NLP models indicate a 43.66% macro F1-score (i.e., 38.29% precision (P), 50.8% recall (R)) for a dialog state-tracking model that includes information from the sender only, and a 40.9% F1-score (i.e., 38.7% P, 43.36% R) for the same model that includes information from both the sender and the target of the micro-behavior. These are significantly higher compared to simple random forest models that classify behaviors strictly based on speech content and do not consider iterative team dynamics, achieving a 36.07% F1-score (i.e., 39.04% R, 33.53% P). Our findings demonstrate potential ways to leverage large conversational datasets to better capture complex team dynamics. We will discuss future directions including proposed models that can incorporate additional mission days and tasks beyond the TIB for objectively quantifying team behavior at high temporal resolution in space exploration missions.

Projna Paromita↗
DOE OSTI2020

Robust Assessment of Real-World Adversarial Examples

We explore rigorous, systematic, and controlled experimental evaluation of adversarial examples in the real world and propose a testing regimen for evaluation of real world adversarial objects. We show that for small scene/ environmental perturbations, large adversarial performance differences exist. Current state of adversarial reporting exists largely as a frequency count over a dynamic collections of scenes. Our work underscores the need for either a more complete report or a score that incorporates scene changes and baseline performance for models and environments tested by adversarial developers. We put forth a score that attempts to address the above issues in a straight-forward exemplar application for multiple generated adversary examples. We contribute the following: 1. a testbed for adversarial assessment, 2. a score for adversarial examples, and 3. a collection of additional evaluations on testbed data.

deep learning, adversarial machine learning, machi↗
DOE OSTI2021

Local and global decision fusion for cyber-physical system abnormality detection

Monitoring nodes may generate a series of current monitoring node values over time representing current operation of a cyber-physical system. A decision fusion computer platform may receive, from a local status determination module, an indication of whether each node has an initial local status of “normal”/“abnormal” and a local certainty score (with higher values of the local certainty score representing greater likelihood of abnormality). The computer platform may also receive, from a global status determination module, an indication of whether the system has an initial global status of “normal”/“abnormal” and a global certainty score. The computer platform may output, for each node, a fused local status of “normal” or “abnormal,” at least one fused local status being based on the initial global status. The decision fusion computer platform may also output a fused global status of “normal” or “abnormal” based on at least one initial local status.

Abbaszadeh, Masoud↗
DOE OSTI2021

Prioritizing Countries by Concern Regarding Access to Weapons of Mass Destruction Materials

This article presents a global prioritization methodology that evaluates the relative risks of non-state actor acquisition of materials that could be used in chemical, biological, radiological, nuclear and high explosive Weapons of Mass Destruction (WMD) from the country’s relevant infrastructure. Prioritization is based on three domains: 1. Assessing relative scale of materials in each country, 2. The country’s corresponding security posture, and 3. The presence of threat actors. The output is a list of countries prioritized from greatest risk to least. Rather than providing an overall 1 to N ranking, however, the results are placed into tiers based upon their natural groupings within the three domains. The countries in the highest tiers are flagged as potential US national security concern; those scoring in the middle and at the bottom are flagged as posing lower US national security concern. A systematic approach assesses each country by leveraging many disciplines, such as risk and decision analysis, as well as expert judgement. A quantitative value model based on Multi-Attribute Value Theory (MAVT) organizes the objectives scoring criteria into a value tree using lessons learned from previous studies, published literature, and expert judgement. The article presents the prioritization categories and corresponding value model scoring criteria to include measurement type, weight, range, and value preference. Country names and data are notional in order to share the details on the underlying methodology and model without identification of actual security risks. A deliberative process addresses factors external to the model and scrutinizes inputs, methodology, model, and results.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2021

Risk-informed autonomous adaptive cyber controllers

Technology related to risk-informed autonomous adaptive cyber controllers is disclosed. In one example of the disclosed technology, a method includes generating probabilities of a cyber-attack occurring along an attack surface of a network. The probabilities can be generated using sensor and operational data of a network as inputs to an attack graph. The risk scores can be determined using a plurality of fault trees and the generated probabilities from the attack graph. The respective risk scores can correspond to respective nodes of an event tree. The event tree and the determined risk scores can be used to determine risk estimates for a plurality of configurations of the network. The risk estimates for the plurality of configurations of the network can be used to reconfigure the network to reduce a risk from the cyber-attack.

Veeramany, Arun↗
DOE OSTI2022

Automated Shift Detection in Sensor-Based PV Power and Irradiance Time Series

PV power and irradiance sensor-based measurements are prone to error, resulting in issues such as abrupt time series data shifts. These shifts, which are usually unintentional, may be caused by software or hardware configuration changes on a PV system, and do not reflect an actual change in overall system performance. Locating these shifts and segmenting the associated time series aids in more accurate future PV analysis. In this research, an offline changepoint detection (CPD) algorithm that automatically detects these abrupt data shifts in sensor-based time series is introduced. Data shift periods in 101 daily PV power and irradiance time series were labeled manually by two solar experts. These data streams represent sensor-based measurements, and display a variety of data shift behaviors. A changepoint detection algorithm was tuned using the 101 labeled data streams, with each model configuration's ability to detect labeled changepoints benchmarked using metrics such as F1-score, recall, and Rand Index. Best performing models on seasonality-corrected data streams include the Pruned Exact Linear (PELT) method, the Binary Segmentation method, and the Bottom-Up method, all scoring an average F1-score of 0.76 or greater at detecting labeled changepoints within a 30-day window for the labeled data sets. To promote further research in this space, we are releasing the labeled data shift sets on U.S. Department of Energy's (DOE) DuraMAT Data Hub, and the associated algorithm in the Python PVAnalytics package.

changepoint detection↗
DOE OSTI2022

Automated Shift Detection in Sensor-Based PV Power and Irradiance Time Series

PV power and irradiance sensor-based measurements are prone to error, resulting in issues such as time series data shifts. In this research, a changepoint detection (CPD) algorithm that automatically detects data shifts in sensor-based time series is introduced. Data shift periods in 101 daily PV power and irradiance time series were labeled manually by two solar experts. These data streams represent sensor-based measurements, and display a variety of data shift behaviors. A changepoint detection algorithm was tuned using the 101 labeled data streams, with each model configuration's ability to detect labeled changepoints benchmarked using metrics such as F1-score, recall, and Rand Index. Best performing models on seasonality-corrected data streams include the Pruned Exact Linear (PELT) method, the Binary Segmentation method, and the Bottom-Up method, all scoring an average F1-score of 0.76 or greater at detecting labeled changepoints within a 30-day window across the labeled data sets. Pending approval, we plan to release the labeled data sets for this research on NREL's DuraMAT Data Hub, and the associated algorithm in the Python PVAnalytics package. By supplying the training sets and algorithm, we hope to encourage further development in this research space.

data shift↗
DOE OSTI2023

CyTRICS Impact-Based Prioritization Process

Cyber Testing for Resilient Industrial Control Systems™ (CyTRICS™) is the Department of Energy’s (DOE’s) program for cybersecurity vulnerability testing, digital subcomponent enumeration, and forensic assessment. CyTRICS leverages best-in-class test facilities and analytic capabilities at six DOE National Laboratories and strategic partnerships with key stakeholders including technology developers, manufacturers, asset owners and operators, and interagency partners. During the program’s development, CyTRICS established a unique methodology for prioritizing digital components within operational technology (OT) and industrial control systems (ICS) in the Energy Sector Industrial Base (ESIB) for cyber vulnerability testing. The CyTRICS prioritization process leverages multiple characteristics of systems, components, and their contextual deployment to calculate a quantification of individual digital components for CyTRICS testing. The initial version of the CyTRICS prioritization process was premised largely upon the impact which could result to an energy sector industrial control system if the digital component under testing was compromised, either through malicious means, faulty engineering, or other modes. CyTRICS has termed this process the “CyTRICS Impact-based Prioritization Process.” This paper describes the factors identified for use in the Impact-based Prioritization process and identifies the rationale for inclusion. During development, three National Laboratories piloted this prioritization process and generated prioritization scores for seven systems. Following the piloting of the process, laboratory subject matter experts (SME) validated that the numerical scores generated by the prioritization process were consistent with their knowledge of the impact that may occur should any of these systems be disrupted. The following document explains how to perform the prioritization process to generate prioritization scores for energy sector systems. After outlining assumptions required to conduct the process, it describes how to identify and elicit data which can be leveraged to evaluate a system and assign numerical values for each factor. The prioritization process uses different weights on different factors; rationale for each weight is included within the paper. Additionally, the paper includes some recommendations for future enhancements to prioritization, including lessons learned from developing and piloting the process. Finally, a comprehensive appendix includes example documents to be leveraged by those looking to execute the prioritization process.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2020

System Health and Status Reporting is Used to Improve Infrastructure Systems at the Hanford Site - 20449

Reliable physical infrastructure systems are vital to completing the clean-up mission and the start-up and commissioning of new nuclear waste processing facilities at the Hanford Site. Key infrastructure systems at Hanford include water, sewer, and electrical utilities, roads, telecommunications/network systems, and fire systems. All of the infrastructure systems are non-nuclear, General Service systems that are being maintained under formal engineering configuration control and none are specifically credited in any documented safety analysis (DSA). Starting in calendar year 2017, the Hanford Site's infrastructure and services contractor started producing system health and status reports for a total of 15 critical infrastructure systems as means to get a better understanding of each system's health and how it could be improved to ensure better and more reliable service delivery. As the reporting, tracking, and use of the metrics has matured, the number of systems for which reports are being developed has been expanded to a total of 23. Each of the critical infrastructure systems are assigned primary and alternate design authorities that complete qualifications on each assigned system. The design authorities actively manage and maintain configuration control on each of their systems and track metrics associated with availability, maintenance (corrective and preventive), and configuration of the technical baseline. The metrics are based on the principles established in Chapter V of DOE O 420.1C, Facility Safety, [1] and have been tailored as appropriate to each of the key infrastructure systems. An overall assessment of system health is performed and documented on a routine basis (e.g., quarterly). Included in this documentation is a quantitative score for each of the metrics (i.e., availability, maintenance, configuration management) along with results of system walk-downs, description of any permanent design modifications, identification of new or emerging issues and vulnerabilities affecting the system, and an assessment by the design authority of actions necessary to improve the system health score. The overall status scores are calculated using a formula and based on a pre-determined grading scale, a color is assigned for simple presentation. The information gained from developing the reports is being used to (1) assist with making timely and better informed decisions through the use of solid technical bases that are increasing the overall reliability and capabilities of the Site's infrastructure systems as new expense and capital projects in the Reliability Projects Investment Portfolio (RPIP) are prioritized and executed; (2) feed the risk management program, and (3) integrate the technical content of Master Plans that describe the Site's needs for the system and the system's capabilities to meet those future needs. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2024

Data-Informed Evaluation Framework for Integrated Energy Systems: Insights from Power, Process Heat, and Hydrogen Production Applications

The multi-criteria decision analysis (MCDA) framework provides a systematic evaluation of the diverse preferences and performance metrics associated with alternative solutions. This approach is advantageous over a single-criterion methodology, which are only valid under conditions that assume ceteris paribus or an "apples-to-apples" comparison. However, selecting suitable technologies for integrated energy systems (IES) can be likened to an "apples-to-oranges" comparison, given the heterogeneous factors at stake. These factors include economics and performance parameters, geological compatibility, and environmental impacts. Consequently, past research has often employed a mixture of qualitative and quantitative criteria tailored to the specific interests of each study. While the method proves effective in handling the intricate interplay of criteria, the resulting rankings and scores can vary from study to study. This inconsistency is introduced from the use of subjectively defined thresholds and weights. As a result, decision-makers frequently find it challenging to establish clear connections between specific criteria and the resulting scores, as the transformation of criteria into ordinal scores results in a substantial loss of information. To address this challenge, we introduce a data-informed IES evaluation framework that offers comprehensive, interpretable, and traceable evaluations backed by quantifiable rationale. First, we identified key IES evaluation criteria from a decade of literature, focusing on relevant IES applications in power, process heat, and hydrogen production. We leveraged state-of-the-art cost estimates from the Idaho National Laboratory (INL) and technical data from 78 reactor designs from the International Atomic Energy Agency (IAEA) and the Organization for Economic Co-operation and Development - Nuclear Energy Agency (OECD-NEA). Lastly, we established thresholds by analyzing the mean, variance, root mean square, and slope of values across alternatives, categorizing the preferences of decision-makers into distinct utility functions, such as linear, saturating, exponential, and stepwise. Our approach yielded two main outcomes: (1) it provided consistent assessments across different stakeholder groups and (2) it visualized uncertainties in the decision-making context via comprehensive sensitivity analysis. To demonstrate the impact of our framework, we conducted case studies on 6 reactor designs (AP1000, NuScale, BWRX-300, Xe-100, eVinci, iMSR) for the three applications. Our data-driven framework proved to be highly effective in addressing heterogenous uncertainties faced by varied decision-makers? preferences and IES applications, as well as cost and technical estimates of advanced reactors.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2025

Evaluation of the 2022 West Nile virus forecasting challenge, USA

Abstract Background West Nile virus (WNV) is the most common cause of mosquito-borne disease in the continental USA, with an average of ~1200 severe, neuroinvasive cases reported annually from 2005 to 2021 (range 386–2873). Despite this burden, efforts to forecast WNV disease to inform public health measures to reduce disease incidence have had limited success. Here, we analyze forecasts submitted to the 2022 WNV Forecasting Challenge, a follow-up to the 2020 WNV Forecasting Challenge. Methods Forecasting teams submitted probabilistic forecasts of annual West Nile virus neuroinvasive disease (WNND) cases for each county in the continental USA for the 2022 WNV season. We assessed the skill of team-specific forecasts, baseline forecasts, and an ensemble created from team-specific forecasts. We then characterized the impact of model characteristics and county-specific contextual factors (e.g., population) on forecast skill. Results Ensemble forecasts for 2022 anticipated a season at or below median long-term WNND incidence for nearly all (> 99%) counties. More counties reported higher case numbers than anticipated by the ensemble forecast median, but national caseload (826) was well below the 10-year median (1386). Forecast skill was highest for the ensemble forecast, though the historical negative binomial baseline model and several team-submitted forecasts had similar forecast skill. Forecasts utilizing regression-based frameworks tended to have more skill than those that did not and models using climate, mosquito surveillance, demographic, or avian data had less skill than those that did not, potentially due to overfitting. County-contextual analysis showed strong relationships with the number of years that WNND had been reported and permutation entropy (historical variability). Evaluations based on weighted interval score and logarithmic scoring metrics produced similar results. Conclusions The relative success of the ensemble forecast, the best forecast for 2022, suggests potential gains in community ability to forecast WNV, an improvement from the 2020 Challenge. Similar to the previous challenge, however, our results indicate that skill was still limited with general underprediction despite a relative low incidence year. Potential opportunities for improvement include refining mechanistic approaches, integrating additional data sources, and considering different approaches for areas with and without previous cases. Graphical Abstract

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Assessment of oil and gas fields in California as potential CO 2 storage sites

California's total annual greenhouse gas (GHG) emissions (425.3 MtCO 2 e) in 2018 were about 6.4% of the US total (6,677 MtCO 2 e) and around 1% of global emissions. About 39% of 2018 GHG emissions in California were from the industrial and electrical sectors. Many of these emissions were from large stationary point sources and were suitable for carbon capture retrofit with subsequent storage of the captured carbon dioxide (CO 2 ) in geological formations. Previous studies of California found suitable geology and CO 2 storage resource. This study refines and furthers prior work using a three-stage screening process of oil fields, gas fields, and underground natural gas storage (UGS) sites by combining criteria from previous studies while excluding sites that pose technical risk or are located in regions with surface restrictions including sensitive habitats and dense populations. In the first stage, 129 CO 2 storage sites in California were identified using qualification criteria based upon formation properties including geological conditions and pore pressure. The second stage identified sensitive sites by applying conservative screens including seismic activity, faulting, population density, restricted lands, and sensitive habitats. During the third stage, 61 CO 2 potential storage sites were identified by subtraction of stage 2 areas from stage 1. The potential storage volume in the third stage ranged from 1.0 to 2.0 GtCO 2 . Finally, we applied a scoring system with seven parameters to rank the 61 potential sites based on subsurface technical criteria. The scored sites are classified as high priority, medium priority, and sites for future study. Prospective CO 2 storage sites with high and moderate priority were selected and linked to CO 2 sources. There are 14 prospective sites (above 20 MtCO 2 storage resource per site) with a total storage resource of 1024 MtCO 2 distributed in Northern and Southern California. Of these sites, there are 9 potential CO 2 -EOR sites and 1 depleted oil field with a total estimated CO 2 storage volume of ~800 MtCO 2 in the Southern San Joaquin and Ventura Basin. These 10 prospective sites with a storage resource greater than 20 MtCO 2 could potentially deliver more than 20 years of storage with an average injection rate of 40 MtCO 2 /year. The remaining 4 highly prospective sites are in Northern California. Additionally, study results suggest that saline formations should be re-evaluated in concert with storage in oil, gas, and natural gas storage reservoirs.

58 GEOSCIENCES↗
DOE OSTI2025

Joint state-parameter estimation for the reduced fracture model via the united filter

Here, in this paper, we introduce an effective United Filter method for jointly estimating the solution state and physical parameters in flow and transport problems within fractured porous media. Fluid flow and transport in fractured porous media are critical in subsurface hydrology, geophysics, and reservoir geomechanics. Reduced fracture models, which represent fractures as lower-dimensional interfaces, enable efficient multi-scale simulations. However, reduced fracture models also face accuracy challenges due to modeling errors and uncertainties in physical parameters such as permeability and fracture geometry. To address these challenges, we propose a United Filter method, which integrates the Ensemble Score Filter (EnSF) for state estimation with the Direct Filter for parameter estimation. EnSF, based on a score-based diffusion model framework, produces ensemble representations of the state distribution without deep learning. Meanwhile, the Direct Filter, a recursive Bayesian inference method, estimates parameters directly from state observations. The United Filter combines these methods iteratively: EnSF estimates are used to refine parameter values, which are then fed back to improve state estimation. Numerical experiments demonstrate that the United Filter method surpasses the state-of-the-art Augmented Ensemble Kalman Filter, delivering more accurate state and parameter estimation for reduced fracture models. This framework also provides a robust and efficient solution for PDE-constrained inverse problems with uncertainties and sparse observations.

Bayesian inference↗
DOE OSTI2025

Randomized Algorithms for Symmetric Nonnegative Matrix Factorization

Symmetric Nonnegative Matrix Factorization (SymNMF) is a technique in data analysis and machine learning that approximates a matrix with a product of a nonnegative, low-rank matrix and it transpose. To design faster and more scalable algorithms for SymNMF we develop two randomized algorithms for its computation. The first method uses randomized matrix sketching to compute an initial low-rank approximation to the input matrix and proceeds to uses this as a low-rank input to rapidly compute a SymNMF. The second methods uses randomized leverage score sampling to approximately solve constrained least squares problems. Many successful methods for SymNMF rely on (approximately) solving sequences of constrained least squares problems. Here, we prove theoretically that leverage score sampling can approximately solve constrained least squares problems to e-accuracy. Finally we demonstrate both methods work in practice by applying them to graph clustering tasks on large real world data sets. These experiments show that our methods approximately maintain solution quality and achieve significant speed ups for both large dense and large sparse problems.

97 MATHEMATICS AND COMPUTING↗