Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “reliability metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Factors controlling injection-induced rupture of intersecting faults during geological sequestration of CO 2

This study addresses coupled multiphase fluid flow and geomechanics effects on potential fault activation associated with subsurface CO 2 injection around intersecting faults. An enhanced fault-representation model is used to capture geomechanical responses of two intersecting faults with finite length during CO 2 injection. The faults are embedded in a strike-slip stress regime of a caprock-reservoir-basement system with the faults represented by zero-thickness interfaces with adjacent finite-thickness damage zones. A sensitivity analysis is conducted to study the effect of fault permeability, slip-weakening behavior, well location relative to the orientation of faults, and well placement (the number and location of injection wells). Five metrics (pressure, CO 2 plume, shear state on the fault, as well as shear displacement and stress path at selected fault monitoring points) are selected to assess CO 2 migration and reactivation of intersecting faults. The results show that induced ruptures are favored by low permeability faults due to high pressure buildup and by slip-weakening behavior resulting from fault strength reduction. The location of one injection well relative to fault orientation determines the magnitude of changes in effective normal stress and shear stress, affecting the location of induced ruptures. Well placement (two injection wells used in the paper) dominates pressure diffusion around the intersection and tips of faults. This redistributes changes in effective normal stress caused by each injection well, influencing the spatial distribution of ruptures along faults. A larger injection volume induces far-field ruptures that are controlled by stress transfer within the injection layer. The findings presented here can provide valuable insights into engineering operations for a long-term, safe, and reliable geologic CO 2 storage.

Fault permeability↗
DOE OSTI2025

A Framework for Assessing Economic and Environmental Trade-offs of Internalized Emission Costs in ERCOT Grid Planning

The power grid is on the cusp of a massive transition driven by three major areas: 1) the growth in demand for electricity, 2) efforts to decarbonize the United States economy, and 3) a desire to mitigate social disparities from the impact of electricity generation on local populations. However, most studies of the electricity sector do not include equity impacts in their models. This study seeks to do so by developing a comprehensive and generalizable model tailored to the Electric Reliability Council of Texas (ERCOT) grid, designed to incorporate the equity impacts of electricity generation in a decarbonized and resilient framework. To integrate equity into our research, we incorporate environmental externalities into our capacity expansion model of ERCOT. Specifically, we factor in intermediate-level local marginal damages of precursor pollutants (NH3, NOx, primary PM2.5, SO2, and VOC) and global pollutant CO2 into the cost of generating electricity. We do this by taking into account county population, county ambient pollution concentration, and generator emission rates. Leveraging open-source modeling tools, such as PowerGenome, pyGRETA, and GenX we construct a county-level model to account for these costs. We integrate these marginal damages into the variable operations and maintenance costs of generators, for both existing and potential future builds. This study’s findings suggest that the value of a dynamic social cost of carbon (SSC) will cover criteria pollutant marginal damages within the ERCOT grid and solar and wind is expected to increase out to 2035. Key metrics evaluated within this research include fuel mix distribution across technologies, transmission and grid infrastructure costs, CO2 emissions, local pollutants marginal damages, and the variation in generation capacity built by the model. These results and framework can be used to support grid decisions that explicitly include distributional and procedural equity within a decarbonized and sustainable grid framework.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Review of data-driven models for quantifying load shed by non-residential buildings in the United States

Shifting and shedding power demand in buildings can be cost-effective techniques for grids to function reliably and for end users to earn compensation. Grid operators reimburse customers in proportion to the quantity of load shed. Simple data-driven methods are used to quantify this shed, which is the difference between a measured load during the event and modeled "baseline" that would have occurred in absence of the event. These methods have evolved over the years and in many cases have been integrated with building physics, to make them a hybrid between physics based and empirical models. However, there is no comprehensive analysis that provides guidance to building operators, grid operators and researchers in selecting appropriate models based on their specific needs and available data. Here, this work aims to fill this gap by critically assessing the performance of baseline models put forward from the year 2000 through 2023. The literature reviewed includes reports generated by grid operators, reports from national laboratories and academic journal articles. The work outlines modeling features like the inputs, training period, estimation method, adjustments to fine tune the predictions and metrics to evaluate the performance. A comprehensive list of 50 models has been provided. For each model, the study explores the applicability of the model to weather sensitive buildings, variability in the building profile, timing of the event, and whether the building reduces energy consumption before an event. The work identifies the situations in which a particular model works and draws lessons based on evidence of performance. Finally, recommendations to aid in model selection are given.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Primary Heat Transport System Design Considerations for Xcimer Energy’s Athena Fusion Pilot Plant

Fusion energy promises a reliable, carbon-free source of power; however, significant challenges remain before it can be deployed as an economical energy source. In addition to achieving fusion conditions, power plants must operate under extreme temperatures, radiation, and mechanical loads while maintaining high efficiency and availability. These requirements place strong demands on engineering design and plant operation. This work focuses on the engineering challenges associated with balance of plant analysis for inertial fusion energy systems. In particular, this paper examines the design considerations for primary heat transfer systems in fusion pilot plants employing molten fluoride salt coolants, with particular emphasis on system layout optimization and the balance between competing design objectives using the Xcimer Energy Athena inertial pilot plant design as a case study. Through systematic analysis of candidate system configurations and parametric sensitivity studies, we identify key engineering trade-offs governing salt inventory, pumping power requirements, and operational flexibility. The analysis employs system-level modeling tools to explore the design space and establish relationships between geometric parameters and system performance metrics.

Greenwood, Scott [ORNL] (ORCID:0000000333480736)↗
DOE OSTI2022

Wind and Solar Hybrid Power Plants for Energy Resilience

Wind-solar-storage hybrid power plants represent a significant and growing share of new proposed projects in the United States (U.S.). Their uptake is supported by increasing renewable energy market share, technical abilities for dispatch and control, and decreasing wind, solar, and battery storage costs. Simultaneously, generation and storage resources are increasingly used in distributed power systems. While concerns around the reliability of the aging, transforming U.S. electric grid are growing, diversifying energy resources through hybridization or spatial distribution provides an opportunity to enhance power system resilience compared to single-source generation. Understanding where to build hybrids for resilience value, rather than bulk power supply, has not been fully explored in previous studies. Therefore, in this study, we complete a national complementarity analysis to identify areas in the U.S. that are particularly suited for wind-solar hybrid power plant development. We show the importance of seasonal and diurnal patterns in assessing complementarity, and identify that regions in the Great Plains, Midwest, and Southeast are particularly suited for hybrid power plants. We demonstrate the resilience value of hybridization for a reference system based near Memphis, Tennessee, and show optimal sizing of wind, solar, and storage assets given 1.0 and 0.9 critical load factors. Results indicate that pairing wind and solar assets better meet constant load demand and reduce storage requirements compared to solar alone. These results enable future work integrating complementarity metrics in resilience frameworks and indicate a need for more finer resolution of local resource, demand, and hazard data.

14 SOLAR ENERGY↗
DOE OSTI2020

The Value of Information From Horizontal Distributed Acoustic Sensing Compared to Multicomponent Geophones Via Machine Learning

Abstract Faults play an important role in recharging many geothermal reservoirs, and seismic information can image the locations of these faults. The value of information (VOI) metric is used to objectively quantify and compare the value of two types of seismic receiver data via a machine learning approach. The demonstrated VOI methodology is novel by including spatial models from seismic data and obtaining the information statistics from machine learning. Our two-dimensional numerical experiments compare images created from sparsely spaced (80 m), two-component geophone sampling to high spatial resolution (1 m), single-component DAS. We used a three-fold cross validation of a U-Net convolutional neural networks to achieve average classification statistics. The results suggest that when horizontal sources are utilized, geophones and DAS identify reflectors and non-reflectors at roughly the same rate. The average F1 score for horizontal DAS is 0.939 and 0.931 for geophones. For images created from a vertical source, DAS performed marginally better (F1 = 0.919) than geophones (F1 = 0.877). Our transferrable methodology can provide guidance on which acquisition scenarios can improve images of important structures in the subsurface and present an efficient method for obtaining reliability statistics from high-dimensional, spatial data.

Energy & Fuels↗
DOE OSTI2023

Uncertainty quantification in machine learning for engineering design and health prognostics: A tutorial

On top of machine learning (ML) models, uncertainty quantification (UQ) functions as an essential layer of safety assurance that could lead to more principled decision making by enabling sound risk assessment and management. The safety and reliability improvement of ML models empowered by UQ has the potential to significantly facilitate the broad adoption of ML solutions in high-stakes decision settings, such as healthcare, manufacturing, and aviation, to name a few. In this tutorial, we aim to provide a holistic lens on emerging UQ methods for ML models with a particular focus on neural networks and the applications of these UQ methods in tackling engineering design as well as prognostics and health management problems. Towards this goal, we start with a comprehensive classification of uncertainty types, sources, and causes pertaining to UQ of ML models. Next, we provide a tutorial-style description of several state-of-the-art UQ methods: Gaussian process regression, Bayesian neural network, neural network ensemble, and deterministic UQ methods focusing on spectral-normalized neural Gaussian process. Established upon the mathematical formulations, we subsequently examine the soundness of these UQ methods quantitatively and qualitatively (by a toy regression example) to examine their strengths and shortcomings from different dimensions. Then, we review quantitative metrics commonly used to assess the quality of predictive uncertainty in classification and regression problems. Afterward, we discuss the increasingly important role of UQ of ML models in solving challenging problems in engineering design and health prognostics. In conclusion, two case studies with source codes available on GitHub are used to demonstrate these UQ methods and compare their performance in the life prediction of lithium-ion batteries at the early stage (case study 1) and the remaining useful life prediction of turbofan engines (case study 2).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Comparison of methodologies used to determine aromatic lignin unit ratios in lignocellulosic biomass

Abstract Background Multiple analytical methods have been developed to determine the ratios of aromatic lignin units, particularly the syringyl/guaiacyl (S/G) ratio, of lignin biopolymers in plant cell walls. Chemical degradation methods such as thioacidolysis produce aromatic lignin units that are released from certain linkages and may induce chemical changes rendering it difficult to distinguish and determine the source of specific aromatic lignin units released, as is the case with nitrobenzene oxidation methodology. NMR methods provide powerful tools used to analyze cell walls for lignin composition and linkage information. Pyrolysis-mass spectrometry methods are also widely used, particularly as high-throughput methodologies. However, the different techniques used to analyze aromatic lignin unit ratios frequently yield different results within and across particular studies, making it difficult to interpret and compare results. This also makes it difficult to obtain meaningful insights relating these measurements to other characteristics of plant cell walls that may impact biomass sustainability and conversion metrics for the production of bio-derived fuels and chemicals. Results The authors compared the S/G lignin unit ratios obtained from thioacidolysis, pyrolysis-molecular beam mass spectrometry (py-MBMS), HSQC liquid-state NMR and solid-state (ss) NMR methodologies of pine, several genotypes of poplar, and corn stover biomass. An underutilized approach to deconvolute ssNMR spectra was implemented to derive S/G ratios. The S/G ratios obtained for the samples did not agree across the different methods, but trends were similar with the most agreement among the py-MBMS, HSQC NMR and deconvoluted ssNMR methods. The relationship between S/G, thioacidolysis yields, and linkage analysis determined by HSQC is also addressed. Conclusions This work demonstrates that different methods using chemical, thermal, and non-destructive NMR techniques to determine native lignin S/G ratios in plant cell walls may yield different results depending on species and linkage abundances. Spectral deconvolution can be applied to many hardwoods with lignin dominated by S and G units, but the results may not be reliable for some woody and grassy species of more diverse lignin composition. HSQC may be a better method for analyzing lignin in those species given the wealth of information provided on additional aromatic moieties and bond linkages. Additionally, trends or correlations in lignin characteristics such as S/G ratios and lignin linkages within the same species such as poplar may not necessarily exhibit the same trends or correlations made across different biomass types. Careful consideration is required when choosing a method to measure S/G ratios and the benefits and shortcomings of each method discussed here are summarized.

09 BIOMASS FUELS↗
DOE OSTI2023

Characterization of high-temperature figure of merit for solar-thermal absorbers

The figure of merit (FOM) is a widely used metric to characterize the solar-thermal conversion performance of solar absorber layers. Many of previous works have reported FOMs estimated from the radiative properties measured at room temperature. However, advanced concentrated solar power (CSP) plants operate at very high temperatures (~750°C), which can strongly affect the absorber’s radiative properties. Here, the present work proposes using a 1-D conduction-based reference bar technique to directly measure the converted thermal energy to determine the FOM at high absorber temperatures (≳ 750°C) while operating under a large solar concentration (~ 1200 x suns). Using the finite difference method, we carefully analyze several key parameters influencing the design of the experimental setup, such as temperature distribution, material properties, heat loss, and sensor accuracy. In addition, a rigorous uncertainty analysis is incorporated into the FOM to ascertain the confidence level of the result. We validate this new technique on a custom-built tungsten reference bar with an oxidized Inconel 625 solar-thermal absorber. A good agreement between the model and the experiment shows the reliability of the FOM measurement with a well-defined uncertainty. The same method can be implemented to measure the high-temperature FOM of most surface or volumetric solar absorbers for CSP.

14 SOLAR ENERGY↗
DOE OSTI2026

Toward an AI-Powered Software Pipeline for Real-Time Tracking and Analysis of Wildfire and Smoke

Real-time tracking of wildfires and smoke is crucial for effective response, minimizing damage, protecting lives, and efficiently managing resources during fire emergencies. We develop a web-based AI-powered pipeline that detects wildfires in aerial video and estimates deployment-relevant behavior metrics, including cumulative burned area, burned-area growth rate, fire spread direction, and smoke dispersion. The system combines a YOLO-based detector with YCbCr-based fire segmentation, HSV-based smoke segmentation, Farneback optical flow, and centroid-based spatiotemporal tracking. Using ground sampling distance (GSD), pixel-level fire masks are converted to physical burned-area measurements by correlating fire pixel counts with camera altitude and tilt angle. We benchmark YOLO variants and non-YOLO baselines (GoogLeNet, CNN, DBN, Autoencoder, U-Net, and AlexNet) on the IEEE FLAME dataset and a newly created aerial frame dataset, Wildfire-DB. Cross-dataset evaluation uses a strict threshold-transfer protocol: decision thresholds are selected on FLAME validation and transferred unchanged to Wildfire-DB to quantify generalization under domain shift. YOLOv6 achieves the strongest cross-dataset frame-level fire detection on Wildfire-DB (ROC-AUC 0.8200, PR-AUC 0.8044, and transferred-threshold F1 0.7596). For tracking-oriented deployment requiring oriented localization, YOLO11-OBB provides the most reliable cross-dataset behavior among OBB-capable models while remaining computationally feasible. To analyze the feasibility of UAV deployment, we further measure inference efficiency using synchronized GPU and CPU power logs on a fixed workload of 1569 frames. YOLO-family models process the video in 5.73–12.47 seconds with net energy of 1247.28–1775.39 J, substantially lower latency and energy than heavier classification and reconstruction baselines. Overall, model optimality depends on operational objectives: YOLOv6 is best for cross-dataset detection robustness, whereas YOL...

Color segmentation↗
DOE OSTI2021

Guided Resilience Self Assessment Application

Reliable electricity access is vital for everything from hospitals to national security. When grid disturbances occur, it is crucial to minimize the amount of time it takes to recover. Grid resiliency considers both the preparation for and recovery from high-impact low-frequency disturbances (HILF) such as weather events and cyber attacks. The Department of Energy (DOE) Wind Energy Technologies Office (WETO) funds the multi-laboratory Microgrids, Infrastructure Resilience, and Advanced Control Launchpad (MIRACL) project to investigate technologies to support and grow distributed wind. Researchers at Idaho National Laboratory (INL) contribute to resilience and cybersecurity efforts for the project, and have produced a resilience framework to evaluate system resilience based on individual system specifications. In order to make this framework easy for anyone to apply, I have coded a web application that allows users to input their power system qualities, goals, and perceived risks and then receive suggestions on how to improve their grid resiliency. The web application follows the seven steps in the planning stage of the framework process, allowing the user to input information about their grid and select from common goals, metrics, and hazards, as well as add their own. The application currently provides information about how to complete each step, requiring minimal prior knowledge of the framework. Future work for this application will include allowing the user to submit line drawings of their system and incorporate modeling tools so that the hazard simulations of applying the framework is completed for the user.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

A Practical Comparison of Data-Driven Prognostics Methods for Energy Systems

This study explores data-driven prognostics for nuclear power plant (NPP) condensers, focusing on tube fouling. We utilized the Asherah nuclear power plant simulator (ANS) to compare four methods: Random Forest (RF), Support Vector Regressor (SVR), Fully Connected Neural Network (FCNN), and Long Short-Term Memory Neural Network (LSTM). By simulating various fouling scenarios in the ANS, we generated data with different degradation rates under transient operations. The models were trained and tested on these data, with performance evaluated visually and numerically including uncertainty assessment. The LSTM model excelled, exhibiting minimal prediction noise and the most accurate remaining useful life estimates across all degradation levels. Its ability to capture long-term dependencies and produce cleaner outputs makes it a strong candidate, although accurate training data across the entire component lifespan are crucial. The RF model emerged as a robust alternative, providing reliable predictions with high confidence. The FCNN and SVR models, while less effective overall, showed potential under specific conditions. FCNN offers a less complex alternative to LSTM and might benefit from larger datasets. SVR excels in precision when the quality of the training data is high. Furthermore, this study highlights the operational benefits of advanced prognostics in the energy sector and emphasizes the need for further research in NPP condenser health management through real-life experiments.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2021

Inter-well connectivity detection in CO 2 WAG projects using statistical recurrent unit models

Routine well-wise injection and production measurements contain significant information on subsurface structure and properties. Data-driven technology that interprets surface data into subsurface structure or properties can assist operators in making informed decisions by providing a better understanding of field assets. Our machine-learning framework is built on the statistical recurrent unit (SRU) model and interprets well-based injection/production data into inter-well connectivity without relying on a geologic model. We test it on synthetic and field-scale CO 2 EOR projects utilizing the water-alternating-gas (WAG) process. SRU is a special type of recurrent neural network (RNN) that allows for better characterization of temporal trends, by learning various statistics of the input at different time scales. In our application, the complete states (injection rate, pressure and cumulative injection) at injectors and pressure states at producers are fed to SRU as the input and the phase rates at producers are treated as the output. Once the SRU is trained and validated, it is then used to assess the connectivity of each injector to any producer using permutation variable importance method, wherein inputs corresponding to an injector are shuffled and the increase in prediction error at a given producer is recorded as the importance (connectivity metric) of the injector to the producer. This method is tested in both synthetic and field-scale cases. The validation of the proposed data-driven inter-well connectivity assessment is performed using synthetic data from simulation models where inter-well connectivity can be easily measured using the streamline-based flux allocation. The SRU model is shown to offer excellent prediction performance on the synthetic case. Despite significant measurement noise and frequent well shut-ins imposed in the field-scale case, the SRU model offers good prediction accuracy, the overall relative error of the phase production rates at most producers ranges from 10% to 30%. It is shown that the dominant connections identified by the data-driven method and streamline method are in close agreement. This significantly improves confidence in our data-driven procedure. The novelty of this work is that it is purely data-driven method and can directly interpret routine surface measurements to intuitive subsurface knowledge. Furthermore, the streamline-based validation procedure provides physics-based backing to the results obtained from data analytics. This study results in a reliable and efficient data analytics framework that is well-suited for large field applications.

42 ENGINEERING↗
DOE OSTI2024

Sensitivity analysis, surrogate modeling, and optimization of pebble-bed reactors considering normal and accident conditions

This research provides a valuable tool that streamlines the optimization process while significantly increasing its accuracy. This study creates a robust framework for reactor design optimization by incorporating comprehensive modeling using the Comprehensive Reactor Analysis Bundle, or BlueCRAB, within the Multiphysics Object-Oriented Simulation Environment (MOOSE). BlueCRAB is the United States Nuclear Regulatory Commission's code suite for non-light water reactor analysis and includes the Griffin, Pronghorn, and Bison applications. This not only improves the efficiency of the optimization process but also enhances the reliability of the results. Such a tool is essential for advancing the state-of-the-art in pebble-bed reactor technology and is critical for achieving the goals of Generation IV reactors, which aim for safe, sustainable, and economically viable nuclear energy solutions. This work presents and applies this workflow on pebble-bed reactors while considering both normal and off-normal conditions. A representative gas-cooled pebble-bed reactor at equilibrium core conditions serves as the nominal design specification for normal operation and is based on previous research. The depressurized loss-of-forced-cooling accident is deployed for off-normal conditions in this work. After defining design-related parameters and quantities of interest regarding reactor safety and performance, this multiphysics model is sampled using the MOOSE stochastic tools module. The result is a comprehensive dataset of configurations, enabling sensitivity analysis and the generation of surrogate models. Subsequently, the dataset and surrogate models are employed in two optimization studies aimed at maximizing fuel utilization and economic profit while adhering to safety and operational constraints. Performing the optimization process with fuel utilization as the metric leads to an improvement of approximately 10%, compared to engineering-judgment-based nominal conditions. The optimization on economic profit leads to an estimated increase of ~300 million USD over the lifetime of the reactor.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Grid Planning Impacts of Hydropower Growth and Decline

Hydropower and pumped storage hydropower (PSH) have a complex and uncertain future in the U.S. electricity system. On one hand, existing assets have the potential for an expanded role in integrating variable renewable energy while new deployment, particularly of PSH, can help meet growing needs for grid flexibility and improved reliability. On the other hand, challenging environmental and cost considerations could limit the extent of new investments in hydropower and PSH capacity and flexibility. This technical presentation demonstrates the use of a high-fidelity capacity expansion model of the U.S. electric grid (the National Renewable Energy Laboratory's Regional Energy Deployment System) to understand the impacts of alternative futures for the hydropower and PSH fleet, including scenarios of both growth and decline. Growth scenarios include new PSH deployment or improved hydropower flexibility, while scenarios of decline reduce the capacity or energy production potential of hydropower and PSH. Economic, environmental, and performance outcomes of the grid are compared across these scenarios to reveal the potential contributions of hydropower and PSH in the U.S. electric sector over the next several decades, focusing on changes to the grid technology mix, electric sector costs, electricity prices, and air emissions. This broad scenario approach demonstrates how flexible hydropower and PSH can help reduce air emissions and cost by complementing variable renewables, but the opposite can occur with reduced hydropower availability, particularly in the next decade. It is important to consider a wide range of scenarios and metrics to gain a national and regional understanding of hydropower's future in the United States.

capacity expansion↗
DOE OSTI2023

Hybrid renewable energy systems: the value of storage as a function of PV-wind variability

As shares of variable renewable energy (VRE) on the electric grid increase, sources of grid flexibility will become increasingly important for maintaining the reliability and affordability of electricity supply. Lithium-ion battery energy storage has been identified as an important and cost-effective source of flexibility, both by itself and when coupled with VRE technologies like solar photovoltaics (PV) and wind. In this study, we explored the current and future value of utility-scale hybrid energy systems comprising PV, wind, and lithium-ion battery technologies (PV-wind-battery systems). Using a price-taker model with simulated hourly energy and capacity prices, we simulated the revenue-maximizing dispatch of a range of PV-wind-battery configurations across Texas, from the present through 2050. Holding PV capacity and point-of-interconnection capacity constant, we modeled configurations with varying wind-to-PV capacity ratios and battery-to-PV capacity ratios. We found that coupling PV, wind, and battery technologies allows for more effective utilization of interconnection capacity by increasing capacity factors to 60%–80%+ and capacity credits to close to 100%, depending on battery capacity. We also compared the energy and capacity values of PV-wind and PV-wind-battery systems to the corresponding stability coefficient metric, which describes the location-and configuration-specific complementarity of PV and wind resources. Our results show that the stability coefficient effectively predicts the configuration-location combinations in which a smaller battery component can provide comparable economic performance in a PV-wind-battery system (compared to a PV-battery system). These PV-wind-battery hybrids can help integrate more VRE by providing smoother, more predictable generation and greater flexibility.

14 SOLAR ENERGY↗
DOE OSTI2026

An improved dataset for predicting mammal infecting viruses from genetic sequence information

There have been several attempts to develop machine learning (ML) models to identify human infecting viruses from their genomic sequences, with varying degrees of success. Direct comparison between models is problematic, because these models are typically trained and evaluated on different datasets with alternative data splitting schemes, features, and model performance metrics. In this paper we present a standardized dataset of mammal infecting and non-infecting viral pathogens, refined from the previous work of Mollentze et al. to include the latest literature evidence, roughly doubling the number of curated host-virus records available to the community, and new host target labels, primate and mammal. The new host labels were included for several reasons, including previous reports that classification performance is better at broader taxonomic ranks and the idea that there may be more data for primate infection that might serve as a suitable proxy for zoonotic potential and avoidance of false positives for human infection due to absence of evidence. On this dataset, we report the performance of eight machine learning models for predicting mammal-infecting viruses from their genomic sequences. We find that randomly assigning cases in our improved dataset to training/testing sets, when compared to the original assignments into training/testing in Mollentze et al., increases the overall average ROC AUC of prediction of human infection from 0.663 ± 0.070 to 0.784 ± 0.013, consistent with the reduction in phylogenetic distance between train and test sets (relative entropy change from 3.00 to 0.08). The broadest host category of mammal infection can be predicted most reliably at 0.850 ± 0.020. We share our improved dataset and code to enable standardized comparisons of machine learning methods to predict human host infections. Overall, we have presented preliminary evidence that classification of virus host infection is more tractable at higher taxonomic ranks, that unsurprisingly reducing the phylogenetic distance between training and test sets can improve predictive performance, that peptide kmer features appear to be harmful to out of sample model performance, and we are left with the question of whether models for virus host prediction can reasonably be expected to perform well in out of sample scenarios given the likelihood that viruses do not share a common ancestor. Consistent with this concern, when the data is resampled such that there is no overlap between viral families in training and test sets (relative entropy > 24), models perform no better than random chance at prediction of human infection regardless of whether kmers are included (ROC AUC 0.50 ± 0.08) or not (ROC AUC 0.50 ± 0.04).

59 BASIC BIOLOGICAL SCIENCES↗