Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Reliability Target”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 577 records · Page 32

Optimal Portfolio Design of Distributed Energy Resources on Puerto Rico Distribution Feeders with Long Outages after Hurricane Maria

This work details a project to design reliable, resilient, and cost-effective networked microgrids considering grid constraints and resilience metrics focused on Puerto Rico distribution feeder locations with long outages after Hurricane Maria. The project consisted primarily of modeling and simulation tasks that accomplished the following objectives: 1. Selected 10 distribution feeder models in vulnerable areas. The sample feeders are geographically distributed across Puerto Rico and vary in length to capture the wide variety of feeders on the island. 2. Determined the optimal location and sizing of distributed energy resources (DERs) on the identified distribution feeders. The systems considered as part of the microgrid solutions were solar photovoltaic (PV), battery energy storage systems (BESS) and distributed fossil fuel generation (DFFG). 3. Estimated the cost-benefit of the proposed DER portfolios. 4. Provided a set of final recommendations that inform decision making on how to do targeted planning analysis for microgrids that can supply energy to critical infrastructures.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Achieving American Leadership in the Grid Storage Supply Chain Factsheet

To meet growing demand for long duration energy storage, domestic manufacturing will have to increase significantly. The use of renewables is rapidly increasing, and the adaption of electric vehicles is on the rise, which will require the national grid to not only produce and deliver electricity, but also store it reliably and cost-effectively. The International Energy Agency (IEA) recently released a report showing that to reach a goal of net-zero emissions by 2050, grid storage will need to grow to almost 2,500 gigawatt hours (GWh) in less than a decade. Currently, across the globe, battery technologies provide over 30 GWh of grid storage(BloombergNEF, 2020) while pumped storage hydropower (PSH)provides 160 gigawatts (GW) of long-duration energy storage (LDES) (PSH) (U.S. Department of Energy, 2020). This fact sheet summarizes strategies to address key vulnerabilities in the grid storage supply chain, the United States. These strategies include: • Developing domestic, sustainable manufacturing and recycling capabilities along the energy storage supply chain. • Maximizing the use of domestic resources by focusing on second-life and recycling technologies. • Enabling the diversification and deployment of grid storage technologies through targeted research activities. Addressing these opportunities will have significant impacts with respect to increasing well-paying skilled domestic jobs, improving the gross domestic product (GDP), and ensuring minimal environmental and climate impacts.

Source record↗

GANISP: A GAN-Assisted Importance Splitting Probability Estimator

Designing manufacturing processes with high yield and strong reliability relies on effective methods for rare event estimation. Genealogical importance splitting reduces the variance of rare event probability estimators by iteratively selecting and replicating realizations that are headed towards a rare event. The replication step is difficult when applied to deterministic systems where the initial conditions of the offspring realizations need to be modified. Typically, a random perturbation is applied to the offspring to differentiate their trajectory from the parent realization. However, this random perturbation strategy may be effective for some systems while failing for others, preventing variance reduction in the probability estimate. This work seeks to address this limitation using a generative model such as a Generative Adversarial Network (GAN) to generate perturbations that are consistent with the attractor of the dynamical system. The proposed GAN-assisted Importance SPlitting method (GANISP) improves the variance reduction for the system targeted. An implementation of the method is available in a companion repository (https://github.com/NREL/GANISP).

generative adversarial network↗

Analysis of the electrical double layer using electrochemical X-ray photoelectron spectroscopy

The element-sensitivity of X-ray spectroscopies offers the potential to disentangle the individual chemistries of water, ions, and adsorbates at the electrode-electrolyte interface in an element-by-element manner. However, targeted experimental design is needed to establish interface-sensitive in situ X-ray spectroscopy in a realistic electrochemical environment. Here, we demonstrate how electrochemical X-ray photoelectron spectroscopy (EC-XPS) in the dip-and-pull geometry can be used to specifically probe the behavior of ions in the electrical double layer. Taking the case study of a polycrystalline Au foil in 50 mM KClO 4 electrolyte, we tracked the electrochemical response of interfacial K + cations across a broad potential range. We show how, in combination with modeling, key parameters such as the potential of zero charge (PZC), ion packing behavior, dielectric saturation, and the electrostatic potential decay in the double layer can be extracted from the data. Importantly, we also analyze how the experimental conditions and non-idealities can influence the results and put forward criteria for reliable experimentation and data analysis.

Dip-and-pull↗

Planning for Seattle at the Convergence of Resilience and Energy

Grid reliability metrics obscure important temporal, spatial, and categorical considerations for increasing energy resilience. Systemwide or feeder-level outage metrics do not identify which kinds of services are affected by outages, where, and for how long. These outage metrics indicate the impacts of outages but cannot measure the consequences to customers that could result from those outages. The consequences of power outages for surrounding community members are caused by disruptions to electricity-dependent critical services, rather than to electricity itself. Power outages can decrease a community's access to healthcare, fuel, safe indoor temperatures, and provisions like food and water. This project developed critical service access, a new consequence-focused resilience metric that quantifies the relative access to critical services provided to households by distribution infrastructure during normal conditions and major disruptions. We use a spatially granular grid analysis that facilitates targeted resilience interventions; dividing feeders into isolatable sections and combining those sections with the critical service access metric allows us to identify where energy improvements like solar-plus-storage microgrids could create the most benefits for community members by protecting access to food, fuel, health, shelter, and public safety services. We identify locations in a South Seattle study area that could be high priorities for resilience investment and summarize their potential neighborhood-scale resilience benefits. This analysis was complemented by direct feedback from study area residents collect through a survey and focus groups. Results can help utilities understand how and where long power outages can create real consequences for customers, set strategic targets based on that understanding, and measure the potential benefits of energy resilience upgrades for more informed decisions.

14 SOLAR ENERGY↗

Representativeness assessment of the pan-Arctic eddy covariance site network and optimized future enhancements

Abstract. Large changes in the Arctic carbon balance are expected as warming linked to climate change threatens to destabilize ancient permafrost carbon stocks. The eddy covariance (EC) method is an established technique to quantify net losses and gains of carbon between the biosphere and atmosphere at high spatiotemporal resolution. Over the past decades, a growing network of terrestrial EC tower sites has been established across the Arctic, but a comprehensive assessment of the network's representativeness within the heterogeneous Arctic region is still lacking. This creates additional uncertainties when integrating flux data across sites, for example when upscaling fluxes to constrain pan-Arctic carbon budgets and changes therein. This study provides an inventory of Arctic (here > = 60∘ N) EC sites, which has also been made available online (https://cosima.nceas.ucsb.edu/carbon-flux-sites/, last access: 25 January 2022). Our database currently comprises 120 EC sites, but only 83 are listed as active, and just 25 of these active sites remain operational throughout the winter. To map the representativeness of this EC network, we evaluated the similarity between environmental conditions observed at the tower locations and those within the larger Arctic study domain based on 18 bioclimatic and edaphic variables. This allows us to assess a general level of similarity between ecosystem conditions within the domain, while not necessarily reflecting changes in greenhouse gas flux rates directly. We define two metrics based on this representativeness score: one that measures whether a location is represented by an EC tower with similar characteristics (ER1) and a second for which we assess if a minimum level of representation for statistically rigorous extrapolation is met (ER4). We find that while half of the domain is represented by at least one tower, only a third has enough towers in similar locations to allow reliable extrapolation. When we consider methane measurements or year-round (including wintertime) measurements, the values drop to about 1/5 and 1/10 of the domain, respectively. With the majority of sites located in Fennoscandia and Alaska, these regions were assigned the highest level of network representativeness, while large parts of Siberia and patches of Canada were classified as underrepresented. Across the Arctic, mountainous regions were particularly poorly represented by the current EC observation network. We tested three different strategies to identify new site locations or upgrades of existing sites that optimally enhance the representativeness of the current EC network. While 15 new sites can improve the representativeness of the pan-Arctic network by 20 %, upgrading as few as 10 existing sites to capture methane fluxes or remain active during wintertime can improve their respective ER1 network coverage by 28 % to 33 %. This targeted network improvement could be shown to be clearly superior to an unguided selection of new sites, therefore leading to substantial improvements in network coverage based on relatively small investments.

54 ENVIRONMENTAL SCIENCES↗

Transportation Energy Analytics Dashboard (TEAD)

The investment decisions within the nation’s transportation sector have traditionally been prioritized and measured against safety and travel efficiency goals. Increasingly, federal, state, and local policies are requiring consideration of energy use and emissions in the design of our transportation infrastructure. Yet the processes, analytics, and knowledge to include these new metrics into transportation decisions are lacking. Although targeted studies and before and after analysis are performed from time to time, no comprehensive real-time monitoring systems provide energy and emissions to the same level as congestion and safety. To address this gap, the Center for Advanced Transportation Technology (CATT Lab) and the Maryland Transportation Institute (MTI) at the University of Maryland (UMD), in partnership with the National Renewable Energy Laboratory (NREL), have developed a Transportation Energy Analytics Dashboard, or TEAD, to raise awareness of the energy and emission impacts to the same level of observability as that of safety and mobility concerns. The overall goal of this project was to develop and demonstrate an online tool to monitor transportation energy use and emissions in real-time and to archive this data for retrospective analysis. By combining surface transportation energy and emissions evaluation capability with more traditional safety, mobility and reliability system evaluation, our goal is for TEAD to create a dynamic new paradigm in transportation systems management and performance analyses, for both real-time traffic operations, and longer-term planning for project and program investment decision-making.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Airport Ground Support Equipment Infrastructure & Logistics Electrification Assessment Tool: 2025 Data Development, Modeling and Analysis for DFW

The aviation industry is increasingly turning to modernize freight facilities by integrating electric Ground Support Equipment (eGSE) to enhance operational efficiency of freight facility moving vehicles and equipment. Airports worldwide are adopting eGSE to streamline cargo movement, reduce fuel and maintenance costs, and improve logistics coordination.1 North America, with its advanced aviation infrastructure, leads this transition, leveraging Internet of things (IoT)-enabled automation and zero emission technologies to boost reliability and reduce human errors.2 Electrification of freight facility moving vehicles and equipment boosts turnaround times, improves equipment reliability, and optimizes logistics coordination, giving operators a competitive advantage. With rising fuel price volatility and the pressure to meet stringent performance benchmarks, airports are focusing on cost-effective, scalable solutions for long-term financial and operational gains. To further accelerate electrification, airports are integrating Zero Emission Vehicles (ZEVs) into rental car fleets and deploying electric baggage carts, requiring strategic investments in charging infrastructure. 3 The shift, however, presents challenges, such as limited technical expertise, high capital costs, and complex procurement processes. By forging strategic partnerships, leveraging advanced technologies, and optimizing infrastructure investments, airports can create a resilient, future-ready ecosystem that enhances the movement of people and goods through electrification-driven efficiency. Supported by the U.S. Department of Energy (DOE) Vehicle Technologies Office (VTO), this electrification effort provides a scalable, cost-effective solution to improve airport freight operations. Through targeted investments and innovation, airports enhance efficiency, reduce costs, and meet performance benchmarks while advancing toward a resilient, electrified future.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

MOD-Amp System Design Spring: Spring 2026 – Georgetown University, SYSM–5620

High-energy lasers (HELs) play an important role in both national defense and scientific research. In defense applications, HELs are used for target detection, tracking, and engagement. In research environments, they support studies of extreme physical conditions relevant to fusion energy and plasma science. These systems depend on the amplification of light through stimulated emission of radiation, allowing optical energy to be increased to the levels required for operation. This amplification occurs when light passes through an energized gain medium that receives energy from an external optical or electrical source. To achieve the desired output, laser systems often use multiple amplification stages, including high-gain preamplifiers and lower-gain power or booster amplifiers. At Lawrence Livermore National Laboratory (LLNL) and other national laboratories, many large-aperture laser amplifier systems are aging and rely on system-specific hardware, obsolete technologies, and incomplete documentation. These legacy systems create challenges for maintenance, supportability, and long-term operation. Their lack of standardization also increases the difficulty of sustaining reliable performance over time. As this infrastructure continues to age, the likelihood of unplanned downtime grows, which can negatively affect both national security missions and scientific research programs that depend on dependable HEL capabilities. The purpose of this document is to demonstrate the application of systems engineering fundamentals and design thinking through the development of a laser amplifier case study. The proposed system concept is intended as an academic exercise and not as a finalized engineering design. As a result, the development presented in this document is incomplete and may contain technical assumptions or errors that would require further investigation before any real-world implementation.

42 ENGINEERING↗

Decarbonizing Building Thermal Systems: A How-to Guide for Heat Pump Systems and Beyond

Buildings account for a substantial portion of carbon emissions, primarily due to the widespread use of fossil fuels in heating systems. Decarbonization of heating is essential to meet climate targets and reduce the environmental impact of buildings. Heat pumps are capable of leveraging renewable energy sources and can provide heating and cooling in an energy-efficient manner. By leveraging heat pump technology, buildings can significantly reduce their carbon footprint, minimize energy consumption, and decrease their reliance on fossil fuels. The design and construction community plays a pivotal role in facilitating the transition to heat pump systems for heating and cooling. However, this transition requires specialized knowledge and expertise. This resource was developed for architects, engineers, and contractors in response to an industry need for a comprehensive technical resource that guides them through the intricacies of heat pump system design, installation, and maintenance. This resource provides detailed information on system sizing, selection of appropriate heat systems, heat sources, and integration with existing building systems. Moreover, it emphasizes best practices for ensuring operational efficiency, system longevity, and reliability. The development of this recourse was a collaborative effort between NREL/DOE Better Buildings Design and Construction Allies and ASHRAE Task Force for Building Decarbonization. This resource is composed of two complimentary portions that will be completed and released on separate time frames. The first portion will be completed and released in 2023, while the second portion will be released in 2024.

building thermal systems↗

Enabling Cybersecurity, Situational Awareness and Resilience in Distribution Grids with High Penetration of Photovoltaics (CARE-PV) (Final Report)

Since legacy distribution systems have very limited visibility beyond the substation, high penetration of PV at the grid edge presents some unique operational challenges. One approach to address these challenges is to use information from advanced metering infrastructure (AMI) and µPMUs. However, exploiting this information is impacted by a number of factors, including multi-timescale measurements, volume of data generated, communication network impairments (e.g., information loss and latency) and susceptibility to cyber-attacks. Therefore, one of the critical tasks involved in the management of a distribution grid is to develop complete situational awareness by integrating cyber-security mechanisms with state estimation strategies and leveraging this situational awareness to assure energy services at strategic locations while exploiting AMI/PV inverter/ µPMU data. This CARE-PV project addresses the fundamental challenges in situational awareness and resilience to cyber and physical vectors by exploiting the synergy between innovative modeling, estimation, data analytics, testing and validation using smart PV inverters designed at K-State and facilities at NREL. Specifically, the project involved the development, testing and validation of the following novel enabling technologies: (Thrust 1) Resilience to cyber vectors that impact data integrity was addressed via a two-level defense strategy that combines cyber intrusion detection using self-learning, cooperative smart PV inverters, and a novel moving target defense framework to combat data integrity attacks. (Thrust 2) Resilience to cyber-physical vectors that impact situational awareness by limiting data availability was addressed via novel centralized and decentralized, sparsity-based static and dynamic state estimation approaches that enhance observability even when the underlying system is unobservable. (Thrust 3) Leveraging a unique probabilistic sensitivity analysis approach accompanied by one-of-a-kind dominant influencer set computation, the vulnerability of critical infrastructure at strategic locations was evaluated so that proactive PV-based control strategies can be used to support operations under normal/outage scenarios. These CARE-PV project innovations were demonstrated on both small-scale IEEE and larger utility-scale testbeds (Thrust 4). Feedback from Industry Advisory Board members was used to formulate a commercialization pathway for a subset of CARE-PV technologies. These CARE-PV technologies will ultimately lead to reliable and secure, large-scale integration of renewable energy and mitigate the risk of energy disruption resulting from cyber incidents and other emerging threats within the energy environment.

14 SOLAR ENERGY↗

Department of Energy Water Power Technologies Office Cyber Response & Recovery Flipbook [Slides]

Protecting hydroelectric plants from incidents that adversely impact their cyber-physical systems presents unique challenges due to the plants’ widely dispersed geographic locations and varied configurations as well as the relative nascent nature of the cyberattacks targeting these facilities. To help hydroelectric plants better respond to and mitigate cybersecurity incidents, this Department of Energy Water Power Technologies Office Cyber Response & Recovery Flipbook is to be used at a hydroelectric plant to quickly respond to an anomalous event. In addition to this product, there are three other products meant to be distributed to a hydroelectric plant to assist in their cyber incident response and recovery. The first, a report on the processes of building this flip book based on a large set of existing guidance. The second, a handy guide of hydroelectric and cyber guidance in responding to the cyber and physical systems within a hydroelectric plant. And the third is a correlated alignment of the steps an hydroelectric plant operator would take for both a cyber incident as well as an emergency response process if the event rises to a cyber incident affecting the safe and reliable operations of a hydroelectric plant.

13 HYDRO ENERGY↗

The Los Angeles 100% Renewable Energy Study (LA100): Chapter 6. Renewable Energy Investments and Operations

The City of Los Angeles has set ambitious goals to transform its electricity supply, aiming to achieve a 100% renewable energy power system by 2045, along with aggressive electrification targets for buildings and vehicles. To reach these goals, and assess the implications for jobs, electricity rates, the environment, and environmental justice, the Los Angeles City Council passed a series of motions directing the Los Angeles Department of Water and Power (LADWP) to determine the technical feasibility and investment pathways of a 100% renewable energy portfolio standard. The Los Angeles 100% Renewable Energy Study (LA100) is a first-of-its-kind objective, rigorous, and science-based power systems analysis to determine what investments could be made to achieve these goals. The LA100 final report is presented as a collection of 12 chapters and an executive summary, each of which is available as an individual download. This chapter explores pathways to 100% renewable electricity, describing the types of generation resources added, their costs, and how the systems maintain sufficient resources to serve customer demand, including resource adequacy and transmission reliability.

100% Renewable↗

Dark Energy Survey Year 6 Results: improved mitigation of spatially varying observational systematics with masking

As photometric surveys reach unprecedented statistical precision, systematic uncertainties increasingly dominate large-scale structure probes relying on galaxy number density. Defining the final survey footprint is critical, as it excludes regions affected by artefacts or suboptimal observing conditions. For galaxy clustering, spatially varying observational systematics, such as seeing, are a leading source of bias. Template maps of contaminants are used to derive spatially dependent corrections, but extreme values may fall outside the applicability range of mitigation methods, compromising correction reliability. The complexity and accuracy of systematics modelling depend on footprint conservativeness, with aggressive masking enabling simpler, robust mitigation. We present a unified approach to define the DES Year 6 joint footprint, integrating observational systematics templates and artefact indicators that degrade mitigation performance. This removes extreme values from an initial seed footprint, leading to the final joint footprint. By evaluating the DES Year 6 lens sample MagLim++ plus plus on this footprint, we enhance the Iterative Systematics Decontamination (ISD) method, detecting non-linear systematic contamination and improving correction accuracy. While the mask's impact on clustering is less significant than systematics decontamination, it remains non-negligible, comparable to statistical uncertainties in certain w(theta) scales and redshift bins. Supporting coherent analyses of galaxy clustering and cosmic shear, the final footprint spans 4031.04 deg2, setting the basis for DES Year 6 1x2pt, 2x2pt, and 3x2pt analyses. This work highlights how targeted masking strategies optimise the balance between statistical power and systematic control in Stage-III and -IV surveys.

Rodríguez-Monroy, M. [Madrid, IFT; IJCLab, Orsay]↗

Uncertainty quantification of a deep learning fuel property prediction model

Deep learning models are being widely used in the field of combustion. Given the black-box nature of typical neural network based models, uncertainty quantification (UQ) is critical to ensure the reliability of predictions as well as the training datasets, and for a principled quantification of noise and its various sources. Deep learning surrogate models for predicting properties of chemical compounds and mixtures have been recently shown to be promising for enabling data-driven fuel design and optimization, with the ultimate goal of improving efficiency and lowering emissions from combustion engines. In this study, UQ is performed for a multi-task deep learning model that simultaneously predicts the research octane number (RON), Motor Octane Number (MON), and Yield Sooting Index (YSI) of pure components and multicomponent blends. The deep learning model is comprised of three smaller networks: Extractor 1, Extractor 2, and Predictor, and a mixing operator. The molecular fingerprints of individual components are encoded via Extractor 1 and Extractor 2, the mixing operator generates fingerprints for mixtures/blends based on linear mixing operation, and the predictor maps the fingerprint to the target properties. Two different classes of UQ methods, Monte Carlo ensemble methods and Bayesian neural networks (BNNs), are employed for quantifying the epistemic uncertainty. Combinations of Bernoulli and Gaussian distributions with DropConnect and DropOut techniques are explored as ensemble methods. All the DropConnect, DropOut and Bayesian layers are applied to the predictor network. Aleatoric uncertainty is modeled by assuming that each data point has an independent uncertainty associated with it. The results of the UQ study are further analyzed to compare the performance of BNN and ensemble methods. Although this study is confined to UQ of fuel property prediction, the methodologies are applicable to other deep learning frameworks that are being widely used in the combustion community.

33 ADVANCED PROPULSION SYSTEMS↗

Predicting molecular dipole moments by combining atomic partial charges and atomic dipoles

The molecular dipole moment ( μ ) is a central quantity in chemistry. It is essential in predicting infrared and sum-frequency generation spectra as well as induction and long-range electrostatic interactions. Furthermore, it can be extracted directly—via the ground state electron density—from high-level quantum mechanical calculations, making it an ideal target for machine learning (ML). Here, we choose to represent this quantity with a physically inspired ML model that captures two distinct physical effects: local atomic polarization is captured within the symmetry-adapted Gaussian process regression framework which assigns a (vector) dipole moment to each atom, while the movement of charge across the entire molecule is captured by assigning a partial (scalar) charge to each atom. The resulting “MuML” models are fitted together to reproduce molecular μ computed using high-level coupled-cluster theory and density functional theory (DFT) on the QM7b dataset, achieving more accurate results due to the physics-based combination of these complementary terms. The combined model shows excellent transferability when applied to a showcase dataset of larger and more complex molecules, approaching the accuracy of DFT at a small fraction of the computational cost. We also demonstrate that the uncertainty in the predictions can be estimated reliably using a calibrated committee model. The ultimate performance of the models—and the optimal weighting of their combination—depends, however, on the details of the system at hand, with the scalar model being clearly superior when describing large molecules whose dipole is almost entirely generated by charge separation. These observations point to the importance of simultaneously accounting for the local and non-local effects that contribute to μ ; furthermore, they define a challenging task to benchmark future models, particularly those aimed at the description of condensed phases.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Effects of non-equilibrium ionization and excitation on radiation absorption in plasma plumes induced by ablation of metal targets with nanosecond laser pulses

Ionization and radiation absorption in nanosecond laser-induced plasma plumes are routinely modeled using the Saha–Boltzmann equilibrium ionization model (EQM). However, the equilibrium assumption can be inaccurate during the laser pulse when non-equilibrium effects significantly impact radiation absorption. In the present work, the EQM and non-equilibrium collisional-radiative model (CRM) are compared to reveal the effect of plasma non-equilibrium on radiation absorption in non-homogeneous plumes and degree of plasma shielding. Simulations of plume expansion induced by irradiation of a copper target in 1 atm argon background gas with a 10 ns Gaussian pulse at a fluence from 8 Jcm −2 to 14 Jcm −2 are performed with a hybrid computational model that couples a lumped particle direct simulation Monte Carlo method with either CRM or EQM. The simulations show that the EQM strongly underestimates the effects of ionization and radiation absorption compared to CRM and, contrary to the CRM, predict strong ionization of the background gas. The differences between the models are explained by the qualitatively different coupling between plume expansion and dynamics of ionization and exitation processes in the CRM and EQM under conditions when the characteristic times for most radiation- and electron-induced processes are longer than the pulse duration. The CRM-based predictions are also found to agree much better with available experimental data. In conclusion, these results indicate that the model of Saha–Boltzmann equilibrium cannot be used for reliable prediction of the degree of plasma shielding in plumes induced by nanosecond laser pulses or for processing results of spectroscopic measurements at early stages of expansion of such plumes.

Physics - Plasma physics↗

An improved dataset for predicting mammal infecting viruses from genetic sequence information

There have been several attempts to develop machine learning (ML) models to identify human infecting viruses from their genomic sequences, with varying degrees of success. Direct comparison between models is problematic, because these models are typically trained and evaluated on different datasets with alternative data splitting schemes, features, and model performance metrics. In this paper we present a standardized dataset of mammal infecting and non-infecting viral pathogens, refined from the previous work of Mollentze et al. to include the latest literature evidence, roughly doubling the number of curated host-virus records available to the community, and new host target labels, primate and mammal. The new host labels were included for several reasons, including previous reports that classification performance is better at broader taxonomic ranks and the idea that there may be more data for primate infection that might serve as a suitable proxy for zoonotic potential and avoidance of false positives for human infection due to absence of evidence. On this dataset, we report the performance of eight machine learning models for predicting mammal-infecting viruses from their genomic sequences. We find that randomly assigning cases in our improved dataset to training/testing sets, when compared to the original assignments into training/testing in Mollentze et al., increases the overall average ROC AUC of prediction of human infection from 0.663 ± 0.070 to 0.784 ± 0.013, consistent with the reduction in phylogenetic distance between train and test sets (relative entropy change from 3.00 to 0.08). The broadest host category of mammal infection can be predicted most reliably at 0.850 ± 0.020. We share our improved dataset and code to enable standardized comparisons of machine learning methods to predict human host infections. Overall, we have presented preliminary evidence that classification of virus host infection is more tractable at higher taxonomic ranks, that unsurprisingly reducing the phylogenetic distance between training and test sets can improve predictive performance, that peptide kmer features appear to be harmful to out of sample model performance, and we are left with the question of whether models for virus host prediction can reasonably be expected to perform well in out of sample scenarios given the likelihood that viruses do not share a common ancestor. Consistent with this concern, when the data is resampled such that there is no overlap between viral families in training and test sets (relative entropy > 24), models perform no better than random chance at prediction of human infection regardless of whether kmers are included (ROC AUC 0.50 ± 0.08) or not (ROC AUC 0.50 ± 0.04).

59 BASIC BIOLOGICAL SCIENCES↗