Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “variational analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,369 records · Page 76
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Synchrotron‐source micro‐x‐ray computed tomography for examining butterfly eyes

Comparative anatomy is an important tool for investigating evolutionary relationships among species, but the lack of scalable imaging tools and stains for rapidly mapping the microscale anatomies of related species poses a major impediment to using comparative anatomy approaches for identifying evolutionary adaptations. We describe a method using synchrotron source micro-x-ray computed tomography (syn-μXCT) combined with machine learning algorithms for high-throughput imaging of Lepidoptera (i.e., butterfly and moth) eyes. Our pipeline allows for imaging at rates of ~15 min/mm 3 at 600 nm 3 resolution. Image contrast is generated using standard electron microscopy labeling approaches (e.g., osmium tetroxide) that unbiasedly labels all cellular membranes in a species-independent manner thus removing any barrier to imaging any species of interest. To demonstrate the power of the method, we analyzed the 3D morphologies of butterfly crystalline cones, a part of the visual system associated with acuity and sensitivity and found significant variation within six butterfly individuals. Despite this variation, a classic measure of optimization, the ratio of interommatidial angle to resolving power of ommatidia, largely agrees with early work on eye geometry across species. We show that this method can successfully be used to determine compound eye organization and crystalline cone morphology. Our novel pipeline provides for fast, scalable visualization and analysis of eye anatomies that can be applied to any arthropod species, enabling new questions about evolutionary adaptations of compound eyes and beyond.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

A propagation-based fault detection and discrimination method and the optimization of sensor deployment

Industrial processes can be affected by faults having a serious impact on operation when not promptly detected and diagnosed. Here in this paper, a propagation-based fault detection and discrimination(PFDD) method is proposed to develop a strategy for fault diagnosis while in the design phase of a system. The PFDD method constructs the system model using the Integrated System Fault Analysis(ISFA) technique. Based on the system model, the propagation of hardware and software faults are simulated qualitatively. Given the results of the simulation, the process by which a fault propagates can be characterized using the qualitative features of system variables including the deviation of the system variables from their expected values, the variation of the system variables over time, and the order in which each variable is influenced during the propagation of the fault. The strategy by which a fault can be detected and discriminated is defined using those features. The PFDD method supports the detection and discrimination of faults in both steady states and transient states. Based on the PFDD method, the optimization of sensor deployment in a system is discussed. A brute force algorithm is developed to examine the system’s capability at diagnosing faults and the cost of sensor deployment for all possible configurations of sensors. The optimal sensor deployment strategy can be derived accordingly. However, the brute force method is only applicable to small-scale systems due to its high computational cost. A genetic algorithm is used to optimize sensor deployment in large-scale systems. The PFDD and sensor deployment optimization methods are applied to the Experimental Breeder Reactor II (EBR-II) for verification.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Monitoring of mechanical changes in a pipe assembly with complex geometry using multi-mode acoustic signals

Guided wave ultrasonic testing has been widely used for nondestructive and long-range evaluation of diverse mechanical structures, such as pipes and plates. In general, a large number of guided wave modes exist even in the simplest geometries, and they exhibit highly dispersive characteristics. Thus, sophisticated excitation methods and elaborate modal analyses need to be implemented to excite, acquire, and process well-defined pure modes as part of such a measurement method. If the interrogated structure has complex geometry, for example, with cross-sectional variations or multiple joints, the utilization of pure modes becomes significantly more difficult. To overcome this difficulty, this paper proposes the use of multi-mode, multi-frequency guided waves for long-range inspection of structures with arbitrarily complex geometries. Furthermore, using broadband chirp signals and multi-mode time–frequency analysis in a prototypical complex pipe assembly that includes multiple pipe diameters, elbows, and flange connections, we experimentally show sensitive detection of mechanical changes (both material addition and loss). In addition, we demonstrate that the complexity of the structure can be modeled in the digital domain with sufficient detail to allow accurate numerical simulation of the acoustic response using this method.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Investigation of 3D printed lightweight hybrid composites via theoretical modeling and machine learning

Hybrid composites combine two or more different fillers to achieve multifunctional or advanced material properties, such as lightweight and enhanced mechanical properties. The properties of the composites significantly depend on their microstructures, which can be tailored via advanced 3D printing processes. Understanding the process-structure-property relationships is critical to enable the design and engineering of novel hybrid composites for applications in aerospace, automotive, and protective coatings. Here, for this work, we develop 3D printable and lightweight hybrid composites and leverage the conventional design of experiments, a theoretical hybrid model, and an image-driven machine learning (ML) method to investigate their mechanical behaviors. The hybrid composites are formulated with elastomer matrix, microfillers, and thin-shell particles, enabling a significant degree of design freedom of microstructures with densities and mechanical properties varying up to 70% and 91%, respectively. Our statistical analysis indicates that the 3D printing path direction and the microfibers fraction are dominating process parameters with contribution percentages of 45.3% and 57.7% on the specific stiffness and strength, respectively. A hybrid mechanics model is developed based on a simple Weibull distribution function and classical single-filler models to effectively capture the variations in mechanical properties, however, it overestimates the values due to its statistical constraints and idealization of experimental uncertainty. The image-driven ML model leverages the microscale images directly without losing the structural details, shows more accurate predictions with experimental data, and has 48.6% lower root mean square error than the theoretical model.

3D printing↗
DOE OSTI2025

Nonlinearity of the post-spinel transition and its expression in slabs and plumes worldwide

Phase transitions in the mantle control its internal dynamics and structure. The post-spinel transition marks the upper–lower mantle boundary, where ringwoodite dissociates into bridgmanite plus ferropericlase, and its Clapeyron slope regulates mantle flow across it. This interaction has previously been assumed to have no lateral spatial variations, based on the assumption of a linear post-spinel boundary in pressure and temperature. Here we present laser-heated diamond anvil cell experiments with synchrotron X-ray diffraction to better constrain this boundary, especially at higher temperatures. Combining our data with results from the literature, and using a global analysis based on machine learning, we find a pronounced nonlinearity in the post-spinel boundary, with its slope ranging from –4 MPa/K at 2100 K, to –2 MPa/K at 1950 K, and to 0 MPa/K at 1600 K. Changes in temperature over time and space can therefore cause the post-spinel transition to have variable effects on mantle convection and the movement of subducting slabs and upwelling plumes.

58 GEOSCIENCES↗
DOE OSTI2021

Forest stand and canopy development unaltered by 12 years of CO2 enrichment*

Abstract Canopy structure—the size and distribution of tree crowns and the spatial and temporal distribution of leaves within them—exerts dominant control over primary productivity, transpiration and energy exchange. Stand structure—the spatial arrangement of trees in the forest (height, basal area and spacing)—has a strong influence on forest growth, allocation and resource use. Forest response to elevated atmospheric CO2 is likely to be dependent on the canopy and stand structure. Here, we investigated elevated CO2 effects on the forest structure of a Liquidambar styraciflua L. stand in a free-air CO2 enrichment experiment, considering leaves, tree crowns, forest canopy and stand structure. During the 12-year experiment, the trees increased in height by 5 m and basal area increased by 37%. Basal area distribution among trees shifted from a relatively narrow distribution to a much broader one, but there was little evidence of a CO2 effect on height growth or basal area distribution. The differentiation into crown classes over time led to an increase in the number of unproductive intermediate and suppressed trees and to a greater concentration of stand basal area in the largest trees. A whole-tree harvest at the end of the experiment permitted detailed analysis of canopy structure. There was little effect of CO2 enrichment on the relative leaf area distribution within tree crowns and there was little change from 1998 to 2009. Leaf characteristics (leaf mass per unit area and nitrogen content) varied with crown depth; any effects of elevated CO2 were much smaller than the variation within the crown and were consistent throughout the crown. In this young, even-aged, monoculture plantation forest, there was little evidence that elevated CO2 accelerated tree and stand development, and there were remarkably small changes in canopy structure. Questions remain as to whether a more diverse, mixed species forest would respond similarly.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Tune compensation in nearly scaling fixed field alternating gradient accelerators

In this paper, we investigate the stability of the particle trajectories in fixed field alternating gradient accelerators (FFAs) in the presence of field errors. The emphasis is on the scaling radial sector FFA type: A collaboration work is ongoing in view of better understanding the properties of the 150 MeV scaling FFA at Kyoto University Institute for Integrated Radiation and Nuclear Science in Japan and progress toward high-intensity operation. Analysis of certain types of field imperfections revealed some interesting features that required the development of an analytical model based on the scalloping angle of the orbits. This helped explain some of the experimental results as well as generalize the concept of a scaling FFA to a nonscaling one for which the tune variations obey a well-defined law. Based on this, a compensation scheme of tune variations in imperfect scaling FFAs is presented. This is the cornerstone of a novel concept of a fixed tune FFA in which the scaling is not achieved at every azimuthal position of the ring but rather in an average sense.

43 PARTICLE ACCELERATORS↗
DOE OSTI2022

SDSS-IV MaNGA: How the Stellar Populations of Passive Central Galaxies Depend on Stellar and Halo Mass

We analyze spatially resolved and co-added SDSS-IV MaNGA spectra with signal-to-noise ratio ~100 from 2200 passive central galaxies (z ~ 0.05) to understand how central galaxy assembly depends on stellar mass (M*) and halo mass (M h ). We control for systematic errors in M h by employing a new group catalog from Tinker and the widely used Yang et al. catalog. At fixed M*, the strengths of several stellar absorption features vary systematically with M h . Completely model-free, this is one of the first indications that the stellar populations of centrals with identical M* are affected by the properties of their host halos. To interpret these variations, we applied full spectral fitting with the code alf. At fixed M*, centrals in more massive halos are older, show lower [Fe/H], and have higher [Mg/Fe] with 3.5σ confidence. We conclude that halos not only dictate how much M* galaxies assemble but also modulate their chemical enrichment histories. Turning to our analysis at fixed M h , high-M* centrals are older, show lower [Fe/H], and have higher [Mg/Fe] for M h > 10 12 h –1 M⊙ with confidence >4σ. While massive passive galaxies are thought to form early and rapidly, our results are among the first to distinguish these trends at fixed M h . They suggest that high-M* centrals experienced unique early formation histories, either through enhanced collapse and gas fueling or because their halos were early forming and highly concentrated, a possible signal of galaxy assembly bias.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

Data for Spatial Analysis of Cell Patterning to Aid Genetic and Phenotypic Understanding of Grass Stomatal Density: A Case Study in Maize

Biological processes involve complex hierarchies where composite traits result from multiple component traits. However, holistically understanding of how sets of component traits interact to underpin genotype-to-phenotype relationships is generally lacking. Stomatal density (SD) is a tractable model system for exploring how high-throughput phenotyping (HTP) data could be exploited by a new spatial analysis approach to better understand a developmentally and functionally important trait. SD is a composite trait, resulting from various components related to cell identity and size, which are themselves governed by a series of spatio-developmental processes. Data from 192 recombinant inbred lines of maize [Zea mays (L.)] were analyzed by a new stomatal patterning phenotype (SPP) to (1) describe the average spatial probability distribution of the nearest neighboring stomata; (2) derive a core set of component traits related to cell size, cell packing, and positional probabilities; (3) build a structural equation model of component traits underlying SD; and (4) identify stomatal patterning quantitative trait loci (QTL). The core set of SPP-derived traits explained 74% of the variation in SD. Analyzing SPP component traits allowed some loci previously identified as generic SD QTL to be recognized as specific to lateral versus longitudinal elements of stomatal patterning. Therefore, this study highlights how novel insights can be gained by decomposing a composite trait (e.g., SD) into a set of component traits that were present in HTP data but not previously exploited.

AI/ML↗
DOE OSTI2022

Temporal variation in skin microbiota of cohabitating amphibians

Temporal changes and transmission patterns in host-associated microbial communities have important implications for host health. The diversity of amphibian skin microbial communities is associated with disease outcome in amphibians exposed to the fungal pathogen Batrachochytrium dendrobatidis (Bd). To successfully develop conservation strategies against Bd, we need a comprehensive understanding of how skin microbes are maintained and transmitted over time within populations. Here we used 16S rRNA sequence analysis to compare Epipedobates anthonyi frogs housed with one conspecific to frogs housed singly at four time points over the course of 1 year. We found that both α and β diversity of frog skin bacterial communities changed significantly over the course of the experiment. Specifically, we found that bacterial communities of cohabitating frogs became more similar over time. We also observed that some bacterial taxa were differentially abundant between frogs housed singly and frogs housed with a conspecific. These results suggest that conspecific contact may play a role in mediating amphibian skin microbial diversity and that turnover of skin microbial communities can occur across time. Our findings provide rationale for future studies exploring horizontal transmission as a potential mechanism of host-associated microbial maintenance in amphibians.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Probabilistic Analysis of Uncertainty in ATRC Flux Profiles

ATRC is a replica of the larger ATR design and is used to conduct research and obtain data such as flux measurements, excess reactivity, and loading requirements before being loaded into ATR. One method for determining the impact an experiment will have at ATR is by looking at the axial flux profile along the fuel rod in the corresponding ATRC experiment; however, flux wand measurements includes large amounts of variation which makes drawing conclusions from the data difficult. This poster describes a definitive method to propagate the uncertainty from ATRC measurements using Python code.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2021

Side-channel Leakage Assessment Metrics: A Case Study of GIFT Block Ciphers

Determination of an adequate level of security and providing subsequent mechanisms to achieve it, is one of the most pressing problems regarding embedded computing devices. While there are some solutions available for resource-rich computer systems, direct application of these solutions to resource-constrained environments are often unfeasible. The fundamental problem for such resource-constrained systems is the fact that current cryptographic algorithms utilize significant energy consumption and storage overhead. Both the cryptographic algorithm and its physical implementation affect the resilience of a cryptosystem against side-channel attacks. A side-channel attack represents a process that exploits leakages in order to extract sensitive information such as the key. This paper focuses on Correlation Power Analysis (CPA) which is side-channel attack based on the power consumption leakage. In 2016 the U.S. Commerce Department’s National Institute of Standards and Technology (NIST) initiated the call for proposals of new cryptographic algorithms to strengthen the cryptographic defense of networked devices against cyberattacks and to protect the data created by those innumerable device. This work evaluates S-boxes used by NIST candidates PICCOLO, GIFT, and PRESENT, as well as several S-box variants that demonstrated sufficient weaknesses against classical cryptanalysis, for a quantitative comparison in terms of resiliency to CPA attack. Three well-known theoretical metrics are evaluated: transparency order (TO and RTO), nonlinearity, and signal-to-noise (SNR) ratio, aiming to characterize the resistance of these S-boxes against adversaries exploiting physical leakages. Experimental results from attacks on an 8- bit XMEGA were obtained via the ChipWhisperer platform and of all the S-boxes evaluated, GIFT64 with a PICCOLO S-box was found to be the most susceptible to CPA. Results showed that variations in TO and RTO were not sufficient to ensure practical CPA resistance and that among S-boxes with equal non-linearity there were no significant differences in the TO and SNR variants.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Side-channel Leakage Assessment Metrics: A Case Study of GIFT Block Ciphers

Determination of an adequate level of security and providing subsequent mechanisms to achieve it, is one of the most pressing problems regarding embedded computing devices. While there are some solutions available for resource-rich computer systems, direct application of these solutions to resource-constrained environments are often unfeasible. The fundamental problem for such resource-constrained systems is the fact that current cryptographic algorithms utilize significant energy consumption and storage overhead. Both the cryptographic algorithm and its physical implementation affect the resilience of a cryptosystem against side-channel attacks. A side-channel attack represents a process that exploits leakages in order to extract sensitive information such as the key. This paper focuses on Correlation Power Analysis (CPA) which is side-channel attack based on the power consumption leakage. In 2016 the U.S. Commerce Department’s National Institute of Standards and Technology (NIST) initiated the call for proposals of new cryptographic algorithms to strengthen the cryptographic defense of networked devices against cyberattacks and to protect the data created by those innumerable device. This work evaluates S-boxes used by NIST candidates PICCOLO, GIFT, and PRESENT, as well as several S-box variants that demonstrated sufficient weaknesses against classical cryptanalysis, for a quantitative comparison in terms of resiliency to CPA attack. Three well-known theoretical metrics are evaluated: transparency order (TO and RTO), nonlinearity, and signal-to-noise (SNR) ratio, aiming to characterize the resistance of these S-boxes against adversaries exploiting physical leakages. Experimental results from attacks on an 8- bit XMEGA were obtained via the ChipWhisperer platform and of all the S-boxes evaluated, GIFT64 with a PICCOLO S-box was found to be the most susceptible to CPA. Results showed that variations in TO and RTO were not sufficient to ensure practical CPA resistance and that among S-boxes with equal non-linearity there were no significant differences in the TO and SNR variants.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Computational fluid dynamics analysis of char conversion in Sandia’s pressurized entrained flow reactor

Design and analysis of practical reactors utilizing solid feedstocks rely on reaction rate parameters that are typically generated in lab-scale reactors. Evaluation of the reaction rate information often relies on assumptions of uniform temperature, velocity, and species distributions in the reactor, in lieu of detailed measurements that provide local information. This assumption might be a source of substantial error, since reactor designs can impose significant inhomogeneities, leading to data misinterpretation. Spatially resolved reactor simulations help understand the key processes within the reactor and support the identification of severe variations of temperature, velocity, and species distributions. In this work, Sandia’s pressurized entrained flow reactor is modeled to identify inhomogeneities in the reaction zone. Tracer particles are tracked through the reactor to estimate the residence times and burnout ratio of introduced coal char particles in gasifying environments. The results reveal a complex mixing environment for the cool gas and particles entering the reactor along the centerline and the main high-speed hot gas reactor flow. Furthermore, the computational fluid dynamics (CFD) results show that flow asymmetries are introduced through the use of a horizontal gas pre-heating section that connects to the vertical reactor tube. Computed particle temperatures and residence times in the reactor differ substantially from the idealized plug flow conditions typically evoked in interpreting experimental measurements. Furthermore, experimental measurements and CFD analysis of heat flow through porous refractory insulation suggest that for the investigated conditions (1350 °C, <20 atm), the thermal conductivity of the insulation does not increase substantially with increasing pressure.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

Nucleon momentum fraction, helicity and transversity from 2+1-flavor lattice QCD

A detailed analysis of the systematic uncertainties in the calculation of the isovector momentum fraction, ( x ) u - d , helicity moment, ( x ) Δ u - Δ d , and the transversity moment, ( x ) δu - δd , of the nucleon is presented using high-statistics data on seven ensembles of gauge configurations generated by the JLab/W&M/LANL/MIT collaborations using 2 + 1-flavors of dynamical Wilson-clover quarks. The much higher statistics have facilitated better control over all systematics compared to previous lattice calculations. The least understood systematic — excited-state contamination — is quantified by studying the variation of the results as a function of different estimates of the mass gap of the first excited state, obtained from two- and three-point correlation functions, and as a function of the pion mass M π . The final results are obtained using a simultaneous fit in the lattice spacing a , pion mass M π and the finite volume parameter M π L keeping leading order corrections. The data show no significant dependence on the lattice spacing and some evidence for finite-volume corrections. Our final results, in the $ \overline{\mathrm{MS}} $ scheme at 2 GeV, are ( x ) u - d = 0 . 155(17)(20), ( x ) Δ u - Δ d = 0 . 183(14)(20) and ( x ) δu - δd = 0 . 220(18)(20), where the first error is the overall analysis uncertainty assuming excited-state contributions have been removed, and the second is an additional systematic uncertainty due to possible residual excited-state contributions. These results are consistent with phenomenological global fit values.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

Sensitivity of Optical Satellites to Estimate Windthrow Tree-Mortality in a Central Amazon Forest

Windthrow (i.e., trees broken and uprooted by wind) is a major natural disturbance in Amazon forests. Images from medium-resolution optical satellites combined with extensive field data have allowed researchers to assess patterns of windthrow tree-mortality and to monitor forest recovery over decades of succession in different regions. Although satellites with high spatial-resolution have become available in the last decade, they have not yet been employed for the quantification of windthrow tree-mortality. Here, we address how increasing the spatial resolution of satellites affects plot-to-landscape estimates of windthrow tree-mortality. We combined forest inventory data with Landsat 8 (30 m pixel), Sentinel 2 (10 m), and WorldView 2 (2 m) imagery over an old-growth forest in the Central Amazon that was disturbed by a single windthrow event in November 2015. Remote sensing estimates of windthrow tree-mortality were produced from Spectral Mixture Analysis and evaluated with forest inventory data (i.e., ground true) by using Generalized Linear Models. Field measured windthrow tree-mortality (3 transects and 30 subplots) crossing the entire disturbance gradient was 26.9 ± 11.1% (mean ± 95% CI). Although the three satellites produced reliable and statistically similar estimates (from 26.5% to 30.3%, p < 0.001), Landsat 8 had the most accurate results and efficiently captured field-observed variations in windthrow tree-mortality across the entire gradient of disturbance (Sentinel 2 and WorldView 2 produced the second and third best results, respectively). As expected, mean-associated uncertainties decreased systematically with increasing spatial resolution (i.e., from Landsat 8 to Sentinel 2 and WorldView 2). However, the overall quality of model fits showed the opposite pattern. We suggest that this reflects the influence of a relatively minor disturbance, such as defoliation and crown damage, and the fast growth of natural regeneration, which were not measured in the field nor can be captured by coarser resolution imagery. Our results validate the reliability of Landsat imagery for assessing plot-to-landscape patterns of windthrow tree-mortality in dense and heterogeneous tropical forests. Satellites with high spatial resolution can improve estimates of windthrow severity by allowing the quantification of crown damage and mortality of lower canopy and understory trees. However, this requires the validation of remote sensing metrics using field data at compatible scales.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Towards a Robust Adaptive Digital Twin for Fusion Applications

The development of a digital twin system for fusion applications is essential for enhancing the prediction, analysis, and optimization of complex plasma processes. Machine learning (ML), particularly deep learning has demonstrated strong capabilities in modeling such highly nonlinear and intricate systems. However, two critical challenges limit the deployment of deep learning-based digital twins: Uncertainty Quantification (UQ) and data drift. UQ is vital for ensuring trustworthy predictions, especially in decision-support scenarios. Additionally, data-driven models are often sensitive to changes in the underlying data distribution, such as shot-to-shot variations in fusion experiments, which can lead to performance degradation over time. To address these challenges, we are developing an uncertainty-aware, adaptive digital twin framework. Our approach incorporates deep learning models enhanced with Gaussian Process approximations for predictive uncertainty estimation, coupled with an online learning mechanism that enables continuous model adaptation to new experimental data. This adaptive capability allows the data driven models to respond effectively to evolving plasma behaviors and equipment conditions. Specifically, to mitigate the effects of shot-to-shot drift, our system updates itself incrementally as new data becomes available, improving both robustness and fidelity. Our vision is to evolve this data driven model into a self-sustaining digital twin system that leverages UQ based feedback to continuously refine itself and potentially support real-time decision making. This presentation will cover a brief background on uncertainty quantification for ML, our ongoing effort on development of UQ capabilities for ML, our data science pipeline from data collection to model development and analysis and online learning framework for modeling coil deflection at DIII-D. I will also briefly touch upon opportunities and challenges in development of digital twin framework.

Sammuli, Brian [General Atomics]↗
DOE OSTI2023

Evaluation of an open forecasting challenge to assess skill of West Nile virus neuroinvasive disease prediction

Abstract Background West Nile virus (WNV) is the leading cause of mosquito-borne illness in the continental USA. WNV occurrence has high spatiotemporal variation, and current approaches to targeted control of the virus are limited, making forecasting a public health priority. However, little research has been done to compare strengths and weaknesses of WNV disease forecasting approaches on the national scale. We used forecasts submitted to the 2020 WNV Forecasting Challenge, an open challenge organized by the Centers for Disease Control and Prevention, to assess the status of WNV neuroinvasive disease (WNND) prediction and identify avenues for improvement. Methods We performed a multi-model comparative assessment of probabilistic forecasts submitted by 15 teams for annual WNND cases in US counties for 2020 and assessed forecast accuracy, calibration, and discriminatory power. In the evaluation, we included forecasts produced by comparison models of varying complexity as benchmarks of forecast performance. We also used regression analysis to identify modeling approaches and contextual factors that were associated with forecast skill. Results Simple models based on historical WNND cases generally scored better than more complex models and combined higher discriminatory power with better calibration of uncertainty. Forecast skill improved across updated forecast submissions submitted during the 2020 season. Among models using additional data, inclusion of climate or human demographic data was associated with higher skill, while inclusion of mosquito or land use data was associated with lower skill. We also identified population size, extreme minimum winter temperature, and interannual variation in WNND cases as county-level characteristics associated with variation in forecast skill. Conclusions Historical WNND cases were strong predictors of future cases with minimal increase in skill achieved by models that included other factors. Although opportunities might exist to specifically improve predictions for areas with large populations and low or high winter temperatures, areas with high case-count variability are intrinsically more difficult to predict. Also, the prediction of outbreaks, which are outliers relative to typical case numbers, remains difficult. Further improvements to prediction could be obtained with improved calibration of forecast uncertainty and access to real-time data streams (e.g. current weather and preliminary human cases). Graphical Abstract

54 ENVIRONMENTAL SCIENCES↗