Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “ensemble integration”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10

3D diffractive imaging of nanoparticle ensembles using an x-ray laser

Single particle imaging at x-ray free electron lasers (XFELs) has the potential to determine the structure and dynamics of single biomolecules at room temperature. Two major hurdles have prevented this potential from being reached, namely, the collection of sufficient high-quality diffraction patterns and robust computational purification to overcome structural heterogeneity. We report the breaking of both of these barriers using gold nanoparticle test samples, recording around 10 million diffraction patterns at the European XFEL and structurally and orientationally sorting the patterns to obtain better than 3-nm-resolution 3D reconstructions for each of four samples. With these new developments, integrating advancements in x-ray sources, fast-framing detectors, efficient sample delivery, and data analysis algorithms, we illuminate the path towards sub-nanometer biomolecular imaging. The methods developed here can also be extended to characterize ensembles that are inherently diverse to obtain their full structural landscape.

47 OTHER INSTRUMENTATION↗

Synthetic Atmospheric River Ensembles Generated by Deep-AR

This dataset contains 35,850 synthetic landfalling atmospheric river (AR) realizations generated by the Deep-AR two-stage deep-learning framework over the Northeast Pacific and U.S. West Coast. The archive contains 25 stochastic ensemble members for each of 1,434 held-out observed seed events. Each synthetic realization is initialized from conditions 48 hours before the corresponding observed AR landfall and is generated autoregressively at 6-hour intervals over a 144-hour period. Deep-AR combines a deterministic residual network (ResNet) that advances the large-scale atmospheric state with a Wasserstein generative adversarial network (WGAN) that produces stochastic, high-resolution fields. Each HDF5 file contains 0.25° gridded synthetic integrated vapor transport components (qu, qv), 10 m wind components (u10, v10), and 6-hour accumulated precipitation on a common 200 × 480 grid. The files also include coordinate and datetime arrays. This dataset supports AR hazard analysis, ensemble-based uncertainty characterization, precipitation-extremes research, and regional stress testing. Synthetic files follow the naming convention deepar.model.YYYYMMDD.HHMMSS.vNN.h5. YYYYMMDD.HHMMSS identifies the UTC initial-condition timestamp, which occurs 48 hours before the diagnosed observed landfall, and vNN identifies the zero-padded ensemble member, ranging from v01 through v25. Each synthetic file can be paired with its corresponding observed file by matching the initial-condition timestamp. The paired observed file follows the naming convention deepar.obs.YYYYMMDD.HHMMSS.h5 and is available in the separately registered oracle/deepar.obs dataset at https://wdh.energy.gov/ds/oracle/deepar.obs (DOI: https://doi.org/10.21947/3377671).

17 WIND ENERGY↗

OSW Consortium 2 - Validated National Offshore Wind Resource Dataset with Uncertainty Quantification (CRADA Report)

This research has led to the development of the 2023 National Offshore Wind data set (NOW-23), which offers the latest wind resource information for offshore regions in the United States. NOW-23 supersedes, for its offshore component, the Wind Integration National Dataset (WIND) Toolkit, which was published a decade ago and is currently a primary resource for wind resource assessments and grid integration studies in the contiguous United States. By incorporating advancements in the Weather Research and Forecasting (WRF) model, NOW-23 delivers an updated and cutting-edge product to stakeholders. As part of this project, we also developed a summary of the uncertainty quantification in NOW-23, along with NOW-WAKES, a 1-year post-construction data set that quantifies expected offshore wake effects in the US Mid-Atlantic lease areas. Stakeholders can access the NOW-23 data set at https://doi.org/10.25984/1821404.

17 WIND ENERGY↗

Integrated Modeling Driven Evaluation of Opportunities for Climate‐Resilient Perennial Biomass Crop Plantings in Flood‐Prone Agricultural Landscapes

Adapting to future climate change in flood-prone landscapes will require climate-resilient agricultural systems. Planting perennial crops, like switchgrass and willow, along river corridors can mitigate future flooding while supporting bioenergy markets. We developed an integrated assessment linking climate, hydrologic, and inundation model results to assess future flood risk to river-adjacent agricultural lands in the Mid-Atlantic Region (MAR) and explore this opportunity. We produced ensemble streamflow projections for every MAR stream using a hydrologic model driven by a suite of downscaled and bias-corrected Coupled Model Intercomparison Project Phase 6 climate projections. We then conducted high-resolution inundation mapping based on projected flood frequencies for baseline and future periods. Results show that in the near-term future, at least two-thirds of the streams will experience 100-year floods more severe than the baseline 200-year floods. Riparian zones are projected to face a median rise of inundation by 9.5%–24.1%. Results show that there is an opportunity to mitigate flooding in over half of MAR's counties with the quantities of switchgrass and willow plantings anticipated for mature bioenergy markets, even under the most extreme (200-year) flood events. Our integrated modeling framework can guide similar regions to evaluate opportunities for flood-resilient agricultural systems under climate change.

60 APPLIED LIFE SCIENCES↗

Estimating irrigation water use from remotely sensed evapotranspiration data: Accuracy and uncertainties at field, water right, and regional scales

Irrigated agriculture is the dominant user of water globally, but most water withdrawals are not monitored or reported. As a result, it is largely unknown when, where, and how much water is used for irrigation. Here, we evaluated the ability of remotely sensed evapotranspiration (ET) data, integrated with other datasets, to calculate irrigation water withdrawals and applications in an intensively irrigated portion of the United States. We compared irrigation calculations based on an ensemble of satellite-driven ET models from OpenET with reported groundwater withdrawals from hundreds of farmer irrigation application records and a statewide flowmeter database at three spatial scales (field, water right group, and management area). At the field scale, we found that ET-based calculations of irrigation agreed best with reported irrigation when the OpenET ensemble mean was aggregated to the growing season timescale (bias = 1.6–4.9%, R 2 = 0.53–0.74), and agreement between calculated and reported irrigation was better for multi-year averages than for individual years. At the water right group scale, linking pumping wells to specific irrigated fields was the primary source of uncertainty. At the management area scale, calculated irrigation exhibited similar temporal patterns as flowmeter data but tended to be positively biased with more interannual variability. Disagreement between calculated and reported irrigation was strongly correlated with annual precipitation, and calculated and reported irrigation agreed more closely after statistically adjusting for annual precipitation. The selection of an ET model was also an important consideration, as variability across ET models was larger than the potential impacts of conservation measures employed in the region. From these results, we suggest key practices for working with ET-based irrigation data that include accurately accounting for changes in soil moisture, deep percolation, and runoff; careful verification of irrigated area and well-field linkages; and conducting application-specific evaluations of uncertainty.

59 BASIC BIOLOGICAL SCIENCES↗

Sampling lattices in semi-grand canonical ensemble with autoregressive machine learning

Calculating thermodynamic potentials and observables efficiently and accurately is key for the application of statistical mechanics simulations to materials science. However, naive Monte Carlo approaches, on which such calculations are often dependent, struggle to scale to complex materials in many state-of-the-art disciplines such as the design of high entropy alloys or multi-component catalysts. To address this issue, we adapt sampling tools built upon machine learning-based generative modeling to the materials space by transforming them into the semi-grand canonical ensemble. Furthermore, we show that the resulting models are transferable across wide ranges of thermodynamic conditions and can be implemented with any internal energy model U, allowing integration into many existing materials workflows. We demonstrate the applicability of this approach to the simulation of benchmark systems (AgPd, CuAu) that exhibit diverse thermodynamic behavior in their phase diagrams. Finally, we discuss remaining challenges in model development and promising research directions for future improvements.

36 MATERIALS SCIENCE↗

Scalable Generation of High-fidelity Synthetic Population Ensembles

Used within social simulations, synthetic population ensembles enable uncertainty quantification (UQ) methods for obtaining more robust model inference and prediction. A synthetic population ensemble is a series of plausible virtual reconstructions of an area’s population at the granularity of people and residences, generated stochastically to preserve privacy of the source population survey’s respondents. In this paper, we demonstrate the production of large synthetic population ensembles for the U.S. via Oak Ridge National Laboratory’s UrbanPop framework to support modeling of high spatial resolution energy affordability metrics from nationwide social surveys in collaboration with the fusionACS project. The study involves two scenarios: creating ensembles for (1) 17 U.S. metropolitan areas in 2019 and (2) full U.S. Census Divisions in 2023, with each scenario consisting of 41 population instances (a base realization and 40 replicates). To accomplish this task at scale, we configured an integrated system within a research cloud, comprised of virtual containerizations, GPU-enhanced functionality, and orchestrated deployments of UrbanPop’s maturing Likeness Python ecosystem. Results demonstrate we maintained high-fidelity approximations of residential totals by areas of interest and the demographic characteristics of neighborhoods while reducing manual workflow burdens. Finally, we discuss plans to fine-tune and further develop our automated workflows for truly distributed job orchestration to increase computational efficiency, as well as provide an outlook for broadening applications of the ensembles.

Cluster computing↗

Integrating ytopt and libEnsemble to autotune OpenMC

Ytopt is a Python machine-learning-based autotuning software package developed within the ECP PROTEAS-TUNE project. The ytopt software adopts an asynchronous search framework that consists of sampling a small number of input parameter configurations and progressively fitting a surrogate model over the input-output space until exhausting the user-defined maximum number of evaluations or the wall-clock time. libEnsemble is a Python toolkit for coordinating workflows of asynchronous and dynamic ensembles of calculations across massively parallel resources developed within the ECP PETSc/TAO project. libEnsemble helps users take advantage of massively parallel resources to solve design, decision, and inference problems and expands the class of problems that can benefit from increased parallelism. In this paper we present our methodology and framework to integrate ytopt and libEnsemble to take advantage of massively parallel resources to accelerate the autotuning process. Specifically, we focus on using the proposed framework to autotune the ECP ExaSMR application OpenMC, an open source Monte Carlo particle transport code. OpenMC has seven tunable parameters some of which have large ranges such as the number of particles in-flight, which is in the range of 100,000 to 8 million, with its default setting of 1 million. Setting the proper combination of these parameter values to achieve the best performance is extremely time-consuming. Therefore, we apply the proposed framework to autotune the MPI/OpenMP offload version of OpenMC based on a user-defined metric such as the figure of merit (FoM) (particles/s) or energy efficiency energy-delay product (EDP) on Crusher at Oak Ridge Leadership Computing Facility. In conclusion, the experimental results show that we achieve the improvement up to 29.49% in FoM and up to 30.44% in EDP.

Autotuning↗

Nonlinear Thomson scattering of a tightly focused relativistically intense laser pulse by an ensemble of particles

We report a study of the process of nonlinear Thomson scattering (NTS) of a tightly focused relativistically intense laser pulse by an ensemble of electrons simulating the result of ionisation of an ultrathin nanofoil. The description of all six components of a laser pulse is based on the Stratton – Chu integrals describing radiation focused by an off-axis parabolic mirror into a spot down to the diffraction limit, which is of practical interest for modern experiments with extreme-intensity lasers, including diagnostics of their parameters. The interaction of a laser pulse with electrons is simulated by the test-particle method, and the resulting trajectories are used to calculate the characteristics of secondary electromagnetic radiation. The effect of the laser-pulse focal spot diameter in the range λ – 6λ on the angular distributions of the radiated energy and NTS spectra, as well as of a laser-beam peak intensity of 10{sup 21} – 10{sup 22} W cm{sup −2} on the spectral width of the secondary radiation is studied in relation to the experimentally implemented focusing scheme using the example of a laser pulse with an initial homogeneous spatial distribution and a duration of ∼25 fs. (interaction of laser radiation with matter. laser plasma)

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Isotope effects in molecular structures and electronic properties of liquid water via deep potential molecular dynamics based on the SCAN functional

Here, Feynman path-integral deep potential molecular dynamics (PI-DPMD) calculations have been employed to study both light (H 2 O) and heavy water (D 2 O) within the isothermalisobaric ensemble. In particular, the deep neural network is trained based on ab initio data obtained from the strongly constrained and appropriately normed (SCAN) exchange-correlation functional. Because of the lighter mass of hydrogen than deuteron, the properties of light water are more influenced by nuclear quantum effect than those of heavy water. Clear isotope effects are observed and analyzed in terms of hydrogen-bond structure and electronic properties of water that are closely associated with experimental observables. The molecular structures of both liquid H 2 O and D 2 O agree well with the data extracted from scattering experiments. The delicate isotope effects on radial distribution functions and angular distribution functions are well reproduced as well. Our approach demonstrates that deep neural network combined with SCAN functional based ab initio molecular dynamics provides an accurate theoretical tool for modeling water and its isotope effects.

36 MATERIALS SCIENCE↗

Sensitivity Study for Forecasting Variables of WRF-Solar Using a Tangent Linear Approach

Integrating solar generation in recent years has highlighted the need for improved accuracy in predicting solar power. Confidence in solar power forecasting can be achieved by designing an ensemble that provides reliable probabilistic information for solar radiation with reduced uncertainty and error. Ideally, ensemble members are created through the optimized perturbation of the initial conditions in numerical weather prediction (NWP) models. Tangent linear models are capable of efficiently investigating the sensitivity of solar radiation to model input parameters because they do not require individual perturbation of each variable. This sensitivity study using tangent linear models provide us the capability to identify the right variables to perturb in an ensemble prediction system. In this study, we developed tangent linear models for WRF-Solar modules that directly impact the computation of solar radiation and the simulation of cloud formation and dissipation including the Fast All-sky Model for Solar Applications (FARMS), the Noah land surface model (LSM), the Thompson microphysics, the Mello-Yamada-Nakanishi-Niino (MYNN) boundary layer parameterization, and the Deng scheme for a shallow-convection parameterization. A sensitivity analysis was conducted under various scenarios based on satellite observations and model simulations from the National Solar Radiation Data Base (NSRDB) and WRF-Solar, respectively. Critical forecasting variables that are highly sensitive to the forecasting of global horizontal irradiance (GHI), direct normal irradiance (DNI), cloud mixing ratio, cloud tendency, cloud fraction, and sensible and latent heat fluxes were determined using the relevant WRF-Solar module. This study will be used as a guidance on future research leading to high-quality probabilistic solar forecasting. In this presentation, we discuss the validation of tangent linear approach for WRF-Solar modules and illustrate how the sensitivity results are valuable in the improvement of probabilistic solar prediction.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Imaging of a van der Waals spin-orbit torque system using spin ensembles in hBN

Recently, optically active spin defects embedded in two-dimensional (2D) van der Waals (vdW) crystals have emerged as a transformative quantum sensing platform to explore cutting-edge materials science. Taking advantage of excellent solid-state integrability, this new class of spin defects can be readily arranged in nanoscale proximity to target materials, showing great promise for realizing in-situ quantum sensing of microscopic spin and charge behaviors in vdW heterostructures. Here we report hexagonal boron nitride-based quantum imaging of field-free deterministic magnetic switching and electric current distributions in an all-vdW spin-orbit torque (SOT) system. By visualizing variations of nanoscale magnetic stray field profile of room-temperature 2D magnet Fe 3 GaTe 2 under different SOT conditions, we show how the magnetic switching evolves from deterministic to stochastic behavior due to the interplay between spin orientations, anisotropy and Joule heating. Micromagnetic simulations rationalize our results well, revealing the role of field-like SOT in inhibiting thermal fluctuation driven stochastic switching and chaotic multi-domain competition. This understanding, which is otherwise difficult to access by conventional transport measurements, offers valuable insights into material design, testing, and performance evaluation of next-generation vdW spintronic devices.

Imaging techniques↗

Controlled Spalling of 4H Silicon Carbide with Investigated Spin Coherence for Quantum Engineering Integration

We detail scientific and engineering advances which enable the controlled spalling and layer transfer of single crystal 4H silicon carbide (4H-SiC) from bulk substrates. 4HSiC’s properties, including high thermal conductivity and a wide bandgap, make it an ideal semiconductor for power electronics. Moreover, 4H-SiC is an excellent host of solid-state atomic defect qubits for quantum computing and quantum networking. Because 4H-SiC substrates are expensive (due to long growth times and limited yield), techniques for removal and transfer of bulk-quality films are desirable for substrate reuse and integration of the separated films. In this work, we utilize updated approaches for stressor layer thickness control and spalling crack initiation to demonstrate controlled spalling of 4H-SiC, the highest fracture toughness crystal spalled to date. We achieve coherent spin control of neutral divacancy (VV 0 ) qubit ensembles and measure a quasi-bulk spin T 2 of 79.7 μs in the spalled films.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

STITCHES: a Python package to amalgamate existing Earth system model output into new scenario realizations

Understanding the interaction between humans and the Earth system is a computationally daunting task, with many possible approaches depending on resources available and questions of interest. For example, state-of-the-art impact models require decade-long time series of relatively high frequency, spatially resolved and often multiple variables representing climatic impact-drivers (Ruane et al., 2022). Most commonly these are derived from the outputs of detailed, computationally expensive Earth System Models (ESMs) run according to a standard, limited set of future scenarios, the latest being the SSP-RCPs run under CMIP6/ScenarioMIP (Eyring et al., 2016; O’Neill et al., 2016). At the time of writing, O’Neill et al. (2016) has been cited more than 1750 times and Eyring et al. (2016) more than 5000 times, highlighting the broad, general applications of this data. Often, however, impact modeling seeks to explore new scenarios that were not part of the ScenarioMIP protocol, and/or needs a larger set of initial condition ensemble members than are typically available to quantify the effects of ESM internal variability. In addition, the recognition that the human and Earth systems are fundamentally intertwined, and may feature potentially significant feedback loops, is making integrated, simultaneous modeling of the coupled human-Earth system increasingly necessary, if computationally challenging with most existing tools (Thornton et al., 2017). For impact modelers, climate model emulators can be the answer to meet both the needs of: 1) creating realizations for novel scenarios and 2) achieving a simplified, computationally tractable representation of ESM behavior in a coupled human-Earth system modeling framework. We proposed a new, comprehensive approach to such emulation of gridded, multivariate ESM outputs for novel scenarios without the computational cost of a full ESM, STITCHES (Tebaldi et al., 2022). The approach outlined in Tebaldi et al. (2022) should be extensible to future CMIP eras, although the STITCHES software at present is strictly focused on CMIP6/ScenarioMIP data hosted on Pangeo (https://gallery.pangeo.io/repos/pangeo-gallery/cmip6/). The corresponding STITCHES Python package uses existing archives of ESMs’ scenario experiments from CMIP6/ScenarioMIP to construct gridded, multivariate realizations of new scenarios provided by reduced complexity climate models (Hartin et al., 2015; Meinshausen et al., 2011; Smith et al., 2018), or to enrich existing initial condition ensembles. Its output provides the same characteristics as the emulated ESM output: multivariate (spanning potentially all variables that the ESM has saved), spatially resolved (down to the native grid of the ESM), and preserving the same high frequency as the original data. A new realization of multiple variables can be generated on the order of minutes with STITCHES, rather than the hours or sometimes days that ESMs require.

97 MATHEMATICS AND COMPUTING↗

Differentiable Multiphysics Codes: A Breakthrough Technology for Simulation and Computing

This document summarizes the findings of a strategic planning exercise commissioned by the Weapons Simulation and Computing, Computational Physics (WSC/CP) program at the Lawrence Livermore National Laboratory (LLNL) in FY24. During the year, the committee met with multiple stakeholder communities to gather input, opinions, suggestions and concerns which have been incorporated throughout this document. The key findings from this exercise are summarized: • The development of multiphysics modelling and simulation (mod/sim) codes and software technologies, their deployment on exascale compute platforms, and their broad adoption across the NNSA is a major success of the Advanced Simulation and Computing (ASC) program and the Exascale Computing Project (ECP). Sustained investment in these core technologies is essential. • Today’s state of the art involves running ensembles of O(100K) simulations to perform uncertainty quantification (UQ) and design studies using multiple statistical methods such as Bayesian optimization to understand sensitivities of our models and explore parameterized design spaces. Even with exascale computing, we are practically limited to O(10) parameters in these studies since the number of simulations required to sample the space scales exponentially with the number of design parameters. • The data from these simulation ensembles is increasingly being used to train machine learned (ML) surrogates (or reduced order models, ROMs) which can then be used for optimization or real time design exploration. However, the trained surrogates are still limited in the number of parameters they can represent due to the sampling limitations previously noted. • Augmenting our suite of integrated multiphysics simulation codes, both current and emerging, with the ability to compute gradients (solution derivatives) of arbitrary simulation outputs with respect to (some or all) simulation inputs would be a breakthrough technology, opening the door to a new era of efficient and automated inverse design based on verified and validated mod/sim capabilities. • This capability, which we refer to as differentiable multiphysics codes (DMCs), would revolutionize both UQ and optimization studies by breaking the curse of dimensionality that presently limits our “gradient-free” ensemble based computing approach. A similar breakthrough occurred in the AI/ML community once the ability to compute gradients of arbitrary loss functions using back-propagation became commonplace. Gradient information from the multiphysics codes can also be used to dramatically improve the efficiency and scale of training of ML/ROM surrogates for rapid assessments. • Achieving this in our suite of codes will be a grand challenge, similar to the amount of effort that was required to transition from CPU to GPU computing. It will require buy-in from the entire WSC/CP program and beyond, including all integrated codes, physics and engineering models, third-party library dependencies and performance portability abstractions. It will also require investment in research and development of numerical methods for computing adjoints of coupled physics across multiple adaptively refined moving meshes and of stochastic (Monte Carlo) and mesh free (SPH) methods. • New software and numerical techniques, largely pioneered by the AI/ML community, make this feasible. Chief among these is automatic differentiation (AD), the ability to employ AD at point-wise locations in a physics calculation (instead of traditional black-box approaches) and the ability to perform “back-propagation in time” (or reverse mode AD) for non-linear partial differential equations (PDEs). Fundamentally, the conclusion of this strategic planning exercise is that the time is right to undertake a large scale effort in WSC, centered on the existing integrated codes, to continue the natural evolution of mod/sim in the age of AI/ML. Instead of attempting to replace mod/sim with purely data driven AI/ML models, we believe the key to success is to integrate AI/ML by building on top of the decades of hard-won knowledge and the verified/validated multiphysics modelling capability that is the hallmark of the ASC program.

97 MATHEMATICS AND COMPUTING↗

Subsystem Rényi entropy of thermal ensembles for SYK-like models

The Sachdev-Ye-Kitaev model is an N N -modes fermionic model with infinite range random interactions. In this work, we study the thermal Rényi entropy for a subsystem of the SYK model using the path-integral formalism in the large- N N limit. The results are consistent with exact diagonalization and can be well approximated by thermal entropy with an effective temperature when subsystem size M\leq N/2 M ≤ N / 2 . We also consider generalizations of the SYK model with quadratic random hopping term or U(1) U ( 1 ) charge conservation.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

EFIT-Prime: Probabilistic and physics-constrained reduced-order neural network model for equilibrium reconstruction in DIII-D

We introduce EFIT-Prime, a novel machine learning surrogate model for EFIT (Equilibrium FIT) that integrates probabilistic and physics-informed methodologies to overcome typical limitations associated with deterministic and ad hoc neural network architectures. EFIT-Prime utilizes a neural architecture search-based deep ensemble for robust uncertainty quantification, providing scalable and efficient neural architectures that comprehensively quantify both data and model uncertainties. Physically informed by the Grad–Shafranov equation, EFIT-Prime applies a constraint on the current density J tor and a smoothness constraint on the first derivative of the poloidal flux, ensuring physically plausible solutions. Furthermore, the spatial location of the diagnostics is explicitly incorporated in the inputs to account for their spatial correlation. Extensive evaluations demonstrate EFIT-Prime's accuracy and robustness across diverse scenarios, most notably showing good generalization on negative-triangularity discharges that were excluded from training. Timing studies indicate an ensemble inference time of 15 ms for predicting a new equilibrium, offering the possibility of plasma control in real-time, if the model is optimized for speed.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

A Scalable Real-Time Data Assimilation Framework for Predicting Turbulent Atmosphere Dynamics

AI-based foundation models like FourCastNet, GraphCast are revolutionizing weather and climate predictions but are not yet ready for operational use. Their limitation lies in the absence of a data assimilation system to incorporate real-time Earth system observations, crucial for accurately forecasting events like tropical cyclones. To overcome these obstacles, we introduce a generic real-time data assimilation framework and demonstrate its end-to-end performance on the Frontier supercomputer. This framework comprises two primary modules: an ensemble score filter (EnSF), which significantly outperforms the state-of-the-art data assimilation method, and a vision transformer-based surrogate capable of real-time adaptation through the integration of observational data. We demonstrate both the strong and weak scaling of our framework up to 1024 GPUs on the Exascale supercomputer, Frontier. Our results not only illustrate the framework's exceptional scalability on high-performance computing systems, but also demonstrate the importance of supercomputers in real-time data assimilation for weather and climate predictions.

Lu, Dan↗