Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Ensemble models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Statistically bias-corrected and downscaled climate models underestimate the adverse effects of extreme heat on U.S. maize yields

Abstract Efforts to understand and quantify how a changing climate can impact agriculture often rely on bias-corrected and downscaled climate information, making it important to quantify potential biases of this approach. Here, we use a multi-model ensemble of statistically bias-corrected and downscaled climate models, as well as the corresponding parent models from the Coupled Model Intercomparison Project Phase 5 (CMIP5), to drive a statistical panel model of U.S. maize yields that incorporates season-wide measures of temperature and precipitation. We analyze uncertainty in annual yield hindcasts, finding that the CMIP5 models considerably overestimate historical yield variability while the bias-corrected and downscaled versions underestimate the largest weather-induced yield declines. We also find large differences in projected yields and other decision-relevant metrics throughout this century, leaving stakeholders with modeling choices that require navigating trade-offs in resolution, historical accuracy, and projection confidence.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Probabilistic projections of the Amery Ice Shelf catchment, Antarctica, under conditions of high ice-shelf basal melt

Abstract. Antarctica's Lambert Glacier drains about one-sixth of the ice from the East Antarctic Ice Sheet and is considered stable due to the strong buttressing provided by the Amery Ice Shelf. While previous projections of the sea-level contribution from this sector of the ice sheet have predicted significant mass loss only with near-complete removal of the ice shelf, the ocean warming necessary for this was deemed unlikely. Recent climate projections through 2300 indicate that sufficient ocean warming is a distinct possibility after 2100. This work explores the impact of parametric uncertainty on projections of the response of the Lambert–Amery system (hereafter “the Amery sector”) to abrupt ocean warming through Bayesian calibration of a perturbed-parameter ice-sheet model ensemble. We address the computational cost of uncertainty quantification for ice-sheet model projections via statistical emulation, which employs surrogate models for fast and inexpensive parameter space exploration while retaining critical features of the high-fidelity simulations. To this end, we build Gaussian process (GP) emulators from simulations of the Amery sector at a medium resolution (4–20 km mesh) using the Model for Prediction Across Scales (MPAS)-Albany Land Ice (MALI) model. We consider six input parameters that control basal friction, ice stiffness, calving, and ice-shelf basal melting. From these, we generate 200 perturbed input parameter initializations using space filling Sobol sampling. For our end-to-end probabilistic modeling workflow, we first train emulators on the simulation ensemble and then calibrate the input parameters using observations of the mass balance, grounding line movement, and calving front movement with priors assigned via expert knowledge. Next, we use MALI to project a subset of simulations to 2300 using ocean and atmosphere forcings from a climate model for both low- and high-greenhouse-gas-emission scenarios. From these simulation outputs, we build multivariate emulators by combining GP regression with principal component dimension reduction to emulate multivariate sea-level contribution time series data from the MALI simulations. We then use these emulators to propagate uncertainty from model input parameters to predictions of glacier mass loss through 2300, demonstrating that the calibrated posterior distributions have both greater mass loss and reduced variance compared to the uncalibrated prior distributions. Parametric uncertainty is large enough through about 2130 that the two projections under different emission scenarios are indistinguishable from one another. However, after rapid ocean warming in the first half of the 22nd century, the projections become statistically distinct within decades. Overall, this study demonstrates an efficient Bayesian calibration and uncertainty propagation workflow for ice-sheet model projections and identifies the potential for large sea-level rise contributions from the Amery sector of the Antarctic Ice Sheet after 2100 under high-greenhouse-gas-emission scenarios.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

ORION: Operational FoRecastIng Of INduced Seismicity

The Operational Forecasting of Induced Seismicity toolkit “ORION” (ORION) is an open-source, observation-based ensemble forecasting toolkit which is geared towards helping operators understand the seismic hazard (i.e., probabilistic assessment of the magnitude and frequency of induced seismic events) at a site. ORION analyzes how the seismic hazard evolves during injection and suggests possible mitigation strategies to employ if an earthquake that exceeds certain threshold is observed. Through its ensemble modeling approach, ORION leverages the benefits of statistical-, physics-, and machine learning-based forecasting methodologies, while reducing the impact of each model’s respective limitations. The ORION toolkit consists of an easy-to-use GUI interface that affords a user as much or as little interaction as desired. Advanced capabilities allow the user to upload local, high-precision earthquake catalogs, projected injection profiles and/or spatiotemporal estimates of pressure/stress, and to tune various model parameters. ORION will then provide a spatial and temporal ensemble forecast of seismicity defined as the probability of exceedance of a given earthquake magnitude over a forecast period. Additionally, ORION will provide probability distribution of the statistically derived maximum possible earthquake magnitude that may be expected. Finally, ORION will provide suggested operational management strategies (e.g. reduce injection volumes at specific wells) based on the level of hazard.

NRAP,NRAP Tools,ORION,Operational Forcasting of In↗
DOE OSTI2026

The AWAKEN wind farm benchmark, Part 2: Modeling results

Accurately modeling wind farm performance in complex atmospheric flows remains a challenge. This paper presents the modeling results of the American WAKE experimeNt (AWAKEN) wind farm benchmark, a collaborative effort involving 16 research groups from academia and industry within the International Energy Agency Wind Technology Collaboration Programme Task 57. The study evaluates a diverse suite of simulation tools, ranging from fast-running engineering wake models to high-fidelity large-eddy simulations, against a diurnal case study observed during the AWAKEN campaign. The benchmark utilized a three-phase structure to progressively assess model performance as observational data availability increased. Initial blind predictions showed that higher-fidelity models did not uniformly outperform simpler simulation tools. A distinct spatial bias was observed where models struggled to resolve the interplay between a low-level jet, wakes, and terrain-induced flow acceleration. In subsequent phases, leveraging additional measurements for model improvement led to a reduction in mean absolute error across the model ensemble; however, this effect was most pronounced in engineering wake models, where targeted calibration reduced error by up to 40~\%. Overall, the study demonstrates that inflow characterization remains a primary prerequisite for accuracy, particularly for models relying on coarse forcing datasets. While the limited ability to resolve local terrain-flow interactions under single-day conditions represent a recognized constraint, the overall findings on wake modeling and real-world validation still provide valuable guidance for model application and for mitigating this limitation.

Bodini, Nicola↗
DOE OSTI2024

Uncertainty Visualization Challenges in Decision Systems with Ensemble Data & Surrogate Models

Uncertainty visualization is a key component in translating important insights from ensemble simulation data into actionable decision-making by visually conveying various aspects of uncertainty within a system. With the recent advent of fast surrogate models trained on ensemble data, we can substitute computationally expensive simulations, which allows users to interact with more aspects of data spaces than ever before. However, the use of ensemble data with surrogate models in a decision-making tool brings up new challenges for uncertainty visualization, namely how to reconcile and communicate the new and different types of uncertainties brought in by surrogates and how to utilize these new data estimates in actionable ways. In this work, we examine these issues as they relate to high-dimensional data visualization, the integration of discrete datasets and the continuous representations of those datasets, and the unique difficulties associated with systems that allow users to iterate between input and output spaces. We assess the role of uncertainty visualization in facilitating intuitive and actionable interaction with ensemble data and surrogate models, and highlight key challenges in this new frontier of computational simulation.

ensemble data↗
DOE OSTI2026

Contrasting Parametric Sensitivities in Two Global Vegetation Models Using Parameter Perturbation Ensembles

Uncertainty in land model projections remains high and the roles of parametric and structural uncertainty are difficult to disentangle. To compare parametric sensitivity across model structures we present two parameter perturbation ensembles using the Community Land Model (CLM) operating in satellite phenology mode. The ensembles contrast two vegetation modules: (a) the default CLM vegetation module and (b) the Functionally Assembled Terrestrial Ecosystem Simulator (CLM-FATES). We perturbed over 300 parameters and quantified their effects on biophysical fluxes globally and across biomes. Most parameters have minimal impact on biophysical fluxes, with only a few substantially influencing results. While both models exhibit similar parameter sensitivity for some fluxes, CLM-FATES shows larger spread in gross primary productivity (GPP), driven by strong sensitivity to carboxylation rate. CLM-FATES also shows a weaker GPP response to soil hydrology parameters and exhibits higher water use efficiency (WUE). Cross-model comparisons reveal similar sensitivities for some parameters (e.g., leaf dimension) but divergent responses to others (e.g., stomatal intercept), highlighting underlying structural differences. Differences in WUE and sensitivity to hydrology and stomatal conductance parameters underscore how model structure fundamentally alters parametric sensitivity. The data sets generated from these ensembles can be used to identify influential parameters and guide future calibration efforts.

Foster, A. C. [NSF National Center for Atmospheric↗
DOE OSTI2023

A Pacific Tropical Decadal Variability Challenge for Climate Models

Understanding and forecasting Tropical Pacific Decadal-scale Variability (TPDV) strongly rely on climate model simulations. Using a Linear Inverse Modeling (LIM) diagnostic approach, we reveal Coupled Model Intercomparison Project Phase 6 models have significant challenges in reproducing the spatial structure and dominant mechanisms of TPDV. Specifically, while the models' ensemble mean pattern of TPDV resembles that of observations, the spread across models is very large and most models show significant differences from observations. In observations, removing the coupling between extratropics and tropics reduces TPDV by ~60%–70%, and removing the tropical thermocline variability makes the central tropical Pacific a key center of action for TPDV and El Niño Southern Oscillation variability. These characteristics are only confirmed in a subset of models. Differences between observations and simulations are outside the range of natural internal TPDV noise and pose important questions regarding our ability to model the impacts of natural internal low-frequency variability superimposed on long-term climate change.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Ensemble approximate control variate estimators: Applications to multi-fidelity importance sampling.

The recent growth in multifidelity uncertainty quantification has given rise to a large set of variance reduction techniques that leverage information from model ensembles to provide variance reduction for estimates of the statistics of a high-fidelity model. In this paper we provide two contributions: (1) we utilize an ensemble estimator to account for uncertainties in the optimal weights of approximate control variate (ACV) approaches and derive lower bounds on the number of samples required to guarantee variance reduction; and (2) we extend an existing multifidelity importance sampling (MFIS) scheme to leverage control variates. Our approach directly addresses a limitation of many multifidelity sampling strategies that require the usage of pilot samples to estimate covariances. As such we make significant progress towards both increasing the practicality of approximate control variates—for instance, by accounting for the effect of pilot samples—and using multifidelity approaches more effectively for estimating low-probability events. The numerical results indicate our hybrid MFIS-ACV estimator achieves up to 50% improvement in variance reduction over the existing state-of-the-art MFIS estimator, which had already shown an outstanding convergence rate compared to the Monte Carlo method, on several problems of computational mechanics.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

A process-based evaluation of biases in extratropical stratosphere–troposphere coupling in subseasonal forecast systems

Abstract. Two-way coupling between the stratosphere and troposphere is recognized as an important source of subseasonal-to-seasonal (S2S) predictability and can open windows of opportunity for improved forecasts. Model biases can, however, lead to a poor representation of such coupling processes; drifts in a model's circulation related to model biases, resolution, and parameterizations have the potential to feed back on the circulation and affect stratosphere–troposphere coupling. We introduce a set of diagnostics using readily available data that can be used to reveal these biases and then apply these diagnostics to 22 S2S forecast systems. In the Northern Hemisphere, nearly all S2S forecast systems underestimate the strength of the observed upward coupling from the troposphere to the stratosphere, downward coupling within the stratosphere, and the persistence of lower-stratospheric temperature anomalies. While downward coupling from the lower stratosphere to the near surface is well represented in the multi-model ensemble mean, there is substantial intermodel spread likely related to how well each model represents tropospheric stationary waves. In the Southern Hemisphere, the stratospheric vortex is oversensitive to upward-propagating wave flux in the forecast systems. Forecast systems generally overestimate the strength of downward coupling from the lower stratosphere to the troposphere, even as most underestimate the radiative persistence in the lower stratosphere. In both hemispheres, models with higher lids and a better representation of tropospheric quasi-stationary waves generally perform better at simulating these coupling processes.

Garfinkel, Chaim I. (ORCID:000000017258666X)↗
DOE OSTI2025

Evaluating Sea Breezes and Associated Convective Cloud Evolution in the Model Gray Zone

We characterize convective clouds associated with sea‐breeze circulations (SBC) using multi‐agency observations and multi‐case ensemble model simulations. The focus is on assessing convective cloud lifecycle properties and their merging behavior, as well as the environmental conditions they are embedded in, particularly SBC features. In total, 46 SBC days over the Houston‐Galveston region are selected and simulated using the Weather Research and Forecasting (WRF) model at a gray zone scale with a forecast‐like parameterization setup. Advanced techniques, including change‐point detection, a Lagrangian cloud tracking method, and a newly developed cell merging and splitting detection algorithm, are applied and/or developed for this study. Our findings indicate that the WRF model at 1 km grid spacing well represents the thermodynamic conditions over the region, as well as SBC timing and intensity. However, for the associated convective cells, WRF overestimates the 30‐dBZ echo top height, cell area, and maximum radar reflectivity compared to radar observations. This overestimation is potentially due to under‐resolved entrainment processes, an overestimated merging frequency, and the overestimation of updraft intensity. Furthermore, the model exhibits a deficiency in simulating congestus clouds, showing a more rapid transition from shallow to deep convection compared to observed behavior. Moreover, observations indicate stronger, deeper, and wider clouds when merging happens. Conversely, in simulations, the merging process does not necessarily lead to higher or longer‐lived cells, as many cases experience rapid and frequent merging and splitting which may result in more variance in convective updraft velocity during the convection lifetime.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Addressing bias in bagging and boosting regression models

As artificial intelligence (AI) becomes widespread, there is increasing attention on investigating bias in machine learning (ML) models. Previous research concentrated on classification problems, with little emphasis on regression models. This paper presents an easy-to-apply and effective methodology for mitigating bias in bagging and boosting regression models, that is also applicable to any model trained through minimizing a differentiable loss function. Our methodology measures bias rigorously and extends the ML model's loss function with a regularization term to penalize high correlations between model errors and protected attributes. We applied our approach to three popular tree-based ensemble models: a random forest model (RF), a gradient-boosted model (GBT), and an extreme gradient boosting model (XGBoost). We implemented our methodology on a case study for predicting road-level traffic volume, where RF, GBT, and XGBoost models were shown to have high accuracy. Despite high accuracy, the ML models were shown to perform poorly on roads in minority-populated areas. Our bias mitigation approach reduced minority-related bias by over 50%.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Downscaled CMIP5 projections of physical fire risk understate historical trends

Reliable projections of wildfire risk are important for multi-sector impacts analysis. Statistically downscaled and bias-corrected Earth system model ensemble products are routinely used to analyze regional physical wildfire risk, but evaluations of historical observed trends and variability are lacking. Here, we evaluate physical fire risk over the western United States using the Canadian Forest Fire Weather Index (FWI) by comparing model outputs from the Coupled Model Intercomparison Project Phase 5 (CMIP5), statistically downscaled via the Multivariate Adaptive Constructed Analogs (MACA) approach, against the observational target dataset gridMET, a gridded high-resolution surface meteorological product. We analyze multidecadal trends and interannual variability in seasonal average FWI for the historical period and future projections under two emissions scenarios, and we compare MACA-CMIP5 ensemble results with a simple time series model that generates historical and future projections of seasonal FWI based on bootstrapping observed historical trends and variability. Our findings indicate that MACA-CMIP5 accurately captures the magnitude and spatial patterns of seasonally averaged FWI but tends to underestimate historical decadal trends. We show that future increases in fire risk may be underestimated relative to the simple time series model that projects historical variability into the future. We also highlight that model biases in relative humidity contribute significantly to model-data differences. Our results underscore the importance of historical hindcasting exercises for informing broader multi-sector applications.

FWI↗
DOE OSTI2024

Regional climate change: consensus, discrepancies, and ways forward

Climate change has emerged across many regions. Some observed regional climate changes, such as amplified Arctic warming and land-sea warming contrasts have been predicted by climate models. However, many other observed regional changes, such as changes in tropical sea surface temperature and monsoon rainfall are not well simulated by climate model ensembles even when taking into account natural internal variability and structural uncertainties in the response of models to anthropogenic radiative forcing. This suggests climate model predictions may not fully reflect what our future will look like. The discrepancies between models and observations are not well understood due to several real and apparent puzzles and limitations such as the “signal-to-noise paradox” and real-world record-shattering extremes falling outside of the possible range predicted by models. Addressing these discrepancies, puzzles and limitations is essential, because understanding and reliably predicting regional climate change is necessary in order to communicate effectively about the underlying drivers of change, provide reliable information to stakeholders, enable societies to adapt, and increase resilience and reduce vulnerability. The challenges of achieving this are greater in the Global South, especially because of the lack of observational data over long time periods and a lack of scientific focus on Global South climate change. To address discrepancies between observations and models, it is important to prioritize resources for understanding regional climate predictions and analyzing where and why models and observations disagree via testing hypotheses of drivers of biases using observations and models. Gaps in understanding can be discovered and filled by exploiting new tools, such as artificial intelligence/machine learning, high-resolution models, new modeling experiments in the model hierarchy, better quantification of forcing, and new observations. Conscious efforts are needed toward creating opportunities that allow regional experts, particularly those from the Global South, to take the lead in regional climate research. This includes co-learning in technical aspects of analyzing simulations and in the physics and dynamics of regional climate change. Finally, improved methods of regional climate communication are needed, which account for the underlying uncertainties, in order to provide reliable and actionable information to stakeholders and the media.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Concurrent Inter-Model Spread of Boreal Winter Westerly Jet Meridional Positions Between the Northern and Southern Hemispheres in CMIP6 Models

Here, this study investigates the inter-model spread of climatological extratropical westerly jets in boreal winter, using the historical simulation of 52 Coupled Model Intercomparison Project phase 6 (CMIP6) models from 1851 to 2014. The results show that there is a substantial spread in the latitude of the upper-tropospheric westerly jet across models, characterised by large inter-model standard deviations to both the poleward and equatorward sides of the jet axis, although the multi-model ensemble mean (MME) performs well in simulating meridional position of westerly jets. Furthermore, we detect the consistency of inter-model jet position spread between the Northern and Southern Hemispheres, based on the inter-model empirical orthogonal function (EOF) decomposition and correlation of regional-averaged zonal winds. Specifically, the models that simulate the westerly jets poleward/equatorward relative to the MME position in one hemisphere also tend to simulate the jets poleward/equatorward in the other hemisphere. Accordingly, we define a global jet spread index to depict the concurrence of jet shift in the two hemispheres. The results of inter-model regression analyses based on this index indicate that the models positioning the jets poleward than the MME tend to simulate a wider Hadley Cell, a poleward-shifted Ferrel Cell in the Southern Hemisphere, enhanced precipitation in the subtropics and suppressed precipitation in the tropics, and warmer sea surface temperatures in the subtropics and mid-latitudes. The present results suggest that improving the simulation of jet positions in climate models requires a comprehensive consideration of thermal states in the tropics and subtropics/mid latitudes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Statistical Generation of Ocean Forcing With Spatiotemporal Variability for Ice Sheet Models

Melting of ice at the base of floating ice shelves that fringe the Antarctic ice sheet has been identified as a significant source of uncertainty in sea level rise projections. Part of this uncertainty derives from chaotic internal variability of the coupled ocean-atmosphere system. For numerical ice sheet model projections, this uncertainty has not previously been quantified because of the prohibitive computational expense of running large climate model ensembles. Here, we develop and demonstrate a technique that generates independent realizations of internal climate variability from a single climate model simulation. Building on previous developments in model emulation, this technique uses empirical orthogonal function decomposition and Fourier-phase randomization to generate statistically consistent realizations of spatiotemporal variability fields for the target climate variable. The method facilitates efficient sampling of a wide range of climate trajectories, which can also be incorporated within ice sheet or other physical models to represent feedback processes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Amplified drought trends in Nepal increase the potential for Himalayan wildfires

Here we report in the spring 2021, Nepal underwent a record wildfire season in which active fires were detected at a rate 10 times greater than the 2002–2020 average. Prior to these major wildfire events, the country experienced a prolonged precipitation deficit and extreme drought during the post-monsoon period (starting in October 2020). An analysis using observational, reanalysis, and climate model ensemble data indicates that both climate variability and climate change-induced severe drought conditions were at play. Further analysis of climate model outputs suggests the likely reoccurrence of drought conditions, thus favoring active wildfire seasons in Nepal throughout the twenty-first century. While the inter-model uncertainty is large and direct modeling of wildfire spread and suppression has not been completed, the demonstrated relationship between a drought index (the standardized precipitation and evapotranspiration index) and subsequent fire activity may offer actionable opportunities for forest managers to employ the monitoring and projection of climate anomalies at sub-seasonal to decadal timescales to inform their management strategies for Nepal’s wildlands.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

LTAU-FF: Loss Trajectory Analysis for Uncertainty in atomistic Force Fields

Model ensembles are effective tools for estimating prediction uncertainty in deep learning atomistic force fields. However, their widespread adoption is hindered by high computational costs and overconfident error estimates. In this work, we address these challenges by leveraging distributions of per-sample errors obtained during training and employing a distance-based similarity search in the model latent space. Our method, which we call LTAU (Loss Trajectory Analysis for Uncertainty), efficiently estimates the full probability distribution function of errors for any test point using the logged training errors, achieving speeds that are 2–3 orders of magnitudes faster than typical ensemble methods and allowing it to be used for tasks where training or evaluating multiple models would be infeasible. We apply LTAU towards estimating parametric uncertainty in atomistic force fields (LTAU-FF), demonstrating that it produces well-calibrated confidence intervals and predicts errors that correlate strongly with the true errors for data near the training domain. Furthermore, we show that the errors predicted by LTAU-FF can be used in practical applications for detecting out-of-domain data, tuning model performance, and predicting failure during simulations. We believe that LTAU will be a valuable tool for uncertainty quantification in atomistic force fields and is a promising method that should be further explored in other domains of machine learning.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Machine Learning for Mapping Multipactor Susceptibility in RF Systems: Capabilities and Generalization Constraints

Multipactor is a surface-driven electron avalanche phenomenon that degrades the performance and reliability of radio-frequency (RF) systems in particle accelerator and vacuum electronics applications. Multipactor behavior in a given device structure is conventionally assessed through susceptibility charts, which provide a parameter-space characterization of the instability. In this work, we assess the capabilities of machine-learning (ML) models to learn and predict such susceptibility charts and analyze the constraints governing their generalization across materials. Using a simulation-derived dataset spanning six distinct secondary-electron-yield material profiles in a canonical two-surface planar geometry, we train supervised regression models and artificial neural networks to predict the time-averaged electron growth rate, δavg, across the relevant parameter space. Model performance is evaluated using metrics that explicitly probe the structure of susceptibility charts, including Intersection over Union, Structural Similarity Index, and correlation analysis. Tree-based ensemble models outperform neural-network models in reconstructing susceptibility regions and in generalizing across material domains. Principal-component analysis reveals disjoint material feature distributions, indicating that the piecewise mode structure of multipactor susceptibility is difficult to represent with a single global model and that generalization is constrained by data coverage rather than by model complexity. An exhaustive reduced-coverage study further shows that sparse material-space coverage can yield mean performance in the same general range but producing large variability in the susceptibility-region overlap. These results clarify the capabilities of ML-based surrogate models for parameter-space characterization of multipactor discharge. They also provide guidance for their appropriate use in RF system design.

43 PARTICLE ACCELERATORS↗