Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Ensemble models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Multi‐Model Ensembles in Ecosystem Modeling: Challenges and Best Practices for Decision‐Making

Ecosystem models are increasingly central to the decision-making for environmental policy, conservation planning, and climate-related investments. Yet, the growing reliance on Multi-Model Ensembles (MMEs) of ecosystem models by practitioners and policymakers, sometimes under tight timelines and imperfect information, has frequently outpaced the scientific rigor required to ensure ensemble reliability. Here, MMEs refer to approaches that combine targeted predictions from multiple models with the expectation of improving robustness and quantifying predictive uncertainty. Poorly designed MMEs may create a false sense of confidence and lead to suboptimal policy and market decisions. This perspective argues that robust decision-making-relevant MMEs must be grounded on two pillars: (1) rigorous Model Intercomparison Projects (MIPs), which identify inter-model agreement and disagreement, characterize model uncertainties, and evaluate robustness with observationally based benchmarks—MIPs' diagnostic evaluation is so critical that it must be needed to drive MME's decision in model selection and weighting, especially when only a limited number of models available; and (2) co-design by both stakeholders and scientists to ensure that scenarios, metrics and uncertainty requirements provide decision-relevant information. Building upon the past success and lessons from the existing MIPs-MMEs efforts (e.g., climate/Earth system/crop), we derived the theoretical basis for MMEs, addressed their specific challenges in ecosystem modeling, and highlighted proper consideration of model numbers and diversity, risk of model inter-dependence, effective calibration of model parameters, possible overdue of some ecosystem model development, critical roles of open benchmark data across a wide range of conditions, and suggested use of Artificial Intelligence to support MIPs-MMEs. We highlighted the under-recognized opportunity for MIPs and MMEs to drive scientific progress and innovation through identifying better performing models, systematic benchmarking, feedback loops, and targeted model improvement. By following actionable best practice guidelines, MMEs can evolve from ad hoc aggregation of models into a trusted backbone of environmental policy and decision-making.

ecosystem modeling↗

Advanced Atmospheric Ensemble Modeling Techniques

Ensemble modeling (EM), the creation of multiple atmospheric simulations for a given time period, has become an essential tool for characterizing uncertainties in model predictions. We explore two novel ensemble modeling techniques: (1) perturbation of model parameters (Adaptive Programming, AP), and (2) data assimilation (Ensemble Kalman Filter, EnKF). The current research is an extension to work from last year and examines transport on a small spatial scale (<100 km) in complex terrain, for more rigorous testing of the ensemble technique. Two different release cases were studied, a coastal release (SF6) and an inland release (Freon) which consisted of two release times. Observations of tracer concentration and meteorology are used to judge the ensemble results. In addition, adaptive grid techniques have been developed to reduce required computing resources for transport calculations. Using a 20- member ensemble, the standard approach generated downwind transport that was quantitatively good for both releases; however, the EnKF method produced additional improvement for the coastal release where the spatial and temporal differences due to interior valley heating lead to the inland movement of the plume. The AP technique showed improvements for both release cases, with more improvement shown in the inland release. This research demonstrated that transport accuracy can be improved when models are adapted to a particular location/time or when important local data is assimilated into the simulation and enhances SRNL’s capability in atmospheric transport modeling in support of its current customer base and local site missions, as well as our ability to attract new customers within the intelligence community.

54 ENVIRONMENTAL SCIENCES↗

Regional and Model-Specific Response Types in A Global Gridded Crop Model Ensemble

Crop models are often employed to project crop yields under changing conditions such as global warming and associated management change for adaptation. Multi-model ensembles are promoted to enhance the robustness of projections, but questions remain on what causes often large differences between projections of individual models. Global Gridded Crop Models (GGCMs) are especially exposed to this question when applied for assessing climate change impacts, adaptation, environmental impacts of agricultural production, because their results are used in downstream analyses, such as in integrated assessment or economic modeling for projecting future land-use change. Even though global gridded crop models are often based on detailed field-scale models or have implemented similar modeling principles in other ecosystem models, global-scale models are subject to substantial uncertainties from both model structure and parametrization as well as from calibration and input data quality. AgMIP’s Global Gridded Crop Model Intercomparison (GGCMI) has thus set out to intercompare GGCMs in order to evaluate model performance, describe model uncertainties, identify inconsistencies within the ensemble and underlying reasons, and to ultimately improve models and modeling capacities. In phase 2 of the GGCMI activities, 12 modeling groups followed a modeling protocol that asked for up to 1404 31-year global simulations at 0.5 arc-degree spatial resolution to assess models’ sensitivities to changes in carbon dioxide (C; 4 different levels) temperature (T; 7 different offset levels), water supply (W; 9 levels), and nitrogen (N; 3 levels), the so-called CTWN experiment (Franke et al. 2020; http://dx.doi.org/10.5194/gmd-13-2315-2020). We here present analyses of model response types using impact response surfaces along the C, T, W, and N dimensions, respectively and collectively. Doing so, we can understand differences in simulated responses per driver rather than aggregated changes in yields. We find that models’ sensitivities to the individual driver dimensions are substantially different and often more different across models than across regions. A cluster analysis finds regional and model-specific patterns. There is some agreement across models with respect to the spatial patterns of response types but strong differences in the distribution of response type clusters across models suggests that models need to undergo further scrutiny. We suggest establishing standards in model process evaluation not only against historical dynamics but also against dedicated experiments across the CTWN dimensions.

crop models↗

Dynamic Metabolic Model Building Based on the Ensemble Modeling Approach

Ensemble modeling of kinetic systems addresses the challenges of kinetic model construction, with respect to parameter value selection, and still allows for the rich insights possible from kinetic models. This project aimed to show that constructing, implementing, and analyzing such models is a useful tool for the metabolic engineering toolkit, and that they can result in actionable insights from models. Key concepts are developed and deliverable publications and results are presented.

09 BIOMASS FUELS↗

Teleconnections of the Quasi–Biennial Oscillation in a multi–model ensemble of QBO–resolving models

The quasi-biennial oscillation (QBO) dominates the interannual variability of the tropical stratosphere and influences other regions of the atmosphere. The high predictability of the QBO implies that its teleconnections could lead to in-creased skill of seasonal and decadal forecasts provided the relevant mechanisms are accurately represented in models. Here modelling and sampling uncertainties of QBO teleconnections are examined using a multi-model ensemble of QBO-resolving atmospheric general circulation models that have carried out a set of coordinated experiments as part of the Stratosphere-troposphere Processes And their Role in Climate (SPARC) QBO initiative (QBOi). During Northern Hemisphere winter the stratospheric polar vortex in most of these models strengthens when the QBO near 50 hPa is westerly and weakens when it is easterly, consistent with, but weaker than, the observed response. These weak responses are likely due to model errors, such as systematically weak QBO amplitudes near 50 hPa, affecting the teleconnection. The teleconnection to the North Atlantic Oscillation is less well captured overall, but of similar strength to the observed signal in the few models that do show it. Furthermore, the models do not show clear evidence of a QBO teleconnection to the Northern Hemisphere Pacific sector subtropical jet.

54 ENVIRONMENTAL SCIENCES↗

Machine Learning Predictions of a Multiresolution Climate Model Ensemble

Statistical models of high-resolution climate models are useful for many purposes, including sensitivity and uncertainty analyses, but building them can be computationally prohibitive. We generated a unique multiresolution perturbed parameter ensemble of a global climate model. We use a novel application of a machine learning technique known as random forests to train a statistical model on the ensemble to make high-resolution model predictions of two important quantities: global mean top-of-atmosphere energy flux and precipitation. The random forests leverage cheaper low-resolution simulations, greatly reducing the number of high-resolution simulations required to train the statistical model. We demonstrate that high-resolution predictions of these quantities can be obtained by training on an ensemble that includes only a small number of high-resolution simulations. Finally, we also find that global annually averaged precipitation is more sensitive to resolution changes than to any of the model parameters considered.

54 ENVIRONMENTAL SCIENCES↗

Multi-Model Ensemble Wake Vortex Prediction

Several multi-model ensemble methods are investigated for predicting wake vortex transport and decay. This study is a joint effort between National Aeronautics and Space Administration and Deutsches Zentrum fuer Luft- und Raumfahrt to develop a multi-model ensemble capability using their wake models. An overview of different multi-model ensemble methods and their feasibility for wake applications is presented. The methods include Reliability Ensemble Averaging, Bayesian Model Averaging, and Monte Carlo Simulations. The methodologies are evaluated using data from wake vortex field experiments.

Koerner, Stephan↗

Multi-Model Ensemble Wake Vortex Prediction

Several multi-model ensemble methods are investigated for predicting wake vortex transport and decay. This study is a joint effort between National Aeronautics and Space Administration and Deutsches Zentrum fuer Luft- und Raumfahrt to develop a multi-model ensemble capability using their wake models. An overview of different multi-model ensemble methods and their feasibility for wake applications is presented. The methods include Reliability Ensemble Averaging, Bayesian Model Averaging, and Monte Carlo Simulations. The methodologies are evaluated using data from wake vortex field experiments.

Koerner, Stephan↗

The North American Multi-Model Ensemble (NMME): Phase-1 Seasonal to Interannual Prediction, Phase-2 Toward Developing Intra-Seasonal Prediction

The recent US National Academies report "Assessment of Intraseasonal to Interannual Climate Prediction and Predictability" was unequivocal in recommending the need for the development of a North American Multi-Model Ensemble (NMME) operational predictive capability. Indeed, this effort is required to meet the specific tailored regional prediction and decision support needs of a large community of climate information users. The multi-model ensemble approach has proven extremely effective at quantifying prediction uncertainty due to uncertainty in model formulation, and has proven to produce better prediction quality (on average) then any single model ensemble. This multi-model approach is the basis for several international collaborative prediction research efforts, an operational European system and there are numerous examples of how this multi-model ensemble approach yields superior forecasts compared to any single model. Based on two NOAA Climate Test Bed (CTB) NMME workshops (February 18, and April 8, 2011) a collaborative and coordinated implementation strategy for a NMME prediction system has been developed and is currently delivering real-time seasonal-to-interannual predictions on the NOAA Climate Prediction Center (CPC) operational schedule. The hindcast and real-time prediction data is readily available (e.g., http://iridl.ldeo.columbia.edu/SOURCES/.Models/.NMME/) and in graphical format from CPC (http://origin.cpc.ncep.noaa.gov/products/people/wd51yf/NMME/index.html). Moreover, the NMME forecast are already currently being used as guidance for operational forecasters. This paper describes the new NMME effort, presents an overview of the multi-model forecast quality, and the complementary skill associated with individual models.

NOAA CPC↗

Quantifying sensitivity in numerical weather prediction-modeled offshore wind speeds through an ensemble modeling approach

A decade of research has shown that numerical weather prediction (NWP)-modeled wind speeds can be highly sensitive to the inputs and setups within the NWP model. For wind resource characterization applications, this sensitivity is often addressed by constructing a range of setups and selecting the one that best validates against observations. However, this approach is not possible in areas that lack high-quality hub height observations, especially offshore wind areas. In such cases, techniques to quantify and disseminate confidence in NWP-modeled wind speeds in the absence of observations are needed. We address this need in the present study and propose best practices for quantifying the spread in NWP-modeled wind speeds. We implement an ensemble approach in which we consider 24 different setups to the Weather Research and Forecasting (WRF) model. We construct the ensemble by considering variations in WRF version, WRF namelist, atmospheric forcing, and sea surface temperature (SST) forcing. Our analysis finds that the standard deviation produces more consistent estimates compared to the interquartile range and tends to be the more conservative estimator for ensemble variability. We further find that model spread increases closer to the surface and on shorter time scales. In conclusion, we explore methods to attribute total ensemble variability to the different ensemble components (e.g., atmospheric forcing and SST product) and find that contributions by components also vary depending on time scale. We anticipate that the methods and results presented in this paper will provide a reasonable foundation for further research into ensemble-based wind resource data sets.

17 WIND ENERGY↗

Crop Model Improvement Reduces the Uncertainty of the Response to Temperature of Multi-Model Ensembles

To improve climate change impact estimates and to quantify their uncertainty, multi-model ensembles (MMEs) have been suggested. Model improvements can improve the accuracy of simulations and reduce the uncertainty of climate change impact assessments. Furthermore, they can reduce the number of models needed in a MME. Herein, 15 wheat growth models of a larger MME were improved through re-parameterization and/or incorporating or modifying heat stress effects on phenology, leaf growth and senescence, biomass growth, and grain number and size using detailed field experimental data from the USDA Hot Serial Cereal experiment (calibration data set). Simulation results from before and after model improvement were then evaluated with independent field experiments from a CIMMYT worldwide field trial network (evaluation data set). Model improvements decreased the variation (10th to 90th model ensemble percentile range) of grain yields simulated by the MME on average by 39% in the calibration data set and by 26% in the independent evaluation data set for crops grown in mean seasonal temperatures greater than 24 C. MME mean squared error in simulating grain yield decreased by 37%. A reduction in MME uncertainty range by 27% increased MME prediction skills by 47%. Results suggest that the mean level of variation observed in field experiments and used as a benchmark can be reached with half the number of models in the MME. Improving crop models is therefore important to increase the certainty of model-based impact assessments and allow more practical, i.e. smaller MMEs to be used effectively.

damage assessment↗

Multi-Wheat-Model Ensemble Responses to Interannual Climate Variability

We compare 27 wheat models' yield responses to interannual climate variability, analyzed at locations in Argentina, Australia, India, and The Netherlands as part of the Agricultural Model Intercomparison and Improvement Project (AgMIP) Wheat Pilot. Each model simulated 1981e2010 grain yield, and we evaluate results against the interannual variability of growing season temperature, precipitation, and solar radiation. The amount of information used for calibration has only a minor effect on most models' climate response, and even small multi-model ensembles prove beneficial. Wheat model clusters reveal common characteristics of yield response to climate; however models rarely share the same cluster at all four sites indicating substantial independence. Only a weak relationship (R2 0.24) was found between the models' sensitivities to interannual temperature variability and their response to long-termwarming, suggesting that additional processes differentiate climate change impacts from observed climate variability analogs and motivating continuing analysis and model development efforts.

uncertainty↗

Using a numerical cumulus ensemble model as a tool for studying cloud processes

A numerical cumulus ensemble model is presented, which covers a large area, and resolves individual cumulus clouds. The clouds produced by the model simulation are illustrated. Scatter plots of cloud amount vs relative humidity are examined. Plans for a cumulus ensemble model study on the effects of boundary layer processes on cloud formation are discussed.

Krueger, Steven K.↗

“Which Projections Do I Use?” Strategies for Climate Model Ensemble Subset Selection Based on Regional Stakeholder Needs

Climate model (or earth system model) projections are increasingly used for climate adaptation planning and impact assessments. As part of this process, many end‐users evaluate a subset of downscaled climate projections without being aware of the implications of downscaling methodology for statistics or event outcomes. Approaches for determining a subset of global climate models to use often focus on values from the raw models, rather than from their downscaled counterparts, in other words assuming that the statistical distribution of the multi‐model ensemble does not change post downscaling. This study demonstrates that a downscaled ensemble will typically retain the change distribution as a raw ensemble, but individual models can differ dramatically post‐downscaling. We recommend that subset‐selection methods account for this possibility and that decision‐relevant downscaled climate projections provide proper descriptions of fitness‐for‐purpose and essential caveats, so that non‐specialists can interpret the results with an appropriate level of confidence.

54 ENVIRONMENTAL SCIENCES↗

Conditioning multi-model ensembles for disease forecasting

In this study we investigate how an ensemble of disease models can be conditioned to observational data, in a bid to improve its predictive skill. We use the ensemble of influenza forecasting models gathered by the US Centers for Disease Control and Prevention (CDC) as the exemplar. This ensemble is used every year to forecast the annual influenza outbreak in the United States. The models constituting this ensemble draw on very different modeling assumptions and approximations and are a diverse collection of methods to approximate epidemiological dynamics. Currently, each models' predictions are accorded the same importance, or weight, when compiling the ensemble's forecast. We consider this equally-weighted ensemble as the baseline case which has to be improved upon. In this study, we explore whether an ensemble forecast can be improved by "conditioning" the ensemble to whatever observational data is available from the ongoing outbreak. "Conditioning" can imply according the ensemble's members different weights which evolve over time, or simply perform the forecast using the top k (equally-weighted) models. In the latter case, the composition of the "top-k-see of models evolves over time. This is called "model averaging" in statistics. We explore four methods to perform model-averaging, three of which are new. We find that the CDC ensemble responds best to the "top-k-models" approach to model-averaging. All the new MA methods perform better than the baseline equally-weighted ensemble. The four model-averaging methods treat the models as black-boxes and simply use their forecasts as inputs i.e., one does not need access to the models at all, but rather only their forecasts. The model-averaging approaches reviewed in this report thus form a general framework for model-averaging any model ensemble.

59 BASIC BIOLOGICAL SCIENCES↗

A scalable framework for the global offline community land model ensemble simulation

Current earth system models have a large range of uncertainty, owing to differences in the simulation of feedbacks and insufficient information to constrain model parameters. Parameter disturbance experiment provides a straightforward method to quantify the variation (uncertainty) outputs caused by model inputs. Owing to the software complexity and computational intensity of earth system models, a large-scale simulation framework is needed to support ensemble simulation required by parameter disturbance experiment. This paper presents a parallel framework for the community land model ensemble simulation. After a software structure review of the community land model simulation, a single factor parameter disturbance experiment of a reference computational experiment design is used to demonstrate the software design principles, computational characteristics of individual application, parallel ensemble simulation implementation, as well as the weak scalability of this simulation framework on a high-end computer. Lastly, the paper discusses some preliminary diagnostic analysis results of the single factor parameter disturbance experiments. The framework design considerations and implementation details described in this paper can be beneficial to many other research programmes involving large scale, legacy modelling system.

58 GEOSCIENCES↗