Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “ensemble modeling system”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

The Representation of Aerosols in GMAO’s Newest Reanalyses

Over the past few years, NASA’s Global Modelling and Assimilation Office has been working on the configuration and production of three new reanalysis products, GEOS-IT, GiOcean, and MERRA-21C. GEOS-IT, or the Goddard Earth Observing System for Instrument Teams, is a 3D variational data assimilation system that runs in a near real time framework however retrospectively provides data back through 1998 to deliver a consistent view of the Earth-atmosphere system for the production of observational NASA products. Retrospective production for GEOS-IT is complete and the meteorology has since been used to produce the one way weakly coupled GiOcean reanalysis. Due to differences in the atmospheric model, particularly related to scavenging, aerosols are not identical in GEOS-IT and GiOcean. MERRA-21C, or the Modern Era Restrospective analysis for Research and Applications in the 21st century, is a hybrid 4D ensemble variational system at a finer horizontal resolution of 0.25 degrees. Although different in their intended use, and therefore configuration, these systems prominently feature coupling between meteorology and aerosols. The differences and similarities in the aerosol configuration between the three systems will be discussed, covering biomass burning and anthropogenic emissions as well as observations used for the assimilation of aerosol optical depth. A large emphasis will be placed on the version of the underlying aerosol module, GOCART, which underwent a complete refactoring and the addition of radiatively active brown carbon between GEOS-IT and MERRA-21C. Independent observations will be used to evaluate the performance of aerosols in both reanalyses, focusing on aerosol optical depth, surface particulate matter, and vertical profiles of aerosol backscatter.

Allison Collow↗
DOE OSTI2021

Deep Learning for Ensemble Forecasting

Focal Area: (2) Predictive modeling through the use of AI techniques and AI-derived model components and the use of AI and other tools to design a prediction system comprising a hierarchy of models. Science Challenge: While both climate and weather forecast systems have continued to improve due to substantial efforts to improve computational capabilities, observations, and numerical models, the atmosphere is a chaotic system, and this puts a fundamental limit on our ability to make predictions. Forecasts made by high-resolution models initialized with only slightly different atmospheric states can quickly diverge. Quantifying uncertainty in forecasts is essential to adequately understand them and to make the best-informed policy decisions particularly when it comes to hydrology, extreme weather (including extreme precipitation events), and climate.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRSDate not supplied

Using Federated Learning to Overcome Data Gravity in Space

Humans intend to take longer missions to outer space. Understanding the impact that space has on human health is paramount to the success of these missions. Controlled experiments with model organisms are run to infer the impact of space conditions on human health, but the data these experiments generate are too large to transfer to Earth for building models. The same is true for space-relevant data generated on Earth. Ideally, these datasets should be combined to improve statistical power and model accuracy without having to transfer data. Federated learning is such a method which trains an algorithm across decentralized computing systems, each of which has their own local copy of training and testing data. In this research, made possible by NASA@Work, the AI for Life in Space group at NASA demonstrates the use of federated learning to train an ensemble of causality inference models on a combination of data residing on the International Space Station (ISS) and in the cloud. Our work leverages CRISP, a causal inference platform developed during the 2020 Frontier Development Lab’s “Astronaut Health Challenge.” We also leverage the OpenFL federated learning library which was collaboratively developed at Intel and UPenn. We used publicly available data from the NASA Ames Life Sciences Data Archive to identify features in ionizing radiation experiments as causal of changes in cardiac blood velocity. This research demonstrates, for the first time, the possibility of running machine learning algorithms on datasets separated by astronomical distances. In this experiment, all the data were generated in terra, half of which were transferred to the ISS and analyzed on the Spaceborne Computer. In the future, our research will leverage federated learning on data generated in situ on the ISS with data generated terrestrially to predict the impact of spaceflight on mammalian female reproductive capacity.

James Casaletto↗
DOE OSTI2025

A Machine Learning Framework for Modeling Ensemble Properties of Atomically Disordered Materials

Atomic disorder can strongly influence material properties such as charge transport, optical response, and catalytic activity. However, efficiently modeling these disorder effects remains challenging for first-principles methods due to the cost of sampling large configurational spaces and computing complex physical quantities. Recent advances of machine learning techniques, particularly graph neural networks (GNNs), has enabled the efficient and accurate predictions of complex material properties, offering promising tools for studying disordered systems. In this work, we present a general machine-learning-assisted computational framework that integrates equivariant GNNs with Monte Carlo simulations to compute the thermodynamic and ensemble-averaged functional properties of disordered materials. Using the surface-termination-disordered MXene monolayer Ti 3 C 2 T 2–x as a representative system, we find that electrical conductivity exhibits an emergent peak near the order–disorder phase transition temperature due to the interplay between electron scattering and doping. In contrast, optical conductivity remains largely insensitive to local atomic disorder and reflects the global surface chemical composition. These results highlight the role of atomic disorder in affecting material properties and demonstrate the potential of our approach for statistically modeling disorder effects in a wide range of materials such as high-entropy alloys and spin liquids.

MXene↗
DOE OSTI2021

Soil Water Potentials (1990-2018) from a calibrated ELM-FATES, and rooting depth analyses scripts, PA-BCI, Panama

This dataset has ELM-FATES simulation outputs and scripts that accompany and reproduce the manuscript, Chitra-Tarak et al. (2021) Hydraulically-vulnerable trees survive on deep-water access during droughts in a tropical forest. New Phytologist. https: //doi.org/10.1111/nph.17464”. This dataset is organized into four parts: (1) source code for the model, Energy Exascale Earth System Land Model-Functionally Assembled Advanced Terrestrial simulator (ELM-FATES) (2) R scripts associated with ELM-FATES simulations: to prepare files for parameter ensemble simulation runs, calibrate the model with observed hydrological fluxes observed at BCI, re-run the model with tuned parameters and obtain long-term soil matric potentials by depth (3) ELM-FATES hydrological simulation outputs at PA-BCI, Panama (1990-2018, daily), and (4) R scripts for inverse rooting depth modeling and post-processing: prepare datasets and inversely model tree rooting depths at BCI, test hypotheses about the relationship of rooting depths with plant hydraulic traits and drought-induced mortality, and reproduce the manuscript itself. n.b. 1. Data needed to run R scripts in (2) and (4) and thus to reproduce the manuscript are sourced from various public datasets. Refer to the data availability section of the accompanying manuscript. 2. This dataset version (prepared on May 12, 2021) matches the revised and published version of the accompanying manuscript (published May 16, 2021) replacing a previous version. In the publication, this dataset is cited as follows: Chitra-Tarak R, Xu C, Aguilar S, Anderson-Teixeira K, Chambers J, Detto M, Faybishenko B, Fisher R, Knox R, Koven C et al. 2020. Soil water potentials (1990–2018) from a calibrated ELM-FATES, and rooting depth analyses scripts, PA-BCI, Panama. 2.0. NGEE Tropics Data Collection. doi: 10.15486/ngt/1696806

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2018

Recent Updates in the SMAP Level-4 Soil Moisture Algorithm

The NASA Soil Moisture Active Passive (SMAP) mission generates, among other data sets, the Level-4 Soil Moisture (L4_SM) product. The L4_SM data are published with a mean latency of ~2.5 days from the time of observation and provide global, three-hourly, 9-km resolution estimates of surface and root-zone soil moisture and related land surface states and fluxes. The L4_SM algorithm is based on the assimilation of SMAP radiometer brightness temperature observations into the NASA Catchment land surface model using a spatially distributed ensemble Kalman filter (EnKF). In 2018, the L4_SM algorithm was upgraded from Version 3 to Version 4. Underlying the new version is a revised modeling system that includes improved input parameter datasets for land cover, topography, and vegetation height that are based on recent, high-quality, space-borne remote sensing observations. Additionally, SMAP Level-2 soil moisture retrievals and in situ soil moisture measurements were used to calibrate a particular Catchment model parameter that governs the recharge of surface soil moisture from below under non-equilibrium conditions, which brings the model's surface soil moisture more in line with the SMAP Level-2 and in situ soil moisture. Moreover, the calibration of the assimilated SMAP brightness temperatures changed substantially from Version 3 to Version 4, and the "catchment deficit" model variable was removed from the EnKF state vector to avoid degrading the model's groundwater estimates.Considerable effort went into the version upgrade, creating an expectation that the new version is improved over the old version. Indeed, some aspects of the new version are clearly better. However, other aspects are not. In this presentation we summarize the skill of the new and old versions vs. independent in situ measurements and in terms of data assimilation diagnostics, including, for example, the statistics of the (soil moisture) analysis increments and the observation-minus-forecast (brightness temperatures) residuals. We share our experience with trying to improve to the L4_SM product and the lessons learned from the effort.

Reichle, Rolf H.↗
NASA NTRS2019

ESD Reviews: Model Dependence in Multi-Model Climate Ensembles: Weighting, Sub-Selection and Out-Of-Sample Testing

The rationale for using multi-model ensembles in climate change projections and impacts research is often based on the expectation that different models constitute independent estimates; therefore, a range of models allows a better characterisation of the uncertainties in the representation of the climate system than a single model. However, it is known that research groups share literature, ideas for representations of processes, parameterisations, evaluation data sets and even sections of model code. Thus, nominally different models might have similar biases because of similarities in the way they represent a subset of processes, or even be near-duplicates of others, weakening the assumption that they constitute independent estimates. If there are near-replicates of some models, then treating all models equally is likely to bias the inferences made using these ensembles. The challenge is to establish the degree to which this might be true for any given application. While this issue is recognised by many in the community, quantifying and accounting for model dependence in anything other than an ad-hoc way is challenging. Here we present a synthesis of the range of disparate attempts to define, quantify and address model dependence in multi-model climate ensembles in a common conceptual framework, and provide guidance on how users can test the efficacy of approaches that move beyond the equally weighted ensemble. In the upcoming Coupled Model Intercomparison Project phase 6 (CMIP6), several new models that are closely related to existing models are anticipated, as well as large ensembles from some models. We argue that quantitatively accounting for dependence in addition to model performance, and thoroughly testing the effectiveness of the approach used will be key to a sound interpretation of the CMIP ensembles in future scientific studies.

Abramowitz, Gab↗
NASA NTRS2018

GEOS S2S-2_1 File Specification: GMAO Seasonal and Sub-Seasonal Forecast Output

The NASA GMAO seasonal (9 months) and subseasonal (45 days) forecasts are produced with the Goddard Earth Observing System (GEOS) Atmosphere-Ocean General Circulation Model and Data Assimilation System Version S2S-2_1. The new system replaces version S2S-1.0 described in Borovikov et al (2017), and includes upgrades to many components of the system. The atmospheric model includes an upgrade from a pre-MERRA-2 version running on a latitude-longitude grid at approx. 1 degree resolution to a current version running on a cubed sphere grid at approximately 1/2 degree resolution. The important developments are related to the dynamical core (Putman et al., 2011), the moist physics (''two-moment microphysics'' of Barahona et al., 2014) and the cryosphere (Cullather et al., 2014). As in the previous GMAO S2S system, the land model is that of Koster et al (2000). GMAO S2S-2_1 now includes the Goddard Chemistry Aerosol Radiation and Transport (GOCART, Colarco et al., 2010) single moment interactive aerosol model that includes predictive aerosols including dust, sea salt and several species of carbon and sulfate. The previous version of GMAO S2S specified aerosol amounts from climatology, which were used to inform the atmospheric radiation only. The ocean model includes an upgrade from MOM4 to MOM5 (Griffies 2012), and continues to be run on the tripolar grid at approximately 1/2 degree resolution in the tropics with 40 vertical levels. As in S2S-1.0, the sea ice model is from the Los Alamos Sea Ice model (CICE4, Hunke and Lipscomb 2010). The Ocean Data Assimilation System (ODAS) has been upgraded from the one described in Borovikov et al., 2017 to one that uses a modified version of the Penny, 2014 Local Ensemble Transform Kalman Filter (LETKF), and now assimilates along-track altimetry. The ODAS also does a nudging to MERRA-2 SST and sea ice boundary conditions. The atmospheric data assimilation fields used to constrain the atmosphere in the ODAS have been upgraded from MERRA to a MERRA-2 like system. The system is initialized using a MERRA-2-like atmospheric reanalysis (Gelaro et al. 2017) and the GMAO S2S-2_1 ocean analysis. Additional ensemble members for forecasts are produced with initial states at 5-day intervals, with additional members based on perturbations of the atmospheric and ocean states. Both subseasonal and seasonal forecasts are submitted to the National MultiModel Ensemble (NMME) project, and are part of the US/Canada multimodel seasonal forecasts (http://www.cpc.ncep.noaa.gov/products/NMME/). A large suite of retrospective forecasts (''hindcasts'') have been completed, and contribute to the calculation of the model's baseline climatology and drift, anomalies from which are the basis of the seasonal forecasts.

Seasonal Forecast↗
DOE OSTI2024

Evaluation of Autoconversion Representation in E3SMv2 Using an Ensemble of Large-Eddy Simulations of Low-Level Warm Clouds

In numerical atmospheric models that treat cloud and rain droplet populations as separate condensate categories, precipitation initiation in warm clouds is often represented by an autoconversion rate (Au), which is the rate of formation of new rain droplets through the collisions of cloud droplets. Being a function of the cloud droplet size distribution (DSD), the local Au is commonly parameterized as a function of DSD moments: cloud droplet number (n c ) and mass (q c ) concentrations. When applied in a large-scale model, the grid-mean Au must also include a correction, or enhancement factor, to account for the horizontal variability of the cloud properties across the model grid. In this study, we evaluate the Au representation in the Energy Exascale Earth System Model version 2 (E3SMv2) climate model using large-eddy simulations (LES), which explicitly resolve cloud droplet spectra, and therefore the local Au, as well as its spatial variability. The analysis of an ensemble of warm low-level cloud cases shows that the E3SMv2 formulation represents the Au reasonably well compared to the horizontally averaged explicitly computed rate from LES. The agreement, however, comes from a combination of an underestimated E3SM-tuned local Au rate and an overestimated subgrid cloud variability enhancement factor. The latter bias is traced to neglecting the horizontal variability of n c and its co-variability with q c in parameterizing the grid-mean Au.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Solar Uncertainty Management and Mitigation for Exceptional Reliability in Grid Operations (SUMMER-GO): Project Final Report

The Solar Uncertainty Management and Mitigation for Exceptional Reliability in Grid Operation (SUMMER-GO) project was recently completed through a collaboration among the National Renewable Energy Laboratory, Maxar, the Electric Reliability Council of Texas (ERCOT), the University of Texas at Dallas, the University of California Berkeley, and the University of Colorado Boulder. The project made significant advances in probabilistic solar power forecasting, both through the development of Bayesian model averaging methods for ensemble forecasting and in bringing these and other advancements into practice with Maxar's delivery of operational forecasts to ERCOT. In addition to creating more reliable solar power forecasts, the project developed methods for their utilization in power system operations. These include the development of risk-aware unit commitment and economic dispatch algorithms and methods to reformulate probabilistic forecasts to be used in these power system operational models. Dynamic power system reserve methods were also developed, which have been shown in silico to create economic savings and reliability improvements on an ERCOT-like system as well as financial savings in the ERCOT system through more granular consideration of the uncertainty associated with solar power forecasts. Finally, a situational awareness tool to help grid operators better understand solar power forecast uncertainty in daily operations was developed and extensively vetted.

14 SOLAR ENERGY↗
DOE OSTI2022

TDCOSMO VIII. A key test of systematics in the hierarchical method of time-delay cosmography

The largest source of systematic errors in the time-delay cosmography method likely arises from the lens model mass distribution, where an inaccurate choice of model could in principle bias the value of $H$ 0 . A Bayesian hierarchical framework has been proposed which combines lens systems with kinematic data, constraining the mass profile shape at a population level. The framework has been previously validated using a small sample of lensing galaxies drawn from hydro-simulations. The goal of this work is to expand the validation to a more general set of lenses consistent with observed systems, as well as confirm the capacity of the method to combine two lens populations: one which has time delay information and one which lacks time delays and has systematically different image radii. For this purpose, we generated samples of analytic lens mass distributions made of baryons+dark matter and fit the subsequent mock images with standard power-law models. Corresponding kinematics data were also emulated. The hierarchical framework applied to an ensemble of time-delay lenses allowed us to correct the $H$ 0 bias associated with model choice to find $H$ 0 within 1.5σ of the fiducial value. We then combined this set with a sample of corresponding lens systems which have no time delays and have a source at lower $z$, resulting in a systematically smaller image radius relative to their effective radius. The hierarchical framework has successfully accounted for this effect, recovering a value of $H$ 0 which is both more precise (σ ~ 2%) and more accurate (0.7% median offset) than the time-delay set alone. This result confirms that non-time-delay lenses can nonetheless contribute valuable constraining power to the determination of $H$ 0 via their kinematic constraints, assuming they come from the same global population as the time-delay set.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Arctic Tipping Points Triggering Global Change (LDRD Final Report)

The Arctic is warming and feedbacks in the coupled Earth system may be driving the Arctic to tipping events that could have critical downstream impacts for the rest of the globe. In this project we have focused on analyzing sea ice variability and loss in the coupled Earth system Summer sea ice loss is happening rapidly and although the loss may be smooth and reversible, it has significant consequences for other Arctic systems as well as geopolitical and economic implications. Accurate seasonal predictions of sea ice minimum extent and long-term estimates of timing for a seasonally ice-free Arctic depend on a better understanding of the factors influencing sea ice dynamics and variation in this strongly coupled system. Under this project we have investigated the most influential factors in accurate predictions of September Arctic sea ice extent using machine learning models trained separately on observational data and on simulation data from five E3SM historical ensembles. Monthly averaged data from June, July, and August for a selection of ice, ocean, and atmosphere variables were used to train a random forest regression model. Gini importance measures were computed for each input feature with the testing data. We found that sea ice volume is most important earlier in the season (June) and sea ice extent became a more important predictor closer to September. Results from this study provide insight into how feature importance changes with forecast length and illustrates differences between observational data and simulated Earth system data. We have additionally performed a global sensitivity analysis (GSA) using a fully coupled ultra- low resolution configuration E3SM. To our knowledge, this is the first global sensitivity analysis involving the fully-coupled E3SM Earth system model. We have found that parameter variations show significant impact on the Arctic climate state and atmospheric parameters related to cloud parameterizations are the most significant. We also find significant interactions between parameters from different components of E3SM. The results of this study provide invaluable insight into the relative importance of various parameters from the sea ice, atmosphere and ocean components of the E3SM (including cross-component parameter interactions) on various Arctic-focused quantities of interest (QOIs).

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2020

Toward Coupled Data Assimilation in NASA’s GEOS: Developments in the Ocean Context

The Global Modeling & Assimilation Office (GMAO) at NASA GSFC produces analyses and predictions of the Earth system using various configurations of the Goddard Earth Observing System (GEOS) model and assimilation system. The current sub-seasonal-to-seasonal prediction system (GEOS-S2S) is based on a coupled atmosphere-ocean-land-ice configuration of GEOS which includes the Modular Ocean Model version 5 (MOM5) run at approximately 50-km resolution and a de-coupled OI-based ocean analysis that uses an initialization of MOM5 forced by the MERRA-2 reanalysis. GMAO will soon implement an updated GEOS-S2S system that will run at 25-km resolution and adopt aspects of the hybrid four-dimensional ensemble-variational (H4DEnVar) system already running in the production-version atmospheric analysis system, including a Local Ensemble Transform Kalman Filter (LETKF) to provide initial conditions for the oceanic state. This presentation will focus on developments to sustain the GMAO's systems on longer time horizons, where more radical transformations will be required to adapt to advanced computing environments, higher resolution and more diverse model components, and new observations for the Earth system. Results will describe progress toward a version of the GEOS coupled system that will be based around the Joint Effort for Data assimilation Integration (JEDI) framework being developed within Joint Center for Satellite Data Assimilation (JCSDA) and include an updated ocean model, MOM6. Discussion will focus specifically on the use of a Unified Forward Operator (UFO) for simulating observations and the Object Oriented Prediction System (OOPS) for providing the state estimate. These features are being developed as a multi-agency effort under the auspices of the JCSDA and are being adopted in the GMAO for all its applications of coupled data assimilation including S2S, numerical weather prediction, and reanalysis.

Mahajan, Rahul↗
DOE OSTI2024

Scenario storyline discovery for complex multi-actor human-natural systems

This poster was presented at the AGU Fall Meeting 2024. Abstract:Scenario analysis is a useful tool for assessing the impacts of future conditions or alternative strategies. However, the common practice of focusing on a small number of predetermined scenarios can limit our understanding of key uncertainties, and fail to represent diverse stakeholder impacts. Exploratory modeling approaches have been developed to address these issues by simulating a wide range of possible futures and system perspectives. A challenge with these approaches is that they often involve large ensemble experiments which limit interpretability and usability. We recently introduced the FRamework for Narrative Storylines and Impact Classification (FRNSIC; pronounced ``forensic''), a scenario discovery framework that helps users identify scenario storylines that capture key system dynamics and as well as important outcomes. In this poster presentation, we present training materials to support the generalizable application of the framework to other multi-actor systems with complex dynamics. Specifically, we will present a step-by-step methodological typology of tools and methods that can be used to generate and classify plausible states of the world on key metrics and consequential dynamics. The typology will also discuss potential implications of these choices and their applicability to different systems.

Colorado River↗
DOE OSTI2026

HydraGNN_OPF_GFM_2026 - Ensemble of predictive graph foundation models for power grid applications

This dataset supports research on graph foundation models for optimal power flow (OPF) on electric grids using HydraGNN. It contains heterogeneous graph representations of PGLib-OPF cases spanning systems from 14 to 13,659 buses, together with packed HDF5 datasets for pretraining, feasibility classification, and N-1 contingency analysis. The release includes OPF solution data, downstream fine-tuning datasets, pretrained HeteroSAGE and HeteroHEAT model checkpoints, hyperparameter-optimization summaries across multiple heterogeneous GNN architectures, and aggregated fine-tuning results for sample-efficiency studies. The dataset is designed to enable scalable training, evaluation, and transfer-learning studies for OPF surrogate modeling, including node-level AC-OPF solution prediction, graph-level prediction, feasibility classification, operating-condition generalization, and contingency-response tasks.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

EPIsembleVis: A geo-visual analysis and comparison of the prediction ensembles of multiple COVID-19 models

In this work, we present EPIsembleVis, a web-based comparative visual analysis tool for evaluating the consistency of multiple COVID-19 prediction models. Our approach analyzes a collection of COVID-19 predictions from different epidemiological models as an ensemble and utilizes two metrics to quantify model performance. These metrics include (a) prediction uncertainty (represented as the dispersion of predictions in each ensemble) and (b) prediction error (calculated by comparing individual model predictions with the recorded data). Through an interactive visual interface, our approach provides a data-driven workflow for (a) selecting and constructing the COVID-19 model prediction ensemble based on the spatiotemporal overlap of available predictions of multiple epidemiological models, (b) quantifying the model performance using both the uncertainty of each model prediction ensemble, and the error of each ensemble member that represents individual model predictions, and (c) visualizing the spatiotemporal variability in the projection performance of individual models using a suite of novel ensemble visualization techniques, such as the data availability map, a spatiotemporal textured-tile calendar, multivariate rose chart, and time-series leaflet glyph. We demonstrate the capability of our ensemble visual interface through a case study that investigates the performance of weekly COVID-19 predictions, which are provided through the COVID-19 Forecast Hub UMass-Amherst Influenza Forecasting Center of Excellence [47] for the United States and United States Territories. The EPIsembleVis tool is implemented using open-source web technologies and adaptive system design, rendering it interoperable with Elasticsearch and Kibana for automatically ingesting COVID-19 predictions from online repositories, and it is generalizable for analyzing worldwide projections from more epidemiological models.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Deficient precipitation sensitivity to Sahel land surface forcings among CMIP5 models

Abstract The overall performance of the simulated seasonal precipitation response to local terrestrial forcings, namely vegetation abundance and soil moisture, in the Sahel among the Coupled Model Intercomparison Project Phase Five (CMIP5) Earth System Models (ESMs) is systematically investigated and compared with its observational counterpart using a multivariate statistical method. The observed seasonal precipitation response is evaluated against a large ensemble of observational, reanalysis, and satellite data sets to provide quantification of uncertainties. The behaviour of models with and without a Dynamic Global Vegetation Model (DGVM) component is also explored, along with the mechanisms responsible for terrestrial feedback on rainfall. In general, the CMIP5 models can reasonably capture the seasonal evolution of Sahel precipitation and soil moisture, albeit with wet biases during the pre‐monsoon period and dry biases during the peak monsoon period. The non‐DGVM ESMs simulate comparable leaf area indices (LAIs) with observations, while DGVM‐enabled ESMs simulate too much year‐round LAI. The variance of precipitation that is attributed to oceanic forcings in CMIP5 is comparable with observations; however, the variance of precipitation that is attributed to terrestrial forcings is smaller in CMIP5 models than observed, especially for non‐DGVM ESMs. CMIP5 models, especially those without DGVMs, undervalue precipitation's observed response strength to soil moisture anomalies. In both observations and CMIP5 models, none of the atmospheric variables show significant responses to direct vegetation forcing, except for the response in transpiration. Although vegetation has minimal direct effect on the atmospheric state, it can affect the atmosphere by modifying soil moisture and transpiration rate indirectly, which helps explain the more realistic simulation of rainfall in DGVM‐enabled ESMs than non‐DGVM ESMs. Coupling of an ESM to a DGVM is critical in generating reasonable land–atmosphere feedback and examining future ecological and climatic changes over the Sahel.

54 ENVIRONMENTAL SCIENCES↗