SEARCH · Engineering Papers

Results for “ensemble modeling system”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

All sources NASA NTRS DOE OSTI

DOE OSTI2023

Modeling single-molecule stretching experiments using statistical thermodynamics

Single-molecule stretching experiments are widely utilized within the fields of physics and chemistry to characterize the mechanics of individual bonds or molecules, as well as chemical reactions. Analytic relations describing these experiments are valuable, and these relations can be obtained through the statistical thermodynamics of idealized model systems representing the experiments. Since the specific thermodynamic ensembles manifested by the experiments affect the outcome, primarily for small molecules, the stretching device must be included in the idealized model system. Though the model for the stretched molecule might be exactly solvable, including the device in the model often prevents analytic solutions. In the limit of large or small device stiffness, the isometric or isotensional ensembles can provide effective approximations, but the device effects are missing. Here a dual set of asymptotically correct statistical thermodynamic theories are applied to develop accurate approximations for the full model system that includes both the molecule and the device. In conclusion, the asymptotic theories are first demonstrated to be accurate using the freely jointed chain model and then using molecular dynamics calculations of a single polyethylene chain.

74 ATOMIC AND MOLECULAR PHYSICS↗

DOE OSTI2023

Quantifying uncertainty for deep learning based forecasting and flow-reconstruction using neural architecture search ensembles

Classical problems in computational physics such as data-driven forecasting and signal reconstruction from sparse sensors have recently seen an explosion in deep neural network (DNN) based algorithmic approaches. However, most DNN models do not provide uncertainty estimates, which are crucial for establishing the trustworthiness of these techniques in downstream decision making tasks and scenarios. In recent years, ensemble-based methods have achieved significant success for the uncertainty quantification in DNNs on a number of benchmark problems. However, their performance on real-world applications remains under-explored. In this work, we present an automated approach to DNN discovery and demonstrate how this may also be utilized for ensemble-based uncertainty quantification. Specifically, we propose the use of a scalable neural and hyperparameter architecture search for discovering an ensemble of DNN models for complex dynamical systems. We highlight how the proposed method not only discovers high-performing neural network ensembles for our tasks, but also quantifies uncertainty seamlessly. This is achieved by using genetic algorithms and Bayesian optimization for sampling the search space of neural network architectures and hyperparameters. Subsequently, a model selection approach is used to identify candidate models for an ensemble set construction. Afterwards, a variance decomposition approach is used to estimate the uncertainty of the predictions from the ensemble. We demonstrate the feasibility of this framework for two tasks — forecasting from historical data and flow reconstruction from sparse sensors for the sea-surface temperature. In conclusion, we demonstrate superior performance from the ensemble in contrast with individual high-performing models and other benchmarks.

Deep ensembles↗

NASA NTRS2003

Various Numerical Applications on Tropical Convective Systems Using a Cloud Resolving Model

In recent years, increasing attention has been given to cloud resolving models (CRMs or cloud ensemble models-CEMs) for their ability to simulate the radiative-convective system, which plays a significant role in determining the regional heat and moisture budgets in the Tropics. The growing popularity of CRM usage can be credited to its inclusion of crucial and physically relatively realistic features such as explicit cloud-scale dynamics, sophisticated microphysical processes, and explicit cloud-radiation interaction. On the other hand, impacts of the environmental conditions (for example, the large-scale wind fields, heat and moisture advections as well as sea surface temperature) on the convective system can also be plausibly investigated using the CRMs with imposed explicit forcing. In this paper, by basically using a Goddard Cumulus Ensemble (GCE) model, three different studies on tropical convective systems are briefly presented. Each of these studies serves a different goal as well as uses a different approach. In the first study, which uses more of an idealized approach, the respective impacts of the large-scale horizontal wind shear and surface fluxes on the modeled tropical quasi-equilibrium states of temperature and water vapor are examined. In this 2-D study, the imposed large-scale horizontal wind shear is ideally either nudged (wind shear maintained strong) or mixed (wind shear weakened), while the minimum surface wind speed used for computing surface fluxes varies among various numerical experiments. For the second study, a handful of real tropical episodes (TRMM Kwajalein Experiment - KWAJEX, 1999; TRMM South China Sea Monsoon Experiment - SCSMEX, 1998) have been simulated such that several major atmospheric characteristics such as the rainfall amount and its associated stratiform contribution, the Qlheat and Q2/moisture budgets are investigated. In this study, the observed large-scale heat and moisture advections are continuously applied to the 2-D model. The modeled cloud generated from such an approach is termed continuously forced convection or continuous large-scale forced convection. A third study, which focuses on the respective impact of atmospheric components on upper Ocean heat and salt budgets, will be presented in the end. Unlike the two previous 2-D studies, this study employs the 3-D GCE-simulated diabatic source terms (using TOGA COARE observations) - radiation (longwave and shortwave), surface fluxes (sensible and latent heat, and wind stress), and precipitation as input for the Ocean mixed-layer (OML) model.

Shie, C.-L.↗

NASA NTRS2003

Sensitivity of Precipitation Processes to Microphysics and Resolution in a Cloud-Resolving Model

The Goddard Cumulus Ensemble (GCE) model is used to examine the impact of various microphysical schemes, and vertical and horizontal resolution on the development, intensity and rainfall associated with mesoscale convective systems, idealized hurricanes and an ensemble of clouds. The model variables include horizontal and vertical velocities, potential temperature, perturbation pressure, turbulent kinetic energy, and mixing ratios of all water phases (vapor, liquid, and ice). The major characteristics of the GCE model are the explicit representation of warm rain and ice microphysical processes, and their complex interactions with solar and infrared radiative transfer processes, and with surface processes. For idealized hurricane, an axisymmetric version of the GCE model was developed and used successfully to simulate the tropical cyclogenesis process using both a Rankin vortex and saturated air within a specified radius as initial conditions. For mesoscale convective systems, the 3-D version of the GCE model was used to simulate squall lines that developed in the western Pacific, eastern Atlantic and central US. For the cloud ensemble, the GCE model was integrated for several days in order to have good sampling of cloud statistics. In this paper, the sensitivities of hurricane intensity to various microphysical processes and model grid resolution will be examined.

Tao, Wei-Kuo↗

NASA NTRS2003

Sensitivity of Precipitation Processes to Microphysics and Resolution in a Cloud-resolving Model

Tao, Wei-Kuo↗

NASA NTRS2003

Sensitivity of Precipitation Processes to Microphysics and Resolution in a Cloud-Resolving Model

The Goddard Cumulus Ensemble (GCE) model is used to examine the impact of various microphysical schemes, and vertical and horizontal resolution on the development, intensity and rainfall associated with mesoscale convective systems, idealized hurricanes and an ensemble of clouds. The model variables include horizontal and vertical velocities, potential temperature, perturbation pressure, turbulent kinetic energy, and mixing ratios of all water phases (vapor, liquid, and ice). The major characteristics of the GCE model are the explicit representation of warm rain and ice microphysical processes, and their complex interactions with solar and infrared radiative transfer processes, and with surface processes. For idealized hurricane, an axisymmetric version of the GCE model was developed and used successfully to simulate the tropical cyclogenesis process using both a Rankin vortex and saturated air within a specified radius as initial conditions. For mesoscale convective systems, the 3-D version of the GCE model was used to simulate squall lines that developed in the western Pacific, South China Sea, eastern Atlantic, South America and central U.S. For the cloud ensemble, the GCE model was integrated for several days in order to have good sampling of cloud statistics. In this paper, the sensitivities of hurricane intensity to various microphysical processes and model grid resolution will be examined. This will be mainly achieved by performing sensitivity tests using various horizontal (from 1- to 5-kilometers) and vertical resolutions (from 20- to 200-meters in the lower troposphere to 200- to 500-m in the middle and upper troposphere). Sensitivity tests using various microphysical schemes (warm rain only, and three ice with either graupel or hail) will also be performed. The PBL and diurnal variation of precipitation processes will also be evaluated. The budgets will be calculated for different regions (i.e., convective and stratiform regions).

Tao, Wei-Kao↗

DOE OSTI2021

Crack fault diagnosis of rotating machine in nuclear power plant based on ensemble learning

Crack faults in rotating machines can cause machine shutdown or scrapping, endangering the normal operation and safety of nuclear power plants. Intelligent diagnostic techniques based on machine learning have the potential to diagnose crack faults. However, problems such as scarcity of field fault data and high noise of plant measurements pose challenges to the application of machine learning. Here this study proposes an ensemble learning approach to mitigate the negative impacts of the problems. Ensemble learning is a strategy for combining multiple machine learning models into a composite model. The basic idea of ensemble learning is that even if one model makes a mistake, other models can correct it. Case studies based on bearing and gear system fault experiments show that the proposed ensemble learning models have better diagnostic results than the single model in the presence of noise and small data.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

DOE OSTI2025

A Greening Future Elevates Flash Drought Risk in Northern Mid‐to‐High Latitudes

Flash droughts have become a growing concern, as they can emerge rapidly and increase the risk of crop failure. Although past studies have investigated the meteorological drivers and future changes of flash drought, why flash drought is more frequent over humid and vegetated regions remains underexplored. This study delves further into the mechanism by which vegetation regulates flash drought and its future change using observations from multiple data sets and large ensemble simulations from three Earth system models. On an interannual timescale, both observations and simulations show robust increases in flash drought frequency and a higher flash-to-sub-seasonal drought ratio during spring or antecedent conditions with dense vegetation, supporting the important role of vegetation in flash drought occurrence, especially in the northern mid-to-high latitudes. In the latter regions, the large ensemble simulations show robust increases in flash drought (e.g., 67% and 46% increases in Eastern U.S. and North Asia in 2050–2100 relative to 1950–2000 under the high emission scenario), where the growing season is lengthening. Although greening might suggest reduced drought stress, it drives precipitation-soil moisture-evapotranspiration decoupling by increasing evapotranspiration partitioning to transpiration. As transpiration can access deep soil water through the plant root system, its increased portion can weaken the constraints of concurrent precipitation on evapotranspiration, thus accelerating soil moisture depletion under high evaporative demand, driving a slow-to-rapid drought transition. How vegetation regulates flash drought by regulating surface moisture budget is supported by observations and simulations. Although warming supports early planting, agriculture may increasingly be threatened by surging flash drought risk.

Drought↗

DOE OSTI2023

Ensemble models for circuit topology estimation, fault detection and classification in distribution systems

This paper presents a methodology for simultaneous fault detection, classification, and topology estimation for adaptive protection of distribution systems. The methodology estimates the probability of the occurrence of each one of these events by using a hybrid structure that combines three sub-systems, a convolutional neural network for topology estimation, a fault detection based on predictive residual analysis, and a standard support vector machine with probabilistic output for fault classification. The input to all these sub-systems is the local voltage and current measurements. A convolutional neural network uses these local measurements in the form of sequential data to extract features and estimate the topology conditions. The fault detector is constructed with a Bayesian stage (a multitask Gaussian process) that computes a predictive distribution (assumed to be Gaussian) of the residuals using the input. Since the distribution is known, these residuals can be transformed into a Standard distribution, whose values are then introduced into a one-class support vector machine. The structure allows using a one-class support vector machine without parameter cross-validation, so the fault detector is fully unsupervised. Finally, a support vector machine uses the input to perform the classification of the fault types. All three sub-systems can work in a parallel setup for both performance and computation efficiency. In conclusion, we test all three sub-systems included in the structure on a modified IEEE123 bus system, and we compare and evaluate the results with standard approaches.

24 POWER TRANSMISSION AND DISTRIBUTION↗

NASA NTRS2021

Harmonized Emissions Component (HEMCO) 3.0 as a Versatile Emissions Component for Atmospheric Models: Application in the GEOS-Chem, NASA GEOS, WRF-GC, CESM2, NOAA GEFS-Aerosol, and NOAA UFS Models

Emissions are a central component of atmospheric chemistry models. The Harmonized Emissions Component (HEMCO) is a software component for computing emissions from a user-selected ensemble of emission inventories and algorithms. It allows users to re-grid, combine, overwrite, subset, and scale emissions from different inventories through a configuration file and with no change to the model source code. The configuration file also maps emissions to model species with appropriate units. HEMCO can operate in offline stand-alone mode, but more importantly it provides an online facility for models to compute emissions at runtime. HEMCO complies with the Earth System Modeling Framework (ESMF) for portability across models. We present a new version here, HEMCO 3.0, that features an improved three-layer architecture to facilitate implementation into any atmospheric model and improved capability for calculating emissions at any model resolution including multiscale and unstructured grids. The three-layer architecture of HEMCO 3.0 includes (1) the Data Input Layer that reads the configuration file and accesses the HEMCO library of emission inventories and other environmental data, (2) the HEMCO Core that computes emissions on the user-selected HEMCO grid, and (3) the Model Interface Layer that re-grids (if needed) and serves the data to the atmospheric model and also serves model data to the HEMCO Core for computing emissions dependent on model state (such as from dust or vegetation). The HEMCO Core is common to the implementation in all models, while the Data Input Layer and the Model Interface Layer are adaptable to the model environment. Default versions of the Data Input Layer and Model Interface Layer enable straightforward implementation of HEMCO in any simple model architecture, and options are available to disable features such as re-gridding that may be done by independent couplers in more complex architectures. The HEMCO library of emission inventories and algorithms is continuously enriched through user contributions so that new inventories can be immediately shared across models. HEMCO can also serve as a general data broker for models to process input data not only for emissions but for any gridded environmental datasets. We describe existing implementations of HEMCO 3.0 in (1) the GEOS-Chem “Classic” chemical transport model with shared-memory infrastructure, (2) the high-performance GEOS-Chem (GCHP) model with distributed-memory architecture, (3) the NASA GEOS Earth System Model (GEOS ESM), (4) the Weather Research and Forecasting model with GEOS-Chem (WRF-GC), (5) the Community Earth System Model Version 2 (CESM2), and (6) the NOAA Global Ensemble Forecast System – Aerosols (GEFS-Aerosols), as well as the planned implementation in the NOAA Unified Forecast System (UFS). Implementation of HEMCO in CESM2 contributes to the Multi-Scale Infrastructure for Chemistry and Aerosols (MUSICA) by providing a common emissions infrastructure to support different simulations of atmospheric chemistry across scales.

Haipeng Lin↗

NASA NTRS2002

The Surface Energy Budget and Precipitation Efficiency for Convective Systems During TOGA, COARE, GATE, SCSMEX and ARM: Cloud-Resolving Model Simulations

A two-dimensional version of the Goddard Cumulus Ensemble (GCE) Model is used to simulate convective systems that developed in various geographic locations. Observed large-scale advective tendencies for potential temperature, water vapor mixing ratio, and horizontal momentum derived from field campaigns are used as the main forcing. By examining the surface energy budgets, the model results show that the two largest terms are net condensation (heating/drying) and imposed large-scale forcing (cooling/moistening) for tropical oceanic cases. These two terms arc opposite in sign, however. The contributions by net radiation and latent heat flux to the net condensation vary in these tropical cases, however. For cloud systems that developed over the South China Sea and eastern Atlantic, net radiation (cooling) accounts for about 20% or more of the net condensation. However, short-wave heating and long-wave cooling are in balance with each other for cloud systems over the West Pacific region such that the net radiation is very small. This is due to the thick anvil clouds simulated in the cloud systems over the Pacific region. Large-scale cooling exceeds large-scale moistening in the Pacific and Atlantic cases. For cloud systems over the South China Sea, however, there is more large-scale moistening than cooling even though the cloud systems developed in a very moist environment. though For three cloud systems that developed over a mid-latitude continent, the net radiation and sensible and latent heat fluxes play a much more important role. This means the accurate measurement of surface fluxes and radiation is crucial for simulating these mid-latitude cases.

Tao, W.-K.↗

DOE OSTI2023

Pyrocumulonimbus Events Over British Columbia in 2017: An Ensemble Model Study of Parameter Sensitivities and Climate Impacts of Wildfire Smoke in the Stratosphere

Abstract The Pyrocumulonimbus (pyroCb) events over British Columbia in 2017 were observed in the lower stratosphere for about 8–10 months after the smoke injections. Several previous studies used global climate models to investigate the physical parameters for the 2017 pyroCb events, but the conclusions show strong model dependency. In this study, we use the Energy Exascale Earth System Model (E3SM) and complete an ensemble of runs exploring three injection parameters: smoke aerosol mass, the percentage of black carbon within the smoke aerosols, and plume injection height. Additionally, we consider the heterogeneous reaction of ozone and primary organic matter. According to the satellite daily observed aerosol optical depth (AOD), we find that the best ensemble member is the simulation with 0.4 Tg of smoke, 3% of which is black carbon, a 13.5 km smoke injection height, and a 10 −5 probability factor of the heterogeneous reaction. Besides AOD, we examine the ensemble score based on the metrics used in previous studies: the metrics of the extinction coefficient at 18 km altitude and the maximum plume height. The conclusion of the best estimate of the injection parameters for 2017 pyroCb events shows strong not only model but also evaluation metric dependency. We use the Random Forest machine learning technique to quantify the relative importance of each parameter in accurately simulating the 2017 pyroCb events and find that the injection height is the most critical feature, no matter which metric is used to score the ensemble members.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS2020

Designing an Optimal Ensemble Strategy for GMAO S2S Forecast System

The NASA Global Modeling and Assimilation Office (GMAO) Sub-seasonal to Seasonal (S2S) prediction system is being readied for a major upgrade. An important factor in successful extended range forecasting is the definition of the ensemble. Our overall strategy is to run a relatively large ensemble of about 40 members up to 3 months (focusing on the sub-seasonal forecast problem), after which we sub-sample the ensemble, and continue the forecast with about 10 members (up to 12 months). Here we present the results of our testing of various ways to generate the initial perturbations and the validation of a stratified sampling approach for choosing the members of the smaller ensemble. For the initialization of the ensemble we propose a combination of lagged and burst initial conditions. To generate perturbations for the burst ensemble members we used scaled differences of pairs of analysis states (chosen randomly from the corresponding season) separated by 1-10 days. We consider perturbing separately the atmosphere and the ocean, or both. By varying the separation times between the analysis states, we are able to produce perturbations that resemble well-known modes of variability. Focusing on the ENSO SST indices, we found that all types of perturbations are important for the ensemble spread with, however, considerable differences in the timing of the impacts on spread for the atmospheric and oceanic perturbations.Our initial (larger) ensemble size was determined so as to maximize the skill of predicting some of the leading modes of boreal winter atmospheric modes (namely the NAO, PNA and AO). Since it is not feasible for us to run with the larger ensemble beyond about 3 months, we employ a stratified sampling procedure that identifies the emerging directions of error growth to subset the ensemble. By comparing the results from the stratified ensemble with that of the randomly sampled ensemble of the same size, we find that the former provides substantially better estimates the mean of the original large ensemble.

Borovikov, Anna↗

DOE OSTI2020

Safety Risk Reliability Model Library

SR2ML is a software package which contains a set of safety and reliability models designed to be interfaced with the INL developed RAVEN code. These models can be employed to perform both static and dynamic system risk analysis and determine risk importance of specific elements of the considered system. Two classes of reliability models have been developed; the first class includes all classical reliability models (Fault-Trees, Event-Trees, Markov models and Reliability Block Diagrams) which have been extended to deal not only with Boolean logic values but also time dependent values. The second class includes several components aging models. Models of these two classes are designed to be included in a RAVEN ensemble model to perform time dependent system reliability analysis (dynamic analysis). Similarly, these models can be interfaced with system analysis codes to determine failure time of systems and evaluate accident progression (static analysis).

Wang, Congjian↗

NASA NTRS2008

Application of an Ensemble Smoother to Precipitation Assimilation

Assimilation of precipitation in a global modeling system poses a special challenge in that the observation operators for precipitation processes are highly nonlinear. In the variational approach, substantial development work and model simplifications are required to include precipitation-related physical processes in the tangent linear model and its adjoint. An ensemble based data assimilation algorithm "Maximum Likelihood Ensemble Smoother (MLES)" has been developed to explore the ensemble representation of the precipitation observation operator with nonlinear convection and large-scale moist physics. An ensemble assimilation system based on the NASA GEOS-5 GCM has been constructed to assimilate satellite precipitation data within the MLES framework. The configuration of the smoother takes the time dimension into account for the relationship between state variables and observable rainfall. The full nonlinear forward model ensembles are used to represent components involving the observation operator and its transpose. Several assimilation experiments using satellite precipitation observations have been carried out to investigate the effectiveness of the ensemble representation of the nonlinear observation operator and the data impact of assimilating rain retrievals from the TMI and SSM/I sensors. Preliminary results show that this ensemble assimilation approach is capable of extracting information from nonlinear observations to improve the analysis and forecast if ensemble size is adequate, and a suitable localization scheme is applied. In addition to a dynamically consistent precipitation analysis, the assimilation system produces a statistical estimate of the analysis uncertainty.

Zhang, Sara↗

DOE OSTI2021

Penalized ensemble Kalman filters for high dimensional non-linear systems

The ensemble Kalman filter (EnKF) is a data assimilation technique that uses an ensemble of models, updated with data, to track the time evolution of a usually non-linear system. It does so by using an empirical approximation to the well-known Kalman filter. However, its performance can suffer when the ensemble size is smaller than the state space, as is often necessary for computationally burdensome models. This scenario means that the empirical estimate of the state covariance is not full rank and possibly quite noisy. To solve this problem in this high dimensional regime, we propose a computationally fast and easy to implement algorithm called the penalized ensemble Kalman filter (PEnKF). Under certain conditions, it can be theoretically proven that the PEnKF will be accurate (the estimation error will converge to zero) despite having fewer ensemble members than state dimensions. Further, as contrasted to localization methods, the proposed approach learns the covariance structure associated with the dynamical system. These theoretical results are supported with simulations of several non-linear and high dimensional systems.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2023

Assessing decadal variability of subseasonal forecasts of opportunity using explainable AI

Abstract Identifying predictable states of the climate system allows for enhanced prediction skill on the generally low-skill subseasonal timescale via forecasts with higher confidence and accuracy, known as forecasts of opportunity. This study takes a neural network approach to explore decadal variability of subseasonal predictability, particularly during forecasts of opportunity. Specifically, this work quantifies subseasonal prediction skill provided by the tropics within the Community Earth System Model Version 2 (CESM2) Large Ensemble and assesses how this skill evolves on decadal timescales. Utilizing the networks’ confidence and explainable artificial intelligence, physically meaningful sources of predictability associated with periods of enhanced skill are identified. Using these networks, we find that tropically-driven subseasonal predictability varies on decadal timescales during forecasts of opportunity. Further, we investigate the drivers of the low frequency modulation of the tropical-extratropical teleconnection and discuss the implications. Analysis is extended to ECMWF Reanalysis v5 data, revealing that the relationships learned within the CESM2-Large Ensemble holds in modern reanalysis data. These results indicate that the neural networks are capable of identifying predictable decadal states of the climate system within CESM2 that are useful for making confident, accurate subseasonal precipitation predictions in the real world.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2022

Using neural network ensembles to separate ocean biogeochemical and physical drivers of phytoplankton biogeography in Earth system models

Abstract. Earth system models (ESMs) are useful tools for predicting and understanding past and future aspects of the climate system. However, the biological and physical parameters used in ESMs can have wide variations in their estimates. Even small changes in these parameters can yield unexpected results without a clear explanation of how a particular outcome was reached. The standard method for estimating ESM sensitivity is to compare spatiotemporal distributions of variables from different runs of a single ESM. However, a potential pitfall of this method is that ESM output could match observational patterns because of compensating errors. For example, if a model predicts overly weak upwelling and low nutrient concentrations, it might compensate for this by allowing phytoplankton to have a high sensitivity to nutrients. Recently, we demonstrated that neural network ensembles (NNEs) are capable of extracting relationships between predictor and target variables within ocean biogeochemical models. Being able to view the relationships between variables, along with spatiotemporal distributions, allows for a more mechanistically based examination of ESM outputs. Here, we investigated whether we could apply NNEs to help us determine why different ESMs produce different spatiotemporal distributions of phytoplankton biomass. We tested this using three cases. The first and second case used different runs of the same ESM, except that the physical circulations differed between them in the first case, while the biological equations differed between them in the second. Our results indicated that the NNEs were capable of extracting the relationships between variables for different runs of a single ESM, allowing us to distinguish between differences due to changes in circulation (which do not change relationships) from changes in biogeochemical formulation (which do change relationships). In the third case, we applied NNEs to two different ESMs. The results of the third case highlighted the capability of NNEs to contrast the apparent relationships of different ESMs and some of the challenges it presents. Although applied specifically to the ocean components of an ESM, our study demonstrates that Earth system modelers can use NNEs to separate the contributions of different components of ESMs. Specifically, this allows modelers to compare the apparent relationships across different ESMs and observational datasets.

54 ENVIRONMENTAL SCIENCES↗