Engineering topics
Publications and source records attributed to Lu, Dan.
Focal Area(s): (2) Predictive modeling through the use of AI techniques and AI-derived model components: This white paper focuses on using artificial intelligence (AI) to discover connections across processes and scales and to represent those processes in predictive models of anthropogenic impacts on the water cycle.
Focal Area: (2) Predictive modeling through the use of AI techniques and AI-derived model components. (3) Insight gleaned from complex data using AI, big data analytics, and other advanced methods. We propose an AI-enabled model-experiment (MODEX) framework to improve the predictability of subsurface water storage (SWS) from local to conus scales in a changing environment by taking advantage of DOE’s observation and simulation capabilities, as well as to inform the model and the observation development.
Focal Area(s): The white paper focuses on areas 2 and 3. The proposed deep learning modeling approach can provide forecasting capabilities and mechanism-based understanding of terrestrial and aquatic hydro- biogeochemical processes at watershed and river basin scales. Knowledge gleaned here can also provide insights for Earth System Models.
While detailed chemical kinetic models have been successful in representing rates of chemical reactions in continuum scale computational fluid dynamics (CFD) simulations, applying the models in simulations for engineering device conditions is computationally prohibitive. To reduce the cost, data-driven methods, e.g., autoencoders, have been used to construct reduced chemical kinetic models for CFD simulations. Despite their success, data-driven methods rely heavily on training data sets and can be unreliable when used in out-of-distribution (OOD) regions (i.e., when extrapolating outside of the training set). In this paper, we present an enhanced autoencoder model for combustion chemical kinetics with uncertainty quantification to enable the detection of model usage in OOD regions, and thereby creating an OOD-aware autoencoder model that contributes to more robust CFD simulations of reacting flows.Here, we first demonstrate the effectiveness of the method in OOD detection in two well-known datasets, MNIST and Fashion-MNIST, in comparison with the deep ensemble method, and then present the OOD-aware autoencoder for reduced chemistry model in syngas combustion.
Hydrologic predictions at rural watersheds are important but also challenging due to data shortage. Long short-term memory (LSTM) networks are a promising machine learning approach and have demonstrated good performance in streamflow predictions. However, due to its data-hungry nature, most LSTM applications focus on well-monitored catchments with abundant and high-quality observations. In this work, we investigate predictive capabilities of LSTM in poorly monitored watersheds with short observation records. To address three main challenges of LSTM applications in data-scarce locations, i.e., overfitting, uncertainty quantification (UQ), and out-of-distribution prediction, we evaluate different regularization techniques to prevent overfitting, apply a Bayesian LSTM for UQ, and introduce a physics-informed hybrid LSTM to enhance out-of-distribution prediction. Through case studies in two diverse sets of catchments with and without snow influence, we demonstrate that 1) when hydrologic variability in the prediction period is similar to the calibration period, LSTM models can reasonably predict daily streamflow with Nash–Sutcliffe efficiency above 0.8, even with only 2 years of calibration data; 2) when the hydrologic variability in the prediction and calibration periods is dramatically different, LSTM alone does not predict well, but the hybrid model can improve the out-of-distribution prediction with acceptable generalization accuracy; 3) L2 norm penalty and dropout can mitigate overfitting, and Bayesian and hybrid LSTM have no overfitting; and 4) Bayesian LSTM provides useful uncertainty information to improve prediction understanding and credibility. In conclusion, these insights have vital implications for streamflow simulation in watersheds where data quality and availability are a critical issue.
Predictive modeling through the use of AI techniques and AI-derived model components; the use of AI and other tools to design a prediction system comprising of a hierarchy of models (e.g.,AI driven model/component/parameterization selection). We propose a Bayesian neural network ensemble approach for improving large-scale streamflow predictability and understanding in a changing environment that combines multiple Earth system land model predictions by calculating spatiotemporally varying model weights and biases while accounting for various types of observations at multiple scales with uncertainty.
Sensitivity analysis in terrestrial ecosystem modeling is important for understanding controlling processes, guiding model development, and targeting new observations to reduce parameter and prediction uncertainty. Complex and computationally expensive terrestrial ecosystem models (TEM) limit the number of ensemble simulations, requiring sophisticated and efficient methods to analyze sensitivities of multiple model responses to different types of parameter uncertainties. In this study, we propose a distance-based global sensitivity analysis (DGSA) method. DGSA first classifies model response samples into a small set of discrete classes and then calculates the distance between parameter frequency distributions in different classes to measure the parameter sensitivity. The principle is that, if the parameter distribution is the same in each class, then the model response is insensitive to the parameter, while a large difference in the distributions indicates the parameter is influential to the response. Built on this idea, DGSA can be applied to analyze sensitivity of a single and a group of responses to different kinds of parameter uncertainties including continuous, discrete and even stochastic. Besides the main-effect sensitivity from a single parameter, DGSA can also quantify the sensitivity from parameter interactions. Additionally, DGSA is computationally efficient which can use a small number of model evaluations to obtain an accurate and statistically significant result. We applied DGSA to two TEMs, one having eight parameters and three kinds of model responses, and the other having 47 parameters and a long-period response. We demonstrated that DGSA can be used for sensitivity problems with multiple responses and high-dimensional parameters efficiently.
Abstract Mechanistic photosynthesis models are at the heart of terrestrial biosphere models (TBMs) simulating the daily, monthly, annual and decadal rhythms of carbon assimilation ( A ). These models are founded on robust mathematical hypotheses that describe how A responds to changes in light and atmospheric CO 2 concentration. Two predominant photosynthesis models are in common usage: Farquhar (FvCB) and Collatz (CBGB). However, a detailed quantitative comparison of these two models has never been undertaken. In this study, we unify the FvCB and CBGB models to a common parameter set and use novel multi‐hypothesis methods (that account for both hypothesis and parameter variability) for process‐level sensitivity analysis. These models represent three key biological processes: carboxylation, electron transport, triose phosphate use (TPU) and an additional model process: limiting‐rate selection. Each of the four processes comprises 1–3 alternative hypotheses giving 12 possible individual models with a total of 14 parameters. To broaden inference, TBM simulations were run and novel, high‐resolution photosynthesis measurements were made. We show that parameters associated with carboxylation are the most influential parameters but also reveal the surprising and marked dominance of the limiting‐rate selection process (accounting for 57% of the variation in A vs. 22% for carboxylation). The limiting‐rate selection assumption proposed by CBGB smooths the transition between limiting rates and always reduces A below the minimum of all potentially limiting rates, by up to 25%, effectively imposing a fourth limitation on A . Evaluation of the CBGB smoothing function in three TBMs demonstrated a reduction in global A by 4%–10%, equivalent to 50%–160% of current annual fossil fuel emissions. This analysis reveals a surprising and previously unquantified influence of a process that has been integral to many TBMs for decades, highlighting the value of multi‐hypothesis methods.
Incomplete representations of physical processes often lead to structural errors in process-based (PB) hydrologic models. Machine learning (ML) algorithms can reduce streamflow modeling errors but do not enforce physical consistency. As a result, ML algorithms may be unreliable if used to provide future hydroclimate projections where climates and land use patterns are outside the range of training data. Here we test hybrid models built by integrating PB model outputs with a ML algorithm known as Long Short-Term Memory (LSTM) network on their ability to simulate streamflow in 531 catchments representing diverse conditions across the Conterminous United States. Model performance of hybrid models as measured by Nash-Sutcliffe efficiency (NSE) improved relative to standalone PB and LSTM models. More importantly, hybrid models provide highest improvement in catchments where PB models fail completely (i.e., NSE < 0). However, all models performed poorly in catchments with extended low flow periods, suggesting need for additional research.
Raw and processed output from the Multi-Assumption Architecture & Testbed (MAAT, https://github.com/walkeranthonyp/MAAT, tag: v1.2.1_walker2020; Walker et al., 2018). The dataset contains leaf-scale carbon assimilation data from three MAAT simulations and their post-processing and plotting scripts. Simulation 1, labelled "FvC_4proc_proc," is a four-process, process sensitivity analysis (Dai et al., 2017) of carbon assimilation based on the Farquhar et al. (1980) photosynthesis model, and its variant proposed by Collatz et al. (1991). Simulation 2, labelled "FvC_4proc_salt," is a 14 parameter sensitivity analysis (Saltelli et al., 2010) of carbon assimilation based on the Farquhar and Collatz photosynthesis models. Simulation 3, labelled "FvC," is a factorial combination of the alternative process representations and parameter values presented in those original papers across a factorial combination of incident photosynthetically active radiation (50-2000 µmol m-2 s-1, in increments of 50) and atmospheric CO2 concentration (50-1500 µmol m-2 s-1, in increments of 50). Additional scripts/functions for post-processing and plotting are part of the MAAT repository. These data were generated to evaluate and compare the Farquhar and Collatz models of leaf-scale photosynthesis across a range of CO2 and incident light conditions. Analysis of results can be found in Walker et al. (2020). This dataset is comprised of R scripts to process data and produce figures; .RDS files, a binary format specific to R and read with the 'readRDS' function, to store raw MAAT output; R and bash scripts (.bs) that help to run a MAAT simulation, .XML files that are output from MAAT describing a run, and .csv files which contain the calculated sensitivity indexes published in Walker et al. (2020).
Despite their wide and successful applications, deep learning (DL) models are prone to overfitting for small training datasets, produce a poor predictive performance for uncertain data, and provide point estimations without any indication of the accuracy and credibility. These limitations of the deterministic DL models hinder their effective application in Earth system science where the labelled data are sparse, noisy and incomplete with large uncertainty and where the predictive uncertainty quantification is needed for scientific understanding and policy decision making. Integration of Bayesian inference into DL models adds an estimate of uncertainty and regularization in the predictions. However, traditional Bayesian methods are computationally unaffordable and inflexible for high-dimensional problems, which limits their application to DL systems that typically have millions of model parameters. In this effort, we propose an efficient and general-purpose Bayesian inference method to advance DL model optimization and uncertainty quantification, so as to facilitate the adoption of DL in Earth sciences. In a demonstration, we integrate the proposed Bayesian method with a feedforward neural network (NN) to build a fast-to-evaluate surrogate of the complex Energy Exascale Earth System Land Model for efficient modeling. The formulated Bayesian NN, using a small number of training data, produces an accurate prediction with high credibility, whereas with the same small training size, the deterministic NN cannot yield a reasonable estimation and does not provide confidence information. The proposed Bayesian method is computationally efficient and flexible, capable of integration with diverse network variants such as convolutional NNs and recurrent NNs to advance the application of DL in Earth sciences.
Ecosystem model prediction is important for understanding ecosystem responses to climate change and for management support. Model prediction and quantification of predictive uncertainty of ecosystems have long been investigated. The traditional workflow, which calibrates models to match observations and then uses the calibrated models for predictions, relies heavily on inverse modeling to constrain uncertain parameters in complex forward models. This inversion-based prediction approach is infeasible for complex models with heterogeneous parameter uncertainties and incapable of rapid integration of streaming and multiple sources of data because of the difficulty and computational cost in the model inversion, which is typically ill-posed and can require hundreds of thousands of expensive forward simulations to be performed iteratively. We propose to circumvent inverse modeling by precomputing an ensemble of unconstrained forward simulations and then using machine learning (ML) methods to learn the statistical relationship between simulated observation and prediction quantities. Once the ML model has learned the relationship, it can be used to make predictions of future system behavior with uncertainty quantification based on observations. The proposed learningbased inversion-free model prediction (LIMP) framework is computationally efficient which only requires a few thousands of fully parallelizable forward simulations. Additionally, LIMP can continually update predictions based on streaming observations from multiple locations and sources without necessarily requiring extra model simulations. In this study, we apply LIMP to a regional terrestrial ecosystem model with 47 parameters for testing, refining, and evaluating the approach. We demonstrate that LIMP can be used for efficient model prediction, rapid data assimilation, and cost-effective experimental design for improving robust predictive understanding of ecosystems.
Abstract. Improving predictive understanding of Earth system variability and change requires data–model integration. Efficient data–model integration for complex models requires surrogate modeling to reduce model evaluation time. However, building a surrogate of a large-scale Earth system model (ESM) with many output variables is computationally intensive because it involves a large number of expensive ESM simulations. In this effort, we propose an efficient surrogate method capable of using a few ESM runs to build an accurate and fast-to-evaluate surrogate system of model outputs over large spatial and temporal domains. We first use singular value decomposition to reduce the output dimensions and then use Bayesian optimization techniques to generate an accurate neural network surrogate model based on limited ESM simulation samples. Our machine-learning-based surrogate methods can build and evaluate a large surrogate system of many variables quickly. Thus, whenever the quantities of interest change, such as a different objective function, a new site, and a longer simulation time, we can simply extract the information of interest from the surrogate system without rebuilding new surrogates, which significantly reduces computational efforts. We apply the proposed method to a regional ecosystem model to approximate the relationship between eight model parameters and 42 660 carbon flux outputs. Results indicate that using only 20 model simulations, we can build an accurate surrogate system of the 42 660 variables, wherein the consistency between the surrogate prediction and actual model simulation is 0.93 and the mean squared error is 0.02. This highly accurate and fast-to-evaluate surrogate system will greatly enhance the computational efficiency of data–model integration to improve predictions and advance our understanding of the Earth system.
A collection of scientific analyses, metrics, and visualizations for robustvalidation of ice sheet models is presented using the Land Ice Verificationand Validation toolkit (LIVVkit), version 2.1, and the LIVVkit Extensionsrepository (LEX), version 0.1. This software collection targets stand-aloneice sheet or coupled Earth system models, and handles datasets and analysesthat require high-performance computing and storage. LIVVkit aims to enableefficient and fully reproducible workflows for postprocessing, analysis, andvisualization of observational and model-derived datasets in a shareableformat, whereby all data, methodologies, and output are distributed to usersfor evaluation. Extending from the initial LIVVkit software framework, wedemonstrate Greenland ice sheet simulation validation metrics using thecoupled Community Earth System Model (CESM) as well as an idealizedstand-alone high-resolution Community Ice Sheet Model, version 2 (CISM2),coupled to the Albany/FELIX velocity solver (CISM-Albany or CISM-A). As oneexample of the capability, LIVVkit analyzes the degree to which modelscapture the surface mass balance (SMB) and identifies potential sources ofbias, using recently available in situ and remotely sensed data ascomparison. Related fields within atmosphere and land surface models, e.g.,surface temperature, radiation, and cloud cover, are also diagnosed. Appliedto the CESM1.0, LIVVkit identifies a positive SMB bias that is focusedlargely around Greenland's southwest region that is due to insufficientablation.
We present that in numerical modeling of geological carbon sequestration (GCS), uncertainty quantification (UQ) is usually needed to evaluate the impact of uncertain model parameters on model predictions caused by limited measurements and incomplete knowledge of the parameters. However, UQ for GCS is computationally expensive due to the large ensemble of complex and lengthy model simulations. In this study, we propose an adaptive Kriging method to build a fast-to-evaluate surrogate of the GCS model to alleviate the heavy computational burden. The surrogate model is efficiently generated using a Taylor expansion-based adaptive experimental design algorithm that combines a distance-based exploration criterion and an exploitation criterion to adaptively search for informative training samples. In addition, we analyze the uncertainty brought by substituting the surrogate for the actual simulation model and explore its influence on UQ results. Lastly, our method is demonstrated in a synthetic GCS model and its performance is evaluated in comparison with the conventional Monte Carlo sampling. Results indicate that our method can greatly improve the computational efficiency in UQ and provide an effective and reliable UQ solution with the consideration of surrogate uncertainty.
Abstract. Here, computer models are ubiquitous tools used to represent systems across many scientific and engineering domains. For any given system, many computer models exist, each built on different assumptions and demonstrating variability in the ways in which these systems can be represented. This variability is known as epistemic uncertainty, i.e. uncertainty in our knowledge of how these systems operate. Two primary sources of epistemic uncertainty are (1) uncertain parameter values and (2) uncertain mathematical representations of the processes that comprise the system. Many formal methods exist to analyse parameter-based epistemic uncertainty, while process-representation-based epistemic uncertainty is often analysed post hoc, incompletely, informally, or is ignored. In this model description paper we present the multi-assumption architecture and testbed (MAAT v1.0) designed to formally and completely analyse process-representation-based epistemic uncertainty. MAAT is a modular modelling code that can simply and efficiently vary model structure (process representation), allowing for the generation and running of large model ensembles that vary in process representation, parameters, parameter values, and environmental conditions during a single execution of the code. MAAT v1.0 approaches epistemic uncertainty through sensitivity analysis, assigning variability in model output to processes (process representation and parameters) or to individual parameters. In this model description paper we describe MAAT and, by using a simple groundwater model example, verify that the sensitivity analysis algorithms have been correctly implemented. The main system model currently coded in MAAT is a unified, leaf-scale enzyme kinetic model of C 3 photosynthesis. In the Appendix we describe the photosynthesis model and the unification of multiple representations of photosynthetic processes. The numerical solution to leaf-scale photosynthesis is verified and examples of process variability in temperature response functions are provided. For rapid application to new systems, the MAAT algorithms for efficient variation of model structure and sensitivity analysis are agnostic of the specific system model employed. Therefore MAAT provides a tool for the development of novel or toy models in many domains, i.e. not only photosynthesis, facilitating rapid informal and formal comparison of alternative modelling approaches.
Abstract Calibration of the Energy Exascale Earth System Model (E3SM), land model (ELMv0) is challenging because of its model complexity, strong model nonlinearity, and significant computational requirements. Therefore, only a limited number of simulations can be allowed in any attempt to find a near‐optimal solution within an affordable time. The goal of this study is to calibrate some of the ELMv0 parameters to improve model projection of carbon fluxes. We propose a computationally efficient global optimization procedure using sparse‐grid based surrogates. We first use advanced sparse grid (SG) interpolation to construct a surrogate system of the ELMv0, and then calibrate the surrogate model in the optimization process. As the surrogate model is a polynomial whose evaluation is fast, it can be efficiently evaluated a sufficiently large number of times in the optimization, which facilitates the global search. We calibrate eight parameters against five years of net ecosystem exchange, total leaf area index, and latent heat flux data from the U.S. Missouri Ozark flux tower. The calibrated model is then used for predicting the three variables in the following 4 years. The results indicate that an accurate surrogate model can be created for the ELMv0 with a relatively small number of SG points, i.e., a few ELMv0 simulations that can be fully parallel. And, the application of the optimized parameters leads to a better model performance and a higher predictive capability than the default parameter values in the ELMv0.
Multiphase flow and transport processes in the subsurface environment are extremely important in a number of industrial and environmental applications at various spatial and temporal scales. Thus, it is necessary to identify, understand, and predict these processes to improve the production of conventional and unconventional oil and gas, to increase the safety of geological sequestration of carbon dioxide and nuclear waste disposal, and to make remediation of contaminated aquifers more effective.