Engineering topics
Liu, Siyan
Publications and source records attributed to Liu, Siyan.
ORBIT: Oak Ridge Base Foundation Model for Earth System Predictability
Earth system predictability is challenged by the complexity of environmental dynamics and the multitude of variables involved. Current AI foundation models, although advanced by leveraging large and heterogeneous data, are often constrained by their size and data integration, limiting their effectiveness in addressing the full range of Earth system prediction challenges. To overcome these limitations, we introduce the Oak Ridge Base Foundation Model for Earth System Predictability (ORBIT), an advanced vision transformer model that scales up to 113 billion parameters using a novel hybrid tensor-data orthogonal parallelism technique. As the largest model of its kind, ORBIT surpasses the current climate AI foundation model size by a thousandfold. Performance scaling tests conducted on the Frontier supercomputer have demonstrated that ORBIT achieves 684 petaFLOPS to 1.6 exaFLOPS sustained throughput, with scaling efficiency maintained at 41% to 85% across 49,152 AMD GPUs. These breakthroughs establish new advances in AI-driven climate modeling and demonstrate promise to significantly improve the Earth system predictability.
A Scalable Real-Time Data Assimilation Framework for Predicting Turbulent Atmosphere Dynamics
AI-based foundation models like FourCastNet, GraphCast are revolutionizing weather and climate predictions but are not yet ready for operational use. Their limitation lies in the absence of a data assimilation system to incorporate real-time Earth system observations, crucial for accurately forecasting events like tropical cyclones. To overcome these obstacles, we introduce a generic real-time data assimilation framework and demonstrate its end-to-end performance on the Frontier supercomputer. This framework comprises two primary modules: an ensemble score filter (EnSF), which significantly outperforms the state-of-the-art data assimilation method, and a vision transformer-based surrogate capable of real-time adaptation through the integration of observational data. We demonstrate both the strong and weak scaling of our framework up to 1024 GPUs on the Exascale supercomputer, Frontier. Our results not only illustrate the framework's exceptional scalability on high-performance computing systems, but also demonstrate the importance of supercomputers in real-time data assimilation for weather and climate predictions.
Overexpression of soybean GmDHN9 gene enhances drought resistance of transgenic Arabidopsis
Not Available
AIRES-NODE
This software package performs time series forecasting for engineering systems using neural controlled differential equations. Neural controlled differential equations are continuous time analogues to recurrent neural networks. AIRES-NODE is particularly well suited for dynamic systems which are subject to external forcing terms and time-dependent boundary conditions. AIRES-NODE provides an encoder-decoder architecture which allows for variable-length forecasts to be made and is not limited to a fixed time step size when forecasting. Additionally, by utilizing a novel combination of a variational autoencoder and neural controlled differential equations, the AIRES-NODE package is capable of producing probabilistic future forecasts.
Advancing subseasonal reservoir inflow forecasts using an explainable machine learning method
Not Available
Entropy-driven Optimal Sub-sampling of Fluid Dynamics for Developing Machine-learned Surrogates
Optimal sub-sampling of large datasets from fluid dynamics simulations is essential for training reduced-order machine learned models. A method using Shannon entropy was developed to weight flow features according to their level of information content, such that the most informative features can be extracted and used for training a surrogate model. The method is demonstrated in the canonical flow over a cylinder problem simulated with OpenFOAM. Both time-independent predictions and temporal forecasting were investigated as well as two types of prediction targets: local per-grid-point predictions and global per-time-step predictions. When tested on training a surrogate model, results indicate that our entropy-based sampling method typically outperforms random sampling and yields more reproducible results in less iterations. Finally, the method was used to train a surrogate model for modeling turbulence in magnetohydrodynamic flows, which revealed various challenges and opportunities for future research.
Explainable machine learning model for multi-step forecasting of reservoir inflow with uncertainty quantification
We propose an explainable machine learning (ML) model with uncertainty quantification (UQ) to improve multi-step reservoir inflow forecasting. Traditional ML methods have challenges in forecasting inflows multiple days ahead, and lack explainability and UQ. To address these limitations, we introduce an encoder–decoder long short-term memory (ED-LSTM) network for multi-step forecasting, employ the SHapley Additive exPlanation (SHAP) technique for understanding the influence of hydrometeorological factors on inflow prediction, and develop a novel UQ method for prediction trustworthiness. We apply these methods to forecast 7-day inflow in snow-dominant and rain-driven reservoirs. The results demonstrate the effectiveness of the ED-LSTM model, with high forecasting accuracy for short lead times. Our UQ method provides reliable uncertainty estimates, covering 90% of data with a 90% confidence level. The SHAP analysis reveals the importance of historical inflow and precipitation as influential factors. These findings and methods may support reservoir operators in optimizing water resources management decisions.
Uncertainty quantification of the convolutional neural networks on permeability estimation from micro-CT scanned sandstone and carbonate rock images
Rock permeability is one of the most crucial properties affecting subsurface fluid flow behaviors. To accurately and robustly estimate the permeability, Digital Rock Physics, including micro-CT scanning technology and direct flow simulations on scanned images, has prevailed in recent years. Besides, machine learning techniques such as convolutional neural networks (CNNs) have been widely adopted and achieved success in permeability estimations directly from rock images. However, existing ML methods used for permeability estimation from rock images lack uncertainty quantification that causes unreliable predictions and overconfident estimations on out-of-distribution (OOD) samples. Here, in this work, we propose a PI3NN-CNN framework to address this problem. PI3NN-CNN consists of a CNN model for absolute permeability estimation and a PI3NN method to quantify the estimation uncertainty. It is able to quantify the uncertainty for in-distribution (InD) data with a desired confidence level, and identify OOD samples to avoid overconfident predictions. We demonstrate the method using micro-CT scanned images from two sandstone and two carbonate rocks. We found that PI3NN-CNN generates accurate predictions for InD samples, while producing high-quality prediction uncertainties regardless of the prediction accuracy. Meanwhile, PI3NN-CNN identifies OOD samples using its special network initialization scheme. The unique feature of PI3NN-CNN makes it applicable to more complex real-world image-based data for robust learning and predictions without overconfident estimations when the ground-truth information is unavailable.
Knowledge-Informed Uncertainty-Aware Machine Learning for Time Series Forecasting of Dynamical Engineered Systems
The high complexity and multiscale nature of many engineered systems—such as those in nuclear power plants—make representing and forecasting their dynamic behavior challenging. Physics-based models can be overly complex and computationally intractable, whereas machine learning (ML) tools are often data-hungry and prone to unphysical solutions. This study proposes a knowledge-informed ML-aided hybrid residual modeling approach that offers accurate and efficient time series forecasting for the operation of dynamical engineered systems. Hybrid residual modeling entails a baseline solution from domain knowledge and known physics expressions about the system dynamics integrated with an ML model to capture undiscovered information from the mismatch (i.e., residuals) between true states from measurements and baseline-predicted outputs. This study further quantifies the ML model uncertainty to provide trustworthy solutions. Real-time operational data from thermal-hydraulic flow loops of the cryogenic moderator system in Oak Ridge National Laboratory’s Spallation Neutron Source facility were used to demonstrate the potential of knowledge-informed uncertainty-aware ML in real-world applications. The state variables of the cryogenic helium loop were modeled with (1) first principles–based system identification (sysID), (2) long short-term memory (LSTM) neural network, and (3) hybrid sysID (baseline) + LSTM (residual). The superior predictive capability of the sysID+LSTM model versus stand-alone sysID and LSTM is confirmed by average performance metrics and individual data points across different prediction horizons. By creating a robust representation of the underlying physical system, the widely applicable hybrid residual modeling approach will enable the future development of digital twins for performance prediction, prognostics, and operation control.
Uncertainty quantification of machine learning models to improve streamflow prediction under changing climate and environmental conditions
Machine learning (ML) models, and Long Short-Term Memory (LSTM) networks in particular, have demonstrated remarkable performance in streamflow prediction and are increasingly being used by the hydrological research community. However, most of these applications do not include uncertainty quantification (UQ). ML models are data driven and can suffer from large extrapolation errors when applied to changing climate/environmental conditions. UQ is required to quantify the influence of data noises on model predictions and avoid overconfident projections in extrapolation. In this work, we integrate a novel UQ method, called PI3NN, with LSTM networks for streamflow prediction. PI3NN calculates Prediction Intervals by training 3 Neural Networks. It can precisely quantify the predictive uncertainty caused by the data noise and identify out-of-distribution (OOD) data in a non-stationary condition to avoid overconfident predictions. We apply the PI3NN-LSTM method in the snow-dominant East River Watershed in the western US and in the rain-driven Walker Branch Watershed in the southeastern US. Results indicate that for the prediction data which have similar features as the training data, PI3NN precisely quantifies the predictive uncertainty with the desired confidence level; and for the OOD data where the LSTM network fails to make accurate predictions, PI3NN produces a reasonably large uncertainty indicating that the results are not trustworthy and should avoid overconfidence. PI3NN is computationally efficient, robust in performance, and generalizable to various network structures and data with no distributional assumptions. It can be broadly applied in ML-based hydrological simulations for credible prediction.
Coupled Lattice Boltzmann Modeling Framework for Pore-Scale Fluid Flow and Reactive Transport
In this paper, we propose a modeling framework for pore-scale fluid flow and reactive transport based on a coupled lattice Boltzmann model (LBM). We develop a modeling interface to integrate the LBM modeling code parallel lattice Boltzmann solver and the PHREEQC reaction solver using multiple flow and reaction cell mapping schemes. The major advantage of the proposed workflow is the high modeling flexibility obtained by coupling the geochemical model with the LBM fluid flow model. Consequently, the model is capable of executing one or more complex reactions within desired cells while preserving the high data communication efficiency between the two codes. Meanwhile, the developed mapping mechanism enables the flow, diffusion, and reactions in complex pore-scale geometries. We validate the coupled code in a series of benchmark numerical experiments, including 2D single-phase Poiseuille flow and diffusion, 2D reactive transport with calcite dissolution, as well as surface complexation reactions. The simulation results show good agreement with analytical solutions, experimental data, and multiple other simulation codes. In addition, we design an AI-based optimization workflow and implement it on the surface complexation model to enable increased capacity of the coupled modeling framework. Compared to the manual tuning results proposed in the literature, our workflow demonstrates fast and reliable model optimization results without incorporating pre-existing domain knowledge.
A scalable transformer model for real-time decision making in neutron scattering experiments
The U.S. Department of Energy's (DOE's) neutron research facilities at Oak Ridge National Laboratory (ORNL), including the High Flux Isotope Reactor (HFIR) and the Spallation Neutron Source (SNS), are a state-of-the-art neutron scattering facility that allows researchers to study the structure and dynamics of materials at the atomic scale. At the SNS, neutrons are measured using the time-of-flight (TOF) technique as they move through a neutron beamline to interact with a sample. Large volumes of neutron scattering data are collected and recorded in neutron event mode. Optimal productivity of the TOF instrument is limited due to the lack of real-time data analysis tools. The large amount of data generated by the experiments can be challenging to process and analyze in real time, particularly for experiments that require rapid feedback and adjustment of experimental parameters. The regular computer/workstation cannot keep up with the experiment speed to provide real-time feedback to adjust experimental parameters, so connecting the supercomputers available to the neutron facility is necessary to achieve real-time data analysis and experiment steering. To address this challenge, we exploit the Frontier supercomputer at Oak Ridge Leadership Computing Facility (OLCF) to train a scalable temporal fusion transformer model for real-time decision making of TOF neutron scattering experimentation. Here, in this paper, we present the results using Frontier to provide the processing power needed to rapidly process and analyze large volumes of single-crystal diffraction data collected at TOPAZ, a neutron time-of-flight Laue single-crystal diffractometer at the SNS.
Investigation of hydrometeorological influences on reservoir releases using explainable machine learning methods
Long short-term memory (LSTM) networks have demonstrated successful applications in accurately and efficiently predicting reservoir releases from hydrometeorological drivers including reservoir storage, inflow, precipitation, and temperature. However, due to its black-box nature and lack of process-based implementation, we are unsure whether LSTM makes good predictions for the right reasons. In this work, we use an explainable machine learning (ML) method, called SHapley Additive exPlanations (SHAP), to evaluate the variable importance and variable-wise temporal importance in the LSTM model prediction. In application to 30 reservoirs over the Upper Colorado River Basin, United States, we show that LSTM can accurately predict the reservoir releases with NSE ≥ 0.69 for all the considered reservoirs despite of their diverse storage sizes, functionality, elevations, etc. Additionally, SHAP indicates that storage and inflow are more influential than precipitation and temperature. Moreover, the storage and inflow show a relatively long-term influence on the release up to 7 days and this influence decreases as the lag time increases for most reservoirs. These findings from SHAP are consistent with our physical understanding. However, in a few reservoirs, SHAP gives some temporal importances that are difficult to interpret from a hydrological point of view, probably because of its ignorance of the variable interactions. SHAP is a useful tool for black-box ML model explanations, but the hydrological processes inferred from its results should be interpreted cautiously. More investigations of SHAP and its applications in hydrological modeling is needed and will be pursued in our future study.
A deep learning-based direct forecasting of CO 2 plume migration
Accurate and timely forecasts of CO 2 plume evolution in geological reservoirs are crucial for CO 2 migration detection, leakage risk assessment, and operation decision support. Conventional forecasting usually adopts a two-step strategy, first calibrating reservoir model parameters against observations using iterative inverse modeling (or history matching) and then applying the calibrated model for predictions. This method impedes real-time forecasts due to the heavy computational demand in inverse modeling and may suffer from poor prediction accuracy because of the limited observation data. In this work, we propose a deep learning-based latent space mapping framework to forecast CO 2 plume migration directly by avoiding the inverse modeling. We first use the convolutional autoencoder to map the high-dimensional complex plume extents onto low-dimensional latent space. Next, we use neural networks to learn the relationship between the observation variables and the prediction latent variables. And then for given observation data, we infer the prediction values directly. This one-step direct forecasting is computationally efficient which requires a few number of parallelizable reservoir simulations and it can provide accurate predictions with limited observations by learning the observation-prediction relationship in the reduced dimension. Therefore, our proposed method enables an in-time forecast of dynamic CO 2 plume distributions. In this work, we demonstrate the effectiveness and accuracy of our method in predicting the CO 2 plume migration using four metrics such as plume area, centroid movement distance, and plume spreading in the primary and secondary directions. And the spatio-temporal evolution patterns of plume migration under diverse geological complexities are also accurately quantified.
Identifying Hydrometeorological Factors Influencing Reservoir Releases Using Machine Learning Methods
Simulation of reservoir releases plays a critical role in social-economic functioning and our nation's security. How-ever, it is challenging to predict the reservoir release accurately because of many influential factors from natural environments and engineering controls such as the reservoir inflow and storage. Moreover, climate change and hydrological intensification causing the extreme precipitation and temperature make the accurate prediction of reservoir releases even more challenging. Machine learning (ML) methods have shown some successful applications in simulating reservoir releases. However, previous studies mainly used inflow and storage data as inputs and only considered their short-term influences (e.g, previous one or two days). In this work, we use long short-term memory (LSTM) networks for reservoir release prediction based on four input variables including inflow, storage, precipitation, and temperature and consider their long-term influences. We apply the LSTM model to 30 reservoirs in Upper Colorado River Basin, United States. We analyze the prediction performance using six statistical metrics. More importantly, we investigate the influence of the input hydrometeorological factors, as well as their temporal effects on reservoir release decisions. Results indicate that inflow and storage are the most influential factors but the inclusion of precipitation and temperature can further improve the prediction of release especially in low flows. Additionally, the inflow and storage have a relatively long-term effect on the release. These findings can help optimize the water resources management in the reservoirs.
Improving net ecosystem CO2 flux prediction using memory-based interpretable machine learning
Terrestrial ecosystems play a central role in the global carbon cycle and affect climate change. However, our predictive understanding of these systems is still limited due to their complexity and uncertainty about how key drivers and their legacy effects influence carbon fluxes. Here, we propose an interpretable Long Short-Term Memory (iLSTM) network for predicting net ecosystem CO 2 exchange (NEE) and interpreting the influence on the NEE prediction from environmental drivers and their memory effects. We consider five drivers and apply the method to three forest sites in the United States. Besides performing the prediction in each site, we also conduct transfer learning by using the iLSTM model trained in one site to predict at other sites. Results show that the iLSTM model produces good NEE predictions for all three sites and, more importantly, it provides reasonable interpretations on the input driver's importance as well as their temporal importance on the NEE prediction. Additionally, the iLSTM model demonstrates good across-site transferability in terms of both prediction accuracy and interpretability. The transferability can improve the NEE prediction in unobserved forest sites, and the interpretability advances our predictive understanding and guides process-based model development.
An interpretable machine learning model for advancing terrestrial ecosystem predictions
We apply an interpretable Long Short-Term Memory (iLSTM) network for land-atmosphere carbon flux predictions based on time series observations of seven environmental variables. iLSTM enables interpretability of variable importance and variable-wise temporal importance to the prediction of targets by exploring internal network structures. The application results indicate that iLSTM not only improves prediction performance by capturing different dynamics of individual variables, but also reasonably interprets the different contribution of each variable to the target and its different temporal relevance to the target. This variable and temporal importance interpretation of iLSTM advances terrestrial ecosystem model development as well as our predictive understanding of the system.