Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “dynamic model learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

From sequence to protein structure and conformational dynamics with artificial intelligence/machine learning

The 2024 Nobel Prize in Chemistry was awarded in part for de novo protein structure prediction using AlphaFold2, an artificial intelligence/machine learning (AI/ML) model trained on vast amounts of sequence and three-dimensional structure data. AlphaFold2 and related models, including RoseTTAFold and ESMFold, employ specialized neural network architectures driven by attention mechanisms to infer relationships between sequence and structure. At a fundamental level, these AI/ML models operate on the long-standing hypothesis that the structure of a protein is determined by its amino acid sequence. More recently, AlphaFold2 has been adapted for the prediction of multiple protein conformations by subsampling multiple sequence alignments. Herein, we provide an overview of the deterministic relationship between sequence and structure, which was hypothesized over half a century ago with profound implications for the biological sciences ever since. We postulate that protein conformational dynamics are also determined, at least in part, by amino acid sequence and that this relationship may be leveraged for construction of AI/ML models dedicated to predicting protein conformational ensembles. Accordingly, we describe a conceptual model architecture, which may be trained on sequence data in combination with conformationally sensitive structural information, coming primarily from nuclear magnetic resonance (NMR) spectroscopy. Notwithstanding certain limitations in this context, NMR offers abundant structural heterogeneity conducive to conformational ensemble prediction. As NMR and other data continue to accumulate, sequence-informed prediction of protein structural dynamics with AI/ML has the potential to emerge as a transformative capability across the biological sciences.

Artificial intelligence↗
DOE OSTI2021

Simple, low-cost and accurate data-driven geophysical forecasting with learned kernels

Modelling geophysical processes as low-dimensional dynamical systems and regressing their vector field from data is a promising approach for learning emulators of such systems. We show that when the kernel of these emulators is also learned from data (using kernel flows, a variant of cross-validation), then the resulting data-driven models are not only faster than equation-based models but are easier to train than neural networks such as the long short-term memory neural network. In addition, they are also more accurate and predictive than the latter. When trained on geophysical observational data, for example the weekly averaged global sea-surface temperature, considerable gains are also observed by the proposed technique in comparison with classical partial differential equation-based models in terms of forecast computational cost and accuracy. When trained on publicly available re-analysis data for the daily temperature of the North American continent, we observe significant improvements over classical baselines such as climatology and persistence-based forecast techniques. Although our experiments concern specific examples, the proposed approach is general, and our results support the viability of kernel methods (with learned kernels) for interpretable and computationally efficient geophysical forecasting for a large diversity of processes.

58 GEOSCIENCES↗
DOE OSTI2020

Deep anomaly detection for industrial systems: a case study

We explore the use of deep neural networks for anomaly detection of industrial systems where the data are multivariate time series measurements. We formulate the problem as a self-supervised learning where data under normal operation are used to train a deep neural network autoregressive model, i.e., use a window of time series data to predict future data values. The aim of such a model is to learn to represent the system dynamic behavior under normal conditions, while expect higher model vs. measurement discrepancies under faulty conditions. In real world applications, many control settings are discrete in nature. In this paper, vector embedding and joint losses are employed to deal with such situations. Both LSTM and CNN based deep neural network backbones are studied on the Secure Water Treatment (SWaT) testbed datasets. Also, Support Vector Data Description (SVDD) method is adapted to such anomaly detection settings with deep neural networks. Evaluation methods and results are discussed based on the SWaT dataset along with potential pitfalls.

anomaly detection, deep neural network↗
DOE OSTI2025

Correlated dynamic disorder, octahedral tilts, and acoustic phonon softening in CsSnBr 3 and CsPbBr 3

Metal halide perovskites (MHPs) have emerged as highly promising materials for optoelectronic applications, with all-inorganic MHPs presenting enhanced stability compared to their hybrid counterparts. Here, in this study, we investigate the atomic dynamics and structural fluctuations in single crystals of CsSnBr⁢ 3 and CsPbBr⁢ 3 through systematic inelastic neutron scattering (INS) measurements as a function of temperature. Our experiments are compared with first-principle simulations, augmented with large-scale molecular dynamics modeling, based on machine-learned neural network potentials. Through both INS and simulations, we find quasi-elastic diffuse rods in reciprocal space in both compounds, originating from fluctuating planar domains featuring correlated tilts of Br octahedron. The diffuse rods exhibit a slow, overdamped dynamic process, modulated across 𝑸 space, reflecting the strong lattice anharmonicity of the inorganic framework. We do not find evidence for dynamic off-centering of the Sn 2+ ions besides phonon vibrations at the center of the Br octahedron. These results offer valuable insights into the unusual anharmonic atomic dynamics and intricate correlated structural distortions in MHPs, which will be critical for rationalizing and further tailoring their thermal and optoelectronic properties.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Spatio-temporal Fourier Transformer for Long-term Dynamics Prediction (StFT) v1.0

We propose a novel machine learning model spatio-temporal Fourier transformer (StFT) to emulate long-term dynamics of multi-scale and multi-physics systems. Our method StFT overcomes the limitations of rapid error accumulation, particularly in long-term forecasting of systems characterized by complex and coupled dynamics. StFT achieves outstanding accuracy and computational efficiency by effectively capturing multi-scale interactions, and quantify the uncertainties inherent in the predictions. Our model leverages a structured hierarchy of StFT blocks, and explicitly captures dynamics across both macro- and micro- spatial scales. Evaluations conducted on three benchmark datasets (plasma, fluid, and atmospheric dynamics) demonstrate the advantages of our approach over state-of-the-art ML methods.

Bai, Zhe [Lawrence Berkeley National Laboratory (L↗
DOE OSTI2020

Machine learning Frenkel Hamiltonian parameters to accelerate simulations of exciton dynamics

In this manuscript, we develop multiple machine learning (ML) models to accelerate a scheme for parameterizing site-based models of exciton dynamics from all-atom configurations of condensed phase sexithiophene systems. This scheme encodes the details of a system’s specific molecular morphology in the correlated distributions of model parameters through the analysis of many single-molecule excited-state electronic-structure calculations. These calculations yield excitation energies for each molecule in the system and the network of pair-wise intermolecular electronic couplings. In this work, we demonstrate that the excitation energies can be accurately predicted using a kernel ridge regression (KRR) model with Coulomb matrix featurization. We present two ML models for predicting intermolecular couplings. The first one utilizes a deep neural network and bi-molecular featurization to predict the coupling directly, which we find to perform poorly. The second one utilizes a KRR model to predict unimolecular transition densities, which can subsequently be analyzed to compute the coupling. We find that the latter approach performs excellently, indicating that an effective, generalizable strategy for predicting simple bimolecular properties is through the indirect application of ML to predict higher-order unimolecular properties. Such an approach necessitates a much smaller feature space and can incorporate the insight of well-established molecular physics.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Transfer learning nonlinear plasma dynamic transitions in low dimensional embeddings via deep neural networks

Deep learning algorithms provide a new paradigm to study high-dimensional dynamical behaviors, such as those in fusion plasma systems. Development of novel, data-driven model reduction methods, coupled with detection of abnormal modes with plasma physics, opens a unique opportunity to identify plasma instabilities through automated construction of parsimonious models that can be tuned to balance accuracy and cost. Our fusion transfer learning (FTL) model demonstrates success in rapidly reconstructing nonlinear kink mode structures by learning from a limited amount of nonlinear simulation data. The knowledge transfer process leverages a pre-trained neural encoder–decoder network, initially trained on linear simulations, to effectively capture nonlinear dynamics. The low-dimensional embeddings extract the coherent structures of interest, while preserving the inherent dynamics of the complex system. Experimental results highlight FTL’s capacity to capture transitional behaviors and dynamical features in plasma dynamics—a task often challenging for conventional methods. The model developed in this study is generalizable and can be extended broadly through transfer learning to address various magnetohydrodynamics modes.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

Simulating the metabolic pathway dynamics of an organism

Disclosed herein are systems and methods for determining metabolic pathway dynamics using time series multiomics data. In one example, after receiving time series multiomics data comprising time-series metabolomics data associated a metabolic pathway and time-series proteomics data associated with the metabolic pathway, derivatives of the time series multiomics data can be determined. A machine learning model, representing a metabolic pathway dynamics model, can be trained using the time series multiomics data and the derivatives of the time series multiomics data, wherein the metabolic pathway dynamics model relates the time-series metabolomics data and time-series proteomics data to the derivatives of the time series multiomics data. The method can include simulating a virtual strain of the organism using the metabolic pathway dynamics model.

Costello, Zachary↗
DOE OSTI2022

Generative Ensemble Regression: Learning Particle Dynamics from Observations of Ensembles with Physics-informed Deep Generative Models

Here, we propose a new method for inferring the governing stochastic ordinary differential equations (SODEs) by observing particle ensembles at discrete and sparse time instants, i.e., multiple “snapshots.” Particle coordinates at a single time instant, possibly noisy or truncated, are recorded in each snapshot but are unpaired across the snapshots. By training a physics-informed generative model that generates “fake” sample paths, we aim to fit the observed particle ensemble distributions with a curve in the probability measure space, which is induced from the inferred particle dynamics. We employ different metrics to quantify the differences between distributions, e.g., the sliced Wasserstein distances and the adversarial losses in generative adversarial networks. We refer to this method as generative “ensemble-regression” (GER), in analogy to the classic “point-regression,” where we infer the dynamics by performing regression in the Euclidean space. We illustrate the GER by learning the drift and diffusion terms of particle ensembles governed by SODEs with Brownian motions and Lévy processes up to 100 dimensions. We also discuss how to treat cases with noisy or truncated observations. Apart from systems consisting of independent particles, we also tackle nonlocal interacting particle systems with unknown interaction potential parameters by constructing a physics-informed loss function. Finally, we investigate scenarios of paired observations and discuss how to reduce the dimensionality in such cases by proving a convergence theorem that provides theoretical support.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Bayesian inference of heterogeneous epidemic models: Application to COVID-19 spread accounting for long-term care facilities

Here we propose a high dimensional Bayesian inference framework for learning heterogeneous dynamics of a COVID-19 model, with a specific application to the dynamics and severity of COVID-19 inside and outside long-term care (LTC) facilities. We develop a heterogeneous compartmental model that accounts for the heterogeneity of the time-varying spread and severity of COVID-19 inside and outside LTC facilities, which is characterized by time-dependent stochastic processes and time-independent parameters in ~ 1500 dimensions after discretization. To infer these parameters, we use reported data on the number of confirmed, hospitalized, and deceased cases with suitable post-processing in both a deterministic inversion approach with appropriate regularization as a first step, followed by Bayesian inversion with proper prior distributions. To address the curse of dimensionality and the ill-posedness of the high-dimensional inference problem, we propose use of a dimension-independent projected Stein variational gradient descent method, and demonstrate the intrinsic low-dimensionality of the inverse problem. We present inference results with quantified uncertainties for both New Jersey and Texas, which experienced different epidemic phases and patterns. Moreover, we also present forecasting and validation results based on the empirical posterior samples of our inference for the future trajectory of COVID-19.

42 ENGINEERING↗
DOE OSTI2020

Interface learning in fluid dynamics: Statistical inference of closures within micro–macro-coupling models

Many complex multiphysics systems in fluid dynamics involve using solvers with varied levels of approximations in different regions of the computational domain to resolve multiple spatiotemporal scales present in the flow. The accuracy of the solution is governed by how the information is exchanged between these solvers at the interface and several methods have been devised for such coupling problems. In this article, we construct a data-driven model by spatially coupling a microscale lattice Boltzmann method (LBM) solver and macroscale finite difference method (FDM) solver for reaction-diffusion systems. The coupling between the micro-macro solvers has one to many mapping at the interface leading to the interface closure problem, and we propose a statistical inference method based on neural networks to learn this closure relation. The performance of the proposed framework in a bifidelity setting partitioned between the FDM and LBM domain shows its promise for complex systems where analytical relations between micro-macro solvers are not available.

42 ENGINEERING↗
DOE OSTI2024

Multi-resolution partial differential equations preserved learning framework for spatiotemporal dynamics

Traditional data-driven deep learning models often struggle with high training costs, error accumulation, and poor generalizability in complex physical processes. Physics-informed deep learning (PiDL) addresses these challenges by incorporating physical principles into the model. Most PiDL approaches regularize training by embedding governing equations into the loss function, yet this depends heavily on extensive hyperparameter tuning to weigh each loss term. To this end, we propose to leverage physics prior knowledge by “baking” the discretized governing equations into the neural network architecture via the connection between the partial differential equations (PDE) operators and network structures, resulting in a PDE-preserved neural network (PPNN). This method, embedding discretized PDEs through convolutional residual networks in a multi-resolution setting, largely improves the generalizability and long-term prediction accuracy, outperforming conventional black-box models. The effectiveness and merit of the proposed methods have been demonstrated across various spatiotemporal dynamical systems governed by spatiotemporal PDEs, including reaction-diffusion, Burgers’, and Navier-Stokes equations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Utilizing CellBox to Describe Varied Dynamical Systems

CellBox is a hybrid modeling framework that integrates machine-learning approaches with explicit mathematical modeling to determine the behavior of dynamical systems, intended for determining the complex interactions of molecule and phenotype within a cellular environment using perturbation/response-based data. The original work utilized analysis of a melanoma cell line, SK-Mel-133, which we sought to expand to and evaluate effectiveness in describing other nonlinear systems both within and outside of biology. We evaluated with two models. One model is based upon the free-fall of an object under a velocity-dependent drag force; this produced inherent perturbation/response data. The other model is based upon a form of mass-action molecular kinetics; for this, we ran the simulation to steady state in each species for each perturbation and select the steady state as input for CellBox. We determined that CellBox, outside of its original use case, can still strongly predict even non-biological dynamical systems in the small-perturbation regime, with r = 0.99977 to 0.85123, p < 0.05 for freefall. However, CellBox cannot accurately predict behavior within large-perturbation regimes, p >> 0.05 for freefall. In addition, we performed a naïve comparison of CellBox network structure to known network structure in the mass-action model to assess accuracy.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

A Machine Learning Assisted Development of a Model for the Populations of Convective and Stratiform Clouds

Abstract Traditional parameterizations of the interaction between convection and the environment have relied on an assumption that the slowly varying large‐scale environment is in statistical equilibrium with a large number of small and short‐lived convective clouds. They fail to capture nonequilibrium transitions such as the diurnal cycle and the formation of mesoscale convective systems as well as observed precipitation statistics and extremes. Informed by analysis of radar observations, cloud‐permitting model simulation, theory, and machine learning, this work presents a new stochastic cloud population dynamics model for characterizing the interactions between convective and stratiform clouds, with the goal of informing the representation of these interactions in global climate models. Fifteen wet seasons of precipitating cloud observations by a C‐band radar at Darwin, Australia are fed into a machine learning algorithm to obtain transition functions that close a set of coupled equations relating large‐scale forcing, mass flux, the convective cell size distribution, and the stratiform area. Under realistic large‐scale forcing, the derived transition functions show that, on the one hand, interactions with stratiform clouds act to dampen the variability in the size and number of convective cells and therefore in the convective mass flux. On the other, for a given convective area fraction, a larger number of smaller cells is more favorable for the growth of stratiform area than a smaller number of larger cells. The combination of these two factors gives rise to solutions with a few convective cells embedded in a large stratiform area, reminiscent of mesoscale convective systems.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Active Learning of Microgrid Frequency Dynamics Using Neural Ordinary Differential Equations

Accurate frequency modelling of inverter‐based resource (IBR)‐dominated power systems is crucial for ensuring stable, reliable and resilient operations, particularly given their inherent low‐inertia characteristics and fast dynamics that traditional swing equation‐based models inadequately capture. This paper explores neural ordinary differential equations (Neural ODEs) as a computationally efficient, data‐driven framework for modelling power system frequency dynamics, specifically within microgrids integrating high penetrations of distributed energy resources (DERs). The developed neural ODEs framework incorporates a neural network architecture designed to capture input dynamics. By actively perturbing the system with a known signal, the Python‐based neural ODEs framework was trained using measured system states and inputs, without the need for detailed system information. The framework, tested on a model of the Cordova, AK, microgrid, achieved a goodness of fit ranging from 60% to 99% across different state variables and maintained a mean square error in the 10 -6 p.u. range under square and step excitation signals. The proposed approach demonstrated robustness to measurement noise and initial condition variations while maintaining low computational complexity suitable for real‐time power system control applications. Furthermore, transfer learning enabled the neural ODEs model to adapt to the following changes in system topology or generator dispatch, highlighting its effectiveness for dynamic microgrids with frequently evolving configurations and diverse DERs.

Aryal, Tara [South Dakota State Univ., Brookings, ↗
DOE OSTI2023

Deep-Learning-Based Koopman Modeling for Online Control Synthesis of Nonlinear Power System Transient Dynamics

Power system stability and control have become more challenging due to the increasing uncertainty associated with renewable generation. Here, the performance of conventional control is highly driven by the physics-based offline-developed dynamic models that can deviate from the actual system characteristics under different operating conditions and/or configurations. Data-driven approaches based on online measurements can be a better solution to addressing these issues by capturing real-time operation conditions. This article describes a novel fully data-driven probabilistic framework to derive a linear representation of postcontingency grid dynamics and online prescribe control based on the derived model to enhance transient stability. The complex nonlinear power system dynamics is approximated by a linear model by using multiple neural network modules that infer distributions of the observations and introducing a Koopman layer to sample possible Koopman linear models from the inferred distributions. The trained model features linearity that can be easily incorporated into the existing linear control design paradigm and ease the controller design process. The effectiveness of Koopman-based control designs is validated through comparative case studies, which demonstrate increased prediction accuracy and control performance when applied to a power system with heterogeneous generator dynamics.

24 POWER TRANSMISSION AND DISTRIBUTION↗