Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Equation learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Physics-informed hybrid modeling methodology for building infiltration

Infiltration is responsible for one-third to one-half of the space conditioning load of a typical residential home, but the modeling of infiltration for building energy modeling is either represented by over-simplified equations or dependent on over-generalized rules of thumb. Here, this paper develops a physics-informed data-driven methodology for modeling infiltration using building-specific empirical measurements. The developed hybrid methodology combines machine-learning categorization and grey-box sub-modeling to improve the accuracy and generalization of commonly used grey-box infiltration models. The developed methodology excels at predicting infiltration by improving the ability to predict infiltration under unseen environmental conditions using machine learning algorithms with physical significance. In a case study conducted using the iUnit, a modular studio apartment experimental test facility located at the National Renewable Energy Laboratory, we use empirical airtightness measurements to fit an infiltration model using the developed methodology. We find that the developed methodology can improve the overall model accuracy by 43% and improve extrapolation by 38%, compared with the model based on the common grey-box infiltration equation. We also notice that the selected features can improve the performance of a pure machine-learning model, indicating that our methodology identifies the features with the most physical significance to infiltration modeling.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

A methodology for decay heat characterization in molten salt reactors

Accurate decay heat prediction in molten salt reactors (MSRs) faces dual challenges: complex operational uncertainties and the need for interpretable models compatible with engineering workflows. This work presents a hybrid machine learning and segmented polynomial methodology that addresses both requirements through three key innovations. First, a modular data architecture encodes MSR-specific operational parameters (power density: 1-100 W cm -3 , humidity: 0-0.1 wt %, air ingress: 0-0.1 mol %) with uncertainty-aware temporal discretization spanning 15 orders of magnitude. Second, region-optimized machine learning models achieve 92.3 % root mean square error (RMSE) reduction over conventional polynomials while maintaining physical interpretability through automated piecewise equation generation. Third, dual front-end interfaces accelerate safety analyses — a Jupyter environment enables researchers to explore 10,000+ parameter combinations via interactive widgets, while a Streamlit web application reduces design iteration cycles through production-grade visualization tools. Operational deployment demonstrates prediction times of only a couple hundred milliseconds for 10 4 years decay profiles, enabling real-time optimization of spent fuel container designs.

42 - ENGINEERING↗
DOE OSTI2024

A Novel Method to Train Classification Models for Structure Detection in In Situ Spacecraft Data

We present a method for creating spacecraft-like data which can be used to train Machine Learning (ML) models to detect and classify structures in in situ spacecraft data. First, we use the Grad-Shafranov equation to numerically solve for several magnetohydrostatic equilibria which are variations on a known analytic equilibrium. These equilibria are then used as the initial conditions for Particle-In-Cell simulations in which the structures of interest are observed and labeled. We then take one-dimensional slices through the simulations to replicate what a spacecraft collecting data from the simulation would observe. This sliced data then can be used as training data for the initial training of ML models intended for use on spacecraft data. We demonstrate the method applied to the problem of detecting small-scale plasmoids in the magnetotail, which is important for understanding complex magnetotail reconnection dynamics. The simple 1D classifier we train is able to detect more than 70% of the plasmoid points in the data set but also produces a large number of false positives. Our further work on this example problem is detailed, and further potential uses of the method are discussed.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Physics-guided Deep Learning for Power System State Estimation

In the past decade, dramatic progress has been made in the field of machine learning. This paper explores the possibility of applying deep learning in power system state estimation. Traditionally, physics-based models are used including weighted least square (WLS) or weighted least absolute value (WLAV). These models typically consider a single snapshot of the system without capturing temporal correlations of system states. In this paper, a physics-guided deep learning (PGDL) method is proposed. Specifically, inspired by autoencoders, deep neural networks (DNNs) are used to learn the temporal correlations. The estimated system states from DNNs are then checked against physics laws by running through a set of power flow equations. Hence, the proposed PGDL is both data-driven and physics-guided. The accuracy and robustness of the proposed PGDL method are compared with traditional methods in standard IEEE cases. Simulations show promising results and the applicability is further discussed.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Bayesian learning with Gaussian processes for low-dimensional representations of time-dependent nonlinear systems

This work presents a data-driven method for learning low-dimensional time-dependent physics-based surrogate models whose predictions are endowed with uncertainty estimates. We use the operator inference approach to model reduction that poses the problem of learning low-dimensional model terms as a regression of state space data and corresponding time derivatives by minimizing the residual of reduced system equations. Standard operator inference models perform well with accurate training data that are dense in time, but producing stable and accurate models when the state data are noisy and/or sparse in time remains a challenge. Another challenge is the lack of uncertainty estimation for the predictions from the operator inference models. Our approach addresses these challenges by incorporating Gaussian process surrogates into the operator inference framework to (1) probabilistically describe uncertainties in the state predictions and (2) procure analytical time derivative estimates with quantified uncertainties. The formulation leads to a generalized least-squares regression and, ultimately, reduced-order models that are described probabilistically with a closed-form expression for the posterior distribution of the operators. The resulting probabilistic surrogate model propagates uncertainties from the observed state data to reduced-order predictions. Furthermore, we demonstrate the method is effective for constructing low-dimensional models of two nonlinear partial differential equations representing a compressible flow and a nonlinear diffusion–reaction process, as well as for estimating the parameters of a low-dimensional system of nonlinear ordinary differential equations representing compartmental models in epidemiology.

Data-driven model reduction↗
DOE OSTI2024

Reinforcement Learning-Based Approach for EMT Automation of Large-Scale PV Plants

In the pursuit of efficient and precise modeling of large-scale power systems, particularly utility-scale photovoltaic (PV) plants, Electromagnetic Transient (EMT) simulations play a crucial role. As utility-scale PV plants increase in size and complexity, traditional computational methods become inadequate, necessitating more advanced techniques. This paper highlights the progressive efforts made to accelerate EMT simulations. A novel continuous reinforcement learning (RL) strategy is explored to automate the differentiation and categorization of stiff and non-stiff differential algebraic equations (DAEs). The use of stiff and non-stiff integration methods applied to relevant parts of the DAEs assists with the speed-up of the simulations. The paper details the data acquisition, development and offline training of the RL model, leading to its validation that demonstrates a high precision in optimizing simulation methods. The proposed RL promises to significantly enhance the efficacy of EMT simulations, offering a robust framework for the future of power system analysis.

Xia, Qianxue↗
DOE OSTI2022

Classification and Fusion of Two Disparate Data Streams and Nuclear Dissolutions Application

We consider two streams of data or measurements with disparate qualities and time resolutions that need to be classified. The first stream consists of higher quality data at a coarser time resolution, and the other consists of lower quality data at a finer time resolution. We present a fuser-switch method that fuses the set of classifiers of each stream separately and switches between them. We show that this method provides classification decisions at a finer time resolution with superior detection and false alarm probabilities compared to individual classifiers, under the statistical independence and time resolution ratio conditions. When classifiers are trained using machine learning methods, we show that this superior performance is guaranteed with a confidence probability specified by the classifiers' generalization equations. We use these results to provide analytical foundations for previous practical results that achieved significant performance improvements in classifying Pu/Np target dissolution events at a radiochemical processing facility.

Rao, Nageswara↗
DOE OSTI2023

A deep learning upscaling framework: Reactive transport and mineral precipitation in fracture-matrix systems

Pore-scale modeling has limited applicability at large scales due to its high computational cost. One common approach to upscale pore-scale models is the use of effective medium theories, which homogenize small-scale features in a porous structure and characterize the medium by macroscale properties (e.g., permeability) and equations (e.g., Darcy’s law). However, there are classes of physical processes for which effective medium approximations may become inaccurate, e.g., mineral precipitation and clogging during reactive transport. We have developed a deep learning upscaling framework, in which pore-scale modeling is directly employed in macroscale systems, without relying on effective medium approximations. The upscaling framework is first developed for general multiscale systems and then applied to modeling reactive transport with mineral precipitation in the altered layer in fracture-matrix structures. Solute transport from the fractures to the matrix is modeled as a wall boundary condition for the fractures, which, in turn, is predicted by recurrent neural networks using the concentration histories at the fracture-matrix boundary. Specifically, we consider a meter-scale fracture network embedded in sandstones, where the smallest feature is at the micron scale. Here the proposed framework allows us to span five orders of magnitude in length scales by capturing mineral precipitation in the altered layer of the rock matrix at the pore scale across the entire meter-scale fracture network.

42 ENGINEERING↗
DOE OSTI2025

Weak Form Scientific Machine Learning: Test Function Construction for System Identification

Weak form Scientific Machine Learning (WSciML) is a recently developed framework for data-driven modeling and scientific discovery. It leverages the weak form of equation error residuals to provide enhanced noise robustness in system identification via convolving model equations with test functions, reformulating the problem to avoid direct differentiation of data. The performance, however, relies on wisely choosing a set of compactly supported test functions. In this work, we mathematically motivate a novel data-driven method for constructing Single-scale-Local reference functions for creating the set of test functions. Our approach numerically approximates the integration error introduced by the quadrature and identifies the support size for which the error is minimal, without requiring access to the model parameter values. Through numerical experiments across various models, noise levels, and temporal resolutions, we demonstrate that the selected supports consistently align with regions of minimal parameter estimation error. We also compare the proposed method against the strategy for constructing Multi-scale-Global (and orthogonal) test functions introduced in our prior work, demonstrating the improved computational efficiency.

FOS: Computer and information sciences↗
DOE OSTI2022

Automated identification of dominant physical processes

The identification of processes that locally and approximately dominate dynamical system behavior has enabled significant advances in understanding and modeling nonlinear differential dynamical systems. Conventional methods of dominant process identification involve piecemeal and ad hoc (non-rigorous, informal) scaling analyses to identify dominant balances of governing equation terms and to delineate the spatiotemporal boundaries (boundaries in space and/or time) of each dominant balance. For the first time, we present an objective global measure of the fit of dominant balances to observations, which is desirable for automation, and was previously undefined. Furthermore, we propose a formal definition of the dominant balance identification problem in the form of an optimization problem. Here, we show that the optimization can be performed by various machine learning algorithms, enabling the automatic identification of dominant balances. Our method is algorithm agnostic and it eliminates reliance upon expert knowledge to identify dominant balances which are not known beforehand.

42 ENGINEERING↗
DOE OSTI2024

Density functional theory of material design: fundamentals and applications—II

Abstract This is the second and the final part of the review on density functional theory (DFT), referred to as DFT-II. In the first review, DFT-I, we have discussed wavefunction-based methods, their complexity, and basics of density functional theory. In DFT-II, we focus on fundamentals of DFT and their implications for the betterment of the theory. We start our presentation with the exact DFT results followed by the concept of exchange-correlation (xc) or Fermi-Coulomb hole and its relationship with xc energy functional. We also provide the exact conditions for the xc-hole, xc-energy and xc-potential along with their physical interpretation. Next, we describe the extension of DFT for non-integer number of electrons, the piecewise linearity of total energy and discontinuity of chemical potential at integer particle numbers, and derivative discontinuity of the xc potential, which has consequences on fundamental gap of solids. After that, we present how one obtains more accurate xc energy functionals by going beyond the LDA. We discuss the gradient expansion approximation (GEA), generalized gradient approximation (GGA), and hybrid functional approaches to designing better xc energy functionals that give accurate total energies. However, these functionals fail to predict properties like the ionization potential and the band gap. Thus, we next describe different methods of modelling these potentials and results of their application for calculation of the band gaps of different solids to highlight accuracy of different xc potentials. Finally, we conclude with a glimpse on orbital-free density functional theory and the machine learning approach.

36 MATERIALS SCIENCE↗
DOE OSTI2024

A Machine Learning Initializer for Newton-Raphson AC Power Flow Convergence

Power flow computations are fundamental to many power system studies. Obtaining a converged power flow case is not a trivial task especially in large power grids due to the non-linear nature of the power flow equations. One key challenge is that the widely used Newton based power flow methods are sensitive to the initial voltage magnitude and angle estimates, and a bad initial estimate would lead to non-convergence. This paper addresses this challenge by developing a random-forest (RF) machine learning model to provide better initial voltage magnitude and angle estimates towards achieving power flow convergence. This method was implemented on a real ERCOT 6102 bus system under various operating conditions. By providing better Newton-Raphson initialization, the RF model precipitated the solution of 2,106 cases out of 3,899 non-converging dispatches. These cases could not be solved from flat start or by initialization with the voltage solution of a reference case. Finally, results obtained from the RF initializer performed better when compared with DC power flow initialization, Linear regression, and Decision Trees.

random forest↗
DOE OSTI2022

Machine learning prediction on the fractional free volume of polymer membranes

Fractional free volume (FFV) characterizes the microstructural level features of polymers and affects their properties including thermal, mechanical, and separation performance. Experimental measurements and theoretical analyses have been used to quantify the FFV of polymers, but challenges remain because of their limitations. Experimental measurements are laborious and based on semi empirical equations, while Bondi’s group contribution theory involves ambiguities like the determination of van der Waals volume and the choice of factor values in the theoretical equation. To efficiently evaluate the FFV of polymers, this study utilizes high-throughput molecular dynamics (MD) simulations to build a large dataset regarding polymer’s FFV. Based on this large dataset, we further build machine learning (ML) models to establish the composition-structure relation. Inspired by group contribution theory which correlates polymer’s functional groups to FFV, our ML models correlate polymer’s substructures or physico-chemical indexes to FFV. Here, our study first benchmarks the MD simulation protocol to obtain reliable FFV of polymers and then carries out high-throughput MD simulations for more than 6,500 homopolymers and 1,400 polyamides. Such a large and diverse dataset makes the well-trained ML models more generalizable, compared with the group contribution theory. The efficiency of a feed forward neural network model is further demonstrated by applying it to a hypothetical polyimide dataset of more than 8 million chemical structures. The predicted FFVs of hypothetical polyimides are further validated by MD simulations. The obtained FFVs of the 8 million polymers, plus their previously reported gas separation performances, demonstrate the promising capability of ML virtual screening for the discovery of polymer membranes with exceptional permeability/selectivity.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Learning functional priors and posteriors from data and physics

In this work, we develop a new Bayesian framework based on deep neural networks to be able to extrapolate in space-time using historical data and to quantify uncertainties arising from both noisy and gappy data in physical problems. Specifically, the proposed approach has two stages: (1) prior learning and (2) posterior estimation. At the first stage, we employ the physics-informed Generative Adversarial Networks (PI-GAN) to learn a functional prior either from a prescribed function distribution, e.g., Gaussian process, or from historical data and physics. At the second stage, we employ the Hamiltonian Monte Carlo (HMC) method to estimate the posterior in the latent space of PI-GANs. In addition, we use two different approaches to encode the physics: (1) automatic differentiation, used in the physicsinformed neural networks (PINNs) for scenarios with explicitly known partial differential equations (PDEs), and (2) operator regression using the deep operator network (DeepONet) for PDE-agnostic scenarios. We then test the proposed method for (1) meta-learning for one-dimensional regression, and forward/inverse PDE problems (combined with PINNs); (2) PDE-agnostic physical problems (combined with DeepONet), e.g., fractional diffusion as well as saturated stochastic (100-dimensional) flows in heterogeneous porous media; and (3) spatial-temporal regression problems, i.e., inference of a marine riser displacement field using experimental data from the Norwegian Deepwater Programme (NDP). The results demonstrate that the proposed approach can provide accurate predictions as well as uncertainty quantification given very limited scattered and noisy data, since historical data could be available to provide informative priors. In summary, the proposed method is capable of learning flexible functional priors, e.g., both Gaussian and non-Gaussian process, and can be readily extended to big data problems by enabling mini-batch training using stochastic HMC or normalizing flows since the latent space is generally characterized as low dimensional.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Carbon Capture through Membranes - Leveraging Multiphysics Modeling, Dimensional Analysis and Machine Learning to Scale up and Optimize Devices and Processes for Decarbonization

We study the separation performance using membrane modules through dimensional analysis (DA). We formulate the main process equations to identify relevant dimensionless numbers inherent in the physics. In particular, we identify that the critical step in the separation process is mass transfer through the selective layer. Remarkably, the dimensionless feed flow (DFfeed) emerges as a crucial factor in describing this process. Not only does DFfeed directly appear in the governing equations, but it also holds a physical significance associated with the time scales for the mass transfer across the feed side and through the selective layer. Regarding the output performance variables, we consider the recovery, stage cut, productivity and purity. In this context, we profit from experimental data and CFD simulations to evaluate the separation performance of the modules when varying the input flowrate, the scale of the module, and the CO2 permeance. These datasets enable us to establish correlations between performance metrics and the dimensionless feed flow (DFfeed). Using simple power functions of DFfeed, we obtain R2 coefficients exceeding 0.99, indicating the accuracy of the correlations built in the present work. In the future, we wish to use DA to understand key transport mechanisms, predict and control module performance, and challenge the universality of these findings by testing various gas separations across different membrane modules beyond our case study.

Pedrozo, Hector A.↗
DOE OSTI2020

A machine learning study on spinodal clumping in heavy ion collisions

Possible observables of baryon number clustering due to the instabilities occurring at a first order QCD phase transition are discussed. The dynamical formation of baryon clusters at a QCD phase transition can be described by numerical fluid dynamics, augmented with a gradient term and an equation of state with a mechanically unstable region. It is shown that the dynamical description of this phase transition, in nuclear collisions, will lead to the formation of dense baryon clusters at the phase boundary. State-of-the-art machine learning methods find that the coordinate space clumping leaves characteristic imprints on the spatial net density distribution in almost every event. On the other hand the momentum distributions do not show any clear event-by-event features. Lastly, it is shown that the 'third order' cumulant, the skewness, shows a peak at the beam energy where the system, created in the heavy ion collision, reaches the deconfinement phase transition.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2020

Graph interpolating activation improves both natural and robust accuracies in data-efficient deep learning

Improving the accuracy and robustness of deep neural nets (DNNs) and adapting them to small training data are primary tasks in deep learning (DL) research. In this paper, we replace the output activation function of DNNs, typically the data-agnostic softmax function, with a graph Laplacian-based high-dimensional interpolating function which, in the continuum limit, converges to the solution of a Laplace–Beltrami equation on a high-dimensional manifold. Furthermore, we propose end-to-end training and testing algorithms for this new architecture. The proposed DNN with graph interpolating activation integrates the advantages of both deep learning and manifold learning. Compared to the conventional DNNs with the softmax function as output activation, the new framework demonstrates the following major advantages: First, it is better applicable to data-efficient learning in which we train high capacity DNNs without using a large number of training data. Second, it remarkably improves both natural accuracy on the clean images and robust accuracy on the adversarial images crafted by both white-box and black-box adversarial attacks. Third, it is a natural choice for semi-supervised learning. This paper is a significant extension of our earlier work published in NeurIPS, 2018. For reproducibility, the code is available at https://github.com/BaoWangMath/DNN-DataDependentActivation .

Mathematics↗
DOE OSTI2020

NSFnets (Navier-Stokes flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations

In the last 50 years there has been a tremendous progress in solving numerically the Navier-Stokes equations using finite differences, finite elements, spectral, and even meshless methods. Yet, in many real cases, we still cannot incorporate seamlessly (multi-fidelity) data into existing algorithms, and for industrial-complexity applications the mesh generation is time consuming and still an art. Moreover, solving ill-posed problems (e.g., lacking boundary conditions) or inverse problems is often prohibitively expensive and requires different formulations and new computer codes. Here, we employ physics-informed neural networks (PINNs), encoding the governing equations directly into the deep neural network via automatic differentiation, to overcome some of the aforementioned limitations for simulating incompressible laminar and turbulent flows. We develop the Navier-Stokes flow nets (NSFnets) by considering two different mathematical formulations of the Navier-Stokes equations: the velocity-pressure (VP) formulation and the vorticity-velocity (VV) formulation. Since this is a new approach, we first select some standard benchmark problems to assess the accuracy, convergence rate, computational cost and flexibility of NSFnets; analytical solutions and direct numerical simulation (DNS) databases provide proper initial and boundary conditions for the NSFnet simulations. The spatial and temporal coordinates are the inputs of the NSFnets, while the instantaneous velocity and pressure fields are the outputs for the VP-NSFnet, and the instantaneous velocity and vorticity fields are the outputs for the VV-NSFnet. This is unsupervised learning and, hence, no labeled data are required beyond boundary and initial conditions and the fluid properties. The residuals of the VP or VV governing equations, together with the initial and boundary conditions, are embedded into the loss function of the NSFnets. No data is provided for the pressure to the VP-NSFnet, which is a hidden state and is obtained via the incompressibility constraint without extra computational cost. Unlike the traditional numerical methods, NSFnets inherit the properties of neural networks (NNs), hence the total error is composed of the approximation, the optimization, and the generalization errors. Here, we empirically attempt to quantify these errors by varying the sampling (“residual”) points, the iterative solvers, and the size of the NN architecture. For the laminar flow solutions, we show that both the VP and the VV formulations are comparable in accuracy but their best performance corresponds to different NN architectures. The initial convergence rate is fast but the error eventually saturates to a plateau due to the dominance of the optimization error. For the turbulent channel flow, we show that NSFnets can sustain turbulence at , but due to expensive training we only consider part of the channel domain and enforce velocity boundary conditions on the subdomain boundaries provided by the DNS data base. We also perform a systematic study on the weights used in the loss function for balancing the data and physics components, and investigate a new way of computing the weights dynamically to accelerate training and enhance accuracy. In the last part, we demonstrate how NSFnets should be used in practice, namely for ill-posed problems with incomplete or noisy boundary conditions as well as for inverse problems. We obtain reasonably accurate solutions for such cases as well without the need to change the NSFnets and at the same computational cost as in the forward well-posed problems. As a result, we also present a simple example of transfer learning that will aid in accelerating the training of NSFnets for different parameter settings.

97 MATHEMATICS AND COMPUTING↗