SEARCH · Engineering Papers

Results for “equation learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 505 records · Page 28

All sources NASA NTRS DOE OSTI

DOE OSTI2019

Sensor Error Estimation for Reactor Coolant System with Generalization Error Equations

We consider a simplified version of the sensor error estimation problem in pressure sensors in the primary coolant system of a nuclear power plant under the condition that a majority of sensors are error-free. We present a sensor fusion method to estimate sensor errors in the primary coolant system of a power reactor by using measurements collected by multiple sensors. The underlying relationship between the measured and estimated parameters is inferred using measurements collected under controlled no-error conditions of the plant. We propose a machine learning approach to train fusers that provide an estimate of the sensor measurement by using measurements from other sensors, and the difference between the actual sensor measurement and its estimate provides the sensor estimate. We present generalization error equations for these methods that characterize the performance of their sensor error estimates on future measurements using the distribution-free machine learning formulation.

Rao, Nageswara S.↗

DOE OSTI2018

Extracting Knowledge from Data through Catalysis Informatics

Catalysis informatics is a distinct sub-field that lies at the intersection of cheminformatics and materials informatics, but with unique challenges arising from the dynamic, surface-sensitive, multi-scale nature of heterogeneous catalysis. The ideas behind catalysis informatics can be traced back decades, but the field is only recently emerging due to advances in data infrastructure, statistics, machine learning, and computational methods. In this work we review the field from early works on expert systems and knowledge engines to more recent approaches utilizing machine-learning and uncertainty quantification. The data-information-knowledge hierarchy is introduced and used to classify various developments. The chemical master equation and micro-kinetic models are proposed as a quantitative representation of catalysis knowledge, which can be used to generate explanative and predictive hypotheses for the understanding and discovery of catalytic materials. We discuss future prospects for the field, including improved quantitative coupling of experiment/theory, advanced micro-kinetic models, and the development of open-source software tools. As a result, integration of existing chemical and physical models with emerging statistical and computational tools presents a promising route toward the automated design, discovery, and optimization of heterogeneous catalytic processes.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Bayesian learning with Gaussian processes for low-dimensional representations of time-dependent nonlinear systems

This work presents a data-driven method for learning low-dimensional time-dependent physics-based surrogate models whose predictions are endowed with uncertainty estimates. We use the operator inference approach to model reduction that poses the problem of learning low-dimensional model terms as a regression of state space data and corresponding time derivatives by minimizing the residual of reduced system equations. Standard operator inference models perform well with accurate training data that are dense in time, but producing stable and accurate models when the state data are noisy and/or sparse in time remains a challenge. Another challenge is the lack of uncertainty estimation for the predictions from the operator inference models. Our approach addresses these challenges by incorporating Gaussian process surrogates into the operator inference framework to (1) probabilistically describe uncertainties in the state predictions and (2) procure analytical time derivative estimates with quantified uncertainties. The formulation leads to a generalized least-squares regression and, ultimately, reduced-order models that are described probabilistically with a closed-form expression for the posterior distribution of the operators. The resulting probabilistic surrogate model propagates uncertainties from the observed state data to reduced-order predictions. Furthermore, we demonstrate the method is effective for constructing low-dimensional models of two nonlinear partial differential equations representing a compressible flow and a nonlinear diffusion–reaction process, as well as for estimating the parameters of a low-dimensional system of nonlinear ordinary differential equations representing compartmental models in epidemiology.

Data-driven model reduction↗

DOE OSTI2024

Reinforcement Learning-Based Approach for EMT Automation of Large-Scale PV Plants

In the pursuit of efficient and precise modeling of large-scale power systems, particularly utility-scale photovoltaic (PV) plants, Electromagnetic Transient (EMT) simulations play a crucial role. As utility-scale PV plants increase in size and complexity, traditional computational methods become inadequate, necessitating more advanced techniques. This paper highlights the progressive efforts made to accelerate EMT simulations. A novel continuous reinforcement learning (RL) strategy is explored to automate the differentiation and categorization of stiff and non-stiff differential algebraic equations (DAEs). The use of stiff and non-stiff integration methods applied to relevant parts of the DAEs assists with the speed-up of the simulations. The paper details the data acquisition, development and offline training of the RL model, leading to its validation that demonstrates a high precision in optimizing simulation methods. The proposed RL promises to significantly enhance the efficacy of EMT simulations, offering a robust framework for the future of power system analysis.

Xia, Qianxue↗

DOE OSTI2022

Classification and Fusion of Two Disparate Data Streams and Nuclear Dissolutions Application

We consider two streams of data or measurements with disparate qualities and time resolutions that need to be classified. The first stream consists of higher quality data at a coarser time resolution, and the other consists of lower quality data at a finer time resolution. We present a fuser-switch method that fuses the set of classifiers of each stream separately and switches between them. We show that this method provides classification decisions at a finer time resolution with superior detection and false alarm probabilities compared to individual classifiers, under the statistical independence and time resolution ratio conditions. When classifiers are trained using machine learning methods, we show that this superior performance is guaranteed with a confidence probability specified by the classifiers' generalization equations. We use these results to provide analytical foundations for previous practical results that achieved significant performance improvements in classifying Pu/Np target dissolution events at a radiochemical processing facility.

Rao, Nageswara↗

DOE OSTI2023

A deep learning upscaling framework: Reactive transport and mineral precipitation in fracture-matrix systems

Pore-scale modeling has limited applicability at large scales due to its high computational cost. One common approach to upscale pore-scale models is the use of effective medium theories, which homogenize small-scale features in a porous structure and characterize the medium by macroscale properties (e.g., permeability) and equations (e.g., Darcy’s law). However, there are classes of physical processes for which effective medium approximations may become inaccurate, e.g., mineral precipitation and clogging during reactive transport. We have developed a deep learning upscaling framework, in which pore-scale modeling is directly employed in macroscale systems, without relying on effective medium approximations. The upscaling framework is first developed for general multiscale systems and then applied to modeling reactive transport with mineral precipitation in the altered layer in fracture-matrix structures. Solute transport from the fractures to the matrix is modeled as a wall boundary condition for the fractures, which, in turn, is predicted by recurrent neural networks using the concentration histories at the fracture-matrix boundary. Specifically, we consider a meter-scale fracture network embedded in sandstones, where the smallest feature is at the micron scale. Here the proposed framework allows us to span five orders of magnitude in length scales by capturing mineral precipitation in the altered layer of the rock matrix at the pore scale across the entire meter-scale fracture network.

42 ENGINEERING↗

DOE OSTI2025

Weak Form Scientific Machine Learning: Test Function Construction for System Identification

Weak form Scientific Machine Learning (WSciML) is a recently developed framework for data-driven modeling and scientific discovery. It leverages the weak form of equation error residuals to provide enhanced noise robustness in system identification via convolving model equations with test functions, reformulating the problem to avoid direct differentiation of data. The performance, however, relies on wisely choosing a set of compactly supported test functions. In this work, we mathematically motivate a novel data-driven method for constructing Single-scale-Local reference functions for creating the set of test functions. Our approach numerically approximates the integration error introduced by the quadrature and identifies the support size for which the error is minimal, without requiring access to the model parameter values. Through numerical experiments across various models, noise levels, and temporal resolutions, we demonstrate that the selected supports consistently align with regions of minimal parameter estimation error. We also compare the proposed method against the strategy for constructing Multi-scale-Global (and orthogonal) test functions introduced in our prior work, demonstrating the improved computational efficiency.

FOS: Computer and information sciences↗

DOE OSTI2022

Automated identification of dominant physical processes

The identification of processes that locally and approximately dominate dynamical system behavior has enabled significant advances in understanding and modeling nonlinear differential dynamical systems. Conventional methods of dominant process identification involve piecemeal and ad hoc (non-rigorous, informal) scaling analyses to identify dominant balances of governing equation terms and to delineate the spatiotemporal boundaries (boundaries in space and/or time) of each dominant balance. For the first time, we present an objective global measure of the fit of dominant balances to observations, which is desirable for automation, and was previously undefined. Furthermore, we propose a formal definition of the dominant balance identification problem in the form of an optimization problem. Here, we show that the optimization can be performed by various machine learning algorithms, enabling the automatic identification of dominant balances. Our method is algorithm agnostic and it eliminates reliance upon expert knowledge to identify dominant balances which are not known beforehand.

42 ENGINEERING↗

NASA NTRS1993

Application of hardness testing in foundry processing operations: A university and industry partnership

The objectives are the following: (1) to plan a tour of local manufacturing facility to observe hardness testing methods used and how they support the manufacturing process today; (2) to learn the operation of Brinell Hardness Testing and equipment set up with samples of accessory equipment for testing of various materials; (3) to use a Brinell scope (hand magnifier) to measure the diameter of the test indentation (in millimeters); (4) to use the hardness equation to determine the spherical area of the indentation and the calculation of the Brinell Hardness Number; (5) to use standard charts for ASTM standard size indenters to find the hardness number; and (6) to learn a method of conversion of the Brinell number to other hardness scales, their common usage, and also to convert the hardness number to the approximate tensile strength.

Martin, Donald H.↗

DOE OSTI2024

Density functional theory of material design: fundamentals and applications—II

Abstract This is the second and the final part of the review on density functional theory (DFT), referred to as DFT-II. In the first review, DFT-I, we have discussed wavefunction-based methods, their complexity, and basics of density functional theory. In DFT-II, we focus on fundamentals of DFT and their implications for the betterment of the theory. We start our presentation with the exact DFT results followed by the concept of exchange-correlation (xc) or Fermi-Coulomb hole and its relationship with xc energy functional. We also provide the exact conditions for the xc-hole, xc-energy and xc-potential along with their physical interpretation. Next, we describe the extension of DFT for non-integer number of electrons, the piecewise linearity of total energy and discontinuity of chemical potential at integer particle numbers, and derivative discontinuity of the xc potential, which has consequences on fundamental gap of solids. After that, we present how one obtains more accurate xc energy functionals by going beyond the LDA. We discuss the gradient expansion approximation (GEA), generalized gradient approximation (GGA), and hybrid functional approaches to designing better xc energy functionals that give accurate total energies. However, these functionals fail to predict properties like the ionization potential and the band gap. Thus, we next describe different methods of modelling these potentials and results of their application for calculation of the band gaps of different solids to highlight accuracy of different xc potentials. Finally, we conclude with a glimpse on orbital-free density functional theory and the machine learning approach.

36 MATERIALS SCIENCE↗

DOE OSTI2024

A Machine Learning Initializer for Newton-Raphson AC Power Flow Convergence

Power flow computations are fundamental to many power system studies. Obtaining a converged power flow case is not a trivial task especially in large power grids due to the non-linear nature of the power flow equations. One key challenge is that the widely used Newton based power flow methods are sensitive to the initial voltage magnitude and angle estimates, and a bad initial estimate would lead to non-convergence. This paper addresses this challenge by developing a random-forest (RF) machine learning model to provide better initial voltage magnitude and angle estimates towards achieving power flow convergence. This method was implemented on a real ERCOT 6102 bus system under various operating conditions. By providing better Newton-Raphson initialization, the RF model precipitated the solution of 2,106 cases out of 3,899 non-converging dispatches. These cases could not be solved from flat start or by initialization with the voltage solution of a reference case. Finally, results obtained from the RF initializer performed better when compared with DC power flow initialization, Linear regression, and Decision Trees.

random forest↗

DOE OSTI2022

Machine learning prediction on the fractional free volume of polymer membranes

Fractional free volume (FFV) characterizes the microstructural level features of polymers and affects their properties including thermal, mechanical, and separation performance. Experimental measurements and theoretical analyses have been used to quantify the FFV of polymers, but challenges remain because of their limitations. Experimental measurements are laborious and based on semi empirical equations, while Bondi’s group contribution theory involves ambiguities like the determination of van der Waals volume and the choice of factor values in the theoretical equation. To efficiently evaluate the FFV of polymers, this study utilizes high-throughput molecular dynamics (MD) simulations to build a large dataset regarding polymer’s FFV. Based on this large dataset, we further build machine learning (ML) models to establish the composition-structure relation. Inspired by group contribution theory which correlates polymer’s functional groups to FFV, our ML models correlate polymer’s substructures or physico-chemical indexes to FFV. Here, our study first benchmarks the MD simulation protocol to obtain reliable FFV of polymers and then carries out high-throughput MD simulations for more than 6,500 homopolymers and 1,400 polyamides. Such a large and diverse dataset makes the well-trained ML models more generalizable, compared with the group contribution theory. The efficiency of a feed forward neural network model is further demonstrated by applying it to a hypothetical polyimide dataset of more than 8 million chemical structures. The predicted FFVs of hypothetical polyimides are further validated by MD simulations. The obtained FFVs of the 8 million polymers, plus their previously reported gas separation performances, demonstrate the promising capability of ML virtual screening for the discovery of polymer membranes with exceptional permeability/selectivity.

36 MATERIALS SCIENCE↗

DOE OSTI2022

Learning functional priors and posteriors from data and physics

In this work, we develop a new Bayesian framework based on deep neural networks to be able to extrapolate in space-time using historical data and to quantify uncertainties arising from both noisy and gappy data in physical problems. Specifically, the proposed approach has two stages: (1) prior learning and (2) posterior estimation. At the first stage, we employ the physics-informed Generative Adversarial Networks (PI-GAN) to learn a functional prior either from a prescribed function distribution, e.g., Gaussian process, or from historical data and physics. At the second stage, we employ the Hamiltonian Monte Carlo (HMC) method to estimate the posterior in the latent space of PI-GANs. In addition, we use two different approaches to encode the physics: (1) automatic differentiation, used in the physicsinformed neural networks (PINNs) for scenarios with explicitly known partial differential equations (PDEs), and (2) operator regression using the deep operator network (DeepONet) for PDE-agnostic scenarios. We then test the proposed method for (1) meta-learning for one-dimensional regression, and forward/inverse PDE problems (combined with PINNs); (2) PDE-agnostic physical problems (combined with DeepONet), e.g., fractional diffusion as well as saturated stochastic (100-dimensional) flows in heterogeneous porous media; and (3) spatial-temporal regression problems, i.e., inference of a marine riser displacement field using experimental data from the Norwegian Deepwater Programme (NDP). The results demonstrate that the proposed approach can provide accurate predictions as well as uncertainty quantification given very limited scattered and noisy data, since historical data could be available to provide informative priors. In summary, the proposed method is capable of learning flexible functional priors, e.g., both Gaussian and non-Gaussian process, and can be readily extended to big data problems by enabling mini-batch training using stochastic HMC or normalizing flows since the latent space is generally characterized as low dimensional.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

NASA NTRS1992

Learning a trajectory using adjoint functions and teacher forcing

A new methodology for faster supervised temporal learning in nonlinear neural networks is presented which builds upon the concept of adjoint operators to allow fast computation of the gradients of an error functional with respect to all parameters of the neural architecture, and exploits the concept of teacher forcing to incorporate information on the desired output into the activation dynamics. The importance of the initial or final time conditions for the adjoint equations is discussed. A new algorithm is presented in which the adjoint equations are solved simultaneously (i.e., forward in time) with the activation dynamics of the neural network. We also indicate how teacher forcing can be modulated in time as learning proceeds. The results obtained show that the learning time is reduced by one to two orders of magnitude with respect to previously published results, while trajectory tracking is significantly improved. The proposed methodology makes hardware implementation of temporal learning attractive for real-time applications.

Toomarian, Nikzad B.↗

DOE OSTI2024

Carbon Capture through Membranes - Leveraging Multiphysics Modeling, Dimensional Analysis and Machine Learning to Scale up and Optimize Devices and Processes for Decarbonization

We study the separation performance using membrane modules through dimensional analysis (DA). We formulate the main process equations to identify relevant dimensionless numbers inherent in the physics. In particular, we identify that the critical step in the separation process is mass transfer through the selective layer. Remarkably, the dimensionless feed flow (DFfeed) emerges as a crucial factor in describing this process. Not only does DFfeed directly appear in the governing equations, but it also holds a physical significance associated with the time scales for the mass transfer across the feed side and through the selective layer. Regarding the output performance variables, we consider the recovery, stage cut, productivity and purity. In this context, we profit from experimental data and CFD simulations to evaluate the separation performance of the modules when varying the input flowrate, the scale of the module, and the CO2 permeance. These datasets enable us to establish correlations between performance metrics and the dimensionless feed flow (DFfeed). Using simple power functions of DFfeed, we obtain R2 coefficients exceeding 0.99, indicating the accuracy of the correlations built in the present work. In the future, we wish to use DA to understand key transport mechanisms, predict and control module performance, and challenge the universality of these findings by testing various gas separations across different membrane modules beyond our case study.

Pedrozo, Hector A.↗

DOE OSTI2018

Classification of machine learning frameworks for data-driven thermal fluid models

Thermal fluid processes are inherently multi-physics and multi-scale, involving mass-momentum-energy transport phenomena at multiple scales. Thermal fluid simulation (TFS) is based on solving conservative equations, for which – except for “first-principles” direct numerical simulation – closure relations (CRs) are required to provide microscopic interactions or so-called sub-grid-scale physics. In practice, TFS is realized through reduced-order modeling, and its CRs as low-fidelity models can be informed by observations and data from relevant and adequately evaluated experiments and high-fidelity simulations. This paper is focused on data-driven TFS models, specifically on their development using machine learning (ML). Five ML frameworks are introduced including physics-separated ML (PSML or Type I ML), physics-evaluated ML (PEML or Type II ML), physics-integrated ML (PIML or Type III ML), physics-recovered (PRML or Type IV ML), and physics-discovered ML (PDML or Type V ML). The frameworks vary in their performance for different applications depending on the level of knowledge of governing physics, source, type, amount and quality of available data for training. Notably, outlined for the first time in this paper, Type III models present stringent requirements on modeling, substantial computing resources for training, and high potential in extracting value from “big data” in thermal fluid research. Here the current paper demonstrates and investigates ML frameworks in three examples. First, we utilize the heat diffusion equation with a nonlinear conductivity model formulated by convolutional neural networks (CNNs) and feedforward neural networks (FNNs) to illustrate the applications of Type I, Type II, Type III, and Type V ML. The results indicate a preference for Type II ML under deficient data support. Type III ML can effectively utilize field data, potentially generating more robust predictions than Type I and Type II ML. CNN-based closures exhibit more predictability than FNN-based closures, but CNN-based closures require more training data to obtain accurate predictions. Second, we illustrate how to employ Type I ML and Type II ML frameworks for data-driven turbulence modeling using reference works. Third, we demonstrate Type I ML by building a deep FNN-based slip closure for two-phase flow modeling. The results show that deep FNN-based closures exhibit a bounded error in the prediction domain.

42 ENGINEERING↗

DOE OSTI2020

A machine learning study on spinodal clumping in heavy ion collisions

Possible observables of baryon number clustering due to the instabilities occurring at a first order QCD phase transition are discussed. The dynamical formation of baryon clusters at a QCD phase transition can be described by numerical fluid dynamics, augmented with a gradient term and an equation of state with a mechanically unstable region. It is shown that the dynamical description of this phase transition, in nuclear collisions, will lead to the formation of dense baryon clusters at the phase boundary. State-of-the-art machine learning methods find that the coordinate space clumping leaves characteristic imprints on the spatial net density distribution in almost every event. On the other hand the momentum distributions do not show any clear event-by-event features. Lastly, it is shown that the 'third order' cumulant, the skewness, shows a peak at the beam energy where the system, created in the heavy ion collision, reaches the deconfinement phase transition.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2020

Graph interpolating activation improves both natural and robust accuracies in data-efficient deep learning

Improving the accuracy and robustness of deep neural nets (DNNs) and adapting them to small training data are primary tasks in deep learning (DL) research. In this paper, we replace the output activation function of DNNs, typically the data-agnostic softmax function, with a graph Laplacian-based high-dimensional interpolating function which, in the continuum limit, converges to the solution of a Laplace–Beltrami equation on a high-dimensional manifold. Furthermore, we propose end-to-end training and testing algorithms for this new architecture. The proposed DNN with graph interpolating activation integrates the advantages of both deep learning and manifold learning. Compared to the conventional DNNs with the softmax function as output activation, the new framework demonstrates the following major advantages: First, it is better applicable to data-efficient learning in which we train high capacity DNNs without using a large number of training data. Second, it remarkably improves both natural accuracy on the clean images and robust accuracy on the adversarial images crafted by both white-box and black-box adversarial attacks. Third, it is a natural choice for semi-supervised learning. This paper is a significant extension of our earlier work published in NeurIPS, 2018. For reproducibility, the code is available at https://github.com/BaoWangMath/DNN-DataDependentActivation .

Mathematics↗