Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Equation learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

On the Recoverability of Reactor Dynamics from Point Kinetics Data using SINDYc

Data driven models for reactor dynamics tend to face a few notable challenges. First, the broad range of timescales present across the various feedback mechanisms. Next, high standards for safety and importance of model performance in all operational domains. Finally, the correlations between state variables observed in most transients leads to difficulties when methods attempt to attribute certain dynamic phenomena to a particular cause. The current paper seeks to support efforts towards incorporating physics knowledge into one particular data-driven method for finding reactor dynamics called ``Sparse Identification of Nonlinear Dynamics with Control (SINDYc). The incorporation of physical knowledge into SINDYc will help address the challenges listed above by giving the model an initial understanding of the system being modeled. The demonstration of an exact representation of a set of point reactor dynamics equations in SINDYc is provided. Then, this allows for further discussion with mathematical justification as to why SINDYc, and other data-driven methods, may be difficult to apply to nuclear reactor dynamics due to correlations in the state variables. A particularly useful result of this work is the set of candidate functions required in SINDYc to exactly represent the reactor dynamics under a point approximation. In the future, SINDYc can be integrated with point models for reactor dynamics before being applied to the physical system to yield higher accuracy.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS2020

Application of Sparse Identification of Nonlinear Dynamics for Physics-Informed Learning

Advances in machine learning and deep neural networks has enabled complex engineering tasks like image recognition, anomaly detection, regression, and multi-objective optimization, to name but a few. The complexity of the algorithm architecture, e.g., the number of hidden layers in a deep neural network, typically grows with the complexity of the problems they are required to solve, leaving little room for interpreting (or explaining) the path that results in a specific solution. This drawback is particularly relevant for autonomous aerospace and aviation systems, where certifications require a complete understanding of the algorithm behavior in all possible scenarios. Including physics knowledge in such data-driven tools may improve the interpretability of the algorithms, thus enhancing model validation against events with low probability but relevant for system certification. Such events include, for example, spacecraft or aircraft sub-system failures, for which data may not be available in the training phase. This paper investigates a recent physics-informed learning algorithm for identification of system dynamics, and shows how the governing equations of a system can be extracted from data using sparse regression. The learned relationships can be utilized as a surrogate model which, unlike typical data-driven surrogate models, relies on the learned underlying dynamics of the system rather than large number of fitting parameters. The work shows that the algorithm can reconstruct the differential equations underlying the observed dynamics using a single trajectory when no uncertainty is involved. However, the training set size must increase when dealing with stochastic systems, e.g., nonlinear dynamics with random initial conditions.

Corbetta, Matteo↗
DOE OSTI2025

Acceleration of Power System Dynamic Simulations Using a Deep Equilibrium Layer and Neural ODE Surrogate

The dominant paradigm for power system dynamic simulation is to build system-level simulations by combining physics-based models of individual components. The sheer size of the system along with the rapid integration of inverter-based resources exacerbates the computational burden of running time domain simulations. Here, in this paper, we propose a data-driven surrogate model based on implicit machine learningspecifically deep equilibrium layers and neural ordinary differential equationsto learn a reduced order model of a portion of the full underlying system. The data-driven surrogate achieves similar accuracy and reduction in simulation time compared to a physics-based surrogate, without the constraint of requiring detailed knowledge of the underlying dynamic models. This work also establishes key requirements needed to integrate the surrogate into existing simulation workflows; the proposed surrogate is initialized to a steady state operating point that matches the power flow solution by design.

Neural ordinary differential equations↗
DOE OSTI2023

Machine learning of hidden variables in multiscale fluid simulation

Solving fluid dynamics equations often requires the use of closure relations that account for missing microphysics. For example, when solving equations related to fluid dynamics for systems with a large Reynolds number, sub-grid effects become important and a turbulence closure is required, and in systems with a large Knudsen number, kinetic effects become important and a kinetic closure is required. By adding an equation governing the growth and transport of the quantity requiring the closure relation, it becomes possible to capture microphysics through the introduction of ‘hidden variables’ that are non-local in space and time. The behavior of the ‘hidden variables’ in response to the fluid conditions can be learned from a higher fidelity or ab-initio model that contains all the microphysics. In our study, a partial differential equation simulator that is end-to-end differentiable is used to train judiciously placed neural networks against ground-truth simulations. We show that this method enables an Euler equation based approach to reproduce non-linear, large Knudsen number plasma physics that can otherwise only be modeled using Boltzmann-like equation simulators such as Vlasov or particle-in-cell modeling.

42 ENGINEERING↗
DOE OSTI2023

Residual-based error correction for neural operator accelerated infinite-dimensional Bayesian inverse problems

We explore using neural operators, or neural network representations of nonlinear maps between function spaces, to accelerate infinite-dimensional Bayesian inverse problems (BIPs) with models governed by nonlinear parametric partial differential equations (PDEs). Neural operators have gained significant attention in recent years for their ability to approximate the parameter-to-solution maps defined by PDEs using as training data solutions of PDEs at a limited number of parameter samples. The computational cost of BIPs can be drastically reduced if the large number of PDE solves required for posterior characterization are replaced with evaluations of trained neural operators. However, reducing error in the resulting BIP solutions via reducing the approximation error of the neural operators in training can be challenging and unreliable. We provide an a priori error bound result that implies certain BIPs can be ill-conditioned to the approximation error of neural operators, thus leading to inaccessible accuracy requirements in training. To reliably deploy neural operators in BIPs, we consider a strategy for enhancing the performance of neural operators: correcting the prediction of a trained neural operator by solving a linear variational problem based on the PDE residual. We show that a trained neural operator with error correction can achieve a quadratic reduction of its approximation error, all while retaining substantial computational speedups of posterior sampling when models are governed by highly nonlinear PDEs. The strategy is applied to two numerical examples of BIPs based on a nonlinear reaction–diffusion problem and deformation of hyperelastic materials. We demonstrate that posterior representations of the two BIPs produced using trained neural operators are greatly and consistently enhanced by error correction.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Efficient data-driven regression for reduced-order modeling of spatial pattern formation

We present an efficient data-driven regression approach for constructing reduced-order models (ROMs) of reaction-diffusion systems exhibiting pattern formation. The ROMs are learned non-intrusively from available training data of physically accurate numerical simulations. The method can be applied to general nonlinear systems through the use of polynomial model form, while not requiring knowledge of the underlying physical model, governing equations, or numerical solvers. The process of learning ROMs is posed as a low-cost least-squares problem in a reduced-order subspace identified via Proper Orthogonal Decomposition (POD). Numerical experiments on classical pattern-forming systems–including the Schnakenberg and Mimura–Tsujikawa models–demonstrate that higher-order surrogate models significantly improve prediction accuracy while maintaining low computational cost. The proposed method provides a flexible, non-intrusive model reduction framework, well suited for the analysis of complex spatio-temporal pattern formation phenomena.

Data-driven modeling↗
DOE OSTI2023

Multilevel-in-width training for deep neural network regression

A common challenge in regression is that for many problems, the degrees of freedom required for a high-quality solution also allows for overfitting. Regularization is a class of strategies that seek to restrict the range of possible solutions so as to discourage overfitting while still enabling good solutions, and different regularization strategies impose different types of restrictions. In this paper, we present a multilevel regularization strategy that constructs and trains a hierarchy of neural networks, each of which has layers that are wider versions of the previous network's layers. Here, we draw intuition and techniques from the field of Algebraic Multigrid (AMG), traditionally used for solving linear and nonlinear systems of equations, and specifically adapt the Full Approximation Scheme (FAS) for nonlinear systems of equations to the problem of deep learning. Training through V-cycles then encourage the neural networks to build a hierarchical understanding of the problem. We refer to this approach as multilevel-in-width to distinguish from prior multilevel works which hierarchically alter the depth of neural networks. The resulting approach is a highly flexible framework that can be applied to a variety of layer types, which we demonstrate with both fully connected and convolutional layers. We experimentally show with PDE regression problems that our multilevel training approach is an effective regularizer, improving the generalize performance of the neural networks studied.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Neuromorphic Computing is Turing-Complete

Neuromorphic computing is a non-von Neumann computing paradigm that performs computation by emulating the human brain. Neuromorphic systems are extremely energy-efficient and known to consume thousands of times less power than CPUs and GPUs. They have the potential to drive critical use cases such as autonomous vehicles, edge computing and internet of things in the future. For this reason, they are sought to be an indispensable part of the future computing landscape. Neuromorphic systems are mainly used for spike-based machine learning applications, although there are some non-machine learning applications in graph theory, differential equations, and spike-based simulations. These applications suggest that neuromorphic computing might be capable of general-purpose computing. However, general-purpose computability of neuromorphic computing has not been established yet. In this work, we prove that neuromorphic computing is Turing-complete and therefore capable of general-purpose computing. Specifically, we present a model of neuromorphic computing, with just two neuron parameters (threshold and leak), and two synaptic parameters (weight and delay). We devise neuromorphic circuits for computing all the μ-recursive functions (i.e., constant, successor and projection functions) and all the μ-recursive operators (i.e., composition, primitive recursion and minimization operators). Given that the μ-recursive functions and operators are precisely the ones that can be computed using a Turing machine, this work establishes the Turing-completeness of neuromorphic computing.

Date, Prasanna↗
DOE OSTI2022

High-precision inversion of dynamic radiography using hydrodynamic features

While radiography is routinely used to probe complex, evolving density fields in research areas ranging from materials science to shock physics to inertial confinement fusion and other national security applications, complications resulting from noise, scatter, complex beam dynamics, etc. prevent current methods of reconstructing density from being accurate enough to identify the underlying physics with sufficient confidence. In this work, we show that using only features that are robustly identifiable in radiographs and combining them with the underlying hydrodynamic equations of motion using a machine learning approach of a conditional generative adversarial network (cGAN) provides a new and effective approach to determine density fields from a dynamic sequence of radiographs. In particular, we demonstrate the ability of this method to outperform a traditional, direct radiograph to density reconstruction in the presence of scatter, even when relatively small amounts of scatter are present. Our experiments on synthetic data show that the approach can produce high quality, robust reconstructions. We also show that the distance (in feature space) between a testing radiograph and the training set can serve as a diagnostic of the accuracy of the reconstruction.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Deep Active Learning for Solvability Prediction in Power Systems

Traditional methods for solvability region analysis can only have inner approximations with inconclusive conservatism and handle limited types of power flow models. In this letter, we propose a deep active learning framework for solvability prediction in power systems. Compared with passive learning where the training is performed after all instances are labeled, active learning selects most informative instances to be labeled and therefore significantly reduces the size of the labeled dataset for training. In the active learning framework, the acquisition functions, which correspond to different sampling strategies, are defined in terms of the on-the-fly posterior probability from the classifier. First, the IEEE 39-bus system is employed to validate the proposed framework, where a two-dimensional case is illustrated to visualize the effectiveness of the sampling method followed by the high-dimensional numerical experiments. Then, the Northeast Power Coordinating Council (NPCC) 140-bus system is used to validate the performance on large-scale power systems.

42 ENGINEERING↗
NASA NTRS2003

Motor System Development Depends on Experience: A Microgravity Study of Rats

Animals move about their environment by sensing their surroundings and making adjustments according to need. All animals take the force of gravity into account when the brain and spinal cord undertake the planning and execution of movements. To what extent must animals learn to factor in the force of gravity when making neural calculations about movement? Are animals born knowing how to respond to gravity, or must the young nervous system learn to enter gravity into the equation? To study this issue, young rats were reared in two different gravitational environments (the one-G of Earth and the microgravity of low Earth orbit) that necessitated two different types of motor operations (movements) for optimal behavior. We inquired whether those portions of the young nervous system involved in movement, the motor system, can adapt to different gravitational levels and, if so, the cellular basis for this phenomenon. We studied two groups of rats that had been raised for 16 days in microgravity (eight or 14 days old at launch) and compared their walking and righting (ability to go from upside down to upright) and brain structure to those of control rats that developed on Earth. Flight rats were easily distinguished from the age-matched ground control rats in terms of both motor function and central nervous system structure. Mature surface righting predominated in control rats on the day of landing (R+O), while immature righting predominated in the flight rats on landing day and 30 days after landing. Some of these changes appear to be permanent. Several conclusions can be drawn from these studies: (1) Many aspects of motor behavior are preprogrammed into the young nervous system. In addition, several aspects of motor behavior are acquired as a function of the interaction of the developing organism and the rearing environment; (2) Widespread neuroanatomical differences between one-G- and microgravity-reared rats indicate that there is a structural basis for the adaptation to the rearing environment. These observations provide support for the idea that an animal's motor system adapts for optimal function within the environment experienced during a critical period in early postnatal life.

Walton, Kerry D.↗
DOE OSTI2020

TEAM project review, year 1

This note summarizes our research activities within the TEAM project between October 2019 and November 2020, funded by the ASCR Advanced Research in Quantum Computing program. Three research efforts have begun during the review period. The first effort, resulting in the JuQBox software, implemented a novel algorithm for optimal control in closed quantum systems based on symplectic time integration and exact calculations of the gradient of the discretized objective function. In our second effort, resulting in a code called Quandary, we consider the optimal control problem for open quantum systems. Here the evolution of the quantum system is modeled by the Lindblad master equation. In this code, we are considering both the realization of quantum gates as well as the optimal reset problem in which a general mixed quantum state is driven towards its ground state. In our third research effort we have begun the development of computational software for what we call Lindblad learning, i.e. the process of characterizing the equations governing our quantum computing hardware.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Task-oriented machine learning surrogates for tipping points of agent-based models

We present a machine learning framework bridging manifold learning, neural networks, Gaussian processes, and Equation-Free multiscale approach, for the construction of different types of effective reduced order models from detailed agent-based simulators and the systematic multiscale numerical analysis of their emergent dynamics. The specific tasks of interest here include the detection of tipping points, and the uncertainty quantification of rare events near them. Our illustrative examples are an event-driven, stochastic financial market model describing the mimetic behavior of traders, and a compartmental stochastic epidemic model on an Erdös-Rényi network. We contrast the pros and cons of the different types of surrogate models and the effort involved in learning them. Importantly, the proposed framework reveals that, around the tipping points, the emergent dynamics of both benchmark examples can be effectively described by a one-dimensional stochastic differential equation, thus revealing the intrinsic dimensionality of the normal form of the specific type of the tipping point. This allows a significant reduction in the computational cost of the tasks of interest.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Sparse Cholesky factorization for solving nonlinear PDEs via Gaussian processes

In recent years, there has been widespread adoption of machine learning-based approaches to automate the solving of partial differential equations (PDEs). Among these approaches, Gaussian processes (GPs) and kernel methods have garnered considerable interest due to their flexibility, robust theoretical guarantees, and close ties to traditional methods. They can transform the solving of general nonlinear PDEs into solving quadratic optimization problems with nonlinear, PDE-induced constraints. However, the complexity bottleneck lies in computing with dense kernel matrices obtained from pointwise evaluations of the covariance kernel, and its partial derivatives, a result of the PDE constraint and for which fast algorithms are scarce. The primary goal of this paper is to provide a near-linear complexity algorithm for working with such kernel matrices. We present a sparse Cholesky factorization algorithm for these matrices based on the near-sparsity of the Cholesky factor under a novel ordering of pointwise and derivative measurements. The near-sparsity is rigorously justified by directly connecting the factor to GP regression and exponential decay of basis functions in numerical homogenization. We then employ the Vecchia approximation of GPs, which is optimal in the Kullback-Leibler divergence, to compute the approximate factor. This enables us to compute ϵ-approximate inverse Cholesky factors of the kernel matrices with complexity O(N log d (N/ϵ)) in space and O(N log 2d (N/ϵ)) in time. We integrate sparse Cholesky factorizations into optimization algorithms to obtain fast solvers of the nonlinear PDE. We numerically illustrate our algorithm’s near-linear space/time complexity for a broad class of nonlinear PDEs such as the nonlinear elliptic, Burgers, and Monge-Ampère equations. In summary, we provide a fast, scalable, and accurate method for solving general PDEs with GPs and kernel methods.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Lagrangian large eddy simulations via physics-informed machine learning

High-Reynolds number homogeneous isotropic turbulence (HIT) is fully described within the Navier–Stokes (NS) equations, which are notoriously difficult to solve numerically. Engineers, interested primarily in describing turbulence at a reduced range of resolved scales, have designed heuristics, known as large eddy simulation (LES). LES is described in terms of the temporally evolving Eulerian velocity field defined over a spatial grid with the mean-spacing correspondent to the resolved scale. This classic Eulerian LES depends on assumptions about effects of subgrid scales on the resolved scales. Here, we take an alternative approach and design LES heuristics stated in terms of Lagrangian particles moving with the flow. Our Lagrangian LES, thus L-LES, is described by equations generalizing the weakly compressible smoothed particle hydrodynamics formulation with extended parametric and functional freedom, which is then resolved via Machine Learning training on Lagrangian data from direct numerical simulations of the NS equations. The L-LES model includes physics-informed parameterization and functional form, by combining physics-based parameters and physics-inspired Neural Networks to describe the evolution of turbulence within the resolved range of scales. The subgrid-scale contributions are modeled separately with physical constraints to account for the effects from unresolved scales. We build the resulting model under the differentiable programming framework to facilitate efficient training. We experiment with loss functions of different types, including physics-informed ones accounting for statistics of Lagrangian particles. We show that our L-LES model is capable of reproducing Eulerian and unique Lagrangian turbulence structures and statistics over a range of turbulent Mach numbers.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗