SEARCH · Engineering Papers

Results for “Equation learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 847 records · Page 47

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Integrating deep neural networks with full-waveform inversion: Reparameterization, regularization, and uncertainty quantification

Full-waveform inversion (FWI) is an accurate imaging approach for modeling the velocity structure by minimizing the misfit between recorded and predicted seismic waveforms. However, the strong nonlinearity of FWI resulting from fitting oscillatory waveforms can trap the optimization in local minima. We have adopted a neural-network-based full-waveform inversion (NNFWI) method that integrates deep neural networks with FWI by representing the velocity model with a generative neural network. Neural networks can naturally introduce spatial correlations as regularization to the generated velocity model, which suppresses noise in the gradients and mitigates local minima. Furthermore, the velocity model generated by neural networks is input to the same partial differential equation (PDE) solvers used in conventional FWI. The gradients of the neural networks and PDEs are calculated using automatic differentiation, which back propagates gradients through the acoustic PDEs and neural network layers to update the weights of the generative neural network. Experiments on 1D velocity models, the Marmousi model, and the 2004 BP model determine that NNFWI can mitigate local minima, especially for imaging high-contrast features such as salt bodies, and it significantly improves the inversion in the presence of noise. Adding dropout layers to the neural network model also allows analyzing the uncertainty of the inversion results through Monte Carlo dropout. NNFWI opens a new pathway to combine deep learning and FWI for exploiting the characteristics of deep neural networks and the high accuracy of PDE solvers. Because NNFWI does not require extra training data and optimization loops, it provides an attractive and straightforward alternative to conventional FWI.

58 GEOSCIENCES↗

DOE OSTI2021

Verification of Data-Driven Models of Physical Phenomena using Interpretable Approximation

Machine-learned models, specifically neural networks, are increasingly used as “closures” or “constitutive models” in engineering simulators to represent fine-scale physical phenomena that are too computationally expensive to resolve explicitly. However, these neural net models of unresolved physical phenomena tend to fail unpredictably and are therefore not used in mission-critical simulations. In this report, we describe new methods to authenticate them, i.e., to determine the (physical) information content of their training datasets, qualify the scenarios where they may be used and to verify that the neural net, as trained, adhere to physics theory. We demonstrate these methods with neural net closure of turbulent phenomena used in Reynolds Averaged Navier-Stokes equations. We show the types of turbulent physics extant in our training datasets, and, using a test flow of an impinging jet, identify the exact locations where the neural network would be extrapolating i.e., where it would be used outside the feature-space where it was trained. Using Generalized Linear Mixed Models, we also generate explanations of the neural net (à la Local Interpretable Model agnostic Explanations) at prototypes placed in the training data and compare them with approximate analytical models from turbulence theory. Finally, we verify our findings by reproducing them using two different methods.

42 ENGINEERING↗

DOE OSTI2022

Efficient Interdependent Systems Recovery Modeling with DeepONets

Modeling the recovery of interdependent critical infrastructure is a key component of quantifying and optimizing societal resilience to disruptive events. However, simulating the recovery of large-scale interdependent systems under random disruptive events is computationally expensive. Therefore, we propose the application of Deep Operator Networks (DeepONets) in this paper to accelerate the recovery modeling of interdependent systems. DeepONets are ML architectures which identify mathematical operators from data. The form of governing equations DeepONets identify and the governing equation of interdependent systems recovery model are similar. Therefore, we hypothesize that DeepONets can efficiently model the interdependent systems recovery with little training data. We applied DeepONets to a simple case of four interdependent systems with sixteen states. DeepONets, overall, performed satisfactorily in predicting the recovery of these interdependent systems for out of training sample data when compared to reference results.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Robotic Planning under Uncertainty in Spatiotemporal Environments in Expeditionary Science

In the expeditionary sciences, spatiotemporally varying environments -- hydrothermal plumes, algal blooms, lava flows, or animal migrations -- are ubiquitous. Mobile robots are uniquely well-suited to study these dynamic, mesoscale natural environments. We formalize expeditionary science as a sequential decision-making problem, modeled using the language of partially-observable Markov decision processes (POMDPs). Solving the expeditionary science POMDP under real-world constraints requires efficient probabilistic modeling and decision-making in problems with complex dynamics and observational models. Previous work in informative path planning, adaptive sampling, and experimental design have shown compelling results, largely in static environments, using data-driven models and information-based rewards. However, these methodologies do not trivially extend to expeditionary science in spatiotemporal environments: they generally do not make use of scientific knowledge such as equations of state dynamics, they focus on information gathering as opposed to scientific task execution, and they make use of decision-making approaches that scale poorly to large, continuous problems with long planning horizons and real-time operational constraints. In this work, we discuss these and other challenges related to probabilistic modeling and decision-making in expeditionary science, and present some of our preliminary work that addresses these gaps. We ground our results in a real expeditionary science deployment of an autonomous underwater vehicle (AUV) in the deep ocean for hydrothermal vent discovery and characterization. Our concluding thoughts highlight remaining work to be done, and the challenges that merit consideration by the reinforcement learning and decision-making community.

Preston, Victoria↗

DOE OSTI2024

Uncertainty quantification of graph convolution neural network models of evolving processes

The application of neural network models to scientific machine learning tasks has proliferated in recent years. In particular, neural networks have proved to be adept at modeling processes with spatial–temporal complexity. Nevertheless, these highly parameterized models have garnered skepticism in their ability to produce outputs with quantified error bounds over the regimes of interest. Hence there is a need to find uncertainty quantification methods that are suitable for neural networks. In this work we present comparisons of the parametric uncertainty quantification of neural networks modeling complex spatial–temporal processes with Hamiltonian Monte Carlo and Stein variational gradient descent and its projected variant. Specifically we apply these methods to graph convolutional neural network models of evolving systems modeled with recurrent neural network and neural ordinary differential equations architectures. We show that Stein variational inference is a viable alternative to Monte Carlo methods with some clear advantages for complex neural network models. For our exemplars, Stein variational interference gave similar pushed forward uncertainty profiles through time compared to Hamiltonian Monte Carlo, albeit with generally more generous variance. As a result, projected Stein variational gradient descent also produced similar uncertainty profiles to the non-projected counterpart, but large reductions in the active weight space were confounded by the stability of the neural network predictions and the convoluted likelihood landscape.

36 MATERIALS SCIENCE↗

DOE OSTI2026

Data-based filtered dissipation rate modelling for multi-modal turbulent combustion: evaluating a priori model generalizability

Manifold-based models offer a computationally efficient alternative to directly transporting the thermochemical state in computational simulations of turbulent reacting flows, projecting the high-dimensional thermochemical state-space onto a low-dimensional manifold. Recent efforts have yielded a manifold-based model applicable to multi-modal combustion, enabling reconstruction of the thermochemical state from solutions to two-dimensional manifold equations in mixture fraction and generalized progress variable that are parameterised by three scalar dissipation rates. In coarse-grained simulations such as Large Eddy Simulation (LES), closure of the multi-modal manifold equations and subfilter variances/covariance requires closure of three filtered scalar dissipation rates. Here, the present work adopts a data-based approach, providing closure for the three filtered scalar dissipation rates via deep neural networks (DNNs). High-fidelity datasets corresponding to an autoigniting n-dodecane jet flame and a bluff body swirl-stabilized confined lifted spray flame of two aviation fuels (Jet-A and C1) with different ignition propensities are leveraged to generate training data that spans a diverse range of thermodynamic conditions and combustion modes, including low- and high-temperature ignition regimes in addition to premixed and nonpremixed behaviour. A final DNN model is trained to enforce inherent physical constraints by learning nonlinear functional transformations of the three filtered scalar dissipation rates. The generalizability of this constrained DNN model is demonstrated a priori via conditional statistics evaluated on the lifted spray flame with C1–a configuration that had not been included in the training data. Excellent DNN agreement with conditional DNS statistics is observed, and integrated gradients are computed to identify the most sensitive input variables. The similarity of the marginal PDFs of the most informative input variables and outputs across configurations are quantified via the Wasserstein metric, demonstrating that data-based models may successfully generalize to unseen parametric conditions so long as the most informative input variables share similar distributions across training and testing datasets.

Data-based modelling↗

NASA NTRS1998

Expert System Architecture for Rocket Engine Numerical Simulators: A Vision

Simulation of any complex physical system like rocket engines involves modeling the behavior of their different components using mostly numerical equations. Typically a simulation package would contain a set of subroutines for these modeling purposes and some other ones for supporting jobs. A user would create an input file configuring a system (part or whole of a rocket engine to be simulated) in appropriate format understandable by the package and run it to create an executable module corresponding to the simulated system. This module would then be run on a given set of input parameters in another file. Simulation jobs are mostly done for performance measurements of a designed system, but could be utilized for failure analysis or a design job such as inverse problems. In order to use any such package the user needs to understand and learn a lot about the software architecture of the package, apart from being knowledgeable in the target domain. We are currently involved in a project in designing an intelligent executive module for the rocket engine simulation packages, which would free any user from this burden of acquiring knowledge on a particular software system. The extended abstract presented here will describe the vision, methodology and the problems encountered in the project. We are employing object-oriented technology in designing the executive module. The problem is connected to the areas like the reverse engineering of any simulation software, and the intelligent systems for simulation.

Mitra, D.↗

DOE OSTI2022

Scale-invariant machine-learning model accelerates the discovery of quaternary chalcogenides with ultralow lattice thermal conductivity

We design an advanced machine-learning (ML) model based on crystal graph convolutional neural network that is insensitive to volumes (i.e., scale) of the input crystal structures to discover novel quaternary chalcogenides, AMM'Q 3 (A/M/M' = alkali, alkaline earth, post-transition metals, lanthanides, and Q = chalcogens). These compounds are shown to possess ultralow lattice thermal conductivity (κ l ), a desired requirement for thermal-barrier coatings and thermoelectrics. Upon screening the thermodynamic stability of ~1 million compounds using the ML model iteratively and performing density-functional theory (DFT) calculations for a small fraction of compounds, we discover 99 compounds that are validated to be stable in DFT. Taking several DFT-stable compounds, we calculate their κ l using Peierls–Boltzmann transport equation, which reveals ultralow κ l (<2 Wm -1 K -1 at room temperature) due to their soft elasticity and strong phonon anharmonicity. Our work demonstrates the high efficiency of scale-invariant ML model in predicting novel compounds and presents experimental-research opportunities with these new compounds.

36 MATERIALS SCIENCE↗

DOE OSTI2023

Variational deep learning of equilibrium transition path ensembles

Here, we present a time-dependent variational method to learn the mechanisms of equilibrium reactive processes and efficiently evaluate their rates within a transition path ensemble. This approach builds off of the variational path sampling methodology by approximating the time-dependent commitment probability within a neural network ansatz. The reaction mechanisms inferred through this approach are elucidated by a novel decomposition of the rate in terms of the components of a stochastic path action conditioned on a transition. This decomposition affords an ability to resolve the typical contribution of each reactive mode and their couplings to the rare event. The associated rate evaluation is variational and systematically improvable through the development of a cumulant expansion. We demonstrate this method in both over- and under-damped stochastic equations of motion, in low-dimensional model systems, and in the isomerization of a solvated alanine dipeptide. In all examples, we find that we can obtain quantitatively accurate estimates of the rates of the reactive events with minimal trajectory statistics and gain unique insights into transitions through the analysis of their commitment probability.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2024

Non-Intrusive Parallel-in-Time Solvers for Partial Differential Equations (Final Report)

Many time-dependent problems and simulations are often modeled using Partial Differential Equations. Traditional modeling approaches that use sequential time-stepping are reaching a bottleneck in optimizing efficiency. The Center of Applied Science and Computing at Lawrence Livermore National Laboratory extensively works on parallelizing these algorithms to leverage the increasing computational power from the growing number of processors in computer hardware. In particular, they aim to design non-intrusive algorithms that can generalize to a variety of problems and sizes without requiring additional information from or modifications on the original problems. Multigrid Reduction in Time (MGRIT) is a parallel-in-time algorithm that is designed to be non-intrusive. This project focuses on increasing the efficiency of MGRIT by approximating the coarse-grid operator using machine learning approaches as a means to find the most non-intrusive, or general, solution.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Solution of the linear wave-particle kinetic equation for global modes of arbitrary frequency in a tokamak

The linear response of a plasma to perturbations of arbitrary frequency and wavelength is derived for any axisymmetric magnetized toroidal plasma. An explicit transformation to action-angle coordinates is achieved using orthogonal magnetic coordinates and the Littlejohn Lagrangian, establishing the validity of this result to arbitrary order in normalized Larmor radius. The global resonance condition for compressional modes is clarified in more detail than in previous works, confirming that the poloidal orbit-average of the cyclotron frequency gives the desired result at lowest order in Larmor radius. The global plasma response to the perturbation at each resonance is captured by a poloidal and gyroaverage of the perturbing potential. A “global gyroaveraging” of the potential is a natural by-product of this analysis which takes into account the changing of the magnetic field over an orbit. The resonance condition depends on two arbitrary integers which completely separately capture the effects poloidal non-uniformity and finite Larmor radius in generating sidebands. We learn that poloidal sidebands generated for compressional modes are dominated by the change in gyrofrequency over the orbit, which is very different to shear modes where the gyrofrequency only contributes via a finite Larmor radius effect. This increases the number of bounce harmonics required to compute the linear drive, giving a more complicated resonance map. An example calculation is given comparing resonance of shear and compressional modes in a published DIII-D case.

Compressional↗

DOE OSTI2021

First coupled GENE–XGC microturbulence simulations

Covering the core and the edge region of a tokamak, respectively, the two gyrokinetic turbulence codes Gyrokinetic Electromagnetic Numerical Experiment (GENE) and X-point Gyrokinetic Code (XGC) have been successfully coupled by exchanging three-dimensional charge density data needed to solve the gyrokinetic Poisson equation over the entire spatial domain. Certain challenges for the coupling procedure arise from the fact that the two codes employ completely different numerical methods. This includes, in particular, the necessity to introduce mapping procedures for the transfer of data between the unstructured triangular mesh of XGC and the logically rectangular grid (in a combination of real and Fourier space) used by GENE. Constraints on the coupling scheme are also imposed by the use of different time integrators. First, coupled simulations are presented. We have considered collisionless ion temperature gradient turbulence, in both circular and fully shaped plasmas. Coupled simulations successfully reproduce both GENE and XGC reference results, confirming the validity of the code coupling approach toward a whole device model. Here, many lessons learned in the present context, in particular, the need for a coupling procedure as flexible as possible, should be valuable to our and other efforts to couple different kinds of codes in pursuit of a more comprehensive description of complex real-world systems and will drive our further developments of a whole device model for fusion plasmas.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2026

A Statistician’s Overview of Physics-Informed Neural Networks for Spatio-Temporal Data

The recent success of deep neural network models with physical constraints (so-called, Physics-Informed Neural Networks, PINNs) has led to renewed interest in the incorporation of mechanistic information in predictive models. Statisticians and others have long been interested in this problem, which has led to several practical and innovative solutions dating back decades. In this overview, we focus on the problem of data-driven prediction and inference of dynamic spatio-temporal processes that include mechanistic information, such as would be available from partial differential equations, with a strong focus on the quantification of uncertainty associated with data, process, and parameters. Here, we give a brief review of several paradigms and focus our attention on Bayesian implementations given they naturally accommodate uncertainty quantification. We then show that it is straight-forward to include the Bayesian PINN (B-PINN) within the Bayesian hierarchical model (BHM) framework that has long been considered for modeling dynamic spatio-temporal processes. Such a BHM-PINN is illustrated via a simulation study in which a latent nonlinear Burgers’ equation PDE governs the dynamics of Poisson distributed spatio-temporal data. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Bayesian↗

DOE OSTI2025

Surrogate modeling of Cellular-Potts agent-based models as a segmentation task using the U-Net neural network architecture

The Cellular-Potts model is a powerful and ubiquitous framework for developing computational models for simulating complex multicellular biological systems. Cellular-Potts models (CPMs) are often computationally expensive due to the explicit modeling of interactions among large numbers of individual model agents and diffusive fields described by partial differential equations (PDEs). In this work, we develop a convolutional neural network (CNN) surrogate model using a U-Net architecture that accounts for periodic boundary conditions. We use this model to accelerate the evaluation of a mechanistic CPM previously used to investigate in vitro vasculogenesis. The surrogate model was trained to predict 100 computational steps ahead (Monte-Carlo steps, MCS), accelerating simulation evaluations by a factor of 562 times compared to single-core CPM code execution on CPU. Over short timescales of up to 3 recursive evaluations, or 300 MCS, our model captures the emergent behaviors demonstrated by the original Cellular-Potts model such as vessel sprouting, extension and anastomosis, and contraction of vascular lacunae. This approach demonstrates the potential for deep learning to serve as a step toward efficient surrogate models for CPM simulations, enabling faster evaluation of computationally expensive CPM simulations of biological processes.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2013

Analyses Made to Order: Using Transformation to Rapidly Configure a Multidisciplinary Environment

Aerospace problems are highly multidisciplinary. Four or more major disciplines are involved in analyzing any particular vehicle. Moreover, the choice of implementation technology of various subsystems can lead to a change of leading domain or reformation of the driving equations. An excellent example is the change of expertise required to consider aircraft built from composite or metallic structures, or those propelled by chemical or electrical thrusters. Another example is in the major reconfiguration of handling and stability equations with different control surface configuration (e.g., canards, t-tail v four-post tail). Combinatorial problems are also commonplace anytime that a major system is to be designed. If there are only 5 attributes of a design to consider with 4 different options, this is already 1024 options. Adding just 5 more dimensions to the study explodes the space to over one million. Even generous assumptions like the idea that only 10% of the combinations are physically feasible can only contain the problem for so long. To make matters worse, the simple number of combinations is only the beginning. Combining the issue of trade space size with the need to reformulate the design problem for many of the possibilities makes life exponentially more difficult. Advances in software modeling approaches have led to the development of model-driven architecture. This approach uses the transformation of models into inferred models (e.g. inferred execution traces from state machines) or the skeletons for code generation. When the emphasis on transformation is applied to aerospace, it becomes possible to exploit redundancy in the information specified in multiple domain models into a unified system model. F1urther, it becomes possible to overcome the combinatorial nature of specifying integrated system behavior by manually combining the equations governing a given component technology. Transformations from a system specification combined with a system-analysis mapping specification enable one-click combination of domain analyses. This is a flexibility that has been missing from many engineering codes, which often entangle design specification and physical examination much more than is required to conduct the analysis. This capability has been investigated and cultivated within the DARPA F6 program by a team of JPL and Phoenix Integration engineers building the Adapatable Systems Design and Analysis (ASDA) framework. By embracing system modeling with SysML and the Query-View-Transformation (QVT) language, the ASDA team has been able to build a flexible, easily reconfigurable framework for building up and solving large tradespaces. Examples of application and lessons learned in building the framework will be described in this paper. In addition, the motivation will be laid for various tool vendors to develop open model description standards while being able to maintain competitive advantage through proprietary algorithms and approaches. These standards will also be compared to the underpinnings of model-driven architecture and the OMG standards of the Meta-Object Facility (MOF), SysML, and QVT.

Cole, Bjorn↗

DOE OSTI2021

Flow over an espresso cup: inferring 3-D velocity and pressure fields from tomographic background oriented Schlieren via physics-informed neural networks

Tomographic background oriented Schlieren (Tomo-BOS) imaging measures density or temperature fields in three dimensions using multiple camera BOS projections, and is particularly useful for instantaneous flow visualizations of complex fluid dynamics problems. We propose a new method based on physics-informed neural networks (PINNs) to infer the full continuous three-dimensional (3-D) velocity and pressure fields from snapshots of 3-D temperature fields obtained by Tomo-BOS imaging. The PINNs seamlessly integrate the underlying physics of the observed fluid flow and the visualization data, hence enabling the inference of latent quantities using limited experimental data. In this hidden fluid mechanics paradigm, we train the neural network by minimizing a loss function composed of a data mismatch term and residual terms associated with the coupled Navier–Stokes and heat transfer equations. We first quantify the accuracy of the proposed method based on a two-dimensional synthetic data set for buoyancy-driven flow, and subsequently apply it to the Tomo-BOS data set, where we are able to infer the instantaneous velocity and pressure fields of the flow over an espresso cup based only on the temperature field provided by the Tomo-BOS imaging. Moreover, we conduct an independent PIV experiment to validate the PINN inference for the unsteady velocity field at a centre plane. To explain the observed flow physics, we also perform systematic PINN simulations at different Reynolds and Richardson numbers and quantify the variations in velocity and pressure fields. Furthermore, the results in this paper indicate that the proposed deep learning technique can become a promising direction in experimental fluid mechanics.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1989

Optimal selection of space transportation fleet to meet multi-mission space program needs

A space program that spans several decades will be comprised of a collection of missions such as low earth orbital space station, a polar platform, geosynchronous space station, lunar base, Mars astronaut mission, and Mars base. The optimal selection of a fleet of several recoverable and expendable launch vehicles, upper stages, and interplanetary spacecraft necessary to logistically establish and support these space missions can be examined by means of a linear integer programming optimization model. Such a selection must be made because the economies of scale which comes from producing large quantities of a few standard vehicle types, rather than many, will be needed to provide learning curve effects to reduce the overall cost of space transportation if these future missions are to be affordable. Optimization model inputs come from data and from vehicle designs. Each launch vehicle currently in existence has a launch history, giving rise to statistical estimates of launch reliability. For future, not-yet-developed launch vehicles, theoretical reliabilities corresponding to the maturity of the launch vehicles' technology and the degree of design redundancy must be estimated. Also, each such launch vehicle has a certain historical or estimated development cost, tooling cost, and a variable cost. The cost of a launch used in this paper includes the variable cost plus an amortized portion of the fixed and development costs. The integer linear programming model will have several constraint equations based on assumptions of mission mass requirements, volume requirements, and number of astronauts needed. The model will minimize launch vehicle logistic support cost and will select the most desirable launch vehicle fleet.

Morgenthaler, George W.↗

DOE OSTI2022

Efficient high-dimensional variational data assimilation with machine-learned reduced-order models

Abstract. Data assimilation (DA) in geophysical sciences remains the cornerstone of robust forecasts from numerical models. Indeed, DA plays a crucial role in the quality of numerical weather prediction and is a crucial building block that has allowed dramatic improvements in weather forecasting over the past few decades. DA is commonly framed in a variational setting, where one solves an optimization problem within a Bayesian formulation using raw model forecasts as a prior and observations as likelihood. This leads to a DA objective function that needs to be minimized, where the decision variables are the initial conditions specified to the model. In traditional DA, the forward model is numerically and computationally expensive. Here we replace the forward model with a low-dimensional, data-driven, and differentiable emulator. Consequently, gradients of our DA objective function with respect to the decision variables are obtained rapidly via automatic differentiation. We demonstrate our approach by performing an emulator-assisted DA forecast of geopotential height. Our results indicate that emulator-assisted DA is faster than traditional equation-based DA forecasts by 4 orders of magnitude, allowing computations to be performed on a workstation rather than a dedicated high-performance computer. In addition, we describe accuracy benefits of emulator-assisted DA when compared to simply using the emulator for forecasting (i.e., without DA). Our overall formulation is denoted AIEADA (Artificial Intelligence Emulator-Assisted Data Assimilation).

58 GEOSCIENCES↗