Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Equation learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

A Neural Differential Equation Formulation for Modeling Atmospheric Effects in Hyperspectral Images

Atmospheric correction is the process for removing atmospheric effects from spectral data; a necessary step for recovering salient spectral properties. The complex interactions between the atmosphere and light are dominated by absorbance and scattering physics. Existing methods for modeling atmospheric interactions typically rely on deep knowledge of relevant environmental conditions and high-fidelity numerical simulations of the governing physics in order to obtain accurate estimates of these effects. Additionally, existing approaches often require a subject matter expert for pre/post-processing of the data. Model-based approaches for removing atmospheric effects struggle in situations where such domain expertise is not available, and require significant human effort and computational power even when that expertise is available. In contrast, we propose a data-driven approach the uses Neural Differential Equations (NDEs) to accurately learn the interactions between electromagnetic radiation and the atmospheric without access to location specific environmental information. Once trained, the NDE can be applied bi-directionally; to apply or remove atmospheric effects. We demonstrate the effectiveness and utility of these techniques on an example multi-spectral scene.

Koch, James V.↗
DOE OSTI2023

Domain Aware Deep-learning Algorithms Integrated with Scientific-computing Technologies (DADAIST)

This technical report summarized the contribution of the DADAIST project funded by the Data Model Convergence Initiative via the Laboratory Directed Research and Development (LDRD) investments at Pacific Northwest National Laboratory (PNNL). Specifically, we report the development of the NeuroMANCER (Neural Modules with Adaptive Nonlinear Constraints and Efficient Regularizations), a new open-source Scientific Machine Learning library for formulating and solving parametric constrained optimization problems, physics-informed system identification, and parametric optimal control problems. NeuroMANCER is using differentiable programming to combine modern data-driven models and optimization modeling language into a coherent algorithmic and software framework. NeuroMANCER is a Pytorch-based framework and adopts much of its philosophy focused on research and development, rapid prototyping, and streamlined deployment. Strong emphasis is given to extensibility, interoperability with the PyTorch ecosystem, and quick adaptability to custom domain problems. Neuromancer repository contains a comprehensive library of differentiable modules, including custom activation functions, matrix factorizations, deep learning architectures, neural differential equations, differential equation solvers, implicit layers such as iterative solvers, high-level API for symbolic expressions, API for modeling and control of dynamical systems, and extensive set of tutorial code examples in the form of python scripts and jupyter notebooks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Assimilating partial observation to enhance feedback control of stochastic dynamical systems

Here, in this paper, we present a novel methodology to tackle feedback optimal control problems in scenarios where the exact state of the controlled process is unknown. It integrates data assimilation techniques and optimal control solvers to manage partial observation of the state process, a common occurrence in practical scenarios. Traditional stochastic optimal control methods assume full state observation, which is often not feasible in real-world fluid dynamics control problems. Our approach underscores the significance of utilizing observational data to inform control policy design. Specifically, we introduce a kernel learning backward stochastic differential equation (SDE) filter to enhance data assimilation efficiency and propose a sample-wise stochastic optimization method within the stochastic maximum principle framework. We demonstrate the efficacy and accuracy of our method in the control of advection-diffusion-reaction flow problem and the Dubins airplane maneuvering problem with model uncertainty.

data driven↗
DOE OSTI2020

DPM: A deep learning PDE augmentation method with application to large-eddy simulation

A framework is introduced that leverages known physics to reduce overfitting in machine learning for scientific applications. The partial differential equation (PDE) that expresses the physics is augmented with a neural network that uses available data to learn a description of the corresponding unknown or unrepresented physics. Training within this combined system corrects for missing, unknown, or erroneously represented physics, including discretization errors associated with the PDE's numerical solution. For optimization of the network within the PDE, an adjoint PDE is solved to provide high-dimensional gradients, and a stochastic adjoint method (SAM) further accelerates training. Additionally, the approach is demonstrated for large-eddy simulation (LES) of turbulence. High-fidelity direct numerical simulations (DNS) of decaying isotropic turbulence provide the training data used to learn sub-filter-scale closures for the filtered Navier–Stokes equations. Out-of-sample comparisons show that the deep learning PDE method outperforms widely-used models, even for filter sizes so large that they become qualitatively incorrect. It also significantly outperforms the same neural network when a priori trained based on simple data mismatch, not accounting for the full PDE. Measures of discretization errors, which are well-known to be consequential in LES, point to the importance of the unified training formulation's design, which without modification corrects for them. For comparable accuracy, simulation runtime is significantly reduced. A relaxation of the typical discrete enforcement of the divergence-free constraint in the solver is also successful, instead allowing the DPM to approximately enforce incompressibility physics. Since the training loss function is not restricted to correspond directly to the closure to be learned, training can incorporate diverse data, including experimental data.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Gradient flow based phase-field modeling using separable neural networks

Allen–Cahn equation is a reaction–diffusion equation and is widely used for modeling phase separation. Machine learning methods for solving the Allen–Cahn equation in its strong form suffer from inaccuracies in collocation techniques, errors in computing higher-order spatial derivatives, and the large system size required by the space–time approach. To overcome these challenges, we propose solving the gradient flow of the Ginzburg–Landau free energy functional, which is equivalent to the Allen–Cahn equation, thereby avoiding the second-order spatial derivatives associated with the Allen–Cahn equation. A minimizing movement scheme is employed to solve the gradient flow problem, eliminating the complexities of a space–time approach. We utilize a separable neural network that efficiently represents the phase field through low-rank tensor decomposition. As we use the minimizing movement scheme to numerically solve the gradient flow problem, we thus, refer to the proposed method as the Separable Deep Minimizing Movement (SDMM) method. The evaluation of the functional in the minimizing movement scheme using the Gauss quadrature technique bypasses the inaccuracies associated with collocation techniques traditionally used to solve partial differential equations. A hyperbolic tangent transformation is introduced on the phase field prior to the evaluation of the functional to ensure that it remains strictly bounded within the values of the two phases. For this transformation, theoretical guarantee for energy stability of the minimizing movement scheme is established. Our results suggest that this transformation helps to improve the accuracy and efficiency significantly. The proposed method resolves the challenges faced by state-of-the-art machine learning techniques, outperforming them in both accuracy and efficiency. It is also the first machine learning method to achieve an order of magnitude speed improvement over the finite element method. In addition to its formulation and computational implementation, several case studies illustrate the applicability of the proposed method.

42 ENGINEERING↗
DOE OSTI2022

Equation‐Free Surrogate Modeling of Geophysical Flows at the Intersection of Machine Learning and Data Assimilation

Abstract There is a growing interest in developing data‐driven reduced‐order models for atmospheric and oceanic flows that are trained on data obtained either from high‐resolution simulations or satellite observations. The data‐driven models are non‐intrusive in nature and offer significant computational savings compared to large‐scale numerical models. These low‐dimensional models can be utilized to reduce the computational burden of generating forecasts and estimating model uncertainty without losing the key information needed for data assimilation (DA) to produce accurate state estimates. This paper aims at exploring an equation‐free surrogate modeling approach at the intersection of machine learning and DA in Earth system modeling. With this objective, we introduce an end‐to‐end non‐intrusive reduced‐order modeling (NIROM) framework equipped with contributions in modal decomposition, time series prediction, optimal sensor placement, and sequential DA. Specifically, we use proper orthogonal decomposition (POD) to identify the dominant structures of the flow, and a long short‐term memory network to model the dynamics of the POD modes. The NIROM is integrated within the deterministic ensemble Kalman filter (DEnKF) to incorporate sparse and noisy observations at optimal sensor locations obtained through QR pivoting. The feasibility and the benefit of the proposed framework are demonstrated for the NOAA Optimum Interpolation Sea Surface Temperature (SST) V2 data set. Our results indicate that the NIROM is stable for long‐term forecasting and can model dynamics of SST with a reasonable level of accuracy. Furthermore, the prediction accuracy of the NIROM gets improved by almost one order of magnitude by the DEnKF algorithm.

Pawar, Suraj↗
DOE OSTI2024

Learning the boundary-to-domain mapping using Lifting Product Fourier Neural Operators for partial differential equations

Neural operators such as the Fourier Neural Operator (FNO) have been shown to provide resolution-independent deep learning models that can learn mappings between function spaces. For example, an initial condition can be mapped to the solution of a partial differential equation (PDE) at a future time-step using a neural operator. Despite the popularity of neural operators, their use to predict solution functions over a domain given only data over the boundary (such as a spatially varying Dirichlet boundary condition) remains unexplored. In this paper, we refer to such problems as boundary-to-domain problems; they have a wide range of applications in areas such as fluid mechanics, solid mechanics, heat transfer etc. We present a novel FNO-based architecture, named Lifting Product FNO (or LP-FNO) which can map arbitrary boundary functions defined on the lower-dimensional boundary to a solution in the entire domain. Specifically, two FNOs defined on the lower-dimensional boundary are lifted into the higher dimensional domain using our proposed lifting product layer. We demonstrate the efficacy and resolution independence of the proposed LP-FNO for the 2D Poisson equation.

Kashi, Aditya↗
DOE OSTI2023

Model-parallel Fourier neural operators as learned surrogates for large-scale parametric PDEs

Fourier neural operators (FNOs) are a recently introduced neural network architecture for learning solution operators of partial differential equations (PDEs), which have been shown to perform significantly better than comparable deep learning approaches. Once trained, FNOs can achieve speed-ups of multiple orders of magnitude over conventional numerical PDE solvers. However, due to the high dimensionality of their input data and network weights, FNOs have so far only been applied to two-dimensional or small three-dimensional problems. To remove this limited problem-size barrier, we propose a model-parallel version of FNOs based on domain-decomposition of both the input data and network weights. Here, we demonstrate that our model-parallel FNO is able to predict time-varying PDE solutions of over 2.6 billion variables on Perlmutter using up to 512 A100 GPUs and show an example of training a distributed FNO on the Azure cloud for simulating multiphase CO 2 dynamics in the Earth’s subsurface.

58 GEOSCIENCES↗
NASA NTRS2010

The Emergence of Agent-Based Technology as an Architectural Component of Serious Games

The evolution of games as an alternative to traditional simulations in the military context has been gathering momentum over the past five years, even though the exploration of their use in the serious sense has been ongoing since the mid-nineties. Much of the focus has been on the aesthetics of the visuals provided by the core game engine as well as the artistry provided by talented development teams to produce not only breathtaking artwork, but highly immersive game play. Consideration of game technology is now so much a part of the modeling and simulation landscape that it is becoming difficult to distinguish traditional simulation solutions from game-based approaches. But games have yet to provide the much needed interactive free play that has been the domain of semi-autonomous forces (SAF). The component-based middleware architecture that game engines provide promises a great deal in terms of options for the integration of agent solutions to support the development of non-player characters that engage the human player without the deterministic nature of scripted behaviors. However, there are a number of hard-learned lessons on the modeling and simulation side of the equation that game developers have yet to learn, such as: correlation of heterogeneous systems, scalability of both terrain and numbers of non-player entities, and the bi-directional nature of simulation to game interaction provided by Distributed Interactive Simulation (DIS) and High Level Architecture (HLA).

Phillips, Mark↗
DOE OSTI2026

TANTE: Time-adaptive operator learning via neural Taylor expansion

Operator learning for time-dependent partial differential equations (PDEs) has seen rapid progress in recent years, enabling efficient approximation of complex spatiotemporal dynamics. However, most existing methods rely on fixed time step sizes during rollout, which limits their ability to adapt to varying temporal complexity and often leads to error accumulation. In this work, we propose the Time-Adaptive Transformer with Neural Taylor Expansion (TANTE), a novel operator-learning framework that produces continuous-time predictions with adaptive step sizes. TANTE predicts future states by performing a Taylor expansion at the current state, where neural networks learn both the higher-order temporal derivatives and the local radius of convergence. This allows the model to dynamically adjust its rollout based on the local behavior of the solution, thereby reducing cumulative error and improving computational efficiency. We demonstrate the effectiveness of TANTE across a wide range of PDE benchmarks, achieving superior accuracy and adaptability compared to fixed-step baselines, delivering accuracy gains of 60-80 % and speed-ups of 30-40 % at inference time.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Scientific machine learning for modeling and simulating complex fluids

The formulation of rheological constitutive equations—models that relate internal stresses and deformations in complex fluids—is a critical step in the engineering of systems involving soft materials. While data-driven models provide accessible alternatives to expensive first-principles models and less accurate empirical models in many engineering disciplines, the development of similar models for complex fluids has lagged. The diversity of techniques for characterizing non-Newtonian fluid dynamics creates a challenge for classical machine learning approaches, which require uniformly structured training data. Consequently, early machine-learning based constitutive equations have not been portable between different deformation protocols or mechanical observables. Here, we present a data-driven framework that resolves such issues, allowing rheologists to construct learnable models that incorporate essential physical information, while remaining agnostic to details regarding particular experimental protocols or flow kinematics. These scientific machine learning models incorporate a universal approximator within a materially objective tensorial constitutive framework. By construction, these models respect physical constraints, such as frame-invariance and tensor symmetry, required by continuum mechanics. We demonstrate that this framework facilitates the rapid discovery of accurate constitutive equations from limited data and that the learned models may be used to describe more kinematically complex flows. This inherent flexibility admits the application of these “digital fluid twins” to a range of material systems and engineering problems. We illustrate this flexibility by deploying a trained model within a multidimensional computational fluid dynamics simulation—a task that is not achievable using any previously developed data-driven rheological equation of state.

Science & Technology - Other Topics↗
DOE OSTI2022

Discovering equations that govern experimental materials stability under environmental stress using scientific machine learning

Abstract While machine learning (ML) in experimental research has demonstrated impressive predictive capabilities, extracting fungible knowledge representations from experimental data remains an elusive task. In this manuscript, we use ML to infer the underlying differential equation (DE) from experimental data of degrading organic-inorganic methylammonium lead iodide (MAPI) perovskite thin films under environmental stressors (elevated temperature, humidity, and light). Using a sparse regression algorithm, we find that the underlying DE governing MAPI degradation across a broad temperature range of 35 to 85 °C is described minimally by a second-order polynomial. This DE corresponds to the Verhulst logistic function, which describes reaction kinetics analogous to self-propagating reactions. We examine the robustness of our conclusions to experimental variance and Gaussian noise and describe the experimental limits within which this methodology can be applied. Our study highlights the promise and challenges associated with ML-aided scientific discovery by demonstrating its application in experimental chemical and materials systems.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Deducing neutron star equation of state parameters directly from telescope spectra with uncertainty-aware machine learning

Neutron stars provide a unique laboratory for studying matter at extreme pressures and densities. While there is no direct way to explore their interior structure, X-rays emitted from these stars can indirectly provide clues to the equation of state (EOS) of the superdense nuclear matter through the inference of the star's mass and radius. However, inference of EOS directly from a star's X-ray spectra is extremely challenging and is complicated by systematic uncertainties. The current state of the art is to use simulation-based likelihoods in a piece-wise method which relies on certain theoretical assumptions and simplifications about the uncertainties. It first infers the star's mass and radius to reduce the dimensionality of the problem, and from those quantities infer the EOS. Here we demonstrate a series of enhancements to the state of the art, in terms of realistic uncertainty quantification and a path towards circumventing the need for theoretical assumptions to infer physical properties with machine learning. We also demonstrate novel inference of the EOS directly from the high-dimensional spectra of observed stars, avoiding the intermediate mass-radius step. Our network is conditioned on the sources of uncertainty of each star, allowing for natural and complete propagation of uncertainties to the EOS.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Parametric matrix models

We present a general class of machine learning algorithms called parametric matrix models. In contrast with most existing machine learning models that imitate the biology of neurons, parametric matrix models use matrix equations that emulate physical systems. Similar to how physics problems are usually solved, parametric matrix models learn the governing equations that lead to the desired outputs. Parametric matrix models can be efficiently trained from empirical data, and the equations may use algebraic, differential, or integral relations. While originally designed for scientific computing, we prove that parametric matrix models are universal function approximators that can be applied to general machine learning problems. After introducing the underlying theory, we apply parametric matrix models to a series of different challenges that show their performance for a wide range of problems. For all the challenges tested here, parametric matrix models produce accurate results within an efficient and interpretable computational framework that allows for input feature extrapolation.

Computational science↗
NASA NTRSDate not supplied

Turbo-Design: Open-Source Radial Equilibrium Turbomachinery Solver: Part I - Turbines

Advances in 3D Geometrical Designs and Cooling have played a significant role in improving the efficiency of turbomachinery. However, these advancements must be effectively translated back to the modeler. Machine learning can facilitate this transition. Specifically, machine learning–based loss models can bridge the gap between 3D and 1D designs, enabling modelers not only to predict velocity triangles but also to extract additional geometric features. Currently, the design tools used at NASA have not been updated to support such integration—until now. TurboDesign is an open-source, Python-based framework that replaces TD2 (LEW-11029-1) and AXOD2 (LEW-16323-1), both of which are radial equilibrium solvers for axial turbines. The goal of this update is to enable the integration of machine learning loss models into radial equilibrium equations. Additionally, TurboDesign is designed to support radial machines. This paper presents the governing equations, the assumptions underlying the code, the integration of legacy loss models, an example of machine learning model integration, and a validation comparison with CFD. All code, tutorials, and documentation are available at: https://www.github.com/nasa/turbo-design

Radial Equilibrium↗
DOE OSTI2022

Quantum advantage for differential equation analysis

Quantum algorithms for differential equation solving, data processing, and machine learning potentially offer an exponential speedup over all known classical algorithms. However, there also exist obstacles to obtaining this potential speedup in useful problem instances. The essential obstacle for quantum differential equation solving is that outputting useful information may require difficult postprocessing, and the essential obstacle for quantum data processing and machine learning is that inputting the data is a difficult task just by itself. In this study, we demonstrate that, when combined, these difficulties solve one another. We show how the output of quantum differential equation solving can serve as the input for quantum data processing and machine learning, allowing dynamical analysis in terms of principal components, power spectra, and wavelet decompositions. To illustrate this, we consider continuous-time Markov processes on epidemiological and social networks. These quantum algorithms provide an exponential advantage over existing classical Monte Carlo methods.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Machine learning changes the rules for flux limiters

Learning to integrate non-linear equations from highly resolved direct numerical simulations has seen recent interest for reducing the computational load for fluid simulations. Here, we focus on determining a flux-limiter for shock capturing methods. Focusing on flux limiters provides a specific plug-and-play component for existing numerical methods. Since their introduction, an array of flux limiters has been designed. Using the coarse-grained Burgers' equation, we show that flux-limiters may be rank-ordered in terms of their log-error relative to high-resolution data. We then develop a theory to find an optimal flux-limiter and present flux-limiters that outperform others tested for integrating Burgers' equation on lattices with [Formula: see text], and 2x, 3x, 4x, and 8x coarse-grainings. We train a continuous piecewise linear limiter by minimizing the mean-squared misfit to six-grid point segments of high-resolution data, averaged over all segments. While flux limiters are generally designed to have an output of φ(r) = 1 at a flux ratio of r = 1, our limiters are not bound by this rule and yet produce a smaller error than standard limiters. Here we find that our machine learned limiters have distinctive features that may provide new rules-of-thumb for the development of improved limiters. Additionally, we use our theory to learn flux-limiters that outperform standard limiters across a range of values (as opposed to at a specific fixed value) of coarse-graining, number of discretized bins, and diffusion parameter. This demonstrates the ability to produce flux limiters that should be more broadly useful than standard limiters for general applications.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2018

Atmospheric Chemistry Modeling Using a Regression Forest Model

Atmospheric chemistry is central to many environmental issues such as air pollution, climate change, and stratospheric ozone loss. Chemistry Transport Models (CTM) are a central tool for understanding these issues, whether for research or for forecasting. These models split the atmosphere in a large number of grid-boxes and consider the emission of compounds into these boxes and their subsequent transport, deposition, and chemical processing. The chemistry is represented through a series of simultaneous ordinary differential equations, one for each compound. Given the difference in life-times between the chemical compounds (milli-seconds for O1D to years for CH4) these equations are numerically stiff and solving them consists of a significant fraction of the computational burden of a CTM. We have investigated a machine learning approach to solving the differential equations instead of solving them numerically. From an annual simulation of the GEOS-Chem model we have produced a training dataset consisting of the concentration of compounds before and after the differential equations are solved, together with some key physical parameters for every grid-box and time-step. From this dataset we have trained a machine learning algorithm (regression forest) to be able to predict the concentration of the compounds after the integration step based on the concentrations and physical state at the beginning of the time step. We have then included this algorithm back into the GEOS-Chem model, bypassing the need to integrate the chemistry. This machine learning approach shows many of the characteristics of the full simulation and has the potential to be substantially faster. There are a wide range of application for such an approach - generating boundary conditions, for use in air quality forecasts, chemical data assimilation systems, centennial scale climate simulations etc. We discuss our approches' speed and accuracy, and highlight some potential future directions for improving this approach.

Keller, Christoph A.↗