SEARCH · Engineering Papers

Results for “Equation learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 505 records · Page 28

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Encoder–decoder neural network for solving the nonlinear Fokker–Planck–Landau collision operator in XGC

An encoder–decoder neural network has been used to examine the possibility for acceleration of a partial integro-differential equation, the Fokker–Planck–Landau collision operator. This is part of the governing equation in the massively parallel particle-in-cell code XGC, which is used to study turbulence in fusion energy devices. The neural network emphasizes physics-inspired learning, where it is taught to respect physical conservation constraints of the collision operator by including them in the training loss, along with the ℓ 2 loss. In particular, network architectures used for the computer vision task of semantic segmentation have been used for training. A penalization method is used to enforce the ‘soft’ constraints of the system and integrate error in the conservation properties into the loss function. During training, quantities representing the particle density, momentum and energy for all species of the system are calculated at each configuration vertex, mirroring the procedure in XGC. This simple training has produced a median relative loss, across configuration space, of the order of 10 –4 , which is low enough if the error is of random nature, but not if it is of drift nature in time steps. The run time for the current Picard iterative solver of the operator is O(n 2 ), where n is the number of plasma species. As the XGC1 code begins to attack problems including a larger number of species, the collision operator will become expensive computationally, making the neural network solver even more important, especially since its training only scales as O(n). Here, a wide enough range of collisionality has been considered in the training data to ensure the full domain of collision physics is captured. An advanced technique to decrease the losses further will be subject of a subsequent report. Eventual work will include expansion of the network to include multiple plasma species.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2021

Turbulent field fluctuations in gyrokinetic and fluid plasmas

A key uncertainty in the design and development of magnetic confinement fusion energy reactors is predicting edge plasma turbulence. An essential step in overcoming this uncertainty is the validation in accuracy of reduced turbulent transport models. Drift-reduced Braginskii two-fluid theory is one such set of reduced equations that has for decades simulated boundary plasmas in experiment, but significant questions exist regarding its predictive ability. To this end, using a novel physics-informed deep learning framework, we demonstrate the first ever direct quantitative comparisons of turbulent field fluctuations between electrostatic two-fluid theory and electromagnetic gyrokinetic modeling with good overall agreement found in magnetized helical plasmas at low normalized pressure. This framework presents a new technique for the numerical validation and discovery of reduced global plasma turbulence models.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2021

Turbulent field fluctuations in gyrokinetic and fluid plasmas

A key uncertainty in the design and development of magnetic confinement fusion energy reactors is predicting edge plasma turbulence. An essential step in overcoming this uncertainty is the validation in accuracy of reduced turbulent transport models. Drift-reduced Braginskii two-fluid theory is one such set of reduced equations that has for decades simulated boundary plasmas in experiment, but significant questions exist regarding its predictive ability. To this end, using a novel physics-informed deep learning framework, we demonstrate the first ever direct quantitative comparisons of turbulent field fluctuations between electrostatic two-fluid theory and electromagnetic gyrokinetic modelling with good overall agreement found in magnetized helical plasmas at low normalized pressure. This framework is readily adaptable to experimental and astrophysical environments, and presents a new technique for the numerical validation and discovery of reduced global plasma turbulence models.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2023

Using Co-Optimized Machine Learned Manifolds for Modeling Chemically Reacting Flows

Chemically reacting flows play a key role in a wide range of engineered systems, from chemical and polymer processing to combustion-based energy conversion technologies. Simulations of these flows involve solving a coupled set of partial differential equations for mass, momentum, energy, and all relevant chemical species in the system. Chemical reaction pathways may be extremely complex and involve hundreds or more intermediate species, with reactions that occur over timescales varying by several orders of magnitude - presenting a significant numerical stiffness challenge. The combination of these factors makes simulation of chemically reacting flows vastly more expensive than nonreactive simulations, and often makes direct solution of the governing equations intractable. It is necessary to apply lower-fidelity models in place of the detailed governing equations in order to reduce computational cost to enable reacting flow simulation tools to be used in the engineering design process. Many of the models employed for this purpose are based on reducing the dimension of the thermochemical state, motivated by the observation that the observed thermochemical states in a system lie on a low-dimensional manifold in thermochemical state space. This behavior occurs due to the fast equilibration of certain reactive and transport processes, and physics-based manifold models rely on idealized assumptions about the balance of timescales and the way in which chemistry and transport are coupled. In this work, we apply a novel method for data-driven manifold-based modeling that can leverage data from high-fidelity reacting flow simulations to improve model accuracy in cases where the physics-based modeling assumptions break down. The approach is designed to be broadly applicable across chemically reacting flow systems but is applied here to turbulent combustion modeling.

machine learning↗

DOE OSTI2025

DS-TIDE: Harnessing Dynamical Systems for Efficient Time-Independent Differential Equation Solving

Time-Independent Differential Equations (TIDEs) are central to modeling equilibrium behavior across a wide range of scientific and engineering domains, from electrostatics to porous media flow. Conventional numerical solvers offer reliable solutions but incur significant computational costs due to fine-grained discretization and iterative procedures. Machine learning-based approaches address this by replacing iterative solving processes with one-time inference; however, their sophisticated models require extensive training resources that often exceed those of traditional solvers. Consequently, designing a TIDE solver that achieves high accuracy, broad applicability, and exceptional computational efficiency remains a fundamental challenge. In this paper, we propose DS-TIDE, a novel hardware solver that is inspired by, and subsequently leverages, the intrinsic connection between Dynamical Systems (DS) and Differential Equations (DEs) to efficiently and accurately solve TIDEs. DS-TIDE employs a CMOS-compatible DS-based processor, whose physical states evolve under carefully designed DE-driven dynamics and naturally converge to equilibrium -- the solution of the target TIDE -- within ~1µs on a ~1-watt DS-TIDE processor. To enhance expressivity, DS-TIDE incorporates Heterogeneous Dynamics with Temporal Layering (HDTL), which solves TIDEs through a three-stage DS evolution -- conditioning, solving, and decoding -- each governed by specialized dynamics. The entire evolution process is analogous to an infinitely deep neural network temporally unrolled, offering the system the capability of representing complex equations. Furthermore, DS-TIDE is equipped with an on-device DS-DE Auto-Alignment mechanism that dynamically adapts intrinsic hardware dynamics within milliseconds, effectively aligning the system’s dynamics to diverse target DEs. Experimental results across TIDEs from a wide range of scientific and engineering domains demonstrate that DS-TIDE achieves ~10^3× speedup, ~10^5× energy savings, and competitive or superior accuracy compared to state-of-the-art numerical and ML-based solvers.

Liu, Chuan↗

DOE OSTI2024

Physics-Informed Active Learning With Simultaneous Weak-Form Latent Space Dynamics Identification

The parametric greedy latent space dynamics identification (gLaSDI) framework has demonstrated promising potential for accurate and efficient modeling of high-dimensional nonlinear physical systems. However, it remains challenging to handle noisy data. Here, to enhance robustness against noise, we incorporate the weak-form estimation of nonlinear dynamics (WENDy) into gLaSDI. In the proposed weak-form gLaSDI (WgLaSDI) framework, an autoencoder and WENDy are trained simultaneously to discover intrinsic nonlinear latent-space dynamics of high-dimensional data. Compared with the standard sparse identification of nonlinear dynamics (SINDy) employed in gLaSDI, WENDy enables variance reduction and robust latent space discovery, therefore leading to more accurate and efficient reduced-order modeling. Furthermore, the greedy physics-informed active learning in WgLaSDI enables adaptive sampling of optimal training data on the fly for enhanced modeling accuracy. The effectiveness of the proposed framework is demonstrated by modeling various nonlinear dynamical problems, including viscous and inviscid Burgers' equations, time-dependent radial advection, and the Vlasov equation for plasma physics. With data that contains 5%–10% Gaussian white noise, WgLaSDI outperforms gLaSDI by orders of magnitude, achieving 1%–7% relative errors. Compared with the high-fidelity models, WgLaSDI achieves 121 to 1779x speed-up.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Differentiable vertex fitting for jet flavor tagging

This work explores the use of differentiable programming to integrate domain knowledge, in the form of domain specific software, into neural networks to develop scientific machine learning systems. We propose a differentiable vertex fitting algorithm that estimates the crossing point of multiple curves. In the high energy physics setting, these curves are defined by particle equations of motion and the crossing point represents the origin of particle production. This differentiable vertex fitting algorithm can be seamlessly integrated into neural networks, and we show its utility and efficacy in the high energy physics application of the classification of jets, i.e., collimated streams of particles in particle detectors whose originating parent particle we aim to classify. We demonstrate how differentiable vertex fitting can be integrated into larger transformer-based models for jet flavor tagging and show improvements in heavy flavor jet classification when compared to baseline models. Published by the American Physical Society 2024

Smith, Rachel E. C. (ORCID:0000000335851262)↗

DOE OSTI2024

Creation Synthetic Data to Train a Digital Twin to Predict Reactor Operations

Understanding techniques to strengthen the nuclear safeguards regime is crucial in preventing nuclear proliferation due to recent advancements in the nuclear energy industry such as Generation IV reactors and microreactors. Prior to the construction of a nuclear power plant, it is necessary to understand the proliferation potential of the plant's reactor. Digital twins serve as a unique solution to recognizing reactor behavior indicative of nuclear proliferation. A digital twin is defined as a virtual model that works in unison to represent a physical asset, with a transference of data between the virtual and physical assets [1]. This work serves as validation for training a digital twin on synthetic data fabricated via means of Serpent reactor physics and point kinetics equations simulations. In this case this work is based on parameters of Idaho State University's AGN-201 reactor. The synthetic data can then be utilized to train machine learning models in the future to further investigate the utility of these methods. The accuracy of the predicted data is measured against real operational data to verify the reliability of the synthetic data creation methods and decide whether these methods should be used in the future to inform inspectors of a reactor's proliferation potential.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

DOE OSTI2025

Kinetic Deep Learning v0.1

Here, we present a method that uses protein levels to predict times series of metabolite concentrations. Understanding this type of pathway dynamics is important in order to predict the behavior of the pathway and, more pragmatically, to be able to design biological systems (such as strains bioengineered to produce chemical products) reliably. Typically, for this purpose, kinetic models consisting of differential equations based on the Michaelis-Menten dynamics have been used in the past. However, these methods can rarely produce good fits to measured data time series. Possibly, this happens because the kinetic constants are unknown or are different from the ones measured in vivo, or perhaps because Michaelis-Menten dynamics is not a satisfactory description. In order to improve the predictive nature of these kinetic models we have eliminated the Michaelis-Menten description of pathway dynamics and we have substituted it by algorithms that automatically learn these dynamics from previously obtained metabolomics and proteomics data using machine learning approaches. Specifically, kinetic deep learning uses deep learning to map proteomics time series to metabolite concentration time series, instead of learning the first metabolite derivative and integrating in (as in the first version of kinetic learning). This approach is shown to provide good to excellent results with a data set specifically collected for this purpose.

Garcia Martin, Hector [Joint BioEnergy Institute (↗

DOE OSTI2026

Machine Learning Approach to Modeling of Neutral Particles Transport in Plasma

A propagator‐based approach is investigated for Monte‐Carlo (MC) modeling of neutral particle transport in fusion boundary plasmas. The propagator is based on a Green's function for the neutral kinetic equation, which depends on the plasma profiles. A neural network (NN)‐based model for the propagator provides a fast and accurate solution for the neutral distribution function in plasma. Preliminary results from a small 1D test problem look encouraging. The proposed approach, a propagator‐based NN model for neutral transport in plasma, has potential for generalization to higher dimensions and efficient coupling with plasma models.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2021

Challenging Practices of Algebraic Battery Life Models through Statistical Validation and Model Identification via Machine-Learning

Various modeling techniques are used to predict the capacity fade of Li-ion batteries. Algebraic reduced-order models, which are inherently interpretable and computationally fast, are ideal for use in battery controllers, technoeconomic models, and multi-objective optimizations. For Li-ion batteries with graphite anodes, solid-electrolyte-interphase (SEI) growth on the graphite surface dominates fade. This fade is often modeled using physically informed equations, such as square-root of time for predicting solvent-diffusion limited SEI growth, and Arrhenius and Tafel-like equations predicting the temperature and state-of-charge rate dependencies. In some cases, completely empirical relationships are proposed. However, statistical validation is rarely conducted to evaluate model optimality, and only a handful of possible models are usually investigated. This article demonstrates a novel procedure for automatically identifying reduced-order degradation models from millions of algorithmically generated equations via bi-level optimization and symbolic regression. Identified models are statistically validated using cross-validation, sensitivity analysis, and uncertainty quantification via bootstrapping. On a LiFePO 4 /Graphite cell calendar aging data set, automatically identified models utilizing square-root, power law, stretched exponential, and sigmoidal functions result in greater accuracy and lower uncertainty than models identified by human experts, and demonstrate that previously known physical relationships can be empirically "rediscovered" using machine learning.

25 ENERGY STORAGE↗

DOE OSTI2026

Leveraging machine learning to enhance aerosol classification using Single-Particle Mass Spectrometry

Advancing automated classification of atmospheric aerosols from Single-Particle Mass Spectrometry (SPMS) data remains challenging due to overlapping ion signatures, compositional diversity, and limited labeled data. This study evaluates supervised and semi-supervised learning frameworks to enhance aerosol identification by jointly leveraging labeled and unlabeled spectra. Four models were compared: a supervised Support Vector Machine (SVM), a self-training SVM, a stacked autoencoder classifier, and a stacked autoencoder trained using a temporal-ensembling Mean Teacher approach. All models achieved high and stable accuracies (90.0 %–91.1 %), surpassing previous results on the same dataset (87 %) and matching the performance of state-of-the-art deep learning methods. Despite small global metric differences (≤ 1 %), semi-supervised variants yielded up to 5 %–10 % improvements for compositionally rare particle types – such as soot (0.77 % of spectra, F1-score: 0.93–0.97) and hazelnut pollen (0.98 % of spectra, F1-score: 0.97–1.00) – equating to roughly ∼ 187 additional correctly classified spectra. These gains are scientifically significant, as such rare particles exert disproportionate influence on radiative absorption and ice nucleation processes; their improved detection reduces modeled uncertainties in aerosol absorption optical depth and mixed-phase cloud ice nucleation rates. The models' residual misclassifications (≈ 9 %) largely arise from true spectral overlap among chemically adjacent species (e.g., Na- vs. K-feldspar, coated vs. uncoated feldspars), reflecting physical compositional continuity rather than algorithmic error. Collectively, these findings demonstrate that leveraging unlabeled data to learn robust spectral representations and refine classification enhances both fidelity and interpretability, bridging data-driven analysis with aerosol–climate process understanding.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2022

Machine learning prediction of connectivity, biodiversity and resilience in the Coral Triangle

Abstract Even optimistic climate scenarios predict catastrophic consequences for coral reef ecosystems by 2100. Understanding how reef connectivity, biodiversity and resilience are shaped by climate variability would improve chances to establish sustainable management practices. In this regard, ecoregionalization and connectivity are pivotal to designating effective marine protected areas. Here, machine learning algorithms and physical intuition are applied to sea surface temperature anomaly data over a twenty-four-year period to extract ecoregions and assess connectivity and bleaching recovery potential in the Coral Triangle and surrounding oceans. Furthermore, the impacts of the El Niño Southern Oscillation (ENSO) on biodiversity and resilience are quantified. We find that resilience is higher for reefs north of the Equator and that the extraordinary biodiversity of the Coral Triangle is dynamic in time and space, and benefits from ENSO. The large-scale exchange of genetic material is enhanced between the Indian Ocean and the Coral Triangle during La Niña years, and between the Coral Triangle and the central Pacific in neutral conditions. Through machine learning the outstanding biodiversity of the Coral Triangle, its evolution and the increase of species richness are contextualized through geological times, while offering new hope for monitoring its future.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2026

Machine learning-based interatomic potential development and phase transition analysis of ferroelectric hafnium dioxide

The ferroelectric phase (𝑃⁢𝑐⁢𝑎⁢2 1 , which is in orthorhombic symmetry) of hafnium dioxide (HfO 2 ) has gained much attention due to its potential applications in nanoelectronics and advanced memory devices. However, its complex phase behavior under external stimuli, such as pressure and temperature, remains a subject of intense investigation. This study focuses on developing a machine learning-based interatomic potential (MLIP) that is trained with data from density-functional theory (DFT) calculations to simulate phase transitions and mechanical properties of HfO 2 . The developed MLIP predicts lattice parameters, equations of state, bulk and shear moduli, and elastic constants that closely align with DFT predictions for several phases and at various pressures. Once validated, the MLIP is used to investigate the phase transitions of ferroelectric HfO 2 (𝑃⁢𝑐⁢𝑎⁢2 1 ) under both isobaric and constant stress conditions at elevated temperatures ranging from 200 to 2500 K. We used several complementary methods, including local symmetry identification, radial distribution function, and x-ray diffraction characterization, to identify interesting phase transitions among several competitive hafnia phases predicted from our simulations. The suggested methods uniformly reveal that under pure deviatoric condition, the system favors a transition from the orthorhombic 𝑃⁢𝑐⁢𝑎⁢2 1 phase to a tetragonal (𝑃⁢4 2 /𝑛⁢𝑚⁢𝑐) phase, whereas a zero stress condition drives the system from the 𝑃⁢𝑐⁢𝑎⁢2 1 phase to another orthorhombic (𝑃⁢𝑏⁢𝑐⁢𝑛) phase. These findings provide crucial insights into stress and temperature-induced phase behavior of hafnia, guiding future experimental and theoretical studies for optimizing hafnia-based ferroelectric devices.

Ferroelectric HfO2↗

DOE OSTI2026

The high explosives & affected targets (HEAT) dataset

Artificial Intelligence (AI) surrogate models offer a computationally efficient alternative to full-physics simulations, yet no existing datasets are publicly available for training, testing, and validation of machine learning models of the dynamics of high-explosive driven shocks through multiple materials. Shock propagation through materials is a computationally challenging problem because simulations must include material-specific equations of state (EOS) along with descriptions of other physical processes such as plastic deformation, phase change, damage processes, fluid instabilities, and multi-material interactions. Shocks are typically initiated by high-velocity impacts or explosive loading. The latter case necessitates the addition of models of reactive materials to represent high-explosive (HE) detonation. Here, to address the lack of an expansive dataset for multi-material shock propagation in the AI/ML community, we present the High-Explosives and Affected Targets (HEAT) Dataset. HEAT is a physics-rich collection of two-dimensional, cylindrically symmetric, simulations generated using an Eulerian, multi-material, shock-propagation code developed at Los Alamos National Laboratory. The dataset includes two partitions: (1) the expanding shock-cylinder (CYL) simulations, Figs. 1, and (2) the Perturbed Layered Interface (PLI) simulations, Fig. 2. Entries in both partitions consist of time series of arrays of thermodynamic fields (pressure, density, and temperature), kinematic fields (position and velocity), and additional fields that depend on thermodynamic and/or kinematic fields (e.g., material stress). Materials in the CYL partition include solids (aluminium, copper, depleted uranium, stainless steel, tantalum, and a generic polymer), a liquid (water), gases (air, nitrogen), and a generic detonating material (high explosive, HE). The PLI partition spans a highly varying geometry but consists of fixed materials across entries: Copper, aluminium, stainless steel, generic polymer, and generic HE. HEAT captures critical phenomena such as momentum transfer, shock propagation, plastic deformation, and thermal effects, making HEAT a valuable benchmark for development of AI/ML emulation of multi-material shock propagation.

36 MATERIALS SCIENCE↗

DOE OSTI2021

Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projection

Deep neural networks (DNNs) have achieved state-of-the-art performance across a variety of traditional machine learning tasks, e.g., speech recognition, image classification, and segmentation. The ability of DNNs to efficiently approximate high-dimensional functions has also motivated their use in scientific applications, e.g., to solve partial differential equations and to generate surrogate models. In this paper, we consider the supervised training of DNNs, which arises in many of the above applications. We focus on the central problem of optimizing the weights of the given DNN such that it accurately approximates the relation between observed input and target data. Devising effective solvers for this optimization problem is notoriously challenging due to the large number of weights, nonconvexity, data sparsity, and nontrivial choice of hyperparameters. To solve the optimization problem more efficiently, we propose the use of variable projection (VarPro), a method originally designed for separable nonlinear least-squares problems. Our main contribution is the Gauss--Newton VarPro method (GNvpro) that extends the reach of the VarPro idea to nonquadratic objective functions, most notably cross-entropy loss functions arising in classification. These extensions make GNvpro applicable to all training problems that involve a DNN whose last layer is an affine mapping, which is common in many state-of-the-art architectures. In our four numerical experiments from surrogate modeling, segmentation, and classification, GNvpro solves the optimization problem more efficiently than commonly used stochastic gradient descent (SGD) schemes. Finally, GNvpro finds solutions that generalize well, and in all but one example better than well-tuned SGD methods, to unseen data points.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Neural network reconstruction of the dense matter equation of state from neutron star observables

The Equation of State (EoS) of strongly interacting cold and hot ultra-dense QCD matter remains a major challenge in the field of nuclear astrophysics. With the advancements in measurements of neutron star masses, radii, and tidal deformabilities, from electromagnetic and gravitational wave observations, neutron stars play an important role in constraining the ultra-dense QCD matter EoS. Here, in this work, we present a novel method that exploits deep learning techniques to reconstruct the neutron star EoS from mass-radius (M-R) observations. We employ neural networks (NNs) to represent the EoS in a model-independent way, within the range of ~1-7 times the nuclear saturation density. The unsupervised Automatic Differentiation (AD) framework is implemented to optimize the EoS, so as to yield through TOV equations, an M-R curve that best fits the observations. We demonstrate that this method works by rebuilding the EoS on mock data, i.e., mass-radius pairs derived from a randomly generated polytropic EoS. The reconstructed EoS fits the mock data with reasonable accuracy, using just 11 mock M-R pairs observations, close to the current number of actual observations.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2023

QuadConv: Quadrature-based convolutions with applications to non-uniform PDE data compression

We present a new convolution layer for deep learning architectures which we call QuadConv — an approximation to continuous convolution via quadrature. Our operator is developed explicitly for use on non-uniform, mesh-based data, and accomplishes this by learning a continuous kernel that can be sampled at arbitrary locations. Moreover, the construction of our operator admits an efficient implementation which we detail and construct. As an experimental validation of our operator, we consider the task of compressing partial differential equation (PDE) simulation data from fixed meshes. Here, we show that QuadConv can match the performance of standard discrete convolutions on uniform grid data by comparing a QuadConv autoencoder (QCAE) to a standard convolutional autoencoder (CAE). Further, we show that the QCAE can maintain this accuracy even on non-uniform data. In both cases, QuadConv also outperforms alternative unstructured convolution methods such as graph convolution.

Compression↗