Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Deep operator network (DeepONet)”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

52 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Toward a Machine Learning Approach to Interpreting X-ray Spectra of Trace Impurities by Converting XANES to EXAFS

The fact that the photoabsorption spectrum of a material contains information about the atomic structure, commonly understood in terms of multiple scattering theory, is the basis of the popular extended X-ray absorption spectroscopy (EXAFS) technique. How much of the same structural information is present in other complementary spectroscopic signals is not obvious. Here we use a machine learning approach to demonstrate that within theoretical models that accurately predict the EXAFS signal, the extended near-edge region does indeed contain the EXAFS-accessible structural information. We do this by exhibiting deep operator neural networks (DeepONets) that have learned the relationship between the extended and near edge portions of the X-ray absorption spectrum to predict the former from the latter. We find that we can accurately predict the EXAFS spectrum between 6 and 14 Å –1 from the first 6 Å –1 (≈100 eV) of the absorption spectrum of Cu 2 + substitutional defects in the Fe 3+ mineral hematite (α-Fe 2 O 3 ). This surprising finding implies that theoretical analyses of X-ray absorption spectra could be implemented that extract the same conclusions as high-quality EXAFS studies from spectra collected over a much smaller range of photon energies. This relaxes a host of experimental limitations related to the X-ray source and measurement sample, including collection time, minimum dopant concentration, source brilliance, and energy range. We describe the theoretical data sets and DeepONet construction and show that the resulting DeepONets produce EXAFS that recovers linear combination fits to experimental data with accuracy approaching the original ab initio calculations. We discuss the implications of our findings for minor constituent characterization and for understanding the information content of spectroscopic data more broadly, including how this approach might be applied to measured experimental spectra. In conclusion, to encourage similar efforts, the simulated X-ray spectra, machine learning, and fitting code are publicly available.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Synergistic learning with multi-task DeepONet for efficient PDE problem solving

Multi-task learning (MTL) is an inductive transfer mechanism designed to leverage useful information from multiple tasks to improve generalization performance compared to single-task learning. It has been extensively explored in traditional machine learning to address issues such as data sparsity and overfitting in neural networks. In this work, we apply MTL to problems in science and engineering governed by partial differential equations (PDEs). However, implementing MTL in this context is complex, as it requires task-specific modifications to accommodate various scenarios representing different physical processes. To this end, we present a multi-task deep operator network (MT-DeepONet) to learn solutions across various functional forms of source terms in a PDE and multiple geometries in a single concurrent training session. We introduce modifications in the branch network of the vanilla DeepONet to account for various functional forms of a parameterized coefficient in a PDE. Additionally, we handle parameterized geometries by introducing a binary mask in the branch network and incorporating it into the loss term to improve convergence and generalization to new geometry tasks. Our approach is demonstrated on three benchmark problems: (1) learning different functional forms of the source term in the Fisher equation; (2) learning multiple geometries in a 2D Darcy Flow problem and showcasing better transfer learning capabilities to new geometries; and (3) learning 3D parameterized geometries for a heat transfer problem and demonstrate the ability to predict on new but similar geometries. Finally, our MT-DeepONet framework offers a novel approach to solving PDE problems in engineering and science under a unified umbrella based on synergistic learning that reduces the overall training cost for neural operators.

42 ENGINEERING↗
DOE OSTI2025

Computationally efficient models for aqueous organic redox flow batteries

The rising usage of intermittent energy has garnered the need for large scale energy storage systems. Redox flow batteries (RFB) based energy storage system shows promising potential. Numerical simulations and machine learning approaches have been widely used to study RFB performance. The development of autonomous material discovery framework and digital twin of energy storage system usually needs to query cell performance through fast response models. In this study, two computationally efficient models are introduced: a physics-based analytical flow battery model (EZBattery), and a machine learning operator model (Deep Operator Network, denoted by DeepONet). Both models can provide cell performance near instantly, and prediction accuracy was systematically examined on an application of evaluating the performances of a 780 cm 2 aqueous organic redox flow battery (AORFB), using potential anolyte candidates in dihydroxyphenazine (DHP)-based family of organic materials. A validated computationally expansive 3-dimensional multi-physics finite element model by COMSOL was used as the ground truth and provided the training data set for the DeepONet. 1280 samples were generated with 10 properties to mimic the different possible anolyte candidates, and the cell performances were evaluated under 10 different combined operating conditions. The accuracy comparisons for the two computationally efficient models show that both models can provide comparable accuracy in predicting cell charging/discharging voltage curves. DeepONet can provide slightly higher overall accuracy than EZBattery with faster calculation speed, but highly relies on the training dataset. EZBattery does not need a training dataset and can provide interpretable physics-based explanations of the results, while being more flexible to adjust to adapt any different cell designs, flow battery architectures, and electrolyte materials.

Analytical model↗
DOE OSTI2021

A seamless multiscale operator neural network for inferring bubble dynamics

Modelling multiscale systems from nanoscale to macroscale requires the use of atomistic and continuum methods and, correspondingly, different computer codes. Here, we develop a seamless method based on DeepONet, which is a composite deep neural network (a branch and a trunk network) for regressing operators. In particular, we consider bubble growth dynamics, and we model tiny bubbles of initial size from 100 nm to 10 $\mathrm {\mu }\textrm {m}$ , modelled by the Rayleigh–Plesset equation in the continuum regime above 1 $\mathrm {\mu }\textrm {m}$ and the dissipative particle dynamics method for bubbles below 1 $\mathrm {\mu }\textrm {m}$ in the atomistic regime. After an offline training based on data from both regimes, DeepONet can make accurate predictions of bubble growth on-the-fly (within a fraction of a second) across four orders of magnitude difference in spatial scales and two orders of magnitude in temporal scales. The framework of DeepONet is general and can be used for unifying physical models of different scales in diverse multiscale applications.

Mechanics↗
DOE OSTI2025

Self-adaptive weights based on balanced residual decay rate for physics-informed neural networks and deep operator networks

Physics-informed deep learning has emerged as a promising alternative for solving partial differential equations. However, for complex problems, training these networks can still be challenging, often resulting in unsatisfactory accuracy and efficiency. In this work, we demonstrate that the failure of plain physics-informed neural networks arises from the significant discrepancy in the convergence rate of residuals at different training points, where the slowest convergence rate dominates the overall solution convergence. Based on these observations, we propose a pointwise adaptive weighting method that balances the residual decay rate across different training points. The performance of our proposed adaptive weighting method is compared with current state-of-the-art adaptive weighting methods on benchmark problems for both physics-informed neural networks and physics-informed deep operator networks. In conclusion, through extensive numerical results we demonstrate that our proposed approach of balanced residual decay rates offers several advantages, including bounded weights, high prediction accuracy, fast convergence rate, low training uncertainty, low computational cost, and ease of hyperparameter tuning.

Balanced convergence rate↗
DOE OSTI2023

On the influence of over-parameterization in manifold based surrogates and deep neural operators

Constructing accurate and generalizable approximators (surrogate models) for complex physico-chemical processes exhibiting highly non-smooth dynamics is challenging. The main question is what type of surrogate models we should construct and should these models be under-parameterized or over-parameterized. In this work, we propose new developments and perform comparisons for two promising approaches: manifold-based polynomial chaos expansion (m-PCE) and the deep neural operator (DeepONet), and we examine the effect of over-parameterization on generalization. While m-PCE enables the construction of a mapping by first identifying low-dimensional embeddings of the input functions, parameters, and quantities of interest (QoIs), a neural operator learns the nonlinear mapping via the use of deep neural networks. Here, we demonstrate the performance of these methods in terms of generalization accuracy by solving the 2D time-dependent Brusselator reaction-diffusion system with uncertainty sources, modeling an autocatalytic chemical reaction between two species. We first propose an extension of the m-PCE by constructing a mapping between latent spaces formed by two separate embeddings of the input functions and the output QoIs. To further enhance the accuracy of the DeepONet, we introduce weight self-adaptivity in the loss function. We demonstrate that the performance of m-PCE and DeepONet is comparable for cases of relatively smooth input-output mappings. However, when highly non-smooth dynamics is considered, DeepONet shows higher approximation accuracy. We also find that for m-PCE, modest over-parameterization leads to better generalization, both within and outside of distribution, whereas aggressive over-parameterization leads to over-fitting. In contrast, an even highly over-parameterized DeepONet leads to better generalization for both smooth and non-smooth dynamics. Furthermore, we compare the performance of the above models with another recently proposed operator learning model, the Fourier Neural Operator, and show that its over-parameterization also leads to better generalization. Taken together, our studies show that m-PCE can provide very good accuracy at very low training cost, whereas a highly over-parameterized DeepONet can provide better accuracy and robustness to noise but at higher training cost. In both methods, the inference cost is negligible.

97 MATHEMATICS AND COMPUTING↗
NASA NTRSDate not supplied

DeepONet-Assisted Optimization of Surface Topography for Transition Delay in A Mach 4.5 Boundary Layer

We use deep learning, an ensemble variationaltechnique (EnVar), and direct numerical simulations(DNS) to design an optimal topography for a two-dimensional roughness element that delays the on-set of laminar-turbulent transition in a Mach 4.5 flat-plate boundary layer. Deep operator networks (Deep-ONets), which have the known ability to learn com-plex nonlinear operators within dynamical systems,are used for machine learning. For the baseline config-uration of a smooth flat plate, the second-mode wavesat the DNS inflow cause a quick nonlinear breakdownof the high-speed boundary layer within the computa-tional domain. Results reported in the present studyvalidate the ability of DeepONets to model the tran-sition delay via a given topography of the roughnesselement. The computing cost to optimize the rough-ness element for minimal skin-friction drag is substan-tially lowered by the DeepONets-based reduced-ordermodel. In comparison to the baseline method of EnVaroptimization based on DNS alone, the DeepONets-based EnVar optimizer is able to delay transition pastthe outflow boundary of the computational domainwhile utilizing almost 5–6 times fewer DNS.

Machine Learning↗
DOE OSTI2022

Neural operator prediction of linear instability waves in high-speed boundary layers

Here, we investigate if neural operators can predict the linear evolution of instability waves in high-speed boundary layers. To this end, we extend the design of the DeepOnet to ensure accurate and robust predictions, and also to perform data assimilation. In particular, we train DeepONet to take as inputs an upstream disturbance and a downstream location of interest, and to provide as output the perturbation field downstream in the boundary layer. DeepONet thus approximates the linearized and parabolized Navier-Stokes operator for this flow. For successful application to the high-speed boundary layer problem, we add sample weighting and Fourier input features to the regular DeepONet formulation. Once trained, the DeepOnet can perform fast and accurate predictions of the downstream disturbances within the range of training frequencies (inside the distribution). In addition, we show that DeepONet can solve the inverse problem, where downstream wall measurements are adopted as input, and a trained network can predict the upstream disturbances that led to these observations. This capability, along with the forward predictions, allows us to perform a full data assimilation cycle efficiently: starting from wall-pressure data, we predict the upstream disturbance using the inverse DeepONet and its evolution using the forward DeepONet. Finally, we introduce three new metrics to benchmark the training, evaluation and break-even cost of neural operators.

42 ENGINEERING↗
DOE OSTI2021

Forecasting Solar-Thermal Systems Performance under Transient Operation Using a Data-Driven Machine Learning Approach Based on the Deep Operator Network Architecture

Modeling and prediction of the dynamic behavior of thermal systems operating under intermittent energy input and variable load requirements represent one of the greatest challenges in the development of efficient and reliable renewable-based power generation technologies. In this work, a data-driven machine learning modeling framework was developed based on a modified version of the Deep Operator Network architecture where the time coordinate in the trunk net is replaced with historical data of the predicting quantity. The modeling framework can be used to accurately predict the performance of renewable-based energy conversion technologies including wind- and solar-based power plants. This novel framework was applied on a solar-thermal system that consists of a solar collection loop using a flat plate collector, a power generation loop comprising an Organic Rankine Cycle, and a thermal energy storage tank connecting both loops. Variable solar irradiance, air temperature, and power load profiles were used by the Deep Operator Network to predict the State-of-Charge and the efficiency of the thermal system for several days. The results were compared with the State-of-Charge and efficiency functions calculated using a physics-based model. For a simple operation scenario, characterized by a clear sky solar irradiance profile and constant load, the standard deviation in the State-of-Charge prediction by Deep Operator Network is below 0.9% during a seven-day prediction time horizon. For the most realistic operation scenario that considers real solar irradiance and a rough load profile, the maximum standard deviation in the predictions for the State-of-Charge and efficiency are below 6.8% and 2.5%, respectively. A comparison between Deep Operator Network and Long Short Term Memory network was also performed. In general, both networks predict very well the State-of-Charge for different data density conditions; however, a higher accuracy, with a standard deviation below 2.0%, is obtained by the Deep Operator Network during three and half days using sparser training data of 20-minute points. The same accuracy for the State-of-Charge prediction with the Long Short Term Memory network is achieved only for 14 h. Average standard deviations for the State-of-Charge prediction of 1.1% with the Deep Operator Network and 1.5% with the Long Short Term Memory network are obtained for a four-day prediction time using a denser training data of 5-minute points.

DeepONet↗
DOE OSTI2025

Fast meta-solvers for 3D complex-shape scatterers using neural operators trained on a non-scattering problem

Three-dimensional target identification using scattering techniques requires high accuracy solutions and very fast computations for real-time predictions in some critical applications. We first train a deep neural operator (DeepONet) to solve wave propagation problems described by the Helmholtz equation in a domain without scatterers but at different wavenumbers and with a complex absorbing boundary condition. We then design two classes of fast meta-solvers by combining DeepONet with either relaxation methods, such as Jacobi and Gauss-Seidel, or with Krylov methods, such as GMRES and BiCGStab, using the trunk basis of DeepONet as a coarse-scale preconditioner. We leverage the spectral bias of neural networks to account for the lower part of the spectrum in the error distribution while the upper part is handled inexpensively using relaxation methods or fine-scale preconditioners. The meta-solvers are then applied to solve scattering problems with different shape of scatterers, at no extra training cost. We first demonstrate that the resulting meta-solvers are shape-agnostic, fast, and robust, whereas the standard standalone solvers may even fail to converge without the DeepONet. We then apply both classes of meta-solvers to scattering from a submarine, a complex three-dimensional problem. We achieve very fast solutions, especially with the DeepONet-Krylov methods, which require orders of magnitude fewer iterations than any of the standalone solvers.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Long-time integration of parametric evolution equations with physics-informed DeepONets

Ordinary and partial differential equations (ODEs/PDEs) play a paramount role in analyzing and simulating complex dynamic processes across all corners of science and engineering. In recent years machine learning tools are aspiring to introduce new effective ways of simulating such equations, however existing approaches are not able to reliably return stable and accurate predictions across long temporal horizons. We aim to address this challenge by introducing an effective framework for learning evolution operators that map random initial conditions to associated ODE/PDE solutions within a short time interval. Such operators can be parametrized by deep neural networks that are trained in an entirely self-supervised manner without requiring one to generate any paired input-output observations. Global long-time predictions across a range of initial conditions can be then obtained by iteratively evaluating the trained model using each prediction as the initial condition for the next evaluation step. Here, this introduces a new approach to temporal domain decomposition that is shown to be effective in performing accurate long-time simulations for a wide range of parametric ODE and PDE systems, from wave propagation, to reaction-diffusion dynamics and stiff chemical kinetics, introducing a new way of rapidly emulating non-equilibrium processes in science and engineering.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

DeepM&Mnet: Inferring the electroconvection multiphysics fields based on operator approximation by neural networks

Electroconvection is a multiphysics problem involving coupling of the flow field with the electric field as well as the cation and anion concentration fields. Here, we use electroconvection as a benchmark problem to put forward a new data assimilation framework, the DeepM&Mnet, for simulating multiphysics and multiscale problems at speeds much faster than standard numerical methods using pre-trained neural networks. We first pre-train DeepONets that can predict independently each field, given general inputs from the rest of the fields of the coupled system. DeepONets can approximate nonlinear operators and are composed of two sub-networks, a branch net for the input fields and a trunk net for the locations of the output field. DeepONets, which are extremely fast, are used as building blocks in the DeepM&Mnet and form constraints for the multiphysics solution along with some sparse available measurements of any of the fields. We demonstrate the new methodology and document the accuracy of each individual DeepONet, and subsequently we present two different DeepM&Mnet architectures that infer accurately and efficiently 2D electroconvection fields for unseen electric potentials. Furthermore, the DeepM&Mnet framework is general and can be applied for building any complex multiphysics and multiscale models based on very few measurements using pre-trained DeepONets in a “plug-and-play” mode.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Bi-fidelity modeling of uncertain and partially unknown systems using DeepONets

Recent advances in modeling large-scale, complex physical systems have shifted research focuses towards data-driven techniques. However, generating datasets by simulating complex systems can require significant computational resources. Similarly, acquiring experimental datasets can prove difficult. For these systems, often computationally inexpensive, but in general inaccurate models, known as the low-fidelity models, are available. Here in this paper, we propose a bi-fidelity modeling approach for complex physical systems, where we model the discrepancy between the true system's response and a low-fidelity response in the presence of a small training dataset from the true system's response using a deep operator network, a neural network architecture suitable for approximating nonlinear operators. We apply the approach to systems that have parametric uncertainty and are partially unknown. Three numerical examples are used to show the efficacy of the proposed approach to model uncertain and partially unknown physical systems.

MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Sound propagation in realistic interactive 3D scenes with parameterized sources using deep neural operators

We address the challenge of acoustic simulations in three-dimensional (3D) virtual rooms with parametric source positions, which have applications in virtual/augmented reality, game audio, and spatial computing. The wave equation can fully describe wave phenomena such as diffraction and interference. However, conventional numerical discretization methods are computationally expensive when simulating hundreds of source and receiver positions, making simulations with parametric source positions impractical. To overcome this limitation, we propose using deep operator networks to approximate linear wave-equation operators. This enables the rapid prediction of sound propagation in realistic 3D acoustic scenes with parametric source positions, achieving millisecond-scale computations. By learning a compact surrogate model, we avoid the offline calculation and storage of impulse responses for all relevant source/listener pairs. Our experiments, including various complex scene geometries, show good agreement with reference solutions, with root mean squared errors ranging from 0.02 to 0.10 Pa. Notably, our method signifies a paradigm shift as—to our knowledge—no prior machine learning approach has achieved precise predictions of complete wave fields within realistic domains.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

RandONets: Shallow networks with random projections for learning linear and nonlinear operators

Deep neural networks have been extensively used for the solution of both the forward and the inverse problem for dynamical systems. However, their implementation necessitates optimizing a high-dimensional space of parameters and hyperparameters. This fact, along with the requirement of substantial computational resources, pose a barrier to achieving high numerical accuracy, but also interpretability. Here, to address the above challenges, we present Random Projection-based Operator Networks (RandONets): shallow networks with random projections and tailor-made numerical analysis methods that learn accurately and fast linear and nonlinear operators. Building on previous works, we prove that RandOnets are universal approximators of linear and nonlinear operators. Due to their simplicity, RandONets provide a one-step transformation of the input space, facilitating interpretability. For the evaluation of their performance, we focus on operators of PDEs. We show, that RandONets outperform by several orders of magnitude, both in terms of numerical approximation accuracy and computational cost, the “vanilla” DeepONets. Hence, we believe that our method will trigger further developments in the field of scientific machine learning, for the development of new ‘’light”schemes that will provide high accuracy while reducing dramatically the computational cost. A MATLAB toolbox for RandONets, including demos, is available on GitHub at https://github.com/GianlucaFabiani/RandONets.

Interpretable machine learning↗
DOE OSTI2022

A comprehensive and fair comparison of two neural operators (with practical extensions) based on $\mathrm{FAIR}$ data

Neural operators can learn nonlinear mappings between function spaces and offer a new simulation paradigm for real-time prediction of complex dynamics for realistic diverse applications as well as for system identification in science and engineering. Herein, we investigate the performance of two neural operators, which have shown promising results so far, and we develop new practical extensions that will make them more accurate and robust and importantly more suitable for industrial-complexity applications. The first neural operator, DeepONet, was published in 2019 (Lu et al., 2019), and its original architecture was based on the universal approximation theorem of Chen & Chen (1995). The second one, named Fourier Neural Operator or FNO, was published in 2020, and it is based on parameterizing the integral kernel in the Fourier space. DeepONet is represented by a summation of products of neural networks (NNs), corresponding to the branch NN for the input function and the trunk NN for the output function; both NNs are general architectures, e.g., the branch NN can be replaced with a CNN or a ResNet. According to Kovachki et al. (2021), FNO in its continuous form can be viewed conceptually as a DeepONet with a specific architecture of the branch NN and a trunk NN represented by a trigonometric basis. In order to compare FNO with DeepONet computationally for realistic setups, we develop several extensions of FNO that can deal with complex geometric domains as well as mappings where the input and output function spaces are of different dimensions. We also develop an extended DeepONet with special features that provide inductive bias and accelerate training, and we present a faster implementation of DeepONet with cost comparable to the computational cost of FNO, which is based on the Fast Fourier Transform. Here we consider 16 different benchmarks to demonstrate the relative performance of the two neural operators, including instability wave analysis in hypersonic boundary layers, prediction of the vorticity field of a flapping airfoil, porous media simulations in complex-geometry domains, etc. We follow the guiding principles of FAIR (Findability, Accessibility, Interoperability, and Reusability) for scientific data management and stewardship. The performance of DeepONet and FNO is comparable for relatively simple settings, but for complex geometries the performance of FNO deteriorates greatly. We also compare theoretically the two neural operators and obtain similar error estimates for DeepONet and FNO under the same regularity assumptions.

42 ENGINEERING↗