SEARCH · Engineering Papers

Results for “Operator learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10

All sources NASA NTRS DOE OSTI

DOE OSTI2022

A comprehensive and fair comparison of two neural operators (with practical extensions) based on $\mathrm{FAIR}$ data

Neural operators can learn nonlinear mappings between function spaces and offer a new simulation paradigm for real-time prediction of complex dynamics for realistic diverse applications as well as for system identification in science and engineering. Herein, we investigate the performance of two neural operators, which have shown promising results so far, and we develop new practical extensions that will make them more accurate and robust and importantly more suitable for industrial-complexity applications. The first neural operator, DeepONet, was published in 2019 (Lu et al., 2019), and its original architecture was based on the universal approximation theorem of Chen & Chen (1995). The second one, named Fourier Neural Operator or FNO, was published in 2020, and it is based on parameterizing the integral kernel in the Fourier space. DeepONet is represented by a summation of products of neural networks (NNs), corresponding to the branch NN for the input function and the trunk NN for the output function; both NNs are general architectures, e.g., the branch NN can be replaced with a CNN or a ResNet. According to Kovachki et al. (2021), FNO in its continuous form can be viewed conceptually as a DeepONet with a specific architecture of the branch NN and a trunk NN represented by a trigonometric basis. In order to compare FNO with DeepONet computationally for realistic setups, we develop several extensions of FNO that can deal with complex geometric domains as well as mappings where the input and output function spaces are of different dimensions. We also develop an extended DeepONet with special features that provide inductive bias and accelerate training, and we present a faster implementation of DeepONet with cost comparable to the computational cost of FNO, which is based on the Fast Fourier Transform. Here we consider 16 different benchmarks to demonstrate the relative performance of the two neural operators, including instability wave analysis in hypersonic boundary layers, prediction of the vorticity field of a flapping airfoil, porous media simulations in complex-geometry domains, etc. We follow the guiding principles of FAIR (Findability, Accessibility, Interoperability, and Reusability) for scientific data management and stewardship. The performance of DeepONet and FNO is comparable for relatively simple settings, but for complex geometries the performance of FNO deteriorates greatly. We also compare theoretically the two neural operators and obtain similar error estimates for DeepONet and FNO under the same regularity assumptions.

42 ENGINEERING↗

DOE OSTI2022

Fed-DeepONet: Stochastic Gradient-Based Federated Training of Deep Operator Networks

The Deep Operator Network (DeepONet) framework is a different class of neural network architecture that one trains to learn nonlinear operators, i.e., mappings between infinite-dimensional spaces. Traditionally, DeepONets are trained using a centralized strategy that requires transferring the training data to a centralized location. Such a strategy, however, limits our ability to secure data privacy or use high-performance distributed/parallel computing platforms. To alleviate such limitations, in this paper, we study the federated training of DeepONets for the first time. That is, we develop a framework, which we refer to as Fed-DeepONet, that allows multiple clients to train DeepONets collaboratively under the coordination of a centralized server. To achieve Fed-DeepONets, we propose an efficient stochastic gradient-based algorithm that enables the distributed optimization of the DeepONet parameters by averaging first-order estimates of the DeepONet loss gradient. Then, to accelerate the training convergence of Fed-DeepONets, we propose a moment-enhanced (i.e., adaptive) stochastic gradient-based strategy. Finally, we verify the performance of Fed-DeepONet by learning, for different configurations of the number of clients and fractions of available clients, (i) the solution operator of a gravity pendulum and (ii) the dynamic response of a parametric library of pendulums.

Moya, Christian↗

DOE OSTI2022

Data-Driven Operator Theoretic Methods for Phase Space Learning and Analysis

This paper uses data-driven operator theoretic approaches to explore the global phase space of a dynamical system. In this work, we defined conditions for discovering new invariant subspaces in the state space of a dynamical system starting from an invariant subspace based on the spectral properties of the Koopman operator. When the system evolution is known locally in several invariant subspaces in the state space of a dynamical system, a phase space stitching result is derived that yields the global Koopman operator. Additionally, in the case of equivariant systems, a phase space stitching result is developed to identify the global Koopman operator using the symmetry properties between the invariant subspaces of the dynamical system and time-series data from any one of the invariant subspaces. Finally, these results are extended to topologically conjugate dynamical systems; in particular, the relation between the Koopman tuple of topologically conjugate systems is established. The proposed results are demonstrated on several second-order nonlinear dynamical systems including a bistable toggle switch. Our method elucidates a strategy for designing discovery experiments: experiment execution can be done in many steps, and models from different invariant subspaces can be combined to approximate the global Koopman operator.

42 ENGINEERING↗

DOE OSTI2019

Network-based Classification of Authentication Attempts using Machine Learning

Network security operators are challenged with protecting an increasing number of clients from authentication-based attacks such as password guessing. Host-based defenses help in preventing such attacks but are difficult to manage and monitor at scale. These challenges open the door for network-based defenses. In this work, we introduce AuthML. AuthML performs protocol-agnostic authentication modeling to detect successful and unsuccessful authentication attempts at the network level. Using machine learning (ML), AuthML operates directly on network communication to determine the outcome of authentication attempts in real time. To show AuthML’s efficacy, we validate our approach on multiple deployment scenarios. AuthML achieves an accuracy of 99.9% examining 29,015 new flows in this operational phase, demonstrating that we can achieve similar performance in real time to state-of-the-art techniques without manual protocol analysis.

Taylor, Curtis↗

DOE OSTI2023

Machine learning for optimizing the manufacture and operation of intermediate temperature fuel cells

The project developed machine learning and artificial intelligence (ML/AI) methodologies for guiding and accelerating the materials design and processing of new high temperature proton conducing electrode ionomer binders and reinforced membranes for high-temperature proton exchange membrane fuel cell (HT-PEMFC; aka intermediate-temperature polymer electrolyte membrane fuel cells).

08 HYDROGEN↗

DOE OSTI2021

A machine learning degradation model for electrochemical capacitors operated at high temperature

Electrochemical capacitors (ECs) have only recently been considered as an alternative power source for telemetry sensors of drilling equipment for geothermal or oil and gas exploration. The lifecycle analysis and modelling of ECs is underrepresented in literature in comparison to other storage devices e.g. Li-ion batteries. This paper investigates the degradation of ECs when cycled outside the manufacturer-specified operating temperature envelope and proposes a machine learning-based approach for modelling the degradation. Experimental results show that end of life, defined as a 30% decrease in capacitance, occurs at 1,000 cycles when the environmental temperature exceeds the maximum operating temperature by 30%. The life-cycle test data is then used as an input to a Gaussian process regression (GPR) algorithm to predict the capacitance fade trend. The GPR is validated on a total of nine commercial cells from two different manufacturers, achieving an average root mean squared percent error of less than 2% and a mean calibration score of 93% when referenced to a 95% confidence interval. The model can be utilized to determine the EC degradation rate at a range of operating temperature values.

42 ENGINEERING↗

DOE OSTI2019

Learning Deep Neural Network Representations for Koopman Operators of Nonlinear Dynamical Systems

The Koopman operator has recently garnered much attention for its value in dynamical systems analysis and data-driven model discovery. However, its application has been hindered by the computational complexity of extended dynamic mode decomposition; this requires a combinatorially large basis set to adequately describe many nonlinear systems of interest, e.g. cyber-physical infrastructure systems, biological networks, social systems, and fluid dynamics. Often the dictionaries generated for these problems are manually curated, requiring domain-specific knowledge and painstaking tuning. In this paper we introduce a deep learning framework for learning Koopman operators of nonlinear dynamical systems. We show that this novel method automatically selects efficient deep dictionaries, outperforming state-of-the-art methods. We benchmark this method on partially observed nonlinear systems, including the glycolytic oscillator and show it is able to predict quantitatively 100 steps into the future, using only a single timepoint, and qualitative oscillatory behavior 400 steps into the future.

koopman operators, machine learning, deep learning↗

DOE OSTI2021

Non-intrusive nonlinear model reduction via machine learning approximations to low-dimensional operators

Abstract Although projection-based reduced-order models (ROMs) for parameterized nonlinear dynamical systems have demonstrated exciting results across a range of applications, their broad adoption has been limited by their intrusivity: implementing such a reduced-order model typically requires significant modifications to the underlying simulation code. To address this, we propose a method that enables traditionally intrusive reduced-order models to be accurately approximated in a non-intrusive manner. Specifically, the approach approximates the low-dimensional operators associated with projection-based reduced-order models (ROMs) using modern machine-learning regression techniques. The only requirement of the simulation code is the ability to export the velocity given the state and parameters; this functionality is used to train the approximated low-dimensional operators. In addition to enabling nonintrusivity, we demonstrate that the approach also leads to very low computational complexity, achieving up to $$10^3{\times }$$ 10 3 × in run time. We demonstrate the effectiveness of the proposed technique on two types of PDEs. The domain of applications include both parabolic and hyperbolic PDEs, regardless of the dimension of full-order models (FOMs).

42 ENGINEERING↗

DOE OSTI2022

Optimization of Water-Alternating-CO2 Injection Field Operations Using a Machine-Learning-Assisted Workflow

Summary This paper will present a robust workflow to address multiobjective optimization (MOO) of carbon dioxide (CO2)-enhanced oil recovery (EOR)-sequestration projects with a large number of operational control parameters. Farnsworth unit (FWU) field, a mature oil reservoir undergoing CO2 alternating water injection (CO2-WAG) EOR, will be used as a field case to validate the proposed optimization protocol. The expected outcome of this work would be a repository of Pareto-optimal solutions of multiple objective functions, including oil recovery, carbon storage volume, and project economics. FWU’s numerical model is used to demonstrate the proposed optimization workflow. Because using MOO requires computationally intensive procedures, machine-learning-based proxies are introduced to substitute for the high-fidelity model, thus reducing the total computation overhead. The vector machine regression combined with the Gaussian kernel (Gaussian-SVR) is used to construct proxies. An iterative self-adjusting process prepares the training knowledge base to develop robust proxies and minimizes computational time. The proxies’ hyperparameters will be optimally designed using Bayesian optimization to achieve better generalization performance. Trained proxies will be coupled with multiobjective particle swarm Optimization (MOPSO) protocol to construct the Pareto-front solution repository. The outcomes of this workflow will be a repository containing Pareto-optimal solutions of multiple objectives considered in the CO2-WAG project. The proposed optimization workflow will be compared with another established methodology using a multilayer neural network (MLNN) to validate its feasibility in handling MOO with a large number of parameters to control. Optimization parameters used include operational variables that might be used to control the CO2-WAG process, such as the duration of the water/gas injection period, producer bottomhole pressure (BHP) control, and water injection rate of each well included in the numerical model. It is proved that the workflow coupling Gaussian-SVR proxies and the iterative self-adjusting protocol is more computationally efficient. The MOO process is made more rapid by squeezing the size of the required training knowledge base while maintaining the high accuracy of the optimized results. The outcomes of the optimization study show promising results in successfully establishing the solution repository considering multiple objective functions. Results are also verified by validating the Pareto fronts with simulation results using obtained optimized control parameters. The outcome from this work could provide field operators an opportunity to design a CO2-WAG project using as many inputs as possible from the reservoir models. The proposed work introduces a novel concept that couples Gaussian-SVR proxies with a self-adjusting protocol to increase the computational efficiency of the proposed workflow and to guarantee the high accuracy of the obtained optimized results. More importantly, the workflow can optimize a large number of control parameters used in a complex CO2-WAG process, which greatly extends its utility in solving large-scale MOO problems in various projects with similar desired outcomes.

Energy & Fuels↗

DOE OSTI2020

Basic Design Layout of a Modular Pumped Hydro Teaching Unit

This work was originally completed in July 2015 as a proposal to Santa Fe Community College (SFCC) for their engineering program and a follow up to a feasibility study completed for the Community College the year earlier as indicated below. The work was ahead of its time and funding was hard to obtain to implement a project. This work is now being formally released for distribution to a wider audience based on the evolving circumstances in greater society that now favor this approach with funding more readily available. Due to the distribution outside of SFCC, specific architectural drawing sets of the pertinent buildings called out in the design have been removed. No changes to the design have been made. The design layout utilizing a building is not meant to limit the implementation of the design. The elevation difference required may be obtained with a building, a small hill, jacking up the upper tank, or similar means. The design presented is a basic design, identifying major components and how they are connected together. The user may change components under their own liability. All local applicable codes and standards must be followed when building an actual unit, under the authority of a responsible engineer or owner. Appropriate measures must be followed to protect people from hazards such as electricity, water pressure, and general construction. A leak test to verify water tightness of the system should be conducted prior to any operations. The vision of this design is to allow a class of 10 to 15 students to construct the assembly within a month or so time period, and then learn to operate the assembly at their school location under the guidance of their instructor. This will allow the students to learn about the practical concept of storing electricity with water, and gain a hands on experience of the real world challenges faced with full Modular Pumped Hydro Projects.

13 HYDRO ENERGY↗

DOE OSTI2025

A Neural Optimizer With Decision-Focused Learning for Optimal Energy Storage Operation

Here, this article introduces a neural optimizer-based framework for optimizing battery energy storage system (BESS) control for grid services, including demand charge and energy cost reduction. By leveraging decision-focused learning (DFL), the proposed framework ensures seamless integration and adaptation, significantly enhancing control performance. A patch time-series transformer is employed for peak load forecasting, incorporating aleatoric uncertainty quantification to account for forecasting uncertainties within the decision-making process. The framework utilizes a solver-in-the-loop approach to generate optimal BESS actions, which are then used to train the neural optimizer-based agent. By co-optimizing both BESS operational modes and output power within the NN, the system achieves improved performance and robustness. After initial training, the forecasting and control models are jointly fine-tuned to account for forecasting errors, further improving decision precision and efficiency through DFL. Case studies are performed to validate the performance of the framework using multiple real-world datasets, demonstrating superior performance in monthly peak load forecasting compared to state-of-the-art models. In addition, the results are compared against existing decision-making approaches. The results demonstrate a reduction in monthly peak forecasting error by approximately 15% across various performance measures and achieve an optimization gap for BESS operation that is about three times smaller compared to existing methods.

Kim, Hyeonjin [Pacific Northwest National Laborato↗

DOE OSTI2023

Adaptive Data-Driven Model Predictive Control for Heat Pipe Microreactors

To establish a technical basis for self-regulating microreactors, a model predictive control (MPC) system is investigated to proactively respond to anomalies and disturbances in anticipation of potential deviations from operating setpoints. Due to the difficulty of developing a physics-based surrogate model that can accurately match plant data in various operating conditions, machine learning algorithms are used in MPC, which allow for learning from both simulation and operation data, thus efficiently describing the targeted transient with arbitrary accuracy. However, one of the biggest concerns in applying ML algorithms like artificial neural networks (ANNs) is that the predictive capabilities of ANN are limited by training data. If there are gaps between the training and target domain, the accuracy of an ANN can degrade significantly when it is used to predict unseen data. To improve the predictive capability of ANN and enable a confident use of data-driven MPCs outside the training data, this study proposes an adaptive data-driven MPC framework. The system will monitor the discrepancy between plant responses and surrogate predictions, fine-tune the ANN-based surrogate when a large discrepancy is detected, and continue MPC operation with updated surrogates. The framework is demonstrated on a point kinetic model for microreactors. The hyperparameters of the update strategy, including layers to update, error thresholds, learning rate discount, and number of data points used for fine-tuning, are optimized so the simulated microreactor is able to follow changes in setpoint with the smallest of deviations.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2023

Adaptable Data Driven Model Predictive Control for Heat Pipe Microreactors

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2024

Learning Nonlinear Reduced Models from Data with Operator Inference

This review discusses Operator Inference, a nonintrusive reduced modeling approach that incorporates physical governing equations by defining a structured polynomial form for the reduced model, and then learns the corresponding reduced operators from simulated training data. The polynomial model form of Operator Inference is sufficiently expressive to cover a wide range of nonlinear dynamics found in fluid mechanics and other fields of science and engineering, while still providing efficient reduced model computations. The learning steps of Operator Inference are rooted in classical projection-based model reduction; thus, some of the rich theory of model reduction can be applied to models learned with Operator Inference. This connection to projection-based model reduction theory offers a pathway toward deriving error estimates and gaining insights to improve predictions. Furthermore, through formulations of Operator Inference that preserve Hamiltonian and other structures, important physical properties such as energy conservation can be guaranteed in the predictions of the reduced model beyond the training horizon. This review illustrates key computational steps of Operator Inference through a large-scale combustion example.

Mechanics↗

DOE OSTI2025

Reducing Frequency Bias of Fourier Neural Operators in 3D Seismic Wavefield Simulations Through Multistage Training

The recent development of neural operator (NeurOp) learning for solutions to the elastic wave equation shows promising results and provides the basis for fast large-scale simulations for different seismological applications. In this article, we use the Fourier neural operator (FNO) model to directly solve the 3D Helmholtz wave equation for fast seismic ground-motion simulations on different frequencies and show the frequency bias of the FNO model, that is, it learns the lower frequencies better comparing to the higher frequencies. To reduce the frequency bias, we adopt the multistage FNO training, that is, after training a stage 1 FNO model for estimating the ground motion, we use a second FNO model as the stage 2 to learn from the residual, which greatly reduced the errors on the higher frequencies. By adopting this multistage training, the FNO models show reduced biases on higher frequencies, which enhanced the overall results of the ground-motion simulations. Thus the multistage training FNO improves the accuracy and realism of the ground-motion simulations.

earthquakes↗

DOE OSTI2020

Concurrent Relaxation through Accelerated Deep Learning

CRADL captures performance metrics of machine learning algorithms operating on mesh data from multiphysics codes This proxy application is a tool to explore scalability of inference on HPC platforms, and also gather performance metrics for inference on new machine learning specific hardware. CRADL is designed to give users as fine a control as possible over an inference simulation. Users may select the number of cycles, amount of data, and batch size to pass to the accelerator of choice. Additionally the user may select a number of performance optimization libraries and flags. CRADL comes packaged with a repository of anonymized multi-physics simulation data, as well as a pretrained model for inference. The code allows a user to load their own pre-trained model and data if they wish. The code can operate in multiple parallelization schemes, with performance enhancing options such as half-precision libraries, PyTorch benchmarking, and pinned memory with non-blocking data transfers.

Zieb, KristoferJ.↗

DOE OSTI2024

Fourier-MIONet: Fourier-enhanced multiple-input neural operators for multiphase modeling of geological carbon sequestration

Geologic carbon sequestration (GCS) is a safety-critical technology that aims to reduce the amount of carbon dioxide in the atmosphere, which also places high demands on reliability. Multiphase flow in porous media is essential to understand CO 2 migration and pressure fields in the subsurface associated with GCS. However, numerical simulation for such problems in 4D is computationally challenging and expensive, due to the multiphysics and multiscale nature of the highly nonlinear governing partial differential equations (PDEs). It prevents us from considering multiple subsurface scenarios and conducting real-time optimization. Here, we develop a Fourier-enhanced multiple-input neural operator (Fourier-MIONet) to learn the solution operator of the problem of multiphase flow in porous media. Fourier-MIONet utilizes the recently developed framework of the multiple-input deep neural operators (MIONet) and incorporates the Fourier neural operator (FNO) in the network architecture. Once Fourier-MIONet is trained, it can predict the evolution of saturation and pressure of the multiphase flow under various reservoir conditions, such as permeability and porosity heterogeneity, anisotropy, injection configurations, and multiphase flow properties. Compared to the enhanced FNO (U-FNO), the proposed Fourier-MIONet has 90% fewer unknown parameters, and it can be trained in significantly less time (about 3.5 times faster) with much lower CPU memory (<15%) and GPU memory (<35%) requirements, to achieve similar prediction accuracy. In addition to the lower computational cost, Fourier-MIONet can be trained with only 6 snapshots of time to predict the PDE solutions for 30 years. Furthermore, we observed that Fourier-MIONet can maintain good accuracy when predicting out-of-distribution (OOD) data. The excellent generalizability of Fourier-MIONet is enabled by its adherence to the physical principle that the solution to a PDE is continuous over time. Furthermore, the developed Fourier-MIONet makes it possible to solve the long-time evolution of geological carbon sequestration in a large-scale three-dimensional space accurately and efficiently.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Exploring Multidimensional Spatial-Temporal Hydropower Operational Flexibilities by Modeling and Optimizing Water-Constrained Cascading Hydroelectric Systems

Because of unique characteristics such as clean and cost-competitive electricity as well as fast-ramping and storage abilities, the power industry continues to evolve its operation strategies for cascading hydroelectric (CHE) systems for providing enhanced values to the grid, especially under the deeper renewable resource integration. However, existing operation practices of CHEs predate the integration of renewables, which could prohibit the effective utilization of their inherent flexibilities in delivering maximum financial benefits and providing valuable grid services to the power system and electricity market operations. Indeed, modeling and optimizing these resource-limited while flexible CHE assets with uncertainties and imperfect information across multiple spatial-temporal dimensions present significant challenges. To facilitate CHE facility operators in effectively coordinating water usage and hydropower plant operations across multiple timescales, this project aims to fill the existing gaps by developing a suite of accurate water inflow (WI) forecast models as well as enhanced CHE modeling and optimization approaches with proper consideration of their unique characteristics, which would help explore their multidimensional spatial-temporal operational flexibility potentials. The developed approaches could better align reservoir operation strategies with variability and uncertainty of future water availability. They can also promote more effective utilization of multidimensional spatial-temporal hydropower operational flexibility potentials by designing long-term evacuation plans of reservoirs and short-term operation of CHEs, along with their coordination with other types of renewables. The project leverages various resources to facilitate the research and development activities, including actual characteristics data of CHE systems and a library of current and future cases of Portland General Electric (PGE). These realistic data enable the project team to study how to maximize the value of CHEs under current and future portfolios and evaluate opportunities to improve operation practices.

13 HYDRO ENERGY↗