Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “operator learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

MLOps for Beam Controls

Machine learning operations (MLOps) is the standardization and streamlining of the ML development lifecycle to address the challenges associated with large-scale machine learning applications. The full MLOps pipeline consists of open-source tools: DataHub, MinIO and MLflow. It is being used for dataset management and model development to handle changing data dependencies, varying business needs, reproducibility, and diverse teams working with differing tools and skills. To demonstrate the completion of an MLOps pipeline for particle accelerator operations, we are deploying a simple script that computes settings for the Booster’s gradient magnet power supply. Once the demonstration is complete, we will develop and deploy ML-based optimization algorithms to improve Booster’s overall efficiency. This MLOps pipeline opens the gate to systematically develop and deploy ML applications for accelerator controls and diagnostics.

43 PARTICLE ACCELERATORS↗
DOE OSTI2024

MLOps for Beam Controls

Machine learning operations (MLOps) is the standardization and streamlining of the ML development lifecycle to address the challenges associated with large-scale machine learning applications. The full MLOps pipeline consists of open-source tools: DataHub, MinIO and MLflow. It is being used for dataset management and model development to handle changing data dependencies, varying business needs, reproducibility, and diverse teams working with differing tools and skills. To demonstrate the completion of an MLOps pipeline for particle accelerator operations, we are deploying a simple script that computes settings for the Booster’s gradient magnet power supply. Once the demonstration is complete, we will develop and deploy ML-based optimization algorithms to improve Booster’s overall efficiency. This MLOps pipeline opens the gate to systematically develop and deploy ML applications for accelerator controls and diagnostics.

43 PARTICLE ACCELERATORS↗
DOE OSTI2026

AI-Ready Control System for the Fermilab Accelerator Complex

Reliable, high-intensity operation of the Fermilab Accelerator Complex is critical to the success of the Long-Baseline Neutrino Facility and Deep Underground Neutrino Experiment. We describe the requirements and infrastructure necessary to support routine use of artificial intelligence and machine learning (AI/ML) in the accelerator control system. Three capabilities are identified: a machine learning operations (MLOps) framework standardizing the lifecycle of AI/ML automation from data management through deployment and monitoring; a data quality framework defining and enforcing standards required to build trustworthy AI/ML applications; and workflow integration with large language models to assist physicists, engineers, and operators with information retrieval, code development, and routine analysis. Use cases spanning beam diagnostics, beam control, and support system automation illustrate the technical requirements across the complex.

43 PARTICLE ACCELERATORS↗
DOE OSTI2025

Two-level overlapping additive Schwarz preconditioner for training scientific machine learning applications

In this work we introduce a novel two-level overlapping additive Schwarz preconditioner for accelerating the training of scientific machine learning applications. The design of the proposed preconditioner is motivated by the nonlinear two-level overlapping additive Schwarz preconditioner. The neural network parameters are decomposed into groups (subdomains) with overlapping regions. In addition, the network’s feed-forward structure is indirectly imposed through a novel subdomain-wise synchronization strategy and a coarse-level training step. Through a series of numerical experiments, which consider physicsinformed neural networks and operator learning approaches, we demonstrate that the proposed two-level preconditioner significantly speeds up the convergence of the standard (LBFGS) optimizer while also yielding more accurate machine learning models. Moreover, the devised preconditioner is designed to take advantage of model-parallel computations, which can further reduce the training time.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Optimizing the optimizer for physics-informed neural networks and Kolmogorov-Arnold networks

Physics-Informed Neural Networks (PINNs) have revolutionized the computation of PDE solutions by integrating partial differential equations (PDEs) into the neural network’s training process as soft constraints, becoming an important component of the scientific machine learning (SciML) ecosystem. More recently, physics-informed Kolmogorv-Arnold networks (PIKANs) have also shown to be effective and comparable in accuracy with PINNs. In their current implementation, both PINNs and PIKANs are mainly optimized using first-order methods like Adam, as well as quasi-Newton methods such as BFGS and its low-memory variant, L-BFGS. However, these optimizers often struggle with highly nonlinear and non-convex loss landscapes, leading to challenges such as slow convergence, local minima entrapment, and (non)degenerate saddle points. In this study, we investigate the performance of Self- Scaled BFGS (SSBFGS), Self-Scaled Broyden (SSBroyden) methods and other advanced quasi-Newton schemes, including BFGS and L-BFGS with different line search strategies. These methods dynamically rescale updates based on historical gradient information, thus enhancing training efficiency and accuracy. We systematically compare these optimizers – using both PINNs and PIKANs – on key challenging PDEs, including the Burgers, Allen-Cahn, Kuramoto-Sivashinsky, Ginzburg-Landau, and Stokes equations. Additionally, we evaluate the performance of SSBFGS and SSBroyden for Deep Operator Network (DeepONet) architectures, demonstrating their effectiveness for data-driven operator learning. Our findings provide state-of-the-art results with orders-of-magnitude accuracy improvements without the use of adaptive weights or any other enhancements typically employed in PINNs. More broadly, our work reveal insights into the effectiveness of quasi-Newton optimization strategies in significantly improving the convergence and accurate generalization of PINNs and PIKANs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Computationally efficient models for aqueous organic redox flow batteries

The rising usage of intermittent energy has garnered the need for large scale energy storage systems. Redox flow batteries (RFB) based energy storage system shows promising potential. Numerical simulations and machine learning approaches have been widely used to study RFB performance. The development of autonomous material discovery framework and digital twin of energy storage system usually needs to query cell performance through fast response models. In this study, two computationally efficient models are introduced: a physics-based analytical flow battery model (EZBattery), and a machine learning operator model (Deep Operator Network, denoted by DeepONet). Both models can provide cell performance near instantly, and prediction accuracy was systematically examined on an application of evaluating the performances of a 780 cm 2 aqueous organic redox flow battery (AORFB), using potential anolyte candidates in dihydroxyphenazine (DHP)-based family of organic materials. A validated computationally expansive 3-dimensional multi-physics finite element model by COMSOL was used as the ground truth and provided the training data set for the DeepONet. 1280 samples were generated with 10 properties to mimic the different possible anolyte candidates, and the cell performances were evaluated under 10 different combined operating conditions. The accuracy comparisons for the two computationally efficient models show that both models can provide comparable accuracy in predicting cell charging/discharging voltage curves. DeepONet can provide slightly higher overall accuracy than EZBattery with faster calculation speed, but highly relies on the training dataset. EZBattery does not need a training dataset and can provide interpretable physics-based explanations of the results, while being more flexible to adjust to adapt any different cell designs, flow battery architectures, and electrolyte materials.

Analytical model↗
DOE OSTI2025

Neural chaos: A spectral stochastic neural operator

Building surrogate models for operators with uncertainty quantification capabilities is essential for many engineering applications where randomness–such as variability in material properties, boundary conditions, and initial conditions–is unavoidable. Polynomial Chaos Expansion (PCE) is widely recognized as a go-to method for constructing stochastic surrogates in both intrusive and non-intrusive ways, and it has recently been used in the context of operator learning. However, its application becomes challenging for complex or high-dimensional processes, as achieving accuracy requires higher-order polynomials, which can increase computational demand and/or the risk of overfitting. Furthermore, PCE requires specialized treatments to manage random variables that are not independent, and these treatments may be problem-dependent or may fail with increasing complexity. Here, in this work, we adopt the same formalism as the spectral expansion used in PCE; however, we replace the classical polynomial basis functions with neural network (NN) basis functions to leverage their expressivity. To achieve this, we propose an algorithm that identifies NN-parameterized basis functions in a purely data-driven manner, without any prior assumptions about the joint distribution of the random variables involved, whether independent or dependent, or about their marginal distributions. The proposed algorithm identifies each NN-parameterized basis function sequentially, ensuring they are orthogonal with respect to the data distribution. The basis functions are constructed directly on the joint stochastic variables without requiring a tensor product structure or assuming independence of the random variables. This approach may offer greater flexibility for complex stochastic models, while simplifying implementation compared to the tensor product structures typically used in PCE to handle random vectors. This is particularly advantageous given the current state of open-source packages, where building and training neural networks can be done with just a few lines of code and extensive community support. We demonstrate the effectiveness of the proposed scheme through several numerical examples of varying complexity and provide comparisons with classical PCE.

Polynomial chaos expansion↗
DOE OSTI2024

Reducing Operator Complexity of Galerkin Coarse-grid Operators with Machine Learning

Here, we propose a data-driven and machine-learning-based approach to compute non-Galerkin coarse-grid operators in multigrid (MG) methods, addressing the well-known issue of increasing operator complexity. Guided by the MG theory on spectrally equivalent coarse-grid operators, we have developed novel machine learning algorithms that utilize neural networks combined with smooth test vectors from multigrid eigenvalue problems. The proposed method demonstrates promise in reducing the complexity of coarse-grid operators while maintaining overall MG convergence for solving parametric partial differential equation problems. Numerical experiments on anisotropic rotated Laplacian and linear elasticity problems are provided to showcase the performance and comparison with existing methods for computing non-Galerkin coarse-grid operators.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Machine learning models of intermittent operation of RO wellhead water treatment for salinity reduction and nitrate removal

Machine learning models were developed for intermittent multi-mode operation of a wellhead reverse osmosis water purification and desalination system to predict salt passage, nitrate passage, and permeate flux. The models, based on long short-term memory (LSTM) recurrent neural network (RNN) architecture, included an attention mechanism to increase model performance in proximity of the regulatory limit for nitrate. Training and testing of the models for the Startup, Production, Shutdown and Flushing operational modes were based on operational data (consisting of 22 process variables per data sample) acquired every 2–5 s over a six-month period. The significant sets of model input attributes for the different operational modes were assessed via Spearman ranking correlation, Self-Organizing Map (SOM) analysis and feed forward feature selection (FFFS). Although the variability of nitrate passage, salt passage and permeate flux was significant over the four operational modes, prediction performance for the three outcomes were with R2 and Average Absolute Relative Error (AARE) of 0.78–0.95 and 2.96–6.16 %, respectively. Model updates post membrane elements replacement demonstrated similar levels of prediction accuracy. The study results suggest that there is merit in exploring the utility of multi-mode models for sensor fault detection, data imputation, and for potential use in model-predictive control.

Intermittent RO operation↗
DOE OSTI2024

Metric Learning to Accelerate Convergence of Operator Splitting Methods

Recent developments in machine learning have led to promising advances in accelerating the solution of constrained optimization problems. Increasing demand for real-time decision-making capabilities in applications such as artificial intelligence and optimal control has led to a variety of proposed strategies for learning to produce fast solutions to optimization problems. For example, recent works have shown that it is possible to accelerate the convergence of optimization algorithms by learning to select their parameters, such as gradient descent stepsizes. This work proposes a new approach, in which the underlying metric spaces of proximal operator splitting algorithms are learned to maximize convergence rate. While prior works in optimization theory have derived optimal metrics in simple cases, no such result exists for many practical problem forms including general Quadratic Programming (QP). This paper shows how differentiable optimization can enable the end-to-end learning of proximal metrics, enhancing the convergence of proximal algorithms for QP problems beyond what is possible based on known theory. Additionally, the results illustrate a strong connection between the learned proximal metrics and active constraints at the optima, leading to an interpretation in which the predicted proximal metrics can be viewed as a form of active set prediction.

King, Ethan [BATTELLE (PACIFIC NW LAB)]↗
DOE OSTI2025

Optimizing Facility Operations by Applying Machine Learning to the Army Reserve Enterprise Building Control System (Final Report)

Thousands of U.S. Department of Defense (DoD) buildings have building automation systems (BASs) and/or advanced meters. Although these systems have a wealth of data, performance optimization requires time and expertise to review and act on that information. Machine learning (ML) can provide automated and actionable insights to controls operators. This demonstration implemented proven ML methods on the Army Reserve Enterprise Building Control System. ML refers to algorithms that “learn” from data and improve their performance on a given task over time. In the buildings domain these tasks range from predicting future energy consumption, to identifying operational issues before faults occur, to optimizing control decisions. To learn, ML requires input data, which – for buildings – typically consists of instrument data such as energy consumption data and subsystem controls information such as set-point temperatures, and context data consisting of information such as the physical location of the building, the area of the building, and the weather. ML models use the relationships learned from the input data to make predictions with new, previously unseen, data. The team was able to investigate and successfully implement the following ML use cases: labeling consumption data as anomalous or non-anomalous; baseline whole-building load prediction (unknown fault status); fault detection (validation not possible); and site prioritization for energy-related projects. Due to the constraints of the project, interventions were not able to be implemented during the demonstration; therefore, assessments of operational cost savings and maintenance avoided could not be performed. The project has been presented at two leading national building conferences and two additional publications to peer-reviewed journals are currently in preparation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2024

RandONets: Shallow networks with random projections for learning linear and nonlinear operators

Deep neural networks have been extensively used for the solution of both the forward and the inverse problem for dynamical systems. However, their implementation necessitates optimizing a high-dimensional space of parameters and hyperparameters. This fact, along with the requirement of substantial computational resources, pose a barrier to achieving high numerical accuracy, but also interpretability. Here, to address the above challenges, we present Random Projection-based Operator Networks (RandONets): shallow networks with random projections and tailor-made numerical analysis methods that learn accurately and fast linear and nonlinear operators. Building on previous works, we prove that RandOnets are universal approximators of linear and nonlinear operators. Due to their simplicity, RandONets provide a one-step transformation of the input space, facilitating interpretability. For the evaluation of their performance, we focus on operators of PDEs. We show, that RandONets outperform by several orders of magnitude, both in terms of numerical approximation accuracy and computational cost, the “vanilla” DeepONets. Hence, we believe that our method will trigger further developments in the field of scientific machine learning, for the development of new ‘’light”schemes that will provide high accuracy while reducing dramatically the computational cost. A MATLAB toolbox for RandONets, including demos, is available on GitHub at https://github.com/GianlucaFabiani/RandONets.

Interpretable machine learning↗
DOE OSTI2025

Deep Reinforcement Learning for Distribution System Operations: A Tutorial and Survey

Here, the rapid evolution of modern electric power distribution systems into complex networks of interconnected active devices, distributed generation (DG), and storage poses increasing difficulties for system operators. The large-scale integration of distributed energy resources (DERs) and the rapid exchange of measurement data via communication networks present major opportunities for advancing grid operations but also introduce greater uncertainty, higher data dimensionality, more complex network and device models, and challenging control and optimization problems. Deep reinforcement learning (DRL) algorithms are promising in addressing these challenges. However, they have not been effectively adapted for power systems applications, requiring extensive customization for implementation and evaluation. This has resulted in reproducibility challenges and a steep learning curve for researchers new to applying DRL algorithms to the power systems domain. To bridge these gaps, this tutorial aims to serve as a valuable resource for researchers interested in exploring learning-based algorithms to operate active power distribution networks. Specifically, this work presents a generalized process for translating sequential decision-making problems in power distribution systems into Markov decision process (MDP) formulations, illustrated through concrete grid service examples. Additionally, we introduce a simple environment design strategy to develop and evaluate example DRL algorithms for distribution system applications, complete with an included code repository to guide users through environment construction.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Reinforcement Learning to Enhance Optimal Operation of Resilient Community Energy Systems

This paper presents a novel model-free multi-agent Reinforcement Learning (RL) control method to enhance the resilience of community energy systems in island mode, which coordinates multiple objectives without the necessity of identifying system models that require expert knowledge. Specifically, a community-level coordinator agent is designed to allocate renewable energy resources among different buildings, and multiple building-level agents are developed to optimize load schedules based on limited energy resources and requirements of building loads and occupants’ comfort. In a two-day evaluation, our RL approach demonstrated a similar performance against MPC without requiring system models and formulation of optimization problems as required in MPC.

ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATION↗
DOE OSTI2024

Improving operational performance using machine learning analysis of Radiation Portal Monitor measurements

Radiation Portal Monitors (RPMs) have been installed worldwide to scan vehicles and cargo for the presence of radiological and nuclear materials. In field operations, the sensitivity of these systems is typically limited by the relatively high rates of nuisance alarms that usually must be followed up with secondary inspections. We have developed a machine-learning based alarm analysis system that has been deployed at numerous locations in the U.S. and internationally. Our Enhanced Radiological Nuclear Inspection and Evaluation (ERNIE) analysis software and its derivatives have demonstrated increased sensitivity to radiological and nuclear material of concern while reducing nuisance alarms by as much as an order of magnitude.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2026

Explainable Graph Learning for Particle Accelerator Operations

Particle accelerators are vital tools in physics, medicine, and industry, requiring precise tuning to ensure optimal beam performance. However, real-world deviations from idealized simulations make beam tuning a time-consuming and error-prone process. In this work, we propose an explanation-driven framework for providing actionable insight into beamline operations, with a focus on the injector beamline at the Continuous Electron Beam Accelerator Facility (CEBAF). We represent beamline configurations as heterogeneous graphs, where setting nodes represent elements that human operators can actively adjust during beam tuning, and reading nodes passively provide diagnostic feedback. To identify the most influential setting nodes responsible for differences between any two beamline configurations, our approach first predicts the resulting changes in reading nodes caused by variations in settings, and then learns importance scores that capture the joint influence of multiple setting nodes. Experimental results on real-world CEBAF injector data demonstrate the framework’s ability to generate interpretable insights that can assist human operators in beamline tuning and reduce operational overhead.

Wang, Song [Univ. of Virginia, Charlottesville, VA↗