Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “neural operator learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12

Machine Learning Solutions for a Stable Grid Recovery

Grid operating security studies are typically employed to establish operating boundaries, ensuring secure and stable operation for a range of operation under NERC guidelines. However, if these boundaries are severely violated, existing system security margins will be largely unknown, as would be a secure incremental dispatch path to higher security margins while continuing to serve load. As an alternative to the use of complex optimizations over dynamic conditions, this work employs the use of machine learning to identify a sequence of secure state transitions which place the grid in a higher degree of operating security with greater static and dynamic stability margins. Several reinforcement learning solution methods were developed using deep learning neural networks, including Deep Q-learning, Mu-Zero, and the continuous algorithms Proximal Reinforcement Learning, and Advantage Actor Critic Learning. The work is demonstrated on a power grid with three control dimensions but can be scaled in size and dimensionality, which is the subject of ongoing research.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Data-driven estimation of energy consumption for electric bus under real-world driving conditions

Reliable and accurate estimation of an electric bus’s instantaneous energy consumption is critical in evaluating energy impacts of planning and control of electric bus operations. In this study, we developed machine learning-based long short-term memory (LSTM) and artificial neural network (ANN) models to estimate 1 Hz energy consumption of electric buses based on continuous monitoring data of electric buses in Chattanooga, Tennessee, in 2019 and 2020. We propose a data-partitioning algorithm to separate energy charging and discharging modes before applying data-driven estimation models. Here, a K-fold cross-validation-based model selection process was conducted to identify the optimal model structure and input variables in terms of prediction accuracy. The estimation results show the predicted mean absolute percentage error rates of LSTM and ANN models were 3% and 5%, respectively. We compared the proposed models with existing models in the literature based on the same testing data to demonstrate the predictability of our models.

Artificial neural network↗

Physics-constrained deep neural network method for estimating parameters in a redox flow battery

Here, in this paper, we present a physics-constrained deep neural network (PCDNN) method for parameter estimation in the zero-dimensional (0D) model of the vanadium redox flow battery (VRFB). In this approach, we use deep neural networks to approximate the model parameters as functions of the operating conditions. This method allows the integration of VRFB computational models as the physical constraints in the parameter learning process, leading to enhanced accuracy of parameter estimation and cell voltage prediction. Using an experimental dataset, we demonstrate that the PCDNN method can estimate model parameters for a range of operating conditions and improve the 0D model prediction of voltage compared to the 0D model prediction with constant operation-condition-independent parameters estimated with traditional inverse methods. We also demonstrate that the PCDNN approach has an improved generalization ability for estimating parameter values for operating conditions not used in the training process.

25 ENERGY STORAGE↗

Spin-orbit torque induced spike-timing dependent plasticity

Nanoelectronic devices that mimic the functionality of synapses are a crucial requirement for performing cortical simulations of the brain. In this work, we propose a ferromagnet-heavy metal heterostructure that employs spin-orbit torque to implement spike-timing dependent plasticity. The proposed device offers the advantage of decoupled spike transmission and programming current paths, thereby leading to reliable operation during online learning. Possible arrangement of such devices in a crosspoint architecture can pave the way for ultra-dense neural networks. Simulation studies indicate that the device has the potential of achieving pico-Joule level energy consumption (maximum 2 pJ per synaptic event) which is comparable to the energy consumption for synaptic events in biological synapses.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Clifford Neural Operators on Atmospheric Data Influenced Partial Differential Equations

Mathematical representations of the atmosphere are key to forecasting and research tasks across Earth science. Numerically solving the underlying partial differential equations(PDEs) of the atmosphere, however, can be difficult and computationally expensive with numerous trade-offs between computing efficiency and accuracy. Utilizing neural net-works to learn approximations of the PDE solutions from the data can help us model complex phenomena more efficiently than traditional numerical schemes. Here, we have applied Clifford algebra-based neural operators for predicting atmospheric variables. Clifford Fourier neural operators are used with two different backbone architectures, ResNet and UNet, on custom data of U10, V10 and surface pressure as well as U500, V500 and Z500. Clifford Fourier neural operators, coupled with ResNet and UNet architectures, are applied to a key reanalysis dataset. Model performance is initially strong, but we observe increasing errors, resulting in the model becoming highly unstable.

Sujit Roy↗

Ultrafast Bragg coherent diffraction imaging of epitaxial thin films using deep complex-valued neural networks

Abstract Domain wall structures form spontaneously due to epitaxial misfit during thin film growth. Imaging the dynamics of domains and domain walls at ultrafast timescales can provide fundamental clues to features that impact electrical transport in electronic devices. Recently, deep learning based methods showed promising phase retrieval (PR) performance, allowing intensity-only measurements to be transformed into snapshot real space images. While the Fourier imaging model involves complex-valued quantities, most existing deep learning based methods solve the PR problem with real-valued based models, where the connection between amplitude and phase is ignored. To this end, we involve complex numbers operation in the neural network to preserve the amplitude and phase connection. Therefore, we employ the complex-valued neural network for solving the PR problem and evaluate it on Bragg coherent diffraction data streams collected from an epitaxial La 2-x Sr x CuO 4 (LSCO) thin film using an X-ray Free Electron Laser (XFEL). Our proposed complex-valued neural network based approach outperforms the traditional real-valued neural network methods in both supervised and unsupervised learning manner. Phase domains are also observed from the LSCO thin film at an ultrafast timescale using the complex-valued neural network.

77 NANOSCIENCE AND NANOTECHNOLOGY↗

PowerModelsGAT-AI: Physics-Informed Graph Attention for Multi-System Power Flow With Continual Learning

Solving the alternating current power flow equations in real time is essential for secure grid operation, yet classical Newton–Raphson solvers can be slow under stressed conditions. Existing graph neural networks for power flow are typically trained on a single system and often degrade on different systems. We present PowerModelsGAT-AI, a physics-informed graph attention network that predicts bus voltages and generator injections. The model uses bus-type-aware masking to handle different bus types and balances multiple loss terms, including a power-mismatch penalty, using learned weights. We evaluate the model on 14 benchmark systems (4 to 6,470 buses) and train a unified model on 13 of these under contingency conditions with up to two branch outages, achieving an average normalized mean absolute error of 0.89% for voltage magnitudes and R 2 >0.99 for voltage angles. We also show continual learning: when adapting a base model to a new 1,354-bus system, standard fine-tuning causes severe forgetting with error increases exceeding 1000% on base systems, while our experience replay and elastic weight consolidation strategy keeps error increases below 2% and in some cases improves base-system performance. Interpretability analysis shows that learned attention weights correlate with physical branch parameters (susceptance: r=0.38 ; thermal limits: r=0.22 ), and feature importance analysis supports that the model captures established power flow relationships.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Power System Event Classification and Localization Using a Convolutional Neural Network

Detection and timely identification of power system disturbances are essential for situation awareness and reliable electricity grid operation. Because records of actual events in the system are limited, ensemble simulation-based events are needed to provide adequate data for building event-detection models through deep learning; e.g., a convolutional neural network (CNN). An ensemble numerical simulation-based training data set have been generated through dynamic simulations performed on the Polish system with various types of faults in different locations. Such data augmentation is proven to be able to provide adequate data for deep learning. The synchronous generators’ frequency signals are used and encoded into images for developing and evaluating CNN models for classification of fault types and locations. With a time-domain stacked image set as the benchmark, two different time-series encoding approaches, i.e., wavelet decomposition-based frequency-domain stacking and polar coordinate system-based Gramian Angular Field (GAF) stacking, are also adopted to evaluate and compare the CNN model performance and applicability. The various encoding approaches are suitable for different fault types and spatial zonation. With optimized settings of the developed CNN models, the classification and localization accuracies can go beyond 84 and 91%, respectively.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Learning plasma dynamics and robust rampdown trajectories with predict-first experiments at TCV

The rampdown phase of a tokamak pulse is difficult to simulate and often exacerbates multiple plasma instabilities. To reduce the risk of disrupting operations, we leverage advances in Scientific Machine Learning (SciML) to combine physics with data-driven models, developing a neural state-space model (NSSM) that predicts plasma dynamics during Tokamak à Configuration Variable (TCV) rampdowns. The NSSM efficiently learns dynamics from a modest dataset of 311 pulses with only five pulses in a reactor-relevant high-performance regime. The NSSM is parallelized across uncertainties, and reinforcement learning (RL) is applied to design trajectories that avoid instability limits. High-performance experiments at TCV show statistically significant improvements in relevant metrics. A predict-first experiment, increasing plasma current by 20% from baseline, demonstrates the NSSM’s ability to make small extrapolations. The developed approach paves the way for designing tokamak controls with robustness to considerable uncertainty and demonstrates the relevance of SciML for fusion experiments.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Deep Learning in Label-free Cell Classification

Label-free cell analysis is essential to personalized genomics, cancer diagnostics, and drug development as it avoids adverse effects of staining reagents on cellular viability and cell signaling. However, currently available label-free cell assays mostly rely only on a single feature and lack sufficient differentiation. Also, the sample size analyzed by these assays is limited due to their low throughput. Here, we integrate feature extraction and deep learning with high-throughput quantitative imaging enabled by photonic time stretch, achieving record high accuracy in label-free cell classification. Our system captures quantitative optical phase and intensity images and extracts multiple biophysical features of individual cells. These biophysical measurements form a hyperdimensional feature space in which supervised learning is performed for cell classification. We compare various learning algorithms including artificial neural network, support vector machine, logistic regression, and a novel deep learning pipeline, which adopts global optimization of receiver operating characteristics. As a validation of the enhanced sensitivity and specificity of our system, we show classification of white blood T-cells against colon cancer cells, as well as lipid accumulating algal strains for biofuel production. In conclusion, this system opens up a new path to data-driven phenotypic diagnosis and better understanding of the heterogeneous gene expressions in cells.

60 APPLIED LIFE SCIENCES↗

Efficient and generalizable nested Fourier-DeepONet for three-dimensional geological carbon sequestration

Geological carbon sequestration (GCS) involves injecting CO2 into subsurface geological formationsfor permanent storage. Numerical simulations could guide decisions in GCS projects by predictingCO 2 migration pathways and the pressure distribution in storage formation. However, these simula-tions are often computationally expensive due to highly coupled physics and large spatial-temporalsimulation domains. Surrogate modelling with data-driven machine learning has become a promis-ing alternative to accelerate physics-based simulations. Among these, the Fourier neural operator(FNO) has been applied to three-dimensional synthetic subsurface models. Despite its good accuracyin simulating CO 2 plume migration, it requires large computational resources in training and alsolacks generalizability. Here, to further improve performance, we have developed a nested Fourier-DeepONet by combining the expressiveness of the FNO with the modularity of a deep operatornetwork (DeepONet). This new framework is twice as efficient as a nested FNO for training and has atleast 80% lower GPU memory requirement due to its flexibility to treat temporal coordinates sepa-rately. These performance improvements are achieved without compromising prediction accuracy.In addition, the generalization and extrapolation ability of nested Fourier-DeepONet beyond thetraining range has been thoroughly evaluated. Nested Fourier-DeepONet outperformed the nestedFNO for extrapolation in time with more than 50% reduced error. It also exhibited good extrapolationaccuracy beyond the training range in terms of reservoir properties, number of wells, and injectionrate.

Lee, Jonathan E. [Department of Chemical and Envir↗

Dynamic Parameter Estimation with Physics-based Neural Ordinary Differential Equations

Accurate estimation of dynamic parameters of gen-erators is crucial to building a reliable model for dynamical studies and reliable operation of the power system. This paper develops a physics-based neural ordinary differential equations (ODE) approach to learn the parameters of generator dynamic model using phasor measurement units (PMU) data. We design a physics-based neural network to represent the swing equations of the power system dynamics. A loss function is defined as the difference between dynamic simulation results from the physics-based neural networks and pseudo PMU measurements. The parameters of generator dynamic model are iteratively updated using the neural ODEs and the adjoint method. By exploiting the mini-batch scheme in neural ODE training, the parameter estimation performance is significantly improved. Numerical study results on a 3-machine 9-bus system show that the proposed algorithm outperforms state-of-the-art baseline method in both computation time and dynamic parameter estimation accuracy.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Confronting Domain Shift in Trained Neural Networks

SAND2021-15138 O This code accompanies a paper published in the Proceedings of Machine Learning Research (PMLR), “Confronting Domain Shift in Trained Neural Networks.” Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Martinez, Carianne↗

Adaptive Data-Driven Model Predictive Control for Heat Pipe Microreactors

To establish a technical basis for self-regulating microreactors, a model predictive control (MPC) system is investigated to proactively respond to anomalies and disturbances in anticipation of potential deviations from operating setpoints. Due to the difficulty of developing a physics-based surrogate model that can accurately match plant data in various operating conditions, machine learning algorithms are used in MPC, which allow for learning from both simulation and operation data, thus efficiently describing the targeted transient with arbitrary accuracy. However, one of the biggest concerns in applying ML algorithms like artificial neural networks (ANNs) is that the predictive capabilities of ANN are limited by training data. If there are gaps between the training and target domain, the accuracy of an ANN can degrade significantly when it is used to predict unseen data. To improve the predictive capability of ANN and enable a confident use of data-driven MPCs outside the training data, this study proposes an adaptive data-driven MPC framework. The system will monitor the discrepancy between plant responses and surrogate predictions, fine-tune the ANN-based surrogate when a large discrepancy is detected, and continue MPC operation with updated surrogates. The framework is demonstrated on a point kinetic model for microreactors. The hyperparameters of the update strategy, including layers to update, error thresholds, learning rate discount, and number of data points used for fine-tuning, are optimized so the simulated microreactor is able to follow changes in setpoint with the smallest of deviations.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Adaptable Data Driven Model Predictive Control for Heat Pipe Microreactors

To establish a technical basis for self-regulating microreactors, a model predictive control (MPC) system is investigated to proactively respond to anomalies and disturbances in anticipation of potential deviations from operating setpoints. Due to the difficulty of developing a physics-based surrogate model that can accurately match plant data in various operating conditions, machine learning algorithms are used in MPC, which allow for learning from both simulation and operation data, thus efficiently describing the targeted transient with arbitrary accuracy. However, one of the biggest concerns in applying ML algorithms like artificial neural networks (ANNs) is that the predictive capabilities of ANN are limited by training data. If there are gaps between the training and target domain, the accuracy of an ANN can degrade significantly when it is used to predict unseen data. To improve the predictive capability of ANN and enable a confident use of data-driven MPCs outside the training data, this study proposes an adaptive data-driven MPC framework. The system will monitor the discrepancy between plant responses and surrogate predictions, fine-tune the ANN-based surrogate when a large discrepancy is detected, and continue MPC operation with updated surrogates. The framework is demonstrated on a point kinetic model for microreactors. The hyperparameters of the update strategy, including layers to update, error thresholds, learning rate discount, and number of data points used for fine-tuning, are optimized so the simulated microreactor is able to follow changes in setpoint with the smallest of deviations.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Use of Convolutional Neural Network Image Classification and High-Speed Ion Probe Data Toward Real-Time Detonation Characterization in a Water-Cooled Rotating Detonation Engine

As rotating detonation engines (RDEs) progress in maturity, the importance of monitoring advancements toward development of active control becomes more critical. Experimental RDE data processing at time scales which satisfy real-time diagnostics will likely require the use of machine learning. This study aims to develop and deploy a novel real-time monitoring technique capable of determining detonation wave number, direction, frequency, and individual wave speeds throughout experimental RDE operational windows. To do so, the diagnostic integrates image classification by a convolutional neural network (CNN) and ionization current signal analysis. Wave mode identification through single-image CNN classification bypasses the need to evaluate sequential images and offers instantaneous identification of the wave mode present in the RDE annulus. Here, real-time processing speeds are achieved due to low data volumes required by the methodology, namely one short-exposure image and a short window of sensor data to generate each diagnostic output. The diagnostic acquires live data using a modified experimental setup alongside Pylon and PyDAQmx libraries within a python data acquisition environment. Lab-deployed diagnostic results are presented across varying wave modes, operating conditions, and data quality, currently executed at 3–4 Hz with a variety of iteration speed optimization options to be considered as future work. These speeds exceed that of conventional techniques and offer a proven structure for real-time RDE monitoring. The demonstrated ability to analyze detonation wave presence and behavior during RDE operation will certainly play a vital role in the development of RDE active control, necessary for RDE technology maturation toward industrial integration.

42 ENGINEERING↗

A physics-informed operator regression framework for extracting data-driven continuum models

The application of deep learning toward discovery of data-driven models requires careful application of inductive biases to obtain a description of physics which is both accurate and robust. We present here a framework for discovering continuum models from high fidelity molecular simulation data. Our approach applies a neural network parameterization of governing physics in modal space, allowing a characterization of differential operators while providing structure which may be used to impose biases related to symmetry, isotropy, and conservation form. Here, we demonstrate the effectiveness of our framework for a variety of physics, including local and nonlocal diffusion processes and single and multiphase flows. For the flow physics we demonstrate this approach leads to a learned operator that generalizes to system characteristics not included in the training sets, such as variable particle sizes, densities, and concentration.

42 ENGINEERING↗

Cislunar Trajectory Design and Maneuver Autonomy for NASA's Moon to Mars Architecture

NASA’s Moon to Mars architecture is an ambitious roadmap of manned cislunar and deep space exploration. The extensive amount of orbital assets required will place a significant burden on ground-based resources, such as communication networks and operations facilities. Spacecraft autonomy is essential for maintaining a vast number of complex missions beyond Earth orbit. To achieve full autonomy, spacecraft must be able to employ methods of robust maneuver design without an explicit dependence on commands sent from the ground. This level of autonomy is needed not only for stationkeeping, but also for outbound transfers. To address the need of spacecraft maneuver design autonomy, this work investigates the use of neural networks (NNs) in a supervised learning environment. A supervised learning approach for NNs allows for a curated training data set, consisting exclusively of perturbations applied to a desired mission concept of operations (ConOps). The proposed approach allows humans on the ground to design a specific mission ConOps before flight, then employ NNs to fly the mission robustly and autonomously. This investigation numerically tests maneuver autonomy in four highly sensitive regions of flight: orbit raising, translunar injection burns, powered lunar flybys, and invariant manifold insertion burns. These straining cases are contextualized by testing them in a demonstration mission, targeting an Earth-Moon L3 orbit. The study first establishes feasibility by automating impulsive burn maneuvers. However, some guidance algorithms will need more intensive commands, such as inertial pointing and angular rates. To validate this method, NN maneuver autonomy is applied to a finite burn model of the demonstration mission. The use of sequential, mission specific maneuvers provide an appropriate testbed to demonstrate the robustness of a NN trained on feasible perturbed states. Moreover, these scenarios provide preliminary proof-of-concept for fully autonomous missions that execute maneuvers without dependence upon explicit command uplinks. As a result, the technological advancement proposed in this work may significantly ease the strain on ground-based mission operations. This would enable complex and autonomous mission execution in cislunar and deep space regimes, filling a technology gap required to support future manned missions.

NASA↗