Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “physical learning, networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

A dimensionality reduction algorithm for mapping tokamak operational regimes using a variational autoencoder (VAE) neural network

Abstract A variational autoencoder (VAE) is a type of unsupervised neural network which is able to learn meaningful data representations in a reduced dimensional space. We present an application of VAE in identifying the operational stability boundary of tokamak plasma discharges. This model was implemented using a dataset of over 3000 discharges from the high beta tokamak-extended pulse (HBT-EP) device. We found the VAE model to be capable of forming a continuous low-dimensional operational space map and identifying the operational boundaries using a specified warning time window. By projecting the operational parameters onto the same reduced space, this provides an intuitive way for the machine operator or an automated control system to perform disruption avoidance using a relevant control actuator as a discharge approaches a boundary. Pre-programmed GPU control experiments were conducted to demonstrate this control technique using HBT-EP’s saddle control coils as a horizontal position actuator, showing the ability to avoid the oncoming disruptive event and extend the duration of the discharge.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

ExaTN: Scalable GPU-Accelerated High-Performance Processing of General Tensor Networks at Exascale

We present ExaTN (Exascale Tensor Networks), a scalable GPU-accelerated C++ library which can express and process tensor networks on shared- as well as distributed-memory high-performance computing platforms, including those equipped with GPU accelerators. Specifically, ExaTN provides the ability to build, transform, and numerically evaluate tensor networks with arbitrary graph structures and complexity. It also provides algorithmic primitives for the optimization of tensor factors inside a given tensor network in order to find an extremum of a chosen tensor network functional, which is one of the key numerical procedures in quantum many-body theory and quantum-inspired machine learning. Numerical primitives exposed by ExaTN provide the foundation for composing rather complex tensor network algorithms. We enumerate multiple application domains which can benefit from the capabilities of our library, including condensed matter physics, quantum chemistry, quantum circuit simulations, as well as quantum and classical machine learning, for some of which we provide preliminary demonstrations and performance benchmarks just to emphasize a broad utility of our library.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Lightweight and effective tensor sensitivity for atomistic neural networks

Atomistic machine learning focuses on the creation of models that obey fundamental symmetries of atomistic configurations, such as permutation, translation, and rotation invariances. In many of these schemes, translation and rotation invariance are achieved by building on scalar invariants, e.g., distances between atom pairs. There is growing interest in molecular representations that work internally with higher rank rotational tensors, e.g., vector displacements between atoms, and tensor products thereof. Here, we present a framework for extending the Hierarchically Interacting Particle Neural Network (HIP-NN) with Tensor Sensitivity information (HIP-NN-TS) from each local atomic environment. Crucially, the method employs a weight tying strategy that allows direct incorporation of many-body information while adding very few model parameters. We show that HIP-NN-TS is more accurate than HIP-NN, with negligible increase in parameter count, for several datasets and network sizes. As the dataset becomes more complex, tensor sensitivities provide greater improvements to model accuracy. In particular, HIP-NN-TS achieves a record mean absolute error of 0.927 $\frac{\text{kcal}}{\text{mol}}$ for conformational energy variation on the challenging COMP6 benchmark, which includes a broad set of organic molecules. We also compare the computational performance of HIP-NN-TS to HIP-NN and other models in the literature.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

SPIDARman: System-Level Physics-Informed Detection of Anomalies in Reactor Collected Data Considering Human Errors

In nuclear power plants (NPPs), anomalies arising from sensors or human errors (HEs) can undermine the performance and reliability of plant operations. Anomaly detection models can be employed to detect sensor errors and HEs. Additionally, physics-informed machine learning models can utilize the known physics of the system, as described by mathematical equations, to ensure that sensor values are consistent with physical laws. Hence, we propose SPIDARman: System-level Physics-Informed Detection of Anomalies in Reactor Collected Data Considering Human Errors, a holistic physics-informed anomaly detection approach based on generative adversarial networks (GANs) to detect anomalies in both automatically collected sensor data and manually collected surveillance data. Here we test our approach on data collected from a flow loop testbed, showcasing its potential to detect anomalies. Results demonstrate that the proposed model performs better than the baseline GAN-based models in detecting sensor and surveillance anomalies, suggesting the potential of physics-informed anomaly detection GAN models in NPPs.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Exciting DeePMD: Learning excited-state energies, forces, and non-adiabatic couplings

We extend the DeePMD neural network architecture to predict electronic structure properties necessary to perform non-adiabatic dynamics simulations. While learning the excited state energies and forces follows a straightforward extension of the DeePMD approach for ground-state energies and forces, how to learn the map between the non-adiabatic coupling vectors (NACV) and the local chemical environment descriptors of DeePMD is less trivial. Most implementations of machine-learning-based non-adiabatic dynamics inherently approximate the NACVs, with an underlying assumption that the energy-difference-scaled NACVs are conservative fields. We overcome this approximation, implementing the method recently introduced by Richardson [J. Chem. Phys. 158, 011102 (2023)], which learns the symmetric dyad of the energy-difference-scaled NACV. Furthermore, the efficiency and accuracy of our neural network architecture are demonstrated through the example of the methaniminium cation CH 2 NH 2 + .

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Hybrid Quantum-Classical Neural Network for Calculating Ground State Energies of Molecules

We present a hybrid quantum-classical neural network that can be trained to perform electronic structure calculation and generate potential energy curves of simple molecules. The method is based on the combination of parameterized quantum circuits and measurements. With unsupervised training, the neural network can generate electronic potential energy curves based on training at certain bond lengths. To demonstrate the power of the proposed new method, we present the results of using the quantum-classical hybrid neural network to calculate ground state potential energy curves of simple molecules such as H 2 , LiH, and BeH 2 . The results are very accurate and the approach could potentially be used to generate complex molecular potential energy surfaces.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2026

Survey of Deep Learning and Physics-Based Approaches in Computational Wave Imaging

Computational wave imaging (CWI) extracts hidden structure and physical properties of a volume of material by analyzing wave signals that traverse that volume. Applications include seismic exploration of the Earth’s subsurface, acoustic imaging and nondestructive testing (NDT) in material science, and ultrasound computed tomography (USCT) in medicine. Current approaches for solving CWI problems can be divided into two categories: those rooted in traditional physics and those based on deep learning. Physics-based methods stand out for their ability to provide high-resolution and quantitatively accurate estimates of acoustic properties within the medium. However, they can be computationally intensive and are susceptible to ill-posedness and nonconvexity typical of CWI problems. Machine learning (ML)-based computational methods have recently emerged, offering a different perspective to address these challenges. Diverse scientific communities have independently pursued the integration of deep learning in CWI. This review discusses how contemporary scientific ML techniques, and deep neural networks in particular, have been developed to enhance and integrate with traditional physics-based methods for solving CWI problems. We present a structured framework that consolidates existing research spanning multiple domains, including computational imaging, wave physics, and data science. This study concludes with important lessons learned from existing ML-based methods and identifies technical hurdles and emerging trends through a systematic analysis of the extensive literature on this topic.

42 ENGINEERING↗
DOE OSTI2020

PANNA: Properties from Artificial Neural Network Architectures

We report prediction of material properties from first principles is often a computationally expensive task. Recently, artificial neural networks and other machine learning approaches have been successfully employed to obtain accurate models at a low computational cost by leveraging existing example data. Here, we present a software package “Properties from Artificial Neural Network Architectures” (PANNA) that provides a comprehensive toolkit for creating neural network models for atomistic systems following the Behler–Parrinello topology. Besides the core routines for neural network training, it includes data parser, descriptor builder for Behler–Parrinello class of symmetry functions and force-field generator suitable for integration within molecular dynamics packages. PANNA offers a variety of activation and cost functions, regularization methods, as well as the possibility of using fully-connected networks with custom size for each atomic species. PANNA benefits from the optimization and hardware-flexibility of the underlying TensorFlow engine which allows it to be used on multiple CPU/GPU/TPU systems, making it possible to develop and optimize neural network models based on large datasets.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Transfer Learning of High-Fidelity Opacity Spectra in Autoencoders and Surrogate Models

Simulations of high energy density physics are expensive, largely in part for the need to produce nonlocal thermodynamic equilibrium opacities. High-fidelity spectra may reveal new physics in the simulations not seen with low-fidelity spectra, but the cost of these simulations also scales with the level of fidelity of the opacities being used. Neural networks are capable of reproducing these spectra, but neural networks need data to train them, which limits the level of fidelity of the training data. Here this article demonstrates that it is possible to reproduce high-fidelity spectra with median errors in the realm of 3%–4% using as few as 50 samples of high-fidelity Krypton data by performing transfer learning on a neural network trained on many times more low-fidelity data.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Encrypted Control Using Modified Learning With Errors-based Schemes

Cyber-physical systems (CPSs) require reliable, safe, and secure control of critical infrastructure, combining computational and networking capabilities, which heighten the risk of cyber attacks. These attacks can disrupt the physical process, causing unforeseen consequences. One solution is the use of fully homomorphic encryption (FHE) to protect the control loop, allowing for secure computations and communications without compromising signal and control system privacy. The challenge with FHE, however, is its requirement for inputs to be integers. This paper introduces a modified Learning With Errors (LWE) FHE approach that encodes control system dynamics and signals into integers. Our proposed scheme leverages a generalized LWE encoding function and modifies the Gentry-Sahai-Waters (GSW) gadget decomposition tool to encrypt the control system. Using the modified LWE scheme, we formalize a fully encrypted control system, supported by simulated results.

42 - ENGINEERING↗
DOE OSTI2024

Solving the Orszag–Tang vortex magnetohydrodynamics problem with physics-constrained convolutional neural networks

We study the 2D Orszag–Tang vortex magnetohydrodynamics (MHD) problem through the use of physics-constrained convolutional neural networks (PCNNs) for forecasting the density, ρ, and the magnetic field, B, as well as the prediction of B given the velocity field v of the fluid. In addition to translation equivariance from the convolutional architecture, other physics constraints were embedded: absence of magnetic monopoles, non-negativity of ρ, use of only relevant variables, and the periodic boundary conditions of the problem. The use of only relevant variables and the hard constraint of non-negative ρ were found to facilitate learning greatly. The divergenceless condition ∇·B=0 was implemented as a hard constraint up to machine precision through the use of a magnetic potential to define B=∇×A. Residual networks and data augmentation were also used to improve performance. This allowed for some of the residual models to function as surrogate models and provide reasonably accurate simulations. For the prediction task, the PCNNs were evaluated against a physics-informed neural network, which had the ideal MHD induction equation as a soft constraint. Several models were able to generate highly accurate fields, which are visually almost indistinguishable and have low mean squared error. Only methods with built-in hard constraints produced physical fields with ∇·B=0. The use of PCNNs for MHD has the potential to produce physically consistent real-time simulations to serve as virtual diagnostics in cases where inferences must be made with limited observables.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

Statistical Treatment of Convolutional Neural Network Superresolution of Inland Surface Wind for Subgrid-Scale Variability Quantification

Abstract Machine learning models have been employed to perform either physics-free data-driven or hybrid dynamical downscaling of climate data. Most of these implementations operate over relatively small downscaling factors because of the challenge of recovering fine-scale information from coarse data. This limits their compatibility with many global climate model outputs, often available between ∼50- and 100-km resolution, to scales of interest such as cloud resolving or urban scales. This study systematically examines the capability of a type of superresolving convolutional neural network (SR-CNNs) to downscale surface wind speed data over land from different coarse resolutions (25-, 48-, and 100-km resolution) to 3 km. For each downscaling factor, we consider three convolutional neural network (CNN) configurations that generate superresolved predictions of fine-scale wind speed, which take between one and three input fields: coarse wind speed, fine-scale topography, and diurnal cycle. In addition to fine-scale wind speeds, probability density function parameters are generated through which sample wind speeds can be generated, accounting for the intrinsic stochasticity of wind speed. For assessing generalization to new data, CNN models are tested on regions with different topography and climate that are unseen during training. The evaluation of superresolved predictions focuses on subgrid-scale variability and the recovery of extremes. Models with coarse wind and fine topography as inputs exhibit the best performance when compared with other model configurations, operating across the same downscaling factor. Our diurnal cycle encoding results in lower out-of-sample generalizability when compared with other input configurations.

17 WIND ENERGY↗
DOE OSTI2020

Machine learning for nonintrusive model order reduction of the parametric inviscid transonic flow past an airfoil

Fluid flow in the transonic regime finds relevance in aerospace engineering, particularly in the design of commercial air transportation vehicles. Computational fluid dynamics models of transonic flow for aerospace applications are computationally expensive to solve because of the high degrees of freedom as well as the coupled nature of the conservation laws. While these issues pose a bottleneck for the use of such models in aerospace design, computational costs can be significantly minimized by constructing special, structure-preserving surrogate models called reduced-order models. In this work, we propose a machine learning method to construct reduced-order models via deep neural networks and we demonstrate its ability to preserve accuracy with a significantly lower computational cost. In addition, our machine learning methodology is physics-informed and constrained through the utilization of an interpretable encoding by way of proper orthogonal decomposition. Application to the inviscid transonic flow past the RAE2822 airfoil under varying freestream Mach numbers and angles of attack, as well as airfoil shape parameters with a deforming mesh, shows that the proposed approach adapts to high-dimensional parameter variation well. Notably, the proposed framework precludes the knowledge of numerical operators utilized in the data generation phase, thereby demonstrating its potential utility in the fast exploration of design space for diverse engineering applications. Comparison against a projection-based nonintrusive model order reduction method demonstrates that the proposed approach produces comparable accuracy and yet is orders of magnitude computationally cheap to evaluate, despite being agnostic to the physics of the problem.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

A data analytical approach for assessing the efficacy of Operational Technology active defenses against insider threats

In recent years, the need for Operational Technology (OT) defenses has been recognized, serving as an additional line of defense when Information Technology (IT) defenses are bypassed. This is no longer considered an uncommon possibility when dealing with advanced persistent threat (APT) actors expected to be state-sponsored and receiving insider assistance. In these extreme adversarial situations, OT defenses aim to provide another layer of defense for the system, introduced directly at the physical process level, as described by the sensors data, the system model, and control actions. Just like IT defenses, two schools of thought, i.e., passive and active defenses, have emerged to address this challenge. In active defenses, representing the focus of this paper, known signatures, synthesized based on the system's unique characteristics, are inserted into the system. In contradistinction, passive methods rely solely on observing system behavior in search of patterns of normal behavior with deviations thereof representing abnormal behavior. In their most sophisticated implementations, both passive and active defenses rely on the use of data analytics to identify the patterns and synthesize the observed and/or inserted signatures. Past research has shown that passive defenses may be bypassed by APT actors relying on data analytics and their intimate knowledge of the system to evade detection by respecting the patterns identified by the defenders. Here this manuscript explores the use of active defenses under the assumption that the attacker has privileged access to the system, including access to the system's model and sensors data. Specifically, this manuscript assesses the ability of active defenses to remain invisible to the attackers, and discusses the associated challenges that must be addressed to ensure their resiliency against APT actors.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2022

Autoregressive Neural Network for Simulating Open Quantum Systems via a Probabilistic Formulation

The theory of open quantum systems lays the foundation for a substantial part of modern research in quantum science and engineering. Rooted in the dimensionality of their extended Hilbert spaces, the high computational complexity of simulating open quantum systems calls for the development of strategies to approximate their dynamics. In this Letter, we present an approach for tackling open quantum system dynamics. Using an exact probabilistic formulation of quantum physics based on positive operator-valued measure, we compactly represent quantum states with autoregressive neural networks; such networks bring significant algorithmic flexibility due to efficient exact sampling and tractable density. We further introduce the concept of string states to partially restore the symmetry of the autoregressive neural network and improve the description of local correlations. Efficient algorithms have been developed to simulate the dynamics of the Liouvillian superoperator using a forward-backward trapezoid method and find the steady state via a variational formulation. Our approach is benchmarked on prototypical one-dimensional and two-dimensional systems, finding results which closely track the exact solution and achieve higher accuracy than alternative approaches based on using Markov chain Monte Carlo method to sample restricted Boltzmann machines. Our Letter provides general methods for understanding quantum dynamics in various contexts, as well as techniques for solving high-dimensional probabilistic differential equations in classical setups.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Addressing GPU memory limitations for Graph Neural Networks in High-Energy Physics applications

Introduction Reconstructing low-level particle tracks in neutrino physics can address some of the most fundamental questions about the universe. However, processing petabytes of raw data using deep learning techniques poses a challenging problem in the field of High Energy Physics (HEP). In the Exa.TrkX Project, an illustrative HEP application, preprocessed simulation data is fed into a state-of-art Graph Neural Network (GNN) model, accelerated by GPUs. However, limited GPU memory often leads to Out-of-Memory (OOM) exceptions during training, due to the large size of models and datasets. This problem is exacerbated when deploying models on High-Performance Computing (HPC) systems designed for large-scale applications. Methods We observe a high workload imbalance issue during GNN model training caused by the irregular sizes of input graph samples in HEP datasets, contributing to OOM exceptions. We aim to scale GNNs on HPC systems, by prioritizing workload balance in graph inputs while maintaining model accuracy. Our paper introduces diverse balancing strategies aimed at decreasing the maximum GPU memory footprint and avoiding the OOM exception, across various datasets. Results Our experiments showcase memory reduction of up to 32.14% compared to the baseline. We also demonstrate the proposed strategies can avoid OOM in application. Additionally, we create a distributed multi-GPU implementation using these samplers to demonstrate the scalability of these techniques on the HEP dataset. Discussion By assessing the performance of these strategies as data loading samplers across multiple datasets, we can gauge their effectiveness in both single-GPU and distributed environments. Our experiments, conducted on datasets of varying sizes and across multiple GPUs, broaden the applicability of our work to various GNN applications that handle input datasets with irregular graph sizes.

Lee, Claire Songhyun↗