Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “NEURAL NETWORKS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 991 records · Page 55
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Enhancing Neural Network Explainability with Variational Autoencoders

Machine intelligence has been used to tackle increasingly complex problems and deep learning solutions are at the forefront of tackling these problems. In general, these architectures have a great number of parameters that are methodically updated in training. The vast number and complexity of deep neural networks makes it very difficult to decipher the inner workings of the neurons and layers that make up the network. This paper posits that trustworthiness and trust in autonomous systems are increased through eXplainable Artificial Intelligence (XAI) and presents a method that enhances the explainability and understanding of a neural network decision. We leverage variational autoencoders to produce human interpretable features from complex data sets. We show that the explainable features can then be used for machine learning applications. Explainability inspires trust in autonomous systems that use deep learning, which is necessary for safety critical systems.

Loc Tran↗
DOE OSTI2022

Results From Invoking Artificial Neural Networks to Measure Insider Threat Detection & Mitigation

Advances on differentiating between malicious intent and natural “organizational evolution” to explain observed anomalies in operational workplace patterns suggest benefit from evaluating collective behaviors observed in the facilities to improve insider threat detection and mitigation (ITDM). Advances in artificial neural networks (ANN) provide more robust pathways for capturing, analyzing, and collating disparate data signals into quantitative descriptions of operational workplace patterns. In response, a joint study by Sandia National Laboratories and the University of Texas at Austin explored the effectiveness of commercial artificial neural network (ANN) software to improve ITDM. Overall, this research demonstrates the benefit of learning patterns of organizational behaviors, detecting off-normal (or anomalous) deviations from these patterns, and alerting when certain types, frequencies, or quantities of deviations emerge for improving ITDM. Evaluating nearly 33,000 access control data points and over 1,600 intrusion sensor data points collected over a nearly twelve-month period, this study's results demonstrated the ANN could recognize operational patterns at the Nuclear Engineering Teaching Laboratory (NETL) and detect off-normal behaviors—suggesting that ANNs can be used to support a data-analytic approach to ITDM. Several representative experiments were conducted to further evaluate these conclusions, with the resultant insights supporting collective behavior-based analytical approaches to quantitatively describe insider threat detection and mitigation.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2026

A unified neural-network framework for nucleon imaging from numerical simulations of QCD

Parton distributions encode the momentum-space structure and, in their generalizations, the spatial tomography of quarks and gluons inside hadrons, the building blocks of visible matter. We present a unified neural-network approach that learns these distributions directly from matrix elements calculated via numerical simulations of quantum chromodynamics (QCD) on the lattice by fitting two complementary inputs simultaneously: data matched to physical quantities via known momentum-space and coordinate-space formalisms. Utilizing data from both methods stabilizes the extraction and mitigates biases that can arise when either is used alone. We validate the method on controlled mock data and apply it to lattice-QCD matrix elements to extract parton distribution functions (PDFs). We show benefits of such an approach for determining the physical quantities. We further extend the framework to zero-skewness generalized parton distributions and demonstrate nucleon tomography within the same neural-network parameterization. Our results provide an adaptable and systematically improvable approach for extracting partonic distributions from Euclidean correlators. It can incorporate polarization, additional channels, and future experimental constraints from current and future facilities, such as the Electron-Ion Collider.

Hadronic Spectroscopy↗
NASA NTRSDate not supplied

Enhancing Neural Network Decision-Making with Variational Autoencoders

Machine intelligence has been used to tackle increasingly complex problems and deep learning solutions are at the forefront of tackling these problems. In general, these architectures have a great number of parameters that are methodically updated in training. The vast number and complexity of deep neural networks makes it very difficult to decipher the inner workings of the neurons and layers that make up the network. This paper posits that trustworthiness and trust in autonomous systems are increased through eXplainable Artificial Intelligence (XAI) and presents a method that enhances the explainability and understanding of a neural network decision. We leverage variational autoencoders to produce human interpretable features from complex data sets. We show that the explainable features can then be used for machine learning applications. This explainability encourages people to be more inclined to justifiably trust machine decision-making.

Loc Tran↗
DOE OSTI2021

Flow over an espresso cup: inferring 3-D velocity and pressure fields from tomographic background oriented Schlieren via physics-informed neural networks

Tomographic background oriented Schlieren (Tomo-BOS) imaging measures density or temperature fields in three dimensions using multiple camera BOS projections, and is particularly useful for instantaneous flow visualizations of complex fluid dynamics problems. We propose a new method based on physics-informed neural networks (PINNs) to infer the full continuous three-dimensional (3-D) velocity and pressure fields from snapshots of 3-D temperature fields obtained by Tomo-BOS imaging. The PINNs seamlessly integrate the underlying physics of the observed fluid flow and the visualization data, hence enabling the inference of latent quantities using limited experimental data. In this hidden fluid mechanics paradigm, we train the neural network by minimizing a loss function composed of a data mismatch term and residual terms associated with the coupled Navier–Stokes and heat transfer equations. We first quantify the accuracy of the proposed method based on a two-dimensional synthetic data set for buoyancy-driven flow, and subsequently apply it to the Tomo-BOS data set, where we are able to infer the instantaneous velocity and pressure fields of the flow over an espresso cup based only on the temperature field provided by the Tomo-BOS imaging. Moreover, we conduct an independent PIV experiment to validate the PINN inference for the unsteady velocity field at a centre plane. To explain the observed flow physics, we also perform systematic PINN simulations at different Reynolds and Richardson numbers and quantify the variations in velocity and pressure fields. Furthermore, the results in this paper indicate that the proposed deep learning technique can become a promising direction in experimental fluid mechanics.

97 MATHEMATICS AND COMPUTING↗
NASA NTRSDate not supplied

Design and Analysis of Convolutional Neural Network for RF Signal Modulation Classification for In-Orbit Deployment

To effectively transmit data to and from satellites requires a complex and robust RF communication system. Commonly, several different types of signal modulations may be required to maximize satellite efficiency depending on a variety of unexpected channel impairments. We propose a neural network algorithm capable of learning these RF signal modulations using a supervised learning technique designed for low power, high-efficiency in-orbit deployment. The work presented demonstrates a convolutional neural network (CNN) capable of learning and recognizing a set of modulation schemes commonly used to transmit RF information. We are capable of recognizing the modulation scheme from the I and Q data channels directly, with no preprocessing or data conversion required other than breaking the incoming signal into a set of uniform normalized samples. We perform a network design and size analysis, showing that reasonably high accuracy can be obtained using networks with a relatively low number of trainable parameters. Given that a user of a system such as this may wish to receive a signal using a modulation scheme that the network has not previously learned, we demonstrate that transfer learning can learn new modulation schemes by retraining only the fully connected layers in the CNN. Thus, this type of network would excel in outer space deployment using high-efficiency transfer learning hardware. Modulation recognition can be performed through rapid feedforward computation, and the CNN training process is significantly simplified when learning new modulations is required.

CNN↗
DOE OSTI2026

Neural-Network-Enhanced COTSIM: Advancing Predictive Capabilities for Fast DIII-D Simulations

Sustaining fusion reactions in tokamaks requires heating plasma to thermonuclear temperatures while maintaining confinement and stability. Neutral beam injection (NBI) provides heating, current drive, torque, and fueling, while electron cyclotron (EC) waves are widely used for heating and current drive; together, these actuators shape the plasma current, temperature, and density profiles. The control-oriented tokamak simulator (COTSIM), a predictive, control-oriented code, has been enhanced with neural-network surrogates for transport and sources. Turbulent transport is predicted by MMMnet—a neural-network version of the updated multimode model (MMM 9.0.10)—with significantly reduced computation time relative to MMM; neoclassical transport follows the Chang–Hinton model. NUBEAMnet, a surrogate of the Monte Carlo NUBEAM module, predicts beam-driven heating, current, and torque. EC heating and current drive use a control-oriented, empirically scaled source model; plasma resistivity follows the Spitzer formulation; bootstrap current uses the Sauter model. Equilibrium is computed using both prescribed and fixed-boundary solvers (FBSs), and the pedestal structure is modeled with an empirical pedestal model. For a representative DIII-D discharge, COTSIM predicts electron and ion temperature and safety-factor profiles in close agreement with TRANSP predictive and interpretive simulations while extending predictions through the pedestal region to the plasma edge (versus 80% of the minor radius in TRANSP). Furthermore, the equivalent COTSIM simulation runs in under 3 min compared to about 2 h for TRANSP, enabling rapid scenario planning, optimization of tokamak operation, and between-pulse control design.

Control-oriented tokamak simulator (COTSIM)↗
DOE OSTI2020

Fast predictions of liquid-phase acid-catalyzed reaction rates using molecular dynamics simulations and convolutional neural networks

The rates of liquid-phase, acid-catalyzed reactions relevant to the upgrading of biomass into high-value chemicals are highly sensitive to solvent composition and identifying suitable solvent mixtures is theoretically and experimentally challenging. We show that the complex atomistic configurations of reactant–solvent environments generated by classical molecular dynamics simulations can be exploited by 3D convolutional neural networks to enable accurate predictions of Brønsted acid-catalyzed reaction rates for model biomass compounds. We develop a 3D convolutional neural network, which we call SolventNet, and train it to predict acid-catalyzed reaction rates using experimental reaction data and corresponding molecular dynamics simulation data for seven biomass-derived oxygenates in water–cosolvent mixtures. We show that SolventNet can predict reaction rates for additional reactants and solvent systems an order of magnitude faster than prior simulation methods. This combination of machine learning with molecular dynamics enables the rapid, high-throughput screening of solvent systems and identification of improved biomass conversion conditions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Scalable balanced training of conditional generative adversarial neural networks on image data

Here, we propose a distributed approach to train deep convolutional generative adversarial neural network (DC-CGANs) models. Our method reduces the imbalance between generator and discriminator by partitioning the training data according to data labels, and enhances scalability by performing a parallel training where multiple generators are concurrently trained, each one of them focusing on a single data label. Performance is assessed in terms of inception score, Fréchet inception distance, and image quality on MNIST, CIFAR10, CIFAR100, and ImageNet1k datasets, showing a significant improvement in comparison to state-of-the-art techniques to training DC-CGANs. Weak scaling is attained on all the four datasets using up to 1000 processes and 2000 NVIDIA V100 GPUs on the OLCF supercomputer Summit.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Surrogate modeling of Monte Carlo radiation transport with convolutional neural networks for shielding optimization

Here, we present a machine learning (ML)-based surrogate model using convolutional neural networks (CNN) designed to emulate the attenuation of neutron fields as they pass through various shielding materials. This model can compute the outgoing neutron flux almost instantaneously and achieves reasonable accuracy compared to traditional Monte Carlo (MC)-based codes, which are computationally intensive. This emulator alleviates the complexity of neutron radiation transport through shielding materials by reducing the dimensionality and enables shielding optimization for a known radiation environment. This optimization process, which would have taken an unrealistic timeline due to several complex radiation transport simulations, can now be achieved in minutes, thus increasing computational capabilities in radiation shielding assessment. We demonstrate the applications of this emulator in computing effective dose rates and optimizing shielding solutions for a heavy-ion accelerator facility, such as the Facility for Rare Isotope Beams, where secondary neutrons produced via beam interactions dominate the radiation environment.

accelerator shielding↗
DOE OSTI2023

Abisko: Deep codesign of an architecture for spiking neural networks using novel neuromorphic materials

The Abisko project aims to develop an energy-efficient spiking neural network (SNN) computing architecture and software system capable of autonomous learning and operation. The SNN architecture explores novel neuromorphic devices that are based on resistive-switching materials, such as memristors and electrochemical RAM. Equally important, Abisko uses a deep codesign approach to pursue this goal by engaging experts from across the entire range of disciplines: materials, devices and circuits, architectures and integration, software, and algorithms. Here, the key objectives of our Abisko project are threefold. First, we are designing an energy-optimized high-performance neuromorphic accelerator based on SNNs. This architecture is being designed as a chiplet that can be deployed in contemporary computer architectures and we are investigating novel neuromorphic materials to improve its design. Second, we are concurrently developing a productive software stack for the neuromorphic accelerator that will also be portable to other architectures, such as field-programmable gate arrays and GPUs. Third, we are creating a new deep codesign methodology and framework for developing clear interfaces, requirements, and metrics between each level of abstraction to enable the system design to be explored and implemented interchangeably with execution, measurement, a model, or simulation. As a motivating application for this codesign effort, we target the use of SNNs for an analog event detector for a high-energy physics sensor.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Scalable training of graph convolutional neural networks for fast and accurate predictions of HOMO-LUMO gap in molecules

Abstract Graph Convolutional Neural Network (GCNN) is a popular class of deep learning (DL) models in material science to predict material properties from the graph representation of molecular structures. Training an accurate and comprehensive GCNN surrogate for molecular design requires large-scale graph datasets and is usually a time-consuming process. Recent advances in GPUs and distributed computing open a path to reduce the computational cost for GCNN training effectively. However, efficient utilization of high performance computing (HPC) resources for training requires simultaneously optimizing large-scale data management and scalable stochastic batched optimization techniques. In this work, we focus on building GCNN models on HPC systems to predict material properties of millions of molecules. We use HydraGNN, our in-house library for large-scale GCNN training, leveraging distributed data parallelism in PyTorch. We use ADIOS, a high-performance data management framework for efficient storage and reading of large molecular graph data. We perform parallel training on two open-source large-scale graph datasets to build a GCNN predictor for an important quantum property known as the HOMO-LUMO gap. We measure the scalability, accuracy, and convergence of our approach on two DOE supercomputers: the Summit supercomputer at the Oak Ridge Leadership Computing Facility (OLCF) and the Perlmutter system at the National Energy Research Scientific Computing Center (NERSC). We present our experimental results with HydraGNN showing (i) reduction of data loading time up to 4.2 times compared with a conventional method and (ii) linear scaling performance for training up to 1024 GPUs on both Summit and Perlmutter.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Reionization history constraints from neural network based predictions of high-redshift quasar continua

ABSTRACT Observations of the early Universe suggest that reionization was complete by z ∼ 6, however, the exact history of this process is still unknown. One method for measuring the evolution of the neutral fraction throughout this epoch is via observing the Lyα damping wings of high-redshift quasars. In order to constrain the neutral fraction from quasar observations, one needs an accurate model of the quasar spectrum around Lyα, after the spectrum has been processed by its host galaxy but before it is altered by absorption and damping in the intervening intergalactic medium (IGM). In this paper, we present a novel machine learning approach, using artificial neural networks, to reconstruct quasar continua around Lyα. Our Quasar Spectra from Artificial Neural Network based predictive Regression Algorithm(QSANNdRA) improves the error in this reconstruction compared to the state-of-the-art principal component analysis (PCA) based model in the literature by 14.2 per cent on average, and provides an improvement of 6.1 per cent on average when compared to an extension thereof. In comparison with the extended PCA model, QSANNdRA further achieves an improvement of 22.1 per cent and 16.8 per cent when evaluated on low-redshift quasars most similar to the two high-redshift quasars under consideration, ULAS J1120+0641 at z = 7.0851 and ULAS J1342+0928 at z = 7.5413, respectively. Using our more accurate reconstructions of these two z > 7 quasars, we estimate the neutral fraction of the IGM using a homogeneous reionization model and find $\bar{x}_\mathrm{H\, \small{I}} = 0.25^{+0.05}_{-0.05}$ at z = 7.0851 and $\bar{x}_\mathrm{H\, \small{I}} = 0.60^{+0.11}_{-0.11}$ at z = 7.5413. Our results are consistent with the literature and favour a rapid end to reionization.

Ďurovčíková, Dominika↗
DOE OSTI2025

CONCURRENT, CONDENSED STEIN VARIATIONAL GRADIENT DESCENT FOR UNCERTAINTY QUANTIFICATION OF NEURAL NETWORKS

In this work, we propose a Stein variational gradient descent (SVGD) method to concurrently sparsify, train, and provide uncertainty quantification (UQ) of a complexly parameterized model, such as a neural network (NN). It employs a graph reconciliation and condensation process to reduce complexity and increase similarity in the Stein ensemble of parameterizations. Therefore, the proposed concurrent, condensed SVGD (ccSVGD) method can provide UQ on parameters, not just outputs. Furthermore, the parameter reduction speeds up the convergence of the Stein gradient descent as it reduces the combinatorial complexity by aligning and differentiating the sensitivity to parameters. These properties are demonstrated with an illustrative example and an application to a mechanical response representation problem in solid mechanics.

42 ENGINEERING↗
DOE OSTI2021

Extraction of the muon signals recorded with the surface detector of the Pierre Auger Observatory using recurrent neural networks

The Pierre Auger Observatory, at present the largest cosmic-ray observatory ever built, is instrumented with a ground array of 1600 water-Cherenkov detectors, known as the Surface Detector (SD). The SD samples the secondary particle content (mostly photons, electrons, positrons and muons) of extensive air showers initiated by cosmic rays with energies ranging from 1017eV up to more than 1020eV. Measuring the independent contribution of the muon component to the total registered signal is crucial to enhance the capability of the Observatory to estimate the mass of the cosmic rays on an event-by-event basis. However, with the current design of the SD, it is difficult to straightforwardly separate the contributions of muons to the SD time traces from those of photons, electrons and positrons. In this paper, we present a method aimed at extracting the muon component of the time traces registered with each individual detector of the SD using Recurrent Neural Networks. We derive the performances of the method by training the neural network on simulations, in which the muon and the electromagnetic components of the traces are known. We conclude this work showing the performance of this method on experimental data of the Pierre Auger Observatory. We find that our predictions agree with the parameterizations obtained by the AGASA collaboration to describe the lateral distributions of the electromagnetic and muonic components of extensive air showers.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Accelerate microstructure evolution simulation using graph neural networks with adaptive spatiotemporal resolution

Abstract Surrogate models driven by sizeable datasets and scientific machine-learning methods have emerged as an attractive microstructure simulation tool with the potential to deliver predictive microstructure evolution dynamics with huge savings in computational costs. Taking 2D and 3D grain growth simulations as an example, we present a completely overhauled computational framework based on graph neural networks with not only excellent agreement to both the ground truth phase-field methods and theoretical predictions, but enhanced accuracy and efficiency compared to previous works based on convolutional neural networks. These improvements can be attributed to the graph representation, both improved predictive power and a more flexible data structure amenable to adaptive mesh refinement. As the simulated microstructures coarsen, our method can adaptively adopt remeshed grids and larger timesteps to achieve further speedup. The data-to-model pipeline with training procedures together with the source codes are provided.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Automated Stellar Spectra Classification with Ensemble Convolutional Neural Network

Large sky survey telescopes have produced a tremendous amount of astronomical data, including spectra. Machine learning methods must be employed to automatically process the spectral data obtained by these telescopes. Classification of stellar spectra by applying deep learning is an important research direction for the automatic classification of high-dimensional celestial spectra. In this paper, a robust ensemble convolutional neural network (ECNN) was designed and applied to improve the classification accuracy of massive stellar spectra from the Sloan digital sky survey. We designed six classifiers which consist six different convolutional neural networks (CNN), respectively, to recognize the spectra in DR16. Then, according the cross-entropy testing error of the spectra at different signal-to-noise ratios, we integrate the results of different classifiers in an ensemble learning way to improve the effect of classification. The experimental result proved that our one-dimensional ECNN strategy could achieve 95.0% accuracy in the classification task of the stellar spectra, a level of accuracy that exceeds that of the classical principal component analysis and support vector machine model.

79 ASTRONOMY AND ASTROPHYSICS↗