Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Neural network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,297 records · Page 72
All sourcesNASA NTRSDOE OSTI
NASA NTRS1994

Higher-Order Neural Networks Applied to 2D and 3D Object Recognition

A Higher-Order Neural Network (HONN) can be designed to be invariant to geometric transformations such as scale, translation, and in-plane rotation. Invariances are built directly into the architecture of a HONN and do not need to be learned. Thus, for 2D object recognition, the network needs to be trained on just one view of each object class, not numerous scaled, translated, and rotated views. Because the 2D object recognition task is a component of the 3D object recognition task, built-in 2D invariance also decreases the size of the training set required for 3D object recognition. We present results for 2D object recognition both in simulation and within a robotic vision experiment and for 3D object recognition in simulation. We also compare our method to other approaches and show that HONNs have distinct advantages for position, scale, and rotation-invariant object recognition. The major drawback of HONNs is that the size of the input field is limited due to the memory required for the large number of interconnections in a fully connected network. We present partial connectivity strategies and a coarse-coding technique for overcoming this limitation and increasing the input field to that required by practical object recognition problems.

Spirkovska, Lilly↗
DOE OSTI2022

Invertible neural networks for E3SM land model calibration and simulation

We apply an invertible neural network (INN) for E3SM land model calibration and simulation with eight parameters at the Missouri Ozark AmeriFlux forest site. INN provides bijective (two-way) mappings between inputs and outputs, thus it can solve probabilistic inverse problems and forward approximations simultaneously. We demonstrate INN's inverse and forward capability in both synthetic and real-data applications. Results indicate that INN produces accurate parameter posterior distributions similar to Markov Chain Monte Carlo sampling and it generates model outputs close to the forward model simulations. Additionally, both the inverse and forward evaluations in INN are computationally efficient which allows for rapid integration of observations for parameter estimation and fast model predictions.

Lu, Dan↗
DOE OSTI2022

A Neural Network Approach to Predict Gibbs Free Energy of Ternary Solid Solutions

Here, we present a data-centric deep learning (DL) approach using neural networks (NNs) to predict the thermodynamics of ternary solid solutions. We explore how NNs can be trained with a dataset of Gibbs free energies computed from a CALPHAD database to predict ternary systems as a function of composition and temperature. We have chosen the energetics of the FCC solid solution phase in 226 binaries consisting of 23 elements at 11 different temperatures to demonstrate the feasibility. The number of binary data points included in the present study is 102,000. We select six ternaries to augment the binary dataset to investigate their influence on the NN prediction accuracy. We examine the sensitivity of data sampling on the prediction accuracy of NNs over selected ternary systems. It is anticipated that the current DL workflow can be further elevated by integrating advanced descriptors beyond the elemental composition and more curated training datasets to improve prediction accuracy and applicability.

42 ENGINEERING↗
DOE OSTI2024

Analytic Neural Network Gaussian Process Enabled Chance-Constrained Voltage Regulation for Active Distribution Systems with PVs, Batteries and EVs

This paper proposes an analytic neural network Gaussian process (NNGP)-based chance-constrained real-time voltage regulation method for active distribution systems with photovoltaics (PVs), batteries, and electric vehicles (EVs). NNGP can utilize historical measurement data to achieve real-time probabilistic node voltage estimation through Bayesian inference. Then, NNGP is fully analytically embedded into the optimal power flow model to perform voltage regulation and adapt to various topological changes. The uncertainties of voltage estimations are easily considered via the chance constraint, and it has been shown that the adoption of this chance constraint can significantly improve the reliability of voltage regulation under various scenarios. The comparison results with other methods, carried out on a real 759-node distribution system located in western Colorado, U.S., show that the proposed method can achieve accurate voltage estimation across different topologies and reliably perform voltage regulation considering PVs, batteries, and EVs.

active distribution systems↗
DOE OSTI2019

ECabc: A feature tuning program focused on Artificial Neural Network hyperparameters

ECabc is an open source Python package based on an Artificial Bee Colony, used for tuning functions such as hyperparameters of artificial neural networks. The method is modelled after the honey foraging techniques of bees, where the search space for solutions is in as many dimensions as the number of parameters being tuned.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Contrasting advantages of learning with random weights and backpropagation in non-volatile memory neural networks

Recently, a Cambrian explosion of novel, non-volatile memory (NVM) devices known as memristive devices have inspired effort in building hardware neural networks that learn like the brain. Early experimental prototypes built simple perceptrons from nanosynapses, and recently, fully-connected multi-layer perceptron (MLP) learning systems have been realized. However, while backpropagating learning systems pair well with high-precision computer memories and achieve state-of-the-art performances, this typically comes with a massive energy budget. For future Internet of Things/peripheral use cases, system energy footprint will be a major constraint, and emerging NVM devices may fill the gap by sacrificing high bit precision for lower energy. In this work, we contrast the well known MLP approach with the Extreme Learning Machine (ELM) or NoProp approach, which uses a large layer of random weights to improve the separability of high-dimensional tasks, and is usually considered inferior in a software context. However, we find that when taking device non-linearity into account, NoProp manages to equal hardware MLP system in terms of accuracy. While also using a sign-based adaptation of the delta rule for energy-savings, we find that NoProp can learn effectively with four to six ’bits’ of device analog capacity, while MLP requires eight bit capacity with the same rule. This may allow the requirements for memristive devices to be relaxed in the context of online learning. By comparing the energy footprint of these systems for several candidate nanosynapses, as well as realistic peripherals, we confirm that memristive NoProp systems save energy compared to MLP systems. Lastly, we show that ELM/NoProp systems can achieve better generalization abilities than nanosynaptic MLP systems when paired with pre-processing layers (which do not require backpropagated error). Collectively, these advantages make such systems worthy of consideration in future accelerators or embedded hardware.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Neural-network diagnostics of electrochemical machining

In the high-precision electrochemical shaping of new high-strength and hard materials, diagnostics may be based on neural-network identification algorithms incorporated in the automated control system of the equipment.

42 ENGINEERING↗
DOE OSTI2021

Fault Detection Utilizing Convolution Neural Network on Timeseries Synchrophasor Data From Phasor Measurement Units

An end-to-end supervised learning method is proposed for fault detection in the electric grid using Big Data from multiple Phasor Measurement Units (PMUs). The approach consists of preprocessing steps aimed at reducing data noise and dimensionality, followed by utilization of six classification models considered for detecting faults. Three of the models were variants of Convolutional Neural Network (CNN) architectures that consider a single type of measurement (voltage, current or frequency) at all PMUs or all types together also at all PMUs. CNN based models were compared to traditional methods of Logistic Regression (LR), Multi-layer Perceptron (MLP) and Support Vector Machine (SVM). Evaluation was conducted on two-year data measured by PMUs at 37 locations in a large electric grid. Here, the response variable for classification were extracted from the grid-wide outage event log. Experiments show that CNN-based models outperformed traditional methods on one year out-of-sample outage detection over the entire grid.

42 ENGINEERING↗
DOE OSTI2022

Nonvolatile Memories in Spiking Neural Network Architectures: Current and Emerging Trends

A sustainable computing scenario demands more energy-efficient processors. Neuromorphic systems mimic biological functions by employing spiking neural networks for achieving brain-like efficiency, speed, adaptability, and intelligence. Current trends in neuromorphic technologies address the challenges of investigating novel materials, systems, and architectures for enabling high-integration and extreme low-power brain-inspired computing. This review collects the most recent trends in exploiting the physical properties of nonvolatile memory technologies for implementing efficient in-memory and in-device computing with spike-based neuromorphic architectures.

42 ENGINEERING↗
DOE OSTI2019

Neural network based quasi-diabatic Hamiltonians with symmetry adaptation and a correct description of conical intersections

Previously, we demonstrated in a report that artificial neural networks (NNs) can be used to generate quasidiabatic Hamiltonians (H d ) that are capable of representing adiabatic energies, energy gradients, and derivative couplings. In this work, two additional issues are addressed. First, symmetry-adapted functions such as permutation invariant polynomials are introduced to account for complete nuclear permutation inversion symmetry. Second, a partially diagonalized representation is introduced to facilitate a better description of near degeneracy points. The diabatization of 1, 2 1 A states of NH 3 is used as an example. The NN fitting findings are compared to that of a previous fitting with symmetry adapted polynomials.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

WellPINN: Accurate Well Representation for Transient Fluid Pressure Diffusion in Subsurface Reservoirs With Physics‐Informed Neural Networks

Accurate representation of pumping wells is essential for reliable reservoir characterization and simulation of operational scenarios in subsurface flow models. Physics-informed neural networks (PINNs) are emerging as a promising alternative to numerical models for reservoir modeling, offering seamless integration of monitoring data and governing physical equations. However, existing PINN-based studies face major challenges in capturing fluid pressure near wells when using a source/sink term, particularly during the early stages after pumping begins. We address this problem by introducing WellPINN, a workflow in which an initially trained PINN infers fluid pressure across the entire reservoir domain using a large equivalent well radius. This initial PINN solution is then locally refined around the well by a set of subdomain PINNs that are trained for smaller equivalent well radii. Continuity across these subdomain interfaces as well as at the initial condition is ensured by hard-constraining each PINN on its subdomain boundary. Our results demonstrate WellPINN as the first workflow of its kind to focus on accurate inference of fluid pressure from pumping rates throughout the entire injection period, significantly advancing the potential of PINNs for inverse modeling and operational scenario simulations. All data and code for this paper are openly available at https://doi.org/10.20350/DIGITALCSIC/17260.

58 GEOSCIENCES↗
DOE OSTI2025

Application of a Physics-Informed Convolutional Neural Network for Monitoring the Temperature Fields in High-Temperature Gas Reactors

Here, this work presents current advances in applying a physics-informed convolutional neural network (CNN) to evaluate temperature distributions in advanced reactors. Our goal is to demonstrate that the CNN can reconstruct temperature fields within the solid region of a prismatic fuel assembly in a high-temperature gas reactor (HTGR) with sensor data available in only a few cooling channels. Before that, we showcase the superior performance of the physics-informed CNN in comparison to a purely data-driven multilayer perceptron (MLP), considering a canonical heated channel setup. This analysis shows the advantages of our approach and justifies its choice. The datasets employed here are obtained upon numerical simulations performed with codes under the Nuclear Energy Advanced Modeling and Simulation program. This work is important, as industry experience indicates that the assembly material in HTGR concepts is prone to large thermal-mechanical loads nearing operational limits. This makes it crucial to characterize peak temperatures and their distributions near hot spots. Modern thermocouples are unreliable in these types of harsh environments because of the high neutron fluxes and elevated temperatures involved. The CNN-based field reconstruction represents an attractive solution, enabling sensor arrays in less aggressive locations and augmenting indirect predictions for less accessible regions. The results show that the CNN reduces prediction errors by orders of magnitude in comparison to the MLP, considering the simple yet well-representative heated channel case. In the case of the HTGR fuel assembly, the CNN can successfully reconstruct temperature fields over various cooling regimes. Furthermore, we also explore the algorithm’s ability to detect abnormalities. Interestingly, the CNN proves it has the capacity to detect blockage in one of the noninstrumented cooling channels.

Machine learning↗
DOE OSTI2019

Neural network approximation of Bayesian models for the inference of ion and electron temperature profiles at W7-X

Here, we describe a method for training a neural network (NN) to approximate the full model Bayesian inference of plasma profiles from x-ray imaging diagnostic measurements. The modeling is carried out within the Minerva Bayesian modeling framework where models are defined as a set of assumptions, prior beliefs on parameter values and physics knowledge. The goal is to use NNs for fast ion and electron temperature profile inversion from measured image data. The NN is trained solely on artificial data generated by sampling from the joint distribution of the free parameters and model predictions. The training is carried out in such a way that the mapping learned by the network constitutes an approximation of the full model Bayesian inference. The analysis is carried out on images constituted of 20 × 195 pixels corresponding to binned lines of sight and spectral channels, respectively. Through the full model inference, it is possible to infer electron and ion temperature profiles as well as impurity density profiles. When the network is utilized for the inference of the temperature profiles, the analysis time can be reduced down to a few tens of microseconds for a single time point, which is a drastic improvement if compared to the ≈4 h long Bayesian inference. The procedure developed for the generation of the training set does not rely on diagnostic-specific features, and therefore it is in principle applicable to any other model developed within the Minerva framework. The trained NN has been tested on data collected during the first operational campaign at W7-X, and compared to the full model Bayesian inference results.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2026

Improving ideal MHD equilibrium accuracy with physics-informed neural networks

We present a novel approach to compute three-dimensional magnetohydrodynamic equilibria with isotropic pressure profiles and nested surfaces by parametrizing Fourier modes with artificial neural networks (NNs). The full nonlinear global force residual of single equilibria across the volume in real space is then minimized with first order optimizers and compared to equilibria computed by conventional solvers. Already, we observe competitive computational cost to arrive at the same minimum residuals computable with existing codes. With increased computational cost, lower minima of the residual are computable with the NNs than with any other tested solver, establishing a new lower bound for the force residual. We use minimally complex NNs, and we expect significant improvements for solving not only single equilibria with NNs, but also for creating NN models valid over continuous distributions of equilibria.

ideal magnetohydrodynamics↗
DOE OSTI2021

Exploring the Value of Nodes with Multicommunity Membership for Classification with Graph Convolutional Neural Networks

Sampling is an important step in the machine learning process because it prioritizes samples that help the model best summarize the important concepts required for the task at hand. The process of determining the best sampling method has been rarely studied in the context of graph neural networks. In this paper, we evaluate multiple sampling methods (i.e., ascending and descending) that sample based off different definitions of centrality (i.e., Voterank, Pagerank, degree) to observe its relation with network topology. We find that no sampling method is superior across all network topologies. Additionally, we find situations where ascending sampling provides better classification scores, showing the strength of weak ties. Two strategies are then created to predict the best sampling method, one that observes the homogeneous connectivity of the nodes, and one that observes the network topology. In both methods, we are able to evaluate the best sampling direction consistently.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Modern deep neural networks for Direct Normal Irradiance forecasting: A classification approach

The escalating energy demand and the adverse environmental impacts of fossil-fuel use necessitate a shift towards cleaner and renewable alternatives. Concentrated Solar Power (CSP) technology emerges as a promising solution, offering a carbon-free alternative for power generation. The efficiency and profitability of CSP depend on the Direct Normal Irradiance (DNI) component of solar radiation; hence, accurate DNI forecasting can help optimize CSP plants’ operations and performance. The unpredictable nature of weather phenomena, particularly cloud cover, introduces uncertainty into DNI projections. Existing DNI forecasting models use meteorological factors, which are both challenging to estimate numerically over short prediction windows and expensive to model through data at a sufficiently high spatial and temporal resolution. This research addresses the challenge by presenting a novel approach that formulates DNI prediction as a multi-class classification problem, departing from conventional regression-based methods. The primary objective of this classification framework is to identify optimal periods aligning with specific operational thresholds for CSP plants, contributing to enhanced dispatch optimization strategies. We model the DNI classification problem using four advanced deep neural networks – rectified linear unit (ReLU) networks, 1D residual networks (ResNets), bidirectional long short-term memory (BiLSTM) networks, and transformers – achieving accuracies up to 93.5% without requiring meteorological parameters.

14 SOLAR ENERGY↗