Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Neural Network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,333 records · Page 74

Simulation tests of the optimization method of Hopfield and Tank using neural networks

The method proposed by Hopfield and Tank for using the Hopfield neural network with continuous valued neurons to solve the traveling salesman problem is tested by simulation. Several researchers have apparently been unable to successfully repeat the numerical simulation documented by Hopfield and Tank. However, as suggested to the author by Adams, it appears that the reason for those difficulties is that a key parameter value is reported erroneously (by four orders of magnitude) in the original paper. When a reasonable value is used for that parameter, the network performs generally as claimed. Additionally, a new method of using feedback to control the input bias currents to the amplifiers is proposed and successfully tested. This eliminates the need to set the input currents by trial and error.

Paielli, Russell A.↗

Neural network approaches versus statistical methods in classification of multisource remote sensing data

Neural network learning procedures and statistical classificaiton methods are applied and compared empirically in classification of multisource remote sensing and geographic data. Statistical multisource classification by means of a method based on Bayesian classification theory is also investigated and modified. The modifications permit control of the influence of the data sources involved in the classification process. Reliability measures are introduced to rank the quality of the data sources. The data sources are then weighted according to these rankings in the statistical multisource classification. Four data sources are used in experiments: Landsat MSS data and three forms of topographic data (elevation, slope, and aspect). Experimental results show that two different approaches have unique advantages and disadvantages in this classification application.

Benediktsson, Jon A.↗

The identification of helicopter noise using a neural network

Experiments were carried out to demonstrate the ability of an artificial neural network (ANN) system to distinguish between the noise of two helicopters. The ANN is taught to identify helicopters by using two types of features: one that is associated with the ratio of the main-rotor to tail-rotor blade passage frequency (BPF), and the ohter that describes the distribution of peaks in the main-rotor spectrum, which is independent of the tail-rotor. It is shown that the ability of the ANN to identify helicopters is comparable to that of a conventional recognition system using the ratio of the main-rotor BPF to the tail-rotor BPF (when both the main- and the tail-rotor noise are present), but the performoance of ANN exceeds the conventional-method performance when the tail-rotor noise is absent. In addition, the results of ANN can be obtained as a function of propagation distance.

Cabell, Randolph H.↗

Neural-Network Object-Recognition Program

HONTIOR computer program implements third-order neural network exhibiting invariance under translation, change of scale, and in-plane rotation. Invariance incorporated directly into architecture of network. Only one view of each object needed to train network for two-dimensional-translation-invariant recognition of object. Also used for three-dimensional-transformation-invariant recognition by training network on only set of out-of-plane rotated views. Written in C language.

Spirkovska, L.↗

The 3-D image recognition based on fuzzy neural network technology

Three dimensional stereoscopic image recognition system based on fuzzy-neural network technology was developed. The system consists of three parts; preprocessing part, feature extraction part, and matching part. Two CCD color camera image are fed to the preprocessing part, where several operations including RGB-HSV transformation are done. A multi-layer perception is used for the line detection in the feature extraction part. Then fuzzy matching technique is introduced in the matching part. The system is realized on SUN spark station and special image input hardware system. An experimental result on bottle images is also presented.

Hirota, Kaoru↗

Automated Induction Of Rule-Based Neural Networks

Prototype expert systems implemented in software and are functionally equivalent to neural networks set up automatically and placed into operation within minutes following information-theoretic approach to automated acquisition of knowledge from large example data bases. Approach based largely on use of ITRULE computer program.

Smyth, Padhraic J.↗

Higher-Order Neural Networks Applied to 2D and 3D Object Recognition

A Higher-Order Neural Network (HONN) can be designed to be invariant to geometric transformations such as scale, translation, and in-plane rotation. Invariances are built directly into the architecture of a HONN and do not need to be learned. Thus, for 2D object recognition, the network needs to be trained on just one view of each object class, not numerous scaled, translated, and rotated views. Because the 2D object recognition task is a component of the 3D object recognition task, built-in 2D invariance also decreases the size of the training set required for 3D object recognition. We present results for 2D object recognition both in simulation and within a robotic vision experiment and for 3D object recognition in simulation. We also compare our method to other approaches and show that HONNs have distinct advantages for position, scale, and rotation-invariant object recognition. The major drawback of HONNs is that the size of the input field is limited due to the memory required for the large number of interconnections in a fully connected network. We present partial connectivity strategies and a coarse-coding technique for overcoming this limitation and increasing the input field to that required by practical object recognition problems.

Spirkovska, Lilly↗

Invertible neural networks for E3SM land model calibration and simulation

We apply an invertible neural network (INN) for E3SM land model calibration and simulation with eight parameters at the Missouri Ozark AmeriFlux forest site. INN provides bijective (two-way) mappings between inputs and outputs, thus it can solve probabilistic inverse problems and forward approximations simultaneously. We demonstrate INN's inverse and forward capability in both synthetic and real-data applications. Results indicate that INN produces accurate parameter posterior distributions similar to Markov Chain Monte Carlo sampling and it generates model outputs close to the forward model simulations. Additionally, both the inverse and forward evaluations in INN are computationally efficient which allows for rapid integration of observations for parameter estimation and fast model predictions.

Lu, Dan↗

A Neural Network Approach to Predict Gibbs Free Energy of Ternary Solid Solutions

Here, we present a data-centric deep learning (DL) approach using neural networks (NNs) to predict the thermodynamics of ternary solid solutions. We explore how NNs can be trained with a dataset of Gibbs free energies computed from a CALPHAD database to predict ternary systems as a function of composition and temperature. We have chosen the energetics of the FCC solid solution phase in 226 binaries consisting of 23 elements at 11 different temperatures to demonstrate the feasibility. The number of binary data points included in the present study is 102,000. We select six ternaries to augment the binary dataset to investigate their influence on the NN prediction accuracy. We examine the sensitivity of data sampling on the prediction accuracy of NNs over selected ternary systems. It is anticipated that the current DL workflow can be further elevated by integrating advanced descriptors beyond the elemental composition and more curated training datasets to improve prediction accuracy and applicability.

42 ENGINEERING↗

Analytic Neural Network Gaussian Process Enabled Chance-Constrained Voltage Regulation for Active Distribution Systems with PVs, Batteries and EVs

This paper proposes an analytic neural network Gaussian process (NNGP)-based chance-constrained real-time voltage regulation method for active distribution systems with photovoltaics (PVs), batteries, and electric vehicles (EVs). NNGP can utilize historical measurement data to achieve real-time probabilistic node voltage estimation through Bayesian inference. Then, NNGP is fully analytically embedded into the optimal power flow model to perform voltage regulation and adapt to various topological changes. The uncertainties of voltage estimations are easily considered via the chance constraint, and it has been shown that the adoption of this chance constraint can significantly improve the reliability of voltage regulation under various scenarios. The comparison results with other methods, carried out on a real 759-node distribution system located in western Colorado, U.S., show that the proposed method can achieve accurate voltage estimation across different topologies and reliably perform voltage regulation considering PVs, batteries, and EVs.

active distribution systems↗

ECabc: A feature tuning program focused on Artificial Neural Network hyperparameters

ECabc is an open source Python package based on an Artificial Bee Colony, used for tuning functions such as hyperparameters of artificial neural networks. The method is modelled after the honey foraging techniques of bees, where the search space for solutions is in as many dimensions as the number of parameters being tuned.

97 MATHEMATICS AND COMPUTING↗

Contrasting advantages of learning with random weights and backpropagation in non-volatile memory neural networks

Recently, a Cambrian explosion of novel, non-volatile memory (NVM) devices known as memristive devices have inspired effort in building hardware neural networks that learn like the brain. Early experimental prototypes built simple perceptrons from nanosynapses, and recently, fully-connected multi-layer perceptron (MLP) learning systems have been realized. However, while backpropagating learning systems pair well with high-precision computer memories and achieve state-of-the-art performances, this typically comes with a massive energy budget. For future Internet of Things/peripheral use cases, system energy footprint will be a major constraint, and emerging NVM devices may fill the gap by sacrificing high bit precision for lower energy. In this work, we contrast the well known MLP approach with the Extreme Learning Machine (ELM) or NoProp approach, which uses a large layer of random weights to improve the separability of high-dimensional tasks, and is usually considered inferior in a software context. However, we find that when taking device non-linearity into account, NoProp manages to equal hardware MLP system in terms of accuracy. While also using a sign-based adaptation of the delta rule for energy-savings, we find that NoProp can learn effectively with four to six ’bits’ of device analog capacity, while MLP requires eight bit capacity with the same rule. This may allow the requirements for memristive devices to be relaxed in the context of online learning. By comparing the energy footprint of these systems for several candidate nanosynapses, as well as realistic peripherals, we confirm that memristive NoProp systems save energy compared to MLP systems. Lastly, we show that ELM/NoProp systems can achieve better generalization abilities than nanosynaptic MLP systems when paired with pre-processing layers (which do not require backpropagated error). Collectively, these advantages make such systems worthy of consideration in future accelerators or embedded hardware.

97 MATHEMATICS AND COMPUTING↗

Linear eddy mixing based tabulation and artificial neural networks for large eddy simulations of turbulent flames

A large eddy simulation (LES) sub-grid model is developed based on the artificial neural network (ANN) approach to calculate the species instantaneous reaction rates for multi-step, multi-species chemical kinetics mechanisms. The proposed methodology depends on training the ANNs off-line on a thermo-chemical database representative of the actual composition and turbulence (but not the actual geometrical problem) of interest, and later using them to replace the stiff ODE solver (direct integration (DI)) to calculate the reaction rates in the sub-grid. The thermo-chemical database is tabulated with respect to the thermodynamic state vector without any reduction in the number of state variables. The thermo-chemistry is evolved by stand-alone linear eddy mixing (LEM) model simulations under both premixed and non-premixed conditions, where the unsteady interaction of turbulence with chemical kinetics is included as a part of the training database. The proposed methodology is tested in LES and in stand-alone LEM studies of three distinct test cases with different reduced mechanisms and conditions. LES of premixed flame-turbulence-vortex interaction provides direct comparison of the proposed ANN method against DI and ANNs trained on thermo-chemical database created using another type of tabulation method. It is shown that the ANN trained on the LEM database can capture the correct flame physics with accuracy comparable to DI, which cannot be achieved by ANN trained on a laminar premix flame database. A priori evaluation of the ANN generality within and outside its training domain is carried out using stand-alone LEM simulations as well. Results in general are satisfactory, and it is shown that the ANN provides considerable amount of memory saving and speed-up with reasonable and reliable accuracy. The speed-up is strongly affected by the stiffness of the reduced mechanism used for the computations, whereas the memory saving is considerable regardless. (author)

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Neural-network diagnostics of electrochemical machining

In the high-precision electrochemical shaping of new high-strength and hard materials, diagnostics may be based on neural-network identification algorithms incorporated in the automated control system of the equipment.

42 ENGINEERING↗

Fault Detection Utilizing Convolution Neural Network on Timeseries Synchrophasor Data From Phasor Measurement Units

An end-to-end supervised learning method is proposed for fault detection in the electric grid using Big Data from multiple Phasor Measurement Units (PMUs). The approach consists of preprocessing steps aimed at reducing data noise and dimensionality, followed by utilization of six classification models considered for detecting faults. Three of the models were variants of Convolutional Neural Network (CNN) architectures that consider a single type of measurement (voltage, current or frequency) at all PMUs or all types together also at all PMUs. CNN based models were compared to traditional methods of Logistic Regression (LR), Multi-layer Perceptron (MLP) and Support Vector Machine (SVM). Evaluation was conducted on two-year data measured by PMUs at 37 locations in a large electric grid. Here, the response variable for classification were extracted from the grid-wide outage event log. Experiments show that CNN-based models outperformed traditional methods on one year out-of-sample outage detection over the entire grid.

42 ENGINEERING↗

Nonvolatile Memories in Spiking Neural Network Architectures: Current and Emerging Trends

A sustainable computing scenario demands more energy-efficient processors. Neuromorphic systems mimic biological functions by employing spiking neural networks for achieving brain-like efficiency, speed, adaptability, and intelligence. Current trends in neuromorphic technologies address the challenges of investigating novel materials, systems, and architectures for enabling high-integration and extreme low-power brain-inspired computing. This review collects the most recent trends in exploiting the physical properties of nonvolatile memory technologies for implementing efficient in-memory and in-device computing with spike-based neuromorphic architectures.

42 ENGINEERING↗

Neural network based quasi-diabatic Hamiltonians with symmetry adaptation and a correct description of conical intersections

Previously, we demonstrated in a report that artificial neural networks (NNs) can be used to generate quasidiabatic Hamiltonians (H d ) that are capable of representing adiabatic energies, energy gradients, and derivative couplings. In this work, two additional issues are addressed. First, symmetry-adapted functions such as permutation invariant polynomials are introduced to account for complete nuclear permutation inversion symmetry. Second, a partially diagonalized representation is introduced to facilitate a better description of near degeneracy points. The diabatization of 1, 2 1 A states of NH 3 is used as an example. The NN fitting findings are compared to that of a previous fitting with symmetry adapted polynomials.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Multi-Level Interval Estimation for Locating damage in Structures by Using Artificial Neural Networks

A new analysis technique, called multi-level interval estimation method, is developed for locating damage in structures. In this method, the artificial neural networks (ANN) analysis method is combined with the statistics theory to estimate the range of damage location. The ANN is multilayer perceptron trained by back-propagation. Natural frequencies and modal shape at a few selected points are used as input to identify the location and severity of damage. Considering the large-scale structures which have lots of elements, multi-level interval estimation method is developed to reduce the estimation range of damage location step-by-step. Every step, estimation range of damage location is obtained from the output of ANN by using the method of interval estimation. The next ANN training cases are selected from the estimation range after linear transform, and the output of new ANN estimation range of damage location will gained a reduced estimation range. Two numerical example analyses on 10-bar truss and 100-bar truss are presented to demonstrate the effectiveness of the proposed method.

97 MATHEMATICS AND COMPUTING↗