Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Neural Networks (Computer)”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30

Parallel programming of an ionic floating-gate memory array for scalable neuromorphic computing

Neuromorphic computers could overcome efficiency bottlenecks inherent to conventional computing through parallel programming and readout of artificial neural network weights in a crossbar memory array. However, selective and linear weight updates and <10-nanoampere read currents are required for learning that surpasses conventional computing efficiency. We introduce an ionic floating-gate memory array based on a polymer redox transistor connected to a conductive-bridge memory (CBM). Selective and linear programming of a redox transistor array is executed in parallel by overcoming the bridging threshold voltage of the CBMs. Synaptic weight readout with currents <10 nanoamperes is achieved by diluting the conductive polymer with an insulator to decrease the conductance. The redox transistors endure >1 billion write-read operations and support >1-megahertz write-read frequencies.

bio-inspired, energy storage (including batteries ↗

Enhanced read resolution in reconfigurable memristive synapses for Spiking Neural Networks

Abstract The synapse is a key element circuit in any memristor-based neuromorphic computing system. A memristor is a two-terminal analog memory device. Memristive synapses suffer from various challenges including high voltage, SET or RESET failure, and READ margin issues that can degrade the distinguishability of stored weights. Enhancing READ resolution is very important to improving the reliability of memristive synapses. Usually, the READ resolution is very small for a memristive synapse with a 4-bit data precision. This work considers a step-by-step analysis to enhance the READ current resolution or the read current difference between two resistance levels for a current-controlled memristor-based synapse. An empirical model is used to characterize the $${\hbox {HfO}}_{2}$$ HfO 2 based memristive device. $$1\textrm{st}$$ 1 st and $$2\textrm{nd}$$ 2 nd stage device of our proposed synapse design can be scaled to enhance the READ current margin up to $$\sim$$ ∼ 4.3 $$\times$$ × and $$\sim$$ ∼ 21%, respectively. Moreover, READ current resolution can be enhanced with run-time adaptation techniques such as READ voltage scaling and body biasing. The READ voltage scaling and body biasing can improve the READ current resolution by about 46% and 15%, respectively. TENNLab’s neuromorphic computing framework is leveraged to evaluate the effect of READ current resolution on classification, control, and reservoir computing applications. Higher READ current resolution shows better accuracy than lower resolution even when facing different levels of read noise.

97 MATHEMATICS AND COMPUTING↗

Surrogate construction via weight parameterization of residual neural networks

Surrogate model development is a critical step for uncertainty quantification or other sample-intensive tasks for complex computational models. Here, in this work, we develop a multi-output surrogate form using a class of neural networks (NNs) that employ shortcut connections, namely Residual NNs (ResNets). ResNets are known to regularize the surrogate learning problem and improve the efficiency and accuracy of the resulting surrogate. Inspired by the continuous, Neural ODE analogy, we augment ResNets with weight parameterization strategy with respect to ResNet depth. Weight-parameterized ResNets regularize the NN surrogate learning problem and allow better generalization with a drastically reduced number of learnable parameters. We demonstrate that weight-parameterized ResNets are more accurate and efficient than conventional feed-forward multi-layer perceptron networks. We also compare various options for parameterization of the weights as functions of ResNet depth. We demonstrate the results on both synthetic examples and a large scale earth system model of interest.

97 MATHEMATICS AND COMPUTING↗

FEDERATED LEARNING ON STOCHASTIC NEURAL NETWORKS

Federated learning is a machine learning paradigm that leverages edge computing on client devices to optimize models while maintaining user privacy by ensuring that local data remain on the device. However, since all data are collected by clients, federated learning is susceptible to latent noise in local datasets. Factors such as limited measurement capabilities or human errors may introduce inaccuracies in client data. To address this challenge, we propose the use of a stochastic neural network as the local model within the federated learning framework. Stochastic neural networks not only facilitate the estimation of the true underlying states of the data but also enable the quantification of latent noise. We refer to our federated learning approach, which incorporates stochastic neural networks as local models, as federated stochastic neural networks. In this work we will present numerical experiments demonstrating the performance and effectiveness of our method, particularly in handling nonindependent and identically distributed data.

97 MATHEMATICS AND COMPUTING↗

Application of artificial neural networks to the design optimization of aerospace structural components

The application of artificial neural networks to capture structural design expertise is demonstrated. The principal advantage of a trained neural network is that it requires trivial computational effort to produce an acceptable new design. For the class of problems addressed, the development of a conventional expert system would be extremely difficult. In the present effort, a structural optimization code with multiple nonlinear programming algorithms and an artificial neural network code NETS were used. A set of optimum designs for a ring and two aircraft wings for static and dynamic constraints were generated by using the optimization codes. The optimum design data were processed to obtain input and output pairs, which were used to develop a trained artificial neural network with the code NETS. Optimum designs for new design conditions were predicted by using the trained network. Neural net prediction of optimum designs was found to be satisfactory for most of the output design parameters. However, results from the present study indicate that caution must be exercised to ensure that all design variables are within selected error bounds.

Berke, Laszlo↗

Reconfigurable Cascaded Thermal Neuristors for Neuromorphic Computing

While the complementary metal-oxide semiconductor (CMOS) technology is the mainstream for the hardware implementation of neural networks, an alternative route is explored based on a new class of spiking oscillators called “thermal neuristors”, which operate and interact solely via thermal processes. Utilizing the insulator-to-metal transition (IMT) in vanadium dioxide, a wide variety of reconfigurable electrical dynamics mirroring biological neurons is demonstrated. Notably, inhibitory functionality is achieved just in a single oxide device, and cascaded information flow is realized exclusively through thermal interactions. To elucidate the underlying mechanisms of the neuristors, a detailed theoretical model is developed, which accurately reflects the experimental results. In conclusion, this study establishes the foundation for scalable and energy-efficient thermal neural networks, fostering progress in brain-inspired computing.

36 MATERIALS SCIENCE↗

Predicting Melt Properties Using Atomistic Simulations With A Highly Accurate Physically Informed Neural Network Interatomic Potential

The use of a recently developed machine learning (ML) interatomic potential for molecular dynamics simulations of aluminum melt properties will be presented. Such properties are critical for process modeling in additive manufacturing, including the melt pool size, solidification, and formation of solidification microstructures. Direct first-principles modeling of these processes is computationally prohibitive whereas simulations employing ML potentials combine the high accuracy of quantum-mechanical methods with high computational speeds. The physically-informed neural network (PINN) method used herein, integrates a high-dimensional regression implemented by an artificial neural network with a physics-based bond-order interatomic potential. PINN potentials can accurately reproduce many properties of aluminum in both crystalline-solid and liquid phases. We examine the accuracy of a PINN Al potential in predicting the density, self-diffusivity, viscosity, and the tension of the liquid surface and liquid-solid interfaces. Comparison with experimental data and ab initio molecular dynamics calculations shows very good agreement for all properties tested.

molecular dynamics↗

An Approach to Realize Generalized Optimal Motion Primitives Using Physics Informed Neural Networks

Autonomous manipulation is a challenging problem in field robotics due to uncertainty in object properties, constraints, and coupling phenomenon with robot control systems. Humans learn motion primitives over time to effectively interact with the environment. We postulate that autonomous manipulation can be enabled by basic sets of motion primitives as well, but do not necessitate mimicking human motion primitives. Here, this work presents an approach to generalized optimal motion primitives using physics-informed neural networks. Our simulated and experimental results demonstrate that optimality is notionally maintained where the mean maximum observed final position percent error was 0.564% and the average mean error for all the trajectories was 1.53%. These results indicate that notional generalization is attained using a physics-informed neural network approach that enables near optimal real-time adaptation of primitive motion profiles.

97 MATHEMATICS AND COMPUTING↗

Enabling Innovation in Wind Turbine Design Using Artificial Intelligence

The Inverse Network Transformations for Efficient Generation of Robust Airfoil and Turbine Enhancements (INTEGRATE) project is developing a new inverse-design capability for wind turbine rotors using invertible neural networks. This artificial intelligence (AI)-based technology can capture complex nonlinear aerodynamic effects 100 times faster than alternative design approaches.

aerodynamics↗

A Taxonomy for Classification and Comparison of Dataflows for GNN Accelerators

Recently, Graph Neural Networks (GNNs) have received a lot of interest because of their success in learning representations from graph structured data. However, GNNs exhibit different compute and memory characteristics compared to traditional Deep Neural Networks (DNNs). Graph convolutions require feature aggregations from neighboring nodes (known as the aggregation phase), which leads to highly irregular data accesses. GNNs also have a very regular compute phase that can be broken down to matrix multiplications (known as the combination phase). All recently proposed GNN accelerators utilize different dataflows and microarchitecture optimizations for these two phases. Different communication strategies between the two phases have been also used. However, as more custom GNN accelerators are proposed, the harder it is to qualitatively classify them and quantitatively contrast them. In this work, we present a taxonomy to describe several diverse dataflows for running GNN inference on accelerators. This provides a structured way to describe and compare the design-space of GNN accelerators.

97 MATHEMATICS AND COMPUTING↗

A Comprehensive Analysis of PINNs for Power System Transient Stability

The integration of machine learning in power systems, particularly in stability and dynamics, addresses the challenges brought by the integration of renewable energies and distributed energy resources (DERs). Traditional methods for power system transient stability, involving solving differential equations with computational techniques, face limitations due to their time-consuming and computationally demanding nature. This paper introduces physics-informed Neural Networks (PINNs) as a promising solution for these challenges, especially in scenarios with limited data availability and the need for high computational speed. PINNs offer a novel approach for complex power systems by incorporating additional equations and adapting to various system scales, from a single bus to multi-bus networks. Our study presents the first comprehensive evaluation of physics-informed Neural Networks (PINNs) in the context of power system transient stability, addressing various grid complexities. Additionally, we introduce a novel approach for adjusting loss weights to improve the adaptability of PINNs to diverse systems. Our experimental findings reveal that PINNs can be efficiently scaled while maintaining high accuracy. Furthermore, these results suggest that PINNs significantly outperform the traditional ode45 method in terms of efficiency, especially as the system size increases, showcasing a progressive speed advantage over ode45.

97 MATHEMATICS AND COMPUTING↗

Prediction of the thickness of the compensator filter in radiation therapy using computational intelligence

In this study, artificial neural networks (ANNs) and adaptive neuro-fuzzy inference system (ANFIS) are investigated to predict the thickness of the compensator filter in radiation therapy. In the proposed models, the input parameters are field size (S), off-axis distance, and relative dose (D/D{sub 0}), and the output is the thickness of the compensator. The obtained results show that the proposed ANN and ANFIS models are useful, reliable, and cheap tools to predict the thickness of the compensator filter in intensity-modulated radiation therapy.

61 RADIATION PROTECTION AND DOSIMETRY↗

Towards practical control design using neural computation

The objective is to develop neural network based control design techniques which address the issue of performance/control effort tradeoff. Additionally, the control design needs to address the important issue if achieving adequate performance in the presence of actuator nonlinearities such as position and rate limits. These issues are discussed using the example of aircraft flight control. Given a set of pilot input commands, a feedforward net is trained to control the vehicle within the constraints imposed by the actuators. This is achieved by minimizing an objective function which is the sum of the tracking errors, control input rates and control input deflections. A tradeoff between tracking performance and control smoothness is obtained by varying, adaptively, the weights of the objective function. The neurocontroller performance is evaluated in the presence of actuator dynamics using a simulation of the vehicle. Appropriate selection of the different weights in the objective function resulted in the good tracking of the pilot commands and smooth neurocontrol. An extension of the neurocontroller design approach is proposed to enhance its practicality.

Troudet, Terry↗

Multispectral image fusion using neural networks

A prototype system is being developed to demonstrate the use of neural network hardware to fuse multispectral imagery. This system consists of a neural network IC on a motherboard, a circuit card assembly, and a set of software routines hosted by a PC-class computer. Research in support of this consists of neural network simulations fusing 4 to 7 bands of Landsat imagery and fusing (separately) multiple bands of synthetic imagery. The simulations, results, and a description of the prototype system are presented.

Kagel, J. H.↗

Digital Twin for 3D Printing on CNC Machines

A system has been developed for 3D printing by means of arc welding in CNC machines. Modules for high-speed computation and for the training of neural networks (with feedback) may be connected to this 3D printing system so as to optimize the welding process and improve product quality. A digital twin expands the capabilities of CAM systems in preparing control programs and equipment for CNC machines.

3D PRINTING↗

Neural networks for parameter estimation in intractable models

The goal is to use deep learning models to estimate parameters in statistical models when standard likelihood estimation methods are computationally infeasible. For instance, inference for max-stable processes is exceptionally challenging even with small datasets, but simulation is straightforward. Data from model simulations are used to train deep neural networks and learn statistical parameters from max-stable models. The proposed neural network-based method provides a competitive alternative to current approaches, as demonstrated by considerable accuracy and computational time improvements. Finally, it serves as a proof of concept for deep learning in statistical parameter estimation and can be extended to other estimation problems.

97 MATHEMATICS AND COMPUTING↗

Efficient human activity recognition with spatio-temporal spiking neural networks

In this study, we explore Human Activity Recognition (HAR), a task that aims to predict individuals' daily activities utilizing time series data obtained from wearable sensors for health-related applications. Although recent research has predominantly employed end-to-end Artificial Neural Networks (ANNs) for feature extraction and classification in HAR, these approaches impose a substantial computational load on wearable devices and exhibit limitations in temporal feature extraction due to their activation functions. To address these challenges, we propose the application of Spiking Neural Networks (SNNs), an architecture inspired by the characteristics of biological neurons, to HAR tasks. SNNs accumulate input activation as presynaptic potential charges and generate a binary spike upon surpassing a predetermined threshold. This unique property facilitates spatio-temporal feature extraction and confers the advantage of low-power computation attributable to binary spikes. We conduct rigorous experiments on three distinct HAR datasets using SNNs, demonstrating that our approach attains competitive or superior performance relative to ANNs, while concurrently reducing energy consumption by up to 94%.

60 APPLIED LIFE SCIENCES↗