Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Neural Network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 757 records · Page 42

Neural Network for Coated/Uncoated Hydrosol Particles

Why Neural Network - Accurately reproduce inherent optical properties (IOPs) for coated/uncoated hydrosol particles while reducing required disk space by 100x - Provide IOP properties at least 100x faster than the advanced look-up-table

Grant Sims↗

Anomaly detection with spiking neural networks

Detecting anomalies with a spiking neural network is provided. An input layer receives a number of inputs and converts them into phase-coded spikes, wherein each input is contained within a number of progressively larger neighborhoods of surrounding inputs. From the phase-coded spikes, a median value of each input is computed for each size neighborhood. An absolute difference of each input from its median value is computed for each size neighborhood. A median absolute difference (MAD) of each input is computed for each size neighborhood. For each input, an adaptive median filter (AMF) determines if a MAD for any size neighborhood exceeds a respective threshold. If one or more neighborhoods exceeds its threshold, the AMF outputs the median value of the input for the smallest neighborhood. If none of the neighborhoods exceeds the threshold, the AMF outputs the original value of the input.

Verzi, Stephen Joseph↗

Physics-informed heterogeneous graph neural networks for DC blocker placement

The threat of geomagnetic disturbances (GMDs) to the reliable operation of the bulk energy system has spurred the development of effective strategies for mitigating their impacts. One such approach involves placing transformer neutral blocking devices, which interrupt the path of geomagnetically induced currents (GICs) to limit their impact. The high cost of these devices and the sparsity of transformers that experience high GICs during GMD events, however, calls for a sparse placement strategy that involves high computational cost. To address this challenge, we developed a physics-informed heterogeneous graph neural network (PIHGNN) for solving the graph-based dc-blocker placement problem. Our approach combines a heterogeneous graph neural network (HGNN) with a physics-informed neural network (PINN) to capture the diverse types of nodes and edges in ac/dc networks and incorporates the physical laws of the power grid. We train the PIHGNN model using a surrogate power flow model and validate it using case studies. Results demonstrate that PIHGNN can effectively and efficiently support the deployment of GIC dc-current blockers, ensuring the continued supply of electricity to meet societal demands. Furthermore, our approach has the potential to contribute to the development of more reliable and resilient power grids capable of withstanding the growing threat that GMDs pose.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Simulation of Foam Divot Weight on External Tank Utilizing Least Squares and Neural Network Methods

Simulation of divot weight in the insulating foam, associated with the external tank of the U.S. space shuttle, has been evaluated using least squares and neural network concepts. The simulation required models based on fundamental considerations that can be used to predict under what conditions voids form, the size of the voids, and subsequent divot ejection mechanisms. The quadratic neural networks were found to be satisfactory for the simulation of foam divot weight in various tests associated with the external tank. Both linear least squares method and the nonlinear neural network predicted identical results.

Chamis, Christos C.↗

Bayesian Entropy Neural Networks for physics-aware prediction

This article addresses the need for deep learning models to integrate well-defined constraints into their outputs, driven by their application in surrogate models, learning with limited data and partial information, and scenarios requiring flexible model behavior to incorporate non-data sample information. We introduce Bayesian Entropy Neural Networks (BENN), a framework grounded in Maximum Entropy (MaxEnt) principles, designed to impose constraints on Bayesian Neural Network (BNN) predictions. BENN is capable of constraining not only the predicted values but also their derivatives and variances, ensuring a more robust and reliable model output. To achieve simultaneous uncertainty quantification and constraint satisfaction, we employ the method of multipliers approach. This allows for the concurrent estimation of neural network parameters and the Lagrangian multipliers associated with the constraints. Our experiments, spanning diverse applications such as beam deflection modeling and microstructure generation, demonstrate the effectiveness of BENN. The results highlight significant improvements over traditional BNNs and showcase competitive performance relative to contemporary constrained deep learning methods.

14 SOLAR ENERGY↗

Application of Neural Networks to Wind tunnel Data Response Surface Methods

The integration of nonlinear neural network methods with conventional linear regression techniques is demonstrated for representative wind tunnel force balance data modeling. This work was motivated by a desire to formulate precision intervals for response surfaces produced by neural networks. Applications are demonstrated for representative wind tunnel data acquired at NASA Langley Research Center and the Arnold Engineering Development Center in Tullahoma, TN.

Lo, Ching F.↗

Uncertainties in Parameters Estimated with Neural Networks: Application to Strong Gravitational Lensing

In Hezaveh et al. (2017) we showed that deep learning can be used for model parameter estimation and trained convolutional neural networks to determine the parameters of strong gravitational lensing systems. Here we demonstrate a method for obtaining the uncertainties of these parameters. We review the framework of variational inference to obtain approximate posteriors of Bayesian neural networks and apply it to a network trained to estimate the parameters of the Singular Isothermal Ellipsoid plus external shear and total flux magnification. We show that the method can capture the uncertainties due to different levels of noise in the input data, as well as training and architecture-related errors made by the network. To evaluate the accuracy of the resulting uncertainties, we calculate the coverage probabilities of marginalized distributions for each lensing parameter. By tuning a single hyperparameter, the dropout rate, we obtain coverage probabilities approximately equal to the confidence levels for which they were calculated, resulting in accurate and precise uncertainty estimates. Our results suggest that neural networks can be a fast alternative to Monte Carlo Markov Chains for parameter uncertainty estimation in many practical applications, allowing more than seven orders of magnitude improvement in speed.

Perreault Levasseur, Laurence↗

Short-Term Forecasting of Thermostatic and Residential Loads Using Long Short-Term Memory Recurrent Neural Networks

Internet of Things (IoT) devices in smart grids enable intelligent energy management for grid managers and personalized energy services for consumers. Investigating a smart grid with IoT devices requires a simulation framework with IoT devices modeling. However, there lack comprehensive study on the modeling of IoT devices in smart grids. This paper investigates the IoT device modeling of a thermostatic load and implements the recurrent neural networks model for short-term load forecasting in this IoT-based thermostatic load. The recurrent neural network structure is leveraged to build a load forecasting model on temporal correlation. The temporal recurrent neural network layers including long short-term memory cells are employed to learn the data from both the simulation platform and New South Wales residential datasets. The simulation results are provided for demonstration.

electric load forecasting↗

A physics-constrained neural network for multiphase flows

The present study develops a physics-constrained neural network (PCNN) to predict sequential patterns and motions of multiphase flows (MPFs), which includes strong interactions among various fluid phases. To predict the order parameters, which locate individual phases in the future time, a neural network (NN) is applied to quickly infer the dynamics of the phases by encoding observations. The multiphase consistent and conservative boundedness mapping algorithm (MCBOM) is next implemented to correct the predicted order parameters. This enforces the predicted order parameters to strictly satisfy the mass conservation, the summation of the volume fractions of the phases to be unity, the consistency of reduction, and the boundedness of the order parameters. Then, the density of the fluid mixture is updated from the corrected order parameters. Finally, the velocity in the future time is predicted by another NN with the same network structure, but the conservation of momentum is included in the loss function to shrink the parameter space. The proposed PCNN for MPFs sequentially performs (NN)-(MCBOM)-(NN), which avoids nonphysical behaviors of the order parameters, accelerates the convergence, and requires fewer data to make predictions. Numerical experiments demonstrate that the proposed PCNN is capable of predicting MPFs effectively.

Mechanics↗

Auto-associative nanoelectronic neural network

In this paper, an auto-associative neural network using single-electron tunneling (SET) devices is proposed and simulated at low temperature. The nanoelectronic auto-associative network is able to converge to a stable state, previously stored during training. The recognition of the pattern involves decreasing the energy of the input state until it achieves a point of local minimum energy, which corresponds to one of the stored patterns.

77 NANOSCIENCE AND NANOTECHNOLOGY↗

Artificial neural network modeling of the spontaneous combustion occurring in the industrial-scale coal stockpiles with 10-18 mm coal grain sizes

Companies consuming large amounts of coal should work with coal stocks in order to not face problems due to production delays. The industrial-scale stockpiles formed for the aforementioned reasons cause environmental problems and economic losses for the companies. This study was performed in a coal stock area of a large company in Konya, which uses large amounts of coal in its manufacturing units. The coal stockpile with 5 m width, 10 m length, 3 m height, and having 120 tons of weight was formed in the coal stock area of the company. The inner temperature data of the stockpile was recorded by 17 temperature sensors placed inside the stockpile at certain points. In order to achieve this goal, the electrical signal conversion of temperatures sensed by 17 temperature sensors placed in certain points inside the coal stockpile, the transfer of these electrical signals into computer media by using analog-digital conversion unit after applying necessary filtration and upgrading processes, and the record of these information into a database in particular time intervals are provided. Additionally, the data relating to the air temperature, air humidity, atmospheric pressure, wind velocity, and wind direction that are the parameters affecting the coal stockpile were also recorded. Afterwards, these measurement values were used for training and testing of an artificial neural network model. Comparison of the experimental and artificial neural network results, accuracy rates of training and testing were found to be 99.5% and 99.17%, respectively. It is shown that possible coal stockpile behavior with this artificial neural network model is powerfully estimated.

01 COAL, LIGNITE, AND PEAT↗

Method for Constructing Composite Response Surfaces by Combining Neural Networks with Polynominal Interpolation or Estimation Techniques

A method and system for data modeling that incorporates the advantages of both traditional response surface methodology (RSM) and neural networks is disclosed. The invention partitions the parameters into a first set of s simple parameters, where observable data are expressible as low order polynomials, and c complex parameters that reflect more complicated variation of the observed data. Variation of the data with the simple parameters is modeled using polynomials; and variation of the data with the complex parameters at each vertex is analyzed using a neural network. Variations with the simple parameters and with the complex parameters are expressed using a first sequence of shape functions and a second sequence of neural network functions. The first and second sequences are multiplicatively combined to form a composite response surface, dependent upon the parameter values, that can be used to identify an accurate mode

Rai, Man Mohan↗

Learning in Neural Networks: VLSI Implementation Strategies

Fully-parallel hardware neural network implementations may be applied to high-speed recognition, classification, and mapping tasks in areas such as vision, or can be used as low-cost self-contained units for tasks such as error detection in mechanical systems (e.g. autos). Learning is required not only to satisfy application requirements, but also to overcome hardware-imposed limitations such as reduced dynamic range of connections.

Neural Networks↗

Transformer Neural Networks with Spatiotemporal Attention for Predictive Control and Optimization of Industrial Processes

In the context of real-time optimization and model predictive control of industrial systems, machine learning, and neural networks represent cutting-edge tools that hold promise for enhancing dynamic modeling. This work presents a novel transformer neural network architecture for real-time optimization and model predictive control. This network design includes a modified attention mechanism inspired by positional embedding attention from vision transformers and task-specific modifications to the input-output structure of the transformer’s decoder stack. Experiments were conducted using data from a 450 MW coal-fired power plant to evaluate this approach's effectiveness. The transformer neural network was compared with conventional recurrent models, including GRU and LSTM. The transformer exhibited a 6% increase in the R-squared (R2) value of predictions and an 83% reduction in mean squared error (MSE). Computation time was also reduced by 84% compared to conventional recurrent models.

Gallup, Ethan R.↗

Nutrient Stress Detection in Corn Using Neural Networks and AVIRIS Hyperspectral Imagery

AVIRIS image cube data has been processed for the detection of nutrient stress in corn by both known, ratio-type algorithms and by trained neural networks. The USDA Shelton, NE, ARS Variable Rate Nitrogen Application (VRAT) experimental farm was the site used in the study. Upon application of ANOVA and Dunnett multiple comparsion tests on the outcome of both the neural network processing and the ratio-type algorithm results, it was found that the neural network methodology provides a better overall capability to separate nutrient stressed crops from in-field controls.

Estep, Lee↗

A neural network for controlling the configuration of frame structure with elastic members

A neural network for controlling the configuration of frame structure with elastic members is proposed. In the present network, the structure is modeled not by using the relative angles of the members but by using the distances between the joint locations alone. The relationship between the environment and the joints is also defined by their mutual distances. The analog neural network attains the reaching motion of the manipulator as a minimization problem of the energy constructed by the distances between the joints, the target, and the obstacles. The network can generate not only the final but also the transient configurations and the trajectory. This framework with flexibility and parallelism is very suitable for controlling the Space Telerobotic systems with many degrees of freedom.

Tsutsumi, Kazuyoshi↗

A Parallel Alternative for Energy-Efficient Neural Network Training and Inferencing

Energy efficiency of training and inferencing with large neural network models is a critical challenge facing the future of sustainable large-scale machine learning workloads. This paper introduces an alternative strategy, called phantom parallelism, to minimize the net energy consumption of traditional tensor (model) parallelism, the most energy-inefficient component of large neural network training. The approach is presented in the context of feed-forward network architectures as a preliminary, but comprehensive, proof-of-principle study of the proposed methodology. We derive new forward and backward propagation operators for phantom parallelism, implement them as custom autograd operations within an end-to-end phantom parallel training pipeline and compare its parallel performance and energy-efficiency against those of conventional tensor parallel training pipelines. Formal analyses that predict lower bandwidth and FLOP counts are presented with supporting empirical results on up to 256 GPUs that corroborate these gains. Experiments are shown to deliver ∼50% reduction in the energy consumed to train FFNs using the proposed phantom parallel approach when compared with conventional tensor parallel methods. Additionally, the proposed approach is shown to train smaller phantom models to the same model loss on smaller GPU counts as larger tensor parallel models on larger GPU counts offering the possibility for even greater energy savings.

Seal, Sudip [ORNL] (ORCID:0000000332330656)↗

Trainability of Dissipative Perceptron-Based Quantum Neural Networks

Several architectures have been proposed for quantum neural networks (QNNs), with the goal of efficiently performing machine learning tasks on quantum data. Rigorous scaling results are urgently needed for specific QNN constructions to understand which, if any, will be trainable at a large scale. Here, we analyze the gradient scaling (and hence the trainability) for a recently proposed architecture that we call dissipative QNNs (DQNNs), where the input qubits of each layer are discarded at the layer’s output. We find that DQNNs can exhibit barren plateaus, i.e., gradients that vanish exponentially in the number of qubits. Moreover, we provide quantitative bounds on the scaling of the gradient for DQNNs under different conditions, such as different cost functions and circuit depths, and show that trainability is not always guaranteed. Here our work represents the first rigorous analysis of the scalability of a perceptron-based QNN.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗