Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Backpropagation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
NASA NTRS1992

Extrapolation limitations of multilayer feedforward neural networks

The limitations of backpropagation used as a function extrapolator were investigated. Four common functions were used to investigate the network's extrapolation capability. The purpose of the experiment was to determine whether neural networks are capable of extrapolation and, if so, to determine the range for which networks can extrapolate. The authors show that neural networks cannot extrapolate and offer an explanation to support this result.

Haley, Pamela J.↗
NASA NTRS1992

Neural network error correction for solving coupled ordinary differential equations

A neural network is presented to learn errors generated by a numerical algorithm for solving coupled nonlinear differential equations. The method is based on using a neural network to correctly learn the error generated by, for example, Runge-Kutta on a model molecular dynamics (MD) problem. The neural network programs used in this study were developed by NASA. Comparisons are made for training the neural network using backpropagation and a new method which was found to converge with fewer iterations. The neural net programs, the MD model and the calculations are discussed.

Shelton, R. O.↗
NASA NTRS1992

LAI inversion from optical reflectance using a neural network trained with a multiple scattering model

The inversion of the leaf area index (LAI) canopy parameter from optical spectral reflectance measurements is obtained using a backpropagation artificial neural network trained using input-output pairs generated by a multiple scattering reflectance model. The problem of LAI estimation over sparse canopies (LAI < 1.0) with varying soil reflectance backgrounds is particularly difficult. Standard multiple regression methods applied to canopies within a single homogeneous soil type yield good results but perform unacceptably when applied across soil boundaries, resulting in absolute percentage errors of >1000 percent for low LAI. Minimization methods applied to merit functions constructed from differences between measured reflectances and predicted reflectances using multiple-scattering models are unacceptably sensitive to a good initial guess for the desired parameter. In contrast, the neural network reported generally yields absolute percentage errors of <30 percent when weighting coefficients trained on one soil type were applied to predicted canopy reflectance at a different soil background.

Smith, James A.↗
NASA NTRS1992

Inversion of surface parameters using fast learning neural networks

A neural network approach to the inversion of surface scattering parameters is presented. Simulated data sets based on a surface scattering model are used so that the data may be viewed as taken from a completely known randomly rough surface. The fast learning (FL) neural network and a multilayer perceptron (MLP) trained with backpropagation learning (BP network) are tested on the simulated backscattering data. The RMS error of training the FL network is found to be less than one half the error of the BP network while requiring one to two orders of magnitude less CPU time. When applied to inversion of parameters from a statistically rough surface, the FL method is successful at recovering the surface permittivity, the surface correlation length, and the RMS surface height in less time and with less error than the BP network. Further applications of the FL neural network to the inversion of parameters from backscatter measurements of an inhomogeneous layer above a half space are shown.

Dawson, M. S.↗
NASA NTRS1992

Sea ice classification using fast learning neural networks

A first learning neural network approach to the classification of sea ice is presented. The fast learning (FL) neural network and a multilayer perceptron (MLP) trained with backpropagation learning (BP network) were tested on simulated data sets based on the known dominant scattering characteristics of the target class. Four classes were used in the data simulation: open water, thick lossy saline ice, thin saline ice, and multiyear ice. The BP network was unable to consistently converge to less than 25 percent error while the FL method yielded an average error of approximately 1 percent on the first iteration of training. The fast learning method presented can significantly reduce the CPU time necessary to train a neural network as well as consistently yield higher classification accuracy than BP networks.

Dawson, M. S.↗
NASA NTRS1992

Inversion of parameters for semiarid regions by a neural network

Microwave brightness temperatures obtained from a passive radiative transfer model are inverted through use of a neural network. The model is applicable to semiarid regions and produces dual-polarized brightness temperatures for 6.6-, 10.7-, and 37-GHz frequencies. A range of temperatures is generated by varying three geophysical parameters over acceptable ranges: soil moisture, vegetation moisture, and soil temperature. A multilayered perceptron (MLP) neural network is trained with a subset of the generated temperatures, and the remaining temperatures are inverted using a backpropagation method. Several synthetic terrains are devised and inverted by the network under local constraints. All the inversions show good agreement with the original geophysical parameters, falling within 5 percent of the actual value of the parameter range.

Zurk, Lisa M.↗
NASA NTRS1993

A review and analysis of neural networks for classification of remotely sensed multispectral imagery

A literature survey and analysis of the use of neural networks for the classification of remotely sensed multispectral imagery is presented. As part of a brief mathematical review, the backpropagation algorithm, which is the most common method of training multi-layer networks, is discussed with an emphasis on its application to pattern recognition. The analysis is divided into five aspects of neural network classification: (1) input data preprocessing, structure, and encoding; (2) output encoding and extraction of classes; (3) network architecture, (4) training algorithms; and (5) comparisons to conventional classifiers. The advantages of the neural network method over traditional classifiers are its non-parametric nature, arbitrary decision boundary capabilities, easy adaptation to different types of data and input structures, fuzzy output values that can enhance classification, and good generalization for use with multiple images. The disadvantages of the method are slow training time, inconsistent results due to random initial weights, and the requirement of obscure initialization values (e.g., learning rate and hidden layer size). Possible techniques for ameliorating these problems are discussed. It is concluded that, although the neural network method has several unique capabilities, it will become a useful tool in remote sensing only if it is made faster, more predictable, and easier to use.

Paola, Justin D.↗
NASA NTRS1991

Training product unit neural networks with genetic algorithms

The training of product neural networks using genetic algorithms is discussed. Two unusual neural network techniques are combined; product units are employed instead of the traditional summing units and genetic algorithms train the network rather than backpropagation. As an example, a neural netork is trained to calculate the optimum width of transistors in a CMOS switch. It is shown how local minima affect the performance of a genetic algorithm, and one method of overcoming this is presented.

Janson, D. J.↗
NASA NTRS1993

Retrieval of ice thickness from polarimetric SAR data

We describe a potential procedure for retrieving ice thickness from multi-frequency polarimetric SAR data for thin ice. This procedure includes first masking out the thicker ice types with a simple classifier and then deriving the thickness of the remaining pixels using a model-inversion technique. The technique used to derive ice thickness from polarimetric observations is provided by a numerical estimator or neural network. A three-layer perceptron implemented with the backpropagation algorithm is used in this investigation with several improved aspects for a faster convergence rate and a better accuracy of the neural network. These improvements include weight initialization, normalization of the output range, the selection of offset constant, and a heuristic learning algorithm. The performance of the neural network is demonstrated by using training data generated by a theoretical scattering model for sea ice matched to the database of interest. The training data are comprised of the polarimetric backscattering coefficients of thin ice and the corresponding input ice parameters to the scattering model. The retrieved ice thickness from the theoretical backscattering coefficients is compare with the input ice thickness to the scattering model to illustrate the accuracy of the inversion method. Results indicate that the network convergence rate and accuracy are higher when multi-frequency training sets are presented. In addition, the dominant backscattering coefficients in retrieving ice thickness are found by comparing the behavior of the network trained backscattering data at various incidence angels. After the neural network is trained with the theoretical backscattering data at various incidence anges, the interconnection weights between nodes are saved and applied to the experimental data to be investigated. In this paper, we illustrate the effectiveness of this technique using polarimetric SAR data collected by the JPL DC-8 radar over a sea ice scene.

Kwok, R.↗
NASA NTRS1994

Neural Network for Positioning Space Station Solar Arrays

As a shuttle approaches the Space Station Freedom for a rendezvous, the shuttle's reaction control jet firings pose a risk of excessive plume impingement loads on Freedom solar arrays. The current solution to this problem, in which the arrays are locked in a feathered position prior to the approach, may be neither accurate nor robust, and is also expensive. An alternative solution is proposed here: the active control of Freedom's beta gimbals during the approach, positioning the arrays dynamically in such a way that they remain feathered relative to the shuttle jet most likely to cause an impingement load. An artificial neural network is proposed as a means of determining the gimbal angles that would drive plume angle of attack to zero. Such a network would be both accurate and robust, and could be less expensive to implement than the current solution. A network was trained via backpropagation, and results, which compare favorably to the current solution as well as to some other alternatives, are presented. Other training options are currently being evaluated.

Graham, Ronald E.↗
NASA NTRS1993

Neural network prediction of aluminum-lithium weld strengths from acoustic emission amplitude data

AE flaw growth activity was monitored in aluminum-lithium weld specimens from the onset of tensile loading to failure. Data on actual ultimate strengths together with AE data from the beginning of loading up to 25 percent of the expected ultimate strength were used to train a backpropagation neural network to predict ultimate strengths. Architecturally, the fully interconnected network consisted of an input layer for the AE amplitude data, a hidden layer to accommodate failure mechanism mapping, and an output layer for ultimate strength prediction. The trained network was then applied to the prediction of ultimate strengths in the remaining six specimens. The worst case prediction error was found to be +2.6 percent.

Hill, Eric V. K.↗
NASA NTRS1994

Learning to train neural networks for real-world control problems

Over the past three years, our group has concentrated on the application of neural network methods to the training of controllers for real-world systems. This presentation describes our approach, surveys what we have found to be important, mentions some contributions to the field, and shows some representative results. Topics discussed include: (1) executing model studies as rehearsal for experimental studies; (2) the importance of correct derivatives; (3) effective training with second-order (DEKF) methods; (4) the efficacy of time-lagged recurrent networks; (5) liberation from the tyranny of the control cycle using asynchronous truncated backpropagation through time; and (6) multistream training for robustness. Results from model studies of automotive idle speed control serve as examples for several of these topics.

Feldkamp, Lee A.↗
NASA NTRS1995

Locomotion training of legged robots using hybrid machine learning techniques

In this study artificial neural networks and fuzzy logic are used to control the jumping behavior of a three-link uniped robot. The biped locomotion control problem is an increment of the uniped locomotion control. Study of legged locomotion dynamics indicates that a hierarchical controller is required to control the behavior of a legged robot. A structured control strategy is suggested which includes navigator, motion planner, biped coordinator and uniped controllers. A three-link uniped robot simulation is developed to be used as the plant. Neurocontrollers were trained both online and offline. In the case of on-line training, a reinforcement learning technique was used to train the neurocontroller to make the robot jump to a specified height. After several hundred iterations of training, the plant output achieved an accuracy of 7.4%. However, when jump distance and body angular momentum were also included in the control objectives, training time became impractically long. In the case of off-line training, a three-layered backpropagation (BP) network was first used with three inputs, three outputs and 15 to 40 hidden nodes. Pre-generated data were presented to the network with a learning rate as low as 0.003 in order to reach convergence. The low learning rate required for convergence resulted in a very slow training process which took weeks to learn 460 examples. After training, performance of the neurocontroller was rather poor. Consequently, the BP network was replaced by a Cerebeller Model Articulation Controller (CMAC) network. Subsequent experiments described in this document show that the CMAC network is more suitable to the solution of uniped locomotion control problems in terms of both learning efficiency and performance. A new approach is introduced in this report, viz., a self-organizing multiagent cerebeller model for fuzzy-neural control of uniped locomotion is suggested to improve training efficiency. This is currently being evaluated for a possible patent by NASA, Johnson Space Center. An alternative modular approach is also developed which uses separate controllers for each stage of the running stride. A self-organizing fuzzy-neural controller controls the height, distance and angular momentum of the stride. A CMAC-based controller controls the movement of the leg from the time the foot leaves the ground to the time of landing. Because the leg joints are controlled at each time step during flight, movement is smooth and obstacles can be avoided. Initial results indicate that this approach can yield fast, accurate results.

Simon, William E.↗
NASA NTRS1993

Conjugate-Gradient Neural Networks in Classification of Multisource and Very-High-Dimensional Remote Sensing Data

Application of neural networks to classification of remote sensing data is discussed. Conventional two-layer backpropagation is found to give good results in classification of remote sensing data but is not efficient in training. A more efficient variant, based on conjugate-gradient optimization, is used for classification of multisource remote sensing and geographic data and very-high-dimensional data. The conjugate-gradient neural networks give excellent performance in classification of multisource data, but do not compare as well with statistical methods in classification of very-high-dimentional data.

Benediktsson, J. A.↗
NASA NTRS1993

Neural Network Prediction of Aluminum-Lithium Weld Strengths from Acoustic Emission Amplitude Data

Acoustic Emission (AE) flaw growth activity was monitored in aluminum-lithium weld specimens from the onset tensile loading to failure. Data on actual ultimate strengths together with AE data from the beginning of loading up to 25 percent of the expected ultimate strength were used to train a backpropagation neural network to predict ultimate strengths. Architecturally, the fully interconnected network consisted of an input layer for the AE amplitude data, a hidden layer to accommodate failure mechanism mapping, and an output layer for ultimate strength prediction. The trained network was the applied to the prediction of ultimate strengths in the remaining six specimens. The worst case prediction error was found to be +2.6 percent.

Hill, Eric v. K.↗
NASA NTRS1998

Feature Extraction of Event-Related Potentials Using Wavelets: An Application to Human Performance Monitoring

This report describes the development and evaluation of mathematical models for predicting human performance from discrete wavelet transforms (DWT) of event-related potentials (ERP) elicited by task-relevant stimuli. The DWT was compared to principal components analysis (PCA) for representation of ERPs in linear regression and neural network models developed to predict a composite measure of human signal detection performance. Linear regression models based on coefficients of the decimated DWT predicted signal detection performance with half as many f ree parameters as comparable models based on PCA scores. In addition, the DWT-based models were more resistant to model degradation due to over-fitting than PCA-based models. Feed-forward neural networks were trained using the backpropagation,-, algorithm to predict signal detection performance based on raw ERPs, PCA scores, or high-power coefficients of the DWT. Neural networks based on high-power DWT coefficients trained with fewer iterations, generalized to new data better, and were more resistant to overfitting than networks based on raw ERPs. Networks based on PCA scores did not generalize to new data as well as either the DWT network or the raw ERP network. The results show that wavelet expansions represent the ERP efficiently and extract behaviorally important features for use in linear regression or neural network models of human performance. The efficiency of the DWT is discussed in terms of its decorrelation and energy compaction properties. In addition, the DWT models provided evidence that a pattern of low-frequency activity (1 to 3.5 Hz) occurring at specific times and scalp locations is a reliable correlate of human signal detection performance.

Trejo, Leonard J.↗
NASA NTRS2004

NASA Experimental Program to Stimulate Competitive Research: South Carolina

The use of an appropriate relationship model is critical for reliable prediction of future urban growth. Identification of proper variables and mathematic functions and determination of the weights or coefficients are the key tasks for building such a model. Although the conventional logistic regression model is appropriate for handing land use problems, it appears insufficient to address the issue of interdependency of the predictor variables. This study used an alternative approach to simulation and modeling urban growth using artificial neural networks. It developed an operational neural network model trained using a robust backpropagation method. The model was applied in the Myrtle Beach region of South Carolina, and tested with both global datasets and areal datasets to examine the strength of both regional models and areal models. The results indicate that the neural network model not only has many theoretic advantages over other conventional mathematic models in representing the complex urban systems, but also is practically superior to the logistic model in its capability to predict urban growth with better - accuracy and less variation. The neural network model is particularly effective in terms of successfully identifying urban patterns in the rural areas where the logistic model often falls short. It was also found from the area-based tests that there are significant intra-regional differentiations in urban growth with different rules and rates. This suggests that the global modeling approach, or one model for the entire region, may not be adequate for simulation of a urban growth at the regional scale. Future research should develop methods for identification and subdivision of these areas and use a set of area-based models to address the issues of multi-centered, intra- regionally differentiated urban growth.

Sutton, Michael A.↗
NASA NTRS1999

Feature extraction of event-related potentials using wavelets: an application to human performance monitoring

This report describes the development and evaluation of mathematical models for predicting human performance from discrete wavelet transforms (DWT) of event-related potentials (ERP) elicited by task-relevant stimuli. The DWT was compared to principal components analysis (PCA) for representation of ERPs in linear regression and neural network models developed to predict a composite measure of human signal detection performance. Linear regression models based on coefficients of the decimated DWT predicted signal detection performance with half as many free parameters as comparable models based on PCA scores. In addition, the DWT-based models were more resistant to model degradation due to over-fitting than PCA-based models. Feed-forward neural networks were trained using the backpropagation algorithm to predict signal detection performance based on raw ERPs, PCA scores, or high-power coefficients of the DWT. Neural networks based on high-power DWT coefficients trained with fewer iterations, generalized to new data better, and were more resistant to overfitting than networks based on raw ERPs. Networks based on PCA scores did not generalize to new data as well as either the DWT network or the raw ERP network. The results show that wavelet expansions represent the ERP efficiently and extract behaviorally important features for use in linear regression or neural network models of human performance. The efficiency of the DWT is discussed in terms of its decorrelation and energy compaction properties. In addition, the DWT models provided evidence that a pattern of low-frequency activity (1 to 3.5 Hz) occurring at specific times and scalp locations is a reliable correlate of human signal detection performance. Copyright 1999 Academic Press.

Evoked Potentials↗