Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Bayesian neural networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2019

An Ensemble of Bayesian Neural Networks for Exoplanetary Atmospheric Retrieval

Machine learning (ML) is now used in many areas of astrophysics, from detecting exoplanets in Kepler transit signals to removing telescope systematics. Recent work demonstrated the potential of using ML algorithms for atmospheric retrieval by implementing a random forest (RF) to perform retrievals in seconds that are consistent with the traditional, computationally expensive nested-sampling retrieval method. We expand upon their approach by presenting a new ML model, plan-net, based on an ensemble of Bayesian neural networks (BNNs) that yields more accurate inferences than the RF for the same data set of synthetic transmission spectra. We demonstrate that an ensemble provides greater accuracy and more robust uncertainties than a single model. In addition to being the first to use BNNs for atmospheric retrieval, we also introduce a new loss function for BNNs that learns correlations between the model outputs. Importantly, we show that designing ML models to explicitly incorporate domain-specific knowledge both improves performance and provides additional insight by inferring the covariance of the retrieved atmospheric parameters. We apply plan-net to the Hubble Space Telescope Wide Field Camera 3 transmission spectrum for WASP-12b and retrieve an isothermal temperature and water abundance consistent with the literature. We highlight that our method is flexible and can be expanded to higher resolution spectra and a larger number of atmospheric parameters.

Adam D. Cobb↗
NASA NTRS2020

Genetic Algorithm for Optimization of Neural Networks for Bayesian Inference of Model Uncertainty

The objective of this work was to develop a genetic optimization algorithm that can design a neural network capable of producing uncertainty estimates along with predictions. This algorithm is necessary because the inclusion of uncertainty modeling in a neural network greatly complicates the network’s design space, making the development of a converging model extremely difficult and time consuming. The genetic algorithm presented in this work uses a number of value ranges for various configurable neural network parameters to create a randomly generated population of network architectures. The initially generated population is then evolved over the course of several generations, with the best performing models breeding to produce novel network configurations. Mutations are randomly applied to the network designs to facilitate the development of adaptations beneficial to the task being performed. An experiment was conducted to validate the proposed algorithm, in which the genetic optimizer was tasked with producing a neural network capable of predicting the sound pressure level (SPL) resulting from jet-surface interaction (JSI) noise. The data used for this task was generated at the NASA Glenn Research Center in the Aero-Acoustic Propulsion Laboratory. Starting with an initial population size of 35 randomly generated networks, and evolved over the course of 10 generations, the genetic algorithm produced a design able to predict SPL as a result of JSI noise within 0.272 dB, on average.

Genetic algorithm↗
NASA NTRS2024

Effective structural impact detection and localization using convolutional neural network and Bayesian information fusion with limited sensors

Due to their unpredictable nature, many impact events (e.g., overheight vehicles striking on bridges) go unnoticed or get reported many hours later. However, they can induce structural failures or hidden damage that accelerates the structure’s long-term degradation. Therefore, prompt impact detection and localization strategies are essential for early warning of impact events and rapid maintenance of structures. Most existing impact detection strategies are developed for aircraft composite panels utilizing high-rate synchronized measurement from densely deployed sensors. Limited efforts have been made for infrastructure or human habitats which generally require large-scale but low-rate measurement. In particular, due to harsh environments (e.g., deep space habitats under meteoroids), structural impact localization must be robust to limited sensors (e.g., sensor damage during impacts) and multi-source errors (e.g., measurement errors). In this study, an effective impact detection and localization strategy is proposed using a limited number of vibration measurements, especially in harsh environments (e.g. in deep space). Convolutional neural networks are trained for each sensor node and are fused using Bayesian theory to improve the accuracy of impact localization. Special considerations are paid to evaluate the effect of both measurement error and modeling error in the analysis. The proposed strategy is illustrated using 1D structure, and further validated in 3D geodesic dome structure numerically. The results demonstrate that it can detect and localize impact events accurately and robustly on structures.

Yuguang Fu↗
NASA NTRSDate not supplied

A Framework for the Analysis of Deep Neural Networks in Autonomous Aerospace Applications using Bayesian Statistics

Deep Neural Networks (DNNs) are considered to be key components in many autonomous systems. Applications range from vision-based obstacle avoidance to intelligent/learning control and planning. Safety-critical applications as found in the aerospace domain require that the behavior of the DNN is validated and tested rigorously for safety of the autonomous system (AUS). In this paper, we present a framework to support testing of DNNs and the analysis of the network structure. Our framework employs techniques from statistical modeling and active learning to effectively generate test cases for DNN safety testing and performance analysis. We will present results of a case study on a physics-based Deep recurrent residual neural network (DR-RNN), which has been trained to emulate the aerodynamics behavior of a fixed-wing aircraft.

Deep Neural networks↗
NASA NTRS2004

Neural network uncertainty assessment using Bayesian statistics: a remote sensing application

Neural network (NN) techniques have proved successful for many regression problems, in particular for remote sensing; however, uncertainty estimates are rarely provided. In this article, a Bayesian technique to evaluate uncertainties of the NN parameters (i.e., synaptic weights) is first presented. In contrast to more traditional approaches based on point estimation of the NN weights, we assess uncertainties on such estimates to monitor the robustness of the NN model. These theoretical developments are illustrated by applying them to the problem of retrieving surface skin temperature, microwave surface emissivities, and integrated water vapor content from a combined analysis of satellite microwave and infrared observations over land. The weight uncertainty estimates are then used to compute analytically the uncertainties in the network outputs (i.e., error bars and correlation structure of these errors). Such quantities are very important for evaluating any application of an NN model. The uncertainties on the NN Jacobians are then considered in the third part of this article. Used for regression fitting, NN models can be used effectively to represent highly nonlinear, multivariate functions. In this situation, most emphasis is put on estimating the output errors, but almost no attention has been given to errors associated with the internal structure of the regression model. The complex structure of dependency inside the NN is the essence of the model, and assessing its quality, coherency, and physical character makes all the difference between a blackbox model with small output errors and a reliable, robust, and physically coherent model. Such dependency structures are described to the first order by the NN Jacobians: they indicate the sensitivity of one output with respect to the inputs of the model for given input data. We use a Monte Carlo integration procedure to estimate the robustness of the NN Jacobians. A regularization strategy based on principal component analysis is proposed to suppress the multicollinearities in order to make these Jacobians robust and physically meaningful.

Neural Networks (Computer)↗
NASA NTRS2019

Deep Learning and Uncertainty Quantification for Climate Resilience

Modeling and monitoring of earth’s processes through physical models and satellite observations at high resolutions is crucial for ensuring society’s ability to adapt to climate change. Deep learning (DL) has been shown to be a valuable tool for generating high resolution data, emulating physical models, and detecting weather patterns which can then be used to inform stakeholders and decision makers. However, both the data and model parameters contain substantial uncertainties that may alter users’ decisions. In this work we present two DL applications on high-resolution climate and satellite datasets using Bayesian neural networks to generate well calibrated uncertainty estimates.

Vandal, Thomas↗
NASA NTRS1990

Neural network classification - A Bayesian interpretation

The relationship between minimizing a mean squared error and finding the optimal Bayesian classifier is reviewed. This provides a theoretical interpretation for the process by which neural networks are used in classification. A number of confidence measures are proposed to evaluate the performance of the neural network classifier within a statistical framework.

Wan, Eric A.↗
NASA NTRS2011

Markov Chain Monte Carlo Bayesian Learning for Neural Networks

Conventional training methods for neural networks involve starting al a random location in the solution space of the network weights, navigating an error hyper surface to reach a minimum, and sometime stochastic based techniques (e.g., genetic algorithms) to avoid entrapment in a local minimum. It is further typically necessary to preprocess the data (e.g., normalization) to keep the training algorithm on course. Conversely, Bayesian based learning is an epistemological approach concerned with formally updating the plausibility of competing candidate hypotheses thereby obtaining a posterior distribution for the network weights conditioned on the available data and a prior distribution. In this paper, we developed a powerful methodology for estimating the full residual uncertainty in network weights and therefore network predictions by using a modified Jeffery's prior combined with a Metropolis Markov Chain Monte Carlo method.

Goodrich, Michael S.↗
NASA NTRS2004

Fossil Signatures Using Elemental Abundance Distributions and Bayesian Probabilistic Classification

Elemental abundances (C6, N7, O8, Na11, Mg12, Al3, P15, S16, Cl17, K19, Ca20, Ti22, Mn25, Fe26, and Ni28) were obtained for a set of terrestrial fossils and the rock matrix surrounding them. Principal Component Analysis extracted five factors accounting for the 92.5% of the data variance, i.e. information content, of the elemental abundance data. Hierarchical Cluster Analysis provided unsupervised sample classification distinguishing fossil from matrix samples on the basis of either raw abundances or PCA input that agreed strongly with visual classification. A stochastic, non-linear Artificial Neural Network produced a Bayesian probability of correct sample classification. The results provide a quantitative probabilistic methodology for discriminating terrestrial fossils from the surrounding rock matrix using chemical information. To demonstrate the applicability of these techniques to the assessment of meteoritic samples or in situ extraterrestrial exploration, we present preliminary data on samples of the Orgueil meteorite. In both systems an elemental signature produces target classification decisions remarkably consistent with morphological classification by a human expert using only structural (visual) information. We discuss the possibility of implementing a complexity analysis metric capable of automating certain image analysis and pattern recognition abilities of the human eye using low magnification optical microscopy images and discuss the extension of this technique across multiple scales.

Hoover, Richard B.↗
NASA NTRS2005

Probabilisitc Geobiological Classification Using Elemental Abundance Distributions and Lossless Image Compression in Recent and Modern Organisms

Last year we presented techniques for the detection of fossils during robotic missions to Mars using both structural and chemical signatures[Storrie-Lombardi and Hoover, 2004]. Analyses included lossless compression of photographic images to estimate the relative complexity of a putative fossil compared to the rock matrix [Corsetti and Storrie-Lombardi, 2003] and elemental abundance distributions to provide mineralogical classification of the rock matrix [Storrie-Lombardi and Fisk, 2004]. We presented a classification strategy employing two exploratory classification algorithms (Principal Component Analysis and Hierarchical Cluster Analysis) and non-linear stochastic neural network to produce a Bayesian estimate of classification accuracy. We now present an extension of our previous experiments exploring putative fossil forms morphologically resembling cyanobacteria discovered in the Orgueil meteorite. Elemental abundances (C6, N7, O8, Na11, Mg12, Ai13, Si14, P15, S16, Cl17, K19, Ca20, Fe26) obtained for both extant cyanobacteria and fossil trilobites produce signatures readily distinguishing them from meteorite targets. When compared to elemental abundance signatures for extant cyanobacteria Orgueil structures exhibit decreased abundances for C6, N7, Na11, All3, P15, Cl17, K19, Ca20 and increases in Mg12, S16, Fe26. Diatoms and silicified portions of cyanobacterial sheaths exhibiting high levels of silicon and correspondingly low levels of carbon cluster more closely with terrestrial fossils than with extant cyanobacteria. Compression indices verify that variations in random and redundant textural patterns between perceived forms and the background matrix contribute significantly to morphological visual identification. The results provide a quantitative probabilistic methodology for discriminating putatitive fossils from the surrounding rock matrix and &om extant organisms using both structural and chemical information. The techniques described appear applicable to the geobiological analysis of meteoritic samples or in situ exploration of the Mars regolith. Keywords: cyanobacteria, microfossils, Mars, elemental abundances, complexity analysis, multifactor analysis, principal component analysis, hierarchical cluster analysis, artificial neural networks, paleo-biosignatures

Storrie-Lombardi, Michael C.↗
NASA NTRS2003

Use of Soft Computing Technologies For Rocket Engine Control

The problem to be addressed in this paper is to explore how the use of Soft Computing Technologies (SCT) could be employed to further improve overall engine system reliability and performance. Specifically, this will be presented by enhancing rocket engine control and engine health management (EHM) using SCT coupled with conventional control technologies, and sound software engineering practices used in Marshall s Flight Software Group. The principle goals are to improve software management, software development time and maintenance, processor execution, fault tolerance and mitigation, and nonlinear control in power level transitions. The intent is not to discuss any shortcomings of existing engine control and EHM methodologies, but to provide alternative design choices for control, EHM, implementation, performance, and sustaining engineering. The approaches outlined in this paper will require knowledge in the fields of rocket engine propulsion, software engineering for embedded systems, and soft computing technologies (i.e., neural networks, fuzzy logic, and Bayesian belief networks), much of which is presented in this paper. The first targeted demonstration rocket engine platform is the MC-1 (formerly FASTRAC Engine) which is simulated with hardware and software in the Marshall Avionics & Software Testbed laboratory that

Trevino, Luis C.↗
NASA NTRS2001

Use of Soft Computing Technologies for a Qualitative and Reliable Engine Control System for Propulsion Systems

The problem to be addressed in this paper is to explore how the use of Soft Computing Technologies (SCT) could be employed to improve overall vehicle system safety, reliability, and rocket engine performance by development of a qualitative and reliable engine control system (QRECS). Specifically, this will be addressed by enhancing rocket engine control using SCT, innovative data mining tools, and sound software engineering practices used in Marshall's Flight Software Group (FSG). The principle goals for addressing the issue of quality are to improve software management, software development time, software maintenance, processor execution, fault tolerance and mitigation, and nonlinear control in power level transitions. The intent is not to discuss any shortcomings of existing engine control methodologies, but to provide alternative design choices for control, implementation, performance, and sustaining engineering, all relative to addressing the issue of reliability. The approaches outlined in this paper will require knowledge in the fields of rocket engine propulsion (system level), software engineering for embedded flight software systems, and soft computing technologies (i.e., neural networks, fuzzy logic, data mining, and Bayesian belief networks); some of which are briefed in this paper. For this effort, the targeted demonstration rocket engine testbed is the MC-1 engine (formerly FASTRAC) which is simulated with hardware and software in the Marshall Avionics & Software Testbed (MAST) laboratory that currently resides at NASA's Marshall Space Flight Center, building 4476, and is managed by the Avionics Department. A brief plan of action for design, development, implementation, and testing a Phase One effort for QRECS is given, along with expected results. Phase One will focus on development of a Smart Start Engine Module and a Mainstage Engine Module for proper engine start and mainstage engine operations. The overall intent is to demonstrate that by employing soft computing technologies, the quality and reliability of the overall scheme to engine controller development is further improved and vehicle safety is further insured. The final product that this paper proposes is an approach to development of an alternative low cost engine controller that would be capable of performing in unique vision spacecraft vehicles requiring low cost advanced avionics architectures for autonomous operations from engine pre-start to engine shutdown.

Trevino, Luis↗
NASA NTRS2004

Support vector machines

Support Vector Machines (SVMs) are a type of supervised learning algorith,, other examples of which are Artificial Neural Networks (ANNs), Decision Trees, and Naive Bayesian Classifiers. Supervised learning algorithms are used to classify objects labled by a 'supervisor' - typically a human 'expert.'.

multiangle↗
NASA NTRS2006

Probabilistic Classification Using Elemental Abundance Distributions and Lossless Image Compression in Apollo 17 Lunar Dust Samples from Mare Serenitatis

We have previously outlined a strategy for the detection of fossils [Storrie-Lombardi and Hoover, 2004] and extant microbial life [Storrie-Lombaudi and Hoover, 20051 during robotic missions to Mars using co-registered structural and chemical signatures. Data inputs included image lossless compression indices to estimate relative textural complexity and elemental abundance distributions. Two exploratory classification algorithms (principal component analysis and hierarchical cluster analysis) provide an initial tentative classification of all targets. Nonlinear stochastic neural networks are then trained to produce a Bayesian estimate of algorithm classification accuracy. The strategy previously has been successful in distinguishing regions of biotic and abiotic alteration of basalt glass from unaltered samples. [Storrie-Lombardi and Fisk, 2004; Storrie-Lombardi and Fisk, 2004] Such investigations of abiotic versus biotic alteration of terrestrial mineralogy on Earth are compromised by .the difficulty finding mineralogy completely unaffected by the ubiquitous presence of microbial life on the planet. The renewed interest in lunar exploration offers an opportunity to investigate geological materials that may exhibit signs of aqueous alteration, but are highly unlikely to contain contaminating biological weathering signatures. We here present an extension of our earlier data set to include lunar dust samples obtained during the Apollo 17 mission. Apollo 17 landed in the Taurus-Littrow Valley in Mare Serenitatis. Most of the rock samples from this region of the lunar highlands are basalts comprised primarily of plagioclase and pyroxene and selected examples of orange and black volcanic glass. SEM images and elemental abundances (C6, N7, O8, Na11, Mg12, Al13, Si14, P15, S16, Cll7, K19, Ca20, Fe26) for a series of targets in the lunar dust samples are compared to the extant cyanobacteria, fossil trilobites, Orgueil meteorite, and terrestrial basalt targets previously discussed. The data set provides a first step in producing a quantitative probabilistic methodology for geobiological analysis of returned lunar samples or in situ exploration.

Storrie-Lombardi, Michael C.↗
NASA NTRS1998

A Conceptual Approach to Assimilating Remote Sensing Data to Improve Soil Moisture Profile Estimates in a Surface Flux/Hydrology Model: Overview - Part 1

Knowledge of the amount of water in the soil is of great importance to many earth science disciplines. Soil moisture is a key variable in controlling the exchange of water and energy between the land surface and the atmosphere. Thus, soil moisture information is valuable in a wide range of applications including weather and climate, runoff potential and flood control, early warning of droughts, irrigation, crop yield forecasting, soil erosion, reservoir management, geotechnical engineering, and water quality. Despite the importance of soil moisture information, widespread and continuous measurements of soil moisture are not possible today. Although many earth surface conditions can be measured from satellites, we still cannot adequately measure soil moisture from space. Research in soil moisture remote sensing began in the mid 1970s shortly after the surge in satellite development. Recent advances in remote sensing have shown that soil moisture can be measured, at least qualitatively, by several methods. Quantitative measurements of moisture in the soil surface layer have been most successful using both passive and active microwave remote sensing, although complications arise from surface roughness and vegetation type and density. Early attempts to measure soil moisture from space-borne microwave instruments were hindered by what is now considered sub-optimal wavelengths (shorter than 5 cm) and the coarse spatial resolution of the measurements. L-band frequencies between 1 and 3 GHz (10-30 cm) have been deemed optimal for detection of soil moisture in the upper few centimeters of soil. The Electronically Steered Thinned Array Radiometer (ESTAR), an aircraft-based instrument operating a 1,4 GHz, has shown great promise for soil moisture determination. Initiatives are underway to develop a similar instrument for space. Existing space-borne synthetic aperture radars (SARS) operating at C- and L-band have also shown some potential to detect surface wetness. The advantage of radar is its much higher resolution than passive microwave systems, but it is currently hampered by surface roughness effects and the lack of a good algorithm based on a single frequency and single polarization. In addition, its repeat frequency is generally low (about 40 days). In the meantime, two new radiometers offer some hope for remote sensing of soil moisture from space. The Tropical Rainfall Measuring Mission (TRMM) Microwave Imager (TMI), launched in November 1997, possesses a 10.65 GHz channel and the Advanced Microwave Scanning Radiometer (AMSR) on both the ADEOS-11 and Earth Observing System AM-1 platforms to be launched in 1999 possesses a 6.9 GHz channel. Aside from issues about interference from vegetation, the coarse resolution of these data will provide considerable challenges pertaining to their application. The resolution of TMI is about 45 km and that of AMSR is about 70 km. These resolutions are grossly inconsistent with the scale of soil moisture processes and the spatial variability of factors that control soil moisture. Scale disparities such as these are forcing us to rethink how we assimilate data of various scales in hydrologic models. Of particular interest is how to assimilate soil moisture data by reconciling the scale disparity between what we can expect from present and future remote sensing measurements of soil moisture and modeling soil moisture processes. It is because of this disparity between the resolution of space-based sensors and the scale of data needed for capturing the spatial variability of soil moisture and related properties that remote sensing of soil moisture has not met with more widespread success. Within a single footprint of current sensors at the wavelengths optimal for this application, in most cases there is enormous heterogeneity in soil moisture created by differences in landcover, soils and topography, as well as variability in antecedent precipitation. It is difficult to interpret the meaning of 'mean' soil moisture under such conditions and even more difficult to apply such a value. Because of the non-linear relationships between near-surface soil moisture and other variables of interest, such as surface energy fluxes and runoff, mean soil moisture has little applicability at such large scales. It is for these reasons that the use of remote sensing in conjunction with a hydrologic model appears to be of benefit in capturing the complete spatial and temporal structure of soil moisture. This paper is Part I of a four-part series describing a method for intermittently assimilating remotely-sensed soil moisture information to improve performance of a distributed land surface hydrology model. The method, summarized in section II, involves the following components, each of which is detailed in the indicated section of the paper or subsequent papers in this series: Forward radiative transfer model methods (section II and Part IV); Use of a Kalman filter to assimilate remotely-sensed soil moisture estimates with the model profile (section II and Part IV); Application of a soil hydrology model to capture the continuous evolution of the soil moisture profile within and below the root zone (section III); Statistical aggregation techniques (section IV and Part II); Disaggregation techniques using a neural network approach (section IV and Part III); and Maximum likelihood and Bayesian algorithms for inversely solving for the soil moisture profile in the upper few cm (Part IV).

Crosson, William L.↗
NASA NTRS1994

A Guide to the Literature on Learning Graphical Models

This literature review discusses different methods under the general rubric of learning Bayesian networks from data, and more generally, learning probabilistic graphical models. Because many problems in artificial intelligence, statistics and neural networks can be represented as a probabilistic graphical model, this area provides a unifying perspective on learning. This paper organizes the research in this area along methodological lines of increasing complexity.

Buntine, Wray L.↗