Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Bayesian Neural Network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Bayesian SegNet for Semantic Segmentation with Improved Interpretation of Microstructural Evolution During Irradiation of Materials

Understanding the relationship between the evolution of microstructures of irradiated LiAlO2pellets and tritium diffusion, retention and release could improve predictions of tritium performance. Given expert-labeled segmented images of irradiated and unirradiated pellets, we trained Deep Convolutional Neural Networks to segment images into defect, grain, and boundary classes. Qualitative microstructural information was calculated from these segmented images to facilitate the comparison of unirradiated and irradiated pellets. We tested modifications to improve the sensitivity of the model, including incorporating meta-data into the model and utilizing uncertainty quantification. The predicted segmentation was similar to the expert-labeled segmentation for most methods of microstructural qualification, including pixel proportion, defect area, and defect density. Overall, the high performance metrics for the best models for both irradiated and unirradiated images shows that utilizing neural network models is a viable alternative to expert-labeled images.

Oostrom, Marjolein T.↗
DOE OSTI2024

Power generation forecasting for solar plants based on Dynamic Bayesian networks by fusing multi-source information

A Dynamic Bayesian network (DBN) model for solar power generation forecasting in solar plants is proposed in this paper. The key idea is to fuse sensor data, operational indicators, meteorological data, lagged output power information, and model errors for more accurate short-term (e.g., hours) and mid-term (e.g., days to weeks) power generation forecasting. The proposed DBN augments automated data-driven structure learning with expert knowledge encoding using continuous and categorical data given constraints to represent causal relationships within a solar inverter system. Additionally, an error compensation mechanism is proposed to capture temporal fluctuation. The effectiveness of the DBN on solar power generation forecasting was evaluated by rolling window analysis with one-year testing data collected from a local solar plant. The proposed DBN is compared with four state-of-art methods including support-vector regression (SVR), k-nearest neighbors (kNN), artificial neural network (ANN), and long short-term memory (LSTM) models. The result show that the proposed DBN achieves better accuracy in general, and it is not as data-hungry as some neural network-based models. The proposed DBN is also shown to have robust and consistent forecasting power with different forecasting horizons. The accuracy is 92% - 95% from one hour to one week ahead forecasting.

14 SOLAR ENERGY↗
NASA NTRS2022

Optimal Estimation Framework for Ocean Color Atmospheric Correction and Pixel-level Uncertainty Quantification

Ocean color remote sensing requires compensation for atmospheric scattering and absorption (aerosol, Rayleigh, and trace gases), referred to as atmospheric correction (AC). AC allows inference of parameters such as spectrally resolved remote sensing reflectance ( R rs )(λ) ; sr 1 ) at the ocean surface from the top-of-atmosphere reflectance. Often, the uncertainty of this process is not fully explored. Bayesian inference techniques provide a simultaneous AC and uncertainty assessment via a full posterior distribution of the relevant variables, given the prior distribution of those variables and the radiative transfer (RT) likelihood function. Given uncertainties in the algorithm inputs, the Bayesian framework enables better constraints on the AC process by using the complete spectral information compared to traditional approaches that use only a subset of bands for AC. This paper investigates a Bayesian inference research method (Optimal Estimation, OE) for ocean color AC by simultaneously retrieving atmospheric and ocean properties using all visible and near-infrared spectral bands. The OE algorithm analytically approximates the posterior distribution of parameters based on normality assumptions and provides a potentially viable operational algorithm with a reduced computational expense. We developed a Neural Network (NN) RT forward model look-up-table-based emulator to increase algorithm efficiency further and thus speed up the likelihood computations. We then applied the OE algorithm to synthetic data and observations from the MODerate resolution Imaging Spectroradiometer (MODIS) on NASA’s Aqua spacecraft. We compared the R rs )(λ) retrieval and its uncertainty estimates from the OE method with in-situ validation data from the SeaWiFS Bio-optical Archive and Storage System (SeaBASS) and Aerosol Robotic Network Ocean Color (AERONET-OC) datasets. The OE algorithm improved R rs )(λ) estimates relative to the NASA standard operational algorithm by improving all statistical metrics at 443, 555, and 667 nm. Unphysical negative R rs )(λ) , which often appear in complex water conditions, was reduced by a factor of 3. The OE-derived pixel-level R rs )(λ) uncertainty estimates were also assessed relative to in-situ data and were shown to have skill.

Atmospheric correction↗
DOE OSTI2022

Transfer learning driven design optimization for inertial confinement fusion

Transfer learning is a promising approach to create predictive models that incorporate simulation and experimental data into a common framework. In this technique, a neural network is first trained on a large database of simulations and then partially retrained on sparse sets of experimental data to adjust predictions to be more consistent with reality. Previously, this technique has been used to create predictive models of Omega [Humbird et al., IEEE Trans. Plasma Sci. 48, 61–70 (2019)] and NIF [Humbird et al., Phys. Plasmas 28, 042709 (2021); Kustowski et al., Mach. Learn. 3, 015035 (2022)] inertial confinement fusion (ICF) experiments that are more accurate than simulations alone. Here, in this work, we conduct a transfer learning driven hypothetical ICF campaign in which the goal is to maximize experimental neutron yield via Bayesian optimization. The transfer learning model achieves yields within 5% of the maximum achievable yield in a modest-sized design space in fewer than 20 experiments. Furthermore, we demonstrate that this method is more efficient at optimizing designs than traditional model calibration techniques commonly employed in ICF design. Such an approach to ICF design could enable robust optimization of experimental performance under uncertainty.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Training Spiking Neural Networks with Synaptic Plasticity under Integer Representation

Neuromorphic computing is emerging as a promising Beyond Moore computing paradigm that employs event-triggered computation and non-von Neumann hardware. Spike Timing Dependent Plasticity (STDP) is a well-known bio-inspired learning rule that relies on activities of locally connected neurons to adjust the weights of their respective synapses. In this work, we analyze a basic STDP rule and its sensitivity on the different hyperparameters for training spiking neural networks (SNNs) with supervision, customized for a neuromorphic hardware implementation with integer weights. We compare the classification performance on four UCI datasets (iris, wine, breast cancer and digits) that depict varying levels of complexity. We perform a search for optimal set of hyperparameters using both grid search and Bayesian optimization. Through the use of Bayesian optimization, we show the general trends in hyperparameter sensitivity in SNN classification problem. With the best sets of hyperparameters, we achieve accuracies comparable to some of the best performing SNNs on these four datasets. With a highly optimized supervised STDP rule we show that these accuracies can be achieved with just 20 epochs of training.

Kulkarni, Shruti↗
DOE OSTI2023

Bayesian sequential optimal experimental design for nonlinear models using policy gradient reinforcement learning

We present a mathematical framework and computational methods for optimally designing a finite sequence of experiments. This sequential optimal experimental design (sOED) problem is formulated as a finite-horizon partially observable Markov decision process (POMDP) under a Bayesian setting and with information-theoretic utilities. The formulation is general and may accommodate continuous random variables, non-Gaussian posteriors, and nonlinear forward models. The sOED design policy incorporates elements of feedback and lookahead simultaneously, and we show it to generalize the commonly-used batch and greedy design strategies. We solve for the sOED policy using the policy gradient (PG) method from reinforcement learning, and provide a derivation for the PG expression in the sOED context. Adopting an actor-critic approach, the policy and value functions are parameterized using deep neural networks and improved via PG estimates produced from simulated episodes of designs and observations. The new PG-sOED algorithm is first validated on a linear-Gaussian benchmark, and then compared against other design baselines on a sensor movement problem for contaminant source inversion in a convection-diffusion field. As a result, we provide explanation for the policy behaviors using knowledge of the underlying physical process.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Techno-economic implications and cost of forecasting errors in solar PV power production using optimized deep learning models

Accurate solar Photovoltaic (PV) power forecasting is important for enhancing both the performance and economic feasibility of PV systems. This study evaluates several deep learning models, including Dense Neural Networks (DNN), Long Short-Term Memory (LSTM), Convolutional Neural Networks (CNN), and a hybrid LSTMCNN model, for predicting PV power production one day in advance. Prior to optimization, the models exhibited relatively high errors, with the best model (DNN) achieving a Root Mean Square Error (RMSE) of 31.13 kW and a coefficient of determination (R 2 ) of 62.15 %. After employing Bayesian optimization, the LSTM-CNN model demonstrated the best performance, with the RMSE reduced to 9.79 kW and R 2 improved to 97.62 %, showcasing significant enhancement in predictive accuracy. Here, the economic evaluation considered three cases: rewards for underestimation (0.08 USD/kWh), no rewards, and penalties for both over-and underestimation (120 % of the utility tariff). In the rewards scenario, the LSTM-CNN model reduced the Levelized Cost of Electricity (LCOE) by 4 %, while in the penalty scenario, a backup diesel generator would have increased the LCOE by 49 %. Additionally, the LSTM-CNN model minimized financial losses, achieving the lowest penalties and maximizing net cash flow compared to other models, demonstrating its overall technical and economic superiority.

Deep learning↗
DOE OSTI2023

Image-based solar estimates

An example device is configured to determine, based on a sky image of a portion of sky over a power distribution network and using a convolutional neural network (CNN)-based image regression model, an estimated global horizontal irradiance (GHI) value and manage or control the power distribution network using the estimated GHI value. The device may also be configured to determine, based on GHI values and aggregate load values for at least a portion of the power distribution network, using a Bayesian Structural Time Series model, an estimated photovoltaic power output value for the at least a portion of the power distribution network. The device may manage or control the power distribution network using the estimated photovoltaic power output value.

Bernstein, Andrey↗
DOE OSTI2024

Stochastic machine learning via sigma profiles to build a digital chemical space

This work establishes a different paradigm on digital molecular spaces and their efficient navigation by exploiting sigma profiles. To do so, the remarkable capability of Gaussian processes (GPs), a type of stochastic machine learning model, to correlate and predict physicochemical properties from sigma profiles is demonstrated, outperforming state-of-the-art neural networks previously published. The amount of chemical information encoded in sigma profiles eases the learning burden of machine learning models, permitting the training of GPs on small datasets which, due to their negligible computational cost and ease of implementation, are ideal models to be combined with optimization tools such as gradient search or Bayesian optimization (BO). Gradient search is used to efficiently navigate the sigma profile digital space, quickly converging to local extrema of target physicochemical properties. While this requires the availability of pretrained GP models on existing datasets, such limitations are eliminated with the implementation of BO, which can find global extrema with a limited number of iterations. A remarkable example of this is that of BO toward boiling temperature optimization. Holding no knowledge of chemistry except for the sigma profile and boiling temperature of carbon monoxide (the worst possible initial guess), BO finds the global maximum of the available boiling temperature dataset (over 1,000 molecules encompassing more than 40 families of organic and inorganic compounds) in just 15 iterations (i.e., 15 property measurements), cementing sigma profiles as a powerful digital chemical space for molecular optimization and discovery, particularly when little to no experimental data is initially available.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Deducing neutron star equation of state from telescope spectra with machine-learning-derived likelihoods

The interiors of neutron stars reach densities and temperatures beyond the limits of terrestrial experiments, providing vital laboratories for probing nuclear physics. While the star's interior is not directly observable, its pressure and density determine the star's macroscopic structure which affects the spectra observed in telescopes. The relationship between the observations and the internal state is complex and partially intractable, presenting difficulties for inference. Previous work has focused on the regression from stellar spectra of parameters describing the internal state. We demonstrate a calculation of the full likelihood of the internal state parameters given observations, accomplished by replacing intractable elements with machine learning models trained on samples of simulated stars. Our machine-learning-derived likelihood allows us to perform maximum a posteriori estimation of the parameters of interest, as well as full scans. We demonstrate the technique by inferring stellar mass and radius from an individual stellar spectrum, as well as equation of state parameters from a set of spectra. Our results are more precise than pure regression models, reducing the width of the parameter residuals by 11.8% in the most realistic scenario. The neural networks will be released as a tool for fast simulation of neutron star properties and observed spectra.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

A Hybrid Gradient Method to Designing Bayesian Experiments for Implicit Models

Bayesian experimental design (BED) aims at designing an experiment to maximize the information gathering from the collected data. The optimal design is usually achieved by maximizing the mutual information (MI) between the data and the model parameters. When the analytical expression of the MI is unavailable, e.g.,having implicit models with intractable data distributions, a neural network-based lower bound of the MI was recently proposed and a gradient ascent method was used to maximize the lower bound [1]. However, the approach in [1] requires a pathwise sampling path to compute the gradient of the MI lower bound with respect to the design variables, and such a pathwise sampling path is usually inaccessible for implicit models. In this work, we propose a hybrid gradient approach that leverages recent advances in variational MI estimator and evolution strategies (ES)combined with black-box stochastic gradient ascent (SGA) to maximize the MI lower bound. This allows the design process to be achieved through a unified scalable procedure for implicit models without sampling path gradients. Several experiments demonstrate that our approach significantly improves the scalability of BED for implicit models in high-dimensional design space.

Zhang, Jiaxin↗
DOE OSTI2024

Chaotic neural dynamics facilitate probabilistic computations through sampling

Cortical neurons exhibit highly variable responses over trials and time. Theoretical works posit that this variability arises potentially from chaotic network dynamics of recurrently connected neurons. Here, we demonstrate that chaotic neural dynamics, formed through synaptic learning, allow networks to perform sensory cue integration in a sampling-based implementation. We show that the emergent chaotic dynamics provide neural substrates for generating samples not only of a static variable but also of a dynamical trajectory, where generic recurrent networks acquire these abilities with a biologically plausible learning rule through trial and error. Furthermore, the networks generalize their experience in the stimulus-evoked samples to the inference without partial or all sensory information, which suggests a computational role of spontaneous activity as a representation of the priors as well as a tractable biological computation for marginal distributions. These findings suggest that chaotic neural dynamics may serve for the brain function as a Bayesian generative model.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2023

A flexible event reconstruction based on machine learning and likelihood principles

Event reconstruction is a central step in many particle physics experiments, turning detector observables into parameter estimates; for example estimating the energy of an interaction given the sensor readout of a detector. A corresponding likelihood function is often intractable, and approximations need to be constructed. Here, in our work, we first show how the full likelihood for a many-sensor detector can be broken apart into smaller terms, and secondly how we can train neural networks to approximate all terms solely based on forward simulation. Our technique results in a fast, flexible, and close-to-optimal surrogate model proportional to the likelihood and can be used in conjunction with standard inference techniques allowing for a consistent treatment of uncertainties. We illustrate our technique for parameter inference in neutrino telescopes based on maximum likelihood and Bayesian posterior sampling. Given its great flexibility, we also showcase our method for geometry optimization enabling to learn optimal detector designs. Lastly, we apply our method to realistic simulation of a ton-scale water-based liquid scintillator detector.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
NASA NTRS2015

Modeling the Swift BAT Trigger Algorithm with Machine Learning

To draw inferences about gamma-ray burst (GRB) source populations based on Swift observations, it is essential to understand the detection efficiency of the Swift burst alert telescope (BAT). This study considers the problem of modeling the Swift BAT triggering algorithm for long GRBs, a computationally expensive procedure, and models it using machine learning algorithms. A large sample of simulated GRBs from Lien et al. (2014) is used to train various models: random forests, boosted decision trees (with AdaBoost), support vector machines, and artificial neural networks. The best models have accuracies of approximately greater than 97% (approximately less than 3% error), which is a significant improvement on a cut in GRB flux which has an accuracy of 89:6% (10:4% error). These models are then used to measure the detection efficiency of Swift as a function of redshift z, which is used to perform Bayesian parameter estimation on the GRB rate distribution. We find a local GRB rate density of eta(sub 0) approximately 0.48(+0.41/-0.23) Gpc(exp -3) yr(exp -1) with power-law indices of eta(sub 1) approximately 1.7(+0.6/-0.5) and eta(sub 2) approximately -5.9(+5.7/-0.1) for GRBs above and below a break point of z(sub 1) approximately 6.8(+2.8/-3.2). This methodology is able to improve upon earlier studies by more accurately modeling Swift detection and using this for fully Bayesian model fitting. The code used in this is analysis is publicly available online.

gamma rays: general↗
DOE OSTI2022

Optimization of Water-Alternating-CO2 Injection Field Operations Using a Machine-Learning-Assisted Workflow

Summary This paper will present a robust workflow to address multiobjective optimization (MOO) of carbon dioxide (CO2)-enhanced oil recovery (EOR)-sequestration projects with a large number of operational control parameters. Farnsworth unit (FWU) field, a mature oil reservoir undergoing CO2 alternating water injection (CO2-WAG) EOR, will be used as a field case to validate the proposed optimization protocol. The expected outcome of this work would be a repository of Pareto-optimal solutions of multiple objective functions, including oil recovery, carbon storage volume, and project economics. FWU’s numerical model is used to demonstrate the proposed optimization workflow. Because using MOO requires computationally intensive procedures, machine-learning-based proxies are introduced to substitute for the high-fidelity model, thus reducing the total computation overhead. The vector machine regression combined with the Gaussian kernel (Gaussian-SVR) is used to construct proxies. An iterative self-adjusting process prepares the training knowledge base to develop robust proxies and minimizes computational time. The proxies’ hyperparameters will be optimally designed using Bayesian optimization to achieve better generalization performance. Trained proxies will be coupled with multiobjective particle swarm Optimization (MOPSO) protocol to construct the Pareto-front solution repository. The outcomes of this workflow will be a repository containing Pareto-optimal solutions of multiple objectives considered in the CO2-WAG project. The proposed optimization workflow will be compared with another established methodology using a multilayer neural network (MLNN) to validate its feasibility in handling MOO with a large number of parameters to control. Optimization parameters used include operational variables that might be used to control the CO2-WAG process, such as the duration of the water/gas injection period, producer bottomhole pressure (BHP) control, and water injection rate of each well included in the numerical model. It is proved that the workflow coupling Gaussian-SVR proxies and the iterative self-adjusting protocol is more computationally efficient. The MOO process is made more rapid by squeezing the size of the required training knowledge base while maintaining the high accuracy of the optimized results. The outcomes of the optimization study show promising results in successfully establishing the solution repository considering multiple objective functions. Results are also verified by validating the Pareto fronts with simulation results using obtained optimized control parameters. The outcome from this work could provide field operators an opportunity to design a CO2-WAG project using as many inputs as possible from the reservoir models. The proposed work introduces a novel concept that couples Gaussian-SVR proxies with a self-adjusting protocol to increase the computational efficiency of the proposed workflow and to guarantee the high accuracy of the obtained optimized results. More importantly, the workflow can optimize a large number of control parameters used in a complex CO2-WAG process, which greatly extends its utility in solving large-scale MOO problems in various projects with similar desired outcomes.

Energy & Fuels↗
DOE OSTI2021

Multi-Objective Hyperparameter Optimization for Spiking Neural Network Neuroevolution

Neuroevolution has had significant success over recent years, but there has been relatively little work applying neuroevolution approaches to spiking neural networks (SNNs). SNNs are a type of neural network that includes temporal processing component, are not easily trained using other methods, and can be deployed into energy-efficient neuromorphic hardware. In this work, we investigate two evolutionary approaches for training SNNs. We explore the impact of the hyperparameters of the evolutionary approaches, including tournament size, population size, and representation type, on the performance of the algorithms. We present a multi-objective Bayesian-based hyperparameter optimization approach to tune the hyperparameters to produce the most accurate and smallest SNNs. We show that the hyperparameters can significantly affect the performance of these algorithms. We also perform sensitivity analysis and demonstrate that every hyperparameter value has the potential to perform well, assuming other hyperparameter values are set correctly.

Parsa, Maryam↗
NASA NTRS2016

Modeling the Swift Bat Trigger Algorithm with Machine Learning

To draw inferences about gamma-ray burst (GRB) source populations based on Swift observations, it is essential to understand the detection efficiency of the Swift burst alert telescope (BAT). This study considers the problem of modeling the Swift / BAT triggering algorithm for long GRBs, a computationally expensive procedure, and models it using machine learning algorithms. A large sample of simulated GRBs from Lien et al. is used to train various models: random forests, boosted decision trees (with AdaBoost), support vector machines, and artificial neural networks. The best models have accuracies of greater than or equal to 97 percent (less than or equal to 3 percent error), which is a significant improvement on a cut in GRB flux, which has an accuracy of 89.6 percent (10.4 percent error). These models are then used to measure the detection efficiency of Swift as a function of redshift z, which is used to perform Bayesian parameter estimation on the GRB rate distribution. We find a local GRB rate density of n (sub 0) approaching 0.48 (sup plus 0.41) (sub minus 0.23) per cubic gigaparsecs per year with power-law indices of n (sub 1) approaching 1.7 (sup plus 0.6) (sub minus 0.5) and n (sub 2) approaching minus 5.9 (sup plus 5.7) (sub minus 0.1) for GRBs above and below a break point of z (redshift) (sub 1) approaching 6.8 (sup plus 2.8) (sub minus 3.2). This methodology is able to improve upon earlier studies by more accurately modeling Swift detection and using this for fully Bayesian model fitting.

gamma-ray burst: general – gamma-rays: general â↗
NASA NTRS2023

Predicting Patterns of Solar Energy Buildout to Identify Opportunities for Biodiversity Conservation

The construction of solar energy facilities can have positive or negative impacts on biodiversity depending on siting and associated land use transitions. We identified drivers of solar siting and quantified patterns of buildout in states surrounding the Chesapeake Bay watershed – a biodiversity hotspot with numerous ecosystem services. Using a convolutional neural network, we mapped the footprints of ground-mounted solar arrays present in satellite imagery annually from 2017 to 2021 in Delaware, Maryland, Pennsylvania, New York, Virginia, and West Virginia. As of 2021, we identified 958 solar arrays covering 52.3 km2 built primarily on previously cultivated land, while avoiding natural landcover. We fit a binomial-Weibull model to these solar timeseries data in a hierarchical, Bayesian framework to quantify the relationship between geospatial covariates and rate of solar development. Solar array construction rate increased in cultivated areas, areas of lower agricultural suitability, lower slope, lower forest cover, lower biodiversity protection, and greater distances from roads. We also estimated changes in the rate of solar construction over time and found differences among states: acceleration in Virginia and deceleration in New York. We used parameter estimates to map the relative likelihood of future solar development across the study area. This methodology can be used to anticipate where solar is likely to be built in different landscapes and how these patterns align with conservation goals. Around the Chesapeake Bay watershed, the selection of lower quality agricultural areas for solar energy minimizes removal of important habitat and provides opportunities for native plant and pollinator restoration.

Artificial Intelligence↗