Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “bayesian optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 775 records · Page 43

SPIKE-Dx : A Low-Power High-Throughput Fault Diagnostics Tool using Spiking Neural Networks for Constrained Systems

Diagnostic systems are important for many aerospace systems, which are severely limited in available power, like cubesats or UAVs. Therefore, traditional diagnostics systems cannot be used due to their substantial footprint and constraints. In this paper, we present our very low power diagnostic tool SPIKE-DX to monitor critical systems with constrained computational and energy resources. This is made possible through spiking neural networks (SNNs), which are executable within optimized simulation environments and further implemented on on cutting-edge neuromorphic hardware. Based upon FMEA (Failure Mode and Effect Analysis) framework, Diagnostic Bayesian Networks (DBNs) can be constructed that provide powerful means for diagnostic reasoning. In this paper, we describe such DBNs and a method to automatically translate the DBN into highly structured networks of spiking neurons for execution in SPIKE-DX.

Spiking Neural Networks↗

Variational Bayesian strategies for high-dimensional, stochastic design problems

This paper is concerned with a lesser-studied problem in the context of model-based, uncertainty quantification (UQ), that of optimization/design/control under uncertainty. The solution of such problems is hindered not only by the usual difficulties encountered in UQ tasks (e.g. the high computational cost of each forward simulation, the large number of random variables) but also by the need to solve a nonlinear optimization problem involving large numbers of design variables and potentially constraints. We propose a framework that is suitable for a class of such problems and is based on the idea of recasting them as probabilistic inference tasks. To that end, we propose a Variational Bayesian (VB) formulation and an iterative VB–Expectation-Maximization scheme that is capable of identifying a local maximum as well as a low-dimensional set of directions in the design space, along which, the objective exhibits the largest sensitivity. We demonstrate the validity of the proposed approach in the context of two numerical examples involving thousands of random and design variables. In all cases considered the cost of the computations in terms of calls to the forward model was of the order of 100 or less. The accuracy of the approximations provided is assessed by information-theoretic metrics.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Deep Sigma Point Processes-Assisted Chance-Constrained Power System Transient Stability Preventive Control

Here this paper proposes a deep sigma point processes (DSPP)-assisted chance-constrained power system transient stability preventive control method to deal with uncertain renewable energy and loads-induced stability risk. The traditional transient stability-constrained preventive control is reformulated as a chance-constrained optimization problem. To deal with the computational bottleneck of the time-domain simulation-based probabilistic transient stability assessment, the DSPP is developed. DSPP is a parametric Bayesian approach that allows us to predict system transient stability with high computational efficiency while accurately quantifying the confidence intervals of the predictions that can be used to inform system instability risk. To this end, with a given preset confidence probability, we embed DSPP into the primal dual interior point method to help solve the chance-constrained preventive control problem, where the corresponding Jacobian and Hessian matrices are derived. Comparison results with other existing methods show that the proposed method can significantly speed up preventive control while maintaining high accuracy and convergence

97 MATHEMATICS AND COMPUTING↗

Estimation of hydraulic conductivity in a watershed using sparse multi-source data via Gaussian process regression and Bayesian experimental design

Enhanced water management systems depend on accurate estimation of subsurface hydraulic properties. However, geologic formations can vary significantly, so information from a single source (e.g., widely spaced boreholes) is insufficient in characterizing subsurface aquifer properties. Therefore, multiple sources of information are needed to complement the hydrogeology understanding of a region. Here, this study presents a numerical framework in which information from different measurement sources is combined to characterize the 3D random field in a multi-fidelity prediction model. Coupled with the model, a Bayesian experimental design was used to determine the best future sampling locations. The Upper Sangamon watershed in east-central Illinois was selected as the case study site, where the multi-fidelity Gaussian process model was used to estimate the hydraulic conductivity in the region of interest. Multi-source observation data were obtained from electrical resistivity and borehole pumping tests. The accuracy of the model prediction is dependent on the locations and the distribution of both high- and low-fidelity data. Furthermore, the multi-fidelity model was compared with the single-fidelity model. The uncertainties and confidence in the measurements and parameter estimates were quantified and used to design future cycles of data collection to further improve the confidence intervals.

54 ENVIRONMENTAL SCIENCES↗

Three-dimensional multiphase segmentation of X-ray CT data of porous materials using a Bayesian Markov random field framework

Advancements in noninvasive imaging methods such as X-ray computed tomography (CT) have led to a recent surge of applications in porous media research with objectives ranging from theoretical aspects of pore-scale fluid and interfacial dynamics to practical applications such as enhanced oil recovery and advanced contaminant remediation. While substantial efforts and resources have been devoted to advance CT technology, microscale analysis, and fluid dynamics simulations, the development of efficient and stable three-dimensional multiphase image segmentation methods applicable to large data sets is lacking. To eliminate the need for wet-dry or dual-energy scans, image alignment, and subtraction analysis, commonly applied in X-ray micro-CT, a segmentation method based on a Bayesian Markov random field (MRF) framework amenable to true three-dimensional multiphase processing was developed and evaluated. Furthermore, several heuristic and deterministic combinatorial optimization schemes required to solve the labeling problem of the MRF image model were implemented and tested for computational efficiency and their impact on segmentation results. Test results for three grayscale data sets consisting of dry glass beads, partially saturated glass beads, and partially saturated crushed tuff obtained with synchrotron X-ray micro-CT demonstrate great potential of the MRF image model for three-dimensional multiphase segmentation. While our results are promising and the developed algorithm is stable and computationally more efficient than other commonly applied porous media segmentation models, further potential improvements exist for fully automated operation.

42 ENGINEERING↗

Standing on the shoulders of dwarfs: the kepler asteroseismic legacy sample. I. Oscillation mode parameters

The advent of space-based missions like Kepler has revolutionized the study of solar-type stars, particularly through the measurement and modeling of their resonant modes of oscillation. Here we analyze a sample of 66 Kepler main-sequence stars showing solar-like oscillations as part of the Kepler seismic LEGACY project. We use Kepler short-cadence data, of which each star has at least 12 months, to create frequency-power spectra optimized for asteroseismology. For each star, we identify its modes of oscillation and extract parameters such as frequency, amplitude, and line width using a Bayesian Markov chain Monte Carlo “peak-bagging” approach. We report the extracted mode parameters for all 66 stars, as well as derived quantities such as frequency difference ratios, the large and small separations Δν and δν{sub 02}; the behavior of line widths with frequency and line widths at ν{sub max} with T{sub eff}, for which we derive parametrizations; and behavior of mode visibilities. These average properties can be applied in future peak-bagging exercises to better constrain the parameters of the stellar oscillation spectra. The frequencies and frequency ratios can tightly constrain the fundamental parameters of these solar-type stars, and mode line widths and amplitudes can test models of mode damping and excitation.

79 ASTRONOMY AND ASTROPHYSICS↗

Sensitivity Analysis of Genome-Scale Metabolic Flux Prediction

TRIMER, Transcription Regulation Integrated with MEtabolic Regulation, is a genome-scale modeling pipeline targeting at metabolic engineering applications. Using TRIMER, regulated metabolic reactions can be effectively predicted by integrative modeling of metabolic reactions with a Transcription Factor (TF)-gene regulatory network (TRN), which is modeled via a Bayesian network (BN). In this paper, we focus on sensitivity analysis of metabolic flux prediction for uncertainty quantification of BN structures for TRN modeling in TRIMER. We propose a computational strategy to construct the uncertainty class of TRN models based on the inferred regulatory order uncertainty given transcriptomic expression data. With that, we analyze the prediction sensitivity of the TRIMER pipeline for the metabolite yields of interest. The obtained sensitivity analyses can guide Optimal Experimental Design (OED) to help acquire new data that can enhance TRN modeling and achieve specific metabolic engineering objectives, including metabolite yield alterations. Here we have performed small- and large-scale simulated experiments, demonstrating the effectiveness of our developed sensitivity analysis strategy for BN structure learning to quantify the edge importance in terms of metabolic flux prediction uncertainty reduction and its potential to effectively guide OED.

59 BASIC BIOLOGICAL SCIENCES↗

Stan : A Probabilistic Programming Language

Stan is a probabilistic programming language for specifying statistical models. A Stan program imperatively defines a log probability function over parameters conditioned on specified data and constants. As of version 2.14.0, Stan provides full Bayesian inference for continuous-variable models through Markov chain Monte Carlo methods such as the No-U-Turn sampler, an adaptive form of Hamiltonian Monte Carlo sampling. Penalized maximum likelihood estimates are calculated using optimization methods such as the limited memory Broyden-Fletcher-Goldfarb-Shanno algorithm. Stan is also a platform for computing log densities and their gradients and Hessians, which can be used in alternative algorithms such as variational Bayes, expectation propagation, and marginal inference using approximate integration. To this end, Stan is set up so that the densities, gradients, and Hessians, along with intermediate quantities of the algorithm such as acceptance probabilities, are easily accessible. Stan can also be called from the command line using the cmdstan package, through R using the rstan package, and through Python using the pystan package. All three interfaces support sampling and optimization-based inference with diagnostics and posterior analysis. rstan and pystan also provide access to log probabilities, gradients, Hessians, parameter transforms, and specialized plotting.

97 MATHEMATICS AND COMPUTING↗

A Dirichlet process mixture model for automatic {sup 18}F-FDG PET image segmentation: Validation study on phantoms and on lung and esophageal lesions

Purpose: The aim of this study was to implement a Dirichlet process mixture (DPM) model for automatic tumor edge identification on {sup 18}F-fluorodeoxyglucose positron emission tomography ({sup 18}F-FDG PET) images by optimizing the parameters on which the algorithm depends, to validate it experimentally, and to test its robustness. Methods: The DPM model belongs to the class of the Bayesian nonparametric models and uses the Dirichlet process prior for flexible nonparametric mixture modeling, without any preliminary choice of the number of mixture components. The DPM algorithm implemented in the statistical software package R was used in this work. The contouring accuracy was evaluated on several image data sets: on an IEC phantom (spherical inserts with diameter in the range 10–37 mm) acquired by a Philips Gemini Big Bore PET-CT scanner, using 9 different target-to-background ratios (TBRs) from 2.5 to 70; on a digital phantom simulating spherical/uniform lesions and tumors, irregular in shape and activity; and on 20 clinical cases (10 lung and 10 esophageal cancer patients). The influence of the DPM parameters on contour generation was studied in two steps. In the first one, only the IEC spheres having diameters of 22 and 37 mm and a sphere of the digital phantom (41.6 mm diameter) were studied by varying the main parameters until the diameter of the spheres was obtained within 0.2% of the true value. In the second step, the results obtained for this training set were applied to the entire data set to determine DPM based volumes of all available lesions. These volumes were compared to those obtained by applying already known algorithms (Gaussian mixture model and gradient-based) and to true values, when available. Results: Only one parameter was found able to significantly influence segmentation accuracy (ANOVA test). This parameter was linearly connected to the uptake variance of the tested region of interest (ROI). In the first step of the study, a calibration curve was determined to automatically generate the optimal parameter from the variance of the ROI. This “calibration curve” was then applied to contour the whole data set. The accuracy (mean discrepancy between DPM model-based contours and reference contours) of volume estimation was below (1 ± 7)% on the whole data set (1 SD). The overlap between true and automatically segmented contours, measured by the Dice similarity coefficient, was 0.93 with a SD of 0.03. Conclusions: The proposed DPM model was able to accurately reproduce known volumes of FDG concentration, with high overlap between segmented and true volumes. For all the analyzed inserts of the IEC phantom, the algorithm proved to be robust to variations in radius and in TBR. The main advantage of this algorithm was that no setting of DPM parameters was required in advance, since the proper setting of the only parameter that could significantly influence the segmentation results was automatically related to the uptake variance of the chosen ROI. Furthermore, the algorithm did not need any preliminary choice of the optimum number of classes to describe the ROIs within PET images and no assumption about the shape of the lesion and the uptake heterogeneity of the tracer was required.

60 APPLIED LIFE SCIENCES↗

Analyzing Data Privacy for Edge Systems

Internet-of-Things (IoT)-based streaming applications are all around us. Currently, we are transitioning from IoT processing being performed on the cloud to the edge. While moving to the edge provides significant networking efficiency benefits, IoT edge computing creates significant data privacy concerns. We propose a methodology that can successfully privacy protect the continual data streams generated by sensors on the edge device. We implement local differential privacy on streaming data and incorporate Bayesian inference and Gaussian process to evaluate the privacy policy. We demonstrate our methodology on a real-world smart meter testbed and identify the optimal privacy protection settings.

Kotevska, Olivera↗

Load Balancing Using Time Series Analysis for Soft Real Time Systems with Statistically Periodic Loads

This thesis provides design and analysis of techniques for global load balancing on ensemble architectures running soft-real-time object-oriented applications with statistically periodic loads. It focuses on estimating the instantaneous average load over all the processing elements. The major contribution is the use of explicit stochastic process models for both the loading and the averaging itself. These models are exploited via statistical time-series analysis and Bayesian inference to provide improved average load estimates, and thus to facilitate global load balancing. This thesis explains the distributed algorithms used and provides some optimality results. It also describes the algorithms' implementation and gives performance results from simulation. These results show that the authors' techniques allow more accurate estimation of the global system loading, resulting in fewer object migrations than local methods. The authors' method is shown to provide superior performance, relative not only to static load-balancing schemes but also to many adaptive load-balancing methods. Results from a preliminary analysis of another system and from simulation with a synthetic load provide some evidence of more general applicability.

LOAD CONTROL↗

Load Balancing Using Time Series Analysis for Soft Real Time Systems with Statistically Periodic Loads

This thesis provides design and analysis of techniques for global load balancing on ensemble architectures running soft-real-time object-oriented applications with statistically periodic loads. It focuses on estimating the instantaneous average load over all the processing elements. The major contribution is the use of explicit stochastic process models for both the loading and the averaging itself. These models are exploited via statistical time-series analysis and Bayesian inference to provide improved average load estimates, and thus to facilitate global load balancing. This thesis explains the distributed algorithms used and provides some optimality results. It also describes the algorithms' implementation and gives performance results from simulation. These results show that our techniques allow more accurate estimation of the global system load ing, resulting in fewer object migration than local methods. Our method is shown to provide superior performance, relative not only to static load-balancing schemes but also to many adaptive methods.

REAL TIME↗

BayFlux: A Bayesian method to quantify metabolic Fluxes and their uncertainty at the genome scale

Metabolic fluxes, the number of metabolites traversing each biochemical reaction in a cell per unit time, are crucial for assessing and understanding cell function. 13 C Metabolic Flux Analysis ( 13 C MFA) is considered to be the gold standard for measuring metabolic fluxes. 13 C MFA typically works by leveraging extracellular exchange fluxes as well as data from 13 C labeling experiments to calculate the flux profile which best fit the data for a small, central carbon, metabolic model. However, the nonlinear nature of the 13 C MFA fitting procedure means that several flux profiles fit the experimental data within the experimental error, and traditional optimization methods offer only a partial or skewed picture, especially in “non-gaussian” situations where multiple very distinct flux regions fit the data equally well. Here, we present a method for flux space sampling through Bayesian inference (BayFlux), that identifies the full distribution of fluxes compatible with experimental data for a comprehensive genome-scale model. This Bayesian approach allows us to accurately quantify uncertainty in calculated fluxes. We also find that, surprisingly, the genome-scale model of metabolism produces narrower flux distributions (reduced uncertainty) than the small core metabolic models traditionally used in 13 C MFA. The different results for some reactions when using genome-scale models vs core metabolic models advise caution in assuming strong inferences from 13 C MFA since the results may depend significantly on the completeness of the model used. Based on BayFlux, we developed and evaluated novel methods (P- 13 C MOMA and P- 13 C ROOM) to predict the biological results of a gene knockout, that improve on the traditional MOMA and ROOM methods by quantifying prediction uncertainty.

59 BASIC BIOLOGICAL SCIENCES↗

Convolutional neural networks on the HEALPix sphere: a pixel-based algorithm and its application to CMB data analysis

We describe a novel method for the application of convolutional neural networks (CNNs) to fields defined on the sphere, using the Hierarchical Equal Area Latitude Pixelization scheme (HEALPix). Specifically, we have developed a pixel-based approach to implement convolutional and pooling layers on the spherical surface, similarly to what is commonly done for CNNs applied to Euclidean space. The main advantage of our algorithm is to be fully integrable with existing, highly optimized libraries for NNs (e.g., PyTorch, TensorFlow, etc.). We present two applications of our method: (i) recognition of handwritten digits projected on the sphere; (ii) estimation of cosmological parameter from simulated maps of the cosmic microwave background (CMB). The latter represents the main target of this exploratory work, whose goal is to show the applicability of our CNN to CMB parameter estimation. We have built a simple NN architecture, consisting of four convolutional and pooling layers, and we have used it for all the applications explored herein. Concerning the recognition of handwritten digits, our CNN reaches an accuracy of ~95%, comparable with other existing spherical CNNs, and this is true regardless of the position and orientation of the image on the sphere. For CMB-related applications, we tested the CNN on the estimation of a mock cosmological parameter, defining the angular scale at which the power spectrum of a Gaussian field projected on the sphere peaks. We estimated the value of this parameter directly from simulated maps, in several cases: temperature and polarization maps, presence of white noise, and partially covered maps. For temperature maps, the NN performances are comparable with those from standard spectrum-based Bayesian methods. For polarization, CNNs perform about a factor four worse than standard algorithms. Nonetheless, our results demonstrate, for the first time, that CNNs are able to extract information from polarization fields, both in full-sky and masked maps, and to distinguish between E and B -modes in pixel space. Lastly, we have applied our CNN to the estimation of the Thomson scattering optical depth at reionization ( τ ) from simulated CMB maps. Even without any specific optimization of the NN architecture, we reach an accuracy comparable with standard Bayesian methods. This work represents a first step towards the exploitation of NNs in CMB parameter estimation and demonstrates the feasibility of our approach.

79 ASTRONOMY AND ASTROPHYSICS↗

Dark Energy Survey Year 3 results: optimized $w$CDM simulation-based inference with weak lensing map-level hybrid statistics

We present cosmological constraints from the Dark Energy Survey Year 3 (DES Y3) weak lensing data using hierarchical hybrid statistics within a Bayesian simulation-based inference framework that is based on the Gower Street simulations. To maximize the precision of the inference, we have developed a new, information-theory based, data compression of the weak lensing maps to just seven highly informative summary statistics. The hybrid scheme exploits the high information content of the power spectrum, compressing both the power spectrum and neural-based summaries that are designed to extract further information. Our simulation-based approach enables principled forward modelling of all major sources of systematic uncertainty and survey properties into realistic mock observations, including the survey mask, photometric redshift uncertainties, intrinsic galaxy alignments, multiplicative shear calibration bias, source galaxy clustering, non-Gaussian shape noise, and non-linear structure formation. The summary statistics are then used in a Bayesian simulation-based inference pipeline. The inference is validated through coverage tests and checks for robustness against baryonic feedback. Assuming a $w$CDM cosmology, our analysis yields $S_8 = 0.808 \pm 0.017$, $Ω_{\rm m} = 0.325 \pm 0.024$, and $w < -0.766$ (marginalized posterior 68 per cent credible intervals). This rigorous combination of information theory, physics- and neural network-based extreme data compression, and principled Bayesian analysis improves the figure of merit for $(Ω_{\rm m}, S_8, w)$ by 60 per cent over the previous state-of-the-art, and by almost a factor of 3 over two-point analyses of the same data. They are the most precise joint constraints on $(Ω_{\rm m}, S_8, w)$ from weak gravitational lensing data alone of any survey to date. We intend to apply this analysis to the more recent DES Y6 data.

Williamson, J. [University Coll. London]↗

Accurate Automatic Delineation of Heterogeneous Functional Volumes in Positron Emission Tomography for Oncology Applications

Purpose: Accurate contouring of positron emission tomography (PET) functional volumes is now considered crucial in image-guided radiotherapy and other oncology applications because the use of functional imaging allows for biological target definition. In addition, the definition of variable uptake regions within the tumor itself may facilitate dose painting for dosimetry optimization. Methods and Materials: Current state-of-the-art algorithms for functional volume segmentation use adaptive thresholding. We developed an approach called fuzzy locally adaptive Bayesian (FLAB), validated on homogeneous objects, and then improved it by allowing the use of up to three tumor classes for the delineation of inhomogeneous tumors (3-FLAB). Simulated and real tumors with histology data containing homogeneous and heterogeneous activity distributions were used to assess the algorithm's accuracy. Results: The new 3-FLAB algorithm is able to extract the overall tumor from the background tissues and delineate variable uptake regions within the tumors, with higher accuracy and robustness compared with adaptive threshold (T{sub bckg}) and fuzzy C-means (FCM). 3-FLAB performed with a mean classification error of less than 9% +- 8% on the simulated tumors, whereas binary-only implementation led to errors of 15% +- 11%. T{sub bckg} and FCM led to mean errors of 20% +- 12% and 17% +- 14%, respectively. 3-FLAB also led to more robust estimation of the maximum diameters of tumors with histology measurements, with <6% standard deviation, whereas binary FLAB, T{sub bckg} and FCM lead to 10%, 12%, and 13%, respectively. Conclusion: These encouraging results warrant further investigation in future studies that will investigate the impact of 3-FLAB in radiotherapy treatment planning, diagnosis, and therapy response evaluation.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Development of DPD coarse-grained models: From bulk to interfacial properties

A new Bayesian method was recently introduced for developing coarse-grain (CG) force fields for molecular dynamics. The CG models designed for dissipative particle dynamics (DPD) are optimized based on trajectory matching. Here we extend this method to improve transferability across thermodynamic conditions. We demonstrate the capability of the method by developing a CG model of n-pentane from constant-NPT atomistic simulations of bulk liquid phases and we apply the CG-DPD model to the calculation of the surface tension of the liquid-vapor interface over a large range of temperatures. The coexisting densities, vapor pressures, and surface tensions calculated with different CG and atomistic models are compared to experiments. Depending on the database used for the development of the potentials, it is possible to build a CG model which performs very well in the reproduction of the surface tension on the orthobaric curve.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Quantifying properties of hot and dense QCD matter through systematic model-to-data comparison

We systematically compare an event-by-event heavy-ion collision model to data from the CERN Large Hadron Collider. Using a general Bayesian method, we probe multiple model parameters including fundamental quark-gluon plasma properties such as the specific shear viscosity η/s, calibrate the model to optimally reproduce experimental data, and extract quantitative constraints for all parameters simultaneously. Furthermore, the method is universal and easily extensible to other data and collision models.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗