Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “EM algorithm”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Mixture densities, maximum likelihood, and the EM algorithm

The problem of estimating the parameters which determine a mixture density is reviewed as well as maximum likelihood estimation for it. A particular iterative procedure for numerically approximating maximum likelihood estimates for mixture density problems is considered. This EM algorithm, is a specialization to the mixture density context of a general algorithm of the same name used to approximate maximum likelihood estimates for incomplete data problems. The formulation and theoretical and practical properties of the EM algorithm for mixture densities are discussed focussing in particular on mixtures of densities from exponential families.

Redner, R. A.↗

Probabilistic partition of unity networks for high–dimensional regression problems

We explore the probabilistic partition of unity network (PPOU-Net) model in the context of high-dimensional regression problems and propose a general framework focusing on adaptive dimensionality reduction. With the proposed framework, the target function is approximated by a mixture of experts model on a low-dimensional manifold, where each cluster is associated with a fixed-degree polynomial. We present a training strategy that leverages the expectation maximization (EM) algorithm. During the training, we alternate between (i) applying gradient descent to update the DNN coefficients; and (ii) using closed-form formulae derived from the EM algorithm to update the mixture of experts model parameters. Under the probabilistic formulation, step (ii) admits the form of embarrassingly paralleliazable weighted least-squares solves. The PPOU-Nets consistently outperform the baseline fully-connected neural networks of comparable sizes in numerical experiments of various data dimensions. Here, we also explore the proposed model in applications of quantum computing, where the PPOU-Nets act as surrogate models for cost landscapes associated with variational quantum circuits.

97 MATHEMATICS AND COMPUTING↗

Symmetric normal mixtures

We consider mixture density estimation under the symmetry constraint x = Az for an orthogonal matrix A. This distributional constraint implies a corresponding constraint on the mixture parameters. Focusing on the gaussian case, we derive an expectation-maximization (EM) algorithm to enforce the constraint and show results for modeling of image feature vectors.

symmetry constraint↗

A Load Switching Group based Feeder-level Microgrid Energy Management Algorithm for Service Restoration in Power Distribution System

This paper presents a load switching group based energy management system (LSG-EMS) for operating microgrids on a distribution feeder powered by one or multiple grid-forming distributed energy resources. Loads on a distribution feeder are divided into load switching groups that can be remotely switched on and off. The LSG-EMS algorithm, formulated as a mixed-integer linear programming (MILP) problem, has an objective function of maximizing the served loads while minimizing the total number of switching actions. A new set of topology constraints are developed for allowing multiple microgrids to be formed on the feeder and selecting the optimal supply path. Customer comfort is accounted for by maximizing the supply duration in the customer preferred service period and enforcing a minimum service duration. The proposed method is demonstrated on a modified IEEE 33-bus system using actual customer data. Simulation results show that the LSG-EMS successfully coordinates multiple grid-forming sources by selecting an optimal supply topology that maximizes the supply period of both the critical and noncritical loads while minimizing customer service interruptions in the service restoration process.

Hu, Rongxing↗

Accounting for uncertainty in complex alluvial aquifer modeling by Bayesian multi-model approach

Alluvial aquifers by nature are complex caused by varied depositional environments. Developing a reliable groundwater model to represent an alluvial aquifer is non-trivial. Also, relying on a single best calibrated model may not be sufficient because of an inadequate choice of model parameter values. To better understand groundwater dynamics and improve model prediction reliability, this study presents a Bayesian multi-model uncertainty quantification (BMMUQ) framework to account for model parameter uncertainty in complex alluvial groundwater modeling. The methodology was applied to the agriculturally intensive Mississippi River alluvial aquifer (MRAA), Northeast Louisiana. An aquifer architecture was first constructed using 7,259 well logs in the MRAA area which covers three fluvial deposits (alluvium, braided-stream terrace, and braided-stream terrace-loess). A 12-layer MODFLOW model was then developed to address the alluvial aquifer complexity and well calibrated through a genetic algorithm. This study quantified model parameter uncertainty in hydraulic conductivity and specific storage of sand facies. Bayesian model averaging (BMA) with the Expectation Maximization (EM) algorithm was adopted to derive posterior model weights and head variances of 50 alternative conceptual groundwater flow models, and thereby obtains BMA ensemble model predictions instead of only relying on the best calibrated conceptual model. Overall, results show that an estimated around 950 million m3 of groundwater storage loss occurs in 2015 with respect to the beginning of 2004, due to high groundwater demand for irrigation in the MRAA area. Explicitly quantifying model uncertainty can produce more reliable groundwater level predictions from BMA ensemble model. The presented groundwater modeling framework improves our understanding of the MRAA and provides a valuable tool to assist agricultural water management.

54 ENVIRONMENTAL SCIENCES↗

A Graphical Model for Fusing Diverse Microbiome Data

This paper develops a Bayesian graphical model for fusing disparate types of count data. The motivating application is the study of bacterial communities from diverse high-dimensional features, in this case, transcripts, collected from different treatments. In such datasets, there are no explicit correspondences between the communities and each corresponds to different factors, making data fusion challenging. We introduce a flexible multinomial-Gaussian generative model for jointly modeling such count data. This latent variable model jointly characterizes the observed data through a common multivariate Gaussian latent space that parameterizes the set of multinomial probabilities of the transcriptome counts. The covariance matrix of the latent variables induces a covariance matrix of co-dependencies between all the transcripts, effectively fusing multiple data sources. We present a computationally scalable variational Expectation-Maximization (EM) algorithm for inferring the latent variables and the parameters of the model. Here, the inferred latent variables provide a common dimensionality reduction for visualizing the data and the inferred parameters provide a predictive posterior distribution. In addition to simulation studies that demonstrate the variational EM procedure, we apply our model to a bacterial microbiome dataset.

59 BASIC BIOLOGICAL SCIENCES↗

Software for Data Analysis with Graphical Models

Probabilistic graphical models are being used widely in artificial intelligence and statistics, for instance, in diagnosis and expert systems, as a framework for representing and reasoning with probabilities and independencies. They come with corresponding algorithms for performing statistical inference. This offers a unifying framework for prototyping and/or generating data analysis algorithms from graphical specifications. This paper illustrates the framework with an example and then presents some basic techniques for the task: problem decomposition and the calculation of exact Bayes factors. Other tools already developed, such as automatic differentiation, Gibbs sampling, and use of the EM algorithm, make this a broad basis for the generation of data analysis software.

Buntine, Wray L.↗

Data Analysis with Graphical Models: Software Tools

Probabilistic graphical models (directed and undirected Markov fields, and combined in chain graphs) are used widely in expert systems, image processing and other areas as a framework for representing and reasoning with probabilities. They come with corresponding algorithms for performing probabilistic inference. This paper discusses an extension to these models by Spiegelhalter and Gilks, plates, used to graphically model the notion of a sample. This offers a graphical specification language for representing data analysis problems. When combined with general methods for statistical inference, this also offers a unifying framework for prototyping and/or generating data analysis algorithms from graphical specifications. This paper outlines the framework and then presents some basic tools for the task: a graphical version of the Pitman-Koopman Theorem for the exponential family, problem decomposition, and the calculation of exact Bayes factors. Other tools already developed, such as automatic differentiation, Gibbs sampling, and use of the EM algorithm, make this a broad basis for the generation of data analysis software.

Buntine, Wray L.↗

Statistical methods for astronomical data with upper limits. II - Correlation and regression

Statistical methods for calculating correlations and regressions in bivariate censored data where the dependent variable can have upper or lower limits are presented. Cox's regression and the generalization of Kendall's rank correlation coefficient provide significant levels of correlations, and the EM algorithm, under the assumption of normally distributed errors, and its nonparametric analog using the Kaplan-Meier estimator, give estimates for the slope of a regression line. Monte Carlo simulations demonstrate that survival analysis is reliable in determining correlations between luminosities at different bands. Survival analysis is applied to CO emission in infrared galaxies, X-ray emission in radio galaxies, H-alpha emission in cooling cluster cores, and radio emission in Seyfert galaxies.

Isobe, T.↗

Density Estimation with Mercer Kernels

We present a new method for density estimation based on Mercer kernels. The density estimate can be understood as the density induced on a data manifold by a mixture of Gaussians fit in a feature space. As is usual, the feature space and data manifold are defined with any suitable positive-definite kernel function. We modify the standard EM algorithm for mixtures of Gaussians to infer the parameters of the density. One benefit of the approach is it's conceptual simplicity, and uniform applicability over many different types of data. Preliminary results are presented for a number of simple problems.

Macready, William G.↗

Particle Filtering for Model-Based Anomaly Detection in Sensor Networks

A novel technique has been developed for anomaly detection of rocket engine test stand (RETS) data. The objective was to develop a system that postprocesses a csv file containing the sensor readings and activities (time-series) from a rocket engine test, and detects any anomalies that might have occurred during the test. The output consists of the names of the sensors that show anomalous behavior, and the start and end time of each anomaly. In order to reduce the involvement of domain experts significantly, several data-driven approaches have been proposed where models are automatically acquired from the data, thus bypassing the cost and effort of building system models. Many supervised learning methods can efficiently learn operational and fault models, given large amounts of both nominal and fault data. However, for domains such as RETS data, the amount of anomalous data that is actually available is relatively small, making most supervised learning methods rather ineffective, and in general met with limited success in anomaly detection. The fundamental problem with existing approaches is that they assume that the data are iid, i.e., independent and identically distributed, which is violated in typical RETS data. None of these techniques naturally exploit the temporal information inherent in time series data from the sensor networks. There are correlations among the sensor readings, not only at the same time, but also across time. However, these approaches have not explicitly identified and exploited such correlations. Given these limitations of model-free methods, there has been renewed interest in model-based methods, specifically graphical methods that explicitly reason temporally. The Gaussian Mixture Model (GMM) in a Linear Dynamic System approach assumes that the multi-dimensional test data is a mixture of multi-variate Gaussians, and fits a given number of Gaussian clusters with the help of the wellknown Expectation Maximization (EM) algorithm. The parameters thus learned are used for calculating the joint distribution of the observations. However, this GMM assumption is essentially an approximation and signals the potential viability of non-parametric density estimators. This is the key idea underlying the new approach.

Solano, Wanda↗

Shape Estimation for Elongated Deformable Object using B-spline Chained Multiple Random Matrices Model

In this paper, a B-spline chained multiple random matrix models (RMMs) representation is proposed to model geometric characteristics of an elongated deformable object. The hyper degrees of freedom structure of the elongated deformable object make its shape estimation challenging. Based on the likelihood function of the proposed B-spline chained multiple RMMs, an expectation-maximization (EM) method is derived to estimate the shape of the elongated deformable object. A split and merge method based on the Euclidean minimum spanning tree (EMST) is proposed to provide initialization for the EM algorithm. The proposed algorithm is evaluated for the shape estimation of the elongated deformable objects in scenarios, such as the static rope with various configurations (including configurations with intersection), the continuous manipulation of a rope and a plastic tube, and the assembly of two plastic tubes. The execution time is computed and the accuracy of the shape estimation results is evaluated based on the comparisons between the estimated width values and its ground-truth, and the intersection over union (IoU) metric.

Gang Yao↗

Sensor Data Quality and Angular Rate Down-Selection Algorithms on SLS EM-1

The NASA Space Launch System Block 1 launch vehicle is equipped with an Inertial Navigation System (INS) and multiple Rate Gyro Assemblies (RGA) that are used in the Guidance, Navigation, and Control (GN&C) algorithms. The INS provides the inertial position, velocity, and attitude of the vehicle along with both angular rate and specific force measurements. Additionally, multiple sets of co-located rate gyros supply angular rate data. The collection of angular rate data, taken along the launch vehicle, is used to separate out vehicle motion from flexible body dynamics. Since the system architecture uses redundant sensors, the capability was developed to evaluate the health (or validity) of the independent measurements. A suite of Sensor Data Quality (SDQ) algorithms is responsible for assessing the angular rate data from the redundant sensors. When failures are detected, SDQ will take the appropriate action and disqualify or remove faulted sensors from forward processing. Additionally, the SDQ algorithms contain logic for down-selecting the angular rate data used by the GNC software from the set of healthy measurements. This paper explores the trades and analyses that were performed in selecting a set of robust fault-detection algorithms included in the GN&C flight software. These trades included both an assessment of hardware-provided health and status data as well as an evaluation of different algorithms based on time-to-detection, type of failures detected, and probability of detecting false positives. We then provide an overview of the algorithms used for both fault-detection and measurement down selection. We next discuss the role of trajectory design, flexible-body models, and vehicle response to off-nominal conditions in setting the detection thresholds. Lastly, we present lessons learned from software integration and hardware-in-the-loop testing.

Park, Thomas↗

Sensor Data Quality and Angular Rate Down-Selection Algorithms on SLS EM-1

The NASA Space Launch System Block 1 launch vehicle is equipped with an Inertial Navigation System (INS) and multiple Rate Gyro Assemblies (RGA) that are used in the Guidance, Navigation, and Control (GN&C) algorithms. The INS provides the inertial position, velocity, and attitude of the vehicle along with both angular rate and specific force measurements. Additionally, multiple sets of co-located rate gyros supply angular rate data. The collection of angular rate data, taken along the launch vehicle, is used to separate out vehicle motion from flexible body dynamics. Since the system architecture uses redundant sensors, the capability was developed to evaluate the health (or validity) of the independent measurements. A suite of Sensor Data Quality (SDQ) algorithms is responsible for assessing the angular rate data from the redundant sensors. When failures are detected, SDQ will take the appropriate action and disqualify or remove faulted sensors from forward processing. Additionally, the SDQ algorithms contain logic for down-selecting the angular rate data used by the GN&C software from the set of healthy measurements. This paper provides an overview of the algorithms used for both fault-detection and measurement down selection.

Park, Thomas↗

Structures via Reasoning - Applying AI to Cryo Electron Microscopy to Reveal Structural Variability

There have been breakthroughs of latest cryo electron microscopy (cryo-EM) data analysis algorithms to classify cryo-EM image data. However, most of these cryo-EM reconstruction methods have focused on classifying distinctly different biomolecule structures. Here, we present our approaches of deep learning to differentiate homologous structures that are distinguishable only with inner morphological differences. We succeeded supervised classification of these subtly different homologues. However, we could not differentiate them with unsupervised methods. Here we discuss what further approaches are likely needed for successful unsupervised classification.

47 OTHER INSTRUMENTATION↗

Computationally efficient Bayesian estimation of graphical networks for omics data

Graphical networks are useful, widely-used modeling approaches to represent complex biological processes with biological measurements generated by platforms such as mass spectrometry. Bayesian analyses of graphical networks for omics data have several advantages over their frequentist counterparts, such as the inclusion of prior knowledge in the estimation of models. However, Bayesian approaches to date have only been feasible for data with a couple hundred biomolecules due to prohibitive computational time, but omics data often contains tens of thousands of biomolecules. Here, we present and illustrate a more computationally efficient approach named BPlane (Bayesian PseudoLikelihood-based Algorithm for Network Estimation) to extend Bayesian modeling capabilities for larger-sized datasets, such as most untargeted proteomics data. Via simulation, we demonstrate that BPlane produces substantial computational savings over a current state-of-the-art Bayesian algorithm while maintaining competitive edge detection accuracy. On a SARS-CoV2 proteomics data with 7000 proteins, the competing algorithm takes three times as long to complete the first iteration as BPlane takes to converge after over 100 iterations.

EM algorithm↗

Neutron source reconstruction using a generalized expectation–maximization algorithm on one-dimensional neutron images from the Z facility

Magnetized Liner Inertial Fusion experiments have been performed at the Z facility at Sandia National Laboratories. These experiments use deuterium fuel, which produces 2.45 MeV neutrons on reaching thermonuclear conditions. To study the spatial structure of neutron production, the one-dimensional imager of neutrons diagnostic was fielded to record axial resolved neutron images. In this diagnostic, neutrons passing through a rolled edge aperture form an image on a CR-39-based solid state nuclear track detector. In this report we present a modified generalized expectation–maximization algorithm to reconstruct an axial neutron emission profile of the stagnated fusion plasma. We validate the approach by comparing the reconstructed neutron emission profile to an x-ray emission profile provided by a time-integrated pinhole camera.

47 OTHER INSTRUMENTATION↗

Estimating Sparse Direct Effects in Multivariate Regression With the Spike-and-Slab LASSO

The multivariate regression interpretation of the Gaussian chain graph model simultaneously parametrizes (i) the direct effects of p predictors on q outcomes and (ii) the residual partial covariances between pairs of outcomes. We introduce a new method for fitting sparse versions of these models with spike-and-slab LASSO (SSL) priors. We develop an Expectation Conditional Maximization algorithm to obtain sparse estimates of the p × q matrix of direct effects and the q × q residual precision matrix. Our algorithm iteratively solves a sequence of penalized maximum likelihood problems with self-adaptive penalties that gradually filter out negligible regression coefficients and partial covariances. Because it adaptively penalizes individual model parameters, our method is seen to outperform fixed-penalty competitors on simulated data. We establish the posterior contraction rate for our model, buttressing our method’s excellent empirical performance with strong theoretical guarantees. Using our method, we estimated the direct effects of diet and residence type on the composition of the gut microbiome of elderly adults.

EM algorithm↗