Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Expectation Maximization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

A Graphical Model for Fusing Diverse Microbiome Data

This paper develops a Bayesian graphical model for fusing disparate types of count data. The motivating application is the study of bacterial communities from diverse high-dimensional features, in this case, transcripts, collected from different treatments. In such datasets, there are no explicit correspondences between the communities and each corresponds to different factors, making data fusion challenging. We introduce a flexible multinomial-Gaussian generative model for jointly modeling such count data. This latent variable model jointly characterizes the observed data through a common multivariate Gaussian latent space that parameterizes the set of multinomial probabilities of the transcriptome counts. The covariance matrix of the latent variables induces a covariance matrix of co-dependencies between all the transcripts, effectively fusing multiple data sources. We present a computationally scalable variational Expectation-Maximization (EM) algorithm for inferring the latent variables and the parameters of the model. Here, the inferred latent variables provide a common dimensionality reduction for visualizing the data and the inferred parameters provide a predictive posterior distribution. In addition to simulation studies that demonstrate the variational EM procedure, we apply our model to a bacterial microbiome dataset.

59 BASIC BIOLOGICAL SCIENCES↗

Autodifferentiable Ensemble Kalman Filters

Data assimilation is concerned with sequentially estimating a temporally evolving state. This task, which arises in a wide range of scientific and engineering applications, is particularly challenging when the state is high-dimensional and the state-space dynamics are unknown. This paper introduces a machine learning framework for learning dynamical systems in data assimilation. Here, our auto-differentiable ensemble Kalman filters (AD-EnKFs) blend ensemble Kalman filters for state recovery with machine learning tools for learning the dynamics. In doing so, AD-EnKFs leverage the ability of ensemble Kalman filters to scale to high-dimensional states and the power of automatic differentiation to train high-dimensional surrogate models for the dynamics. Numerical results using the Lorenz-96 model show that AD-EnKFs outperform existing methods that use expectation-maximization or particle filters to merge data assimilation and machine learning. In addition, AD-EnKFs are easy to implement and require minimal tuning.

autodifferentiation↗

Gaussian Mixture Model Solvers for the Boltzmann Equation

This report documents our experience constructing a numerical method for the collisional Boltzmann equation that is capable of accurately capturing the collisionless through strongly collisional limits. We explore three different functional representations and present a detailed account of a numerical method based on a spatially dependent Gaussian mixture model (GMM). The Kullback-Leibler divergence is used as a closeness measure and various expectation maximization (EM) solution algorithms are implemented to find a compact representation in velocity space for distribution functions that exhibit significant non-Maxwellian character. We discuss issues that appear with this representation over a range of Knudsen numbers for a prototypical test problem and demonstrate that the strongly collisional limit recovers a solution to Euler's equations. Looking forward, this approach is broadly applicable to the non-relativistic and relativistic collisional Vlasov equations.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Monte Carlo Simulation and Reconstruction: Assessment of Myocardial Perfusion Imaging of Tracer Dynamics With Cardiac Motion Due to Deformation and Respiration Using Gamma Camera With Continuous Acquisition

Purpose: Myocardial perfusion imaging (MPI) with single photon emission computed tomography (SPECT) is routinely used for stress testing in nuclear medicine. Recently, our group extended its potential going from 3D visual qualitative image analysis to 4D spatiotemporal reconstruction of dynamically acquired data to capture the time variation of the radiotracer concentration and the estimated myocardial blood flow (MBF) and coronary flow reserve (CFR). However, the quality of reconstructed image is compromised due to cardiac deformation and respiration. The work presented here develops an algorithm that reconstructs the dynamic sequence of separate respiratory and cardiac phases and evaluates the algorithm with data simulated with a Monte Carlo simulation for the continuous image acquisition and processing with a slowly rotating SPECT camera. Methods: A clinically realistic Monte Carlo (MC) simulation is developed using the 4D Extended Cardiac Torso (XCAT) digital phantom with respiratory and cardiac motion to model continuous data acquisition of dynamic cardiac SPECT with slowly rotating gamma cameras by incorporating deformation and displacement of the myocardium due to cardiac and respiratory motion. We extended our previously developed 4D maximum-likelihood expectation-maximization (MLEM) reconstruction algorithm for a data set binned from a continuous list mode (LM) simulation with cardiac and respiratory information. Our spatiotemporal image reconstruction uses splines to explicitly model the temporal change of the tracer for each cardiac and respiratory gate that delineates the myocardial spatial position as the tracer washes in and out. Unlike in a fully list-mode data acquisition and reconstruction the accumulated photons are binned over a specific but very short time interval corresponding to each cardiac and respiratory gate. Reconstruction results are presented showing the dynamics of the tracer in the myocardium as it continuously deforms. These results are then compared with the conventional 4D spatiotemporal reconstruction method that models only the temporal changes of the tracer activity. Mean Stabilized Activity (MSA), signal to noise ratio (SNR) and Bias for the myocardium activities for three different target-to-background ratios (TBRs) are evaluated. Dynamic quantitative indices such as wash-in (K1) and wash-out (k2) rates at each gate were also estimated. Results: The MSA and SNR are higher with higher TBRs while biases were improved with higher TBRs to less than 10%. The correlation between exhalation-inhalation sequence with the ground truth during respiratory cycle was excellent. Our reconstruction method showed better resolved myocardial walls during diastole to systole as compared to the ungated 4D image. Estimated values of K1 and k2 were also consistent with the ground truth. Conclusion: The continuous image acquisition for dynamic scan using conventional two-head gamma cameras can provide valuable information for MPI. Our study demonstrated the viability of using a continuous image acquisition method on a widely used clinical two-head SPECT system. Our reconstruction method showed better resolved myocardial walls during diastole to systole as compared to the ungated 4D image. Precise implementation of reconstruction algorithms, better segmentation techniques by generating images of different tissue types and background activity would improve the feasibility of the method in real clinical environment.

60 APPLIED LIFE SCIENCES↗

A full-ring variable-aperture cadmium zinc telluride system for whole-body single photon emission computed tomography: realistic simulations with phantoms

Single photon emission computed tomography (SPECT) is an imaging modality that has demonstrated its utility in a number of clinical indications. Despite this progress, a high sensitivity, high spatial resolution, multi-tracer SPECT with a large field of view suitable for whole-body imaging of a broad range of radiotracers for theranostics is not available. Purpose We have designed a cadmium zinc telluride (CZT) variable-aperture full-ring SPECT scanner instrumented with a broad-energy tungsten collimator intended to fill this technological gap. The final purpose is to provide a multi-tracer solution for brain and whole-body imaging. Our static SPECT scanner breaks the paradigm of the standard dual- and triple-head rotational SPECT systems, utilizing a larger detector area in each scan increasing the sensitivity. We provide a demonstration of the performance of our design using a realistic model of our detector with simulated body-sized 99mTc phantoms. Methods We developed a realistic model of our detector by using a combination of a Geant4 Monte Carlo simulation and a CZT detector response model based on a finite element model. Our approach models the characteristic low-energy tail effect in CZT that noticeably affects the sensitivity and the quality of the scatter correction in CZT detectors. We implement a modified dual energy window scatter correction adapted to include the CZT low-energy tail effect. A dedicated correction is also developed to eliminate the undesirable truncation observed in images given the presence of detector edges and gaps between detectors, due to the non-rotational nature of our device. Corrections for the attenuation, detector response and the presence of collimators are also included. The images are reconstructed using the maximum-likelihood expectation-maximization algorithm implemented in the reconstruction open software STIR. Detector and reconstruction performance are characterized with a Derenzo phantom and a body-sized National Electrical Manufacturers Association (NEMA) Image Quality (IQ) phantom containing 99mTc. Results Our SPECT design can resolve 6.4mm rods in a Derenzo phantom and obtain a good image contrast with the IQ phantom. Explicit testing of the gap and edge correction is provided, showing an excellent performance in eliminating the image truncation artifacts. Our modified scatter correction shows no overestimation of the contrast-recovery ratio for our realistic CZT detector model, as opposed to the cases without correction and with a standard dual-energy window scatter correction. Conclusions In this paper, we further demonstrate the performance of our design for whole-body imaging purposes. This adds to our previous demonstration of improved qualitative and quantitative 99mTc imaging for brain perfusion and 123I imaging for dopamine transport with respect to state-of-the-art NaI dual-head cameras. We show that our design performs similarly to the VERITON SPECT from Spectrum Dynamics, a commercial full-ring CZT SPECT camera, with the potential advantage of the broader energy range of application given by our custom-design tungsten collimators. Here, our device combines high sensitivity and image resolution with a broad-energy imaging application for the purpose of clinical imaging and theranostics of emerging radionuclides.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

3D Source Reconstruction Using Coded Aperture Gamma-Ray Imaging

Recent measurements with coded-aperture imagers demonstrate material mass determination in a holdup setting to an accuracy within a few percent. This capability is of particular interest to the Surplus Plutonium Disposition (SPD) project, which aims to dilute and dispose of surplus plutonium oxide. Gamma-ray imagers can be used to determine holdup without interrupting normal operations. In this work, we examine techniques for 3D source localization and mass determination using gamma-ray imagers. Coded-aperture imagers provide excellent source localization within the 2D image plane; however, multiple imagers operating in tandem are necessary to identify source location in 3D space. A Maximum Likelihood Expectation-Maximization (MLEM) method for fitting detector mappings is a powerful tool for accomplishing this task. MLEM allows 3D source localization to be simultaneously constrained using multiple gamma-ray imagers by constructing the basis for the MLEM fit using detector mappings from different detector locations stitched together. Each of these basis points represents a singular response from a source in 3D space and is generated using Monte Carlo simulations of sources placed individually at different locations throughout the imager’s field of view. Additionally, implementing knowledge of the physical equipment in the simulations of the glovebox used for the SPD project incorporates attenuation effects that are needed to calculate material holdup.

Laminack, Alex↗

Entanglement maximization and mirror symmetry in two-Higgs-doublet models

We consider 2-to-2 scatterings of Higgs bosons in a CP-conserving two-Higgs-doublet model (2HDM) and study the implication of maximizing the entanglement in the flavor space, where the two doublets Φ a , a = 1, 2, can be viewed as a qubit: Φ 1 = |0⟩ and Φ 2 = |1⟩. More specifically, we compute the scattering amplitudes for Φ a Φ b → Φ c Φ d and require the outgoing flavor entanglement to be maximal for a full product basis such as the computational basis, which consists of {|00⟩, |01⟩, |10⟩, |11⟩}. In the unbroken phase and turning off the gauge interactions, entanglement maximization results in the appearance of an U(2) × U(2) global symmetry among the quartic couplings, which in general is broken softly by the mass terms. Interestingly, once the Higgs bosons acquire vacuum expectation values, maximal entanglement enforces an exact U(2) × U(2) symmetry, which is spontaneously broken to U(1) × U(1). As a byproduct, this gives rise to Higgs alignment as well as to the existence of 6 massless Nambu-Goldstone bosons. The U(2) × U(2) symmetry can be gauged to lift the massless Goldstones, while maintaining maximal entanglement demands the presence of a discrete Z 2 symmetry interchanging the two gauge sectors. The model is custodially invariant in the scalar sector, and the inclusion of fermions requires a mirror dark sector, related to the standard one by the Z 2 symmetry.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Supercharging simulation-based inference for Bayesian optimal experimental design

Abstract Bayesian optimal experimental design (BOED) seeks to maximize the expected information gain (EIG) of experiments. This requires a likelihood estimate, which in many settings is intractable. Simulation-based inference (SBI) provides powerful tools for this regime. However, existing work explicitly connecting SBI and BOED is restricted to a single contrastive EIG bound. We show that the EIG admits multiple formulations which can directly leverage modern SBI density estimators, encompassing neural posterior, likelihood, and ratio estimation. Building on this perspective, we define a novel EIG estimator using neural likelihood estimation. Further, we identify optimization as a key bottleneck of gradient based EIG maximization and show that a simple multi-start parallel gradient ascent procedure can substantially improve reliability and performance. With these innovations, our SBI-based BOED methods are able to match or outperform by up to 22% existing state-of-the-art approaches across standard BOED benchmarks.

97 MATHEMATICS AND COMPUTING↗

Analysis and optimization of seismic monitoring networks with Bayesian optimal experimental design

SUMMARY Monitoring networks increasingly aim to assimilate data from a large number of diverse sensors covering many sensing modalities. Bayesian optimal experimental design (OED) seeks to identify data, sensor configurations or experiments which can optimally reduce uncertainty and hence increase the performance of a monitoring network. Information theory guides OED by formulating the choice of experiment or sensor placement as an optimization problem that maximizes the expected information gain (EIG) about quantities of interest given prior knowledge and models of expected observation data. Therefore, within the context of seismo-acoustic monitoring, we can use Bayesian OED to configure sensor networks by choosing sensor locations, types and fidelity in order to improve our ability to identify and locate seismic sources. In this work, we develop the framework necessary to use Bayesian OED to optimize a sensor network’s ability to locate seismic events from arrival time data of detected seismic phases at the regional-scale. This framework requires five elements: (i) A likelihood function that describes the distribution of detection and traveltime data from the sensor network, (ii) A prior distribution that describes a priori belief about seismic events, (iii) A Bayesian solver that uses a prior and likelihood to identify the posterior distribution of seismic events given the data, (iv) An algorithm to compute EIG about seismic events over a data set of hypothetical prior events, (v) An optimizer that finds a sensor network which maximizes EIG. Once we have developed this framework, we explore many relevant questions to monitoring such as: how to trade off sensor fidelity and earth model uncertainty; how sensor types, number and locations influence uncertainty; and how prior models and constraints influence sensor placement.

58 GEOSCIENCES↗

Off-policy deep reinforcement learning with automatic entropy adjustment for adaptive online grid emergency control

Electric overloading conditions and contingencies put modern power systems at risk of voltage collapse and blackouts. Load shedding is crucial to maintain voltage stability for grid emergency control. However, the rule- or model-based schemes rely on accurate dynamic system models and face considerable challenges in adapting to various operating conditions and uncertain event occurrences. Here, to address these issues, this paper proposes a novel deep reinforcement learning (DRL)-based voltage stability control algorithm with automatic entropy adjustment (AEA) for grid emergency control. Various dynamic network components for complex system operations are modeled to construct the DRL environment. An off-policy soft actor-critic architecture is developed to maximize the expected reward and policy entropy simultaneously. The AEA mechanism is proposed to facilitate the policy maximum entropy procedure, and the proposed method can automatically provide effective discrete and continuous actions against various fault scenarios. Our approach accomplishes high sampling efficiency, scalability, and auto-adaptivity of the control policies under high uncertainties. Comparative studies with the existing DRL-based control methods in IEEE benchmarks indicate salient performance improvement of the proposed method for dynamic system emergency control.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Co-optimization of fuel properties, combustion system geometry, and injection strategy for conventional diesel fuel

Here, studies have shown that fuel properties can impact an engine’s operation in several ways, including ignition delay, sooting tendency, mixture formation, and combustion temperature. In mixing-controlled compression ignition (MCCI) engines, the fuel system design and piston bowl geometry significantly affect combustion performance and emissions. Based on current information, it is difficult to draw conclusions about fuel property effects and sensitivities. The central fuel hypothesis approach used in the US Department of Energy Co-Optima program has worked well for spark ignition fuels: identifying critical fuel property ranges is sufficient to screen fuel blends that are expected to maximize efficiency and reduce pollutant emissions. However, for MCCI-relevant fuels, the information gained from past studies is not sufficient to build such a merit function or to allow for performing a similar screening of fuel blends. It is hypothesized that a co-optimization of a fuel’s physical and chemical properties, combustion system geometry, and injection strategy could leverage synergies between the effects of the fuel properties and geometries, resulting in improved performance over state-of-the-art. A machine learning–assisted unconstrained global optimization algorithm was used to explore a design space comprising 23 independent variables. The results show that physical property effects were minimal even for large variations in fuel properties, and the only interaction effect that was observed was the effect of varied fuel density parameters on fuel/air mixture formation. Nevertheless, these interactions were not sufficient in magnitude to significantly affect optimization results. Therefore, analysis of the results suggests that fuel physical properties cannot be leveraged in a co-optimization context to increase engine efficiency.

33 ADVANCED PROPULSION SYSTEMS↗

The impact of market design and clean energy incentives on strategic generation investments and resource adequacy in low-carbon electricity markets

Well-designed electricity markets play a crucial role in maintaining reliable electric power systems, which are critical in modern society. Here, this study examines the impact of different electricity market designs and clean energy incentive schemes on supporting renewable energy integration and achieving clean energy goals. To this end, we utilize a game-theoretical generation expansion planning model where generation companies make investment and retirement decisions to maximize their expected profit. The model is structured as an equilibrium problem with equilibrium constraints (EPEC) and solved using a diagonalization approach combined with progressive hedging. We analyze three types of electricity market designs: an energy-only market, a capacity market, and a clean energy market, and consider a wide range of market parameters resulting in 14 total scenarios. Wind and solar capacity comprise the majority of new investments in all considered scenarios, but the resultant system planning reserve margin (PRM) can differ significantly depending on market parameters. We also find that profit-driven investments lead to lower PRMs than a traditional system cost minimization approach. These individual scenario results further demonstrate how different market designs and clean energy incentive schemes may influence investor decision-making and impact resource adequacy throughout the clean energy transition.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Reinforcement learning for adaptive maintenance policy optimization under imperfect knowledge of the system degradation model and partial observability of system states

Maintenance policy optimization usually is faced with challenges that arise from an imperfect knowledge of system degradation models and from the partial observability of system degradation states. Here, this paper proposes a reinforcement learning method to address these two challenges for a class of maintenance problems with Markov degradation processes. The reinforcement learning approach consists of a learning component and a planning component. Using sequentially collected observations, at each step of decision-making the learning component improves the knowledge of system degradation in terms of the probability distributions of the transition rates based on sequential Bayesian inference. Using the updated transition rates, at each step of decision-making the maintenance policy optimization problem is then formulated as a partially observable Markov decision problem, and the planning component computes the optimal maintenance policy that maximizes the expected cumulative reward. The proposed method is illustrated using a numerical example with repair and inspection maintenance actions. The result shows that as more observations are collected, the learning component progressively learns the true system degradation process, and the planning component adjusts the optimal maintenance policy accordingly as well, which leads to increased reward.

42 ENGINEERING↗

Probabilistic inference in very large universes

Our current favored cosmological theories allow for the striking and controversial possibility that the observable universe is just a small part of a much larger universe in which parameters that describe the effective, low-energy laws of physics vary from one region to another. The controversy is largely driven by the fact that such a “very large universe” is mostly observationally inaccessible to us, so the issue arises of how we can reasonably assess a theory that describes such a universe. In this paper, we propose a Bayesian method for theory assessment based on theory-generated probability distributions for our observations. We focus on the principles that define this method, leaving aside concerns about how, in practice, one would carry out the required calculations. (One important issue that we set aside is the measure problem.) We argue that cosmological theories can be tested by the standard method of Bayesian updating, but we need to use theoretical predictions for “first-person” probabilities—that is, probabilities that we should use for our observations, taking into account all relevant selection effects. These selection effects can vary from one observer to another and can vary with time, so, in principle, first-person probabilities are defined for each observer instant—an observer at a specific instant of time. Calculations of first-person probabilities should take into account everything that the observer believes about herself and her surroundings, which we refer to as her subjective state. If the universe is very large, a theory might predict that there are many observer instants in the same subjective state; we argue that first-person probabilities should be calculated using a principle of self-locating indifference (PSLI), the assumption that any real observer should make predictions for her future as if she were chosen randomly and uniformly from the theoretically predicted observer instants that share her subjective state. We believe the PSLI is intuitively very reasonable, but we also argue that, if the theory is correct, the use of this principle maximizes the expected fraction of observers who will make correct predictions. A further complication is that cosmological theories are not expected to fully predict the detailed properties of the universe, but rather will predict a set of possible universes, each with a probability. Different possible universes will generically have different numbers of observers. We argue that, in the calculation of first-person probabilities, the probability for each possible universe should be weighted by the number of observer instants in the specified subjective state that it contains. These issues have been controversial in the literature, so we also provide a rebuttal to the claim that principles like the PSLI involve a “selection fallacy”; a rebuttal to what we dub the principle of required certainty; an argument rejecting theories that predict a preponderance of Boltzmann brains; a rebuttal to a parable about humans and Jovians used by Hartle and Srednicki to argue that assumptions of typicality can lead to absurd consequences; and, finally, a discussion about how the use of “old evidence” can be fit into a Bayesian mold.

Azhar, Feraz [University of Notre Dame, IN (United↗

Stochastic Model Predictive Control With Gaussian Wind Direction Preview for Wake Steering

This article addresses the problem of wake steering control for wind farms that explicitly consider the tradeoff between farm-level power generation and yaw duty cycle under variable and uncertain wind conditions. A novel stochastic model predictive control (MPC) algorithm is presented, which utilizes a stochastic model of the freestream wind field components in a receding horizon framework to compute optimal yaw set points that maximize the expected value of the farm power while constraining the yaw actuation. Different configurations of the algorithm are evaluated using a steady-state wind farm simulator. The proposed stochastic MPC algorithm can plan control actions over a future prediction horizon based on probabilistic estimates of the incoming wind magnitude and direction.

17 WIND ENERGY↗

Maximum Likelihood Spectrum Decomposition for Isotope Identification and Quantification

A spectral decomposition method has been implemented to identify and quantify isotopic source terms in high-resolution gamma-ray spectroscopy in static geometry and shielding scenarios. Monte Carlo simulations were used to build the response matrix of a shielded high-purity germanium detector monitoring an effluent stream with a Marinelli configuration. The decomposition technique was applied to a series of calibration spectra taken with the detector using a multi-nuclide standard. These results are compared with decay-corrected values from the calibration certificate. For most nuclei in the standard ( 241 Am, 109 Cd, 137 Cs, and 60 Co), the deviations from the certificate values were generally no more than 6% with a few outliers as high as 10%. Furthermore, for 57 Co, the radionuclide with the lowest activity, the deviations from the standard reached as high as 25%, driven by the meager statistics in the calibration spectra. In addition, a complete treatment of error propagation for the technique is presented.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Estimating Sparse Direct Effects in Multivariate Regression With the Spike-and-Slab LASSO

The multivariate regression interpretation of the Gaussian chain graph model simultaneously parametrizes (i) the direct effects of p predictors on q outcomes and (ii) the residual partial covariances between pairs of outcomes. We introduce a new method for fitting sparse versions of these models with spike-and-slab LASSO (SSL) priors. We develop an Expectation Conditional Maximization algorithm to obtain sparse estimates of the p × q matrix of direct effects and the q × q residual precision matrix. Our algorithm iteratively solves a sequence of penalized maximum likelihood problems with self-adaptive penalties that gradually filter out negligible regression coefficients and partial covariances. Because it adaptively penalizes individual model parameters, our method is seen to outperform fixed-penalty competitors on simulated data. We establish the posterior contraction rate for our model, buttressing our method’s excellent empirical performance with strong theoretical guarantees. Using our method, we estimated the direct effects of diet and residence type on the composition of the gut microbiome of elderly adults.

EM algorithm↗