A likelihood ratio test for shrinkage covariance estimators
Explore the source record for details and available documents.
Anderson, Dylan Zachary↗
SEARCH · Engineering Papers
Results for “likelihood”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Flow-based likelihoods for non-Gaussian inference
We investigate the use of data-driven likelihoods to bypass a key assumption made in many scientific analyses, which is that the true likelihood of the data is Gaussian. In particular, we suggest using the optimization targets of flow-based generative models, a class of models that can capture complex distributions by transforming a simple base distribution through layers of nonlinearities. We call these flow-based likelihoods (FBL). We analyze the accuracy and precision of the reconstructed likelihoods on mock Gaussian data, and show that simply gauging the quality of samples drawn from the trained model is not a sufficient indicator that the true likelihood has been learned. We nevertheless demonstrate that the likelihood can be reconstructed to a precision equal to that of sampling error due to a finite sample size. We then apply FBLs to mock weak lensing convergence power spectra, a cosmological observable that is significantly non-Gaussian (NG). We find that the FBL captures the NG signatures in the data extremely well, while other commonly used data-driven likelihoods, such as Gaussian mixture models and independent component analysis, fail to do so. This suggests that works that have found small posterior shifts in NG data with data-driven likelihoods such as these could be underestimating the impact of non-Gaussianity in parameter constraints. By introducing a suite of tests that can capture different levels of NG in the data, we show that the success or failure of traditional data-driven likelihoods can be tied back to the structure of the NG in the data. Here, unlike other methods, the flexibility of the FBL makes it successful at tackling different types of NG simultaneously. Because of this, and consequently their likely applicability across datasets and domains, we encourage their use for inference when sufficient mock data are available for training.
79 ASTRONOMY AND ASTROPHYSICS↗
Non-Gaussianity in the weak lensing correlation function likelihood – implications for cosmological parameter biases
ABSTRACT We study the significance of non-Gaussianity in the likelihood of weak lensing shear two-point correlation functions, detecting significantly non-zero skewness and kurtosis in 1D marginal distributions of shear two-point correlation functions in simulated weak lensing data. We examine the implications in the context of future surveys, in particular LSST, with derivations of how the non-Gaussianity scales with survey area. We show that there is no significant bias in 1D posteriors of Ωm and σ8 due to the non-Gaussian likelihood distributions of shear correlations functions using the mock data (100 deg2). We also present a systematic approach to constructing approximate multivariate likelihoods with 1D parametric functions by assuming independence or more flexible non-parametric multivariate methods after decorrelating the data points using principal component analysis (PCA). While the use of PCA does not modify the non-Gaussianity of the multivariate likelihood, we find empirically that the 1D marginal sampling distributions of the PCA components exhibit less skewness and kurtosis than the original shear correlation functions. Modelling the likelihood with marginal parametric functions based on the assumption of independence between PCA components thus gives a lower limit for the biases. We further demonstrate that the difference in cosmological parameter constraints between the multivariate Gaussian likelihood model and more complex non-Gaussian likelihood models would be even smaller for an LSST-like survey. In addition, the PCA approach automatically serves as a data compression method, enabling the retention of the majority of the cosmological information while reducing the dimensionality of the data vector by a factor of ∼5.
79 ASTRONOMY AND ASTROPHYSICS↗
Learning likelihood ratios with neural network classifiers
The likelihood ratio is a crucial quantity for statistical inference in science that enables hypothesis testing, construction of confidence intervals, reweighting of distributions, and more. Many modern scientific applications, however, make use of data- or simulation-driven models for which computing the likelihood ratio can be very difficult or even impossible. By applying the so-called “likelihood ratio trick,” approximations of the likelihood ratio may be computed using clever parametrizations of neural network-based classifiers. A number of different neural network setups can be defined to satisfy this procedure, each with varying performance in approximating the likelihood ratio when using finite training data. We present a series of empirical studies detailing the performance of several common loss functionals and parametrizations of the classifier output in approximating the likelihood ratio of two univariate and multivariate Gaussian distributions as well as simulated high-energy particle physics datasets.
72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
Planck 2018 results. V. CMB power spectra and likelihoods
We describe the legacy Planck cosmic microwave background (CMB) likelihoods derived from the 2018 data release. The overall approach is similar in spirit to the one retained for the 2013 and 2015 data release, with a hybrid method using different approximations at low ( ℓ < 30) and high ( ℓ ≥ 30) multipoles, implementing several methodological and data-analysis refinements compared to previous releases. With more realistic simulations, and better correction and modelling of systematic effects, we can now make full use of the CMB polarization observed in the High Frequency Instrument (HFI) channels. The low-multipole EE cross-spectra from the 100 GHz and 143 GHz data give a constraint on the ΛCDM reionization optical-depth parameter τ to better than 15% (in combination with the TT low- ℓ data and the high- ℓ temperature and polarization data), tightening constraints on all parameters with posterior distributions correlated with τ . We also update the weaker constraint on τ from the joint TEB likelihood using the Low Frequency Instrument (LFI) channels, which was used in 2015 as part of our baseline analysis. At higher multipoles, the CMB temperature spectrum and likelihood are very similar to previous releases. A better model of the temperature-to-polarization leakage and corrections for the effective calibrations of the polarization channels (i.e., the polarization efficiencies) allow us to make full use of polarization spectra, improving the ΛCDM constraints on the parameters θ MC , ω c , ω b , and H 0 by more than 30%, and n s by more than 20% compared to TT-only constraints. Extensive tests on the robustness of the modelling of the polarization data demonstrate good consistency, with some residual modelling uncertainties. At high multipoles, we are now limited mainly by the accuracy of the polarization efficiency modelling. Using our various tests, simulations, and comparison between different high-multipole likelihood implementations, we estimate the consistency of the results to be better than the 0.5 σ level on the ΛCDM parameters, as well as classical single-parameter extensions for the joint likelihood (to be compared to the 0.3 σ levels we achieved in 2015 for the temperature data alone on ΛCDM only). Minor curiosities already present in the previous releases remain, such as the differences between the best-fit ΛCDM parameters for the ℓ < 800 and ℓ > 800 ranges of the power spectrum, or the preference for more smoothing of the power-spectrum peaks than predicted in ΛCDM fits. These are shown to be driven by the temperature power spectrum and are not significantly modified by the inclusion of the polarization data. Overall, the legacy Planck CMB likelihoods provide a robust tool for constraining the cosmological model and represent a reference for future CMB observations.
79 ASTRONOMY AND ASTROPHYSICS↗
Effect of likelihood misspecification in Gaussian process-driven autonomous experimentation
In recent years, several groups have designed Autonomous Experiment (AE) models with the aim of using them as an alternative method for neutron scattering scanning. In an AE, Gaussian processes (GPs) are most frequently used due to their interpretability, their non-parametric nature, their universal approximation, and their closed-form predictive distribution. GPs have two key components, namely, the model for the likelihood of a neutron count knowing the underlying dynamic structure factor and the acquisition function. In this paper, we investigate the impact, on the quality of an AE, of the likelihood and acquisition function choices, in energy scans and (Q, ω) ones, with respect to the signal-over-noise ratio. While we hypothesized that the quality of GP predictions would decrease when the normal to Poisson likelihood approximation breaks down at low count rates, we found that the use of the correct Poisson likelihood does not improve the quality of the data collected, as well as yields very poor results in (Q, ω) scans at low count rates. In fact, the best results are obtained with a combination of normal likelihood, including the observation noise, and the change in variance acquisition function. In addition, we find that the performance, or quality of the predictive distribution, is a misleading measure of efficiency, that is, of the quality of the data collected.
Perryman, David Elliott [Inst. Laue-Langevin (ILL)↗
Amortized simulation-based frequentist inference for tractable and intractable likelihoods
Abstract High-fidelity simulators that connect theoretical models with observations are indispensable tools in many sciences. If the likelihood is known, inference can proceed using standard techniques. However, when the likelihood is intractable or unknown, a simulator makes it possible to infer the parameters of a theoretical model directly from real and simulated observations when coupled with machine learning. We introduce an extension of the recently proposed likelihood-free frequentist inference ( LF2I ) approach that makes it possible to construct confidence sets with the p -value function and to use the same function to check the coverage explicitly at any given parameter point. Like LF2I , this extension yields provably valid confidence sets in parameter inference problems for which a high-fidelity simulator is available. The utility of our algorithm is illustrated by applying it to three pedagogically interesting examples: the first is from cosmology, the second from high-energy physics and astronomy, both with tractable likelihoods, while the third, with an intractable likelihood, is from epidemiology 3 3 Code to reproduce all of our results is available on https://github.com/AliAlkadhim/ALFFI . .
Computer Science↗
Deep Learning and Likelihood Approaches for Viral Phylogeography Converge on the Same Answers Whether the Inference Model Is Right or Wrong
Abstract Analysis of phylogenetic trees has become an essential tool in epidemiology. Likelihood-based methods fit models to phylogenies to draw inferences about the phylodynamics and history of viral transmission. However, these methods are often computationally expensive, which limits the complexity and realism of phylodynamic models and makes them ill-suited for informing policy decisions in real-time during rapidly developing outbreaks. Likelihood-free methods using deep learning are pushing the boundaries of inference beyond these constraints. In this paper, we extend, compare, and contrast a recently developed deep learning method for likelihood-free inference from trees. We trained multiple deep neural networks using phylogenies from simulated outbreaks that spread among 5 locations and found they achieve close to the same levels of accuracy as Bayesian inference under the true simulation model. We compared robustness to model misspecification of a trained neural network to that of a Bayesian method. We found that both models had comparable performance, converging on similar biases. We also implemented a method of uncertainty quantification called conformalized quantile regression that we demonstrate has similar patterns of sensitivity to model misspecification as Bayesian highest posterior density (HPD) and greatly overlap with HPDs, but have lower precision (more conservative). Finally, we trained and tested a neural network against phylogeographic data from a recent study of the SARS-Cov-2 pandemic in Europe and obtained similar estimates of region-specific epidemiological parameters and the location of the common ancestor in Europe. Along with being as accurate and robust as likelihood-based methods, our trained neural networks are on average over 3 orders of magnitude faster after training. Our results support the notion that neural networks can be trained with simulated data to accurately mimic the good and bad statistical properties of the likelihood functions of generative phylogenetic models.
Evolutionary Biology↗
Role of the likelihood for elastic scattering uncertainty quantification
In the last decade, uncertainty quantification (UQ) for optical model potentials (OMPs) has become a focal point for nuclear reaction theory, and several competing approaches for OMP UQ have recently been developed. Here, we clarify recent efforts to compare frequentist and Bayesian approaches in the context of OMP UQ [G. B. King et al., Phys. Rev. Lett. 122, 232502 (2019)]. We replicate a portion of that OMP UQ study but use independent statistical tools. Specifically, we compare two methods for OMP parameter inference from elastic scattering data: the Levenberg-Marquardt algorithm for χ 2 minimization on one hand and Markov chain Monte Carlo (MCMC) sampling on the other. Separately, we assess the common practice of using a renormalized likelihood (χ 2 /N), N being the number of data points, instead of the canonical weighted-least-squares likelihood (χ 2 ), as a way of accounting for unknown data correlations. Here, we show that for a generic linear model and for a five-parameter OMP analysis, frequentist and uniform-prior Bayesian approaches recover the same optimum and uncertainty estimates—not systematically larger uncertainties for the Bayesian approach, as was concluded in G. B. King et al., Phys. Rev. Lett. 122, 232502 (2019). Further, we show that if an additional, near-degenerate parameter is introduced into the same OMP analysis such that the parameter posterior becomes non-Gaussian, then covariance-based estimates of uncertainty become unreliable. Finally, we show that regardless of optimization approach, if χ 2 /N is used for the likelihood, the resulting parametric uncertainties increase by $\sqrt{N}$, and that this is responsible for the conclusions drawn in the revisited study. Based on our replication results, we find that a fortuitous cancellation of unreported errors and the renormalization factor can lead to improvement in empirical coverages, as was the case in the original comparative study. We emphasize that developing and applying a realistic likelihood function is an essential task in a UQ analysis, and that several recent UQ studies that employed a renormalized likelihood (i.e., including a 1/N factor) may have yielded unrealistically large uncertainties for elastic-scattering observables. If the parameter posterior deviates from multivariate-normal, a sampling-based approach like MCMC has a clear advantage over methods that assume the Laplace approximation holds. We note that empirical coverage can serve as an important internal check for the analyst whose model or data may have additional, unaccounted-for uncertainties.
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
A flexible event reconstruction based on machine learning and likelihood principles
Event reconstruction is a central step in many particle physics experiments, turning detector observables into parameter estimates; for example estimating the energy of an interaction given the sensor readout of a detector. A corresponding likelihood function is often intractable, and approximations need to be constructed. Here, in our work, we first show how the full likelihood for a many-sensor detector can be broken apart into smaller terms, and secondly how we can train neural networks to approximate all terms solely based on forward simulation. Our technique results in a fast, flexible, and close-to-optimal surrogate model proportional to the likelihood and can be used in conjunction with standard inference techniques allowing for a consistent treatment of uncertainties. We illustrate our technique for parameter inference in neutrino telescopes based on maximum likelihood and Bayesian posterior sampling. Given its great flexibility, we also showcase our method for geometry optimization enabling to learn optimal detector designs. Lastly, we apply our method to realistic simulation of a ton-scale water-based liquid scintillator detector.
46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
Non -degenerate marginal-likelihood calibration with application to quantum characterization
Here, we propose a marginal likelihood strategy within the Kennedy-O’Hagan (KOH) Bayesian framework, where a Gaussian process (GP) models the discrepancy between a physical system and its simulator. Our approach introduces a novel marginalized likelihood by integrating out the degenerate eigenspace of the covariance matrix, rather than approximating the original likelihood. Unlike approximation methods that compromise accuracy for computational efficiency, our method defines an exact likelihood—distinct from the original but preserving all relevant information. This formulation achieves computational efficiency and stability, even for large datasets where the covariance matrix nears degeneracy. Applied to the characterization of a superconducting quantum device at Lawrence Livermore National Laboratory, the approach enhances the predictive accuracy of the Lindblad master equations for modeling Ramsey measurement data by effectively quantifying uncertainties consistent with the quantum data.
general physics↗
Likelihood-based signal and noise analysis for docking of models into cryo-EM maps
Fast, reliable docking of models into cryo-EM maps requires understanding of the errors in the maps and the models. Likelihood-based approaches to errors have proven to be powerful and adaptable in experimental structural biology, finding applications in both crystallography and cryo-EM. Indeed, previous crystallographic work on the errors in structural models is directly applicable to likelihood targets in cryo-EM. Likelihood targets in Fourier space are derived here to characterize, based on the comparison of half-maps, the direction- and resolution-dependent variation in the strength of both signal and noise in the data. Because the signal depends on local features, the signal and noise are analysed in local regions of the cryo-EM reconstruction. The likelihood analysis extends to prediction of the signal that will be achieved in any docking calculation for a model of specified quality and completeness. A related calculation generalizes a previous measure of the information gained by making the cryo-EM reconstruction.
59 BASIC BIOLOGICAL SCIENCES↗
Likelihood Maximization and Moment Matching in Low SNR Gaussian Mixture Models
We derive an asymptotic expansion for the log-likelihood of Gaussian mixture models (GMMs) with equal covariance matrices in the low signal-to-noise regime. The expansion reveals an intimate connection between two types of algorithms for parameter estimation: the method of moments and likelihood optimizing algorithms such as Expectation-Maximization (EM). We show that likelihood optimization in the low SNR regime reduces to a sequence of least squares optimization problems that match the moments of the estimate to the ground truth moments one by one. This connection is a stepping stone towards the analysis of EM and maximum likelihood estimation in a wide range of models. A motivating application for the study of low SNR mixture models is cryo-electron microscopy data, which can be modeled as a GMM with algebraic constraints imposed on the mixture centers. We discuss the application of our expansion to algebraically constrained GMMs, among other example models of interest. © 2022 The Authors. Communications on Pure and Applied Mathematics published by Wiley Periodicals LLC.
97 MATHEMATICS AND COMPUTING↗
Generalized framework for likelihood-based field-level inference of growth rate from velocity and density fields
Measuring the growth rate of large-scale structures ( f ) as a function of redshift has the potential to break degeneracies between modified gravity and dark energy models, when combined with expansion-rate probes. Direct estimates of peculiar velocities of galaxies have attracted interest as a means of estimating fσ 8 . In particular, field-level methods can be used to fit the field nuisance parameter along with cosmological parameters simultaneously. This article aims to provide the community with a unified framework for the theoretical modeling of the likelihood-based field-level inference by performing fast field covariance calculations for velocity and density fields. Our purpose is to lay the foundations for a nonlinear extension of the likelihood-based method at the field level. We have developed a generalized framework, implemented in the dedicated software flip to perform a likelihood-based inference of fσ 8 . We derived a new field covariance model, which includes wide-angle corrections. We also included the models previously described in the literature inside our framework. We compared their performance against ours, and we validated our model by comparing it with the two-point statistics of a recent N-body simulation. The tests we performed have allowed us to validate our software and determine the appropriate wavenumber range to integrate our covariance model and its validity in terms of separation. Our framework allows for a wider wavenumber coverage to be used in our calculations than in previous works, which is particularly interesting for nonlinear model extensions. Finally, our generalized framework allows us to efficiently perform a survey geometry-dependent Fisher forecast of the fσ 8 parameter. We show that the Fisher forecast method we developed gives an error bar that is 30% closer to a full likelihood-based estimation than a standard volume Fisher forecast.
Ravoux, Corentin↗
Reionization effective likelihood from Planck 2018 data
We release relike (reionization effective likelihood), a fast and accurate effective likelihood code based on the latest Planck 2018 data that allows one to constrain any model for reionization between 6 < z < 30 using five constraints from the CMB reionization principal components (PC). We tested the code on two example models which showed excellent agreement with sampling the exact Planck likelihoods using either a simple Gaussian PC likelihood or its full kernel density estimate. This code enables a fast and consistent means for combining Planck constraints with other reionization data sets, such as kinetic Sunyaev-Zeldovich effects, line-intensity mapping, luminosity function, star formation history, quasar spectra, etc., where the redshift dependence of the ionization history is important. Since the PC technique tests any reionization history in the given range, we also derive model-independent constraints for the total Thomson optical depth τ PC = $0.0619$$^{+0.0056}_{–0.0068}$ and its 15 ≤ z ≤ 30 high redshift component τ PC (15,30) < 0.020 (95% C.L.). Furthermore, the upper limits on the high-redshift optical depth is a factor of ~3 larger than those reported in the Planck 2018 cosmological parameter paper using the FlexKnot method and we validate our results with a direct analysis of a two-step model which permits this small high-z component.
79 ASTRONOMY AND ASTROPHYSICS↗
Assessment of Bird Strike Likelihood to Refine Bird Strike Risk Models
In its most basic form, bird strike risk is comprised of a frequency component that reflects the likelihood of a collision and a severity component that reflects the cost (monetary or otherwise) of the incident. The bird strike risk model currently used by United State Department of Agriculture (USDA) Wildlife Services to evaluate the risk posed by individual bird species at airports and establish priorities for management was developed in 2018. The model uses airport-specific data on the number of reported strikes for a species recorded in the Federal Aviation Administration (FAA)’s National Wildlife Strike Database as a measure of frequency and the species’ relative hazard score as a measure of severity. The model was tested against independent data, found to perform well overall, and is being implemented widely across the United States. However, the model has limitations, including that species known to pose risk to aircraft locally, but not present in the strike record database, are not reflected as a major component of risk. Standard bird survey methodology commonly used at airports (e.g. point counts or transects) potentially can be used to complement wildlife strike records to calculate frequency or relative abundance of species. However, these methods generally focus on airport-wide population estimation and often ignore vital information that contributes to the true likelihood of a strike, such as use of runway protection zones and other critical areas, and spatial and temporal overlap with departing or approaching aircraft. As such, a more detailed understanding of space use by birds across landcovers and population fluctuations across the year is needed to accurately estimate the likelihood of bird strikes at airports. In this manuscript, we will review the extant risk model, including a discussion on its limitations. We then discuss approaches for refining our understanding of strike likelihood and briefly touch on needs for estimating probability of strike severity (cost).
bird strike, aircraft collision, damage by wildlif↗
A dynamic likelihood approach to filtering transport processes: advection-diffusion dynamics
A Bayesian data assimilation scheme is formulated for advection-dominated advective and diffusive evolutionary problems, based upon the Dynamic Likelihood (DLF) approach to filtering. The DLF was developed specifically for hyperbolic problems –waves–, and in this paper, it is extended via a split step formulation, to handle advection-diffusion problems. In the dynamic likelihood approach, observations and their statistics are used to propagate probabilities along characteristics, evolving the likelihood in time. The estimate posterior thus inherits phase information. For advection-diffusion the advective part of the time evolution is handled on the basis of observations alone, while the diffusive part is informed through the model as well as observations. We expect, and indeed show here, that in advection-dominated problems, the DLF approach produces better estimates than other assimilation approaches, particularly when the observations are sparse and have low uncertainty. The added computational expense of the method is cubic in the total number of observations over time, which is on the same order of magnitude as a standard Kalman filter and can be mitigated by bounding the number of forward propagated observations, discarding the least informative data.
97 MATHEMATICS AND COMPUTING↗
Likelihood-based docking of models into cryo-EM maps
Optimized docking of models into cryo-EM maps requires exploiting an understanding of the signal expected in the data to minimize the calculation time while maintaining sufficient signal. The likelihood-based rotation function used in crystallography can be employed to establish plausible orientations in a docking search. A phased likelihood translation function yields scores for the placement and rigid-body refinement of oriented models. Optimized strategies for choices of the resolution of data from the cryo-EM maps to use in the calculations and the size of search volumes are based on expected log-likelihood-gain scores computed in advance of the search calculation. Tests demonstrate that the new procedure is fast, robust and effective at placing models into even challenging cryo-EM maps.
59 BASIC BIOLOGICAL SCIENCES↗