Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “FREQUENTIST METHODS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Data from: Comparing traditional and Bayesian approaches to ecological meta-analysis

Despite the wide application of meta-analysis in ecology, some of the traditional methods used for meta-analysis may not perform well given the type of data characteristic of ecological meta-analyses. We reviewed published meta-analyses on the ecological impacts of global climate change, evaluating the number of replicates used in the primary studies (ni) and the number of studies or records (k) that were aggregated to calculate a mean effect size. We used the results of the review in a simulation experiment to assess the performance of conventional frequentist and Bayesian meta-analysis methods for estimating a mean effect size and its uncertainty interval. Our literature review showed that ni and k were highly variable, distributions were right-skewed, and were generally small (median ni =5, median k=44). Our simulations show that the choice of method for calculating uncertainty intervals was critical for obtaining appropriate coverage (close to the nominal value of 0.95). When k was low (<40), 95% coverage was achieved by a confidence interval based on the t-distribution that uses an adjusted standard error (the Hartung-Knapp-Sidik-Jonkman, HKSJ), or by a Bayesian credible interval, whereas bootstrap or z-distribution confidence intervals had lower coverage. Despite the importance of the method to calculate the uncertainty interval, 39% of the meta-analyses reviewed did not report the method used, and of the 61% that did, 94% used a potentially problematic method, which may be a consequence of software defaults. In general, for a simple random-effects meta-analysis, the performance of the best frequentist and Bayesian methods were similar for the same combinations of factors (k and mean replication), though the Bayesian approaches had higher than nominal (>95%) coverage for the mean effect when k was very low (k<15). Our literature review suggests that many meta-analyses that used z-distribution or bootstrapping confidence intervals may have over-estimated the statistical significance of their results when the number of studies was low; more appropriate methods need to be adopted in ecological meta-analyses.

54 ENVIRONMENTAL SCIENCES↗

Analysis of the theoretical bias in dark matter direct detection

Fitting the model ''A'' to dark matter direct detection data, when the model that underlies the data is ''B'', introduces a theoretical bias in the fit. We perform a quantitative study of the theoretical bias in dark matter direct detection, with a focus on assumptions regarding the dark matter interactions, and velocity distribution. We address this problem within the effective theory of isoscalar dark matter-nucleon interactions mediated by a heavy spin-1 or spin-0 particle. We analyze 24 benchmark points in the parameter space of the theory, using frequentist and Bayesian statistical methods. First, we simulate the data of future direct detection experiments assuming a momentum/velocity dependent dark matter-nucleon interaction, and an anisotropic dark matter velocity distribution. Then, we fit a constant scattering cross section, and an isotropic Maxwell-Boltzmann velocity distribution to the simulated data, thereby introducing a bias in the analysis. The best fit values of the dark matter particle mass differ from their benchmark values up to 2 standard deviations. The best fit values of the dark matter-nucleon coupling constant differ from their benchmark values up to several standard deviations. We conclude that common assumptions in dark matter direct detection are a source of potentially significant bias.

79 ASTRONOMY AND ASTROPHYSICS↗

LOCAL NON-GAUSSIANITY IN THE COSMIC MICROWAVE BACKGROUND THE BAYESIAN WAY

We introduce an exact Bayesian approach to search for non-Gaussianity of local type in cosmic microwave background (CMB) radiation data. Using simulated CMB temperature maps, the newly developed technique is compared against the conventional frequentist bispectrum estimator. Starting from the joint probability distribution, we obtain analytic expressions for the conditional probabilities of the primordial perturbations given the data, and for the level of non-Gaussianity, f{sub NL}, given the data and the perturbations. We propose Hamiltonian Monte Carlo sampling as a means to derive realizations of the primordial fluctuations from which we in turn sample f{sub NL}. Although computationally expensive, this approach allows us to construct exactly the full target posterior probability distribution. When compared to the frequentist estimator, applying the Bayesian method to Gaussian CMB maps provides consistent results. For the analysis of non-Gaussian maps, however, the error bars on f{sub NL} do not show excess variance within the Bayesian framework. This finding is of particular relevance in the light of upcoming high-precision CMB measurements obtained by the Planck satellite mission.

79 ASTRONOMY AND ASTROPHYSICS↗

Search for Muon Neutrino Disappearance in the Booster Neutrino Beam of Fermilab; Busqueda de Desaparicion de Neutrinos del Muon en el Haz de Neutrinos del Booster de Fermilab

In this work we carried out the disappearance analysis of muon neutrinos produced in the Fermilab Booster Neutrino Beam, using the data released to the public by the collaborations of the MiniBooNE and SciBooNE experiments. The calculations were made with programs in C and C++, implementing the ROOT libraries. From the analysis, using both the classical Pearson method and the Feldman and Cousins frequentist corrections, we obtained the 90\% C.L. limit for the oscillation parameters sin 2 2θ and Δm 2 in the region 0.1 ≤ Δm 2 ≤ 10 eV 2 using a two neutrino model. The result presented in this work is consistent with the official one, with small deviations ascribed to round-off errors in the format of the used data, as well as statistical fluctuations in the generation of fake experiments used in the Feldman and Cousins method. As the official one, our result is consistent with the null oscillation hypothesis. This work was carried out independently to the MiniBooNE and SciBooNE collaborations and its results are not official.

43 PARTICLE ACCELERATORS↗

Planck 2015 results: XX. Constraints on inflation

In this paper, we present the implications for cosmic inflation of the Planck measurements of the cosmic microwave background (CMB) anisotropies in both temperature and polarization based on the full Planck survey, which includes more than twice the integration time of the nominal survey used for the 2013 release papers. The Planck full mission temperature data and a first release of polarization data on large angular scales measure the spectral index of curvature perturbations to be n s = 0.968 ± 0.006 and tightly constrain its scale dependence to dn s / dlnk = -0.003 ± 0.007 when combined with the Planck lensing likelihood. When the Planck high-ℓ polarization data are included, the results are consistent and uncertainties are further reduced. The upper bound on the tensor-to-scalar ratio is r 0.002 < 0.11 (95% CL). This upper limit is consistent with the B-mode polarization constraint r< 0.12 (95% CL) obtained from a joint analysis of the BICEP2/Keck Array and Planck data. These results imply that V(φ) ∝ φ 2 and natural inflation are now disfavoured compared to models predicting a smaller tensor-to-scalar ratio, such as R 2 inflation. We search for several physically motivated deviations from a simple power-law spectrum of curvature perturbations, including those motivated by a reconstruction of the inflaton potential not relying on the slow-roll approximation. We find that such models are not preferred, either according to a Bayesian model comparison or according to a frequentist simulation-based analysis. Three independent methods reconstructing the primordial power spectrum consistently recover a featureless and smooth P R (k)over the range of scales 0.008 Mpc -1 ≲ k ≲ 0.1 Mpc -1 . At large scales, each method finds deviations from a power law, connected to a deficit at multipoles ℓ ≈ 20-40 in the temperature power spectrum, but at an uncompelling statistical significance owing to the large cosmic variance present at these multipoles. By combining power spectrum and non-Gaussianity bounds, we constrain models with generalized Lagrangians, including Galileon models and axion monodromy models. The Planck data are consistent with adiabatic primordial perturbations, and the estimated values for the parameters of the base Λ cold dark matter (ΛCDM) model are not significantly altered when more general initial conditions are admitted. In correlated mixed adiabatic and isocurvature models, the 95% CL upper bound for the non-adiabatic contribution to the observed CMB temperature variance is | α non - adi | < 1.9%, 4.0%, and 2.9% for CDM, neutrino density, and neutrino velocity isocurvature modes, respectively. We have tested inflationary models producing an anisotropic modulation of the primordial curvature power spectrum finding that the dipolar modulation in the CMB temperature field induced by a CDM isocurvature perturbation is not preferred at a statistically significant level. We also establish tight constraints on a possible quadrupolar modulation of the curvature perturbation. Lastly, these results are consistent with the Planck 2013 analysis based on the nominal mission data and further constrain slow-roll single-field inflationary models, as expected from the increased precision of Planck data using the full set of observations.

79 ASTRONOMY AND ASTROPHYSICS↗

Direct Comparison between Bayesian and Frequentist Uncertainty Quantification for Nuclear Reactions

Until recently, uncertainty quantification in low energy nuclear theory was typically performed using frequentist approaches. However in the last few years, the field has shifted toward Bayesian statistics for evaluating confidence intervals. Although there are statistical arguments to prefer the Bayesian approach, no direct comparison is available. In this work, we compare, directly and systematically, the frequentist and Bayesian approaches to quantifying uncertainties in direct nuclear reactions. Starting from identical initial assumptions, we determine confidence intervals associated with the elastic and the transfer process for both methods, which are evaluated against data via a comparison of the empirical coverage probabilities. Expectedly, the frequentist approach is not as flexible as the Bayesian approach in exploring parameter space and often ends up in a different minimum. We also show that the two methods produce significantly different correlations. In the end, the frequentist approach produces significantly narrower uncertainties on the considered observables than the Bayesian. Lastly, our study demonstrates that the uncertainties on the reaction observables considered here within the Bayesian approach represent reality more accurately than the much narrower uncertainties obtained using the standard frequentist approach.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Modified frequentist determination of confidence intervals for poisson distribution

We propose modified frequentist definition for the determination of confidence intervals for the case of Poisson statistics. Namely, we require that 1−β{sup ′}⩾∑{sub n=0}{sup n=n{sub o}{sub b}{sub s}+k}P(n|λ)⩾α{sup ′}. We show that this definition is equivalent to the Bayesian method with prior π(λ) ∼ λ{sup k}. We also propose modified frequentist definition for the case of nonzero background.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Frequentist cosmological constraints from full-shape clustering measurements in DESI DR1

We present a frequentist analysis of clustering measurements from Data Release 1 of the Dark Energy Spectroscopic Instrument (DESI) using the standard profile likelihood method. While Bayesian inferences for effective field theory models of galaxy clustering can be highly sensitive to prior choices for extended cosmological models, frequentist inferences are not susceptible to such effects. We compare frequentist and Bayesian constraints for the parameter set {σ 8 , H 0 , Ω m , w 0 , w a } using the full-shape power spectrum multipoles, post-reconstruction baryon acoustic oscillation (BAO) measurements, and external datasets from the CMB and type Ia supernovae measurements. The frequentist confidence intervals are significantly shifted relative to the Bayesian credible intervals for the w 0 w a CDM model, unless supernovae data are included. When DESI full-shape and BAO data are fit jointly, we obtain the following 1σ frequentist confidence intervals for ΛCDM (w 0 w a CDM): σ 8 = 0.863 +0.048 -0.040 , H 0 = 68.96 +0.81 -0.80 km s -1 Mpc -1 , Ω m = 0.3034 ± 0.0110 (σ 8 = 0.782 +0.060 -0.036 , H 0 = 63.7 +4.2 -2.0 km s -1 Mpc -1 , Ω m = 0.378 +0.024 -0.047 , w 0 = -0.16 +0.10 -0.50 , w a = -3.0 +1.7 ), corresponding to 0.8σ, 0.3σ, 0.7σ (2.1σ, 4.1σ, 6.5σ, 6.3σ, 6.6σ) shifts between the maximum likelihood estimate and the Bayesian posterior mean for ΛCDM (w 0 w a CDM) respectively.

Bayesian reasoning↗

The use of nested sampling in the extraction of polarisation observables at CLAS

The extraction of polarisation observables from photoproduction experiments provides an insight into the spectrum of nucleon resonances and the 'missing resonance' problem. Experiments carried out at JLab, Mainz and Bonn cover a wide range of reactions, which will soon result in the first 'complete measurement' in pseudoscalar meson photoproduction. Traditionally, these measurements have been analysed using frequentist statistics, where parameters are extracted by fitting distributions. An alternative method is the application of Bayesian statistics, where any existing knowledge about the results can be used in the initial conditions. One such application of this is nested sampling. This work discusses nested sampling and how it can be applied to the extraction of spin observables.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

XRD-PUAT (X-Ray Diffraction - Parameter Uncertainty Analysis Toolkit)

This XRD uncertainty toolkit was built to investigate uncertainty and local least squares topology over a specified parameter space on a single histogram in a gpx GSAS-II file. The parameter uncertainty can be investigated using either a frequentist F-test approach or a Bayesian Inference statistical inversion method on a weighted least squares or peak fit refinements.SAND Number: SAND2020-12227 M Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Moore, Alexander↗

Probing the consistency of cosmological contours for supernova cosmology

As the scale of cosmological surveys increases, so does the complexity in the analyses. This complexity can often make it difficult to derive the underlying principles, necessitating statistically rigorous testing to ensure the results of an analysis are consistent and reasonable. This is particularly important in multi-probe cosmological analyses like those used in the Dark Energy Survey (DES) and the upcoming Legacy Survey of Space and Time, where accurate uncertainties are vital. In this paper, we present a statistically rigorous method to test the consistency of contours produced in these analyses and apply this method to the Pippin cosmological pipeline used for type Ia supernova cosmology with the DES. We make use of the Neyman construction, a frequentist methodology that leverages extensive simulations to calculate confidence intervals, to perform this consistency check. A true Neyman construction is too computationally expensive for supernova cosmology, so we develop a method for approximating a Neyman construction with far fewer simulations. We find that for a simulated dataset, the 68% contour reported by the Pippin pipeline and the 68% confidence region produced by our approximate Neyman construction differ by less than a percent near the input cosmology; however, they show more significant differences far from the input cosmology, with a maximal difference of 0.05 in $Ω$ M and 0.07 in w. In conclusion, this divergence is most impactful for analyses of cosmological tensions, but its impact is mitigated when combining supernovae with other cross-cutting cosmological probes, such as the cosmic microwave background.

79 ASTRONOMY AND ASTROPHYSICS↗

Quantifying Safety Margin Using the Risk-Informed Safety Margin Characterization (RISMC)

The Risk-Informed Safety Margin Characterization (RISMC), developed by Idaho National Laboratory as part of the Light-Water Reactor Sustainability Project, utilizes a probabilistic safety margin comparison between a load and capacity distribution, rather than a deterministic comparison between two values, as is usually done in best-estimate plus uncertainty analyses. The goal is to determine the failure probability, or in other words, the probability of the system load equaling or exceeding the system capacity. While this method has been used in pilot studies, there has been little work conducted investigating the statistical significance of the resulting failure probability. In particular, it is difficult to determine how many simulations are necessary to properly characterize the failure probability. This work uses classical (frequentist) statistics and confidence intervals to examine the impact in statistical accuracy when the number of simulations is varied. Two methods are proposed to establish confidence intervals related to the failure probability established using a RISMC analysis. The confidence interval provides information about the statistical accuracy of the method utilized to explore the uncertainty space, and offers a quantitative method to gauge the increase in statistical accuracy due to performing additional simulations.

Grabaskas, David↗

Improvement of Drop‐Hammer Impact Testing for Safety Assessment of High Explosives Using 10‐mg Samples

Here, in this study, we established an improved method for drop-hammer impact testing of small quantities of high explosives (10 mg). We performed about seven hundred impact tests under various experimental conditions (e.g., sandpaper vs bare anvil, different sample masses, drop-weights, and striker diameters) to determine an optimal set of conditions and reaction detection methods (e.g., gas analysis, video, and sound recordings) that give the most statistically reliable results with 10 mg samples. We used both Frequentist and Bayesian statistical approaches to compare estimates of the drop height (DH50) that initiates a reaction 50% of the time, and to quantify the associated uncertainty. Gas analysis proved to be the most reliable reaction detection method, showing unambiguous rises in HE decomposition products (e.g., CO 2 ) even when the other indicators (e.g., sound, video) were inconclusive. The impact tests performed with a bare anvil showed much better reproducibility than those conducted with sandpaper, reducing the largest uncertainty observed in the data sets by a factor of 1.7. The DH 50 values obtained from three different sample masses (10, 20, and 35 mg) fell within the uncertainties of the measurements. We demonstrated the improved procedure (i.e., 10-mg samples, gas analysis, bare anvil, and Bayesian approach) on a variety of PETN samples having different surface areas and thermal histories.

PETN↗

Kumaraswamy distribution: different methods of estimation

This paper addresses different methods of estimation of the unknown parameters of a two-parameter Kumaraswamy distribution from a frequentist point of view. We briefly describe ten different frequentist approaches, namely, maximum likelihood estimators, moments estimators, L-moments estimators, percentile based estimators, least squares estimators, weighted least squares estimators, maximum product of spacings estimators, Cramér–von-Mises estimators, Anderson–Darling estimators and right tailed Anderson–Darling estimators. Monte Carlo simulations and two real data applications are performed to compare the performances of the estimators for both small and large samples.

97 MATHEMATICS AND COMPUTING↗

Informed total-error-minimizing priors: Interpretable cosmological parameter constraints despite complex nuisance effects

While Bayesian inference techniques are standard in cosmological analyses, it is common to interpret resulting parameter constraints with a frequentist intuition. This intuition can fail, for example, when marginalizing high-dimensional parameter spaces onto subsets of parameters, because of what has come to be known as projection effects or prior volume effects. We present the method of informed total-error-minimizing (ITEM) priors to address this problem. An ITEM prior is a prior distribution on a set of nuisance parameters, such as those describing astrophysical or calibration systematics, intended to enforce the validity of a frequentist interpretation of the posterior constraints derived for a set of target parameters (e.g., cosmological parameters). Our method works as follows. For a set of plausible nuisance realizations, we generate target parameter posteriors using several different candidate priors for the nuisance parameters. We reject candidate priors that do not accomplish the minimum requirements of bias (of point estimates) and coverage (of confidence regions among a set of noisy realizations of the data) for the target parameters on one or more of the plausible nuisance realizations. Of the priors that survive this cut, we select the ITEM prior as the one that minimizes the total error of the marginalized posteriors of the target parameters. As a proof of concept, we applied our method to the density split statistics measured in Dark Energy Survey Year 1 data. We demonstrate that the ITEM priors substantially reduce prior volume effects that otherwise arise and that they allow for sharpened yet robust constraints on the parameters of interest.

79 ASTRONOMY AND ASTROPHYSICS↗

A novel framework for increasing research transparency: Exploring the connection between diversity and innovation

A split sample/dual method research protocol is demonstrated to increase transparency while reducing the probability of false discovery. We apply the protocol to examine whether diversity in ownership teams increases or decreases the likelihood of a firm reporting a novel innovation using data from the 2018 United States Census Bureau’s Annual Business Survey. Transparency is increased in three ways: 1) all specification testing and identifying potentially productive models is done in an exploratory subsample that 2) preserves the validity of hypothesis test statistics fromde novoestimation in the holdout confirmatory sample with 3) all findings publicly documented in an earlier registered report and in this journal publication. Bayesian estimation procedures that leverage information from the exploratory stage included in the confirmatory stage estimation replace traditional frequentist null hypothesis significance testing. In addition to increasing statistical power by using information from the full sample, Bayesian methods directly estimate a probability distribution for the magnitude of an effect, allowing much richer inference. Estimated magnitudes of diversity along academic discipline, race, ethnicity, and foreign-born status dimensions are positively associated with innovation. A maximally diverse ownership team on these dimensions would be roughly six times more likely to report new-to-market innovation than a homophilic team.

Science & Technology - Other Topics↗

Cosmological neutrino mass: a frequentist overview in light of DESI

We derive constraints on the neutrino mass using a variety of recent cosmological datasets, including DESI BAO, the full-shape analysis of the DESI matter power spectrum and the one-dimensional power spectrum of the Lyman-α forest (P1D) from eBOSS quasars as well as the cosmic microwave background (CMB). The constraints are obtained in the frequentist formalism by constructing profile likelihoods and applying the Feldman-Cousins prescription to compute confidence intervals. This method avoids potential prior and volume effects that may arise in a comparable Bayesian analysis. Parabolic fits to the profiles allow one to distinguish changes in the upper limits from variations in the constraining power σ of the different data combinations. We find that all profiles in the ΛCDM model are cut off by the ∑m ν ≥ 0 bound, meaning that the corresponding parabolas reach their minimum in the unphysical sector. The most stringent 95% C.L. upper limit is obtained by the combination of DESI DR2 BAO, Planck PR4 and CMB lensing at 53 meV, below the minimum of 59 meV set by the normal ordering. The corresponding constraining power σ is 43 meV, which highlights the importance of the cut-off by negative values in the determination of the upper limit. Extending ΛCDM to non-zero curvature and w 0 w a CDM relaxes the constraints past 59 meV again, but only w 0 w a CDM exhibits profiles with a minimum at a positive value. Additionally, we extend the formalism to constrain the lightest neutrino mass. For DESI DR2 BAO, Planck PR4 and CMB lensing, we find confidence limits at 20 and 19 meV for normal and inverted ordering, respectively. Using a combination of DESI DR1 full-shape, BBN and eBOSS Lyman-α P1D, we successfully constrain the neutrino mass independently of the CMB. This combination yields m l ≤ 97 and 98 meV in the normal and inverted orderings, and total neutrino mass ∑m ν ≤ 285 meV (95% C.L.). The addition of DESI full-shape or Lyman-α P1D to CMB and DESI BAO results in small but noticeable improvement of the constraining power of the data. Lyman-α free-streaming measurements especially improve the constraint. Since they are based on eBOSS data, this sets a promising precedent for upcoming DESI data.

Frequentist statistics↗

The look-elsewhere effect from a unified Bayesian and frequentist perspective

When searching over a large parameter space for anomalies such as events, peaks, objects, or particles, there is a large probability that spurious signals with seemingly high signi ficance will be found. This is known as the look-elsewhere effect and is prevalent throughout cosmology, (astro)particle physics, and beyond. To avoid making false claims of detection, one must account for this effect when assigning the statistical significance of an anomaly. This is typically accomplished by considering the trials factor, which is generally computed numerically via potentially expensive simulations. In this paper we develop a continuous generalization of the Bonferroni and Sidak corrections by applying the Laplace approximation to evaluate the Bayes factor, and in turn relating the trials factor to the prior-to-posterior volume ratio. Here, we use this to define a test statistic whose frequentist properties have a simple interpretation in terms of the global p-value, or statistical significance. We apply this method to various physics-based examples and show it to work well for the full range of p-values, i.e. in both the asymptotic and non-asymptotic regimes. We also show that this method naturally accounts for other model complexities such as additional degrees of freedom, generalizing Wilks' theorem. This provides a fast way to quantify statistical significance in light of the look-elsewhere effect, without resorting to expensive simulations.

79 ASTRONOMY AND ASTROPHYSICS↗