Second-order statistics of alternated stepwise processes.
Generalization of binary processes, discussing time domain and frequency domain analysis
STATISTICAL ANALYSIS↗
SEARCH · Engineering Papers
Results for “ordered statistics”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
A road map to cosmological parameter analysis with third-order shear statistics: III. Efficient estimation of third-order shear correlation functions and an application to the KiDS-1000 data
Context. Third-order lensing statistics contain a wealth of cosmological information that is not captured by second-order statistics. However, the computational effort it takes to estimate such statistics in forthcoming stage IV surveys is prohibitively expensive. Aims. We derive and validate an efficient estimation procedure for the three-point correlation function (3PCF) of polar fields such as weak lensing shear. We then use our approach to measure the shear 3PCF and the third-order aperture mass statistics on the KiDS-1000 survey. Methods We constructed an efficient estimator for third-order shear statistics that builds on the multipole decomposition of the 3PCF. We then validated our estimator on mock ellipticity catalogs obtained from N -body simulations. Finally, we applied our estimator to the KiDS-1000 data and presented a measurement of the third-order aperture statistics in a tomographic setup. Results. Our estimator provides a speedup of a factor of ∼100–1000 compared to the state-of-the-art estimation procedures. It is also able to provide accurate measurements for squeezed and folded triangle configurations without additional computational effort. We report a significant detection of tomographic third-order aperture mass statistics in the KiDS-1000 data (S/N = 6.69). Conclusions. Our estimator will make it computationally feasible to measure third-order shear statistics in forthcoming stage IV surveys. Furthermore, it can be used to construct empirical covariance matrices for such statistics.
Astronomy & Astrophysics↗
KiDS-1000 cosmology: Combined second- and third-order shear statistics
Aims.In this work, we perform the first cosmological parameter analysis of the fourth release of Kilo Degree Survey (KiDS-1000) data with second- and third-order shear statistics. This paper builds on a series of studies aimed at describing the roadmap to third-order shear statistics. Methods.We derived and tested a combined model of the second-order shear statistic, namely, the COSEBIs and the third-order aperture mass statistics 〈ℳ ap 3 〉 in a tomographic set-up. We validated our pipeline withN-body mock simulations of the KiDS-1000 data release. To model the second- and third-order statistics, we used the latest version of HMCODE2020 for the power spectrum and BIHALOFITfor the bispectrum. Furthermore, we used an analytic description to model intrinsic alignments and hydro-dynamical simulations to model the effect of baryonic feedback processes. Lastly, we decreased the dimension of the data vector significantly by considering only equal smoothing radii for the 〈ℳ ap 3 〉 part of the data vector. This makes it possible to carry out a data analysis of the KiDS-1000 data release using a combined analysis of COSEBIs and third-order shear statistics. Results.We first validated the accuracy of our modelling by analysing a noise-free mock data vector, assuming the KiDS-1000 error budget, finding a shift in the maximum of the posterior distribution of the matter density parameter, ΔΩ m < 0.02 σ Ω m , and of the structure growth parameter, ΔS 8 < 0.05 σ S 8 . Lastly, we performed the first KiDS-1000 cosmological analysis using a combined analysis of second- and third-order shear statistics, where we constrained Ω m = 0.248 −0.055 +0.062 andS 8 = σ 8 √(Ω m /0.3 )= 0.772 ± 0.022. The geometric average on the errors of Ω m andS 8 of the combined statistics decreases, compared to the second-order statistic, by a factor of 2.2.
Astronomy & Astrophysics↗
Dark Energy Survey Year 3 Results: Cosmological constraints from second- and third-order shear statistics
Here, we present a cosmological analysis of the third-order aperture mass statistic using Dark Energy Survey Year 3 (DES Y3) data. We perform a complete tomographic measurement of the three-point correlation function of the Y3 weak lensing shape catalog with the four fiducial source redshift bins. Building upon our companion methodology paper, we apply a pipeline that combines the two-point function ξ ± with the mass aperture skewness statistic ⟨ M ap 3 ⟩ , which is an efficient compression of the full shear three-point function. We use a suite of simulated shear maps to obtain a joint covariance matrix. By jointly analyzing ξ ± and ⟨ M ap 3 ⟩ measured from DES Y3 data with a Λ CDM model, we find S 8 = 0.780 ± 0.015 and Ω m = 0.26 6 - 0.040 + 0.039 , yielding 111% of figure-of-merit improvement in the Ω m - S 8 plane relative to ξ ± alone, consistent with expectations from simulated likelihood analyses. With a w CDM model, we find S 8 = 0.74 9 - 0.026 + 0.027 and w 0 = - 1.39 ± 0.31 , which gives an improvement of 22% on the joint S 8 - w 0 constraint. Our results are consistent with w 0 = - 1 . Our new constraints are compared to CMB data from the Planck satellite, and we find that with the inclusion of ⟨ M ap 3 ⟩ the existing tension between the datasets is at the level of 2.3 σ . We show that the third-order statistic enables us to self-calibrate the mean photometric redshift uncertainty parameter of the highest redshift bin with little degradation in the figure of merit. Our results demonstrate the constraining power of higher-order lensing statistics and establish ⟨ M ap 3 ⟩ as a practical observable for joint analyses in current and future surveys.
Gomes, R. C. H. [University of Pennsylvania] (ORCI↗
Higher Order Cumulant Studies of Ocean Surface Random Fields from Satellite Altimeter Data
Higher order statistics, especially 2nd order statistics, have been used to study ocean processes for many years in the past, and occupy an appreciable part of the research literature on physical oceanography. They in turn form part of a much larger field of study in statistical fluid mechanics.
applied statistical oceanography↗
Cosmology with second- and third-order shear statistics for the Dark Energy Survey: Methods and simulated analysis
We present a new pipeline designed for the robust inference of cosmological parameters using both second- and third-order shear statistics. We build a theoretical model for rapid evaluation of three-point correlations using our fastnc code and integrate it into the cosmosis framework. We measure the two-point functions 𝜉 ± and the full configuration-dependent three-point shear correlation functions across all auto- and cross-redshift bins. We compress the three-point functions into the mass aperture statistic ⟨ℳ$^{3}_{ap}$⟩ for a set of 796 simulated shear maps designed to model the Dark Energy Survey Year 3 data. We estimate from it the full covariance matrix and model the effects of intrinsic alignments, shear calibration biases and photometric redshift uncertainties. We apply scale cuts to minimize the contamination from the baryonic signal as modeled through hydrodynamical simulations. We find a significant improvement of 83% on the figure of merit in the Ω m − 𝑆 8 plane when we add the ⟨ℳ$^{3}_{ap}$⟩ data to 𝜉 ± . Here, we present our findings for all relevant cosmological and systematic uncertainty parameters and discuss the complementarity of third-order and second-order statistics.
79 ASTRONOMY AND ASTROPHYSICS↗
Precise cosmological constraints from BOSS galaxy clustering with a simulation-based emulator of the wavelet scattering transform
For this study, we perform a reanalysis of the BOSS CMASS DR12 galaxy dataset using a simulation-based emulator for the wavelet scattering transform (WST) coefficients. Moving beyond our previous works, which laid the foundation for the first galaxy clustering application of this estimator, we construct a neural net-based emulator for the cosmological dependence of the WST coefficients and the 2-point correlation function multipoles, trained from the state-of-the-art suite of abacussummit simulations combined with a flexible halo occupation distribution (HOD) galaxy model. In order to confirm the accuracy of our pipeline, we subject it to a series of thorough internal and external mock parameter recovery tests, before applying it to reanalyze the CMASS observations in the redshift range 0.46 < z < 0.57. We find that a joint WST+2-point correlation function likelihood analysis allows us to obtain marginalized 1σ errors on the Λ CDM parameters that are tighter by a factor of 2.5–6, compared to the 2-point correlation function, and by a factor of 1.4–2.5 compared to the WST-only results. This corresponds to a competitive 0.9%, 2.3% and 1% level of determination for parameters ω c , σ 8 &n s , respectively, and also to a 0.7% and 2.5% constraint on derived parameters h and ƒ(z)σ 8 (z), in agreement with the Planck 2018 results. Our results reaffirm the constraining power of the WST and highlight the exciting prospect of employing higher-order statistics in order to fully exploit the power of upcoming stage-IV spectroscopic observations.
79 ASTRONOMY AND ASTROPHYSICS↗
Feature extraction and classification algorithms for high dimensional data
Feature extraction and classification algorithms for high dimensional data are investigated. Developments with regard to sensors for Earth observation are moving in the direction of providing much higher dimensional multispectral imagery than is now possible. In analyzing such high dimensional data, processing time becomes an important factor. With large increases in dimensionality and the number of classes, processing time will increase significantly. To address this problem, a multistage classification scheme is proposed which reduces the processing time substantially by eliminating unlikely classes from further consideration at each stage. Several truncation criteria are developed and the relationship between thresholds and the error caused by the truncation is investigated. Next an approach to feature extraction for classification is proposed based directly on the decision boundaries. It is shown that all the features needed for classification can be extracted from decision boundaries. A characteristic of the proposed method arises by noting that only a portion of the decision boundary is effective in discriminating between classes, and the concept of the effective decision boundary is introduced. The proposed feature extraction algorithm has several desirable properties: it predicts the minimum number of features necessary to achieve the same classification accuracy as in the original space for a given pattern recognition problem; and it finds the necessary feature vectors. The proposed algorithm does not deteriorate under the circumstances of equal means or equal covariances as some previous algorithms do. In addition, the decision boundary feature extraction algorithm can be used both for parametric and non-parametric classifiers. Finally, some problems encountered in analyzing high dimensional data are studied and possible solutions are proposed. First, the increased importance of the second order statistics in analyzing high dimensional data is recognized. By investigating the characteristics of high dimensional data, the reason why the second order statistics must be taken into account in high dimensional data is suggested. Recognizing the importance of the second order statistics, there is a need to represent the second order statistics. A method to visualize statistics using a color code is proposed. By representing statistics using color coding, one can easily extract and compare the first and the second statistics.
Lee, Chulhee↗
Representation of Probability Density Functions from Orbit Determination using the Particle Filter
Statistical orbit determination enables us to obtain estimates of the state and the statistical information of its region of uncertainty. In order to obtain an accurate representation of the probability density function (PDF) that incorporates higher order statistical information, we propose the use of nonlinear estimation methods such as the Particle Filter. The Particle Filter (PF) is capable of providing a PDF representation of the state estimates whose accuracy is dependent on the number of particles or samples used. For this method to be applicable to real case scenarios, we need a way of accurately representing the PDF in a compressed manner with little information loss. Hence we propose using the Independent Component Analysis (ICA) as a non-Gaussian dimensional reduction method that is capable of maintaining higher order statistical information obtained using the PF. Methods such as the Principal Component Analysis (PCA) are based on utilizing up to second order statistics, hence will not suffice in maintaining maximum information content. Both the PCA and the ICA are applied to two scenarios that involve a highly eccentric orbit with a lower apriori uncertainty covariance and a less eccentric orbit with a higher a priori uncertainty covariance, to illustrate the capability of the ICA in relation to the PCA.
Mashiku, Alinda K.↗
The application of first order mixing statistics to the variation of the diffusivity of carbon in austenite
Carbon in austenite derived for composition dependence of diffusivity using first order mixing statistics
Mclellan, R. B.↗
Analyzing high dimensional data
Problems encountered in analyzing high dimensional data are discussed and possible solutions are proposed. The increased importance of second-order statistics in analyzing high dimensional data and the shortcoming of the minimum distance classifier in high dimensional data are recognized. By investigating characteristics of high dimensional data, it is shown that second-order statistics must be taken into account in high dimensional data. There is a need to represent second order statistics effectively. As the data dimensionality increases, it becomes more difficult to perceive and compare information present in statistics derived from data. In order to overcome this problem, a method to visualize statistics using color code is proposed. By representing statistics using a color code, the first and the second statistics can be more readily compared.
Lee, Chulhee↗
Analyzing High-Dimensional Multispectral Data
In this paper, through a series of specific examples, we illustrate some characteristics encountered in analyzing high- dimensional multispectral data. The increased importance of the second-order statistics in analyzing high-dimensional data is illustrated, as is the shortcoming of classifiers such as the minimum distance classifier which rely on first-order variations alone. We also illustrate how inaccurate estimation or first- and second-order statistics, e.g., from use of training sets which are too small, affects the performance of a classifier. Recognizing the importance of second-order statistics on the one hand, but the increased difficulty in perceiving and comprehending information present in statistics derived from high-dimensional data on the other, we propose a method to aid visualization of high-dimensional statistics using a color coding scheme.
Lee, Chulhee↗
Applying Monte Carlo Simulation to Launch Vehicle Design and Requirements Verification
This paper is focused on applying Monte Carlo simulation to probabilistic launch vehicle design and requirements verification. The approaches developed in this paper can be applied to other complex design efforts as well. Typically the verification must show that requirement "x" is met for at least "y" % of cases, with, say, 10% consumer risk or 90% confidence. Two particular aspects of making these runs for requirements verification will be explored in this paper. First, there are several types of uncertainties that should be handled in different ways, depending on when they become known (or not). The paper describes how to handle different types of uncertainties and how to develop vehicle models that can be used to examine their characteristics. This includes items that are not known exactly during the design phase but that will be known for each assembled vehicle (can be used to determine the payload capability and overall behavior of that vehicle), other items that become known before or on flight day (can be used for flight day trajectory design and go/no go decision), and items that remain unknown on flight day. Second, this paper explains a method (order statistics) for determining whether certain probabilistic requirements are met or not and enables the user to determine how many Monte Carlo samples are required. Order statistics is not new, but may not be known in general to the GN&C community. The methods also apply to determining the design values of parameters of interest in driving the vehicle design. The paper briefly discusses when it is desirable to fit a distribution to the experimental Monte Carlo results rather than using order statistics.
Hanson, John M.↗
The discrimination of sea ice types using SAR backscatter statistics
X-band (HH) synthetic aperture radar (SAR) data of sea ice collected during the Marginal Ice Zone Experiment in March and April of 1987 was statistically analyzed with respect to discriminating open water, first-year ice, multiyear ice, and Odden. Odden are large expanses of nilas ice that rapidly form in the Greenland Sea and transform into pancake ice. A first-order statistical analysis indicated that mean versus variance can segment out open water and first-year ice, and skewness versus modified skewness can segment the Odden and multilayer categories. In additions to first-order statistics, a model has been generated for the distribution function of the SAR ice data. Segmentation of ice types was also attempted using textural measurements. In this case, the general co-occurency matrix was evaluated. The textural method did not generate better results than the first-order statistical approach.
Shuchman, Robert A.↗
Towards Solving the Mixing Problem in the Decomposition of Geophysical Time Series by Independent Component Analysis
The use of the Principal Component Analysis technique for the analysis of geophysical time series has been questioned in particular for its tendency to extract components that mix several physical phenomena even when the signal is just their linear sum. We demonstrate with a data simulation experiment that the Independent Component Analysis, a recently developed technique, is able to solve this problem. This new technique requires the statistical independence of components, a stronger constraint, that uses higher-order statistics, instead of the classical decorrelation a weaker constraint, that uses only second-order statistics. Furthermore, ICA does not require additional a priori information such as the localization constraint used in Rotational Techniques.
Aires, Filipe↗
Rotation of EOFs by the Independent Component Analysis: Towards A Solution of the Mixing Problem in the Decomposition of Geophysical Time Series
The Independent Component Analysis is a recently developed technique for component extraction. This new method requires the statistical independence of the extracted components, a stronger constraint that uses higher-order statistics, instead of the classical decorrelation, a weaker constraint that uses only second-order statistics. This technique has been used recently for the analysis of geophysical time series with the goal of investigating the causes of variability in observed data (i.e. exploratory approach). We demonstrate with a data simulation experiment that, if initialized with a Principal Component Analysis, the Independent Component Analysis performs a rotation of the classical PCA (or EOF) solution. This rotation uses no localization criterion like other Rotation Techniques (RT), only the global generalization of decorrelation by statistical independence is used. This rotation of the PCA solution seems to be able to solve the tendency of PCA to mix several physical phenomena, even when the signal is just their linear sum.
Aires, Filipe↗
Rotation of EOFs by the Independent Component Analysis: Towards a Solution of the Mixing Problem in the Decomposition of Geophysical Time Series
The Independent Component Analysis is a recently developed technique for component extraction. This new method requires the statistical independence of the extracted components, a stronger constraint that uses higher-order statistics, instead of the classical decorrelation, a weaker constraint that uses only second-order statistics. This technique has been used recently for the analysis of geophysical time series with the goal of investigating the causes of variability in observed data (i.e. exploratory approach). We demonstrate with a data simulation experiment that, if initialized with a Principal Component Analysis, the Independent Component Analysis performs a rotation of the classical PCA (or EOF) solution. This rotation uses no localization criterion like other Rotation Techniques (RT), only the global generalization of decorrelation by statistical independence is used. This rotation of the PCA solution seems to be able to solve the tendency of PCA to mix several physical phenomena, even when the signal is just their linear sum.
Aires, Filipe↗
On the Stability of Jump-Linear Systems Driven by Finite-State Machines with Markovian Inputs
This paper presents two mean-square stability tests for a jump-linear system driven by a finite-state machine with a first-order Markovian input process. The first test is based on conventional Markov jump-linear theory and avoids the use of any higher-order statistics. The second test is developed directly using the higher-order statistics of the machine s output process. The two approaches are illustrated with a simple model for a recoverable computer control system.
Patilkulkarni, Sudarshan↗