Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Covariance Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 631 records · Page 35

X-ray Computed Tomography Data of Dense Metallic Components

The data shared in here are X-ray computed tomography (XCT) scans of a hexagonal fuel nozzle in 3 sections with the Metrotom 800 system at the Manufacturing Demonstration Facility (MDF) at Oak Ridge National Laboratory. The data are used in the paper "Tomographic Sparse View Selection using the View Covariance Loss, by Lin et al. (doi:10.1109/TPAMI.2025.36000720), accepted to the international conference on computational imaging (ICCP 2025). Figures 4-7 in the paper describe the part/XCT scan. File name Descriptions: Bottom section: TCR- Single Channeled SRC L 2019-3-18 12-26-41.hdf5 Medium section: TCR- Single Channeled SRC M 2019-3-18 13-8-9.hdf5 Top section: TCR- Single Channeled SRC T 2019-3-18 13-45-39.hdf5 Each hdf5 file contains projection data, and all the relevant X-ray CT scan setting. The full list of included attributes: distance_unit: Units of all distances specified angle_unit : Units of the angles angles: Array of all angles used voxel_size_xy: Baseline recon (if any) has this voxel size in the in-plane direction voxel_size_z: Baseline recon (if any) has this voxel size in the cross-plane direction det_pixel_size_col: Size of the detector pixels in the column dimension det_pixel_size_row: Size of the detector pixels in the row dimension src_iso_dist: Source to iso-center distance iso_det_dist: Iso-center to detector distance det_angle: If the detector is rotated/tilted, this angle corresponds to that value det_row_offset: Center of rotation offset in the vertical direction det_col_offset: Center of rotation offset in the horizontal direction reconstruction: A baseline reconstruction stored as 3D array BHC params: Beam-hardening parameters - Van De Casteel Model - if it has been used to pre-process the projections We also provided a python script (hdf_io.py) that allows the user to read the relevant data from each hdf5 file.

Ziabari, Amir [Oak Ridge National Laboratory]↗

Satellite and In Situ Observations for Advancing Global Earth Surface Modelling: A Review

In this paper we review the use of satellite-based remote sensing in combination with in situ data to inform Earth surface modelling. This involves verification and optimization methods that can handle both random and systematic errors and result in effective model improvement for both surface monitoring and prediction applications. The reasons for diverse remote sensing data and products include (i) their complementary areal and temporal coverage, (ii) their diverse and covariant information content, and (iii) their ability to complement in situ observations, which are often sparse and only locally representative. To improve our understanding of the complex behavior of the Earth system at the surface and sub-surface, we need large volumes of data from high-resolution modelling and remote sensing, since the Earth surface exhibits a high degree of heterogeneity and discontinuities in space and time. The spatial and temporal variability of the biosphere, hydrosphere, cryosphere and anthroposphere calls for an increased use of Earth observation (EO) data attaining volumes previously considered prohibitive. We review data availability and discuss recent examples where satellite remote sensing is used to infer observable surface quantities directly or indirectly, with particular emphasis on key parameters necessary for weather and climate prediction. Coordinated high-resolution remote-sensing and modelling/assimilation capabilities for the Earth surface are required to support an international application-focused effort.

Earth observations↗

HighDimMixedModels.jl: Robust high-dimensional mixed-effects models across omics data

High-dimensional mixed-effects models are an increasingly important form of regression in which the number of covariates rivals or exceeds the number of samples, which are collected in groups or clusters. The penalized likelihood approach to fitting these models relies on a coordinate descent algorithm that lacks guarantees of convergence to a global optimum. Here, we empirically study the behavior of this algorithm on simulated and real examples of three types of data that are common in modern biology: transcriptome, genome-wide association, and microbiome data. Our simulations provide new insights into the algorithm’s behavior in these settings, and, comparing the performance of two popular penalties, we demonstrate that the smoothly clipped absolute deviation (SCAD) penalty consistently outperforms the least absolute shrinkage and selection operator (LASSO) penalty in terms of both variable selection and estimation accuracy across omics data. To empower researchers in biology and other fields to fit models with the SCAD penalty, we implement the algorithm in a Julia package, HighDimMixedModels.jl .

Gorstein, Evan↗

Bootstrap-determined p values in lattice QCD

We present a general method to determine the probability that stochastic Monte Carlo data, in particular those generated in a lattice QCD calculation, would have been obtained were that data drawn from the distribution predicted by a given theoretical hypothesis. Such a probability, or p -value, is often used as an important heuristic measure of the validity of that hypothesis. The proposed method offers the benefit that it remains usable in cases where the standard Hotelling T 2 methods based on the conventional χ 2 statistic do not apply, such as for uncorrelated fits. Specifically, we analyze q 2 , defined as the correlated χ 2 statistic obtained using an arbitrary covariance matrix estimator, and show how to use the bootstrap as a data-driven method to determine the expected distribution of q 2 for a given hypothesis with minimal assumptions. This distribution can then be used to determine the p -value for a fit to the data. We also describe a bootstrap approach for quantifying the impact upon this p -value of estimating population parameters from a single ensemble of N samples. The overall method is accurate up to a 1 / N bias which we do not attempt to quantify. Published by the American Physical Society 2025

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Redshift evolution and covariances for joint lensing and clustering studies with DESI Y1

ABSTRACT Galaxy–galaxy lensing (GGL) and clustering measurements from the Dark Energy Spectroscopic Instrument Year 1 (DESI Y1) data set promise to yield unprecedented combined-probe tests of cosmology and the galaxy–halo connection. In such analyses, it is essential to identify and characterize all relevant statistical and systematic errors. We forecast the covariances of DESI Y1 GGL + clustering measurements and the systematic bias due to redshift evolution in the lens samples. Focusing on the projected clustering and GGL correlations, we compute a Gaussian analytical covariance, using a suite of N-body and lognormal simulations to characterize the effect of the survey footprint. Using the DESI one percent survey data, we measure the evolution of galaxy bias parameters for the DESI luminous red galaxy (LRG) and bright galaxy survey (BGS) samples. We find mild evolution in the LRGs in $0.4 < z < 0.8$, subdominant to the expected statistical errors. For BGS, we find less evolution for brighter absolute magnitude cuts, at the cost of reduced sample size. We find that for a redshift bin width $\Delta z = 0.1$, evolution effects on DESI Y1 GGL is negligible across all scales, all fiducial selection cuts, all fiducial redshift bins. Galaxy clustering is more sensitive to evolution due to the bias squared scaling. Nevertheless the redshift evolution effect is insignificant for clustering above the 1-halo scale of $0.1h^{-1}$ Mpc. For studies that wish to reliably access smaller scales, additional treatment of redshift evolution is likely needed. This study serves as a reference for GGL and clustering studies using the DESI Y1 sample.

79 ASTRONOMY AND ASTROPHYSICS↗

Trend Detection of Atmospheric Time Series: Incorporating Appropriate Uncertainty Estimates and Handling Extreme Events

This paper is aimed at atmospheric scientists without formal training in statistical theory. Its goal is to, 1) provide a critical review of the rationale for trend analysis of the time series typically encountered in the field of atmospheric chemistry; 2) describe a range of trend-detection methods; and 3) demonstrate effective means of conveying the results to a general audience. Trend detections in atmospheric chemical composition data are often challenged by a variety of sources of uncertainty, which often behave differently to other environmental phenomena such as temperature, precipitation rate, or stream flow, and may require specific methods depending on the science questions to be addressed. Some sources of uncertainty can be explicitly included in the model specification, such as autocorrelation and seasonality, but some inherent uncertainties are difficult to quantify, such as data heterogeneity and measurement uncertainty due to the combined effect of short- and long-term natural variability, instrumental stability, and aggregation of data from sparse sampling frequency. Failure to account for these uncertainties might result in an inappropriate inference of the trends and their estimation errors. On the other hand, the variation in extreme events might be interesting for different scientific questions, for example, the frequency of extremely high surface ozone events and their relevance to human health. In this study we aim to, 1) review trend detection methods for addressing different levels of data complexity in different chemical species; 2) demonstrate that the incorporation of scientifically interpretable covariates can outperform pure numerical curve fitting techniques in terms of uncertainty reduction and improved predictability; 3) illustrate the study of trends based on extreme quantiles that can provide insight beyond standard mean or median based trend estimates; and 4) present an advanced method of quantifying regional trends based on the inter-site correlations of multi-site data. All demonstrations are based on time series of observed trace gases relevant to atmospheric chemistry, but the methods can be applied to other environmental data sets.

Trace gas↗

Super-CMB fluctuations and the Hubble tension

We study the covariance in the angular power spectrum estimates of CMB fluctuations when the primordial fluctuations are non-Gaussian. The non-Gaussian covariance comes from a nonzero connected four-point correlation function -- or the trispectrum in Fourier space -- and can be large when long-wavelength (super-CMB) modes are strongly coupled to short-wavelength modes. The effect of such non-Gaussian covariance can be modeled through additional freedom in the theoretical CMB angular power spectrum and can lead to different inferred values of the standard cosmological parameters relative to those in ΛCDM. Taking the collapsed limit of the primordial trispectrum in the quasi-single field inflation model as an example, we study how the six standard ΛCDM parameters shift when two additional parameters describing the trispectrum are allowed. The reduced statistical significance of the Hubble tension in the extended model allows us to combine the {\it Planck} temperature data and the type Ia supernovae data from Panstarrs with the distance-ladder measurement of the Hubble constant. Here, this combination of data shows strong evidence for a primordial trispectrum-induced non-Gaussian covariance, with a likelihood improvement of Δχ 2 ≈ –15 (with two additional parameters) relative to ΛCDM.

79 ASTRONOMY AND ASTROPHYSICS↗

An Adaptive Buddy Check for Observational Quality Control

An adaptive buddy check algorithm is presented that adjusts tolerances for outlier observations based on the variability of surrounding data. The algorithm derives from a statistical hypothesis test combined with maximum-likelihood covariance estimation. Its stability is shown to depend on the initial identification of outliers by a simple background check. The adaptive feature ensures that the final quality control decisions are not very sensitive to prescribed statistics of first-guess and observation errors, nor on other approximations introduced into the algorithm. The implementation of the algorithm in a global atmospheric data assimilation is described. Its performance is contrasted with that of a non-adaptive buddy check, for the surface analysis of an extreme storm that took place in Europe on 27 December 1999. The adaptive algorithm allowed the inclusion of many important observations that differed greatly from the first guess and that would have been excluded on the basis of prescribed statistics. The analysis of the storm development was much improved as a result of these additional observations.

Dee, Dick P.↗

Exploring HOD-dependent systematics for the DESI 2024 Full-Shape galaxy clustering analysis

We analyze the robustness of the DESI 2024 cosmological inference from the full shape of the galaxy power spectrum to uncertainties in the Halo Occupation Distribution (HOD) model of the galaxy-halo connection and the choice of priors on nuisance parameters. We assess variations in the recovered cosmological parameters across a range of mocks populated with different HOD models and find that shifts are often greater than 20% of the expected statistical uncertainties from the DESI data. We encapsulate the effect of such shifts in terms of a systematic covariance term, C HOD , and an additional diagonal contribution quantifying the impact of our choice of nuisance parameter priors on the ability of the effective field theory (EFT) model to correctly recover the cosmological parameters of the simulations. These two covariance contributions are designed to be added to the usual covariance term, C stat , describing the statistical uncertainty in the power spectrum measurement, in order to fairly represent these sources of systematic uncertainty. This novel approach should be more general and robust to the choice of model or additional external datasets used in cosmological fits than the alternative approach of adding systematic uncertainties to the recovered marginalised parameter posteriors. We compare the approaches within the context of a fixed ΛCDM model and demonstrate that our method gives conservative estimates of the systematic uncertainty that nevertheless have little impact on the final posteriors obtained from DESI data.

79 ASTRONOMY AND ASTROPHYSICS↗

Effect of Medium Symmetries in Limiting the Number of Parameters Estimated with Polarimetric Interferometry

The addition of interferometric backscattering pairs to the conventional polarimetric SAR data over forests and other vegetated areas increases the dimensionality of the data space, in principle enabling the estimation of a larger number of vegetation parameters. Without regard to the sensitivity of these data to vegetation scattering parameters, this paper poses the question: Will increasing the data channels as such result in a one-to-one increase in the number of parameters that can be estimated, or do vegetation and data properties inherently limit that number otherwise? In this paper, the complete polarimetric interferometric covariance matrix is considered and various symmetry properties of the scattering medium are used to study whether any of the correlation pairs can be eliminated. The number of independent pairs has direct consequences in their utility in parameter estimation schemes, since the maximum number of parameters that can be estimated cannot exceed the number of unique measurements. The independent components of the polarimetric interferometric SAR (POL/INSAR) data are derived for media with reflection, rotation, and azimuth symmetries, which are often encountered in vegetated surfaces. Similar derivations have been carried out before for simple polarimetry, i.e., zero baseline. This paper extends those to the interferometric case of general nonzero baselines. It is shown that depending on the type of symmetries present, the number of independent available measurements that can be used to estimate medium parameters will vary. In particular, whereas in the general case there are 27 mathematically independent measurements possible from a polarimetric interferometer, this number can be reduced to 15, 9, and 6 if the medium has reflection, rotation, or azimuthal symmetries, respectively. The results can be used in several ways in the interpretation of SAR data and the development of parameter estimation schemes, which will be discussed at the presentation. Recent POL/INSAR data from the JPL AIRSAR over a forested area will be used to demonstrate the results of this derivation. This work was performed by the Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA, under contract from the National Aeronautics and Space Administration.

Moghaddam, Mahta↗

Characterization of Turbulent Latent and Sensible Heat Flux Exchange Between the Atmosphere and Ocean in MERRA

Turbulent fluxes of heat and moisture across the atmosphere-ocean interface are fundamental components of the Earth's energy and water balance. Characterizing both the spatiotemporal variability and the fidelity of these exchanges of heat and moisture is critical to understanding the global water and energy cycle variations, quantifying atmosphere-ocean feedbacks, and improving model predictability. This study examines the veracity of the recently completed NASA Modern-Era Retrospective analysis for Research and Applications (MERRA) product with respect to its representation of the surface turbulent heat fluxes. A validation of MERRA turbulent heat fluxes and near-surface bulk variables at local, high-resolution space and time scales is achieved by making comparisons to a large suite of direct observations. Both in situ and satellite-observed gridded surface heat flux estimates are employed to investigate the spatial and temporal variability of the surface fluxes with respect to their annual mean climatologies, their seasonal covariability of near-surface bulk parameters, and their representation of extremes. The impact of data assimilation on the near-surface parameters is assessed through evaluation of incremental analysis update tendencies produced by the assimilation procedure. It is found that MERRA turbulent surface heat fluxes are relatively accurate for typical conditions but have systematically weak vertical gradients in moisture and temperature and have a weaker covariability between the near-surface gradients and wind speed than found in observations. This results in an underestimate of the surface latent and sensible heat fluxes over the western boundary current and storm track regions. The assimilation of observations mostly acts to bring MERRA closer to observational products by increasing moisture and temperature near the surface and decreasing the near-surface wind speeds. The major patterns of spatial and temporal variability of the turbulent heat fluxes produced by MERRA compare favorably to observationally based estimates. However, MERRA is distinct in terms of amplitude. These results suggest that MERRA is likely to be a valuable resource for a number of research applications though, as with all turbulent flux estimates, systematic issues should be taken into account

Robert, J. Brent↗

Characterization of Turbulent Latent and Sensible Heat Flux Exchange Between the Atmosphere and Ocean in MERRA

Turbulent fluxes of heat and moisture across the atmosphere-ocean interface are fundamental components of the Earth s energy and water balance. Characterizing both the spatiotemporal variability and the fidelity of these exchanges of heat and moisture is critical to understanding the global water and energy cycle variations, quantifying atmosphere-ocean feedbacks, and improving model predictability. This study examines the veracity of the recently completed NASA Modern-Era Retrospective analysis for Research and Applications (MERRA) product with respect to its representation of the surface turbulent heat fluxes. A validation of MERRA turbulent heat fluxes and near-surface bulk variables at local, high-resolution space and time scales is achieved by making comparisons to a large suite of direct observations. Both in situ and satellite-observed gridded surface heat flux estimates are employed to investigate the spatial and temporal variability of the surface fluxes with respect to their annual mean climatologies, their seasonal covariability of near-surface bulk parameters, and their representation of extremes. The impact of data assimilation on the near-surface parameters is assessed through evaluation of incremental analysis update tendencies produced by the assimilation procedure. It is found that MERRA turbulent surface heat fluxes are relatively accurate for typical conditions but have systematically weak vertical gradients in moisture and temperature and have a weaker covariability between the near-surface gradients and wind speed than found in observations. This results in an underestimate of the surface latent and sensible heat fluxes over the western boundary current and storm track regions. The assimilation of observations mostly acts to bring MERRA closer to observational products by increasing moisture and temperature near the surface and decreasing the near-surface wind speeds. The major patterns of spatial and temporal variability of the turbulent heat fluxes produced by MERRA compare favorably to observationally based estimates. However, MERRA is distinct in terms of amplitude. These results suggest that MERRA is likely to be a valuable resource for a number of research applications though, as with all turbulent flux estimates, systematic issues should be taken into account.

Roberts, J. Brent↗

Lessons Learned from Assimilating Altimeter Data into a Coupled General Circulation Model with the GMAO Augmented Ensemble Kalman Filter

Satellite altimetry measurements have provided global, evenly distributed observations of the ocean surface since 1993. However, the difficulties introduced by the presence of model biases and the requirement that data assimilation systems extrapolate the sea surface height (SSH) information to the subsurface in order to estimate the temperature, salinity and currents make it difficult to optimally exploit these measurements. This talk investigates the potential of the altimetry data assimilation once the biases are accounted for with an ad hoc bias estimation scheme. Either steady-state or state-dependent multivariate background-error covariances from an ensemble of model integrations are used to address the problem of extrapolating the information to the sub-surface. The GMAO ocean data assimilation system applied to an ensemble of coupled model instances using the GEOS-5 AGCM coupled to MOM4 is used in the investigation. To model the background error covariances, the system relies on a hybrid ensemble approach in which a small number of dynamically evolved model trajectories is augmented on the one hand with past instances of the state vector along each trajectory and, on the other, with a steady state ensemble of error estimates from a time series of short-term model forecasts. A state-dependent adaptive error-covariance localization and inflation algorithm controls how the SSH information is extrapolated to the sub-surface. A two-step predictor corrector approach is used to assimilate future information. Independent (not-assimilated) temperature and salinity observations from Argo floats are used to validate the assimilation. A two-step projection method in which the system first calculates a SSH increment and then projects this increment vertically onto the temperature, salt and current fields is found to be most effective in reconstructing the sub-surface information. The performance of the system in reconstructing the sub-surface fields is particularly impressive for temperature, but not as satisfactory for salt.

Keppenne, Christian↗

Monte Carlo control loops for cosmic shear cosmology with DES Year 1 data

Weak lensing by large-scale structure is a powerful probe of cosmology and of the dark universe. This cosmic shear technique relies on the accurate measurement of the shapes and redshifts of background galaxies and requires precise control of systematic errors. Monte Carlo control loops (MCCL) is a forward modeling method designed to tackle this problem. It relies on the ultra fast image generator (UFig) to produce simulated images tuned to match the target data statistically, followed by calibrations and tolerance loops. Here, we present the first end-to-end application of this method, on the Dark Energy Survey (DES) Year 1 wide field imaging data. We simultaneously measure the shear power spectrum $C_ℓ$ and the redshift distribution $n(z)$ of the background galaxy sample. The method includes maps of the systematic sources, point spread function (PSF), an approximate Bayesian computation (ABC) inference of the simulation model parameters, a shear calibration scheme, and a fast method to estimate the covariance matrix. We find a close statistical agreement between the simulations and the DES Y1 data using an array of diagnostics. In a nontomographic setting, we derive a set of $C_ℓ$ and $n(z)$ curves that encode the cosmic shear measurement, as well as the systematic uncertainty. Following a blinding scheme, we measure the combination of $Ω_m$, $σ_8$, and intrinsic alignment amplitude $A_{IA}$, defined as $S_8D_{IA}=σ_8(Ω_m/0.3)^{0.5}D_{IA}$, where $D_{IA}=1-0.11(A_{IA}-1)$. We find $S_8D_{IA}=0.8954_{-0.039}^{+0.054}$, where systematics are at the level of roughly 60% of the statistical errors. We discuss these results in the context of earlier cosmic shear analyses of the DES Y1 data. Our findings indicate that this method and its fast runtime offer good prospects for cosmic shear measurements with future wide-field surveys.

79 ASTRONOMY AND ASTROPHYSICS↗

The Effects of Noise on Polarimetric SAR Data

Polarimetric SAR data can provide a great deal of information about the scattering behavior of the surface under observation. Polarimetric SAR systems often measure the scattering matrices of the areas under observation in linear polarizations (H and V). From the scattering matrix commonly used forms such as the covariance matrix and the Stokes matrix can be easily derived. Other measures derived from polarimetric SAR data include the standard deviation of texture, correlation coefficients between scattering matrix terms, and the mode and variance of phase differences between scattering matrix terms. The effects of additive system noise on these measurements is not often considered in the literature on this subject.

Polarimetric SAR Data analysis↗

The Effects of Noise on Polarimetric SAR Data

Polarimetric SAR data can provide a great deal of information about the scattering behavior of the surface under observation. Polarimetric SAR systems often measure the scattering matrices of the areas under observation in linear polarizations (H and V). From the scattering matrix commonly used forms such as the covariance matrix and the Stokes matrix can be easily derived. Other measures derived from polarimetric SAR data include correlation coefficients between scattering matrix terms and the mode and variance of phase differences between scattering matrix terms. The effects of additive system noise on these measurements is not often considered in the literature on this subject. In this paper, the effects of additive system noise on measurements derived from polarimetric SAR data will be examined. It will be shown how first-order noise effects can be removed and how second-order noise effects can be reduced for some measurements...

Freeman, A.↗

Dark Energy Survey Year 3 Results: Cosmological constraints from second- and third-order shear statistics

Here, we present a cosmological analysis of the third-order aperture mass statistic using Dark Energy Survey Year 3 (DES Y3) data. We perform a complete tomographic measurement of the three-point correlation function of the Y3 weak lensing shape catalog with the four fiducial source redshift bins. Building upon our companion methodology paper, we apply a pipeline that combines the two-point function ξ ± with the mass aperture skewness statistic ⟨ M ap 3 ⟩ , which is an efficient compression of the full shear three-point function. We use a suite of simulated shear maps to obtain a joint covariance matrix. By jointly analyzing ξ ± and ⟨ M ap 3 ⟩ measured from DES Y3 data with a Λ CDM model, we find S 8 = 0.780 ± 0.015 and Ω m = 0.26 6 - 0.040 + 0.039 , yielding 111% of figure-of-merit improvement in the Ω m - S 8 plane relative to ξ ± alone, consistent with expectations from simulated likelihood analyses. With a w CDM model, we find S 8 = 0.74 9 - 0.026 + 0.027 and w 0 = - 1.39 ± 0.31 , which gives an improvement of 22% on the joint S 8 - w 0 constraint. Our results are consistent with w 0 = - 1 . Our new constraints are compared to CMB data from the Planck satellite, and we find that with the inclusion of ⟨ M ap 3 ⟩ the existing tension between the datasets is at the level of 2.3 σ . We show that the third-order statistic enables us to self-calibrate the mean photometric redshift uncertainty parameter of the highest redshift bin with little degradation in the figure of merit. Our results demonstrate the constraining power of higher-order lensing statistics and establish ⟨ M ap 3 ⟩ as a practical observable for joint analyses in current and future surveys.

Gomes, R. C. H. [University of Pennsylvania] (ORCI↗

Optimal Estimation of Clock Values and Trends from Finite Data

We show how to solve two problems of optimal linear estimation from a finite set of phase data. Clock noise is modeled as a stochastic process with stationary dth increments. The covariance properties of such a process are contained in the generalized autocovariance function (GACV). We set up two principles for optimal estimation: with the help of the GACV, these principles lead to a set of linear equations for the regression coefficients and some auxiliary parameters. The mean square errors of the estimators are easily calculated. The method can be used to check the results of other methods and to find good suboptimal estimators based on a small subset of the available data.

stationary increments↗