Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Noise Variance”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Optimizing foreground mitigation for CMB lensing with combined multifrequency and geometric methods

Here, a key challenge for current and upcoming cosmic microwave background lensing measurements is their sensitivity to biases from extragalactic foregrounds, such as Sunyaev-Zel'dovich signals or cosmic infrared background emission. Several methods have been developed to mitigate these lensing foreground biases, dividing broadly into multifrequency cleaning approaches and modifications to the estimator geometry, but how to optimally combine these methods has not yet been explored in detail. In this paper, we examine which combination of lensing foreground mitigation strategies is best able to reduce the impact of foreground contamination for a Simons Observatory-like experiment while preserving maximal signal-to-noise. Although the optimal combination obtained depends on whether bias reduction or variance reduction is prioritized and on whether polarization data is used, generally, we find that combinations involving both geometric (profile hardening, source hardening, or shear) and multifrequency (symmetric cleaning) methods perform best. For lensing power spectrum measurements from temperature (polarization and temperature), our combined estimator methods are able to reduce the bias below σ/4 or 0.3% (0.1%), a factor of 16 (30) lower than the standard quadratic estimator bias, at a modest signal-to-noise cost of only 18% (12%). In contrast, single-method foreground-mitigation approaches struggle to reduce the bias to a negligible level below σ/2 without incurring a large noise penalty. For upcoming and current experiments, our combined methods therefore represent a promising approach for making lensing measurements with negligible foreground bias.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Noise from undetected sources in Dark Energy Survey images

ABSTRACT For ground-based optical imaging with current CCD technology, the Poisson fluctuations in source and sky background photon arrivals dominate the noise budget and are readily estimated. Another component of noise, however, is the signal from the undetected population of stars and galaxies. Using injection of artifical galaxies into images, we demonstrate that the measured variance of galaxy moments (used for weak gravitational lensing measurements) in Dark Energy Survey (DES) images is significantly in excess of the Poisson predictions, by up to 30 per cent, and that the background sky levels are overestimated by current software. By cross-correlating distinct images of ‘empty’ sky regions, we establish that there is a significant image noise contribution from undetected static sources (US), which, on average, are mildly resolved at DES resolution. Treating these US as a stationary noise source, we compute a correction to the moment covariance matrix expected from Poisson noise. The corrected covariance matrix matches the moment variances measured on the injected DES images to within 5 per cent. Thus, we have an empirical method to statistically account for US in weak lensing measurements, rather than requiring extremely deep sky simulations. We also find that local sky determinations can remove most of the bias in flux measurements, at a small penalty in additional, but quantifiable, noise.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

A Lie algebraic theory of barren plateaus for deep parameterized quantum circuits

Variational quantum computing schemes train a loss function by sending an initial state through a parametrized quantum circuit, and measuring the expectation value of some operator. Despite their promise, the trainability of these algorithms is hindered by barren plateaus (BPs) induced by the expressiveness of the circuit, the entanglement of the input data, the locality of the observable, or the presence of noise. Up to this point, these sources of BPs have been regarded as independent. In this work, we present a general Lie algebraic theory that provides an exact expression for the variance of the loss function of sufficiently deep parametrized quantum circuits, even in the presence of certain noise models. Our results allow us to understand under one framework all aforementioned sources of BPs. This theoretical leap resolves a standing conjecture about a connection between loss concentration and the dimension of the Lie algebra of the circuit’s generators.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

A variational framework for residual-based adaptivity in neural PDE solvers and operator learning

Residual-based adaptive strategies are widely used in scientific machine learning yet remain largely heuristic. We introduce a variational framework that formalizes these methods through convex transformations of the residual, where different transformations correspond to distinct objective functionals. For instance, exponential weights target uniform error minimization, while linear weights recover quadratic error minimization. This perspective reveals adaptive weighting as a means of selecting sampling distributions that optimize a primal objective, directly linking discretization choices to error metrics. This principled approach yields three key benefits: it enables systematic design of adaptive schemes, reduces discretization error by lowering estimator variance, and enhances learning dynamics by improving gradient signal-to-noise ratio. Extending the framework to operator learning, we demonstrate substantial performance gains across diverse optimizers and architectures. Our results provide a theoretical perspective for residual-based adaptivity and establish a foundation for principled discretization and training.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Propagation of non-Gaussian voltage angle fluctuations in high-voltage power grids

Recent measurements have reported non-Gaussian tails in the distribution of frequency data in electric power grids. Large frequency deviations may induce grid instabilities and it is therefore crucial to understand how noise disturbances with long, non-Gaussian tails propagate. Here, we investigate how fluctuations in power feed-in, characterized by non-zero cumulants of their distribution, propagate through high-voltage power grids. Unlike previous investigations which focused on the white-noise limit, we consider the limit of long noise correlation time, where power feed-in fluctuates over times longer than the inherent dynamical time scales of the grid - the relevant regime for large-scale, high-voltage distribution grids. In this work, we show that in this limit, the skewness and kurtosis of the power feed-in distribution propagate similarly as its variance, independently of the distribution of inertia. Non-Gaussianities from individual sources of noise therefore persist throughout the entire network. This finding is corroborated by numerical results on a realistic model of the synchronous grid of continental Europe.

42 ENGINEERING↗
DOE OSTI2021

Planck constraints on the tensor-to-scalar ratio

In this work, we present constraints on the tensor-to-scalar ratio r using Planck data. We use the latest release of Planck maps (PR4), processed with the NPIPE code, which produces calibrated frequency maps in temperature and polarization for all Planck channels from 30 GHz to 857 GHz using the same pipeline. We computed constraints on r using the BB angular power spectrum, and we also discuss constraints coming from the TT spectrum. Given Planck's noise level, the TT spectrum gives constraints on r that are cosmic-variance limited (with $\sigma$ r = 0.093), but we show that the marginalized posterior peaks towards negative values of r at about the 1.2$\sigma$ level. We derived Planck constraints using the BB power spectrum at both large angular scales (the 'reionization bump') and intermediate angular scales (the 'recombination bump') from $\ell$=2 to 150, and find a stronger constraint than that from TT, with $\sigma$ r = 0.069. The Planck BB spectrum shows no systematic bias, and is compatible with zero, given both the statistical noise and the systematic uncertainties. The likelihood analysis using B modes yields the constraint r < 0.158 at 95% confidence using more than 50% of the sky. This upper limit tightens to r < 0.069 when Planck EE, BB, and EB power spectra are combined consistently, and it tightens further to r < 0.056 when the Planck TT power spectrum is included in the combination. Finally, combining Planck with BICEP2/Keck 2015 data yields an upper limit of r < 0.044.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Methods for the robust computation of the long-period seismic spectrum of broad-band arrays

SUMMARY We describe array methods to search for low signal-to-noise ratio (SNR) signals in long-period seismic data using Fourier analysis. This is motivated by published results that find evidence of solar free oscillations in the Earth's seismic hum. Previous work used data from only one station. In this paper, we describe methods for computing spectra from array data. Arrays reduce noise level through averaging and provide redundancy that we use to distinguish coherent signal from a random background. We describe two algorithms for calculating a robust spectrum from seismic arrays, an algorithm that automatically removes impulsive transient signals from data, a jackknife method for estimating the variance of the spectrum, and a method for assessing the significance of an entire spectral band. We show examples of their application to data recorded by the Homestake Mine 3-D array in Lead, SD and the Piñon Flats PY array. These are two of the quietest small aperture arrays ever deployed in North America. The underground Homestake data has exceptionally low noise, and the borehole sensors of the PY array also have very low noise, making these arrays well suited to finding very weak signals. We find that our methods remove transient signals effectively from the data so that even low-SNR signals in the seismic background can be found and tested. Additionally, we find that the jackknife variance estimate is comparable to the noise floor, and we present initial evidence for solar g-modes in our data through the T2 test, a multivariate generalization of Student's t-test.

Caton, Ross C.↗
DOE OSTI2020

Hemispherical variance anomaly and reionization optical depth

ABSTRACT Cosmic microwave background (CMB) full-sky temperature data show a hemispherical asymmetry in power nearly aligned with the Ecliptic, with the Northern hemisphere displaying an anomalously low variance, while the Southern hemisphere appears consistent with expectations from the best-fitting theory, Lambda Cold Dark Matter (ΛCDM). The low signal-to-noise ratio in current polarization data prevents a similar comparison. Polarization realizations constrained by temperature data show that in ΛCDM the lack of variance is not expected to be present in polarization data. Therefore, a natural way of testing whether the temperature result is a fluke is to measure the variance of CMB polarization components. In anticipation of future CMB experiments that will allow for high-precision large-scale polarization measurements, we study how the variance of polarization depends on ΛCDM-parameter uncertainties by forecasting polarization maps with Planck’s Markov chain Monte Carlo chains. We show that polarization variance is sensitive to present uncertainties in cosmological parameters, mainly due to current poor constraints on the reionization optical depth τ, which drives variance at low multipoles. We demonstrate how the improvement in the τ measurement seen between Planck’s two latest data releases results in a tighter constraint on polarization variance expectations. Finally, we consider even smaller uncertainties on τ and how more precise measurements of τ can drive the expectation for polarization variance in a hemisphere close to that of the cosmic-variance-limited distribution.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Mitigating the optical depth degeneracy using the kinematic Sunyaev-Zel’dovich effect with CMB-S4 data

The epoch of reionization is one of the major phase transitions in the history of the universe, and is a focus of ongoing and upcoming cosmic microwave background (CMB) experiments with improved sensitivity to small-scale fluctuations. Reionization also represents a significant contaminant to CMB-derived cosmological parameter constraints, due to the degeneracy between the Thomson-scattering optical depth, $\tau$, and the amplitude of scalar perturbations, $A_s$. This degeneracy subsequently hinders the ability of large-scale structure data to constrain the sum of the neutrino masses, a major target for cosmology in the 2020s. In this work, we explore the kinematic Sunyaev-Zel'dovich (kSZ) effect as a probe of reionization, and show that it can be used to mitigate the optical depth degeneracy with high-sensitivity, high-resolution data from the upcoming CMB-S4 experiment. We discuss the dependence of the kSZ power spectrum on physical reionization model parameters, as well as on empirical reionization parameters, namely $\tau$ and the duration of reionization, $\Delta z$. We show that by combining the kSZ two-point function and the reconstructed kSZ four-point function, degeneracies between $\tau$ and $\Delta z$ can be strongly broken, yielding tight constraints on both parameters. We forecast $\sigma(\tau) = 0.003$ and $\sigma(\Delta z) = 0.25$ for a combination of CMB-S4 and Planck data, including detailed treatment of foregrounds and atmospheric noise. The constraint on $\tau$ is nearly identical to the cosmic-variance limit that can be achieved from large-angle CMB polarization data. The kSZ effect thus promises to yield not only detailed information about the reionization epoch, but also to enable high-precision cosmological constraints on the neutrino mass.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Utilizing machine learning to improve the precision of fluorescence imaging of cavity-generated spin squeezed states

We present a supervised learning model to calibrate the photon collection rate during the fluorescence imaging of cold atoms. The linear regression model finds the collection rate at each location on the sensor such that the atomic population difference equals that of a highly precise optical cavity measurement. This 192 variable regression results in a measurement variance 27% smaller than our previous single variable regression calibration. The measurement variance is now in agreement with the theoretical limit due to other known noise sources. This model efficiently trains in less than a minute on a standard personal computer's CPU and requires less than 10 min of data collection. Furthermore, the model is applicable across a large change in population difference and across data collected on different days.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Transitioning from Stage-III to Stage-IV: Cosmology from galaxy×CMB lensing and shear×CMB lensing (Final Report for Grant SC0021429 "Cosomology with Galaxy and CMB Surveys: Mitigating Systematic Effects Through Combining Datasets")

We examine the cosmological constraining power from two cross-correlation probes between galaxy and cosmic microwave background (CMB) surveys: the cross-correlation of lens galaxy density with CMB lensing convergence $\langle δ_g$$^κ\text{CMB}\rangle$, and source galaxy weak lensing shear with CMB lensing convergence $\langle^{γκ}$$_{\text{CMB}}\rangle$. These two cross-correlation probes provide an independent cross-check of other large-scale structure constraints and are insensitive to galaxy-only or CMB-only systematic effects. In addition, when combined with other large-scale structure probes, the cross-correlations can break degeneracies in cosmological and nuisance parameters, improving both the precision and robustness of the analysis. In this work, we study how the constraining power of $\langle δ_g$$^κ\text{CMB}\rangle + \langle^{γκ}$$_{\text{CMB}}\rangle$ changes from Stage-III (ongoing) to Stage-IV (future) surveys. Given the flexibility in selecting the lens galaxy sample, we also explore systematically the impact on cosmological constraints when we vary the redshift range and magnitude limit of the lens galaxies using mock galaxy catalogs. We find that in our setup, the contribution to cosmological constraints from $\langle δ_g$$^κ\text{CMB}\rangle$ and $\langle^{γκ}$$_{\text{CMB}}\rangle$ are comparable in the Stage-III data sets; but in Stage-IV surveys, the noise in$\langle δ_g$$^κ\text{CMB}\rangle$ becomes subdominant to cosmic variance, preventing $\langle δ_g$$^κ\text{CMB}\rangle$ to further improve the constraints. This implies that to maximize the cosmological constraints from future $\langle δ_g$$^κ\text{CMB}\rangle$ + $\langle^{γκ}$$_{\text{CMB}}\rangle$ analyses, we should focus more on the requirements on$^{γκ}$$_{\text{CMB}}\rangle$ instead of$\langle δ_g$$^κ\text{CMB}\rangle$. Furthermore, the selection of the lens sample should be optimized in terms of our ability to characterize its redshift or galaxy bias instead of its number density.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

More buck-per-shot: Why learning trumps mitigation in noisy quantum sensing

Quantum sensing is one of the most promising applications for quantum technologies. However, reaching the ultimate sensitivities enabled by the laws of quantum mechanics can be a challenging task in realistic scenarios where noise is present. While several strategies have been proposed to deal with the detrimental effects of noise, these come at the cost of an extra shot budget. Given that shots are a precious resource for sensing – as infinite measurements could lead to infinite precision – care must be taken to truly guarantee that any shot not being used for sensing is actually leading to some metrological improvement. In this work, we study whether investing shots in error-mitigation, inference techniques, or combinations thereof, can improve the sensitivity of a noisy quantum sensor on a (shot) budget. We present a detailed bias–variance error analysis for various sensing protocols. Our results show that the costs of zero-noise extrapolation techniques outweigh their benefits. We also find that pre-characterizing a quantum sensor via inference techniques leads to the best performance, under the assumption that the sensor is sufficiently stable.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

GALIC: hybrid multi-qubitwise pauli grouping for quantum computing measurement

Abstract Observable estimation is a core primitive in NISQ-era algorithms targeting quantum chemistry applications. To reduce the state preparation overhead required for accurate estimation, recent works have proposed various simultaneous measurement schemes to lower estimator variance. Two primary grouping schemes have been proposed: full commutativity (FC) and qubit-wise commutativity (QWC), with no compelling means of interpolation. In this work we propose a generalized framework for designing and analyzing context-aware hybrid FC/QWC commutativity relations. We use our framework to propose a noise-and-connectivity aware grouping strategy: Generalized backend-Aware pauLI Commutation (GALIC). We demonstrate how GALIC interpolates between FC and QWC, maintaining estimator accuracy in Hamiltonian estimation while lowering variance by an average of 20% compared to QWC. We also explore the design space of near-term quantum devices using the GALIC framework, specifically comparing device noise levels and connectivity. We find that error suppression has a more than 10 × larger impact on device-aware estimator variance than qubit connectivity with even larger correlation differences in estimator biases.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Kinematically enhanced interpolating operators for boosted hadrons

We propose to use interpolating operators for lattice quantum chromodynamics calculations of highly boosted pions and nucleons with kinematically enhanced ground-state overlap factors at large momentum. Because this kinematic enhancement applies to the signal but not the variance of the correlation function, these interpolating operators can achieve better signal-to-noise ratios at large momentum. We perform proof-of-principle calculations with boosted pions and nucleons using close-to-physical and larger quark masses to explore the utility of our proposal. Results for effective energies and matrix elements, as well as Lanczos ground-state energy estimators, are consistent with theoretical expectations for signal-to-noise improvement at large momenta.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

Adversarial autoencoder ensemble for fast and probabilistic reconstructions of few-shot photon correlation functions for solid-state quantum emitters

Second-order photon correlation measurements [g (2) (τ) functions] are widely used to classify single-photon emission purity in quantum emitters or to measure the multiexciton quantum yield of emitters that can simultaneously host multiple excitations – such as quantum dots – by evaluating the value of g (2) (τ = 0). Accumulating enough photons to accurately calculate this value is time consuming and could be accelerated by fitting of few-shot photon correlations. Here, we develop an uncertainty-aware, deep adversarial autoencoder ensemble (AAE) that reconstructs noise-free g (2) (τ) functions from noise-dominated, few-shot inputs. The model is trained with simulated g (2) (τ) functions that are facilely generated by Poisson sampling time bins. The AAE reconstructions are performed orders-of-magnitude faster, with reconstruction errors and estimates of g (2) (τ = 0) that are lower in variance and similar in accuracy compared to Maximum likelihood estimation and Levenberg-Marquardt least-squares fitting approaches, for simulated and experimentally measured few-shot g (2) (τ) functions (~100 two-photon events) of InP/ZnS/ZnSe and CdS/CdSe/CdS quantum dots. The deep-ensemble model comprises eight individual autoencoders, allowing for probabilistic reconstructions of noise-free g (2) (τ) functions, and we show that the predicted variance scales inversely with number of shots, with comparable uncertainties to computationally intensive Markov chain Monte Carlo sampling. Furthermore, this work demonstrates the advantage of machine learning models to perform uncertainty-aware, fast, and accurate reconstructions of simple Poisson-distributed photon correlation functions, allowing for on-the-fly reconstructions and accelerated materials characterization of solid-state quantum emitters.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2025

Denoising Autoencoder for Reconstructing Sensor Observation Data and Predicting Evapotranspiration: Noisy and Missing Values Repair and Uncertainty Quantification

Abstract Machine learning (ML) methods applied in scientific research often deal with interrelated features in high‐dimensional data. Reducing data noise and redundancy is needed to increase prediction accuracy and efficiency especially when dealing with data from field sensors. We explored an unsupervised learning method, the denoising autoencoder (DAE), to extract the underlying data structure from noisy raw data in the context of predicting hydrologic quantities from multiple field sensors. These sensors have intrinsic instrumental noise and occasional malfunctions that cause missing values. Our DAE neural network reconstructed meteorological sensor data containing noise and missing values to predict evapotranspiration in a mountainous watershed. The DAE reconstructed the sensor variables with a mean coefficient of determination value of 0.77 across 15 dimensions representing individual sensors. It reduced variance and bias uncertainties compared to a classical autoencoder model. The reconstruction quality varied across dimensions depending on their cross‐correlation and alignment with the underlying data structure. Uncertainties arising from the model structure were overall higher than those resulting from data corruption. We attached the DAE structure to a downstream ET‐prediction neural network in three formats and achieved reasonably accurate ET predictions . The use of the DAE notably reduced variance uncertainty in ET prediction. However, excessive variance reduction may be accompanied by an increase in bias due to the intrinsic bias‐variance tradeoff. Our method of evaluating and reducing uncertainties in aggregated data from different sources can be used to improve predictive models, process understanding, and uncertainty quantification for better water resource management. Plain Language Summary We present a machine learning method, namely the denoising autoencoder, which reduces the effects of data noise and missing values typically present in scientific data sets collected through sensor measurements. This method selects the most relevant information from noisy raw data collected by the instruments and fills in missing values. To demonstrate the effectiveness of our method, we applied it to predict evapotranspiration, a hydrologic variable that represents the water moved from the land surface to the atmosphere through a combination of evaporation and plant water use (transpiration). We also used a random sampling technique (the Monte Carlo method) to compare the uncertainty in the predictions when using the raw and noisy data versus the reconstructed data. The denoising process produced more accurate predictions of evapotranspiration with less uncertainty. Improved predictions of evapotranspiration can lead to a better understanding and accounting of water budgets. This ML approach is broadly suitable for a wide variety of applications that involve noisy sensor data with missing values. Key Points We used a denoising autoencoder (DAE) neural network to reduce noise in meteorological and soil sensor observations by on average We used Monte Carlo sampling to estimate the bias and variance of all model outputs, including uncertainty sources from data and the model We attached the DAE component to a downstream neural network to predict ET with the variance reduced by , compared to that without the DAE

denoising autoencoder↗
DOE OSTI2023

Effective cosmic density field reconstruction with convolutional neural network

ABSTRACT We present a cosmic density field reconstruction method that augments the traditional reconstruction algorithms with a convolutional neural network (CNN). Following previous work, the key component of our method is to use the reconstructed density field as the input to the neural network. We extend this previous work by exploring how the performance of these reconstruction ideas depends on the input reconstruction algorithm, the reconstruction parameters, and the shot noise of the density field, as well as the robustness of the method. We build an eight-layer CNN and train the network with reconstructed density fields computed from the Quijote suite of simulations. The reconstructed density fields are generated by both the standard algorithm and a new iterative algorithm. In real space at z = 0, we find that the reconstructed field is 90 per cent correlated with the true initial density out to $k\sim 0.5 \, \mathrm{ h}\, \rm {Mpc}^{-1}$, a significant improvement over $k\sim 0.2 \, \mathrm{ h}\, \rm {Mpc}^{-1}$ achieved by the input reconstruction algorithms. We find similar improvements in redshift space, including an improved removal of redshift space distortions at small scales. We also find that the method is robust across changes in cosmology. Additionally, the CNN removes much of the variance from the choice of different reconstruction algorithms and reconstruction parameters. However, the effectiveness decreases with increasing shot noise, suggesting that such an approach is best suited to high density samples. This work highlights the additional information in the density field beyond linear scales as well as the power of complementing traditional analysis approaches with machine learning techniques.

Astronomy & Astrophysics↗