Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Statistical calibration”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Validation Metrics for Fixed Effects and Mixed-Effects Calibration

The modern scientific process often involves the development of a predictive computational model. To improve its accuracy, a computational model can be calibrated to a set of experimental data. A variety of validation metrics can be used to quantify this process. Some of these metrics have direct physical interpretations and a history of use, while others, especially those for probabilistic data, are more difficult to interpret. In this work, a variety of validation metrics are used to quantify the accuracy of different calibration methods. Frequentist and Bayesian perspectives are used with both fixed effects and mixed-effects statistical models. Through a quantitative comparison of the resulting distributions, the most accurate calibration method can be selected. Two examples are included which compare the results of various validation metrics for different calibration methods. It is quantitatively shown that, in the presence of significant laboratory biases, a fixed effects calibration is significantly less accurate than a mixed-effects calibration. This is because the mixed-effects statistical model better characterizes the underlying parameter distributions than the fixed effects model. The results suggest that validation metrics can be used to select the most accurate calibration model for a particular empirical model with corresponding experimental data.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2012

Continued Monitoring of Landsat Reflective Band Calibration Using Pseudo-Invariant Calibration Sites

Though both of the current Landsat instruments, Landsat-7 Enhanced Thematic Mapper+ (ETM+) and Landsat-5 Thematic Mapper (TM), include on-board calibration systems, since 2001, pseudo-invariant calibration sites (PICS) have been added to the suite of metrics to assess the instruments calibration. These sites do not provide absolute calibration data since there are no ground measurements of the sites, but in monitoring these PICS over time, the relative calibration can be tracked. The sites used by the Landsat instruments are primarily in the Saharan Desert. To date, the trending from the PICS sites has confirmed that most of the degradation seen in the ETM+ on-board calibration systems is likely not degradation of the instrument, but rather degradation of the calibration systems themselves. However, the PICS data show statistically significant degradation (at 2-sigma) in all the reflective spectral bands of up to -0.22%/year since July 2003. For the TM, the PICS were instrumental in updating the calibration in 2007 and now suggest two bands may require another update. The data show a statistically significant degradation (at 2-sigma) in Bands 1 and 3 of -0.27 and -0.15%/year, respectively, since March 1999. The data filtering and processing methods are currently being reviewed but these PICS results may lead to an update in the reflective band calibration of both Landsat-7 and Landsat-5.

Barsi, Julia A.↗
NASA NTRS1994

Metrology: Calibration and measurement processes guidelines

The guide is intended as a resource to aid engineers and systems contracts in the design, implementation, and operation of metrology, calibration, and measurement systems, and to assist NASA personnel in the uniform evaluation of such systems supplied or operated by contractors. Methodologies and techniques acceptable in fulfilling metrology quality requirements for NASA programs are outlined. The measurement process is covered from a high level through more detailed discussions of key elements within the process, Emphasis is given to the flowdown of project requirements to measurement system requirements, then through the activities that will provide measurements with defined quality. In addition, innovations and techniques for error analysis, development of statistical measurement process control, optimization of calibration recall systems, and evaluation of measurement uncertainty are presented.

Castrup, Howard T.↗
NASA NTRS2018

A Blueprint for Demonstrating Quantum Supremacy with Superconducting Qubits

Long coherence times and high fidelity control recently achieved in scalable superconducting circuits paved the way for the growing number of experimental studies of many-qubit quantum coherent phenomena in these devices. Albeit full implementation of quantum error correction and fault tolerant quantum computation remains a challenge the near term pre-error correction devices could allow new fundamental experiments despite inevitable accumulation of errors. One such open question foundational for quantum computing is achieving the so called quantum supremacy, an experimental demonstration of a computational task that takes polynomial time on the quantum computer whereas the best classical algorithm would require exponential time and/or resources. It is possible to formulate such a task for a quantum computer consisting of less than a 100 qubits. The computational task we consider is to provide approximate samples from a non-trivial quantum distribution. This is a generalization for the case of superconducting circuits of ideas behind boson sampling protocol for quantum optics introduced by Arkhipov and Aaronson. In this presentation we discuss a proof-of-principle demonstration of such a sampling task on a 9-qubit chain of superconducting gmon qubits developed by Google. We discuss theoretical analysis of the driven evolution of the device resulting in output approximating samples from a uniform distribution in the Hilbert space, a quantum chaotic state. We analyze quantum chaotic characteristics of the output of the circuit and the time required to generate a sufficiently complex quantum distribution. We demonstrate that the classical simulation of the sampling output requires exponential resources by connecting the task of calculating the output amplitudes to the sign problem of the Quantum Monte Carlo method. We also discuss the detailed theoretical modeling required to achieve high fidelity control and calibration of the multi-qubit unitary evolution in the device. We use a novel cross-entropy statistical metric as a figure of merit to verify the output and calibrate the device controls. Finally, we demonstrate the statistics of the wave function amplitudes generated on the 9-gmon chain and verify the quantum chaotic nature of the generated quantum distribution. This verifies the implementation of the quantum supremacy protocol.

Kechedzhi, Kostyantyn↗
DOE OSTI2024

Autocalibration of the E3SM Version 2 Atmosphere Model Using a PCA-Based Surrogate for Spatial Fields

Global Climate Model tuning (calibration) is a tedious and time-consuming process, with high-dimensional input and output fields. Experts typically tune by iteratively running climate simulations with hand-picked values of tuning parameters. Many, in both the statistical and climate literature, have proposed alternative calibration methods, but most are impractical or difficult to implement. We present a practical, robust, and rigorous calibration approach on the atmosphere-only model of the Department of Energy's Energy Exascale Earth System Model (E3SM) version 2. Our approach can be summarized into two main parts: (a) the training of a surrogate that predicts E3SM output in a fraction of the time compared to running E3SM, and (b) gradient-based parameter optimization. To train the surrogate, we generate a set of designed ensemble runs that span our input parameter space and use polynomial chaos expansions on a reduced output space to fit the E3SM output. We use this surrogate in an optimization scheme to identify values of the input parameters for which our model best matches gridded spatial fields of climate observations. To validate our choice of parameters, we run E3SMv2 with the optimal parameter values and compare prediction results to expertly-tuned simulations across 45 different output fields. This flexible, robust, and automated approach is straightforward to implement, and we demonstrate that the resulting model output matches present day climate observations as well or better than the corresponding output from expert tuned parameter values, while considering high-dimensional output and operating in a fraction of the time.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2010

Regression Model Term Selection for the Analysis of Strain-Gage Balance Calibration Data

The paper discusses the selection of regression model terms for the analysis of wind tunnel strain-gage balance calibration data. Different function class combinations are presented that may be used to analyze calibration data using either a non-iterative or an iterative method. The role of the intercept term in a regression model of calibration data is reviewed. In addition, useful algorithms and metrics originating from linear algebra and statistics are recommended that will help an analyst (i) to identify and avoid both linear and near-linear dependencies between regression model terms and (ii) to make sure that the selected regression model of the calibration data uses only statistically significant terms. Three different tests are suggested that may be used to objectively assess the predictive capability of the final regression model of the calibration data. These tests use both the original data points and regression model independent confirmation points. Finally, data from a simplified manual calibration of the Ames MK40 balance is used to illustrate the application of some of the metrics and tests to a realistic calibration data set.

Ulbrich, Norbert Manfred↗
DOE OSTI2021

Breaking the degeneracy between polarization efficiency and cosmological parameters in CMB experiments

Accurate cosmological parameter estimates using polarization data of the cosmic microwave background (CMB) put stringent requirements on map calibration, as highlighted in the recent results from the Planck satellite. In this paper, we point out that a model-dependent determination of polarization calibration can be achieved by the joint fit of the temperature-E-mode cross-power spectrum (TE) and E-mode auto-power spectrum (EE). This provides a valuable cross-check to band-averaged polarization efficiency measurements determined using other approaches. We demonstrate that, in ΛCDM, the combination of the TE and EE constrain polarization calibration with sub-percent uncertainty with Planck data and 2% uncertainty with sptpol data. We arrive at similar conclusions when extending ΛCDM to include the amplitude of lensing A L , the number of relativistic species N eff , or the sum of the neutrino masses Σm ν . Furthermore, the uncertainties on cosmological parameters are minimally impacted when marginalizing over polarization calibration, except, as can be expected, for the uncertainty on the amplitude of the primordial scalar power spectrum ln(10 10 A s ), which increases by 20–50%. However, this information can be fully recovered by adding temperature auto-power spectrum (TT) information. For current and future ground-based experiments, SPT-3G and CMB-S4, we forecast the cosmological parameter uncertainties to be minimally degraded when marginalizing over polarization calibration parameters. In addition, CMB-S4 could constrain its polarization calibration at the level of ~0.2% by combining TE and EE, and reach ~0.06% by also including TT. We therefore conclude that relying on calibrating against Planck polarization maps, whose statistical uncertainty is limited to ~0.5%, would be insufficient for upcoming experiments.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Streaked Sub-ps-resolution X-ray Line Shapes and Implications for Solid-density Plasma Dynamics

A high-resolution x-ray spectrometer was coupled with an ultrafast x-ray streak camera to produce time-resolved line shape spectra measured from hot, solid-density plasmas. A Bragg crystal was placed near a laser-produced plasma to maximize throughput; alignment tolerances were established by raytracing. The streak camera produced single-shot time-resolved spectra, heavily sloped due to photon time-of-flight differences, with sufficient reproducibility to accumulate photon statistics. The images are time-calibrated by the slope of streaked spectra and dewarped to generate spectra emitted at different times defined at the source. The streaked spectra demonstrate the evolution of spectral shoulders and other features on ps timescales, showing the feasibility of plasma parameter measurements on the rapid timescales necessary to study high-energy-density plasmas.

High-energy-density plasmas↗
DOE OSTI2020

How data science methods can improve the quality and efficiency of ICF and HEDP research

Data Science methods (many that are Bayesian based) are widely used in the physical sciences to estimate model parameters from experimental data, synthesize heterogeneous data, calibrate models, design experiments, and determine statistical significance of data. These methods provide a wealth of advantages over traditional analysis techniques because: 1) uncertainties are rigorously defined and propagated naturally through complex systems including covariance, 2) prior information is captured within the analysis framework (including rad-MHD and rad-hydro simulations), 3) competing models can be selected and/or ruled out using quantitative criteria, and 4) complex, heterogeneous data can be incorporated simultaneously. While these methods have been widely adopted as the gold standard in fields such as particle physics, astronomy, and biology, they have been slow to catch on in Inertial Confinement Fusion (ICF) and High Energy Density Physics (HEDP) research. Recently, several teams at LLNL, SNL, LANL, and the LLE have been exploring the use of these tools in their research and have found success. Here we propose that a concerted effort to consolidate these independent research efforts by developing and deploying common tools for use across the complex can revolutionize the way we approach data analysis, assimilation of theory and experiment, and decision making. The Bayesian formalism provides a means to accomplish this, but we are lacking certain infrastructure to make it happen on a large scale. Furthermore, once adopted, these techniques can be used to develop standards by which discoveries can be judged, similar to the so-called 5σ rule in high energy particle physics. Such standards may be used in the future to address the issue of unknown reproducibility in ICF and HED experiments caused by low shot rate and high cost per experiment. Our goals as a group are to advance the state of the art in HED measurement science by enabling: 1) better inferences from data with well-defined uncertainties, 2) better use of the data we have and continue to collect, 3) intelligent synthesis of data, 4) evaluation of the statistical significance of our data, and 5) informed decision making regarding the design of new experiments and instruments.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

Optimal Cosmic Microwave Background Lensing Reconstruction and Parameter Estimation with SPTpol Data

Here, we perform the first simultaneous Bayesian parameter inference and optimal reconstruction of the gravitational lensing of the cosmic microwave background (CMB), using 100 deg 2 of polarization observations from the SPTpol receiver on the South Pole Telescope. These data reach noise levels as low as 5.8 μK arcmin in polarization, which are low enough that the typically used quadratic estimator (QE) technique for analyzing CMB lensing is significantly suboptimal. Conversely, the Bayesian procedure extracts all lensing information from the data and is optimal at any noise level. We infer the amplitude of the gravitational lensing potential to be ${A}_{\phi }=0.949\,\pm \,0.122$ using the Bayesian pipeline, consistent with our QE pipeline result, but with 17% smaller error bars. The Bayesian analysis also provides a simple way to account for systematic uncertainties, performing a similar job as frequentist "bias hardening" or linear bias correction, and reducing the systematic uncertainty on A Φ due to polarization calibration from almost half of the statistical error to effectively zero. Finally, we jointly constrain A Φ along with A L , the amplitude of lensing-like effects on the CMB power spectra, demonstrating that the Bayesian method can be used to easily infer parameters both from an optimal lensing reconstruction and from the delensed CMB, while exactly accounting for the correlation between the two. These results demonstrate the feasibility of the Bayesian approach on real data, and pave the way for future analysis of deep CMB polarization measurements with SPT-3G, Simons Observatory, and CMB-S4, where improvements relative to the QE can reach 1.5 times tighter constraints on A Φ and seven times lower effective lensing reconstruction noise.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS1978

A new measurement of the cosmic ray electron spectrum from 10 GeV to 300 GeV

The spectrum of cosmic-ray electrons has been measured with an instrument that combines a transition-radiation detector with a shower detector. The transition-radiation detector provides unique identification of individual electrons and good discrimination against protons. At the same time, it allows the construction of a large-area instrument (0.48 sq m ? ster) and consequently makes possible a measurement of improved statistical accuracy. The instrument has been calibrated with electron beams of 5-300 GeV at Fermilab, thus eliminating energy-dependent biases. A first balloon flight yielded 30 hours of data at an altitude of 5 g/sq cm. The design of the instrument is described, along with the accelerator calibrations and the analysis of the balloon flight data. The spectrum of electrons is found to be significantly steeper than that of protons over the whole energy range.

Hartmann, G.↗
NASA NTRS2004

Absolute Timing of the Crab Pulsar with RXTE

We have monitored the phase of the main X-ray pulse of the Crab pulsar with the Rossi X-ray Timing Explorer (RXTE) for almost eight years, since the start of the mission in January 1996. The absolute time of RXTE's clock is sufficiently accurate to allow this phase to be compared directly with the radio profile. Our monitoring observations of the pulsar took place bi-weekly (during the periods when it was at least 30 degrees from the Sun) and we correlated the data with radio timing ephemerides derived from observations made at Jodrell Bank. We have determined the phase of the X-ray main pulse for each observation with a typical error in the individual data points of 50 microseconds. The total ensemble is consistent with a phase that is constant over the monitoring period, with the X-ray pulse leading the radio pulse by 0.01025 plus or minus 0.00120 period in phase, or 344 plus or minus 40 microseconds in time. The error estimate is dominated by a systematic error of 40 microseconds, most likely constant, arising from uncertainties in the instrumental calibration of the radio data. The statistical error is 0.00015 period, or 5 microseconds. The separation of the main pulse and interpulse appears to be unchanging at time scales of a year or less, with an average value of 0.4001 plus or minus 0.0002 period. There is no apparent variation in these values with energy over the 2-30 keV range. The lag between the radio and X-ray pulses ma be constant in phase (i.e., rotational in nature) or constant in time (i.e., due to a pathlength difference). We are not (yet) able to distinguish between these two interpretations.

Rots, Arnold H.↗
NASA NTRS2008

Forecasting Lightning Threat using Cloud-Resolving Model Simulations

Two new approaches are proposed and developed for making time and space dependent, quantitative short-term forecasts of lightning threat, and a blend of these approaches is devised that capitalizes on the strengths of each. The new methods are distinctive in that they are based entirely on the ice-phase hydrometeor fields generated by regional cloud-resolving numerical simulations, such as those produced by the WRF model. These methods are justified by established observational evidence linking aspects of the precipitating ice hydrometeor fields to total flash rates. The methods are straightforward and easy to implement, and offer an effective near-term alternative to the incorporation of complex and costly cloud electrification schemes into numerical models. One method is based on upward fluxes of precipitating ice hydrometeors in the mixed phase region at the-15 C level, while the second method is based on the vertically integrated amounts of ice hydrometeors in each model grid column. Each method can be calibrated by comparing domain-wide statistics of the peak values of simulated flash rate proxy fields against domain-wide peak total lightning flash rate density data from observations. Tests show that the first method is able to capture much of the temporal variability of the lightning threat, while the second method does a better job of depicting the areal coverage of the threat. Our blended solution is designed to retain most of the temporal sensitivity of the first method, while adding the improved spatial coverage of the second. Exploratory tests for selected North Alabama cases show that, because WRF can distinguish the general character of most convective events, our methods show promise as a means of generating quantitatively realistic fields of lightning threat. However, because the models tend to have more difficulty in predicting the instantaneous placement of storms, forecasts of the detailed location of the lightning threat based on single simulations can be in error. Although these model shortcomings presently limit the precision of lightning threat forecasts from individual runs of current generation models,the techniques proposed herein should continue to be applicable as newer and more accurate physically-based model versions, physical parameterizations, initialization techniques and ensembles of forecasts become available.

McCaul, Eugene W., Jr.↗
NASA NTRS2010

A GPS-Based Pitot-Static Calibration Method Using Global Output-Error Optimization

Pressure-based airspeed and altitude measurements for aircraft typically require calibration of the installed system to account for pressure sensing errors such as those due to local flow field effects. In some cases, calibration is used to meet requirements such as those specified in Federal Aviation Regulation Part 25. Several methods are used for in-flight pitot-static calibration including tower fly-by, pacer aircraft, and trailing cone methods. In the 1990 s, the introduction of satellite-based positioning systems to the civilian market enabled new inflight calibration methods based on accurate ground speed measurements provided by Global Positioning Systems (GPS). Use of GPS for airspeed calibration has many advantages such as accuracy, ease of portability (e.g. hand-held) and the flexibility of operating in airspace without the limitations of test range boundaries or ground telemetry support. The current research was motivated by the need for a rapid and statistically accurate method for in-flight calibration of pitot-static systems for remotely piloted, dynamically-scaled research aircraft. Current calibration methods were deemed not practical for this application because of confined test range size and limited flight time available for each sortie. A method was developed that uses high data rate measurements of static and total pressure, and GPSbased ground speed measurements to compute the pressure errors over a range of airspeed. The novel application of this approach is the use of system identification methods that rapidly compute optimal pressure error models with defined confidence intervals in nearreal time. This method has been demonstrated in flight tests and has shown 2- bounds of approximately 0.2 kts with an order of magnitude reduction in test time over other methods. As part of this experiment, a unique database of wind measurements was acquired concurrently with the flight experiments, for the purpose of experimental validation of the optimization method. This paper describes the GPS-based pitot-static calibration method developed for the AirSTAR research test-bed operated as part of the Integrated Resilient Aircraft Controls (IRAC) project in the NASA Aviation Safety Program (AvSP). A description of the method will be provided and results from recent flight tests will be shown to illustrate the performance and advantages of this approach. Discussion of maneuver requirements and data reduction will be included as well as potential applications.

Foster, John V.↗
NASA NTRS2011

Calibrated Noise Measurements with Induced Receiver Gain Fluctuations

The lack of well-developed techniques for modeling changing statistical moments in our observations has stymied the application of stochastic process theory in science and engineering. These limitations were encountered when modeling the performance of radiometer calibration architectures and algorithms in the presence of non stationary receiver fluctuations. Analyses of measured signals have traditionally been limited to a single measurement series. Whereas in a radiometer that samples a set of noise references, the data collection can be treated as an ensemble set of measurements of the receiver state. Noise Assisted Data Analysis is a growing field of study with significant potential for aiding the understanding and modeling of non stationary processes. Typically, NADA entails adding noise to a signal to produce an ensemble set on which statistical analysis is performed. Alternatively as in radiometric measurements, mixing a signal with calibrated noise provides, through the calibration process, the means to detect deviations from the stationary assumption and thereby a measurement tool to characterize the signal's non stationary properties. Data sets comprised of calibrated noise measurements have been limited to those collected with naturally occurring fluctuations in the radiometer receiver. To examine the application of NADA using calibrated noise, a Receiver Gain Modulation Circuit (RGMC) was designed and built to modulate the gain of a radiometer receiver using an external signal. In 2010, an RGMC was installed and operated at the National Institute of Standards and Techniques (NIST) using their Noise Figure Radiometer (NFRad) and national standard noise references. The data collected is the first known set of calibrated noise measurements from a receiver with an externally modulated gain. As an initial step, sinusoidal and step-function signals were used to modulate the receiver gain, to evaluate the circuit characteristics and to study the performance of a variety of calibration algorithms. The receiver noise temperature and time-bandwidth product of the NFRad are calculated from the data. Statistical analysis using temporal-dependent calibration algorithms reveals that the natural occurring fluctuations in the receiver are stationary over long intervals (100s of seconds); however the receiver exhibits local non stationarity over the interval over which one set of reference measurements are collected. A variety of calibration algorithms have been applied to the data to assess algorithms' performance with the gain fluctuation signals. This presentation will describe the RGMC, experiment design and a comparative analysis of calibration algorithms.

Racette, Paul↗
DOE OSTI2020

Weak-lensing Analysis of X-Ray-selected XXL Galaxy Groups and Clusters with Subaru HSC Data

We present a weak-lensing analysis of X-ray galaxy groups and clusters selected from the XMM-XXL survey using the first-year data from the Hyper Suprime-Cam (HSC) Subaru Strategic Program. Our joint weaklensing and X-ray analysis focuses on 136 spectroscopically confirmed X-ray-selected systems at 0.031 6 z 6 1.033 detected in the 25 deg2 XXL-N region, which largely overlaps with the HSC-XMM field. With high-quality HSC weak-lensing data, we characterize the mass distributions of individual clusters and establish the concentration–mass (c–M) relation for the XXL sample, by accounting for selection bias and statistical effects and marginalizing over the remaining mass calibration uncertainty. We find the mass-trend parameter of the c–M relation to be β = -0.07 ± 0.28 and the normalization to be c200 = 4.8 ± 1.0 (stat) ± 0.8 (syst) at M 200 = 1014h -1M and z = 0.3. We find no statistical evidence for redshift evolution. Our weaklensing results are in excellent agreement with dark-matter-only c–M relations calibrated for recent ΛCDM cosmologies. The level of intrinsic scatter in c200 is constrained as σ(ln c 200 ) < 24% (99.7% CL), which is smaller than predicted for the full population of ΛCDM halos. This is likely caused in part by the X-ray selection bias in terms of the cool-core or relaxation state. We determine the temperature–mass (TX–M 500 ) relation for a subset of 105 XXL clusters that have both measured HSC lensing masses and X-ray temperatures. The resulting TX–M 500 relation is consistent with the self-similar prediction. Our TX–M 500 relation agrees with the XXL DR1 results at group scales but has a slightly steeper mass trend, implying a smaller mass scale in the cluster regime. The overall offset in the TX–M 500 relation is at the ~ 1.5σ level, corresponding to a mean mass offset of (34 ± 20)%. We also provide bias-corrected, weak-lensing-calibrated M 200 and M 500 mass estimates of individual XXL clusters based on their measured X-ray temperatures.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS1979

Ground attitude determination and gyro calibration procedures for the HEAO missions

The Attitude Ground Support System (AGSS) at Goddard Space Flight Center provides ground attitude support for the High Energy Astronomy Observatory (HEAO) missions. The AGSS uses information telemetered from sun sensors, gyroscopes, star trackers (STAs), and an onboard computer to calculate updates to the onboard attitude reference and gyro calibration parameters. This paper presents the procedures used by the AGSS in computing reference and calibration updates; it also provides HEAO-1 postlaunch statistics that illustrate the ability to maintain 0.25-deg accuracy in the onboard attitude reference. Simulated HEAO-B results show an improvement in reference accuracy through the use of calibration maneuvers.

Fallon, L., III↗