Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “error impact analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 613 records · Page 34

Optimizing Residential HVAC Systems: Evaluating How the Usage of Smart Diagnostic Tools for Quality Installation and Commissioning Impacts System Performance and HVAC Contractor Businesses

The adoption of digital quality installation and fault detection tools has the potential to transform the $14 billon residential heating, ventilating, and air conditioning (HVAC) service and installation industry. Increased market penetration can be accelerated by providing an increased understanding of the energy and economic value proposition of the usage of these tools. In tandem with energy and HVAC system performance impact research and analysis, the Southface Team investigated the economic business implications for HVAC service and installation contractors adopting the use of measureQuick (mQ), a smart diagnostic app that aids in quality installation and fault detection and works agnostically with Bluetooth-enabled HVAC technician tools.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Sensitivity Study for Forecasting Variables of WRF-Solar Using a Tangent Linear Approach

Integrating solar generation in recent years has highlighted the need for improved accuracy in predicting solar power. Confidence in solar power forecasting can be achieved by designing an ensemble that provides reliable probabilistic information for solar radiation with reduced uncertainty and error. Ideally, ensemble members are created through the optimized perturbation of the initial conditions in numerical weather prediction (NWP) models. Tangent linear models are capable of efficiently investigating the sensitivity of solar radiation to model input parameters because they do not require individual perturbation of each variable. This sensitivity study using tangent linear models provide us the capability to identify the right variables to perturb in an ensemble prediction system. In this study, we developed tangent linear models for WRF-Solar modules that directly impact the computation of solar radiation and the simulation of cloud formation and dissipation including the Fast All-sky Model for Solar Applications (FARMS), the Noah land surface model (LSM), the Thompson microphysics, the Mello-Yamada-Nakanishi-Niino (MYNN) boundary layer parameterization, and the Deng scheme for a shallow-convection parameterization. A sensitivity analysis was conducted under various scenarios based on satellite observations and model simulations from the National Solar Radiation Data Base (NSRDB) and WRF-Solar, respectively. Critical forecasting variables that are highly sensitive to the forecasting of global horizontal irradiance (GHI), direct normal irradiance (DNI), cloud mixing ratio, cloud tendency, cloud fraction, and sensible and latent heat fluxes were determined using the relevant WRF-Solar module. This study will be used as a guidance on future research leading to high-quality probabilistic solar forecasting. In this presentation, we discuss the validation of tangent linear approach for WRF-Solar modules and illustrate how the sensitivity results are valuable in the improvement of probabilistic solar prediction.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Data from: Comparing traditional and Bayesian approaches to ecological meta-analysis

Despite the wide application of meta-analysis in ecology, some of the traditional methods used for meta-analysis may not perform well given the type of data characteristic of ecological meta-analyses. We reviewed published meta-analyses on the ecological impacts of global climate change, evaluating the number of replicates used in the primary studies (ni) and the number of studies or records (k) that were aggregated to calculate a mean effect size. We used the results of the review in a simulation experiment to assess the performance of conventional frequentist and Bayesian meta-analysis methods for estimating a mean effect size and its uncertainty interval. Our literature review showed that ni and k were highly variable, distributions were right-skewed, and were generally small (median ni =5, median k=44). Our simulations show that the choice of method for calculating uncertainty intervals was critical for obtaining appropriate coverage (close to the nominal value of 0.95). When k was low (<40), 95% coverage was achieved by a confidence interval based on the t-distribution that uses an adjusted standard error (the Hartung-Knapp-Sidik-Jonkman, HKSJ), or by a Bayesian credible interval, whereas bootstrap or z-distribution confidence intervals had lower coverage. Despite the importance of the method to calculate the uncertainty interval, 39% of the meta-analyses reviewed did not report the method used, and of the 61% that did, 94% used a potentially problematic method, which may be a consequence of software defaults. In general, for a simple random-effects meta-analysis, the performance of the best frequentist and Bayesian methods were similar for the same combinations of factors (k and mean replication), though the Bayesian approaches had higher than nominal (>95%) coverage for the mean effect when k was very low (k<15). Our literature review suggests that many meta-analyses that used z-distribution or bootstrapping confidence intervals may have over-estimated the statistical significance of their results when the number of studies was low; more appropriate methods need to be adopted in ecological meta-analyses.

54 ENVIRONMENTAL SCIENCES↗

Analysis of Airport Ground Delay Program Decisions Using Data Mining Techniques

Air traffic service providers have to make decisions regarding changes to air traffic flow in the event of major weather disturbances and traffic congestions to maintain safety of the system. The behavior of the air traffic management system will be more predictable if consistent decisions are made under similar traffic and weather conditions. Consistency of deciding on control action depends on the weather and traffic conditions as well as accuracy in predicting these conditions. Weather parameters (defined in terms of forecast and actual weather and traffic conditions) on different days can be used to categorize days into days with little decision consistency, days with moderate decision consistency and days with high decision consistency. Four years of traffic, weather and ground delay program decisions data at major airports in the United States are used in the analysis. This paper examines performance of different data mining methods in the three regions of decision consistency. Not surprisingly, data mining methods have the best performance in the region of most decision consistency and have the poorest performance in the region of little decision consistency. In applications where data mining methods have differing performance in differing regions, it would be more useful to characterize the region specific performance instead of characterizing performance by a single parameter. Finally, the results show no significant variation in the performance of different data mining methods for this particular problem. The fact that different mining methods show no significant variation also provides further confidence in the results of data mining methods. Work in this abstract discusses initial results. This paper describes the results in terms of both forecast and actual environmental conditions and discusses how prediction errors impact decision consistency.

Kulkarni, Deepak↗

Run Time Improvement Efforts for the Roman Space Telescope Thermal Analysis

"Observatory thermal models for large, complex missions, such as the Wide Field InfraRed Survey Telescope (WFIRST) mission, produce an immense amount of data to be processed. Configuration management of the model throughout the project life cycle has mainly focused on which versions of the subsystem models form the current observatory level configuration. However, the results produced by the model are not nearly as well The Roman Space Telescope (RST), formerly known as the Wide Field InfraRed Survey Telescope, is the next great astrophysics observatory mission to follow the James Webb Space Telescope with a planned launch in 2026. As a large scale, flagship mission for NASA with challenging wave front error stability requirements, a single model approach for both thermal discipline analysis and thermo-optical distortion analysis has been used since the early days of the project. In alleviating the need to maintain two separate models for different analysis types, it imposes run time penalties on the thermal analysis with a large model with significant radiation heat exchange. Throughout the lifecycle of the project, the component models have steadily grown in size, resulting in a continuous growth of the overall observatory model with each update and consequently a considerable increase in the model run time. While ongoing efforts to reduce run time are continuously investigated, previous efforts had primarily focused on timestep size and total simulation time to reach quasi-equilibrium. More recently, studies were performed on the total number of radiation couplings (radks) included in the model, which has a nearly linear impact on run time, but increases exponentially with node count. As standard practice for spacecraft analysis, small radks were excluded from the temperature solution based on the assumption that their interchange/view factors have a negligible impact on heat flow. Four approaches were investigated to reduce the model run time while minimizing the impact on accuracy: (1) the Equivalent Radiation Network node, (2) Progressive Radk Inclusion as solution proceeds, (3) Targeted Radk Filtering for critical/non critical areas, and lastly (4) Representation of culled radks with Backloads. Furthermore, the investigation of model run time also revealed that cold cases took noticeably longer to run than hot cases; the root computational inefficiencies were explored along with the computation penalty of linearization of the external radks and recalculation of temperature dependent linear couplings at each timestep. This paper outlines the details of each of the above approaches and their impact on run time and model accuracy.

Thermal Analysis↗

Run Time Improvement Efforts for the Roman Space Telescope Thermal Analysis

Observatory thermal models for large, complex missions, such as the Wide Field InfraRed Survey Telescope (WFIRST) mission, produce an immense amount of data to be processed. Configuration management of the model throughout the project life cycle has mainly focused on which versions of the subsystem models form the current observatory level configuration. However, the results produced by the model are not nearly as well The Roman Space Telescope (RST), formerly known as the Wide Field InfraRed Survey Telescope, is the next great astrophysics observatory mission to follow the James Webb Space Telescope with a planned launch in 2026. As a large scale, flagship mission for NASA with challenging wave front error stability requirements, a single model approach for both thermal discipline analysis and thermo-optical distortion analysis has been used since the early days of the project. In alleviating the need to maintain two separate models for different analysis types, it imposes run time penalties on the thermal analysis with a large model with significant radiation heat exchange. Throughout the lifecycle of the project, the component models have steadily grown in size, resulting in a continuous growth of the overall observatory model with each update and consequently a considerable increase in the model run time. While ongoing efforts to reduce run time are continuously investigated, previous efforts had primarily focused on timestep size and total simulation time to reach quasi-equilibrium. More recently, studies were performed on the total number of radiation couplings (radks) included in the model, which has a nearly linear impact on run time, but increases exponentially with node count. As standard practice for spacecraft analysis, small radks were excluded from the temperature solution based on the assumption that their interchange/view factors have a negligible impact on heat flow. Four approaches were investigated to reduce the model run time while minimizing the impact on accuracy: (1) the Equivalent Radiation Network node, (2) Progressive Radk Inclusion as solution proceeds, (3) Targeted Radk Filtering for critical/non critical areas, and lastly (4) Representation of culled radks with Backloads. Furthermore, the investigation of model run time also revealed that cold cases took noticeably longer to run than hot cases; the root computational inefficiencies were explored along with the computation penalty of linearization of the external radks and recalculation of temperature dependent linear couplings at each timestep. This paper outlines the details of each of the above approaches and their impact on run time and model accuracy.

Thermal Analysis↗

Comparing traditional and Bayesian approaches to ecological meta‐analysis

Abstract Despite the wide application of meta‐analysis in ecology, some of the traditional methods used for meta‐analysis may not perform well given the type of data characteristic of ecological meta‐analyses. We reviewed published meta‐analyses on the ecological impacts of global climate change, evaluating the number of replicates used in the primary studies ( n i ) and the number of studies or records ( k ) that were aggregated to calculate a mean effect size. We used the results of the review in a simulation experiment to assess the performance of conventional frequentist and Bayesian meta‐analysis methods for estimating a mean effect size and its uncertainty interval. Our literature review showed that n i and k were highly variable, distributions were right‐skewed and were generally small (median n i = 5, median k = 44). Our simulations show that the choice of method for calculating uncertainty intervals was critical for obtaining appropriate coverage (close to the nominal value of 0.95). When k was low (<40), 95% coverage was achieved by a confidence interval (CI) based on the t distribution that uses an adjusted standard error (the Hartung–Knapp–Sidik–Jonkman, HKSJ), or by a Bayesian credible interval, whereas bootstrap or z distribution CIs had lower coverage. Despite the importance of the method to calculate the uncertainty interval, 39% of the meta‐analyses reviewed did not report the method used, and of the 61% that did, 94% used a potentially problematic method, which may be a consequence of software defaults. In general, for a simple random‐effects meta‐analysis, the performance of the best frequentist and Bayesian methods was similar for the same combinations of factors ( k and mean replication), though the Bayesian approach had higher than nominal (>95%) coverage for the mean effect when k was very low ( k < 15). Our literature review suggests that many meta‐analyses that used z distribution or bootstrapping CIs may have overestimated the statistical significance of their results when the number of studies was low; more appropriate methods need to be adopted in ecological meta‐analyses.

Pappalardo, Paula↗

Detailed Evaluation of MODIS Fire Radiative Power Measurements

Satellite remote sensing is providing us tremendous opportunities to measure the fire radiative energy (FRE) release rate or power (FRP) from open biomass burning, which affects many vegetated regions of the world on a seasonal basis. Knowledge of the biomass burning characteristics and emission source strengths of different (particulate and gaseous) smoke constituents is one of the principal ingredients upon which the assessment, modeling, and forecasting of their distribution and impacts depend. This knowledge can be gained through accurate measurement of FRP, which has been shown to have a direct relationship with the rates of biomass consumption and emissions of major smoke constituents. Over the last decade or so, FRP has been routinely measured from space by both the MODIS sensors aboard the polar orbiting Terra and Aqua satellites, and the SEVIRI sensor aboard the Meteosat Second Generation (MSG) geostationary satellite. During the last few years, FRP has been gaining recognition as an important parameter for facilitating the development of various scientific studies relating to the quantitative characterization of biomass burning and their emissions. Therefore, we are conducting a detailed analysis of the FRP products from MODIS to characterize the uncertainties associated with them, such as those due to the MODIS bow-tie effects and other factors, in order to establish their error budget for use in scientific research and applications. In this presentation, we will show preliminary results of the MODIS FRP data analysis, including comparisons with airborne measurements.

Ichoku, Charles↗

Underlying Fundamentals of Kalman Filtering for River Network Modeling

The grand challenge of producing hydrometeorological estimates every time and everywhere has motivated the fusion of sparse observations with dense numerical models, with a particular interest on discharge in river modeling. Ensemble methods are largely preferred as they enable the estimation of error properties, but at the expense of computational load and generally with underestimations. These imperfect stochastic estimates motivate the use of correction methods, that is, error localization and inflation, although the physical justifications for their optimality are limited. The purpose of this study is to use one of the simplest forms of data assimilation when applied to river modeling and reveal the underlying mechanisms impacting its performance. Our framework based on assimilating daily averaged in situ discharge measurements to correct daily averaged runoff was tested over a 4-yr case study of two rivers in Texas. Results show that under optimal conditions of inflation and localization, discharge simulations are consistently improved such that the mean values of Nash–Sutcliffe efficiency are enhanced from211.32 to 0.55 at observed gauges and from212.24 to21.10 at validation gauges. Yet, parameters controlling the inflation and the localization have a large impact on the performance. Further investigations of these sensitivities showed that optimal inflation occurs when compensating exactly for discrepancies in the magnitude of errors while optimal localization matches the distance traveled during one assimilation window. These results may be applicable to more advanced data assimilation methods as well as for larger applications motivated by upcoming river-observing satellite missions, such as NASA’s Surface Water and Ocean Topography mission.

Streamflow↗

Optical galaxy cluster mock catalogs with realistic projection effects: Validations with the SDSS clusters

Galaxy clusters identified in optical imaging surveys suffer from projection effects: Physically unassociated galaxies along a cluster’s line of sight can be counted as its members and boost the observed richness (the number of cluster members). To model the impact of projection on cluster cosmology analyses, we apply a halo occupation distribution model to 𝑁-body simulations to simulate the red galaxies contributing to cluster members, and we use the number of galaxies in a cylinder along the line of sight (counts in cylinders) to model the impact of projection on cluster richness. We compare three projection models: uniform, quadratic, and Gaussian, and we convert between them by matching their effective cylinder volumes. We validate our mock catalogs using SDSS redMaPPer clusters’ data vectors, including counts vs richness, stacked lensing signal, spectroscopic redshift distribution of member galaxies, and richness remeasured on a redshift grid. We find the former two are insensitive to the projection model, while the latter two favor a quadratic projection model with a width of ≈180 ℎ −1 Mpc (equivalent to the volume of a uniform model with a width of 100 ℎ −1 Mpc and a Gaussian model with a width of 110 ℎ −1 Mpc, or a Gaussian redshift error of 0.04). Furthermore, our framework provides an efficient and flexible way to model optical cluster data vectors, paving the way for a simulation-based joint analysis for clusters, galaxies, and shear.

79 ASTRONOMY AND ASTROPHYSICS↗

The cleanroom case study in the Software Engineering Laboratory: Project description and early analysis

This case study analyzes the application of the cleanroom software development methodology to the development of production software at the NASA/Goddard Space Flight Center. The cleanroom methodology emphasizes human discipline in program verification to produce reliable software products that are right the first time. Preliminary analysis of the cleanroom case study shows that the method can be applied successfully in the FDD environment and may increase staff productivity and product quality. Compared to typical Software Engineering Laboratory (SEL) activities, there is evidence of lower failure rates, a more complete and consistent set of inline code documentation, a different distribution of phase effort activity, and a different growth profile in terms of lines of code developed. The major goals of the study were to: (1) assess the process used in the SEL cleanroom model with respect to team structure, team activities, and effort distribution; (2) analyze the products of the SEL cleanroom model and determine the impact on measures of interest, including reliability, productivity, overall life-cycle cost, and software quality; and (3) analyze the residual products in the application of the SEL cleanroom model, such as fault distribution, error characteristics, system growth, and computer usage.

Green, Scott↗

A Performance Analysis on Soil Dielectric Models Over Organic Soils in Alaska for Passive Microwave Remote Sensing of Soil Moisture

Passive microwave remote sensing of soil moisture (SM) requires a physically based dielectric model that quantitatively converts the volumetric SM into the soil bulk dielectric constant. Mironov 2009 is the dielectric model used in the operational SM retrieval algorithms of the NASA Soil Moisture Active Passive (SMAP) and the ESA Soil Moisture and Ocean Salinity (SMOS) missions. However, Mironov 2009 suffers a challenge in deriving SM over organic soils, as it does not account for the impact of soil organic matter (SOM) on the soil bulk dielectric constant. To this end, we presented a comparative performance analysis of nine advanced soil dielectric models over organic soil in Alaska, four of which incorporate SOM. In the framework of the SMAP single-channel algorithm at vertical polarization (SCA-V), SM retrievals from different dielectric models were derived using an iterative optimization scheme. The skills of the different dielectric models over organic soils were reflected by the performance of their respective SM retrievals, which was measured by four conventional statistical metrics, calculated by comparing satellite-based SM time series with in-situ benchmarks. Overall, SM retrievals of organic-soil-based dielectric models tended to overestimate, while those from mineral-soil-based models displayed dry biases. All the models showed comparable values of unbiased root-mean-square error (ubRMSE) and Pearson Correlation (R), but Mironov 2019 exhibited a slight but consistent edge over the others. An integrated consideration of the model inputs, the physical basis, and the validated accuracy indicated that the separate use of Mironov 2009 and Mironov 2019 in the SMAP SCA-V for mineral soils (SOM < 15%) and organic soils (SOM ≥ 15%) would be the preferred option.

Soil Moisture↗

Dark energy survey internal consistency tests of the joint cosmological probes analysis with posterior predictive distributions

Beyond ΛCDM, physics or systematic errors may cause subsets of a cosmological data set to appear inconsistent when analysed assuming ΛCDM. Here, we present an application of internal consistency tests to measurements from the Dark Energy Survey Year 1 (DES Y1) joint probes analysis. Our analysis relies on computing the posterior predictive distribution (PPD) for these data under the assumption of ΛCDM. We find that the DES Y1 data have an acceptable goodness of fit to ΛCDM, with a probability of finding a worse fit by random chance of p = 0.046. Using numerical PPD tests, supplemented by graphical checks, we show that most of the data vector appears completely consistent with expectations, although we observe a small tension between large- and small-scale measurements. A small part (roughly 1.5 per cent) of the data vector shows an unusually large departure from expectations; excluding this part of the data has negligible impact on cosmological constraints, but does significantly improve the p-value to 0.10. The methodology developed here will be applied to test the consistency of DES Year 3 joint probes data sets.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

High-n Rydberg transition spectroscopy for heavy impurity transport studies in W7-X (invited)

Here, we present a novel spectroscopy approach to investigate impurity transport by analyzing line-radiation following high-n Rydberg transitions. While high-n Rydberg states of impurity ions are unlikely to be populated via impact excitation, they can be accessed by charge exchange (CX) reactions along the neutral beams in high-temperature plasmas. Hence, localized radiation of highly ionized impurities, free of passive contributions, can be observed at multiple wavelengths in the visible range. For the analysis and modeling of the observed Rydberg transitions, a technique for calculating effective emission coefficients is presented that can well reproduce the energy dependence seen in datasets available on the OPEN-ADAS database. By using the rate coefficients and comparing modeling results with the new high-n Rydberg CX measurements, impurity transport coefficients are determined with well-documented 2σ confidence intervals for the first time. This demonstrates that high-n Rydberg spectroscopy provides important constraints on the determination of impurity transport coefficients. By additionally considering Bolometer measurements, which provide constraints on the overall impurity emissivity and, therefore, impurity densities, error bars can be reduced even further.

Instruments & Instrumentation↗

Propagation of Computational Uncertainty Using the Modern Design of Experiments

This paper describes the use of formally designed experiments to aid in the error analysis of a computational experiment. A method is described by which the underlying code is approximated with relatively low-order polynomial graduating functions represented by truncated Taylor series approximations to the true underlying response function. A resource-minimal approach is outlined by which such graduating functions can be estimated from a minimum number of case runs of the underlying computational code. Certain practical considerations are discussed, including ways and means of coping with high-order response functions. The distributional properties of prediction residuals are presented and discussed. A practical method is presented for quantifying that component of the prediction uncertainty of a computational code that can be attributed to imperfect knowledge of independent variable levels. This method is illustrated with a recent assessment of uncertainty in computational estimates of Space Shuttle thermal and structural reentry loads attributable to ice and foam debris impact on ascent.

DeLoach, Richard↗

Error analysis of a hybrid control drum worth model

This paper presents a perturbation-based model for control drum worth prediction which employs both physics-based and statistics-based components. Control drums, or control shims, are cylindrical in shape and span the axial length of the core. A portion of the cylinder is coated in neutron absorbing material and the drum can rotate to introduce the absorbing material to the body of the core to reduce reactivity. The model can be expensive to create due to the requirement for full-core Monte Carlo eigenvalue calculations. Therefore, it is important to analyze how the errors in Monte Carlo calculated k{sub eff} used for model training affect model performance. It was found that the error in predicted criticalities could average to 70 pcm in the most complex form of the model and 215 pcm in the simplest form of the model. Furthermore, it was found that the Monte Carlo uncertainty in quantities calculated with Serpent used to train the models had minimal impact on the error observed from the model. Lastly, one of the forms of the hybrid model could be trained in considerably less computational time if the Monte Carlo calculations were run to higher uncertainty in k{sub eff} with a small penalty to model performance.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Effects of target fragmentation on evaluation of LET spectra from space radiation in low-earth orbit (LEO) environment: impact on SEU predictions

Recent improvements in the radiation transport code HZETRN/BRYNTRN and galactic cosmic ray environmental model have provided an opportunity to investigate the effects of target fragmentation on estimates of single event upset (SEU) rates for spacecraft memory devices. Since target fragments are mostly of very low energy, an SEU prediction model has been derived in terms of particle energy rather than linear energy transfer (LET) to account for nonlinear relationship between range and energy. Predictions are made for SEU rates observed on two Shuttle flights, each at low and high inclination orbit. Corrections due to track structure effects are made for both high energy ions with track structure larger than device sensitive volume and for low energy ions with dense track where charge recombination is important. Results indicate contributions from target fragments are relatively important at large shield depths (or any thick structure material) and at low inclination orbit. Consequently, a more consistent set of predictions for upset rates observed in these two flights is reached when compared to an earlier analysis with CREME model. It is also observed that the errors produced by assuming linear relationship in range and energy in the earlier analysis have fortuitously canceled out the errors for not considering target fragmentation and track structure effects.

STS-51 Shuttle Project↗

Effects of Target Fragmentation on Evaluation of LET Spectra From Space Radiation in Low-Earth Orbit (LEO) Environment: Impact on SEU Predictions

Recent improvements in the radiation transport code HZETRN/BRYNTRN and galactic cosmic ray environmental model have provided an opportunity to investigate the effects of target fragmentation on estimates of single event upset (SEU) rates for spacecraft memory devices. Since target fragments are mostly of very low energy, an SEU prediction model has been derived in terms of particle energy rather than linear energy transfer (LET) to account for nonlinear relationship between range and energy. Predictions are made for SEU rates observed on two Shuttle flights, each at low and high inclination orbit. Corrections due to track structure effects are made for both high energy ions with track structure larger than device sensitive volume and for low energy ions with dense track where charge recombination is important. Results indicate contributions from target fragments are relatively important at large shield depths (or any thick structure material) and at low inclination orbit. Consequently, a more consistent set of predictions for upset rates observed in these two flights is reached when compared to an earlier analysis with CREME model. It is also observed that the errors produced by assuming linear relationship in range and energy in the earlier analysis have fortuitously canceled out the errors for not considering target fragmentation and track structure effects.

Shinn, J. L.↗