Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Statistical Testing Approach”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRS1987

Contextual classification on the massively parallel processor

Classifiers are often used to produce land cover maps from multispectral Earth observation imagery. Conventionally, these classifiers have been designed to exploit the spectral information contained in the imagery. Very few classifiers exploit the spatial information content of the imagery, and the few that do rarely exploit spatial information content in conjunction with spectral and/or temporal information. A contextual classifier that exploits spatial and spectral information in combination through a general statistical approach was studied. Early test results obtained from an implementation of the classifier on a VAX-11/780 minicomputer were encouraging, but they are of limited meaning because they were produced from small data sets. An implementation of the contextual classifier is presented on the Massively Parallel Processor (MPP) at Goddard that for the first time makes feasible the testing of the classifier on large data sets.

Tilton, James C.↗
NASA NTRS2010

A Ground Flash Fraction Retrieval Algorithm for GLM

A Bayesian inversion method is introduced for retrieving the fraction of ground flashes in a set of N lightning observed by a satellite lightning imager (such as the Geostationary Lightning Mapper, GLM). An exponential model is applied as a physically reasonable constraint to describe the measured lightning optical parameter distributions. Population statistics (i.e., the mean and variance) are invoked to add additional constraints to the retrieval process. The Maximum A Posteriori (MAP) solution is employed. The approach is tested by performing simulated retrievals, and retrieval error statistics are provided. The approach is feasible for N greater than 2000, and retrieval errors decrease as N is increased.

Koshak, William J.↗
NASA NTRS2018

Structural Certification of Human-Rated Inflatable Space Structures

This paper details the results of an initial study to develop a certification plan for human-rated inflatable space structures, including guidelines for qualification testing. Habitable softgoods inflatables are multi-layered shell structures that use high-strength webbing, cordage and broadcloth fabric to carry the skin loads of a variety of volumetric shapes and structural architectures. The primary objectives of this study are to define the key parameters that affect these structures and propose a statistically robust approach to defining safety and knockdown factors based on test and analysis. Current NASA standards for habitable inflatable space structures use a factor of safety of 4, which was inherited from airship design criteria. An updated approach to defining a design factor, taking into account material strength variability, load variability in the article, number of test samples, and damage and degradation effects is specified. Accurate analytical modeling of these structures is hindered by the difficulty of obtaining accurate and consistent material data due to load-history- dependent, nonlinear load versus strain behavior. A building block approach to certification is detailed that uses stochastic modeling and statistical test design and analysis to address the unique challenges these high-strength softgoods structures present. Human-rated inflatable modules are a transformative capability for launching much larger habitable volumes into space than is possible with rigid shell structures. This research aims to provide the framework for certifying these structures for future human space exploration missions.

Jones, Thomas C.↗
NASA NTRS2009

Bayes Analysis and Reliability Implications of Stress-Rupture Testing a Kevlar/Epoxy COPV Using Temperature and Pressure Acceleration

Composite Overwrapped Pressure Vessels (COPVs) that have survived a long service time under pressure generally must be recertified before service is extended. Flight certification is dependent on the reliability analysis to quantify the risk of stress rupture failure in existing flight vessels. Full certification of this reliability model would require a statistically significant number of lifetime tests to be performed and is impractical given the cost and limited flight hardware for certification testing purposes. One approach to confirm the reliability model is to perform a stress rupture test on a flight COPV. Currently, testing of such a Kevlar49 (Dupont)/epoxy COPV is nearing completion. The present paper focuses on a Bayesian statistical approach to analyze the possible failure time results of this test and to assess the implications in choosing between possible model parameter values that in the past have had significant uncertainty. The key uncertain parameters in this case are the actual fiber stress ratio at operating pressure, and the Weibull shape parameter for lifetime; the former has been uncertain due to ambiguities in interpreting the original and a duplicate burst test. The latter has been uncertain due to major differences between COPVs in the database and the actual COPVs in service. Any information obtained that clarifies and eliminates uncertainty in these parameters will have a major effect on the predicted reliability of the service COPVs going forward. The key result is that the longer the vessel survives, the more likely the more optimistic stress ratio model is correct. At the time of writing, the resulting effect on predicted future reliability is dramatic, increasing it by about one "nine," that is, reducing the predicted probability of failure by an order of magnitude. However, testing one vessel does not change the uncertainty on the Weibull shape parameter for lifetime since testing several vessels would be necessary.

Phoenix, S. Leigh↗
DOE OSTI2018

Spatially Dependent Multiple Testing Under Model Misspecification, With Application to Detection of Anthropogenic Influence on Extreme Climate Events

The Weather Risk Attribution Forecast (WRAF) is a forecasting tool that uses output from global climate models to make simultaneous attribution statements about whether and how greenhouse gas emissions have contributed to extreme weather across the globe. However, in conducting a large number of simultaneous hypothesis tests, the WRAF is prone to identifying false “discoveries.” A common technique for addressing this multiple testing problem is to adjust the procedure in a way that controls the proportion of true null hypotheses that are incorrectly rejected, or the false discovery rate (FDR). Unfortunately, generic FDR procedures suffer from low power when the hypotheses are dependent, and techniques designed to account for dependence are sensitive to misspecification of the underlying statistical model. In this paper, we develop a Bayesian decision-theoretical approach for dependent multiple testing and a nonparametric hierarchical statistical model that flexibly controls false discovery and is robust to model misspecification. We illustrate the robustness of our procedure to model error with a simulation study, using a framework that accounts for generic spatial dependence and allows the practitioner to flexibly specify the decision criteria. Finally, we apply our procedure to several seasonal forecasts and discuss implementation for the WRAF workflow. Lastly, supplementary materials for this article, including a standardized description of the materials available for reproducing the work, are available as an online supplement.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2016

Nasa Space Flight Instruments: Cost Time Trends

Are NASA’s space flight instruments becoming cheaper or more expensive as time marches forward? After analyzing the costs of hundreds of instruments launched over the last 30 years, the short answer to this question is no… and yes. This paper gives a visual analysis of the cost time trends for various NASA space flight instrument types, such as optical, particles detectors, fields detectors and microwave instruments. In addition to the statistical approaches utilized, such as significance tests, cluster analysis and principle components analysis (PCA), we will also discuss the intangibles which are likely at play, including technological progress, NASA policy and the luck of the draw associated with mission manifests. This analysis was performed as the main driver for the NASA Instrument Cost Model (NICM) recent cost estimating model redesign. Started in 2004, the first version of NICM was based off of instruments launched from 1985-2005, or 20 years’ worth of data. As NICM hit its 10-year anniversary, we wanted to know: should NICM continue to only use the most recent 20 years’ worth of data (1995-2015)? Are instruments becoming cheaper or more expensive as time marches forward? There is evidence in favor of a drop in the median dollar-per-kg value across some instrument types, but little in others. Whereas further research is needed to substantiate, Particles and Optical-Planetary instrument types show moderate to strong evidence of a downward trend in dollar-per-kg. Further research is required to study the nature of this trend (shift, taper, cyclic, etc.). Little evidence for a similar downward trend was detected for Fields or Microwave instruments, or Optical instruments on Earth Orbiting spacecraft. We presented evidence in favor of a drop in the median dollar-per-kg value for Particles and Optical-Planetary instrument types. While similar evidence was weak at best for Fields and Microwave instruments. We can speculate as to the causes for this effect, but we are also equipped to begin to rule out, or at least prioritize, some of the suspected drivers. We observed, for Particles and Optical-Planetary instruments, that perhaps a launch manifest effect was playing part of the role in the observed decrease in dollar-per-kg over the years, noting that the more flagship class missions, which have more money to spend on their instruments, were seen in the earlier years in our data, versus the later years which were dominated by less expensive class missions. However, if this were a dominating driver, would we not have seen the downward trend in the Fields and Microwave instruments as well, which were drawn from that same launch manifest? The fact that we did not observe this helps us rule out the launch manifest effect, and other drivers, such as advances in technology, that seem to be more likely suspects. In that case, however, why would technology advances be helping the Particles and Optical-Planetary instruments only? Why would it not be impacting Optical-Earth Orbiting instruments? Further suspects were looked at as well and ruled out, such as the “Faster, Better, Cheaper”era of NASA development which did not seem to actually impact trends by instrument type on a dollar-perkg scale. VI. Future Work A. Time Series Detailed Statistical Assessment The analysis discussed above sets the foundation for a more rigorous time series analysis of the data. Time series analysis will further explore evidence to-date of time trends for the instrument types which showed the strongest indicators for a decrease in dollar-per-kg: Optical (Planetary) and Particles instruments. More than providing evidence and top-level significance tests, time series analysis would help elucidate what kind of trend that exists in the data, their significance and allow statistically based forecasting (see Figure 10)

Mrozinksi, Joseph↗
DOE OSTI2021

Thermodynamics of order and randomness in dopant distributions inferred from atomically resolved imaging

Abstract Exploration of structure-property relationships as a function of dopant concentration is commonly based on mean field theories for solid solutions. However, such theories that work well for semiconductors tend to fail in materials with strong correlations, either in electronic behavior or chemical segregation. In these cases, the details of atomic arrangements are generally not explored and analyzed. The knowledge of the generative physics and chemistry of the material can obviate this problem, since defect configuration libraries as stochastic representation of atomic level structures can be generated, or parameters of mesoscopic thermodynamic models can be derived. To obtain such information for improved predictions, we use data from atomically resolved microscopic images that visualize complex structural correlations within the system and translate them into statistical mechanical models of structure formation. Given the significant uncertainties about the microscopic aspects of the material’s processing history along with the limited number of available images, we combine model optimization techniques with the principles of statistical hypothesis testing. We demonstrate the approach on data from a series of atomically-resolved scanning transmission electron microscopy images of Mo x Re 1- x S 2 at varying ratios of Mo/Re stoichiometries, for which we propose an effective interaction model that is then used to generate atomic configurations and make testable predictions at a range of concentrations and formation temperatures.

25 ENERGY STORAGE↗
DOE OSTI2020

XRD-PUAT (X-Ray Diffraction - Parameter Uncertainty Analysis Toolkit)

This XRD uncertainty toolkit was built to investigate uncertainty and local least squares topology over a specified parameter space on a single histogram in a gpx GSAS-II file. The parameter uncertainty can be investigated using either a frequentist F-test approach or a Bayesian Inference statistical inversion method on a weighted least squares or peak fit refinements.SAND Number: SAND2020-12227 M Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Moore, Alexander↗
NASA NTRS1980

Estimating the chlorophyll content in the waters of Guanabara Bay from the LANDSAT multispectral scanning digital data

Detection of water quality in Guanabara Bay using multispectral scanning digital data taken from LANDSAT satellites was examined. To test these processes, an empirical (statistical) approach was choosen to observe the degree of relationship between LANDSAT data and the in situ data taken simultaneously. The linear and nonlinear regression analyses were taken from among those developed by INPE in 1978. Results indicate that the major regression was in the number six MSS band, atmospheric effects, which indicated a correction coefficient of 0.99 and an average error of 6.59 micrograms liter. This error was similar to that obtained in the laboratory. The chlorophyll content was between 0 and 100 micrograms/liter, as taken from the MSS of LANDSAT.

Dejesusparada, N.↗
DOE OSTI2016

Integrating Symbolic and Statistical Methods for Testing Intelligent Systems Applications to Machine Learning and Computer Vision

Embedded intelligent systems ranging from tiny im- plantable biomedical devices to large swarms of autonomous un- manned aerial systems are becoming pervasive in our daily lives. While we depend on the flawless functioning of such intelligent systems, and often take their behavioral correctness and safety for granted, it is notoriously difficult to generate test cases that expose subtle errors in the implementations of machine learning algorithms. Hence, the validation of intelligent systems is usually achieved by studying their behavior on representative data sets, using methods such as cross-validation and bootstrapping.In this paper, we present a new testing methodology for studying the correctness of intelligent systems. Our approach uses symbolic decision procedures coupled with statistical hypothesis testing to. We also use our algorithm to analyze the robustness of a human detection algorithm built using the OpenCV open-source computer vision library. We show that the human detection implementation can fail to detect humans in perturbed video frames even when the perturbations are so small that the corresponding frames look identical to the naked eye.

Jha, Sumit Kumar↗
DOE OSTI2018

Bootstrap inversion technique for atmospheric trace gas source detection and quantification using long open-path laser measurements

Advances in natural gas extraction technology have led to increased activity in the production and transport sectors in the United States and, as a consequence, an increased need for reliable monitoring of methane leaks to the atmosphere. We present a statistical methodology in combination with an observing system for the detection and attribution of fugitive emissions of methane from distributed potential source location landscapes such as natural gas production sites. We measure long (> 500 m), integrated open-path concentrations of atmospheric methane using a dual frequency comb spectrometer and combine measurements with an atmospheric transport model to infer leak locations and strengths using a novel statistical method, the non-zero minimum bootstrap (NZMB). The new statistical method allows us to determine whether the empirical distribution of possible source strengths for a given location excludes zero. Using this information, we identify leaking source locations (i.e., natural gas wells) through rejection of the null hypothesis that the source is not leaking. The method is tested with a series of synthetic data inversions with varying measurement density and varying levels of model–data mismatch. It is also tested with field observations of (1) a non-leaking source location and (2) a source location where a controlled emission of 3.1 × 10 -5 kg s -1 of methane gas is released over a period of several hours. This series of synthetic data tests and outdoor field observations using a controlled methane release demonstrates the viability of the approach for the detection and sizing of very small leaks of methane across large distances (4 + km 2 in synthetic tests). The field tests demonstrate the ability to attribute small atmospheric enhancements of 17 ppb to the emitting source location against a background of combined atmospheric (e.g., background methane variability) and measurement uncertainty of 5 ppb (1 σ ), when measurements are averaged over 2 min. The results of the synthetic and field data testing show that the new observing system and statistical approach greatly decreases the incidence of false alarms (that is, wrongly identifying a well site to be leaking) compared with the same tests that do not use the NZMB approach and therefore offers increased leak detection and sizing capabilities.

03 NATURAL GAS↗
DOE OSTI2017

A statistical approach to combining multisource information in one-class classifiers

A new method is introduced in this paper for combining information from multiple sources to support one-class classification. The contributing sources may represent measurements taken by different sensors of the same physical entity, repeated measurements by a single sensor, or numerous features computed from a single measured image or signal. The approach utilizes the theory of statistical hypothesis testing, and applies Fisher's technique for combining p-values, modified to handle nonindependent sources. Classifier outputs take the form of fused p-values, which may be used to gauge the consistency of unknown entities with one or more class hypotheses. The approach enables rigorous assessment of classification uncertainties, and allows for traceability of classifier decisions back to the constituent sources, both of which are important for high-consequence decision support. Application of the technique is illustrated in two challenge problems, one for skin segmentation and the other for terrain labeling. Finally, the method is seen to be particularly effective for relatively small training samples.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Shared micromobility as a first- and last-mile transit solution? Spatiotemporal insights from a novel dataset

The first- and last-mile (FM/LM) problem is a major deterrent to public transit use. With the rise of shared micromobility options such as shared e-scooters in recent years, there is a growing interest in understanding their potential to serve as a last-mile transit solution. However, empirical data regarding the integrated use of shared micromobility and public transit have been limited so far. As a result, much is unknown regarding the spatiotemporal patterns and characteristics of shared micromobility trips serving as an FM/LM connection to transit. Here, this paper addresses these knowledge gaps by leveraging a novel dataset (i.e., the Spin post-ride survey dataset) that records thousands of transit-connecting shared e-scooter trips in Washington DC. Specifically, we used the dataset to reveal the spatiotemporal patterns of transit-connecting shared e-scooter trips in Washington DC, resulting in some major policy insights regarding the integral use of shared e-scooters and public transit. We further leveraged the dataset to validate if and to what extent a commonly applied buffer-zone approach can infer FM/LM micromobility trips accurately. Statistical tests showed that the actual FM/LM Spin e-scooter trips differ from inferred FM/LM Spin e-scooter trips in both spatial and temporal dimensions. This indicates that the common practice of inferring FM/LM micromobility trips with a buffer-zone approach can lead to inaccurate estimates of transit-connecting micromobility trips.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2022

Testing the impact of electromagnetic fields on the directed flow of constituent quarks in heavy-ion collisions

It has been proposed that strong electromagnetic fields produced in the early stages of heavy-ion collisions can lead to splitting of the rapidity-odd directed flow of positive and negative hadrons. For light hadrons, the interpretation of such measurements is complicated by the low magnitude of directed flow as well as by ambiguities arising from transported quarks. To overcome these complications, we propose measurements using only hadrons carrying produced quarks (u¯, d¯, s s¯ ). Here, we discuss how to identify the kinematics where such hadrons are produced via the coalescence mechanism and therefore their flow is the sum of the flow of their constituent quarks. With this sum rule verified for certain combinations of hadrons, the expected systematic violation of this rule with increasing electric charge can be measured, which could be a consequence of the electromagnetic fields produced in the collisions. Our approach can be tested with the high statistics data from Phase II of the Beam Energy Scan program at the Relativistic Heavy Ion Collider.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2015

An Overview of High-performance Parallel Big Data transfers over multiple network channels with Transport Layer Security (TLS) and TLS plus Perfect Forward Secrecy (PFS)

This Technical Note provides an overview of high-performance parallel Big Data transfers with and without encryption for data in-transit over multiple network channels. It shows that with the parallel approach, it is feasible to carry out high-performance parallel "encrypted" Big Data transfers without serious impact to throughput. But other impacts, e.g. the energy-consumption part should be investigated. It also explains our rationales of using a statistics-based approach for gaining understanding from test results and for improving the system. The presentation is of high-level nature. Nevertheless, at the end we will pose some questions and identify potentially fruitful directions for future work.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1974

A statistical approach to deriving subsystem specifications

In order to produce cost effective environmental test programs, the test specifications must be realistic and to be useful, they must be available early in the life of a program. This paper describes a method for achieving such specifications for subsystems by utilizing the results of a statistical analysis of data acquired at subsystem mounting locations during system level environmental tests. The paper describes the details of this statistical analysis. The resultant recommended levels are a function of the subsystems' mounting location in the spacecraft. Methods of determining this mounting 'zone' are described. Recommendations are then made as to which of the various problem areas encountered should be pursued further.

Keegan, W. B.↗
DOE OSTI2018

Challenges of Using Indistinguishable from Background as a Release Criterion at FUSRAP Projects - 18157

The US Army Corps of Engineers (USACE) utilizes the Multi-Agency Radiation Survey and Site Investigation Manual (MARSSIM) approach to site release on Formerly Utilized Sites Remedial Action Program (FUSRAP) projects. Use of Indistinguishable from Background (IFB) as a release criterion is discussed in MARSSIM as Scenario B; however, the discussion is arguably limited and refers the user to other documents. USACE has utilized the IFB approach at select FUSRAP projects. USACE experience is that use of IFB as a release criterion has been difficult to implement at best. There are many challenges in using the IFB approach. These include: release criterion is required per MARSSIM; multiple reference areas need to be established; and data presentation. Potential methods to overcome these challenges are discussed. USACE has utilized alternative statistical approaches as well as the IFB approach discussed in MARSSIM for specific purposes at FUSRAP sites. Alternatives to the IFB approach discussed in this paper include other statistical tests such as the Tukey Kramer test. These alternatives may be better suited to release an area of a property prior to the development of release criteria. USACE has found using the IFB criterion useful in certain specific circumstances. These include post MARSSIM Scenario A final status survey data evaluations and for review of historical data. The challenges, alternatives, and circumstances to use the IFB approach are discussed. Other environmental remediation projects attempting to utilize IFB as a criterion may benefit from the USACE experiences as discussed herein. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
NASA NTRS2010

Development of an Uncertainty Model for the National Transonic Facility

This paper introduces an uncertainty model being developed for the National Transonic Facility (NTF). The model uses a Monte Carlo technique to propagate standard uncertainties of measured values through the NTF data reduction equations to calculate the combined uncertainties of the key aerodynamic force and moment coefficients and freestream properties. The uncertainty propagation approach to assessing data variability is compared with ongoing data quality assessment activities at the NTF, notably check standard testing using statistical process control (SPC) techniques. It is shown that the two approaches are complementary and both are necessary tools for data quality assessment and improvement activities. The SPC approach is the final arbiter of variability in a facility. Its result encompasses variation due to people, processes, test equipment, and test article. The uncertainty propagation approach is limited mainly to the data reduction process. However, it is useful because it helps to assess the causes of variability seen in the data and consequently provides a basis for improvement. For example, it is shown that Mach number random uncertainty is dominated by static pressure variation over most of the dynamic pressure range tested. However, the random uncertainty in the drag coefficient is generally dominated by axial and normal force uncertainty with much less contribution from freestream conditions.

Walter, Joel A.↗