Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Hypothesis testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Upwind Buildings Effects on Test Section Performance of the NFAC 80- by 120-Foot Wind Tunnel

In 2013, a new development began that would put multi-story buildings within the exclusion zone of the 80- by 120- Foot Wind Tunnel (80x120) inlet that exceeded the then existing building height restriction. This raised questions concerning whether the tunnel could operate within the acceptable turbulence limits of ≤ 0.5% in the axial direction over 75% area in the wind tunnel test section. The goal of this testing was to determine the validity of this hypothesis. Testing was performed using the 1/50th-scale model of the 80- by 120-FootWind Tunnel within the full-scale facility test section to characterize the model’s test section turbulence levels as a function of wind direction with the presence of the upwind buildings that are currently being constructed. This paper reports the first testing performed to date on the 80x120 inlet as-built under controlled test conditions for wind magnitude and wind onset direction for a variety of buildings in the inlet external flow field, including no buildings (the control case). This testing determined that, at rated operational conditions the upwind blockages caused no increase in turbulence intensity levels measured in the test section and, in some limited cases, caused the turbulence levels to decrease. Lastly, looking at generic rectangular blockages showed that, to have a discernible effect on test section turbulence levels, scaled 100-ft tall blockages need to be within 600 ft of the existing inlet.

Upwind Buildings Effects↗
DOE OSTI2023

Walking the ‘design–build–test–learn’ cycle: flux analysis and genetic engineering reveal the pliability of plant central metabolism

Oilseeds are of great economic importance for food and animal feed and their contribution to renewable energy production. Soybean seeds (Glycine max (L.) Merr.) contain c. 40% protein, 20% oil, and 30% carbohydrate (Song et al., 2023). Due to the massive scale of soybean production worldwide, even small improvements in seed protein and oil content make economic sense (Song et al., 2023). Successful manipulation of seed composition largely depends on a thorough understanding of the processes and pathways involved in the biosynthesis of fatty acids and amino acids, which are the building blocks of lipids and proteins. Rational engineering of the synthesis of storage reserves, that is, the rerouting of metabolic flux in central metabolism, is difficult to accomplish due to the complexity of the central metabolic network, the intricate regulation of its enzymes at multiple levels, and the often-unpredictable effects of genetic manipulation (Sweetlove et al., 2017). Therefore, the advancement of our understanding of central metabolism and its control of carbon partitioning requires following an iterative ‘design–build–test–learn’ (DBTL) cycle (Lin & Eudes, 2020) where metabolic flux analysis and hypothesis testing by transgenic approaches are important components. Previous metabolic studies on soybeans using isotopic tracers and metabolic flux analysis have provided insight into how lipid and protein biosynthesis occurs simultaneously during seed development (Allen et al., 2009; Allen & Young, 2013; Kambhampati et al., 2021). In an article published in this issue of New Phytologist, Morley et al. (2023; 1834–1851) put the insights they have gained into the delivery of metabolic precursors and energy cofactors to oil synthesis to the test and arrive at a successful metabolic engineering design. They show that an increase in seed oil content in soybeans can be achieved by overexpression of malic enzyme (ME) during seed development. Malic enzyme refers to a class of decarboxylating malate dehydrogenase enzymes that oxidize malate with NAD + or NADP + as redox cofactor while generating pyruvate and CO 2 . Like higher plants in general, soybean has distinct NADH- or NADPH-producing ME isoforms localized to the cytosol, plastid, or mitochondria (Gerrard Wheeler et al., 2016). As Morley et al. show, an increase in seed oil can be achieved in particular when a NADP+-dependent enzyme isoform (EC 1.1.1.40) is overexpressed in the plastid. Given the complex compartmentalization of pyruvate, malate, and redox metabolism (Fig. 1), increased oil production appears to depend on additional pyruvate and reducing equivalents being produced in the same compartment where de novo fatty acid biosynthesis occurs: the plastid.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS1987

An algorithmic approach to crustal deformation analysis

In recent years the analysis of crustal deformation measurements has become important as a result of current improvements in geodetic methods and an increasing amount of theoretical and observational data provided by several earth sciences. A first-generation data analysis algorithm which combines a priori information with current geodetic measurements was proposed. Relevant methods which can be used in the algorithm were discussed. Prior information is the unifying feature of this algorithm. Some of the problems which may arise through the use of a priori information in the analysis were indicated and preventive measures were demonstrated. The first step in the algorithm is the optimal design of deformation networks. The second step in the algorithm identifies the descriptive model of the deformation field. The final step in the algorithm is the improved estimation of deformation parameters. Although deformation parameters are estimated in the process of model discrimination, they can further be improved by the use of a priori information about them. According to the proposed algorithm this information must first be tested against the estimates calculated using the sample data only. Null-hypothesis testing procedures were developed for this purpose. Six different estimators which employ a priori information were examined. Emphasis was put on the case when the prior information is wrong and analytical expressions for possible improvements under incompatible prior information were derived.

Iz, Huseyin Baki↗
NASA NTRS1992

Fault detection using a two-model test for changes in the parameters of an autoregressive time series

This article describes an investigation of a statistical hypothesis testing method for detecting changes in the characteristics of an observed time series. The work is motivated by the need for practical automated methods for on-line monitoring of Deep Space Network (DSN) equipment to detect failures and changes in behavior. In particular, on-line monitoring of the motor current in a DSN 34-m beam waveguide (BWG) antenna is used as an example. The algorithm is based on a measure of the information theoretic distance between two autoregressive models: one estimated with data from a dynamic reference window and one estimated with data from a sliding reference window. The Hinkley cumulative sum stopping rule is utilized to detect a change in the mean of this distance measure, corresponding to the detection of a change in the underlying process. The basic theory behind this two-model test is presented, and the problem of practical implementation is addressed, examining windowing methods, model estimation, and detection parameter assignment. Results from the five fault-transition simulations are presented to show the possible limitations of the detection method, and suggestions for future implementation are given.

Scholtz, P.↗
DOE OSTI2016

Integrating Symbolic and Statistical Methods for Testing Intelligent Systems Applications to Machine Learning and Computer Vision

Embedded intelligent systems ranging from tiny im- plantable biomedical devices to large swarms of autonomous un- manned aerial systems are becoming pervasive in our daily lives. While we depend on the flawless functioning of such intelligent systems, and often take their behavioral correctness and safety for granted, it is notoriously difficult to generate test cases that expose subtle errors in the implementations of machine learning algorithms. Hence, the validation of intelligent systems is usually achieved by studying their behavior on representative data sets, using methods such as cross-validation and bootstrapping.In this paper, we present a new testing methodology for studying the correctness of intelligent systems. Our approach uses symbolic decision procedures coupled with statistical hypothesis testing to. We also use our algorithm to analyze the robustness of a human detection algorithm built using the OpenCV open-source computer vision library. We show that the human detection implementation can fail to detect humans in perturbed video frames even when the perturbations are so small that the corresponding frames look identical to the naked eye.

Jha, Sumit Kumar↗
NASA NTRS1981

Vector wind profile gust model

To enable development of a vector wind gust model suitable for orbital flight test operations and trade studies, hypotheses concerning the distributions of gust component variables were verified. Methods for verification of hypotheses that observed gust variables, including gust component magnitude, gust length, u range, and L range, are gamma distributed and presented. Observed gust modulus has been drawn from a bivariate gamma distribution that can be approximated with a Weibull distribution. Zonal and meridional gust components are bivariate gamma distributed. An analytical method for testing for bivariate gamma distributed variables is presented. Two distributions for gust modulus are described and the results of extensive hypothesis testing of one of the distributions are presented. The validity of the gamma distribution for representation of gust component variables is established.

Adelfang, S. I.↗
NASA NTRSDate not supplied

Individual Response Trends to Urban Air Mobility Noise in A Laboratory Study

Advanced Air Mobility, of which Urban Air Mobility (UAM) is a subset, presents new opportunities for more dynamic aviation transportation systems. It is essential to understand the human response to the noise of these vehicles for sustainable operations. This research aims to investigate the relationship between the noise level and number of events on individual human annoyance to UAM vehicle noise. A “noise and number” laboratory psychoacoustic study was conducted in the Exterior Effects Room at NASA Langley Research Center in January 2023. A total of 38 participants listened to 4-minute audio clips of UAM vehicle flyover noise and provided their annoyance response on an 11-point numerical scale. A test hypothesis was that annoyance to multiple flyovers can be found by adding the annoyance responses to individual flyovers. Results for the overall test subject pool suggest that annoyance grows faster than hypothesized as the number of aircraft events increases. This trend of faster annoyance growth only seems to hold when the individual flyovers are heard as distinct events. Parsing the data down to individuals provides further insight into the mechanisms contributing to the increased annoyance with number of events.

noise and number↗
NASA NTRS2021

Sequential Wald Test Employing a Constrained Filter Bank: Application to Spacecraft Conjunctions

A binary Wald sequential probability ratio test that uses the residuals of norm-inequality-constrained Kalman filters for its likelihood ratio may be employed for a class of compound hypothesis tests on non-stationary systems. The hypotheses concern an inequality constraint on the norm of some elements of the system state. Each of two filters minimizes the summed-squares of its estimation errors subject to one or the other direction of the inequality constraint. This solution is motivated by the problem of satellite conjunction assessment, wherein the constraint concerns the close approach distance between two space objects. The outcome of the test can inform decisions concerning risk mitigation maneuvers.

J. Russell Carpenter↗
NASA NTRS2019

Satellite Conjunction Assessment Risk Analysis for "Dilution Region" Events: Issues and Operational Approaches

An important activity within Space Traffic Management is the detection and prevention of possible on-orbit collisions between space objects. The principal parameter for assessing collision likelihood is the probability of collision, which is widely accepted among conjunction assessment practitioners; but it possesses a known deficiency in that it can produce a false sense of safety when the orbital position uncertainties for the conjuncting objects are high. The probability of collision is said to be “diluted” in such a situation and to understate the possible risk; certain approaches have been recommended by researchers to provide (largely conservative) risk estimates and remediation methodologies in these cases. The present analysis explores two of the main proposals for quantifying and remediating possible risk in the dilution region and quantifies their operational implications. These implications with regard to imputed additional workload are considerable, especially in anticipating the conjunction event levels expected with the deployment of the USAF Space Fence radar. This effort has been undertaken as part of a larger enterprise that seeks to clarify the philosophical and statistical underpinnings of the conjunction risk assessment process. The analysis presented herein argues that a form of hypothesis testing is implicitly used in conjunction assessment risk analysis, and that there are a number of conceptual and practical reasons for constructing the associated null hypothesis to counsel against a satellite conjunction remediation action. In short, it is concluded that, for the purposes of determining whether a conjunction remediation action should be pursued, dilution-region probabilities of collision should be treated no differently from those produced under other circumstances.

Probability of collision↗
NASA NTRS2005

Surveillance system and method having an adaptive sequential probability fault detection test

System and method providing surveillance of an asset such as a process and/or apparatus by providing training and surveillance procedures that numerically fit a probability density function to an observed residual error signal distribution that is correlative to normal asset operation and then utilizes the fitted probability density function in a dynamic statistical hypothesis test for providing improved asset surveillance.

Herzog, James P.↗
NASA NTRS2006

Surveillance system and method having an adaptive sequential probability fault detection test

System and method providing surveillance of an asset such as a process and/or apparatus by providing training and surveillance procedures that numerically fit a probability density function to an observed residual error signal distribution that is correlative to normal asset operation and then utilizes the fitted probability density function in a dynamic statistical hypothesis test for providing improved asset surveillance.

Bickford, Randall L.↗
NASA NTRS2008

Surveillance System and Method having an Adaptive Sequential Probability Fault Detection Test

System and method providing surveillance of an asset such as a process and/or apparatus by providing training and surveillance procedures that numerically fit a probability density function to an observed residual error signal distribution that is correlative to normal asset operation and then utilizes the fitted probability density function in a dynamic statistical hypothesis test for providing improved asset surveillance.

Bickford, Randall L.↗
DOE OSTI2017

Evaluation of Smart Grid Technologies Employed for System Reliability Improvement: Pacific Northwest Smart Grid Demonstration Experience

The Pacific Northwest Smart Grid Demonstration (PNWSGD) was a smart grid technology performance evaluation project that included multiple U.S. states and cooperation from multiple electric utilities in the northwest region. One of the local objectives for the project was to achieve improved distribution system reliability. Toward this end, some PNWSGD utilities automated their distribution systems, including the application of fault detection, isolation, and restoration and advanced metering infrastructure. In light of this investment, a major challenge was to establish a correlation between implementation of these smart grid technologies and actual improvements of distribution system reliability. This paper proposes using Welch’s t-test to objectively determine and quantify whether distribution system reliability is improving over time. The proposed methodology is generic, and it can be implemented by any utility after calculation of the standard reliability indices. The effectiveness of the proposed hypothesis testing approach is demonstrated through comprehensive practical results. It is believed that wider adoption of the proposed approach can help utilities to evaluate a realistic long-term performance of smart grid technologies.

Agalgaonkar, Yashodhan P. (ORCID:0000000290061848)↗
DOE OSTI2018

Spatially Dependent Multiple Testing Under Model Misspecification, With Application to Detection of Anthropogenic Influence on Extreme Climate Events

The Weather Risk Attribution Forecast (WRAF) is a forecasting tool that uses output from global climate models to make simultaneous attribution statements about whether and how greenhouse gas emissions have contributed to extreme weather across the globe. However, in conducting a large number of simultaneous hypothesis tests, the WRAF is prone to identifying false “discoveries.” A common technique for addressing this multiple testing problem is to adjust the procedure in a way that controls the proportion of true null hypotheses that are incorrectly rejected, or the false discovery rate (FDR). Unfortunately, generic FDR procedures suffer from low power when the hypotheses are dependent, and techniques designed to account for dependence are sensitive to misspecification of the underlying statistical model. In this paper, we develop a Bayesian decision-theoretical approach for dependent multiple testing and a nonparametric hierarchical statistical model that flexibly controls false discovery and is robust to model misspecification. We illustrate the robustness of our procedure to model error with a simulation study, using a framework that accounts for generic spatial dependence and allows the practitioner to flexibly specify the decision criteria. Finally, we apply our procedure to several seasonal forecasts and discuss implementation for the WRAF workflow. Lastly, supplementary materials for this article, including a standardized description of the materials available for reproducing the work, are available as an online supplement.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1993

Automated system function allocation and display format: Task information processing requirements

An important consideration when designing the interface to an intelligent system concerns function allocation between the system and the user. The display of information could be held constant, or 'fixed', leaving the user with the task of searching through all of the available information, integrating it, and classifying the data into a known system state. On the other hand, the system, based on its own intelligent diagnosis, could display only relevant information in order to reduce the user's search set. The user would still be left the task of perceiving and integrating the data and classifying it into the appropriate system state. Finally, the system could display the patterns of data. In this scenario, the task of integrating the data is carried out by the system, and the user's information processing load is reduced, leaving only the tasks of perception and classification of the patterns of data. Humans are especially adept at this form of display processing. Although others have examined the relative effectiveness of alphanumeric and graphical display formats, it is interesting to reexamine this issue together with the function allocation problem. Currently, Johnson Space Center is the test site for an intelligent Thermal Control System (TCS), TEXSYS, being tested for use with Space Station Freedom. Expert TCS engineers, as well as novices, were asked to classify several displays of TEXSYS data into various system states (including nominal and anomalous states). Three different display formats were used: fixed, subset, and graphical. The hypothesis tested was that the graphical displays would provide for fewer errors and faster classification times by both experts and novices, regardless of the kind of system state represented within the display. The subset displays were hypothesized to be the second most effective display format/function allocation condition, based on the fact that the search set is reduced in these displays. Both the subset and the graphic display conditions were hypothesized to be processed more efficiently than the fixed display conditions.

Czerwinski, Mary P.↗
DOE OSTI2019

Variational autoencoders for new physics mining at the Large Hadron Collider

Using variational autoencoders trained on known physics processes, we develop a one-sided threshold test to isolate previously unseen processes as outlier events. Since the autoencoder training does not depend on any specific new physics signature, the proposed procedure doesn’t make specific assumptions on the nature of new physics. An event selection based on this algorithm would be complementary to classic LHC searches, typically based on model-dependent hypothesis testing. Such an algorithm would deliver a list of anomalous events, that the experimental collaborations could further scrutinize and even release as a catalog, similarly to what is typically done in other scientific domains. Event topologies repeating in this dataset could inspire new-physics model building and new experimental searches. Running in the trigger system of the LHC experiments, such an application could identify anomalous events that would be otherwise lost, extending the scientific reach of the LHC.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

Time-Based CAN IDS Paper Results Code

Modern vehicles are complex cyber-physical systems made of hundreds of electronic control units (ECUs) that communicate over controller area networks (CANs). This inherited complexity has expanded the CAN attack surface which is vulnerable to message injection attacks. These injections change the overall timing characteristics of messages on the bus, and thus, to detect these malicious messages, time-based intrusion detection systems (IDSs) have been proposed. However, time-based IDSs are usually trained and tested on low-fidelity datasets with unrealistic, labeled attacks. This makes difficult the task of evaluating, comparing, and validating IDSs. Here we detail and benchmark four time-based IDSs against the newly published ROAD dataset, the first open CAN IDS dataset with real (non-simulated) stealthy attacks with physically verified effects. We found that methods that perform hypothesis testing by explicitly estimating message timing distributions have lower performance than methods that seek anomalies in a distribution related statistic. In particular, these “distribution-agnostic” based methods outperform “distribution-based” methods by at least 55% in area under the precision-recall curve (AUC-PR). Our results expand the body of knowledge of CAN time-based IDSs by providing details of these methods and reporting their results when tested on datasets with real advanced attacks. Finally, we develop an after-market plug-in detector using lightweight hardware, which can be used to deploy the best performing IDS method on nearly any vehicle.

Moriano, Pablo [Oak Ridge National Lab. (ORNL), Oa↗
NASA NTRS2006

Using the Bootstrap Method for a Statistical Significance Test of Differences between Summary Histograms

A new method is proposed to compare statistical differences between summary histograms, which are the histograms summed over a large ensemble of individual histograms. It consists of choosing a distance statistic for measuring the difference between summary histograms and using a bootstrap procedure to calculate the statistical significance level. Bootstrapping is an approach to statistical inference that makes few assumptions about the underlying probability distribution that describes the data. Three distance statistics are compared in this study. They are the Euclidean distance, the Jeffries-Matusita distance and the Kuiper distance. The data used in testing the bootstrap method are satellite measurements of cloud systems called cloud objects. Each cloud object is defined as a contiguous region/patch composed of individual footprints or fields of view. A histogram of measured values over footprints is generated for each parameter of each cloud object and then summary histograms are accumulated over all individual histograms in a given cloud-object size category. The results of statistical hypothesis tests using all three distances as test statistics are generally similar, indicating the validity of the proposed method. The Euclidean distance is determined to be most suitable after comparing the statistical tests of several parameters with distinct probability distributions among three cloud-object size categories. Impacts on the statistical significance levels resulting from differences in the total lengths of satellite footprint data between two size categories are also discussed.

Xu, Kuan-Man↗