Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Hypothesis testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

Variational autoencoders for new physics mining at the Large Hadron Collider

Using variational autoencoders trained on known physics processes, we develop a one-sided threshold test to isolate previously unseen processes as outlier events. Since the autoencoder training does not depend on any specific new physics signature, the proposed procedure doesn’t make specific assumptions on the nature of new physics. An event selection based on this algorithm would be complementary to classic LHC searches, typically based on model-dependent hypothesis testing. Such an algorithm would deliver a list of anomalous events, that the experimental collaborations could further scrutinize and even release as a catalog, similarly to what is typically done in other scientific domains. Event topologies repeating in this dataset could inspire new-physics model building and new experimental searches. Running in the trigger system of the LHC experiments, such an application could identify anomalous events that would be otherwise lost, extending the scientific reach of the LHC.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Time-Based CAN IDS Paper Results Code

Modern vehicles are complex cyber-physical systems made of hundreds of electronic control units (ECUs) that communicate over controller area networks (CANs). This inherited complexity has expanded the CAN attack surface which is vulnerable to message injection attacks. These injections change the overall timing characteristics of messages on the bus, and thus, to detect these malicious messages, time-based intrusion detection systems (IDSs) have been proposed. However, time-based IDSs are usually trained and tested on low-fidelity datasets with unrealistic, labeled attacks. This makes difficult the task of evaluating, comparing, and validating IDSs. Here we detail and benchmark four time-based IDSs against the newly published ROAD dataset, the first open CAN IDS dataset with real (non-simulated) stealthy attacks with physically verified effects. We found that methods that perform hypothesis testing by explicitly estimating message timing distributions have lower performance than methods that seek anomalies in a distribution related statistic. In particular, these “distribution-agnostic” based methods outperform “distribution-based” methods by at least 55% in area under the precision-recall curve (AUC-PR). Our results expand the body of knowledge of CAN time-based IDSs by providing details of these methods and reporting their results when tested on datasets with real advanced attacks. Finally, we develop an after-market plug-in detector using lightweight hardware, which can be used to deploy the best performing IDS method on nearly any vehicle.

Moriano, Pablo [Oak Ridge National Lab. (ORNL), Oa↗

Using the Bootstrap Method for a Statistical Significance Test of Differences between Summary Histograms

A new method is proposed to compare statistical differences between summary histograms, which are the histograms summed over a large ensemble of individual histograms. It consists of choosing a distance statistic for measuring the difference between summary histograms and using a bootstrap procedure to calculate the statistical significance level. Bootstrapping is an approach to statistical inference that makes few assumptions about the underlying probability distribution that describes the data. Three distance statistics are compared in this study. They are the Euclidean distance, the Jeffries-Matusita distance and the Kuiper distance. The data used in testing the bootstrap method are satellite measurements of cloud systems called cloud objects. Each cloud object is defined as a contiguous region/patch composed of individual footprints or fields of view. A histogram of measured values over footprints is generated for each parameter of each cloud object and then summary histograms are accumulated over all individual histograms in a given cloud-object size category. The results of statistical hypothesis tests using all three distances as test statistics are generally similar, indicating the validity of the proposed method. The Euclidean distance is determined to be most suitable after comparing the statistical tests of several parameters with distinct probability distributions among three cloud-object size categories. Impacts on the statistical significance levels resulting from differences in the total lengths of satellite footprint data between two size categories are also discussed.

Xu, Kuan-Man↗

Cross-Cutting Computational Modeling Project: Integrative Modeling Approach

A wide range of computational models and analyses have been applied to spaceflight risk assessment and countermeasure development. The benefits of using computational modeling to enhance Human Research Program (HRP) goals include the ability to mathematically represent physiological systems, integrate multiple, discrete experimental measures, span multiple temporal and spatial scales, determine important factors within the system and provide estimates of unmeasurable quantities. In the area of application, computational models provide a means of developing simulations to test hypotheses, determining key factors of the system to aid experimental design and bridging gaps in sparse data by mathematically simulating large populations. Specifically, computational models and their supporting analysis tools have the proven potential to integrate analyses of risk factors to enhance mission planning and preparation capabilities and to inform spacecraft design and countermeasure development. Appropriately applied, computational models may allow intelligent, unbiased physiological parameter assessment to enable hypothesis testing and model based design of experiments. HRP recently formed the Computational Modeling Project (CMP), managed out of Glenn Research Center, as a cross-cutting activity aimed at leveraging the growing power and acceptance of computational modeling in informing clinical, physiological, and biological studies. This presentation will provide an overview of the challenges and opportunities in implementing various forms of computational models in support of the HRPs path to risk reduction.

biomechanics↗

Preventing Failures By Dataset Shift Detection in Safety-Critical Graph Applications

Dataset shift refers to the problem where the input data distribution may change over time (e.g., between training and test stages). Since this can be a critical bottleneck in several safety-critical applications such as healthcare, drug-discovery, etc., dataset shift detection has become an important research issue in machine learning. Though several existing efforts have focused on image/video data, applications with graph-structured data have not received sufficient attention. Therefore, in this paper, we investigate the problem of detecting shifts in graph structured data through the lens of statistical hypothesis testing. Specifically, we propose a practical two-sample test based approach for shift detection in large-scale graph structured data. Our approach is very flexible in that it is suitable for both undirected and directed graphs, and eliminates the need for equal sample sizes. Using empirical studies, we demonstrate the effectiveness of the proposed test in detecting dataset shifts. We also corroborate these findings using real-world datasets, characterized by directed graphs and a large number of nodes.

97 MATHEMATICS AND COMPUTING↗

On the accuracy of Hipparcos using binary stars as a calibration tool

Stellar binary systems, specifically those that present the most accurate available orbital elements, are a reliable tool to test the accuracy of astrometric observations. We selected all 35 binaries with these characteristics. Our objective is to provide standard uncertainties for the positions and parallaxes measured by Hipparcos relative to this trustworthy set, as well as to check supposed correlations between several parameters (measurement residuals, positions, magnitudes, and parallaxes). In addition, using the high-confidence subset of visual–spectroscopic binaries, we implemented a validation test of the Hipparcos trigonometric parallaxes of binary systems that allowed the evaluation of their reliability. Standard and non-standard statistical analysis techniques were applied in order to achieve well-founded conclusions. In particular, errors-in-variables models such as the total least-squares method were used to validate Hipparcos parallaxes by comparison with those obtained directly from the orbital elements. Previously, we executed Thompson's τ technique in order to detect suspected outliers in the data. Furthermore, several statistical hypothesis tests were carried out to verify if our results were statistically significant. A statistically significant trend indicating larger Hipparcos angular separations with respect to the reference values in 5.2 ± 1.4 mas was found at the 10{sup −8} significance level. Uncertainties in the polar coordinates θ and ρ of 1.°8 and 6.3 mas, respectively, were estimated for the Hipparcos observations of binary systems. We also verified that the parallaxes of binary systems measured in this mission are absolutely compatible with the set of orbital parallaxes obtained from the most accurate orbits at least at the 95% confidence level. This methodology allows us to better estimate the accuracy of Hipparcos observations of binary systems. Indeed, further application to the data collected by Gaia should yield a standard procedure to compare both data sets.

79 ASTRONOMY AND ASTROPHYSICS↗

Statistical significance of the sterile-neutrino hypothesis in the context of reactor and gallium data

We evaluate the statistical significance of the 3+1 sterile-neutrino hypothesis using $v_e$ and $\bar{v}_e$ disappearance data from reactor, solar and gallium radioactive source experiments. Concerning the latter, we investigate the implications of the recent BEST results. For reactor data we focus on relative measurements independent of flux predictions. For the problem at hand, the usual χ 2 -approximation to hypothesis testing based on Wilks’ theorem has been shown in the literature to be inaccurate. We therefore present results based on Monte Carlo simulations, and find that this typically reduces the significance by roughly 1σ with respect to the naïve expectation. We find no significant indication in favor of sterile-neutrino oscillations from reactor data. On the other hand, gallium data (dominated by the BEST result) show more than 5σ of evidence supporting the sterile-neutrino hypothesis, favoring oscillation parameters in agreement with constraints from reactor data. This explanation is, however, in significant tension (~3 σ) with solar neutrino experiments. In order to assess the robustness of the signal for gallium experiments we present a discussion of the impact of cross-section uncertainties on the results.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Hybrid statistical testing for nuclear material accounting data and/or process monitoring data in nuclear safeguards

The aim of nuclear safeguards is to ensure that special nuclear material is used for peaceful purposes. Historically, nuclear material accounting (NMA) has provided the quantitative basis for monitoring for nuclear material loss or diversion, and process monitoring (PM) data is collected by the operator to monitor the process. PM data typically support NMA in various ways, often by providing a basis to estimate some of the in-process nuclear material inventory. We develop options for combining PM residuals and NMA residuals (residual = measurement - prediction), using a hybrid of period-driven and data-driven hypothesis testing. The modified statistical tests can be used on time series of NMA residuals (the NMA residual is the familiar material balance), or on a combination of PM and NMA residuals. The PM residuals can be generated on a fixed time schedule or as events occur.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Computational Imaging for Intelligence in Highly Scattering Aerosols (Final Report)

Natural and man-made degraded visual environments pose major threats to national security. The random scattering and absorption of light by tiny particles suspended in the air reduces situational awareness and causes unacceptable down-time for critical systems and operations. To improve the situation, we have developed several approaches to interpret the information contained within scattered light to enhance sensing and imaging in scattering media. These approaches were tested at the Sandia National Laboratory Fog Chamber facility and with tabletop fog chambers. Computationally efficient light transport models were developed and leveraged for computational sensing. The models are based on a weak angular dependence approximation to the Boltzmann or radiative transfer equation that appears to be applicable in both the moderate and highly scattering regimes. After the new model was experimentally validated, statistical approaches for detection, localization, and imaging of objects hidden in fog were developed and demonstrated. A binary hypothesis test and the Neyman-Pearson lemma provided the highest theoretically possible probability of detection for a specified false alarm rate and signal-to-noise ratio. Maximum likelihood estimation allowed estimation of the fog optical properties as well as the position, size, and reflection coefficient of an object in fog. A computational dehazing approach was implemented to reduce the effects of scatter on images, making object features more readily discernible. We have developed, characterized, and deployed a new Tabletop Fog Chamber capable of repeatably generating multiple unique fog-analogues for optical testing in degraded visual environments. We characterized this chamber using both optical and microphysical techniques. In doing so we have explored the ability of droplet nucleation theory to describe the aerosols generated within the chamber, as well as Mie scattering theory to describe the attenuation of light by said aerosols, and correlated the aerosol microphysics to optical properties such as transmission and meteorological optical range (MOR). This chamber has proved highly valuable and has supported multiple efforts inclusive to and exclusive of this LDRD project to test optics in degraded visual environments. Circularly polarized light has been found to maintain its polarization state better than linearly polarized light when propagating through fog. This was demonstrated experimentally in both the visible and short-wave infrared (SWIR) by imaging targets made of different commercially available retroreflective films. It was found that active circularly polarized imaging can increase contrast and range compared to linearly polarized imaging. We have completed an initial investigation of the capability for machine learning methods to reduce the effects of light scattering when imaging through fog. Previously acquired experimental long-wave images were used to train an autoencoder denoising architecture. Overfitting was found to be a problem because of lack of variability in the object type in this data set. The lessons learned were used to collect a well labeled dataset with much more variability using the Tabletop Fog Chamber that will be available for future studies. We have developed several new sensing methods using speckle intensity correlations. First, the ability to image moving objects in fog was shown, establishing that our unique speckle imaging method can be implemented in dynamic scattering media. Second, the speckle decorrelation over time was found to be sensitive to fog composition, implying extensions to fog characterization. Third, the ability to distinguish macroscopically identical objects on a far-subwavelength scale was demonstrated, suggesting numerous applications ranging from nanoscale defect detection to security. Fourth, we have shown the capability to simultaneously image and localize hidden objects, allowing the speckle imaging method to be effective without prior object positional information. Finally, an interferometric effect was presented that illustrates a new approach for analyzing speckle intensity correlations that may lead to more effective ways to localize and image moving objects. All of these results represent significant developments that challenge the limits of the application of speckle imaging and open important application spaces. A theory was developed and simulations were performed to assess the potential transverse resolution benefit of relative motion in structured illumination for radar systems. Results for a simplified radar system model indicate that significant resolution benefits are possible using data from scanning a structured beam over the target, with the use of appropriate signal processing.

58 GEOSCIENCES↗

Quickest Detection for Changes in Maximal kNN Coherence of Random Matrices

This paper addresses the problem of quickest detection of a change in the maximal coherence between columns of a n p random matrix based on a sequence of matrix observations having a single unknown change point. The random matrix is assumed to have identically distributed rows and the maximal coherence is defined as the largest of the ( 2 p) correlation coefficients associated with any row. Likewise the κ nearest neighbor (kNN) coherence is defined as the κ-th largest of these correlation coefficients. The forms of the preand postchange distributions of the observed matrices are assumed to belong to the family of elliptically contoured densities with sparse dispersion matrices but are otherwise unknown. A nonparametric stopping rule is proposed that is based on the maximal kNN sample coherence between columns of each observed random matrix. This is a summary statistic that is related to a test of existence of a hub vertex in a sample correlation graph having degree at least κ. Performance bounds on the delay and false alarm performance of the proposed stopping rule are obtained in the purely high-dimensional regime where p → ∞ and n is fixed. When the prechange dispersion matrix is diagonal it is shown that, among all functions of the proposed summary statistic, the proposed stopping rule is asymptotically optimal under a minimax quickest change detection model as the stopping threshold approaches infinity. Furthermore, the theory developed also applies to sequential hypothesis testing and fixed sample size tests.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

EGS Stimulation Design with Uncertainty Quantification at the EGS Collab Site

Engineering a robust hydraulic connection between wells is one of the most difficult aspects of enhanced geothermal systems (EGS). Designing and constructing such hydraulic connections requires and understanding of the in situ state of stress and the heterogeneities and discontinuities that naturally exist and may control the stimulation. Even with comprehensive stress and formation characterization programs substantial uncertainty remains in these key parameters. This is especially the case in high-temperature EGS environments where drilling conditions are often difficult and a far fewer logging and testing options are available. This paper presents a new approach for explicitly quantifying the uncertainties in the state of stress using a Bayesian Markov Chain Monte Carlo method. This approach produces a probability distribution for the stress tensor, including a general 3D orientation, that reflects the uncertainties in all the observations or indicators used to constrain the stress state. This method is demonstrated on the characterization data for the EGS Collab Experiment 2 site. The output of the analysis is used to guide the design of the planned stimulations. In the case of research projects like EGS Collab, explicitly quantifying the uncertainties in the stress state allow for more rigorous hypothesis testing by allowing conclusions drawn from the experiments to be interpreted in the context of the uncertain knowledge about conditions in the test bed.

Burghardt, Jeffrey A.↗

On the detection of molecules in the atmosphere of HD 189733b using HST NICMOS transmission spectroscopy

The Hubble Space Telescope/NICMOS transmission spectrum measurements of HD 189733b that suggest the detection of methane (CH{sub 4}) in an exoplanet atmosphere have been a source of significant controversy. With what is probably the best analyzed exoplanet spectroscopy data set to date, different teams, using different methods, have claimed evidence both contradicting and supporting the original findings. Here, we report results from a uniform spectral retrieval analysis of the three independent published spectra together with null hypothesis testing. Based on Bayesian model comparison, we find that two of the three spectra show strong evidence (≥3.6σ) for the detection of molecular features mainly due to water and methane while the third is consistent with a weak molecular detection at the 2.2σ level. We interpret the agreement in the spectral modulation established by previous authors and the atmospheric retrieval results presented here, as a confirmation of the original detection of molecular absorbers in the atmosphere of HD 189733b.

79 ASTRONOMY AND ASTROPHYSICS↗

Is the galactic bulge devoid of planets?

We consider a sample of 31 exoplanetary systems detected by gravitational microlensing and investigate whether or not the estimated distances to these systems conform to the Galactic distribution of planets expected from models. We derive the expected distribution of distances and relative proper motions from a simulated microlensing survey, correcting for the dominant selection effects that affect the sensitivity of planet detection as a function of distance, and compare it to the observed distribution using Anderson–Darling (AD) hypothesis testing. Taking the relative abundance of planets in the bulge to that in the disk, f{sub bulge}, as a model parameter, we find that our model is consistent with the observed distribution only for f{sub bulge}<0.54 (for a p-value threshold of 0.01) implying that the bulge may be devoid of planets relative to the disk. Allowing for a dependence of planet abundance on metallicity and host mass, or an additional dependence of planet sensitivity on event timescale, does not restore consistency for f{sub bulge}=1. We examine the distance estimates of some events in detail, and conclude that some parallax-based estimates could be significantly in error. Only by combining the removal of one problematic event from our sample and the inclusion of strong dependences of planet abundance or detection sensitivity on host mass, metallicity, and event timescale are we able to find consistency with the hypothesis that the bulge and disk have equal planet abundance.

79 ASTRONOMY AND ASTROPHYSICS↗

Structural characterization of particle systems using spherical harmonics

Many important properties of particulate materials are heavily influenced by the size and shape of the constituent particles. Thus, in order to control and improve product quality, it is important to develop a good understanding of the shape and size of the particles that make up a given particulate material. In this paper, we show how the spherical harmonics expansion can be used to approximate particles obtained from tomographic 3D images. This yields an analytic representation of the particles which can be used to calculate structural characteristics. We present an estimation method for the optimal length of expansion depending on individual particle shapes, based on statistical hypothesis testing. A suitable choice of this parameter leads to a smooth approximation that preserves the main shape features of the original particle. To show the wide applicability of this procedure, we use it to approximate particles obtained from two different tomographic 3D datasets of particulate materials. The first one describes an anode material from lithium-ion cells that consists of sphere-like particles with different sizes. The second dataset describes a powder of highly non-spherical titanium dioxide particles. - Highlights: • Complex particle shapes are described analytically by spherical harmonics expansion. • The optimal length of the expansion is estimated for each particle individually. • Characteristics like, e.g., particle surface areas can be calculated efficiently. • The method is applied to two tomographic datasets of particulate materials.

36 MATERIALS SCIENCE↗

Perioperative Gemcitabine + Erlotinib Plus Pancreaticoduodenectomy for Resectable Pancreatic Adenocarcinoma: ACOSOG Z5041 (Alliance) Phase II Trial

Background: There is considerable interest in a neoadjuvant approach for resectable pancreatic ductal adenocarcinoma (PDAC). This study evaluated perioperative gemcitabine + erlotinib (G+E) for resectable PDAC. Methods: A multicenter, cooperative group, single-arm, phase II trial was conducted between April 2009 and November 2013 (ACOSOG Z5041). Patients with biopsy-confirmed PDAC in the pancreatic head without evidence of involvement of major mesenteric vessels (resectable) were eligible. Patients (n = 123) received an 8-week cycle of G+E before and after surgery. The primary endpoint was 2-year overall survival (OS), and secondary endpoints included toxicity, response, resection rate, and time to progression. Resectability was assessed retrospectively by central review. The study closed early due to slow accrual, and no formal hypothesis testing was performed. Results: Overall, 114 patients were eligible, consented, and initiated protocol treatment. By central radiologic review, 97 (85%) of the 114 patients met the protocol-defined resectability criteria. Grade 3+ toxicity was reported in 60% and 79% of patients during the neoadjuvant phase and overall, respectively. Twenty-two of 114 (19%) patients did not proceed to surgery; 83 patients (73%) were successfully resected. R0 and R1 margins were obtained in 67 (81%) and 16 (19%) resected patients, respectively, and 54 patients completed postoperative G+E (65%). The 2-year OS rate for the entire cohort (n = 114) was 40% (95% confidence interval [CI] 31–50), with a median OS of 21.3 months (95% CI 17.2–25.9). The 2-year OS rate for resected patients (n = 83) was 52% (95% CI 41–63), with a median OS of 25.4 months (95% CI 21.8–29.6). Conclusions: For resectable PDAC, perioperative G+E is feasible. Further evaluation of neoadjuvant strategies in resectable PDAC is warranted with more active systemic regimens.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Multileaf Collimator Tracking Improves Dose Delivery for Prostate Cancer Radiation Therapy: Results of the First Clinical Trial

Purpose: To test the hypothesis that multileaf collimator (MLC) tracking improves the consistency between the planned and delivered dose compared with the dose without MLC tracking, in the setting of a prostate cancer volumetric modulated arc therapy trial. Methods and Materials: Multileaf collimator tracking was implemented for 15 patients in a prostate cancer radiation therapy trial; in total, 513 treatment fractions were delivered. During each treatment fraction, the prostate trajectory and treatment MLC positions were collected. These data were used as input for dose reconstruction (multiple isocenter shift method) to calculate the treated dose (with MLC tracking) and the dose that would have been delivered had MLC tracking not been applied (without MLC tracking). The percentage difference from planned for target and normal tissue dose-volume points were calculated. The hypothesis was tested for each dose-volume value via analysis of variance using the F test. Results: Of the 513 fractions delivered, 475 (93%) were suitable for analysis. The mean difference and standard deviation between the planned and treated MLC tracking doses and the planned and without-MLC tracking doses for all 475 fractions were, respectively, PTV D{sub 99%} −0.8% ± 1.1% versus −2.1% ± 2.7%; CTV D{sub 99%} −0.6% ± 0.8% versus −0.6% ± 1.1%; rectum V{sub 65%} 1.6% ± 7.9% versus −1.2% ± 18%; and bladder V{sub 65%} 0.5% ± 4.4% versus −0.0% ± 9.2% (P<.001 for all dose-volume results). Conclusion: This study shows that MLC tracking improves the consistency between the planned and delivered doses compared with the modeled doses without MLC tracking. The implications of this finding are potentially improved patient outcomes, as well as more reliable dose-volume data for radiobiological parameter determination.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Demonstrating Hierarchical System Development With the Common Community Physics Package Single‐Column Model: A Case Study Over the Southern Great Plains

This study demonstrates a specific application of the hierarchical system development (HSD) approach to investigate, analyze, and attribute model issues within the Unified Forecast System (UFS), with a focus on process isolation. By evaluating a non‐precipitating, shallow cumulus case at the Atmospheric Radiation Measurement Southern Great Plains site in the UFS global forecast against the observation, the investigation identifies a warmer and deeper daytime convective planetary boundary layer (PBL) and misrepresented nocturnal PBL transition. Hypothesis testing, which employs the Common Community Physics Package (CCPP) single‐column model (SCM) and uses the same physics as the UFS global model, confirms that these issues are attributed to the model physics and initialization. Specifically, misrepresented PBL processes are linked to problematic surface condition and a lack of cloud formation, which may stem from deficiencies in PBL and cloud microphysics parameterizations and their interactions. The UFS initial condition contributes to an earlier, excessively collapsed daytime convective boundary layer and a lack of decoupling between the stable boundary layer and residual layer late in the afternoon. This work introduces an avenue for the community to engage with the application of HSD, along with the CCPP and CCPP SCM, to understand the interplay of model physics, disentangle the roles of model components, as well as facilitate model and forecast improvement.

54 ENVIRONMENTAL SCIENCES↗

A statistical approach to combining multisource information in one-class classifiers

A new method is introduced in this paper for combining information from multiple sources to support one-class classification. The contributing sources may represent measurements taken by different sensors of the same physical entity, repeated measurements by a single sensor, or numerous features computed from a single measured image or signal. The approach utilizes the theory of statistical hypothesis testing, and applies Fisher's technique for combining p-values, modified to handle nonindependent sources. Classifier outputs take the form of fused p-values, which may be used to gauge the consistency of unknown entities with one or more class hypotheses. The approach enables rigorous assessment of classification uncertainties, and allows for traceability of classifier decisions back to the constituent sources, both of which are important for high-consequence decision support. Application of the technique is illustrated in two challenge problems, one for skin segmentation and the other for terrain labeling. Finally, the method is seen to be particularly effective for relatively small training samples.

97 MATHEMATICS AND COMPUTING↗