Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “false positive”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Final Report for CRADA Agreement , AL-C-2006-01 with Microsens Biotechnologies: Detection of the Abnormal Prion Protein in Blood by Improving the Extraction of this Protein

Several conditions were examined to optimize the extraction protocol using Seprion beads for the abnormal prion protein. Different combinations of water, hexafluro-2-propanol and formic acid were used. The results of these extraction protocols showed that the magnetic beads coated with Seprion reagents were subject to degradation, themselves, when the extraction conditions that would solubilize the abnormal prion protein were used. These compounds caused interference in the immunoassay for the abnormal prion protein and rendered these protocols incompatible with the assay systems. In an attempt to overcome this problem, another approach was then used. The coated beads were used as an integral part of the assay platform. After washing away denaturing agents, the beads with the 'captured' abnormal prion were incubated directly in the immunoassay, followed by analysis by the capillary electrophoresis. When a capillary electrophoresis electro-kinetic separation was attempted, the beads disturbed the analysis making it impossible to interpret. A pressure separation method was then developed for capillary electrophoresis analysis. When 20 samples, 5 of which were positive were analyzed, the assay identified 4 of the 5 positives and had no false positives. When a larger number of samples were analyzed the results were not as good - there were false positives and false negatives. It was then observed that the amount of beads that were loaded was dependent upon how long the beads were allowed to settle before loading them into the capillary. This resulted in unacceptable variations in the results and explained that when large numbers of samples were evaluated the results were not consistent. Because the technical difficulties with using the Seprion beads could not be overcome at this time, another approach is underway that is outside of the scope of this CRADA. No further agreements have been developed. Because the results were not favorable, no manuscripts were written nor intellectual property developed.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

A framework to measure myocardial extracellular volume fraction using dual-phase low dose CT images

Purpose: Myocardial extracellular volume fraction (ECVF) is a surrogate imaging biomarker of diffuse myocardial fibrosis, a hallmark of pathologic ventricular remodeling. Low dose cardiac CT is emerging as a promising modality to detect diffuse interstitial myocardial fibrosis due to its fast acquisition and low radiation; however, the insufficient contrast in the low dose CT images poses great challenge to measure ECVF from the image. Methods: To deal with this difficulty, the authors present a complete ECVF measurement framework including a point-guided myocardial modeling, a deformable model-based myocardium segmentation, nonrigid registration of pre- and post-CT, and ECVF calculation. Results: The proposed method was evaluated on 20 patients by two observers. Compared to the manually delineated reference segmentations, the accuracy of our segmentation in terms of true positive volume fraction (TPVF), false positive volume fraction (FPVF), and average surface distance (ASD), were 92.18% ± 3.52%, 0.31% ± 0.10%, 0.69 ± 0.14 mm, respectively. The interobserver variability measured by concordance correlation coefficient regarding TPVF, FPVF, and ASD were 0.95, 0.90, 0.94, respectively, demonstrating excellent agreement. Bland-Altman method showed 95% limits of agreement between ECVF at CT and ECVF at MR. Conclusions: The proposed framework demonstrates its efficiency, accuracy, and noninvasiveness in ECVF measurement and dramatically advances the ECVF at cardiac CT toward its clinical use.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Multi-Stage System for Automatic Target Recognition

A multi-stage automated target recognition (ATR) system has been designed to perform computer vision tasks with adequate proficiency in mimicking human vision. The system is able to detect, identify, and track targets of interest. Potential regions of interest (ROIs) are first identified by the detection stage using an Optimum Trade-off Maximum Average Correlation Height (OT-MACH) filter combined with a wavelet transform. False positives are then eliminated by the verification stage using feature extraction methods in conjunction with neural networks. Feature extraction transforms the ROIs using filtering and binning algorithms to create feature vectors. A feedforward back-propagation neural network (NN) is then trained to classify each feature vector and to remove false positives. The system parameter optimizations process has been developed to adapt to various targets and datasets. The objective was to design an efficient computer vision system that can learn to detect multiple targets in large images with unknown backgrounds. Because the target size is small relative to the image size in this problem, there are many regions of the image that could potentially contain the target. A cursory analysis of every region can be computationally efficient, but may yield too many false positives. On the other hand, a detailed analysis of every region can yield better results, but may be computationally inefficient. The multi-stage ATR system was designed to achieve an optimal balance between accuracy and computational efficiency by incorporating both models. The detection stage first identifies potential ROIs where the target may be present by performing a fast Fourier domain OT-MACH filter-based correlation. Because threshold for this stage is chosen with the goal of detecting all true positives, a number of false positives are also detected as ROIs. The verification stage then transforms the regions of interest into feature space, and eliminates false positives using an artificial neural network classifier. The multi-stage system allows tuning the detection sensitivity and the identification specificity individually in each stage. It is easier to achieve optimized ATR operation based on its specific goal. The test results show that the system was successful in substantially reducing the false positive rate when tested on a sonar and video image datasets.

Chao, Tien-Hsin↗

QUASI-STELLAR OBJECT SELECTION ALGORITHM USING TIME VARIABILITY AND MACHINE LEARNING: SELECTION OF 1620 QUASI-STELLAR OBJECT CANDIDATES FROM MACHO LARGE MAGELLANIC CLOUD DATABASE

We present a new quasi-stellar object (QSO) selection algorithm using a Support Vector Machine, a supervised classification method, on a set of extracted time series features including period, amplitude, color, and autocorrelation value. We train a model that separates QSOs from variable stars, non-variable stars, and microlensing events using 58 known QSOs, 1629 variable stars, and 4288 non-variables in the MAssive Compact Halo Object (MACHO) database as a training set. To estimate the efficiency and the accuracy of the model, we perform a cross-validation test using the training set. The test shows that the model correctly identifies {approx}80% of known QSOs with a 25% false-positive rate. The majority of the false positives are Be stars. We applied the trained model to the MACHO Large Magellanic Cloud (LMC) data set, which consists of 40 million light curves, and found 1620 QSO candidates. During the selection none of the 33,242 known MACHO variables were misclassified as QSO candidates. In order to estimate the true false-positive rate, we crossmatched the candidates with astronomical catalogs including the Spitzer Surveying the Agents of a Galaxy's Evolution LMC catalog and a few X-ray catalogs. The results further suggest that the majority of the candidates, more than 70%, are QSOs.

79 ASTRONOMY AND ASTROPHYSICS↗

BioID/LID System Cartridge Testing Report

LLNL produced two lots of lyophilized LAMP assay containing cartridges, LNL022818 and LNL071818, for testing on Gen3 BioID systems. Thirty-nine cartridges were tested for lot LNL022818. All thirty-nine cartridges performed as expected regarding correct positive control (IPC) and negative control (NTC) reactions. Five different templates were tested on these 39 cartridges. The BioID system detected 79.1% true positives and 9.3% false positives along with 98.9% true negatives and 2.9% false negatives for cartridge lot LNL022818. Thirty cartridges were tested for lot LNL071818. It was noted early that due to a manufacturing issue with the cartridges related to incomplete sonic welding of the cartridge, a large fraction of lot LNL071818 cartridges leaked. Insufficient funds were available to replace this lot of cartridges. Twenty three of 30 cartridges performed as expected regarding correct positive control (IPC) and negative control (NTC) reactions. Four different templates were tested on these 30 cartridges. The BioID system detected 77% true positives and 23% false positives along with 96.9% true negatives and 3.1% false negatives for cartridge lot LNL071818. For all tested cartridges using DNA template, DNA was added at a final concentration of 25 pg/ul. All reactions contained positive control DNA at 2.5 pg/ul.

42 ENGINEERING↗

Discovery and Vetting of Exoplanets. I. Benchmarking K2 Vetting Tools

We have adapted the algorithmic tools developed during the Kepler mission to vet the quality of transit-like signals for use on the K2 mission data. Using the four sets of publicly available light curves at MAST, we produced a uniformly vetted catalog of 772 transiting planet candidates from K2 as listed at the NASA Exoplanet Archive in the K2 Table of Candidates. Our analysis marks 676 of these as planet candidates and 96 as false positives. All confirmed planets pass our vetting tests. Sixty of our false positives are new identifications, effectively doubling the overall number of astrophysical signals mimicking planetary transits in K2 data. Most of the targets listed as false positives in our catalog show either prominent secondary eclipses, transit depths suggesting a stellar companion instead of a planet, or significant photocenter shifts during transit. We packaged our tools into the open-source, automated vetting pipeline Discovery and Vetting of Exoplanets (DAVE), designed to streamline follow-up efforts by reducing the time and resources wasted observing targets that are likely false positives. DAVE will also be a valuable tool for analyzing planet candidates from NASA's TESS mission, where several guest-investigator programs will provide independent light-curve sets—and likely many more from the community. We are currently testing DAVE on recently released TESS planet candidates and will present our results in a follow-up paper.

Kostov, Veselin B.↗

Fault Management Algorithm Risk Assessment for the NASA Space Launch System

This paper presents the false positive (FP) and false negative (FN) risk assessment process currently being conducted for the Space Launch System (SLS) Artemis II Fault Management (FM) detection functions. The analysis scope, general assumptions and guide rules, and key modeling concepts were discussed to establish the basis of the risk assessments conducted. Initial analyses indicated a dominance in the total risk by software and firmware failures. This paper presents efforts applied to refine the software risks and the overall impact of implementing those modifications. Current analyses conducted on the detection functions implemented for the SLS Artemis II mission indicate primary risk drivers for the individual FM detection functions are flight software failures, firmware design failures, and hardware Common Cause Failures (CCFs). There still remains issues of how to account for time and redundancy in the software risk estimations.

probability risk analysis↗

Fault Management Algorithm Risk Assessment for the NASA Space Launch System

This presentation describes the false positive (FP) and false negative (FN) risk assessment process currently being conducted for the Space Launch System (SLS) Artemis II Fault Management (FM) detection functions. The analysis scope, general assumptions and guide rules, and key modeling concepts were discussed to establish the basis of the risk assessments conducted. Initial analyses indicated a dominance in the total risk by software and firmware failures. This paper presents efforts applied to refine the software risks and the overall impact of implementing those modifications. Current analyses conducted on the detection functions implemented for the SLS Artemis II mission indicate primary risk drivers for the individual FM detection functions are flight software failures, firmware design failures, and hardware Common Cause Failures (CCFs). There still remains issues of how to account for time and redundancy in the software risk estimations.

probability risk analysis↗

Dual-time-point FDG PET/CT imaging in prosthetic heart valve endocarditis

Purpose: FDG PET/CT has been of increasing interest in the diagnostic workup of prosthetic heart valve endocarditis (PVE). Some reports advocate later imaging time points to improve the diagnostic accuracy for PVE. In this study, we compared standard and late FDG PET/CT images in patients with a clinical suspicion of PVE. Materials and Methods: Fourteen scans in 13 patients referred for FDG PET/CT for suspicion of PVE performed at standard (60 min post injection) and late (150 min post injection) time points were scored based on visual interpretation and semi-quantitatively with SUVmax and target-to-background ratio (TBR, defined as [SUVmax valve/SUVmean blood pool]). Final diagnosis was based on surgical findings in all cases of infection (n = 6) and unremarkable follow-up in all others (n = 8). Results: Late images were more prone to false positive interpretation for both visual and semi-quantitative analyses. Visual analysis of the standard images yielded 1 false negative and 1 false positive result. On the late images, no scans were false negative but 5 scans were false positive. Conclusion: Late FDG PET/CT imaging for PVE seems prone to false positive results. Therefore, late imaging should be interpreted with caution.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

ALMOST ALL OF KEPLER'S MULTIPLE-PLANET CANDIDATES ARE PLANETS

We present a statistical analysis that demonstrates that the overwhelming majority of Kepler candidate multiple transiting systems (multis) indeed represent true, physically associated transiting planets. Binary stars provide the primary source of false positives among Kepler planet candidates, implying that false positives should be nearly randomly distributed among Kepler targets. In contrast, true transiting planets would appear clustered around a smaller number of Kepler targets if detectable planets tend to come in systems and/or if the orbital planes of planets encircling the same star are correlated. There are more than one hundred times as many Kepler planet candidates in multi-candidate systems as would be predicted from a random distribution of candidates, implying that the vast majority are true planets. Most of these multis are multiple-planet systems orbiting the Kepler target star, but there are likely cases where (1) the planetary system orbits a fainter star, and the planets are thus significantly larger than has been estimated, or (2) the planets orbit different stars within a binary/multiple star system. We use the low overall false-positive rate among Kepler multis, together with analysis of Kepler spacecraft and ground-based data, to validate the closely packed Kepler-33 planetary system, which orbits a star that has evolved somewhat off of the main sequence. Kepler-33 hosts five transiting planets, with periods ranging from 5.67 to 41 days.

79 ASTRONOMY AND ASTROPHYSICS↗

Limits of special material detectability fundamental to idealized dual-energy radiographic systems

Analysis of dual-energy radiographic approaches to the detection of “special” material, i.e. contraband, in otherwise innocuous scenes, such as maritime cargo containers encountered at ports of entry, reveals fundamental limitations of data obtained along a single line-of-sight. The basic equations that describe idealized systems are presented and analyzed so as to identify challenges that are only exacerbated by scatter, noise, and measurement uncertainty. In the mono-energetic limit, the instrumentation and measurement methods reduce to a system of linear equations that are simple to treat: it is observed that there exists a threshold for the minimum amount of high-Z material that can be detected unambiguously (without the risk of false positive or false negative indications), even when the background materials are of finite and of well-known variety. The threshold depends on the quality of prior information about the cargo and the selection of probing energies, and can therefore be minimized by good system design practices and measurement procedures, but persists as a general challenge to security operations. In discussion of this result, it is noted that techniques incorporating other prior information, such as the geometry of cargo and potential contraband, along with data from 2D imaging, have far greater potential. For instance, a simple data analysis method capable of differentiating high-Z spheres from those of more common material is presented. This simple example demonstrates that, although simple line-of-sight transmission radiography may suffer fundamental limitations, there remains good cause to be optimistic about the potential for image analysis in radiographic material identification.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Inferring precocial Chinook Salmon production through single‐parentage assignments

Abstract Objective Parentage analysis is a routine methodology in fisheries research, but study systems exist where it is impractical to sample both parents. The ability to reliably assign offspring to a single parent is beneficial in these situations. We applied single‐parentage assignments to a naturally spawning population of Chinook Salmon Oncorhynchus tshawytscha to quantify production of anadromous returns by unsampled precocial males. Methods We used an approach that focused on two important aspects of parentage analyses: (1) addressing the presence of family structure within the set of sampled parents and (2) controlling for false‐positive and false‐negative assignments. Result Results indicated that 30% of reproductively successful males were precocial males, which produced 20% of the returning anadromous offspring. Conclusion This study provides a framework for applying single‐parent assignments in a salmonid study system while explicitly addressing sources of assignment errors.

Steele, Craig A.↗

Cross-Combined Composite Sampling Designs for Identification of Elevated Regions

Analyzing soils for contaminants can be costly. Generally, discrete samples are gathered from within a study area, analyzed by a laboratory and the results are used in a site-specific statistical analysis. Because of the heterogeneities that exist in soil samples within study areas, a large amount of variability and skewness may be present in the sample population. This necessitates collecting a large number of samples to obtain reliable inference on the mean contaminant concentration and to understand the spatial patterns for future remediation. Composite or Incremental sampling is a commonly applied method for gathering multiple discrete samples and physically combining them, such that each combination of discrete samples requires a single laboratory analysis, which reduces cost and can improve the estimates of the mean concentration. While incremental sampling can reduce cost and improve mean estimates, current implementations do not readily facilitate the characterization of spatial patterns or the detection of elevated regions within study areas. The methods we present in this work provide efficient estimation and inference for the mean contaminant concentration over the entire spatial area and enable the identification of high contaminant regions within the area of interest. We develop sample design methodologies that explicitly define the characteristics of these designs (such as sample grid layout) and quantify the number of incremental samples that must be obtained under a design criteria to control false positive and false negative (Type I and II) decision errors. We present the sample design theory and specifications as well as results on simulated and real data.

Hathaway, John E↗

Development and testing of a performance evaluation methodology to assess the reliability of occupancy sensor systems in residential buildings

With the emergence of advanced occupancy sensor technologies to better detect occupancy in buildings, a universal methodology and metrics are required to evaluate and report sensor systems’ reliability and compare the performance across multiple sensor systems. Herein this research presents a methodology to assess the reliability of occupancy sensor systems in residential buildings in a controlled laboratory environment, including both “typical” and “failure” testing scenarios. The developed methodology was then implemented to evaluate a novel occupancy detection sensor system’s reliability. “Typical” testing evaluates the overall accuracy of the sensor system, which suggest how reliable the occupancy sensor system is over time. Results show that on average, the precision and recall are 0.75 and 0.70, indicating similar numbers of false positives and false negatives across the dataset. The overall accuracy of the tested sensor system was 62.4% to 76.4%. Failure testing results indicate whether there are influential variables impacting the sensor performance. For the tested sensor system, the number of occupants, presence of large objects, presence of interior light sources, and number of doors are not influential, while lighting level, location of occupants, additional door in the entry/exit area, and having the TV on are variables determined to impact the sensor system performance.

47 OTHER INSTRUMENTATION↗

Planning, design and logistics of a decision analysis study: The FBI/Ames study involving forensic firearms examiners

This paper describes design and logistical aspects of a decision analysis study to assess the performance of qualified firearms examiners working in accredited laboratories in the United States in terms of accuracy (error rate), repeatability, and reproducibility of decisions involving comparisons of fired bullets and cartridge cases. The purpose of the study was to validate current practice of the forensic discipline of firearms/toolmarks (F/T) examination. It elicited error rate data by counting the number of false positive and false negative conclusions. Preceded by the experimental design, decisions, and logistics described herein, testing was ultimately administered 173 qualified, practicing F/T examiners in public and private crime laboratories. The first round of testing evaluated accuracy, while two subsequent rounds evaluated repeatability and reproducibility of examiner conclusions. This project expands on previous studies by involving many F/T examiners in challenging comparisons and by executing the study in the recommended double-blind format.

99 GENERAL AND MISCELLANEOUS↗

Using distributed temperature sensing to detect CO 2 leakage along the injection well casing

The objective of this study is to evaluate the sensitivity of Distributed Temperature Sensing (DTS) data to detect CO 2 leakage along an injection well casing. This paper describes the relationship between the CO 2 leakage rate and temperature response at DTS locations, and the method and numerical model used for understanding such a relationship. The uncertainties in the parameters are propagated to the interpretation of DTS measurements, which may lead to false positive and false negative identifications of CO 2 leakage. We propose to identify CO 2 leakage by analyzing both temperature history plots at selected vertical locations as well as the vertical temperature profiles at different times. The analysis should be combined with numerical simulations for the estimation of leakage rates. In addition, leakage needs to be confirmed by data from a warm-up test (temperature recovery after injection of cold CO 2 is stopped) to minimize the probability of false identifications.

42 ENGINEERING↗

A likelihood ratio anomaly detector for identifying within-perimeter computer network attacks

The rapid detection of attackers within firewalls of enterprise computer networks is of paramount importance. Anomaly detectors address this problem by quantifying deviations from baseline statistical models of normal network behavior and signaling an intrusion when the observed data deviates significantly from the baseline model. But, many anomaly detectors do not take into account plausible attacker behavior. As a result, anomaly detectors are prone to a large number of false positives due to unusual but benign activity. Our paper first introduces a stochastic model of attacker behavior which is motivated by real world attacker traversal. Then, we develop a likelihood ratio detector that compares the probability of observed network behavior under normal conditions against the case when an attacker has possibly compromised a subset of hosts within the network. Since the likelihood ratio detector requires integrating over the time each host becomes compromised, we illustrate how to use Monte Carlo methods to compute the requisite integral. We then present Receiver Operating Characteristic (ROC) curves for various network parameterizations that show for any rate of true positives, the rate of false positives for the likelihood ratio detector is no higher than that of a simple anomaly detector and is often lower. Finally, we demonstrate the superiority of the proposed likelihood ratio detector when the network topologies and parameterizations are extracted from real-world networks.

97 MATHEMATICS AND COMPUTING↗

Stabilization of light-induced effects in Si modules for IEC 61215 design qualification

Standardized testing of commercial photovoltaic modules is widely used around the world and decreases risks of module failures. Such testing also reduces financial risks for module manufacturers and customers. This work examines the expected impact of certain defects in silicon (Si) modules during standardized accelerated testing. Specifically, the behavior of boron-oxygen (BO) light-induced degradation (LID) and light- and elevated temperature-induced degradation (LeTID) are simulated during some of the stress tests in IEC 61215. LID and LeTID reaction rates at qualification temperatures are estimated from earlier published data. It is demonstrated the BO-related LID may cause some false positives and false negatives when IEC 61215 tests are performed as prescribed in the 2016 published version. Possible stabilization steps to avoid these false results are suggested. State changes for the defects causing LeTID occur much more slowly than those causing BO LID, and therefore LeTID is predicted to have a lesser impact on the IEC 61215 stress tests results.

14 SOLAR ENERGY↗