Resequencing: The Untold Story - Recognizing False Positives, False Negatives and Structural Variation in User Data
Explore the source record for details and available documents.
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Explore the source record for details and available documents.
Robustness against attacks serves as evidence for complex network structures and failure mechanisms that lie behind them. Most often, due to detection capability limitation or good disguises, attacks on networks are subject to false positives and false negatives, meaning that functional nodes may be falsely regarded as compromised by the attacker and vice versa. In this work, we initiate a study of false positive/negative effects on network robustness against three fundamental types of attack strategies, namely, random attacks (RA), localized attacks (LA), and targeted attack (TA). By developing a general mathematical framework based upon the percolation model, we investigate analytically and by numerical simulations of attack robustness with false positive/negative rate (FPR/FNR) on three benchmark models including Erdős-Rényi (ER) networks, random regular (RR) networks, and scale-free (SF) networks. We show that ER networks are equivalently robust against RA and LA only when FPR equals zero or the initial network is intact. We find several interesting crossovers in RR and SF networks when FPR is taken into consideration. By defining the cost of attack, we observe diminishing marginal attack efficiency for RA, LA, and TA. Our finding highlights the potential risk of underestimating or ignoring FPR in understanding attack robustness. The results may provide insights into ways of enhancing robustness of network architecture and improve the level of protection of critical infrastructures.
Explore the source record for details and available documents.
Purpose: In the past decades, computer-aided detection (CADe) systems have been developed to aid screening radiologists in the detection of malignant microcalcifications. These systems are useful to avoid perceptual oversights and can increase the radiologists’ detection rate. However, due to the high number of false positives marked by these CADe systems, they are not yet suitable as an independent reader. Breast arterial calcifications (BACs) are one of the most frequent false positives marked by CADe systems. In this study, a method is proposed for the elimination of BACs as positive findings. Removal of these false positives will increase the performance of the CADe system in finding malignant microcalcifications. Methods: A multistage method is proposed for the removal of BAC findings. The first stage consists of a microcalcification candidate selection, segmentation and grouping of the microcalcifications, and classification to remove obvious false positives. In the second stage, a case-based selection is applied where cases are selected which contain BACs. In the final stage, BACs are removed from the selected cases. The BACs removal stage consists of a GentleBoost classifier trained on microcalcification features describing their shape, topology, and texture. Additionally, novel features are introduced to discriminate BACs from other positive findings. Results: The CADe system was evaluated with and without BACs removal. Here, both systems were applied on a validation set containing 1088 cases of which 95 cases contained malignant microcalcifications. After bootstrapping, free-response receiver operating characteristics and receiver operating characteristics analyses were carried out. Performance between the two systems was compared at 0.98 and 0.95 specificity. At a specificity of 0.98, the sensitivity increased from 37% to 52% and the sensitivity increased from 62% up to 76% at a specificity of 0.95. Partial areas under the curve in the specificity range of 0.8–1.0 were significantly different between the system without BACs removal and the system with BACs removal, 0.129 ± 0.009 versus 0.144 ± 0.008 (p<0.05), respectively. Additionally, the sensitivity at one false positive per 50 cases and one false positive per 25 cases increased as well, 37% versus 51% (p<0.05) and 58% versus 67% (p<0.05) sensitivity, respectively. Additionally, the CADe system with BACs removal reduces the number of false positives per case by 29% on average. The same sensitivity at one false positive per 50 cases in the CADe system without BACs removal can be achieved at one false positive per 80 cases in the CADe system with BACs removal. Conclusions: By using dedicated algorithms to detect and remove breast arterial calcifications, the performance of CADe systems can be improved, in particular, at false positive rates representative for operating points used in screening.
Readers need to be informed about potential pitfalls of [ 68 Ga]Ga-PSMA-11 PET interpretation. Here we report [ 68 Ga]Ga-PSMA-11 PET findings discordant with the histopathology/composite reference standard in a recently published prospective trial on 635 patients with biochemically recurrent prostate cancer. Consensus reads were false positive in 20 regions of 17/217 (8%) patients with lesion validation. Majority of the false positive interpretations (13 of 20, 65%) occurred in the context of suspected prostate (bed) relapse (T) after radiotherapy ( n = 11); other false positive findings were noted for prostate bed post prostatectomy (T, n = 2), pelvic nodes (N, n = 2), or extra pelvic lesions (M, n = 5). Major sources of false positive findings were PSMA-expressing residual adenocarcinoma with marked post-radiotherapy treatment effect. False negative interpretation occurred in 8 regions of 6/79 (8%) patients with histopathology validation, including prostate (bed) ( n = 5), pelvic nodes ( n = 1), and extra pelvic lesions ( n = 2). Lesions were missed mostly due to small metastases or adjacent bladder/urine uptake.[ 68 Ga]Ga-PSMA-11 PET at biochemical recurrence resulted in less than 10% false positive interpretations. Post-radiotherapy prostate uptake was a major source of [ 68 Ga]Ga-PSMA-11 PET false positivity. In few cases, PET correctly detects residual PSMA expression post-radiotherapy, originating however from treated, benign tissue or potentially indolent tumor remnants.
The Kepler mission is uniquely suited to study the frequencies of extrasolar planets. This goal requires knowledge of the incidence of false positives such as eclipsing binaries in the background of the targets, or physically bound to them, which can mimic the photometric signal of a transiting planet. We perform numerical simulations of the Kepler targets and of physical companions or stars in the background to predict the occurrence of astrophysical false positives detectable by the mission. Using real noise level estimates, we compute the number and characteristics of detectable eclipsing pairs involving main-sequence stars and non-main-sequence stars or planets, and we quantify the fraction of those that would pass the Kepler candidate vetting procedure. By comparing their distribution with that of the Kepler Objects of Interest (KOIs) detected during the first six quarters of operation of the spacecraft, we infer the false positive rate of Kepler and study its dependence on spectral type, candidate planet size, and orbital period. We find that the global false positive rate of Kepler is 9.4%, peaking for giant planets (6-22 R{sub Circled-Plus }) at 17.7%, reaching a low of 6.7% for small Neptunes (2-4 R{sub Circled-Plus }), and increasing again for Earth-size planets (0.8-1.25 R{sub Circled-Plus }) to 12.3%. Most importantly, we also quantify and characterize the distribution and rate of occurrence of planets down to Earth size with no prior assumptions on their frequency, by subtracting from the population of actual Kepler candidates our simulated population of astrophysical false positives. We find that 16.5% {+-} 3.6% of main-sequence FGK stars have at least one planet between 0.8 and 1.25 R{sub Circled-Plus} with orbital periods up to 85 days. This result is a significant step toward the determination of eta-earth, the occurrence of Earth-like planets in the habitable zone of their parent stars. There is no significant dependence of the rates of planet occurrence between 0.8 and 4 Earth radii with spectral type. In the process, we also derive a prescription for the signal recovery rate of Kepler that enables a good match to both the KOI size and orbital period distribution, as well as their signal-to-noise distribution.
Searches for planetary transits find many astrophysical false positives as a by-product. There are four main types analyzed in the literature: a grazing-incidence eclipsing binary (EB) star, an EB star with a small radius companion star, a blend of one or more stars with an unrelated EB star, and a physical triple star system. We present a list of 69 astrophysical false positives that had been identified as candidates of transiting planets of the on-going XO survey. This list may be useful in order to avoid redundant observation and characterization of these particular candidates that have been independently identified by other wide-field searches for transiting planets. The list may be useful for those modeling the yield of the XO survey and surveys similar to it. Subsequent observations of some of the listed stars may improve mass-radius relations, especially for low-mass stars. From the candidates exhibiting eclipses, we report three new spectroscopic double-line binaries and give mass function estimations for 15 single-line spectroscopic binaries.
Objective: The aim of this study was to quantify false-positive and incidental findings from annual surveillance imaging in asymptomatic, American Joint Committee on Cancer stage III melanoma patients. Methods: This was a cohort study of patients treated at Melanoma Institute Australia (2000–2015) with baseline computed tomography (CT) or positron emission tomography (PET)/CT imaging and at least two annual surveillance scans. False-positives were defined as findings suspicious for melanoma recurrence that were not melanoma, confirmed by histopathology, subsequent imaging, or clinical follow-up, while incidental findings were defined as non-melanoma-related findings requiring further action. Outcomes of incidental findings were classified as ‘benign’ if they resolved spontaneously or were not seriously harmful; ‘malignant’ if a second malignancy was identified; or ‘other’ if potentially harmful. Results: Among 154 patients, 1022 scans were performed (154 baseline staging, 868 surveillance) during a median follow-up of 85 months (interquartile range 56–112); 57 patients (37%) developed a recurrence. For baseline and surveillance imaging, 124 false-positive results and incidental findings were identified in 81 patients (53%). The frequency of these findings was 5–14% per year, and an additional 181 tests, procedures, and referrals were initiated to investigate these findings. The diagnosis was benign in 109 findings of 124 findings (88%). Fifteen patients with a benign finding underwent an unnecessary invasive procedure. Surveillance imaging identified distant metastases in 20 patients (13%). Conclusion: False-positive results and incidental findings occur in at least half of all patients undergoing annual surveillance imaging, and the additional healthcare use is substantial. These findings persist over time. Clinicians need to be aware of these risks and discuss them with patients, alongside the expected benefits of surveillance imaging.
Light curves from the Kepler Mission contain valuable information on the nature of the phenomena producing the transit-like signals. To assist in exploring the possibility that they are due to an astrophysical false positive, we describe a procedure (BLENDER) to model the photometry in terms of a 'blend' rather than a planet orbiting a star. A blend may consist of a background or foreground eclipsing binary (or star-planet pair) whose eclipses are attenuated by the light of the candidate and possibly other stars within the photometric aperture. We apply BLENDER to the case of Kepler-9 (KIC 3323887), a target harboring two previously confirmed Saturn-size planets (Kepler-9 b and Kepler-9 c) showing transit timing variations, and an additional shallower signal with a 1.59 day period suggesting the presence of a super-Earth-size planet. Using BLENDER together with constraints from other follow-up observations we are able to rule out all blends for the two deeper signals and provide independent validation of their planetary nature. For the shallower signal, we rule out a large fraction of the false positives that might mimic the transits. The false alarm rate for remaining blends depends in part (and inversely) on the unknown frequency of small-size planets. Based on several realistic estimates of this frequency, we conclude with very high confidence that this small signal is due to a super-Earth-size planet (Kepler-9 d) in a multiple system, rather than a false positive. The radius is determined to be 1.64{sup +0.19}{sub -0.14} R{sub +}, and current spectroscopic observations are as yet insufficient to establish its mass.
It has been hypothesized that the presence of closely orbiting giant planets is associated with enhanced chromospheric emission of their host stars. The main cause for such a relation would likely be enhanced dynamo action induced by the planet. We present measurements of chromospheric emission in 234 planet candidate systems from the Kepler mission. This ensemble includes 37 systems with giant-planet candidates, which show a clear emission enhancement. The enhancement, however, disappears when systems that are also identified as eclipsing binary candidates are removed from the ensemble. This suggests that a large fraction of the giant-planet candidate systems with chromospheric emission stronger than the Sun are not giant-planet systems, but false positives. Such false-positive systems could be tidally interacting binaries with strong chromospheric emission. This hypothesis is supported by an analysis of 188 eclipsing binary candidates that show increasing chromospheric emission as function of decreasing orbital period.
NASA’s Kepler mission has provided several thousand transiting planet candidates during the 4 yr of its nominal mission, yet only a small subset of these candidates have been confirmed as true planets. Therefore, the most fundamental question about these candidates is the fraction of bona fide planets. Estimating the rate of false positives of the overall Kepler sample is necessary to derive the planet occurrence rate. We present the results from two large observational campaigns that were conducted with the Spitzer Space Telescope during the the Kepler mission. These observations are dedicated to estimating the false positive rate (FPR) among the Kepler candidates. We select a sub-sample of 51 candidates, spanning wide ranges in stellar, orbital, and planetary parameter space, and we observe their transits with Spitzer at 4.5 μm. We use these observations to measures the candidate’s transit depths and infrared magnitudes. An authentic planet produces an achromatic transit depth (neglecting the modest effect of limb darkening). Conversely a bandpass-dependent depth alerts us to the potential presence of a blending star that could be the source of the observed eclipse: a false positive scenario. For most of the candidates (85%), the transit depths measured with Kepler are consistent with the transit depths measured with Spitzer as expected for planetary objects, while we find that the most discrepant measurements are due to the presence of unresolved stars that dilute the photometry. The Spitzer constraints on their own yield FPRs between 5% and depending on the Kepler Objects of Interest. By considering the population of the Kepler field stars, and by combining follow-up observations (imaging) when available, we find that the overall FPR of our sample is low. The measured upper limit on the FPR of our sample is 8.8% at a confidence level of 3σ. This observational result, which uses the achromatic property of planetary transit signals that is not investigated by the Kepler observations, provides an independent indication that Kepler’s FPR is low.
In this work, gray-scale invariant ranklet texture features are proposed for false positive reduction (FPR) in computer-aided detection (CAD) of breast masses. Two main considerations are at the basis of this proposal. First, false positive (FP) marks surviving our previous CAD system seem to be characterized by specific texture properties that can be used to discriminate them from masses. Second, our previous CAD system achieves invariance to linear/nonlinear monotonic gray-scale transformations by encoding regions of interest into ranklet images through the ranklet transform, an image transformation similar to the wavelet transform, yet dealing with pixels' ranks rather than with their gray-scale values. Therefore, the new FPR approach proposed herein defines a set of texture features which are calculated directly from the ranklet images corresponding to the regions of interest surviving our previous CAD system, hence, ranklet texture features; then, a support vector machine (SVM) classifier is used for discrimination. As a result of this approach, texture-based information is used to discriminate FP marks surviving our previous CAD system; at the same time, invariance to linear/nonlinear monotonic gray-scale transformations of the new CAD system is guaranteed, as ranklet texture features are calculated from ranklet images that have this property themselves by construction. To emphasize the gray-scale invariance of both the previous and new CAD systems, training and testing are carried out without any in-between parameters' adjustment on mammograms having different gray-scale dynamics; in particular, training is carried out on analog digitized mammograms taken from a publicly available digital database, whereas testing is performed on full-field digital mammograms taken from an in-house database. Free-response receiver operating characteristic (FROC) curve analysis of the two CAD systems demonstrates that the new approach achieves a higher reduction of FP marks when compared to the previous one. Specifically, at 60%, 65%, and 70% per-mammogram sensitivity, the new CAD system achieves 0.50, 0.68, and 0.92 FP marks per mammogram, whereas at 70%, 75%, and 80% per-case sensitivity it achieves 0.37, 0.48, and 0.71 FP marks per mammogram, respectively. Conversely, at the same sensitivities, the previous CAD system reached 0.71, 0.87, and 1.15 FP marks per mammogram, and 0.57, 0.73, and 0.92 FPs per mammogram. Also, statistical significance of the difference between the two per-mammogram and per-case FROC curves is demonstrated by the p-value<0.001 returned by jackknife FROC analysis performed on the two CAD systems.
A novel method of generating artificial scintillation noise is developed and used to evaluate occultation rates and false positive rates for surveys probing the Kuiper Belt with the method of serendipitous stellar occultations. A thorough examination of survey design shows that (1) diffraction-dominated occultations are critically (Nyquist) sampled at a rate of 2 Fsu{sup -1}, corresponding to 40 s{sup -1} for objects at 40 AU, (2) occultation detection rates are maximized when targets are observed at solar opposition, (3) Main Belt asteroids will produce occultations light curves identical to those of Kuiper Belt Objects (KBOs) if target stars are observed at solar elongations of: 116{sup 0} {approx}< {epsilon} {approx}< 125 deg., or 131 deg. {approx}< {epsilon} {approx}< 141 deg., and (4) genuine KBO occultations are likely to be so rare that a detection threshold of {approx}>7-8{sigma} should be adopted to ensure that viable candidate events can be disentangled from false positives.
Astrophysical false positives due to stellar eclipsing binaries pose one of the greatest challenges to ground-based surveys for transiting hot Jupiters. We have used known properties of multiple star systems and hot Jupiter systems to predict, a priori, the number of such false detections and the number of genuine planet detections recovered in two hypothetical but realistic ground-based transit surveys targeting fields close to the galactic plane (b {approx} 10{sup 0}): a shallow survey covering a magnitude range 10 < V < 13 and a deep survey covering a magnitude range 15 < V < 19. Our results are consistent with the commonly reported experience of false detections outnumbering planet detections by a factor of {approx}10 in shallow surveys, while in our synthetic deep survey we find {approx}1-2 false detections for every planet detection. We characterize the eclipsing binary configurations that are most likely to cause false detections and find that they can be divided into three main types: (1) two dwarfs undergoing grazing transits, (2) two dwarfs undergoing low-latitude transits in which one component has a substantially smaller radius than the other, and (3) two eclipsing dwarfs blended with one or more physically unassociated foreground stars. We also predict that a significant fraction of hot Jupiter detections are blended with the light from other stars, showing that care must be taken to identify the presence of any unresolved neighbors in order to obtain accurate estimates of planetary radii. This issue is likely to extend to terrestrial planet candidates in the CoRoT and Kepler transit surveys, for which neighbors of much fainter relative brightness will be important.
Asterodensity profiling (AP) is a relatively new technique for studying transit light curves. By comparing the mean stellar density derived from the transit light curve to that found through an independent method, AP provides information on several useful properties such as orbital eccentricity and blended light. We present an AP survey of 41 Kepler Objects of Interest (KOIs), with a single transiting candidate, for which the target star's mean stellar density has been measured using asteroseismology. The ensemble distribution of the AP measurements for the 31 dwarf stars in our sample shows excellent agreement with the spread expected if the KOIs were genuine and have realistic eccentricities. In contrast, the same test for the 10 giants in our sample reveals significant incompatibility at >4σ confidence. While extreme eccentricities could be invoked, this hypothesis requires four of the KOIs to contact their host star at periastron passage, including the recently claimed confirmation of Kepler-91b. After carefully examining several hypotheses, we conclude that the most plausible explanation is that the transiting objects orbit a different star to that measured with asteroseismology—cases we define as false-positives. Based on the AP distribution, we estimate a false-positive rate (FPR) for Kepler's giant stars with a single transiting object of FPR ≅ 70% ± 30%.
We present a framework to conservatively estimate the probability that any particular planet-like transit signal observed by the Kepler mission is in fact a planet, prior to any ground-based follow-up efforts. We use Monte Carlo methods based on stellar population synthesis and Galactic structure models, and report false positive probabilities (FPPs) for every Kepler Object of Interest, assuming a 20% intrinsic occurrence rate of close-in planets in the radius range 0.5 R{sub +} < R{sub p} < 20 R{sub +}. Nearly 90% of the 1235 candidates have FPP <10%, and over half have FPP <5%. This probability varies with the magnitude and Galactic latitude of the target star, and with the depth of the transit signal-deeper signals generally have higher FPPs than shallower signals. We establish that a single deep high-resolution image will be an effective follow-up tool for the shallowest (Earth-sized) transits, providing the quickest route toward probabilistically validating the smallest candidates by potentially decreasing the FPP of an Earth-sized transit around a faint star from >10% to <1%. Since Kepler has detected many more planetary signals than can be positively confirmed with ground-based follow-up efforts in the near term, these calculations will be crucial to using the ensemble of Kepler data to determine population characteristics of planetary systems. We also describe how our analysis complements the Kepler team's more detailed BLENDER false positive analysis for planet validation.
Purpose: To examine both the IQM’s sensitivity and false positive rate to varying MLC errors. By balancing these two characteristics, an optimal tolerance value can be derived. Methods: An un-modified SBRT Liver IMRT plan containing 7 fields was randomly selected as a representative clinical case. The active MLC positions for all fields were perturbed randomly from a square distribution of varying width (±1mm to ±5mm). These unmodified and modified plans were measured multiple times each by the IQM (a large area ion chamber mounted to a TrueBeam linac head). Measurements were analyzed relative to the initial, unmodified measurement. IQM readings are analyzed as a function of control points. In order to examine sensitivity to errors along a field’s delivery, each measured field was divided into 5 groups of control points, and the maximum error in each group was recorded. Since the plans have known errors, we compared how well the IQM is able to differentiate between unmodified and error plans. ROC curves and logistic regression were used to analyze this, independent of thresholds. Results: A likelihood-ratio Chi-square test showed that the IQM could significantly predict whether a plan had MLC errors, with the exception of the beginning and ending control points. Upon further examination, we determined there was ramp-up occurring at the beginning of delivery. Once the linac AFC was tuned, the subsequent measurements (relative to a new baseline) showed significant (p <0.005) abilities to predict MLC errors. Using the area under the curve, we show the IQM’s ability to detect errors increases with increasing MLC error (Spearman’s Rho=0.8056, p<0.0001). The optimal IQM count thresholds from the ROC curves are ±3%, ±2%, and ±7% for the beginning, middle 3, and end segments, respectively. Conclusion: The IQM has proven to be able to detect not only MLC errors, but also differences in beam tuning (ramp-up). Partially supported by the Susan Scott Foundation.
Identifying liquid water on the surface of planets is a high priority, as this traditionally defines habitability. One proposed signature of oceans is specular reflection ('glint'), which increases the apparent albedo of a planet at crescent phases. We post-process a global climate model of an Earth-like planet to simulate reflected light curves. Significantly, we obtain glint-like phase variations even though we do not include specular reflection in our model. This false positive is the product of two generic properties: (1) for modest obliquities, a planet's poles receive less orbit-averaged stellar flux than its equator, so the poles are more likely to be covered in highly reflective snow and ice; and (2) we show that reflected light from a modest-obliquity planet at crescent phases probes higher latitudes than at gibbous phases, therefore a planet's apparent albedo will naturally increase at crescent phase. We suggest that this 'latitude-albedo effect' will operate even for large obliquities: in that case the equator receives less orbit-averaged flux than the poles, and the equator is preferentially sampled at crescent phase. Using rotational and orbital color variations to map the surfaces of directly imaged planets and estimate their obliquity will therefore be a necessary pre-condition for properly interpreting their reflected phase variations. The latitude-albedo effect is a particularly convincing glint false positive for zero-obliquity planets, and such worlds are not amenable to latitudinal mapping. This effect severely limits the utility of specular reflection for detecting oceans on exoplanets.