Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “false positive analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
NASA NTRS2022

Using Trajectory Smoothness Metrics to Identify Drones in Radar Track Data

The identification of unmanned aircraft systems (UAS) using trajectory data is considered. Specifically, a number of smoothness metrics are proposed, which can be used to distinguish UAS from other aerial objects even when they are engaged in accelerative maneuvers (non-constant-velocity flight). The metrics are evaluated on a data set from a UAS sense-and-avoid field test, which contains track data of aerial objects recorded by a vehicle-board radar system during a flight test. The metrics are found to effectively differentiate UAS from other objects such as birds for this data set. In addition, an initial statistical performance analysis of one of the smoothness metrics is undertaken, using 15 data sets deriving from multiple flight tests. The smoothness metric is shown to identify the target UAS with 95% accuracy (95% true positive rate), while achieving a false positive rate of less than 9%.

Sandip Roy↗
DOE OSTI2025

Machine Learning for Anomaly Detection in Neural Network Security and SRF Cavities

This dissertation explores the development and deployment of machine learning approaches to address critical challenges in anomaly detection across two distinct domains: neural network security in federated learning settings and cavity behavior analysis in particle accelerator operations at Jefferson Lab in Newport News, Virginia. Anomaly detection identifies deviations from expected patterns, safeguarding systems in cybersecurity, industry, and research against malicious activities and failures. This dissertation demonstrates how our machine learning approaches enhance detection accuracy and efficiency in both neural network security and industrial applications. First, we investigate vulnerabilities in deep neural networks deployed in federated learning. Although federated learning preserves user privacy by training models locally, it remains vulnerable to backdoor attacks, in which malicious participants embed hidden triggers that induce targeted misbehavior. We propose a self-supervised contrastive learning framework to detect and mitigate such backdoor attacks. In our experiments, this method achieves higher detection accuracy and lower false positive rates than existing defenses, while operating without access to local model updates or original training data and thus preserving the privacy guarantees of the federated setting. Second, we address the operational reliability of superconducting radio-frequency (SRF) cavities at the Continuous Electron Beam Accelerator Facility (CEBAF). Our research leverages an unsupervised learning approach, combined with Principal Component Analysis (PCA) and k-means clustering, to identify anomalous behaviors in SRF cavities. Our method detects subtle anomalous behavior by analyzing SRF signal data. This knowledge allows for the early detection and resolution of potential faults, significantly improving the efficiency and reliability of operations. Third, we extend these insights to time-series anomaly detection more broadly. We design a contrastive-learning based model tailored to increasingly dynamic environments and academic research. This model improves detection accuracy in settings that require real-time monitoring and predictive maintenance. Our research underscores the broader applicability and impact of advanced machine learning techniques in anomaly detection. By extracting meaningful patterns from complex data, machine learning can significantly enhance security in distributed neural networks and improve the efficiency of particle accelerator operations. This dissertation serves as a stepping stone for future investigations into the vast possibilities of anomaly detection, inspiring further exploration and development of machine learning techniques in this field.

Ferguson, Hal [Old Dominion University]↗
NASA NTRS2011

Kepler

The NASA Kepler mission recently announced over 1200 exoplanet candidates. While some are common Hot Jupiters, a large number are Neptune size and smaller, transit depths suggest sizes down to the radius of Earth. The Kepler project has a fairly high confidence that most of these candidates are real exoplanets. Many analysis steps and lessons learned from Kepler light curves are used during the vetting process. This talk will cover some new results in the areas of stellar variability, solar systems with multiple planets, and how transit-like signatures are vetted for false positives, especially those indicative of small planets.

Howell, Steve B.↗
DOE OSTI2022

Planning, design and logistics of a decision analysis study: The FBI/Ames study involving forensic firearms examiners

This paper describes design and logistical aspects of a decision analysis study to assess the performance of qualified firearms examiners working in accredited laboratories in the United States in terms of accuracy (error rate), repeatability, and reproducibility of decisions involving comparisons of fired bullets and cartridge cases. The purpose of the study was to validate current practice of the forensic discipline of firearms/toolmarks (F/T) examination. It elicited error rate data by counting the number of false positive and false negative conclusions. Preceded by the experimental design, decisions, and logistics described herein, testing was ultimately administered 173 qualified, practicing F/T examiners in public and private crime laboratories. The first round of testing evaluated accuracy, while two subsequent rounds evaluated repeatability and reproducibility of examiner conclusions. This project expands on previous studies by involving many F/T examiners in challenging comparisons and by executing the study in the recommended double-blind format.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2023

Development of a Universal Second-Tier Newborn Screening LC–MS/MS Method for Amino Acids, Lysophosphatidylcholines, and Organic Acids

First-tier MS-based newborn screening by flow injection analysis can have high presumptive positive rates, often due to isomeric/isobaric compounds or poor biomarker specificity. These presumptive positive samples can be analyzed by second-tier screening assays employing separations such as liquid chromatography–mass spectrometry (LC–MS/MS), which increases test specificity and drastically reduces false positive referrals. The ability to screen for multiple disorders in a single multiplexed test simplifies workflows and maximizes public health laboratories’ resources. In this study, we developed and validated a highly multiplexed second-tier method for dried blood spots using a hydrophilic interaction liquid chromatography (HILIC) column coupled to an MS/MS system. The LC–MS/MS method was capable of simultaneously detecting second-tier biomarkers for maple syrup urine disease, homocystinuria, methylmalonic acidemia, propionic acidemia, glutaric acidemia type 1, glutaric acidemia type 2, guanidinoacetate methyltransferase deficiency, short-chain acyl-CoA dehydrogenase deficiency, adrenoleukodystrophy, and Pompe disease.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS2014

Abiotic Ozone and Oxygen in Atmospheres Similar to Prebiotic Earth

The search for life on planets outside our solar system will use spectroscopic identification of atmospheric biosignatures. The most robust remotely detectable potential biosignature is considered to be the detection of oxygen (O 2 ) or ozone (O 3 ) simultaneous to methane (CH 4 ) at levels indicating fluxes from the planetary surface in excess of those that could be produced abiotically. Here we use an altitude-dependent photochemical model with the enhanced lower boundary conditions necessary to carefully explore abiotic O 2 and O 3 production on lifeless planets with a wide variety of volcanic gas fluxes and stellar energy distributions. On some of these worlds, we predict limited O 2 and O 3 buildup, caused by fast chemical production of these gases. This results in detectable abiotic O 3 and CH 4 features in the UV-visible, but no detectable abiotic O 2 features. Thus, simultaneous detection of O 3 and CH 4 by a UV-visible mission is not a strong biosignature without proper contextual information. Discrimination between biological and abiotic sources of O 2 and O 3 is possible through analysis of the stellar and atmospheric context-particularly redox state and O atom inventory-of the planet in question. Specifically, understanding the spectral characteristics of the star and obtaining a broad wavelength range for planetary spectra should allow more robust identification of false positives for life. This highlights the importance of wide spectral coverage for future exoplanet characterization missions. Specifically, discrimination between true and false positives may require spectral observations that extend into infrared wavelengths and provide contextual information on the planet's atmospheric chemistry.

planets and satellites↗
DOE OSTI2020

Whole program Adaptive Error Detection and Mitigation

Our project investigated methods to detect soft errors in computing systems. Such detection schemes must have low overheads and also generate no false positives. Our solutions focused on control-space error detection based on changing addressing schemes so that faults tend to cascade – rather than being masked. Our solutions also addressed data-space error detection by synthesizing error detectors based on rigorous floating-point error analysis.

42 ENGINEERING↗
NASA NTRS2021

Cloud Detection over Sunglint Regions with Observations from the Earth Polychromatic Imaging Camera (EPIC)

With the ability to observe the entire sunlit side of the Earth, EPIC data have become an important resource for studying cloud daily variability. Inaccurate cloud masking is a great source of uncertainty. One main region that is prone to error in cloud masking is the sunglint area over ocean surfaces. Cloud detection over these regions is challenging for the EPIC instrument because of its limited spectral channels. Clear sky ocean surface reflectance from visible channels over sunglint is much larger than that over the non-glint areas and can exceed reflectance from thin clouds. This paper presents an improved EPIC ocean cloud masking algorithm (Version 3). Over sunglint regions (glint angle ≤ 25°), the algorithm utilizes EPIC’s oxygen (O2) A-band ratio (764nm to 780 nm) in addition to the 780 nm reflectance observations in masking tests. Outside the sunglint regions, a dynamic reflectance threshold for the Rayleigh corrected 780 nm reflectance is applied. The thresholds are derived as a function of glint angle. When compared with co-located data from the geosynchronous Earth orbit (GEO) and the low Earth orbit (LEO) observations, the consistency of the new ocean cloud mask algorithm has increased by 4~10% and 4~6% in the glint center and granule edges respectively. The false positive rate is reduced by 10~17%. Overall global ocean cloud detection consistency increases by 2%. This algorithm, along with other improvements to the EPIC cloud masks, has been implemented in the EPIC cloud products Version 3. This algorithm will improve the cloud daily variability analysis by removing the artificial peak at local noon time in the glint center latitudes and reducing biases in the early morning and late afternoon cloud fraction over ocean surfaces.

EPIC↗
DOE OSTI2020

A robust benchmark for detection of germline large deletions and insertions

New technologies and analysis methods are enabling genomic structural variants (SVs) to be detected with ever-increasing accuracy, resolution and comprehensiveness. To help translate these methods to routine research and clinical practice, we developed a sequence-resolved benchmark set for identification of both false-negative and false-positive germline large insertions and deletions. To create this benchmark for a broadly consented son in a Personal Genome Project trio with broadly available cells and DNA, the Genome in a Bottle Consortium integrated 19 sequence-resolved variant calling methods from diverse technologies. The final benchmark set contains 12,745 isolated, sequence-resolved insertion (7,281) and deletion (5,464) calls ≥50 base pairs (bp). The Tier 1 benchmark regions, for which any extra calls are putative false positives, cover 2.51 Gbp and 5,262 insertions and 4,095 deletions supported by ≥1 diploid assembly. We demonstrate that the benchmark set reliably identifies false negatives and false positives in high-quality SV callsets from short-, linked- and long-read sequencing and optical mapping.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS2004

Life Detection on the Early Earth

Finding evidence for first the existence, and then the nature of life on the early Earth or early Mars requires both the recognition of subtle biosignatures and the elimination of false positives. The history of the search for fossils in increasingly older Precambrian strata illustrates these difficulties very clearly, and new observational and theoretical approaches are both needed and being developed. At the microscopic level of investigation, three-dimensional morphological characterization coupled with in situ chemical (isotopic, elemental, structural) analysis is the desirable first step. Geological context is paramount, as has been demonstrated by the controversies over AH84001, the Greenland graphites, and the Apex chert microfossils . At larger scales, the nature of sedimentary bedforms and the structures they display becomes crucial, and here the methods of condensed matter physics prove most useful in discriminating between biological and non-biological constructions. Ultimately, a combination of geochemical, morphological, and contextural evidence may be required for certain life detection on the early Earth or elsewhere.

Runnegar, B.↗
NASA NTRS2018

Optimization of Second Fault Detection Thresholds to Maximize Mission POS

In order to support manned spaceflight safety requirements, the Space Launch System (SLS) has defined program-level requirements for key systems to ensure successful operation under single fault conditions. To accommodate this with regards to Navigation, the SLS utilizes an internally redundant Inertial Navigation System (INS) with built-in capability to detect, isolate, and recover from first failure conditions and still maintain adherence to performance requirements. The unit utilizes multiple hardware- and software-level techniques to enable detection, isolation, and recovery from these events in terms of its built-in Fault Detection, Isolation, and Recovery (FDIR) algorithms. Successful operation is defined in terms of sufficient navigation accuracy at insertion while operating under worst case single sensor outages (gyroscope and accelerometer faults at launch). In addition to first fault detection and recovery, the SLS program has also levied requirements relating to the capability of the INS to detect a second fault, tracking any unacceptable uncertainty in knowledge of the vehicle's state. This detection functionality is required in order to feed abort analysis and ensure crew safety. Increases in navigation state error and sensor faults can drive the vehicle outside of its operational as-designed environments and outside of its performance envelope causing loss of mission, or worse, loss of crew. The criteria for operation under second faults allows for a larger set of achievable missions in terms of potential fault conditions, due to the INS operating at the edge of its capability. As this performance is defined and controlled at the vehicle level, it allows for the use of system level margins to increase probability of mission success on the operational edges of the design space. Due to the implications of the vehicle response to abort conditions (such as a potentially failed INS), it is important to consider a wide range of failure scenarios in terms of both magnitude and time. As such, the Navigation team is taking advantage of the INS's capability to schedule and change fault detection thresholds in flight. These values are optimized along a nominal trajectory in order to maximize probability of mission success, and reducing the probability of false positives (defined as when the INS would report a second fault condition resulting in loss of mission, but the vehicle would still meet insertion requirements within system-level margins). This paper will describe an optimization approach using Genetic Algorithms to tune the threshold parameters to maximize vehicle resilience to second fault events as a function of potential fault magnitude and time of fault over an ascent mission profile. The analysis approach, and performance assessment of the results will be presented to demonstrate the applicability of this process to second fault detection to maximize mission probability of success.

Anzalone, Evan↗
DOE OSTI2026

Unsupervised anomaly detection in MeV ultrafast electron diffraction

MeV ultrafast electron diffraction (MUED) is a pump-probe technique used to study the dynamic structural evolution of materials. An ultrashort laser pulse triggers structural changes, which are then probed by an ultrashort relativistic electron beam. To overcome low signal-to-noise ratios, diffraction patterns are averaged over thousands of shots. However, shot-to-shot instabilities in the electron beam can distort individual patterns, introducing uncertainty. Improving MUED accuracy requires detecting and removing these anomalous patterns from large datasets. In this work, we developed a fully unsupervised methodology for the detection of anomalous diffraction patterns. Using a convolutional autoencoder, we calculate the reconstruction mean squared error of the diffraction patterns. Based on the statistical analysis of this error, we provide the user an estimation of the probability that the pattern is normal, which also allows a posterior visual inspection of the images that are difficult to classify. This method has been trained with only 100 diffraction patterns and tested on 1521 patterns, resulting in a false positive rate between 0.2% and 0.4%, with a training time of 10 s per image and a test time of about 1 s per image. Here, the proposed methodology can also be applied to other diffraction techniques in which large datasets are collected that include faulty images due to instrumental instabilities.

43 PARTICLE ACCELERATORS↗
DOE OSTI2024

Toward Automated Detection of Portability Bugs in Kokkos Parallel Programs

Performance-portable programming frameworks provide abstractions for parallel execution to allow easily porting an application to multiple backend programming models, such as CUDA, HIP, and OpenMP. However, programs may still have portability bugs that manifest only on specific backends. Traditional testing is ineffective in discovering these bugs, as it would require concrete execution on all supported hardware configurations for a potentially infinite set of inputs. To mitigate this issue, we focused on a specific programming framework, Kokkos, and identified several categories of common portability bugs. We then developed Klokkos, a static analysis approach based on symbolic execution that can run on commodity hardware, before execution on supercomputers. As a proof-of-concept, we ran Klokkos on examples encoding the identified bugs. Our results show that Klokkos is effective, efficient, and precise: it detected all the considered bugs, quickly, and without any false positives. Although preliminary, our results motivate further research and development in this direction.

Kale, Vivek↗
NASA NTRS2012

Multi-Sensor Observations of Earthquake Related Atmospheric Signals over Major Geohazard Validation Sites

We are conducting a scientific validation study involving multi-sensor observations in our investigation of phenomena preceding major earthquakes. Our approach is based on a systematic analysis of several atmospheric and environmental parameters, which we found, are associated with the earthquakes, namely: thermal infrared radiation, outgoing long-wavelength radiation, ionospheric electron density, and atmospheric temperature and humidity. For first time we applied this approach to selected GEOSS sites prone to earthquakes or volcanoes. This provides a new opportunity to cross validate our results with the dense networks of in-situ and space measurements. We investigated two different seismic aspects, first the sites with recent large earthquakes, viz.- Tohoku-oki (M9, 2011, Japan) and Emilia region (M5.9, 2012,N. Italy). Our retrospective analysis of satellite data has shown the presence of anomalies in the atmosphere. Second, we did a retrospective analysis to check the re-occurrence of similar anomalous behavior in atmosphere/ionosphere over three regions with distinct geological settings and high seismicity: Taiwan, Japan and Kamchatka, which include 40 major earthquakes (M>5.9) for the period of 2005-2009. We found anomalous behavior before all of these events with no false negatives; false positives were less then 10%. Our initial results suggest that multi-instrument space-borne and ground observations show a systematic appearance of atmospheric anomalies near the epicentral area that could be explained by a coupling between the observed physical parameters and earthquake preparation processes.

Ouzounov, D.↗
NASA NTRS2015

Element Load Data Processor (ELDAP) Users Manual

Often, the shear and tensile forces and moments are extracted from finite element analyses to be used in off-line calculations for evaluating the integrity of structural connections involving bolts, rivets, and welds. Usually the maximum forces and moments are desired for use in the calculations. In situations where there are numerous structural connections of interest for numerous load cases, the effort in finding the true maximum force and/or moment combinations among all fasteners and welds and load cases becomes difficult. The Element Load Data Processor (ELDAP) software described herein makes this effort manageable. This software eliminates the possibility of overlooking the worst-case forces and moments that could result in erroneous positive margins of safety and/or selecting inconsistent combinations of forces and moments resulting in false negative margins of safety. In addition to forces and moments, any scalar quantity output in a PATRAN report file may be evaluated with this software. This software was originally written to fill an urgent need during the structural analysis of the Ares I-X Interstage segment. As such, this software was coded in a straightforward manner with no effort made to optimize or minimize code or to develop a graphical user interface.

Ramsey, John K., Jr.↗
DOE OSTI2025

eDNAjoint: An R package for interpreting paired or semi‐paired environmental DNA and traditional survey data in a Bayesian framework

Abstract Environmental DNA (eDNA) sampling is increasingly used in surveys of species distribution as a potentially sensitive and efficient monitoring method. Yet access to modelling tools designed specifically for interpreting this new data type lags behind its ubiquity. While occupancy modelling software has dominated the analytical landscape for eDNA data analysis of single species, this type of model may not always be the most appropriate. The rate of eDNA detection often corresponds to species density, rather than just occupancy, and researchers often have access to observations from non‐genetic sampling methods at the same sites. To provide users access to a modelling framework designed to maximize the use of all available data, we developed an R package, eDNAjoint . The package provides an easy‐to‐use interface for fitting a ‘joint’ model that integrates data from paired or semi‐paired eDNA and traditional surveys in a Bayesian framework. The model can be used to estimate parameters like the probability of a false positive eDNA detection and mean catch rate at a site, and the package allows access to multiple model variations and Bayesian prior customization. Additional functionality can be used for model selection, summarising posteriors and comparing the relative sensitivities of the two survey methods. We demonstrate the use of eDNAjoint by fitting a variation of the model with site‐level covariates that scale the sensitivity of eDNA sampling relative to traditional sampling. The example workflow uses binary eDNA and seine count data for the endangered tidewater goby ( Eucyclogobius newberryi ) from a study by Schmelzle and Kinziger (2016). This use case includes a prior sensitivity analysis and an evaluation of the relationship between detection rates and environmental variables. eDNAjoint has the potential to greatly increase the range of users who will be able to rigorously analyse eDNA and traditional survey data in a Bayesian framework, understand if and how eDNA can improve monitoring practices, and gain confidence in the interpretability of eDNA data.

Keller, Abigail G. [Department of Environment Scie↗
DOE OSTI2021

A novel experimental system for the KDK measurement of the 40 K decay scheme relevant for rare event searches

Potassium-40 ( 40 K) is a long-lived, naturally occurring radioactive isotope. The decay products are prominent backgrounds for many rare event searches, including those involving NaI-based scintillators. 40 K also plays a role in geochronological dating techniques. The branching ratio of the electron capture directly to the ground state of argon-40 has never been measured, which can cause difficulty in interpretingcertain results or can lead to lack of precision depending on the field and analysis technique. Additionally, the KDK (Potassium (K) Decay (DK)) collaboration is measuring this decay. A composite method has a silicon drift detector with an enriched, thermally deposited 40 K source inside the Modular Total Absorption Spectrometer. This setup has been characterized in terms of energy calibration, gamma tagging efficiency, live time and false negatives and positives. A complementary, homogeneous, method is also discussed; it employs a KSr 2 I 5 :Eu scintillator as source and detector.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
NASA NTRS2016

The IPAC Image Subtraction and Discovery Pipeline for the Intermediate Palomar Transient Factory

We describe the near real-time transient-source discovery engine for the intermediate Palomar Transient Factory (iPTF), currently in operations at the Infrared Processing and Analysis Center (IPAC), Caltech. We coin this system the IPAC/iPTF Discovery Engine (or IDE). We review the algorithms used for PSF-matching, image subtraction, detection, photometry, and machine-learned (ML) vetting of extracted transient candidates. We also review the performance of our ML classifier. For a limiting signal-to-noise ratio of 4 in relatively unconfused regions, bogus candidates from processing artifacts and imperfect image subtractions outnumber real transients by approximately equal to 10:1. This can be considerably higher for image data with inaccurate astrometric and/or PSF-matching solutions. Despite this occasionally high contamination rate, the ML classifier is able to identify real transients with an efficiency (or completeness) of approximately equal to 97% for a maximum tolerable false-positive rate of 1% when classifying raw candidates. All subtraction-image metrics, source features, ML probability-based real-bogus scores, contextual metadata from other surveys, and possible associations with known Solar System objects are stored in a relational database for retrieval by the various science working groups. We review our efforts in mitigating false-positives and our experience in optimizing the overall system in response to the multitude of science projects underway with iPTF.

methods: analytical – methods: data analysis –↗