Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “false positive analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Random forest models accurately classify synthetic opioids using high-dimensionality mass spectrometry datasets

Detection of novel threat agents presents several challenges, a principle one being the development of untargeted methods to screen an increasing number of threat chemicals whose exact structures are unknown. With the use of Machine Learning (ML) tools, we can guide the development of analytical methods for broad-spectrum detection of unbounded threat chemical families in complex mixtures. Toward this goal, we used nominal mass and high-resolution mass spectrometry data for hundreds of synthetic opioids and non-opioid compounds. We tested two ML techniques, logistic regression and random forest, to develop models towards a practical, implementable method for opioid detection. We found that of these tested ML methods, random forest models resulted in the highest validation accuracy (95+%) for both nominal mass and high-resolution classification of opioids versus non-opioids, with low false positive and false negative rates. The RF models were then used to successfully predict the classification of 10 compounds—five opioids and five non-opioids not part of the training and validation analysis. This application of ML is a critical step towards the development of field-deployable nominal mass spectrometers with ML-driven analyses for classification of emergent threats.

Arasteh, Kourosh [Lawrence Livermore National Labo↗
DOE OSTI2026

Improving Building Footprint Extraction Using NAIP and 3DEP Lidar Derived Features with Deep Learning

Accurate building footprint extraction is critical for applications ranging from population estimation to disaster management. Although optical imagery provides detailed spectral information, it often struggles with shadows, occlusions, and background clutter in dense urban environments. Lidar data, by contrast, offer precise elevation and structural attributes but face challenges such as variable point density and noise. This study integrates multispectral imagery from the U.S. Department of Agriculture (USDA) National Agriculture Imagery Program (NAIP) with lidar-derived feature height and intensity from the U.S. Geological Survey (USGS) 3D Elevation Program (3DEP) to improve footprint extraction using a U-Net–based deep learning model. A six-band input stack (RGB, near-infrared, height, intensity) was developed, normalized, and tiled for training and evaluation against Microsoft Global Building Footprints (GBF). Results from the Houston, TX test site show that the six-band model achieved a precision of 0.86, recall of 0.88, F1 score of 0.87, and Intersection-over-Union (IoU) of 0.76, consistently outperforming four-band baselines by reducing false positives while maintaining sensitivity. Predictions on withheld Houston tiles confirmed strong within-region generalization, yielded a precision of 0.78, recall of 0.81, F1 score of 0.79, and IoU of 0.66. Qualitative analysis further revealed limitations stemming from both training label quality and vegetation–building confusion. These findings demonstrate the complementary value of integrating spectral and structural information for robust building footprint extraction and how domain adaptation strategies can be used to enhance cross-regional transferability.

Liu, Jung Kuan [United States Geological Survey (U↗
DOE OSTI2025

Machine Learning for Anomaly Detection in Neural Network Security and SRF Cavities

This dissertation explores the development and deployment of machine learning approaches to address critical challenges in anomaly detection across two distinct domains: neural network security in federated learning settings and cavity behavior analysis in particle accelerator operations at Jefferson Lab in Newport News, Virginia. Anomaly detection identifies deviations from expected patterns, safeguarding systems in cybersecurity, industry, and research against malicious activities and failures. This dissertation demonstrates how our machine learning approaches enhance detection accuracy and efficiency in both neural network security and industrial applications. First, we investigate vulnerabilities in deep neural networks deployed in federated learning. Although federated learning preserves user privacy by training models locally, it remains vulnerable to backdoor attacks, in which malicious participants embed hidden triggers that induce targeted misbehavior. We propose a self-supervised contrastive learning framework to detect and mitigate such backdoor attacks. In our experiments, this method achieves higher detection accuracy and lower false positive rates than existing defenses, while operating without access to local model updates or original training data and thus preserving the privacy guarantees of the federated setting. Second, we address the operational reliability of superconducting radio-frequency (SRF) cavities at the Continuous Electron Beam Accelerator Facility (CEBAF). Our research leverages an unsupervised learning approach, combined with Principal Component Analysis (PCA) and k-means clustering, to identify anomalous behaviors in SRF cavities. Our method detects subtle anomalous behavior by analyzing SRF signal data. This knowledge allows for the early detection and resolution of potential faults, significantly improving the efficiency and reliability of operations. Third, we extend these insights to time-series anomaly detection more broadly. We design a contrastive-learning based model tailored to increasingly dynamic environments and academic research. This model improves detection accuracy in settings that require real-time monitoring and predictive maintenance. Our research underscores the broader applicability and impact of advanced machine learning techniques in anomaly detection. By extracting meaningful patterns from complex data, machine learning can significantly enhance security in distributed neural networks and improve the efficiency of particle accelerator operations. This dissertation serves as a stepping stone for future investigations into the vast possibilities of anomaly detection, inspiring further exploration and development of machine learning techniques in this field.

Ferguson, Hal [Old Dominion University]↗
DOE OSTI2022

Planning, design and logistics of a decision analysis study: The FBI/Ames study involving forensic firearms examiners

This paper describes design and logistical aspects of a decision analysis study to assess the performance of qualified firearms examiners working in accredited laboratories in the United States in terms of accuracy (error rate), repeatability, and reproducibility of decisions involving comparisons of fired bullets and cartridge cases. The purpose of the study was to validate current practice of the forensic discipline of firearms/toolmarks (F/T) examination. It elicited error rate data by counting the number of false positive and false negative conclusions. Preceded by the experimental design, decisions, and logistics described herein, testing was ultimately administered 173 qualified, practicing F/T examiners in public and private crime laboratories. The first round of testing evaluated accuracy, while two subsequent rounds evaluated repeatability and reproducibility of examiner conclusions. This project expands on previous studies by involving many F/T examiners in challenging comparisons and by executing the study in the recommended double-blind format.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2023

Development of a Universal Second-Tier Newborn Screening LC–MS/MS Method for Amino Acids, Lysophosphatidylcholines, and Organic Acids

First-tier MS-based newborn screening by flow injection analysis can have high presumptive positive rates, often due to isomeric/isobaric compounds or poor biomarker specificity. These presumptive positive samples can be analyzed by second-tier screening assays employing separations such as liquid chromatography–mass spectrometry (LC–MS/MS), which increases test specificity and drastically reduces false positive referrals. The ability to screen for multiple disorders in a single multiplexed test simplifies workflows and maximizes public health laboratories’ resources. In this study, we developed and validated a highly multiplexed second-tier method for dried blood spots using a hydrophilic interaction liquid chromatography (HILIC) column coupled to an MS/MS system. The LC–MS/MS method was capable of simultaneously detecting second-tier biomarkers for maple syrup urine disease, homocystinuria, methylmalonic acidemia, propionic acidemia, glutaric acidemia type 1, glutaric acidemia type 2, guanidinoacetate methyltransferase deficiency, short-chain acyl-CoA dehydrogenase deficiency, adrenoleukodystrophy, and Pompe disease.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Whole program Adaptive Error Detection and Mitigation

Our project investigated methods to detect soft errors in computing systems. Such detection schemes must have low overheads and also generate no false positives. Our solutions focused on control-space error detection based on changing addressing schemes so that faults tend to cascade – rather than being masked. Our solutions also addressed data-space error detection by synthesizing error detectors based on rigorous floating-point error analysis.

42 ENGINEERING↗
DOE OSTI2020

A robust benchmark for detection of germline large deletions and insertions

New technologies and analysis methods are enabling genomic structural variants (SVs) to be detected with ever-increasing accuracy, resolution and comprehensiveness. To help translate these methods to routine research and clinical practice, we developed a sequence-resolved benchmark set for identification of both false-negative and false-positive germline large insertions and deletions. To create this benchmark for a broadly consented son in a Personal Genome Project trio with broadly available cells and DNA, the Genome in a Bottle Consortium integrated 19 sequence-resolved variant calling methods from diverse technologies. The final benchmark set contains 12,745 isolated, sequence-resolved insertion (7,281) and deletion (5,464) calls ≥50 base pairs (bp). The Tier 1 benchmark regions, for which any extra calls are putative false positives, cover 2.51 Gbp and 5,262 insertions and 4,095 deletions supported by ≥1 diploid assembly. We demonstrate that the benchmark set reliably identifies false negatives and false positives in high-quality SV callsets from short-, linked- and long-read sequencing and optical mapping.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

Unsupervised anomaly detection in MeV ultrafast electron diffraction

MeV ultrafast electron diffraction (MUED) is a pump-probe technique used to study the dynamic structural evolution of materials. An ultrashort laser pulse triggers structural changes, which are then probed by an ultrashort relativistic electron beam. To overcome low signal-to-noise ratios, diffraction patterns are averaged over thousands of shots. However, shot-to-shot instabilities in the electron beam can distort individual patterns, introducing uncertainty. Improving MUED accuracy requires detecting and removing these anomalous patterns from large datasets. In this work, we developed a fully unsupervised methodology for the detection of anomalous diffraction patterns. Using a convolutional autoencoder, we calculate the reconstruction mean squared error of the diffraction patterns. Based on the statistical analysis of this error, we provide the user an estimation of the probability that the pattern is normal, which also allows a posterior visual inspection of the images that are difficult to classify. This method has been trained with only 100 diffraction patterns and tested on 1521 patterns, resulting in a false positive rate between 0.2% and 0.4%, with a training time of 10 s per image and a test time of about 1 s per image. Here, the proposed methodology can also be applied to other diffraction techniques in which large datasets are collected that include faulty images due to instrumental instabilities.

43 PARTICLE ACCELERATORS↗
DOE OSTI2024

Toward Automated Detection of Portability Bugs in Kokkos Parallel Programs

Performance-portable programming frameworks provide abstractions for parallel execution to allow easily porting an application to multiple backend programming models, such as CUDA, HIP, and OpenMP. However, programs may still have portability bugs that manifest only on specific backends. Traditional testing is ineffective in discovering these bugs, as it would require concrete execution on all supported hardware configurations for a potentially infinite set of inputs. To mitigate this issue, we focused on a specific programming framework, Kokkos, and identified several categories of common portability bugs. We then developed Klokkos, a static analysis approach based on symbolic execution that can run on commodity hardware, before execution on supercomputers. As a proof-of-concept, we ran Klokkos on examples encoding the identified bugs. Our results show that Klokkos is effective, efficient, and precise: it detected all the considered bugs, quickly, and without any false positives. Although preliminary, our results motivate further research and development in this direction.

Kale, Vivek↗
DOE OSTI2025

eDNAjoint: An R package for interpreting paired or semi‐paired environmental DNA and traditional survey data in a Bayesian framework

Abstract Environmental DNA (eDNA) sampling is increasingly used in surveys of species distribution as a potentially sensitive and efficient monitoring method. Yet access to modelling tools designed specifically for interpreting this new data type lags behind its ubiquity. While occupancy modelling software has dominated the analytical landscape for eDNA data analysis of single species, this type of model may not always be the most appropriate. The rate of eDNA detection often corresponds to species density, rather than just occupancy, and researchers often have access to observations from non‐genetic sampling methods at the same sites. To provide users access to a modelling framework designed to maximize the use of all available data, we developed an R package, eDNAjoint . The package provides an easy‐to‐use interface for fitting a ‘joint’ model that integrates data from paired or semi‐paired eDNA and traditional surveys in a Bayesian framework. The model can be used to estimate parameters like the probability of a false positive eDNA detection and mean catch rate at a site, and the package allows access to multiple model variations and Bayesian prior customization. Additional functionality can be used for model selection, summarising posteriors and comparing the relative sensitivities of the two survey methods. We demonstrate the use of eDNAjoint by fitting a variation of the model with site‐level covariates that scale the sensitivity of eDNA sampling relative to traditional sampling. The example workflow uses binary eDNA and seine count data for the endangered tidewater goby ( Eucyclogobius newberryi ) from a study by Schmelzle and Kinziger (2016). This use case includes a prior sensitivity analysis and an evaluation of the relationship between detection rates and environmental variables. eDNAjoint has the potential to greatly increase the range of users who will be able to rigorously analyse eDNA and traditional survey data in a Bayesian framework, understand if and how eDNA can improve monitoring practices, and gain confidence in the interpretability of eDNA data.

Keller, Abigail G. [Department of Environment Scie↗
DOE OSTI2021

A novel experimental system for the KDK measurement of the 40 K decay scheme relevant for rare event searches

Potassium-40 ( 40 K) is a long-lived, naturally occurring radioactive isotope. The decay products are prominent backgrounds for many rare event searches, including those involving NaI-based scintillators. 40 K also plays a role in geochronological dating techniques. The branching ratio of the electron capture directly to the ground state of argon-40 has never been measured, which can cause difficulty in interpretingcertain results or can lead to lack of precision depending on the field and analysis technique. Additionally, the KDK (Potassium (K) Decay (DK)) collaboration is measuring this decay. A composite method has a silicon drift detector with an enriched, thermally deposited 40 K source inside the Modular Total Absorption Spectrometer. This setup has been characterized in terms of energy calibration, gamma tagging efficiency, live time and false negatives and positives. A complementary, homogeneous, method is also discussed; it employs a KSr 2 I 5 :Eu scintillator as source and detector.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Photocatalytic water splitting

With the goal of achieving large-scale H 2 production from renewable resources, water splitting into H 2 and O 2 using semiconductor photocatalysts (sometimes called artificial photosynthesis) has been studied for five decades. Unfortunately, the lack of rigour and reproducibility in the data collection and analysis of experimental results has hindered progress in the field. This Primer provides a comprehensive overview of proper characterization and evaluation of photocatalysts for overall water splitting. In particular, the Primer covers various pitfalls in photocatalysis research, best practices for reproducibility and reliable methods for conducting rigorous experiments. As a result, the recommendations are intended to reduce false positives in the literature and to promote progress towards a practical technology for producing H 2 from water by using sunlight.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Stringy Bubbles Solve de Sitter Troubles

Finding four-dimensional de Sitter spacetime solutions in string theory has been a vexing quest ever since the discovery of the accelerating expansion of the universe. Building on a recent analysis of bubble-nucleation in the decay of (false-vacuum) AdS backgrounds where the interfacing bubbles themselves exhibit a de Sitter geometry we show that this resonates strongly with a stringy cosmic brane construction that naturally provides for an exponential mass-hierarchy and the localization of both gravity and matter, in addition to an exponentially suppressed positive cosmological constant. Finally, we argue that these scenarios can be realized in terms of a generalization of a small resolution of a conifold singularity in the context of a (Lorentzian) Calabi–Yau 5-fold, where the isolated (Lorentzian) two complex dimensional Fano variety is a four-dimensional de Sitter spacetime.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Deep Learning for Automated Identification of Eels in Sonar Data

Freshwater eels, such as the American eel (Anguilla rostrata) present numerous challenges related to safe downstream fish passage at hydroelectric facilities. One of those challenges is effective monitoring of their abundance, movements, and behavior to facilitate design and operation of eel protection and passage facilities. A previous EPRI study documented the ability of human analysts to reliably identify American eels in data obtained with a 1100/1800 kHz, multibeam sonar. This report describes a project to develop deep learning (a subset of artificial intelligence) tools to automate the time-consuming, subjective process of eel identification in multibeam sonar data. The project exploited new data collected in the laboratory and the existing data from the prior EPRI field study to develop and test deep learning and other data analytic tools, including wavelet filtering, differencing for static object removal, and convolutional neural network analysis. The analysis of the laboratory data demonstrated feasibility of the approach, revealed object characteristics observed with the sonar that distinguish eels from similarly sized and shaped acoustic targets, and provided additional data for algorithm selection and training. Deep learning algorithms trained and tested on the laboratory data alone achieved accuracy rates of greater than 98% when classifying acoustic images of eels and similar-sized neutrally buoyant sticks. The algorithm trained and tested on the pre-existing field data alone, and yielded classification accuracy of 9.3% false positives and 13.3% false negatives when distinguishing between eels and sticks/PVC pipes based on video clips (i.e., multiple, consecutive images). This performance is comparable to the classification accuracy achieved by human analysts in the prior study. The deep learning algorithm trained on a combination of video clips obtained in the laboratory and the field and tested on video clips from the field, was able to distinguish eels from sticks and PVC pipes (a river debris analog) of similar size with 100% accuracy. Outreach to the hardware, software, and end-user communities early in the project helped to identify needs and specify the application space. Outreach to those communities at the end of the project communicated project results and opportunities for further development. The project achieved proof of concept for automated identification of eel in multibeam sonar data. Future work should focus on acquisition of additional data for more robust algorithm training and testing; modification of the software tools to accommodate multiple acoustic targets in the acoustic field at a given time; identification of additional object classes; incorporation of motion in the object identification and classification algorithms; operationalizing the software tools, including integration with other existing sonar data analysis tools; and partnering with hardware and software providers for distribution of the software tools with their commercial products.

13 HYDRO ENERGY↗
DOE OSTI2024

Description of Pegethrix niliensis sp. nov., a Novel Cyanobacterium from the Nile River Basin, Egypt: A Polyphasic Analysis and Comparative Study of Related Genera in the Oculatellales Order

In this paper, we examine the filamentous cyanobacterial strain NILCB16 and describe it as a new species within the genus Pegethrix. The original population was sampled from a mat growing in an irrigation canal in the Nile River, Egypt. Initially classified under Plectonema or Planktolyngbya, the strain is a potential producer of the toxins microcystin and β-N-Methylamino-L-Alanine (BMAA). Additionally, we reviewed the taxonomic relationships between the Oculatellales genera. To describe the new species, we conducted a polyphasic study, encompassing 16S rRNA gene phylogenetic analyses performed using both Maximum Likelihood and Bayesian methods, sequence identity (p-distance) analysis, 16S-23S ITS secondary structures, and morphological and habitat comparisons. The phylogenetic analysis revealed that strain NILCB16 clustered within the Pegethrix clade with strong phylogenetic support, but in a distinct position from other species in the genus. The strain shared a maximum 16S rRNA gene identity of 97.3% with P. qiandaoensis and 96.1% with the type species, P. bostrychoides. Morphologically, NILCB16 can be differentiated from other species in the genus by its lack of false branching. Our phylogenetic analyses also show that Pegethrix, Cartusia, Elainella, and Maricoleus are clustered with strong phylogenetic support. They exhibit high 16S rRNA gene identity and are morphologically indistinguishable, suggesting they could potentially be merged into a single genus in the future.

Hentschke, Guilherme Scotta (ORCID:000000034396024↗
DOE OSTI2024

Detection of Anatoxins in Human Urine by Liquid Chromatography Triple Quadrupole Mass Spectrometry and ELISA

Harmful cyanobacterial blooms are becoming more common and persistent around the world. When in bloom, various cyanobacterial strains can produce anatoxins in high concentrations, which, unlike other cyanobacterial toxins, may be present in clear water. Potential human and animal exposures to anatoxins occur mainly through unintentional ingestion of contaminated algal mats and water. To address this public health threat, we developed and validated an LC-MS/MS method to detect anatoxins in human urine to confirm exposures. Pooled urine was fortified with anatoxin-a and dihydroanatoxin at concentrations from 10.0 to 500 ng/mL to create calibrators and quality control samples. Samples were diluted with isotopically labeled anatoxin and solvent prior to LC-MS/MS analysis. This method can accurately quantitate anatoxin-a with inter- and intraday accuracies ranging from 98.5 to 103% and relative standard deviations < 15%, which is within analytical guidelines for mass spectrometry methods. Additionally, this method qualitatively detects a common degradation product of anatoxin, dihydroanatoxin, above 10 ng/mL. We also evaluated a commercial anatoxin-a ELISA kit for potential diagnostic use; however, numerous false positives were detected from unexposed individual human urine samples. In conclusion, we have developed a method to detect anatoxins precisely and accurately in urine samples, addressing a public health area of concern, which can be applied to future exposure events.

urine↗
DOE OSTI2021

Correlations Between Panoramic Imagery and Gamma-Ray Background in an Urban Area

When searching for radiological sources in an urban area, a vehicle-borne detector system will often measure complex, varying backgrounds primarily from natural gamma-ray sources. Much work has been focused on developing spectral algorithms that retain sensitivity and minimize the false-positive rate even in the presence of such spectral and temporal variability. However, information about the environment surrounding the detector system might also provide useful clues about the expected background, which if incorporated into an algorithm, could improve performance. Recent work has focused on extensive measuring and modeling of urban areas with the goal of understanding how these complex backgrounds arise. This work presents an analysis of panoramic video images and gamma-ray background data collected in Oakland, California, by the radiological multisensor analysis platform (RadMAP) vehicle. Features were extracted from the panoramic images by semantically labeling the images and then convolving the labeled regions with the detector response. A linear model was used to relate the image-derived features to gamma-ray spectral features obtained using nonnegative matrix factorization (NMF) under different regularizations. Here we find some gamma-ray background features correlate strongly with image-derived features that measure the response-adjusted solid angle subtended by sky and buildings, and we discuss the implications for the development of future, contextually aware detection algorithms.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Identifying preferential flow from soil moisture time series: Review of methodologies

Abstract Identifying and quantifying preferential flow (PF) through soil—the rapid movement of water through spatially distinct pathways in the subsurface—is vital to understanding how the hydrologic cycle responds to climate, land cover, and anthropogenic changes. In recent decades, methods have been developed that use measured soil moisture time series to identify PF. Because they allow for continuous monitoring and are relatively easy to implement, these methods have become an important tool for recognizing when, where, and under what conditions PF occurs. The methods seek to identify a pattern or quantification that indicates the occurrence of PF. Most commonly, the chosen signature is either (1) a nonsequential response to infiltrated water, in which soil moisture responses do not occur in order of shallowest to deepest, or (2) a velocity criterion, in which newly infiltrated water is detected at depth earlier than is possible by nonpreferential flow processes. Alternative signatures have also been developed that have certain advantages but are less commonly utilized. Choosing among these possible signatures requires attention to their pertinent characteristics, including susceptibility to errors, possible bias toward false negatives or false positives, reliance on subjective judgments, and possible requirements for additional types of data. We review 77 studies that have applied such methods to highlight important information for readers who want to identify PF from soil moisture data and to inform those who aim to develop new methods or improve existing ones. Core Ideas Soil moisture data can be used to identify the occurrence of preferential flow (PF) and its initiating conditions. Various data‐analysis methods to identify PF differ in susceptibility to error, bias, and subjectivity. These methods can utilize vast amounts of data from soil moisture monitoring networks to develop understanding of when, where, and under what conditions PF occurs. Newly developed methods may lead to better accuracy and reliability, and reduce the need for subjective judgments. Plain Language Summary Preferential flow through soil occurs when a large amount of water is suddenly available, as during an intense storm. This type of flow moves rapidly through the soil in distinct narrow pathways rather than moving evenly throughout the body of soil, with major consequences for groundwater resources, ecosystems, spreading of contaminants, and other vital concerns. Methods of detecting preferential flow have been developed that utilize measurements of soil water content made by sensors installed at various depths. This measurement technology has been widely implemented, many locations now having datasets years in length, and various methods have been developed for using these to identify preferential flow. The various methods are based on different features in the soil moisture records and vary in their advantages and shortcomings. In this review, we explain and evaluate these methods, highlighting important information for their implementation to identify preferential flow from soil moisture data and for efforts to develop new methods or improve existing ones.

Nimmo, John R↗