Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “false positive analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

DETECTING FIRE WITH MACHINE LEARNING-ENABLED VISUAL MONITORING FOR NUCLEAR POWER PLANT ENVIRONMENTS

Nuclear power plants are experiencing significant cost challenges to remain competitive with other energy-generation utilities. Unlike other industries, the cost of operation and maintenance activities is mostly attributed to workforce costs. To mitigate this, nuclear power plant stakeholders are increasingly interested in the development and deployment of machine learning methods to potentially automate or augment manually intensive tasks to reduce costs, especially for monitoring activities. One monitoring function that is visually demanding and that can occur frequently to meet the requirements of a fire protection program is visually monitoring an area for fire occurrence. Currently, fire watch activities consist of a worker physically stationed at a given location with the sole responsibility of observing a given area to ensure a fire is detected and mitigated promptly. This effort focused on the development and evaluation of a suitable deep convolutional neural network to classify individual video frames at a sub-second frequency for the occurrence of “fire” and “no fire” in varying industrial environments similar to nuclear power plants. It is believed that a trained neural network model could be integrated with existing facility video surveillance camera feeds to generate alerts when fire inferences occur in individual frames captured at sub-second temporal resolutions. Extensive effort was dedicated to identifying and curating suitable imagery training data representing varying environments and scene settings with and without flame features to maximize generalization in nuclear power plant environments. The data collection effort resulted in the aggregation of a large, labeled image library exceeding 12,000 images to support model training for diverse industrial environments. A deep neural network model incorporating parallel multi-scale capabilities was developed and trained to support accurate image-based detection of flame incidents of varying sizes and spectral feature properties within heterogeneous scenes. Analysis results show that the trained model can achieve high inference accuracy despite heterogeneous scene environments and components. Testing accuracy exceeded 95.0 percent with very low false positive and false negative inferences.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2020

Uranium Mass Attribute System (UMAS) Study: Evaluation of Options against the Arms Control Evaluation Criteria

In this FY19 study, a range of possible measurement systems for confirming a U-235 mass attribute were identified. The systems were reviewed for their applicability in a potential future arms control agreement, and a subset of the eight most plausible systems were selected for further evaluation against the Arms Control Evaluation Criteria (ACEC). The evaluation results were then used to identify a primary and secondary recommended measurement system for confirming a U-235 mass attribute on warheads, nuclear components of warheads, and/or materials as part of a hypothetical arms control agreement.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗
NASA NTRS2020

The NANOGrav 11 yr data set: Evolution of Gravitational-wave Background Statistics

An ensemble of inspiraling supermassive black hole binaries should produce a stochastic background of very low frequency gravitational waves. This stochastic background is predicted to be a power law, with a gravitational-wave strain spectral index of −2/3, and it should be detectable by a network of precisely timed millisecond pulsars, widely distributed on the sky. This paper reports a new “time slicing” analysis of the 11 yr data release from the North American Nanohertz Observatory for Gravitational Waves (NANOGrav) using 34 millisecond pulsars. Methods to flag potential “false-positive” signatures are developed, including techniques to identify responsible pulsars. Mitigation strategies are then presented. We demonstrate how an incorrect noise model can lead to spurious signals, and we show how independently modeling noise across 30 Fourier components, spanning NANOGrav’s frequency range, effectively diagnoses and absorbs the excess power in gravitational-wave searches. This results in a nominal, and expected, progression of our gravitational-wave statistics. Additionally, we show that the first interstellar medium event in PSR J1713+0747 pollutes the common red-noise process with low spectral index noise, and we use a tailored noise model to remove these effects.

J S Hazboun↗
DOE OSTI2022

Condition monitoring and anomaly detection in cyber-physical systems

The modern industrial environment is equipping myriads of smart manufacturing machines where the state of each device can be monitored continuously. Such monitoring can help identify possible future failures and develop a cost-effective maintenance plan. However, it is a daunting task to perform early detection with low false positives and negatives from the huge volume of collected data. This requires developing a holistic machine learning framework to address the issues in condition monitoring of high priority components and develop efficient techniques to detect anomalies that can detect and possibly localize the faulty components. This paper presents a comparative analysis of recent machine learning approaches for robust, cost-effective anomaly detection in cyber-physical systems. While detection has been extensively studied, very few researchers have analyzed the localization of the anomalies. We show that supervised learning outperforms unsupervised algorithms. For supervised cases, we achieve near-perfect accuracy of 98% (specifically for tree-based algorithms). In contrast, the best-case accuracy in the unsupervised cases was 63%—the area under the receiver operating characteristic curve (AUC) exhibits similar outcomes as an additional metric.

Marfo, William↗
DOE OSTI2021

Mapathons versus automated feature extraction: a comparative analysis for strengthening immunization microplanning

Background: Social instability and logistical factors like the displacement of vulnerable populations, the difficulty of accessing these populations, and the lack of geographic information for hard-to-reach areas continue to serve as barriers to global essential immunizations (EI). Microplanning, a population-based, healthcare intervention planning method has begun to leverage geographic information system (GIS) technology and geospatial methods to improve the remote identification and mapping of vulnerable populations to ensure inclusion in outreach and immunization services, when feasible. We compare two methods of accomplishing a remote inventory of building locations to assess their accuracy and similarity to currently employed microplan line-lists in the study area. Methods: The outputs of a crowd-sourced digitization effort, or mapathon, were compared to those of a machine-learning algorithm for digitization, referred to as automatic feature extraction (AFE). The following accuracy assessments were employed to determine the performance of each feature generation method: (1) an agreement analysis of the two methods assessed the occurrence of matches across the two outputs, where agreements were labeled as “befriended” and disagreements as “lonely”; (2) true and false positive percentages of each method were calculated in comparison to satellite imagery; (3) counts of features generated from both the mapathon and AFE were statistically compared to the number of features listed in the microplan line-list for the study area; and (4) population estimates for both feature generation method were determined for every structure identified assuming a total of three households per compound, with each household averaging two adults and 5 children. Results: The mapathon and AFE outputs detected 92,713 and 53,150 features, respectively. A higher proportion (30%) of AFE features were befriended compared with befriended mapathon points (28%). The AFE had a higher true positive rate (90.5%) of identifying structures than the mapathon (84.5%). The difference in the average number of features identified per area between the microplan and mapathon points was larger (t = 3.56) than the microplan and AFE (t = -2.09) (alpha = 0.05). Conclusions: Our findings indicate AFE outputs had higher agreement (i.e., befriended), slightly higher likelihood of correctly identifying a structure, and were more similar to the local microplan line-lists than the mapathon outputs. These findings suggest AFE may be more accurate for identifying structures in high-resolution satellite imagery than mapathons. However, they both had their advantages and the ideal method would utilize both methods in tandem.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS2002

Advanced Ultrasonic Diagnosis of Extremity Trauma: The Faster Exam

Ultrasound is of prO)len accuracy in abdominal and thoracic trauma and may be useful to diagnose extremity injury in situations where radiography is not available such as military and space applications. We prospectively evaluated the utility of extremity , ultrasound performed by trained, non-physician personnel in patients with extremity trauma, to simulate remote aerospace or military applications . Methods: Patients with extremity trauma were identified by history, physical examination, and radiographic studies. Ultrasound examination was performed bilaterally by nonphysician personnel with a portable ultrasound device using a 10-5 MHz linear probe, Images were video-recorded for later analysis against radiography by Fisher's exact test. The average time of examination was 4 minutes. Ultrasound accurately diagnosed extremity, injury in 94% of patients with no false positive exams; accuracy was greater in mid-shaft locations and least in the metacarpa/metatarsals. Soft tissue/tendon injury was readily visualized . Extremity ultrasound can be performed quickly and accurately by nonphysician personnel with excellent accuracy. Blinded verification of the utility of ultrasound in patients with extremity injury should be done to determine if Extremity and Respiratory evaluation should be added to the FAST examination (the FASTER exam) and verify the technique in remote locations such as military and aerospace applications.

Dulchavsky, S. A.↗
NASA NTRS2019

Satellite Conjunction Assessment Risk Analysis for "Dilution Region" Events: Issues and Operational Approaches

An important activity within Space Traffic Management is the detection and prevention of possible on-orbit collisions between space objects. The principal parameter for assessing collision likelihood is the probability of collision, which is widely accepted among conjunction assessment practitioners; but it possesses a known deficiency in that it can produce a false sense of safety when the orbital position uncertainties for the conjuncting objects are high. The probability of collision is said to be “diluted” in such a situation and to understate the possible risk; certain approaches have been recommended by researchers to provide (largely conservative) risk estimates and remediation methodologies in these cases. The present analysis explores two of the main proposals for quantifying and remediating possible risk in the dilution region and quantifies their operational implications. These implications with regard to imputed additional workload are considerable, especially in anticipating the conjunction event levels expected with the deployment of the USAF Space Fence radar. This effort has been undertaken as part of a larger enterprise that seeks to clarify the philosophical and statistical underpinnings of the conjunction risk assessment process. The analysis presented herein argues that a form of hypothesis testing is implicitly used in conjunction assessment risk analysis, and that there are a number of conceptual and practical reasons for constructing the associated null hypothesis to counsel against a satellite conjunction remediation action. In short, it is concluded that, for the purposes of determining whether a conjunction remediation action should be pursued, dilution-region probabilities of collision should be treated no differently from those produced under other circumstances.

Probability of collision↗
NASA NTRS2019

Machine Learning-Based Atmospheric Phenomena Detection Platform

As the number of Earth pointing satellites has increased over the last several decades, the data volume retrieved from instruments onboard these satellites has also increased. It is expected that this trend will continue as more data intensive missions and small satellite constellations are launched. Currently, feature detection - namely atmospheric phenomena - in these datasets is performed manually and is thus not scalable with the growing data archives. Recent advancements in computational efficiency allow for the Earth science community to leverage machine learning to identify interesting atmospheric phenomena. Given the wide range of distinctive features in various atmospheric phenomena, a specialized machine learning model is required for accurate detection of these phenomena independently. The Phenomena Portal, developed at NASA IMPACT, is designed to provide visualization for the output from these machine learning models. In addition, detected events for each atmospheric phenomena are stored in a database that can be used to more easily use/subset larger spatiotemporal datasets. The user interface also incorporates additional features to enhance the user experience including spatiotemporal analysis, multiple base layer images, and a slider to filter events with lower probabilities of positive detection. Each detection supports user feedback on whether the detection is true or false that can then be stored and used to improve the machine learning model performance.

Gurung, Iksha↗
DOE OSTI2026

lumicap v0.1

Automated HDR luminance imaging system designed for daylighting research and building science. It controls a fisheye-lens camera to capture time-lapse bracket sequences, merges them into calibrated HDR images, and runs a full post-processing pipeline — all unattended. Features: - Scheduled LDR bracket capture via gphoto2 - HDR merging with vignetting, ND filter, and fisheye projection corrections - Illuminance and luminance meter integration (Konica Minolta T-10A, LS-100/150) - Daylight glare probability (DGP) and solar position computation - Automated false-color rendering, JPEG thumbnails, and daily time-lapse video - CSV data logging per timestep Uses: - Long-term monitoring of daylight conditions in buildings - Glare analysis for occupant comfort research - Solar irradiance and sky luminance studies Advantages: - End-to-end automation — capture, calibration, analysis, and archiving run without manual intervention - Built on the proven Radiance toolchain, ensuring photometrically accurate HDR output - Hardware-agnostic meter support via serial auto-detection - Lightweight — no GUI overhead, deployable on a headless Raspberry Pi or similar embedded system

Wang, Taoning [Lawrence Berkeley National Laborato↗
DOE OSTI2022

Uncovering novel mutational signatures by de novo extraction with SigProfilerExtractor

Mutational signature analysis is commonly performed in cancer genomic studies. Here, we present SigProfilerExtractor, an automated tool for de novo extraction of mutational signatures, and benchmark it against another 13 bioinformatics tools by using 34 scenarios encompassing 2,500 simulated signatures found in 60,000 synthetic genomes and 20,000 synthetic exomes. For simulations with 5% noise, reflecting high-quality datasets, SigProfilerExtractor outperforms other approaches by elucidating between 20% and 50% more true-positive signatures while yielding 5-fold less false-positive signatures. Applying SigProfilerExtractor to 4,643 whole-genome- and 19,184 whole-exome-sequenced cancers reveals four novel signatures. Two of the signatures are confirmed in independent cohorts, and one of these signatures is associated with tobacco smoking. In summary, this report provides a reference tool for analysis of mutational signatures, a comprehensive benchmarking of bioinformatics tools for extracting signatures, and several novel mutational signatures, including one putatively attributed to direct tobacco smoking mutagenesis in bladder tissues.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

From Data to Insights: A Covariate Analysis of the IARPA BRIAR Dataset for Multimodal Biometric Recognition Algorithms at Altitude and Range

This paper examines covariate effects on fused whole body biometrics performance in the IARPA BRIAR dataset, specifically focusing on UAV platforms, elevated positions, and distances up to 1000 meters. The dataset includes outdoor videos compared with indoor images and controlled gait recordings. Normalized raw fusion scores relate directly to predicted false accept rates (FAR), offering an intuitive means for interpreting model results. A linear model is developed to predict biometric algorithm scores, analyzing their performance to identify the most influential covariates on accuracy at altitude and range. Weather factors like temperature, wind speed, solar loading, and turbulence are also investigated in this analysis. The study found that resolution and camera distance best predicted accuracy and findings can guide future research and development efforts in long-range/elevated/UAV biometrics and support the creation of more reliable and robust systems for national security and other critical domains.

Bolme, David↗
DOE OSTI2023

Denial of Service Attack Detection via Differential Analysis of Generalized Entropy Progressions

Denial-of-Service (DoS) attacks are one the most common and consequential cyber attacks in computer networks. While existing research offers a plethora of detection methods, the issue of achieving scalability, a low false positive rate, and high detection accuracy remains open. In this work, we address this problem by developing a differential method based on generalized entropy progression. In this method, named as DoDGE, we continuously fit the line of best fit to the entropy progression of destination addresses and check if the derivative, that is, the slope of this line is less than the negative of the dynamically computed standard deviation of the derivatives. Furthermore, to distinguish from flash events, we leverage the symmetry that when a flash event occurs, the derivative of the entropy progression of source addresses is positive. With this design, we omit the usage of the thresholds and the results with five real-world network traffic datasets confirm that DoDGE outperforms threshold-based DoS attack detection by two orders of magnitude in terms of false positives on average. When compared to ten machine learning (ML) models, DoDGE achieves a balanced accuracy of 99%, while the average balanced accuracy for the ML models is 52%. Moreover, the results show that DoDGE successfully differentiates between a flash event and a DoS attack. Furthermore, since the main computation cost of DoDGE is the entropy computation, which is linear in the volume of the unit-time network flow, uses integer only operations, and works on a small fraction of the total flow, it is lightweight and scalable.

Cybersecurity, wireless communication↗
DOE OSTI2026

Random forest models accurately classify synthetic opioids using high-dimensionality mass spectrometry datasets

Detection of novel threat agents presents several challenges, a principle one being the development of untargeted methods to screen an increasing number of threat chemicals whose exact structures are unknown. With the use of Machine Learning (ML) tools, we can guide the development of analytical methods for broad-spectrum detection of unbounded threat chemical families in complex mixtures. Toward this goal, we used nominal mass and high-resolution mass spectrometry data for hundreds of synthetic opioids and non-opioid compounds. We tested two ML techniques, logistic regression and random forest, to develop models towards a practical, implementable method for opioid detection. We found that of these tested ML methods, random forest models resulted in the highest validation accuracy (95+%) for both nominal mass and high-resolution classification of opioids versus non-opioids, with low false positive and false negative rates. The RF models were then used to successfully predict the classification of 10 compounds—five opioids and five non-opioids not part of the training and validation analysis. This application of ML is a critical step towards the development of field-deployable nominal mass spectrometers with ML-driven analyses for classification of emergent threats.

Chemistry↗
DOE OSTI2024

Cy-Phy ADS: Cyber Physical Anomaly Detection Framework for EV Charging Systems

Today’s large-scale Electric Vehicle (EV) infrastructures are heavily dependent on information communication technologies to maintain their operation and to support communication within sub-system components as well as the outside world. These technologies are vulnerable to various cyber and physical threats. Timely identification and mitigation of these threats are critical for improving human safety, avoiding economic losses, and preventing catastrophic system failures. By addressing this, our work presents a ResNet Autoencoder (AE) based Cyber-Physical Anomaly Detection System (Cy-Phy ADS) for detecting anomalies in EV Controller Area Network (CAN) protocol communication. It consists of four main components: Cyber-Physical Feature Extractor, ResNet AE-based Anomaly Detection Framework, Cyber-Physical Health Metric (CPHM), and Visualization Dashboard. The presented framework was trained and tested using CAN data collected from the EV charging system testbed at the Idaho National Laboratory. The presented Cy-Phy ADS compared against six widely used unsupervised anomaly detection algorithms: One Class Support Vector Machine (OCSVM), Variational Autoencoder (VAE), LSTM Autoencoder (LSTM AE), Isolation Forest (IForest), Principle Component Analysis (PCA) and Local Outlier Factor (LOF). Here the presented approach showed the highest accuracy among the compared methods. Further, the proposed approach showed comparable performance in terms of precision, F1, and False positive rate. It also showed the lowest training and inference time compared to the neural network-based baseline algorithms compared against with. Additionally, the Cy-Phy ADS has advantages such as unsupervised training, the ability to provide a holistic metric for system health characterization, and non-linear feature extraction.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2025

Random forest models accurately classify synthetic opioids using high-dimensionality mass spectrometry datasets

Detection of novel threat agents presents several challenges, a principle one being the development of untargeted methods to screen an increasing number of threat chemicals whose exact structures are unknown. With the use of Machine Learning (ML) tools, we can guide the development of analytical methods for broad-spectrum detection of unbounded threat chemical families in complex mixtures. Toward this goal, we used nominal mass and high-resolution mass spectrometry data for hundreds of synthetic opioids and non-opioid compounds. We tested two ML techniques, logistic regression and random forest, to develop models towards a practical, implementable method for opioid detection. We found that of these tested ML methods, random forest models resulted in the highest validation accuracy (95+%) for both nominal mass and high-resolution classification of opioids versus non-opioids, with low false positive and false negative rates. The RF models were then used to successfully predict the classification of 10 compounds—five opioids and five non-opioids not part of the training and validation analysis. This application of ML is a critical step towards the development of field-deployable nominal mass spectrometers with ML-driven analyses for classification of emergent threats.

Arasteh, Kourosh [Lawrence Livermore National Labo↗
DOE OSTI2026

Improving Building Footprint Extraction Using NAIP and 3DEP Lidar Derived Features with Deep Learning

Accurate building footprint extraction is critical for applications ranging from population estimation to disaster management. Although optical imagery provides detailed spectral information, it often struggles with shadows, occlusions, and background clutter in dense urban environments. Lidar data, by contrast, offer precise elevation and structural attributes but face challenges such as variable point density and noise. This study integrates multispectral imagery from the U.S. Department of Agriculture (USDA) National Agriculture Imagery Program (NAIP) with lidar-derived feature height and intensity from the U.S. Geological Survey (USGS) 3D Elevation Program (3DEP) to improve footprint extraction using a U-Net–based deep learning model. A six-band input stack (RGB, near-infrared, height, intensity) was developed, normalized, and tiled for training and evaluation against Microsoft Global Building Footprints (GBF). Results from the Houston, TX test site show that the six-band model achieved a precision of 0.86, recall of 0.88, F1 score of 0.87, and Intersection-over-Union (IoU) of 0.76, consistently outperforming four-band baselines by reducing false positives while maintaining sensitivity. Predictions on withheld Houston tiles confirmed strong within-region generalization, yielded a precision of 0.78, recall of 0.81, F1 score of 0.79, and IoU of 0.66. Qualitative analysis further revealed limitations stemming from both training label quality and vegetation–building confusion. These findings demonstrate the complementary value of integrating spectral and structural information for robust building footprint extraction and how domain adaptation strategies can be used to enhance cross-regional transferability.

Liu, Jung Kuan [United States Geological Survey (U↗
NASA NTRS2019

String Midpoint Monitoring for Li-Ion Batteries

The paper introduces the midpoint monitoring method for detecting cell voltage imbalance in a series string of 8 electrochemical cells. Monte Carlo analysis is performed for normal cell End of Discharge open and closed circuit voltage behavior to assess the extent to which derived guard band voltage limits will detect over discharged cells, provide false positive indication, or fail to detect over discharged cells. Forward work is proposed to refine the battery monitoring method for operational use in 28V lithium ion batteries.

Russell, S.↗
NASA NTRS1981

The effect of abnormal cell proportion on specimen classifier performance

An analysis is presented of the results obtained from a cell classifier which is confronted with an abnormal/normal cell ratio which is different from the ratio assumed in the calibration of the classifier. False negative and false positive error rates are determined in advance for classifier operation, along with the necessary sample size in order to validate the predicted distributions. Changes are demonstrated to happen only regarding the false negative rate, where reductions in the abnormal cell rate below the expected rates would cause totally unreliable data. Substantial overproduction of abnormal cells would be quickly noticeable, while production rates beyond, but close to, the expected rates would only require more extensive sampling. Classifier systems for 10% proportions of abnormal cells are concluded to be possible, but difficulties are present with much lower rates

Castleman, K. R.↗