Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “false positive”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

Random forest models accurately classify synthetic opioids using high-dimensionality mass spectrometry datasets

Detection of novel threat agents presents several challenges, a principle one being the development of untargeted methods to screen an increasing number of threat chemicals whose exact structures are unknown. With the use of Machine Learning (ML) tools, we can guide the development of analytical methods for broad-spectrum detection of unbounded threat chemical families in complex mixtures. Toward this goal, we used nominal mass and high-resolution mass spectrometry data for hundreds of synthetic opioids and non-opioid compounds. We tested two ML techniques, logistic regression and random forest, to develop models towards a practical, implementable method for opioid detection. We found that of these tested ML methods, random forest models resulted in the highest validation accuracy (95+%) for both nominal mass and high-resolution classification of opioids versus non-opioids, with low false positive and false negative rates. The RF models were then used to successfully predict the classification of 10 compounds—five opioids and five non-opioids not part of the training and validation analysis. This application of ML is a critical step towards the development of field-deployable nominal mass spectrometers with ML-driven analyses for classification of emergent threats.

Chemistry↗

Numerical characterization of support recovery in sparse regression with correlated design

Sparse regression is employed in diverse scientific settings as a feature selection method. A pervasive aspect of scientific data is the presence of correlations between predictive features. These correlations hamper both feature selection and estimation and jeopardize conclusions drawn from estimated models. On the other hand, theoretical results on sparsity-inducing regularized regression have largely addressed conditions for selection consistency via asymptotics, and disregard the problem of model selection, whereby regularization parameters are chosen. In this numerical study, we address these issues through exhaustive characterization of the performance of several regression estimators, coupled with a range of model selection strategies. These estimators and selection criteria were examined across correlated regression problems with varying degrees of signal to noise, distributions of non-zero model coefficients, and model sparsity. Our results reveal a fundamental tradeoff between false positive and false negative control in all regression estimators and model selection criteria examined. Additionally, we numerically explore a transition point modulated by the signal-to-noise ratio and spectral properties of the design covariance matrix at which the selection accuracy of all considered algorithms degrades. Overall, we find that SCAD coupled with BIC or empirical Bayes model selection performs the best feature selection across the regression problems considered.

97 MATHEMATICS AND COMPUTING↗

The essential character of the neutrino mechanism of core-collapse supernova explosions

Abstract Calibrating with detailed 2D core-collapse supernova (CCSN) simulations, we derive a simple CCSN explosion condition based solely upon the terminal density profiles of state-of-the-art stellar evolution calculations of the progenitor massive stars. This condition captures the vast majority of the behaviour of the one hundred 2D state-of-the-art models we performed to gauge its usefulness. The goal is to predict, without resort to detailed simulation, the explodability of a given massive star. We find that the simple maximum fractional ram pressure jump discriminant we define works well ∼90 per cent of the time and we speculate on the origin of the few false positives and false negatives we witness. The maximum ram pressure jump generally occurs at the time of accretion of the silicon/oxygen interface, but not always. Our results depend upon the fidelity with which the current implementation of our code F ornax adheres to Nature and issues concerning the neutrino–matter interaction, the nuclear equation of state, the possible effects of neutrino oscillations, grid resolution, the possible role of rotation and magnetic fields, and the accuracy of the numerical algorithms employed remain to be resolved. Nevertheless, the explodability condition we obtain is simple to implement, shows promise that it might be further generalized while still employing data from only the unstable Chandrasekhar progenitors, and is a more credible and robust simple explosion predictor than can currently be found in the literature.

Astronomy & Astrophysics↗

Visual attention based detection of signs of anthropogenic activities in satellite imagery

With increasing deployment of satellite imaging systems, only a small fraction of collected data can be subject to expert scrutiny. We present and evaluate a two-tier approach to broad area search for signs of anthropogenic activities in high-resolution commercial satellite imagery. The method filters image information using semantically oriented interest points by combining Harris corner detection and spatial pyramid matching. The idea is that anthropogenic structures, such as rooftop outlines, fence corners, road junctions, are locally arranged in specific angular relations to each other. They are often oriented at approximately right angles to each other (which is known as rectilinearity relation). Detecting the rectilinearity provides an opportunity to highlight regions most likely to contain anthropogenic activity. This is followed by supervised classification of regions surrounding the detected corner points as man-made vs. natural scenes. We consider, in particular, a search for anthropogenic activities in uncluttered areas. In this paper, we proposed and evaluated a two-tier approach to broad area search for signs of anthropogenic activities. Results from experiments on high-resolution ({approx}0.6m) commercial satellite image data showed the potential applicability of this approach and its ability of achieving both high precision and recall rates. The main advantage of combining corner-based cueing with general object recognition is that the incorporation of domain specific knowledge even in its more general form, such as presence of comers, provides a useful cue to narrow the focus of search for signs of anthropogenic activities. Combination of comer based cueing with spatial pyramid matching addressed the issue of comer categorization. An important practical issue for further research is optimizing the balance between false positive and false negative rates. While the results presented in the paper are encouraging, the problem of an automated broad area search for signs of anthropogenic activities remains challenging.

47 OTHER INSTRUMENTATION↗

Leveraging Fine-Grained Occupancy Estimation Patterns for Effective HVAC Control

As occupancy sensing technologies become mature, various occupancy sensors are increasingly deployed in commercial buildings for pervasive occupancy monitoring. These sensors provide occupant-count data, which contains rich spatiotemporal information about occupancy patterns. With long-term occupant-count data collected from a commercial building, we design three different predictive models that capture the occupancy dynamics and examine how a model predictive control of the HVAC system benefits from actual occupancy count prediction. Our analysis reveals that mispredictions of occupancy states, especially false positives and false negatives, may introduce inefficient control that leads to energy waste or user discomfort. To address this issue, we take a step further to design an adaptive model predictive controller that minimizes inefficient control actions according to misprediction types and distributions. A comprehensive evaluation is performed in OpenBuild and EnergyPlus simulators to study the effectiveness of the proposed end-to-end control strategy. The evaluation shows that the proposed solution reduces energy consumption by 29.5% while improving the average weighted occupants comfort by 86.7% in Predicted Mean Vote (PMV) over the fixed schedule strategy.

97 MATHEMATICS AND COMPUTING↗

A Decision Theoretic Approach to Evaluate Radiation Detection Algorithms

There are a variety of sensor systems deployed at U.S. border crossings and ports of entry that scan for illicit nuclear material. In this work, we develop a framework for comparing the performance of detection algorithms that interpret the output of these scans and determine when secondary screening is needed. We optimize each algorithm to minimize its risk, or expected loss. We measure an algorithm’s risk by considering its performance over a sample, the probability distribution of threat sources, and the consequence of detection errors. While it is common to optimize algorithms by fixing one error rate and minimizing another, our framework allows one to simultaneously consider multiple types of detection errors. Our framework is flexible and easily adapted to many different assumptions regarding the probability of a vehicle containing illicit material, and the relative consequences of a false positive and false negative errors. Our methods can therefore inform decision makers of the algorithm family and parameter values which best reduce the threat from illicit nuclear material, given their understanding of the environment at any point in time. To illustrate the applicability of our methods, in this paper, we compare the risk from two families of detection algorithms and discuss the policy implications of our results.

Nobles, Mallory A.↗

A Data-driven Approach for Predicting Nepheline Crystallization in High-Level Waste Glasses

High-level waste (HLW) glasses with high aluminum content are prone to nepheline crystallization during the slow canister cooling that is experienced during large-scale production. Due to its detrimental effects on glass durability, nepheline precipitation must be avoided; however, developing robust, predictive models for nepheline crystallization behavior in HLW glasses is difficult due to their compositional complexity. Using overly conservative constraints to predict nepheline formation can limit the waste loading to lower than the achievable capacity. In this study, a robust data-driven model using five compositional features has been developed to predict nepheline formation. The analysis of the model and the data show that there is an overlap, instead of a distinct compositional boundary, between glasses that form and do not form nepheline. As a result, the model’s predictive accuracy is not the same throughout the feature space and instead is dependent on the location of the glass composition in the dimensionally reduced feature space.

Sargin, Irmak↗

Data Analytics Applied to Coal Fired Boilers for Detecting Leaks

Data analytics were used to detect boiler leaks from five different coal-fired boilers including both subcritical and supercritical systems. Discriminant functions were developed that detected leaks up to two weeks prior to forced plant shutdowns for repairs. The leaks were identified to occur at different sections of the boiler for each plant, including waterwalls, economizer and superheater using conventional process measurement data. Leaking conditions were detected with a high degree of confidence (≪ 1% misclassified observations) and were able to distinguish normal operations from those time periods with steam leaks even while operating the power plants in power cycling mode.Multivariable statistical analyses, including Principal Component (PCA), cluster, and Fischer Discriminant Analysis (FDA) were used to characterize the leak occurrence. Normal and operational states with steam leaks were provided in the original process datasets. These datasets were split into two different groups for training and validation purposes. The data were sorted chronologically, and every third observation was assigned to training the Discriminant Function Model (DFM) while the rest were reserved for validation. PCA was used to reduce dimensionality of the original datasets. Canonical and FDA analyses were used to investigate the relationship between process variables. The outcome of the analyses revealed that nearly 35,000 observations were classified correctly; less than 0.05% of total observations were misclassified to be leaking, i.e. both false positives and false negatives.

Indrawan, Natarianto↗

A scaling transformation for classifier output based on likelihood ratio: Applications to a CAD workstation for diagnosis of breast cancer

Purpose: The authors developed scaling methods that monotonically transform the output of one classifier to the ''scale'' of another. Such transformations affect the distribution of classifier output while leaving the ROC curve unchanged. In particular, they investigated transformations between radiologists and computer classifiers, with the goal of addressing the problem of comparing and interpreting case-specific values of output from two classifiers. Methods: Using both simulated and radiologists' rating data of breast imaging cases, the authors investigated a likelihood-ratio-scaling transformation, based on ''matching'' classifier likelihood ratios. For comparison, three other scaling transformations were investigated that were based on matching classifier true positive fraction, false positive fraction, or cumulative distribution function, respectively. The authors explored modifying the computer output to reflect the scale of the radiologist, as well as modifying the radiologist's ratings to reflect the scale of the computer. They also evaluated how dataset size affects the transformations. Results: When ROC curves of two classifiers differed substantially, the four transformations were found to be quite different. The likelihood-ratio scaling transformation was found to vary widely from radiologist to radiologist. Similar results were found for the other transformations. Our simulations explored the effect of database sizes on the accuracy of the estimation of our scaling transformations. Conclusions: The likelihood-ratio-scaling transformation that the authors have developed and evaluated was shown to be capable of transforming computer and radiologist outputs to a common scale reliably, thereby allowing the comparison of the computer and radiologist outputs on the basis of a clinically relevant statistic.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

SU-D-202-02: Quantitative Imaging: Correlation Between Image Feature Analysis and the Accuracy of Manually Drawn Contours On PET Images

Purpose: To evaluate the correlation between image features and the accuracy of manually drawn target contours on synthetic PET images Methods: A digital PET phantom was used in combination with Monte Carlo simulation to create a set of 26 simulated PET images featuring a variety of tumor shapes and activity heterogeneity. These tumor volumes were used as a gold standard in comparisons with manual contours delineated by 10 radiation oncologist on the simulated PET images. Metrics used to evaluate segmentation accuracy included the dice coefficient, false positive dice, false negative dice, symmetric mean absolute surface distance, and absolute volumetric difference. Image features extracted from the simulated tumors consisted of volume, shape complexity, mean curvature, and intensity contrast along with five texture features derived from the gray-level neighborhood difference matrices including contrast, coarseness, busyness, strength, and complexity. Correlation between these features and contouring accuracy were examined. Results: Contour accuracy was reasonably well correlated with a variety of image features. Dice coefficient ranged from 0.7 to 0.90 and was correlated closely with contrast (r=0.43, p=0.02) and complexity (r=0.5, p<0.001). False negative dice ranged from 0.10 to 0.50 and was correlated closely with contrast (r=0.68, p<0.001) and complexity (r=0.66, p<0.001). Absolute volumetric difference ranged from 0.0002 to 0.67 and was correlated closely with coarseness (r=0.46, p=0.02) and complexity (r=0.49, p=0.008). Symmetric mean absolute difference ranged from 0.02 to 1 and was correlated closely with mean curvature (r=0.57, p=0.02) and contrast (r=0.6, p=0.001). Conclusion: The long term goal of this study is to assess whether contouring variability can be reduced by providing feedback to the practitioner based on image feature analysis. The results are encouraging and will be used to develop a statistical model which will enable a prediction of contour accuracy based purely on image feature analysis.

60 APPLIED LIFE SCIENCES↗

Designing Size Consistent Statistics for Accurate Anomaly Detection in Dynamic Networks

An important task in network analysis is the detection of anomalous events in a network time series. These events could merely be times of interest in the network timeline or they could be examples of malicious activity or network malfunction. Hypothesis testing using network statistics to summarize the behavior of the network provides a robust framework for the anomaly detection decision process. Unfortunately, choosing network statistics that are dependent on confounding factors like the total number of nodes or edges can lead to incorrect conclusions (e.g., false positives and false negatives). In this article, we describe the challenges that face anomaly detection in dynamic network streams regarding confounding factors. We also provide two solutions to avoiding error due to confounding factors: the first is a randomization testing method that controls for confounding factors, and the second is a set of size-consistent network statistics that avoid confounding due to the most common factors, edge count and node count.

97 MATHEMATICS AND COMPUTING↗

PECAN2

PECAN2, Pose Classification with 3D Atomic Network 2, presents an innovative approach to pose classification in computational modeling, particularly in the context of molecular docking. Traditional methods like docking, which rely on physics-based calculations, are prone to inaccuracies in predicting binding poses during experimental testing. While machine learning (ML) approaches, such as ML-driven pose classification, have been introduced to address these issues, they often depend on the availability of crystal structure data. In contrast, PECAN2 introduces a novel pose classification method that operates independently of crystal structures. Instead, it utilizes a 3D atomic neural network with Point Cloud Network (PCN) to establish correlations between docking scores and experimental data. This approach marks a departure from previous studies that heavily relied on crystal structures for labeling. The key innovation lies in the ability of PECAN2 to enhance the performance of molecular docking by filtering out false positives and false negatives through the correlation between docking scores and experimental data.

Shim, Heesung↗

Random forest models accurately classify synthetic opioids using high-dimensionality mass spectrometry datasets

Detection of novel threat agents presents several challenges, a principle one being the development of untargeted methods to screen an increasing number of threat chemicals whose exact structures are unknown. With the use of Machine Learning (ML) tools, we can guide the development of analytical methods for broad-spectrum detection of unbounded threat chemical families in complex mixtures. Toward this goal, we used nominal mass and high-resolution mass spectrometry data for hundreds of synthetic opioids and non-opioid compounds. We tested two ML techniques, logistic regression and random forest, to develop models towards a practical, implementable method for opioid detection. We found that of these tested ML methods, random forest models resulted in the highest validation accuracy (95+%) for both nominal mass and high-resolution classification of opioids versus non-opioids, with low false positive and false negative rates. The RF models were then used to successfully predict the classification of 10 compounds—five opioids and five non-opioids not part of the training and validation analysis. This application of ML is a critical step towards the development of field-deployable nominal mass spectrometers with ML-driven analyses for classification of emergent threats.

Arasteh, Kourosh [Lawrence Livermore National Labo↗

Two-level structural sparsity regularization for identifying lattices and defects in noisy images

Here, this paper presents a regularized regression model with a two-level structural sparsity penalty applied to locate individual atoms in a noisy scanning transmission electron microscopy image (STEM). In crystals, the locations of atoms is symmetric, condensed into a few lattice groups. Therefore, by identifying the underlying lattice in a given image, individual atoms can be accurately located. We propose to formulate the identification of the lattice groups as a sparse group selection problem. Furthermore, real atomic scale images contain defects and vacancies, so atomic identification based solely on a lattice group may result in false positives and false negatives. To minimize error, model includes an individual sparsity regularization in addition to the group sparsity for a within-group selection, which results in a regression model with a two-level sparsity regularization. We propose a modification of the group orthogonal matching pursuit (gOMP) algorithm with a thresholding step to solve the atom finding problem. The convergence and statistical analyses of the proposed algorithm are presented. The proposed algorithm is also evaluated through numerical experiments with simulated images. The applicability of the algorithm on determination of atom structures and identification of imaging distortions and atomic defects was demonstrated using three real STEM images. In conclusion, we believe this is an important step toward automatic phase identification and assignment with the advent of genomic databases for materials.

36 MATERIALS SCIENCE↗

DETECTING FIRE WITH MACHINE LEARNING-ENABLED VISUAL MONITORING FOR NUCLEAR POWER PLANT ENVIRONMENTS

Nuclear power plants are experiencing significant cost challenges to remain competitive with other energy-generation utilities. Unlike other industries, the cost of operation and maintenance activities is mostly attributed to workforce costs. To mitigate this, nuclear power plant stakeholders are increasingly interested in the development and deployment of machine learning methods to potentially automate or augment manually intensive tasks to reduce costs, especially for monitoring activities. One monitoring function that is visually demanding and that can occur frequently to meet the requirements of a fire protection program is visually monitoring an area for fire occurrence. Currently, fire watch activities consist of a worker physically stationed at a given location with the sole responsibility of observing a given area to ensure a fire is detected and mitigated promptly. This effort focused on the development and evaluation of a suitable deep convolutional neural network to classify individual video frames at a sub-second frequency for the occurrence of “fire” and “no fire” in varying industrial environments similar to nuclear power plants. It is believed that a trained neural network model could be integrated with existing facility video surveillance camera feeds to generate alerts when fire inferences occur in individual frames captured at sub-second temporal resolutions. Extensive effort was dedicated to identifying and curating suitable imagery training data representing varying environments and scene settings with and without flame features to maximize generalization in nuclear power plant environments. The data collection effort resulted in the aggregation of a large, labeled image library exceeding 12,000 images to support model training for diverse industrial environments. A deep neural network model incorporating parallel multi-scale capabilities was developed and trained to support accurate image-based detection of flame incidents of varying sizes and spectral feature properties within heterogeneous scenes. Analysis results show that the trained model can achieve high inference accuracy despite heterogeneous scene environments and components. Testing accuracy exceeded 95.0 percent with very low false positive and false negative inferences.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Genometa - A Fast and Accurate Classifier for Short Metagenomic Shotgun Reads

Metagenomic studies use high-throughput sequence data to investigate microbial communities in situ. However, considerable challenges remain in the analysis of these data, particularly with regard to speed and reliable analysis of microbial species as opposed to higher level taxa such as phyla. We here present Genometa, a computationally undemanding graphical user interface program that enables identification of bacterial species and gene content from datasets generated by inexpensive high-throughput short read sequencing technologies. Our approach was first verified on two simulated metagenomic short read datasets, detecting 100% and 94% of the bacterial species included with few false positives or false negatives. Subsequent comparative benchmarking analysis against three popular metagenomic algorithms on an Illumina human gut dataset revealed Genometa to attribute the most reads to bacteria at species level (i.e. including all strains of that species) and demonstrate similar or better accuracy than the other programs. Lastly, speed was demonstrated to be many times that of BLAST due to the use of modern short read aligners. Our method is highly accurate if bacteria in the sample are represented by genomes in the reference sequence but cannot find species absent from the reference. This method is one of the most user-friendly and resource efficient approaches and is thus feasible for rapidly analysing millions of short reads on a personal computer.

59 BASIC BIOLOGICAL SCIENCES↗

Uncertainty quantification in ToxCast high throughput screening

High throughput screening (HTS) projects like the U.S. Environmental Protection Agency’s ToxCast program are required to address the large and rapidly increasing number of chemicals for which we have little to no toxicity measurements. Concentration-response parameters such as potency and efficacy are extracted from HTS data using nonlinear regression, and models and analyses built from these parameters are used to predict in vivo and in vitro toxicity of thousands of chemicals. How these predictions are impacted by uncertainties that stem from parameter estimation and propagated through the models and analyses has not been well explored. While data size and complexity makes uncertainty quantification computationally expensive for HTS datasets, continued advancements in computational resources have allowed these computational challenges to be met. This study uses nonparametric bootstrap resampling to calculate uncertainties in concentration-response parameters from a variety of HTS assays. Using the ToxCast estrogen receptor model for bioactivity as a case study, we highlight how these uncertainties can be propagated through models to quantify the uncertainty in model outputs. Uncertainty quantification in model outputs is used to identify potential false positives and false negatives and to determine the distribution of model values around semi-arbitrary activity cutoffs, increasing confidence in model predictions. At the individual chemical-assay level, curves with high variability are flagged for manual inspection or retesting, focusing subject-matter-expert time on results that need further input. This work improves the confidence of predictions made using HTS data, increasing the ability to use this data in risk assessment.

59 BASIC BIOLOGICAL SCIENCES↗

Leak Detection and H2 Sensor Development for Hydrogen Applications

The objectives of this report are: (1) Develop a low cost, low power, durable, and reliable hydrogen safety sensor for a wide range of vehicle and infrastructure applications; (2) Continually advance test prototypes guided by materials selection, sensor design, electrochemical R&D investigation, fabrication, and rigorous life testing; (3) Disseminate packaged sensor prototypes and control systems to DOE Laboratories and commercial parties interested in testing and fielding advanced prototypes for cross-validation; (4) Evaluate manufacturing approaches for commercialization; and (5) Engage an industrial partner and execute technology transfer. Recent developments in the search for sustainable and renewable energy coupled with the advancements in fuel cell powered vehicles (FCVs) have augmented the demand for hydrogen safety sensors. There are several sensor technologies that have been developed to detect hydrogen, including deployed systems to detect leaks in manned space systems and hydrogen safety sensors for laboratory and industrial usage. Among the several sensing methods electrochemical devices that utilize high temperature-based ceramic electrolytes are largely unaffected by changes in humidity and are more resilient to electrode or electrolyte poisoning. The desired sensing technique should meet a detection threshold of 1% (10,000 ppm) H 2 and response time of {approx_equal}1 min, which is a target for infrastructure and vehicular uses. Further, a review of electrochemical hydrogen sensors by Korotcenkov et.al and the report by Glass et.al suggest the need for inexpensive, low power, and compact sensors with long-term stability, minimal cross-sensitivity, and fast response. This view has been largely validated and supported by the fuel cell and hydrogen infrastructure industries by the NREL/DOE Hydrogen Sensor Workshop held on June 8, 2011. Many of the issues preventing widespread adoption of best-available hydrogen sensing technologies available today outside of cost, derive from excessive false positives and false negatives arising from signal drift and unstable sensor baseline; both of these problems necessitate the need for unacceptable frequent calibration.

08 HYDROGEN↗