Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “misclassification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2017

PREPARING FOR ADVANCED LIGO: A STAR–GALAXY SEPARATION CATALOG FOR THE PALOMAR TRANSIENT FACTORY

The search for fast optical transients, such as the expected electromagnetic counterparts to binary neutron star mergers, is riddled with false positives (FPs) ranging from asteroids to stellar flares. While moving objects are readily rejected via image pairs separated by ∼1 hr, stellar flares represent a challenging foreground, significantly outnumbering rapidly evolving explosions. Identifying stellar sources close to and fainter than the transient detection limit can eliminate these FPs. Here, we present a method to reliably identify stars in deep co-adds of Palomar Transient Factory (PTF) imaging. Our machine-learning methodology utilizes the random forest (RF) algorithm, which is trained using >3×10{sup 6} sources with Sloan Digital Sky Survey (SDSS) spectra. When evaluated on an independent test set, the PTF RF model outperforms the SExtractor star classifier by ∼4%. For faint sources (r{sup ′}⩾21 mag), which dominate the field population, the PTF RF model produces a ∼19% improvement over SExtractor. To avoid false negatives in the PTF transient-candidate stream, we adopt a conservative stellar classification threshold, corresponding to a galaxy misclassification rate of 0.005. Ultimately, ∼1.70×10{sup 8} objects are included in our PTF point-source catalog, of which only ∼10{sup 6} are expected to be galaxies. We demonstrate that the PTF RF catalog reveals transients that otherwise would have been missed. To leverage its superior image quality, we additionally create an SDSS point-source catalog, which is also tuned to have a galaxy misclassification rate of 0.005. These catalogs have been incorporated into the PTF real-time pipelines to automatically reject stellar sources as non-extragalactic transients.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2018

(GO) 2 -SIM: a GCM-oriented ground-observation forward-simulator framework for objective evaluation of cloud and precipitation phase

General circulation model (GCM) evaluation using ground-based observations is complicated by inconsistencies in hydrometeor and phase definitions. Here we describe (GO) 2 -SIM, a forward simulator designed for objective hydrometeor-phase evaluation, and assess its performance over the North Slope of Alaska using a 1-year GCM simulation. For uncertainty assessment, 18 empirical relationships are used to convert model grid-average hydrometeor (liquid and ice, cloud, and precipitation) water contents to zenith polarimetric micropulse lidar and Ka-band Doppler radar measurements, producing an ensemble of 576 forward-simulation realizations. Sensor limitations are represented in forward space to objectively remove from consideration model grid cells with undetectable hydrometeor mixing ratios, some of which may correspond to numerical noise. Phase classification in forward space is complicated by the inability of sensors to measure ice and liquid signals distinctly. However, signatures exist in lidar–radar space such that thresholds on observables can be objectively estimated and related to hydrometeor phase. The proposed phase-classification technique leads to misclassification in fewer than 8 % of hydrometeor-containing grid cells. Such misclassifications arise because, while the radar is capable of detecting mixed-phase conditions, it can mistake water- for ice-dominated layers. However, applying the same classification algorithm to forward-simulated and observed fields should generate hydrometeor-phase statistics with similar uncertainty. Alternatively, choosing to disregard how sensors define hydrometeor phase leads to frequency of occurrence discrepancies of up to 40 %. So, while hydrometeor-phase maps determined in forward space are very different from model reality they capture the information sensors can provide and thereby enable objective model evaluation.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2018

Projecting Trouble: Light Based Adversarial Attacks on Deep Learning Classifiers

This work demonstrates a physical attack on a deep learning image classification system using projected light onto a physical scene. Prior work is dominated by techniques for creating adversarial examples which directly manipulate the digital input of the classifier. Such an attack is limited to scenarios where the adversary can directly update the inputs to the classifier such as Clarifai or Cloud Vision. Such limitations have led to a vein of research around physical attacks where objects are constructed to be inherently adversarial or adversarial modifications are added to cause misclassification. Our work differs from other physical attacks in that we can cause misclassification dynamically without altering physical objects in a permanent way. We construct a test setup which includes a light projection source, an object for classification, and a camera to capture the scene. Experiments are conducted against 2D and 3D objects. For the 2D demonstration, a CIFAR-10 image is printed and attacked with light. In this 2D presentation, the classifier confidence of the target class drops from 98% probability to 22%. The 3D presentation places a toy car in the camera frame, attacks the car with projected light, and recaptures the scene for verification of attack success. The classifier confidence the image is a car drops from 89% probability to 43%

adversarial machine learning, deep learning, adver↗
DOE OSTI2022

Assurance by Design for Cyber Physical Data-Driven Systems

Currently, Cyber Physical Data-Driven Systems (CPDDS) employ machine learning for the classification, data fusion, and control of our nation’s infrastructure, such as the power grid, transportation networks (e.g., fuel distribution, air traffic control), and DoD long-duration collaborative autonomous platforms including unmanned underwater, ground, surface, space, and aerial systems. Many CPDDSs are system-of-systems that should be designed to communicate over disadvantaged networks. It is important to assure that the CPDDSs are resilient against physical and cyber threats by design. Additionally, their design should tolerate misclassification errors resulting from natural and/or adversarial distribution shifts within their data driven components. The all-domain nature of the problem of assuring the design of CPDDSs requires a multi-disciplinary perspective as outlined in this chapter.

Chikkagoudar, Satish↗
DOE OSTI2022

Adversarial classification via distributional robustness with Wasserstein ambiguity

Abstract We study a model for adversarial classification based on distributionally robust chance constraints. We show that under Wasserstein ambiguity, the model aims to minimize the conditional value-at-risk of the distance to misclassification, and we explore links to adversarial classification models proposed earlier and to maximum-margin classifiers. We also provide a reformulation of the distributionally robust model for linear classification, and show it is equivalent to minimizing a regularized ramp loss objective. Numerical experiments show that, despite the nonconvexity of this formulation, standard descent methods appear to converge to the global minimizer for this problem. Inspired by this observation, we show that, for a certain class of distributions, the only stationary point of the regularized ramp loss minimization problem is the global minimizer.

Ho-Nguyen, Nam (ORCID:0000000344647730)↗
DOE OSTI2018

Comparison of Shallow and Deep Learning Methods on Classifying the Regional Pattern of Diffuse Lung Disease

This study aimed to compare shallow and deep learning of classifying the patterns of interstitial lung diseases (ILDs). Using high-resolution computed tomography images, two experienced radiologists marked 1200 regions of interest (ROIs), in which 600 ROIs were each acquired using a GE or Siemens scanner and each group of 600 ROIs consisted of 100 ROIs for subregions that included normal and five regional pulmonary disease patterns (ground-glass opacity, consolidation, reticular opacity, emphysema, and honeycombing). We employed the convolution neural network (CNN) with six learnable layers that consisted of four convolution layers and two fully connected layers. The classification results were compared with the results classified by a shallow learning of a support vector machine (SVM). The CNN classifier showed significantly better performance for accuracy compared with that of the SVM classifier by 6–9%. As the convolution layer increases, the classification accuracy of the CNN showed better performance from 81.27 to 95.12%. Especially in the cases showing pathological ambiguity such as between normal and emphysema cases or between honeycombing and reticular opacity cases, the increment of the convolution layer greatly drops the misclassification rate between each case. Conclusively, the CNN classifier showed significantly greater accuracy than the SVM classifier, and the results implied structural characteristics that are inherent to the specific ILD patterns.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2023

Deep reinforcement learning for class imbalance fault diagnosis of equipment in nuclear power plants

In equipment fault diagnosis in nuclear power plants, there may be far more samples in one class (e.g., a health state) than in another class (e.g., a fault state). The distribution of data in each class is highly skewed. Most machine learning algorithms are suitable for balanced training datasets. When faced with imbalanced samples, these algorithms tend to provide good identification for the majority classes and bias for the minority classes. However, the misclassification of minority classes can lead to high costs. To address the above problem, this paper develops a deep reinforcement learning-based diagnosis method that models fault diagnosis as a sequential decision-making process. At each time step, the agent receives the state of the environment represented by the training samples and then takes a diagnosis action guided by a policy. If the action is correct/incorrect, the agent receives a positive/negative reward. The reward for minority classes is higher than that for majority classes. The agent’s goal is to obtain as many cumulative rewards as possible in the process, i.e., to identify the sample as correctly as possible. Six demonstration scenarios are constructed, depending on the selected fault datasets and the designed model structures. Experiments show that the proposed method achieves a higher weighted-averaged F1 score than the classical supervised learning method in most cases of class imbalance. Finally, the proposed method has potential applications in the field of class imbalance fault diagnosis of equipment in nuclear power plants.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2019

An anomaly detection framework for dynamic systems using a Bayesian hierarchical framework

Complex systems are susceptible to many types of anomalies, faults, and abnormal behavior caused by a variety of off-nominal conditions that may ultimately result in major failures or catastrophic events. Early and accurate detection of these anomalies using system inputs and outputs collected from sensors and smart devices has become a challenging problem and an active area of research in many application domains. In this article, we present a new Bayesian hierarchical framework that is able to model the relationship between system inputs (sensor measurements) and outputs (response variables) without imposing strong distributional/parametric assumptions while using only a subset of training samples and sensor attributes. Then, an optimal cost-sensitive anomaly detection framework is proposed to determine whether a sample is an anomalous one taking into consideration the trade-off between misclassification errors and detection rates. The model can be used for both supervised and unsupervised settings depending on the availability of data regarding the behavior of the system under anomaly conditions. The unsupervised model is particularly useful when it is prohibitive to identify in advance the anomalies that a system may present and where no data are available regarding the behavior of the system under anomaly conditions. A Bayesian hierarchical setting is used to structure the proposed framework and help with accommodating uncertainty, imposing interpretability, and controlling the sparsity and complexity of the proposed anomaly detection framework. A Markov chain Monte Carlo algorithm is also developed for model training using past data. Furthermore, the numerical experiments conducted using a simulated data set and a wind turbine data set demonstrate the successful application of the proposed work for system response modeling and anomaly detection.

17 WIND ENERGY↗
DOE OSTI2020

Data analytics for leak detection in a subcritical boiler

For decades, boiler leaks have been the leading cause of forced outages in the coal-fired unit. The leak occurrences are currently escalating since the existing plants must satisfy faster-ramping rates to support grid operation. Data analytics including Principal Component Analysis, Canonical Variate, and Fisher Discriminant Analysis were combined for detecting and characterizing the leak in a commercial 650 MW subcritical coal-fired power plant. The combined approach was shown to be highly effective in the fault investigation that would not have been easily achieved by an individual technique. The variability in both training and validation datasets was first evaluated using PCA. Then, the CV-FDA was employed to discriminate among faults, and to categorize the processed data into two main groups: no-leak (0) and leak (1), providing the timeframe and location of the leak occurrence. Furthermore, about 8,014 observations from 81 process variables were initially included in the calculation, while the variable count was reduced to 4 with less than 1% misclassification rate in total observations. Finally, the leak was isolated in the waterwall section. Thus, the outcome of this research may provide early detection and isolation of faulty operations in the coal-fired power plant that involves a considerable number of process variables.

20 FOSSIL-FUELED POWER PLANTS↗
DOE OSTI2014

Temporal variability in urinary levels of drinking water disinfection byproducts dichloroacetic acid and trichloroacetic acid among men

Urinary haloacetic acids (HAAs), such as dichloroacetic acid (DCAA) and trichloroacetic acid (TCAA), have been suggested as potential biomarkers of exposure to drinking water disinfection byproducts (DBPs). However, variable exposure to and the short elimination half-lives of these biomarkers can result in considerable variability in urinary measurements, leading to exposure misclassification. Here we examined the variability of DCAA and TCAA levels in the urine among eleven men who provided urine samples on 8 days over 3 months. The urinary concentrations of DCAA and TCAA were measured by gas chromatography coupled with electron capture detection. We calculated the intraclass correlation coefficients (ICCs) to characterize the within-person and between-person variances and computed the sensitivity and specificity to assess how well single or multiple urine collections accurately determined personal 3-month average DCAA and TCAA levels. The within-person variance was much higher than the between-person variance for all three sample types (spot, first morning, and 24-h urine samples) for DCAA (ICC=0.08–0.37) and TCAA (ICC=0.09–0.23), regardless of the sampling interval. A single-spot urinary sample predicted high (top 33%) 3-month average DCAA and TCAA levels with high specificity (0.79 and 0.78, respectively) but relatively low sensitivity (0.47 and 0.50, respectively). Collecting two or three urine samples from each participant improved the classification. The poor reproducibility of the measured urinary DCAA and TCAA concentrations indicate that a single measurement may not accurately reflect individual long-term exposure. Collection of multiple urine samples from one person is an option for reducing exposure classification errors in studies exploring the effects of DBP exposure on reproductive health. - Highlights: • We evaluated the variability of DCAA and TCAA levels in the urine among men. • Urinary DCAA and TCAA levels varied greatly over a 3-month period. • Single measurement may not accurately reflect personal long-term exposure levels. • Collecting multiple samples from one person improved the exposure classification.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2018

Hourly land-use regression models based on low-cost PM monitor data

Highlights: • A network of 23 low-cost PM monitors was deployed across an urban area. • Land use regression models predict the hourly intra-urban PM variations. • Coefficients of determination for hourly PM predictions ranged 0.66–0.76. • Low-cost PM monitor data need to be carefully evaluated before use. • The combination of low-cost monitors and modeling provides useful PM estimates. Land-use regression (LUR) models provide location and time specific estimates of exposure to air pollution and thereby improve the sensitivity of health effects models. However, they require pollutant concentrations at multiple locations along with land-use variables. Often, monitoring is performed over short durations using mobile monitoring with research-grade instruments. Low-cost PM monitors provide an alternative approach that increases the spatial and temporal resolution of the air quality data. LUR models were developed to predict hourly PM concentrations across a metropolitan area using PM concentrations measured simultaneously at multiple locations with low-cost monitors. Monitors were placed at 23 sites during the 2015/16 heating season. Monitors were externally calibrated using co-located measurements including a reference instrument (GRIMM particle spectrometer). LUR models for each hour of the day and weekdays/weekend days were developed using the deletion/substitution/addition algorithm. Coefficients of determination for hourly PM predictions ranged from 0.66 and 0.76 (average 0.7). The hourly-resolved LUR model results will be used in epidemiological studies to examine if and how quickly, increases in ambient PM concentrations trigger adverse health events by reducing the exposure misclassification that arises from using less time resolved exposure estimates.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2013

Automated Patient Identification and Localization Error Detection Using 2-Dimensional to 3-Dimensional Registration of Kilovoltage X-Ray Setup Images

Purpose: To determine whether kilovoltage x-ray projection radiation therapy setup images could be used to perform patient identification and detect gross errors in patient setup using a computer algorithm. Methods and Materials: Three patient cohorts treated using a commercially available image guided radiation therapy (IGRT) system that uses 2-dimensional to 3-dimensional (2D-3D) image registration were retrospectively analyzed: a group of 100 cranial radiation therapy patients, a group of 100 prostate cancer patients, and a group of 83 patients treated for spinal lesions. The setup images were acquired using fixed in-room kilovoltage imaging systems. In the prostate and cranial patient groups, localizations using image registration were performed between computed tomography (CT) simulation images from radiation therapy planning and setup x-ray images corresponding both to the same patient and to different patients. For the spinal patients, localizations were performed to the correct vertebral body, and to an adjacent vertebral body, using planning CTs and setup x-ray images from the same patient. An image similarity measure used by the IGRT system image registration algorithm was extracted from the IGRT system log files and evaluated as a discriminant for error detection. Results: A threshold value of the similarity measure could be chosen to separate correct and incorrect patient matches and correct and incorrect vertebral body localizations with excellent accuracy for these patient cohorts. A 10-fold cross-validation using linear discriminant analysis yielded misclassification probabilities of 0.000, 0.0045, and 0.014 for the cranial, prostate, and spinal cases, respectively. Conclusions: An automated measure of the image similarity between x-ray setup images and corresponding planning CT images could be used to perform automated patient identification and detection of localization errors in radiation therapy treatments.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2023

Identifying common stored product insects using automated deep learning methods

Monitoring stored product insect pests is a common practice for post-harvest management of stored grain and grain-based commodities, which helps ensure product quality from harvest to final consumer. Current methods of sampling and monitoring can be time-consuming, labor-intensive, expensive and require expertise in insect identification. Therefore, this study aims to develop an image-based automated identification system for common stored product insect species using deep-learning methods. Top-down images of the common stored product adult insect species of Rhyzopertha dominica, Cryptolestes ferrugineus, Tribolium castaneum, Sitophilus oryzae, and Oryzaephilus surinamensis were acquired and analyzed. Deep learning-based, state-of-the-art Convolutional Neural Networks (CNN) models (ResNet-50, MobileNet-v2, DarkNet-53, and EfficientNet-b0) were fine-tuned with a transfer learning approach to classify the insect species. All models were able to correctly identify the insect species with at least 96% accuracy and with few misclassifications. One issue with trained CNNs is that they do not explain the reasoning for the classification and are often called a “black box”. Therefore, visualization methods called Gradient-weighted Class Activation Mapping (Grad-CAM) were implemented to explore the black box network. The Grad-CAM uses heat maps to highlight the major image features that the network focused on to make insect species predictions. The Grad-CAM verifies the network's prediction and also helps improve network performance. This study contributes to the overall goal of developing a camera-based system for monitoring stored grain insects. As a result, the developed system would empower warehouse, flour mills, and other food facilities with a tool to quickly and accurately identify insect species in stored product environments and could be implemented as part of a close to real-time monitoring system.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2015

Neutron detection in a high-gamma field using solution-grown stilbene

We present that a solution-based technique for growing large-volume stilbene scintillators was developed in 2013; crystals up to diameters of 10 cm, or larger, have been grown while preserving excellent pulse shape discrimination (PSD) properties. The goal of this study is to evaluate the PSD capabilities of 5.08 by 5.08-cm stilbene crystals grown by Lawrence Livermore National Laboratory and Inrad Optics when exposed to a 1000 to 1 gamma ray-neutron ratio and operating at a 100-kHz count rate. Results were compared to an equivalent EJ-309 liquid scintillation detector. 252 Cf neutron pulses were recorded in two experiments where 60 Co and 137 Cs sources created the high-gamma field. The high count rate created numerous double pulses that were cleaned using fractional and template approaches designed to remove double pulses while preserving neutron counts. PSD was performed at a threshold of 42 keVee (440-keV proton) for stilbene and 60 keVee (610-keV proton) for EJ-309 liquid. The lower threshold in stilbene resulted in a neutron intrinsic efficiency of approximately 14.5%, 10% higher than EJ-309 liquid, for bare 252 Cf and 13% for 252 Cf in the high-gamma field. Despite the lower threshold, the gamma misclassification rate in stilbene was approximately 3×10 $-$6 , nearly a factor-of-five lower than what we found with the EJ-309 liquid.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2020

Event-by-event neutron–photon multiplicity correlations in 252Cf(sf)

Excited nuclear fragments are emitted during nuclear fission. The de-excitation of these fission fragments takes place as sequential emission of neutrons followed by photons. A correlation between neutron and photon multiplicities accompanying fission is thus expected. Fission event generators based on established statistical nuclear physics models predict a negative event-by-event correlation in neutron-photon multiplicity. A survey of published experimental results of an event-by-event covariance between the neutron - photon multiplicities emitted following the spontaneous fission of 252 Cf is presented. Analytic unfolding expressions are developed in this work to determine the bias introduced by background sources, particle misclassification, pulse pileup, and inelastic photon production. The published experimental data are re-analyzed using these unfolding techniques and are found to be in qualitative agreement with the predictions of model-based calculations. Finally, in particular, we have concluded that there exists a significant event-by-event neutron-photon emission competition following the spontaneous fission of 252 Cf.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Expanded analysis of machine learning models for nuclear transient identification using TPOT

Industries around the world are becoming more and more data driven. The nuclear field is no exception with several different applications being proposed. One popular area of research is the use of machine learning in transient detection. This paper seeks to build upon a previous study which made use of the AutoML package TPOT to train traditional machine learning models to classify transient events occurring with a reactor. Synthetic data was once again collected using a GPWR reactor simulator. Data on 12 different events was collected using 15 different initial conditions. Here, a dataset consisting of over 100,000 data points was compiled and used to train 7 different machine learning models using a pre-defined TPOT dictionary with 12 different preprocessing techniques. Three of the trained models were able to produce validation results in the 90s with the expanded dataset. Once the models were trained, it was possible to look into where during the simulation, misclassifications occurred. Using these three models, analysis was done to determine if TPOT could be used to train models that were effective if important features were missing. The results from this were positive with the newly trained models scoring close to the original models. Finally, to conclude this study, the three high performing models were retrained using different random states to see if there was any major variation when different states were used.

42 ENGINEERING↗
DOE OSTI2017

Detectors for Active Interrogation Applications

Active interrogation creates an environment that is particularly challenging from a radiation-detection standpoint: the elevated background levels from the source can mask the desired signatures from the SNM. Neutron based interrogation experiments have shown that nanosecond-level timing is required to discriminate induced-fission neutrons from the scattered source neutrons. Previous experiments using high-energy bremsstrahlung X-rays have demonstrated the ability to induce and detect prompt photofission neutrons from single target materials; however, a real-world application would require spectroscopic capability to discern between photofission neutrons emitted by SNM and neutrons emitted by other reactions in non-SNM. Using digital pulseshape discrimination, organic liquid scintillators are capable of reliably detecting neutrons in an intense gamma-ray field. Photon misclassification rates as low as 1 in 10 6 have been achieved, which is approaching the level of gaseous neutron detectors such as 3 He without the need for neutron moderation. These scintillators also possess nanosecond-timing resolution, making them candidates for both neutron-and photon-driven active interrogation systems. Lastly, we have applied an array of liquid and NaI(Tl) scintillators to successfully image 13.7 kg of HEU interrogated by a DT neutron generator; the system was in the direct presence of the accelerator during the experiment.

36 MATERIALS SCIENCE↗
DOE OSTI2018

Part 1: Tracing Russian VX to its synthetic routes by multivariate statistics of chemical attribution signatures

Chemical attribution signatures (CAS) associated with different synthetic routes used for the production of Russian VX (VR) were identified here. The goal of the study was to retrospectively determine the production method employed for an unknown VR sample. Six different production methods were evaluated, carefully chosen to include established synthetic routes used in the past for large scale production of the agent, routes involving general phosphorus-sulfur chemistry pathways leading to the agent, and routes whose main characteristic is their innate simplicity in execution. Two laboratories worked in parallel and synthesized a total of 37 batches of VR via the six synthetic routes following predefined synthesis protocols. The chemical composition of impurities and byproducts in each route was analyzed by GC/MS-EI and 49 potential CAS were recognized as important markers in distinguishing these routes using Principal Component Analysis (PCA). The 49 potential CAS included expected species based on knowledge of reaction conditions and pathways but also several novel compounds that were fully identified and characterized by a combined analysis that included MS-CI, MS-EI and HR-MS. The CAS profiles of the calibration set were then analyzed using partial least squares discriminant analysis (PLS-DA) and a cross validated model was constructed. The model allowed the correct classification of an external test set without any misclassifications, demonstrating the utility of this methodology for attributing VR samples to a particular production method. This work is part one of a three-part series in this Forensic VSI issue of a Sweden-United States collaborative effort towards the understanding of the CAS of VR in diverse batches and matrices. This part focuses on the CAS in synthesized batches of crude VR and in the following two parts of the series the influence of food matrices on the CAS profiles are investigated.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗