Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Outlier detection”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Hubble Frontier Field Clusters and Their Parallel Fields: Photometric and Photometric Redshift Catalogs

We present a multiband analysis of the six Hubble Frontier Field clusters and their parallel fields, producing catalogs with measurements of source photometry and photometric redshifts. We release these catalogs to the public along with maps of intracluster light and models for the brightest galaxies in each field. This rich data set covers a wavelength range from 0.2 to 8 μm, utilizing data from the Hubble Space Telescope, Keck Observatories, Very Large Telescope array, and Spitzer Space Telescope. We validate our products by injecting into our fields and recovering a population of synthetic objects with similar characteristics to those in real extragalactic surveys. The photometric catalogs contain a total of over 32,000 entries, with 50% completeness at a threshold of mag AB ~ 29.1 for unblended sources and magAB ~ 29 for blended ones, in the IR-weighted detection band. Photometric redshifts were obtained by means of template fitting and have an average outlier fraction of 10.3% and scatter σ = 0.067 when compared to spectroscopic estimates. The software we devised, after being tested in the present work, will be applied to new data sets from ongoing and future surveys.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS2009

Gear Fault Detection Effectiveness as Applied to Tooth Surface Pitting Fatigue Damage

A study was performed to evaluate fault detection effectiveness as applied to gear tooth pitting fatigue damage. Vibration and oil-debris monitoring (ODM) data were gathered from 24 sets of spur pinion and face gears run during a previous endurance evaluation study. Three common condition indicators (RMS, FM4, and NA4) were deduced from the time-averaged vibration data and used with the ODM to evaluate their performance for gear fault detection. The NA4 parameter showed to be a very good condition indicator for the detection of gear tooth surface pitting failures. The FM4 and RMS parameters performed average to below average in detection of gear tooth surface pitting failures. The ODM sensor was successful in detecting a significant amount of debris from all the gear tooth pitting fatigue failures. Excluding outliers, the average cumulative mass at the end of a test was 40 mg.

Lewicki, David G.↗
NASA NTRS2010

Gear Fault Detection Effectiveness as Applied to Tooth Surface Pitting Fatigue Damage

A study was performed to evaluate fault detection effectiveness as applied to gear-tooth-pitting-fatigue damage. Vibration and oil-debris monitoring (ODM) data were gathered from 24 sets of spur pinion and face gears run during a previous endurance evaluation study. Three common condition indicators (RMS, FM4, and NA4 [Ed. 's note: See Appendix A-Definitions D were deduced from the time-averaged vibration data and used with the ODM to evaluate their performance for gear fault detection. The NA4 parameter showed to be a very good condition indicator for the detection of gear tooth surface pitting failures. The FM4 and RMS parameters perfomu:d average to below average in detection of gear tooth surface pitting failures. The ODM sensor was successful in detecting a significant 8lDOunt of debris from all the gear tooth pitting fatigue failures. Excluding outliers, the average cumulative mass at the end of a test was 40 mg.

Lewicki, David G.↗
DOE OSTI2024

CIRCLEZ : Reliable photometric redshifts for active galactic nuclei computed solely using photometry from Legacy Survey Imaging for DESI

Photometric redshifts for galaxies hosting an accreting supermassive black hole in their center, known as active galactic nuclei (AGNs), are notoriously challenging. At present, they are most optimally computed via spectral energy distribution (SED) fittings, assuming that deep photometry for many wavelengths is available. However, for AGNs detected from all-sky surveys, the photometry is limited and provided by a range of instruments and studies. This makes the task of homogenizing the data challenging, presenting a dramatic drawback for the millions of AGNs that wide surveys such as SRG/eROSITA are poised to detect. This work aims to compute reliable photometric redshifts for X-ray-detected AGNs using only one dataset that covers a large area: the tenth data release of the Imaging Legacy Survey (LS10) for DESI. LS10 provides deep grizW1-W4 forced photometry within various apertures over the footprint of the eROSITA-DE survey, which avoids issues related to the cross-calibration of surveys. We present the results from CIRCLEZ, a machine-learning algorithm based on a fully connected neural network. CIRCLEZ is built on a training sample of 14 000 X-ray-detected AGNs and utilizes multi-aperture photometry, mapping the light distribution of the sources. The accuracy (σNMAD) and the fraction of outliers (η) reached in a test sample of 2913 AGNs are equal to 0.067 and 11.6%, respectively. The results are comparable to (or even better than) what was previously obtained for the same field, but with much less effort in this instance. We further tested the stability of the results by computing the photometric redshifts for the sources detected in CSC2 and Chandra-COSMOS Legacy, reaching a comparable accuracy as in eFEDS when limiting the magnitude of the counterparts to the depth of LS10. The method can be applied to fainter samples of AGNs using deeper optical data from future surveys (for example, LSST, Euclid), granting LS10-like information on the light distribution beyond the morphological type. Along with this paper, we have released an updated version of the photometric redshifts (including errors and probability distribution functions) for eROSITA/eFEDS.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS2020

Object and Gas Source Detection with Robotic Platforms in Perceptually-Degraded Environments

In exploration-oriented robotic missions for disaster relief in unknown subterranean environments, it is of prime importance for a human supervisor to rapidly gain situational awareness of salient objects within the environment. In this paper we present an automated object detection pipeline that is adaptable to heterogeneous robots with arbitrary sensor configurations. It has been deployed in time-critical scenarios with multiple collaborative robots in a variety of demanding underground environments. For visually observable objects, detections are made in both the visible and thermal spectra using a state-of-the-art machine learning framework for object detection and classification. Our pipeline can be rapidly adapted to a specific task by using a small, structured dataset to fine-tune a pre-trained convolutional neural network (CNN). Relative localization is separated from the CNN for speed of operation. A robust architecture for localization is used with outlier rejection and a hierarchy of fall-back distance measurement methods. Point-source objects such as gas and WiFi hotspots can also be detected, by tracking signal strength over time and presenting an intuitive visualization on a map. Observations of each object types are presented to the operator in ranked confidence order for final evaluation.

Agha-mohammadi, Ali-akbar↗
NASA NTRS2010

In Situ and Lidar Observations of Tropopause Subvisible Cirrus Clouds During TC4

During the Tropical Composition, Clouds, and Climate Coupling (TC4) experiment in July-August 2007, the NASA WB-57F and ER-2 aircraft made coordinated flights through a tropopause subvisible cirrus (SVC) layer off the Pacific Coast of Central America. The ER-2 aircraft was equipped with a remote sensing payload that included the cloud physics lidar (CPL). The WB-57F payload included cloud microphysical and trace gas measurements, and the aircraft made four vertical profiles through the SVC layer shortly after the ER-2 flew over. The in situ and remotely sensed data are used to quantify the meteorological and microphysical properties of the SVC layer, and these data are compared to the limited set of SVC measurements that have previously been made. It is found that the layer encountered was particularly tenuous, with optical depths (tau) between about 10(exp -4) and 10(exp -3). From the in situ and other meteorological data, radiative heating rate perturbations of approx.0.05-0.1 K/day are calculated. These heating rates are smaller than previous estimates for tropopause SVC, consistent with the smaller tau in the present study. Coverage statistics based on CPL data from other TC4 flights indicate that this cloud was not an outlier among the sampled population. SVC with properties similar to the one presented here are below the detection limit of space \based lidars such as CALIPSO, and a comparison with the TC4 statistics suggests that a majority (>50%) of tropopause SVC (with tau < 0.01) could be unaccounted for in studies using CALIPSO data.

Davis, Sean↗
NASA NTRSDate not supplied

Topographic Slant Range Modeling and Fault Detection for Precision Planetary Landing

This work presents a novel landing site relative topographic measurement model for aslant range sensor being utilized for precision planetary landing operations. The measurement model accounts for the local terrain the slant range sensor captures and leverages knowledge of the estimated landing site provided by the navigation filter. Notably, in contrast to previous works, the new model does not rely on surface normal approximation, reducing the model’s sensitivity to noisy digital elevation maps which represent the local topography. In addition to the measurement model, this work introduces a novel fault detection method, denoted the probabilistic inspection of topographic filter altitude likelihood (PITFAL) algorithm, that implements a statistical outlier rejection algorithm. PITFAL is designed for multi-beam slant range sensors, such as the Navigation Doppler LIDAR (NDL), and identifies statistically inconsistent range estimates through a consensus check on the set of apparent altitudes computed for each individual beam. These models are numerically validated by the Safe and Precise Landing Capability Evolution (SPLICE) project’s high-fidelity terrestrial and lunar lander simulations.

Davis W Adams↗
NASA NTRS2025

Topographic Slant Range Modeling and Fault Detection for Precision Planetary Landing

This work presents a novel landing site relative topographic measurement model for aslant range sensor being utilized for precision planetary landing operations. The measurement model accounts for the local terrain the slant range sensor captures and leverages knowledge of the estimated landing site provided by the navigation filter. Notably, in contrast to previous works, the new model does not rely on surface normal approximation, reducing the model’s sensitivity to noisy digital elevation maps which represent the local topography. In addition to the measurement model, this work introduces a novel fault detection method, denoted the probabilistic inspection of topographic filter altitude likelihood (PITFAL) algorithm, that implements a statistical outlier rejection algorithm. PITFAL is designed for multi-beam slant range sensors, such as the Navigation Doppler LIDAR (NDL), and identifies statistically inconsistent range estimates through a consensus check on the set of apparent altitudes computed for each individual beam. These models are numerically validated by the Safe and Precise Landing Capability Evolution (SPLICE) project’s high-fidelity terrestrial and lunar lander simulations.

Davis W Adams↗
DOE OSTI2022

Photometry on Structured Backgrounds: Local Pixel-wise Infilling by Regression

Photometric pipelines struggle to estimate both the flux and flux uncertainty for stars in the presence of structured backgrounds such as filaments or clouds. However, it is exactly stars in these complex regions that are critical to understanding star formation and the structure of the interstellar medium. We develop a method, similar to Gaussian process regression, which we term local pixel-wise infilling (LPI). Using a local covariance estimate, we predict the background behind each star and the uncertainty of that prediction in order to improve estimates of flux and flux uncertainty. We show the validity of our model on synthetic data and real dust fields. We further demonstrate that the method is stable even in the crowded field limit. While we focus on optical-IR photometry, this method is not restricted to those wavelengths. We apply this technique to the 34 billion detections in the second data release of the Dark Energy Camera Plane Survey. In addition to removing many >3σ outliers and improving uncertainty estimates by a factor of ~2–3 on nebulous fields, we also show that our method is well behaved on uncrowded fields. The entirely post-processing nature of our implementation of LPI photometry allows it to easily improve the flux and flux uncertainty estimates of past as well as future surveys.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Cy-Phy ADS: Cyber Physical Anomaly Detection Framework for EV Charging Systems

Today’s large-scale Electric Vehicle (EV) infrastructures are heavily dependent on information communication technologies to maintain their operation and to support communication within sub-system components as well as the outside world. These technologies are vulnerable to various cyber and physical threats. Timely identification and mitigation of these threats are critical for improving human safety, avoiding economic losses, and preventing catastrophic system failures. By addressing this, our work presents a ResNet Autoencoder (AE) based Cyber-Physical Anomaly Detection System (Cy-Phy ADS) for detecting anomalies in EV Controller Area Network (CAN) protocol communication. It consists of four main components: Cyber-Physical Feature Extractor, ResNet AE-based Anomaly Detection Framework, Cyber-Physical Health Metric (CPHM), and Visualization Dashboard. The presented framework was trained and tested using CAN data collected from the EV charging system testbed at the Idaho National Laboratory. The presented Cy-Phy ADS compared against six widely used unsupervised anomaly detection algorithms: One Class Support Vector Machine (OCSVM), Variational Autoencoder (VAE), LSTM Autoencoder (LSTM AE), Isolation Forest (IForest), Principle Component Analysis (PCA) and Local Outlier Factor (LOF). Here the presented approach showed the highest accuracy among the compared methods. Further, the proposed approach showed comparable performance in terms of precision, F1, and False positive rate. It also showed the lowest training and inference time compared to the neural network-based baseline algorithms compared against with. Additionally, the Cy-Phy ADS has advantages such as unsupervised training, the ability to provide a holistic metric for system health characterization, and non-linear feature extraction.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2023

Fast and efficient identification of anomalous galaxy spectra with neural density estimation

ABSTRACT Current large-scale astrophysical experiments produce unprecedented amounts of rich and diverse data. This creates a growing need for fast and flexible automated data inspection methods. Deep learning algorithms can capture and pick up subtle variations in rich data sets and are fast to apply once trained. Here, we study the applicability of an unsupervised and probabilistic deep learning framework, the probabilistic auto-encoder, to the detection of peculiar objects in galaxy spectra from the SDSS survey. Different to supervised algorithms, this algorithm is not trained to detect a specific feature or type of anomaly, instead it learns the complex and diverse distribution of galaxy spectra from training data and identifies outliers with respect to the learned distribution. We find that the algorithm assigns consistently lower probabilities (higher anomaly score) to spectra that exhibit unusual features. For example, the majority of outliers among quiescent galaxies are E+A galaxies, whose spectra combine features from old and young stellar population. Other identified outliers include LINERs, supernovae, and overlapping objects. Conditional modelling further allows us to incorporate additional information. Namely, we evaluate the probability of an object being anomalous given a certain spectral class, but other information such as metrics of data quality or estimated redshift could be incorporated as well. We make our code publicly available.

Böhm, Vanessa↗
NASA NTRSDate not supplied

On The Processing of Log Files for Monitoring Antenna Health

In order to improve the quality of geodetic results, we have developed an infrastructure for timely processing of telemetry from IVS observing stations. We check every hour for new log files with telemetry from both VLBI observing sessions, single dish experiments, and stow-in data collection and automatically process them. The telemetry data we use is the system temperature, phase calibration phases and amplitudes, system equivalent flux density, and the differences between formatter clock and GPS clock. For the system temperature and phase calibration, processing includes filtering out outliers and computing averages and rms of the scatter in each scan. Furthermore, for the phase calibration we also compute the group delay and detect spurious signals. Cleaned and post-processed telemetry is archived. Our process detects abnormalities, such as, anomalously high system temperature, unstable phase calibration phases, jumps in the GPS and formatter clock differences, and others. With our procedure, the latency of detection of station abnormalities is reduced to less than two hours. Early detection of abnormalities reduces the amount of affected data since station personnel get early alerts. We discuss our experience of running this system since 2022.

Phase Calibration↗
NASA NTRSDate not supplied

On The Processing of Log Files for Monitoring Antenna Health

In order to improve the quality of geodetic results, we have developed an infrastructure for timely processing of telemetry from IVS observing stations. We check every hour for new log files with telemetry from both VLBI observing sessions, single dish experiments, and stow-in data collection and automatically process them. The telemetry data we use is the system temperature, phase calibration phases and amplitudes, system equivalent flux density, and the differences between formatter clock and GPS clock. For the system temperature and phase calibration, processing includes filtering out outliers and computing averages and rms of the scatter in each scan. Furthermore, for the phase calibration we also compute the group delay and detect spurious signals. Cleaned and post-processed telemetry is archived. Our process detects abnormalities, such as, anomalously high system temperature, unstable phase calibration phases, jumps in the GPS and formatter clock differences, and others. With our procedure, the latency of detection of station abnormalities is reduced to less than two hours. Early detection of abnormalities reduces the amount of affected data since station personnel get early alerts. We discuss our experience of running this system since 2022.

VLBI↗
DOE OSTI2024

Coincident learning for unsupervised anomaly detection of scientific instruments

Abstract Anomaly detection is an important task for complex scientific experiments and other complex systems (e.g. industrial facilities, manufacturing), where failures in a sub-system can lead to lost data, poor performance, or even damage to components. While scientific facilities generate a wealth of data, labeled anomalies may be rare (or even nonexistent), and expensive to acquire. Unsupervised approaches are therefore common and typically search for anomalies either by distance or density of examples in the input feature space (or some associated low-dimensional representation). This paper presents a novel approach called coincident learning for anomaly detection (CoAD), which is specifically designed for multi-modal tasks and identifies anomalies based on coincident behavior across two different slices of the feature space. We define an unsupervised metric, F ^ β , out of analogy to the supervised classification F β statistic. CoAD uses F ^ β to train an anomaly detection algorithm on unlabeled data , based on the expectation that anomalous behavior in one feature slice is coincident with anomalous behavior in the other. The method is illustrated using a synthetic outlier data set and a MNIST-based image data set, and is compared to prior state-of-the-art on two real-world tasks: a metal milling data set and our motivating task of identifying RF station anomalies in a particle accelerator.

43 PARTICLE ACCELERATORS↗
DOE OSTI2021

Anomaly detection in the Zwicky Transient Facility DR3

We present results from applying the SNAD anomaly detection pipeline to the third public data release of the Zwicky Transient Facility (ZTF DR3). The pipeline is composed of three stages: feature extraction, search of outliers with machine learning algorithms, and anomaly identification with followup by human experts. Our analysis concentrates in three ZTF fields, comprising more than 2.25 million objects. A set of four automatic learning algorithms was used to identify 277 outliers, which were subsequently scrutinized by an expert. From these, 188 (68 per cent) were found to be bogus light curves – including effects from the image subtraction pipeline as well as overlapping between a star and a known asteroid, 66 (24 per cent) were previously reported sources whereas 23 (8 per cent) correspond to non-catalogued objects, with the two latter cases of potential scientific interest (e.g. one spectroscopically confirmed RS Canum Venaticorum star, four supernovae candidates, one red dwarf flare). Moreover, using results from the expert analysis, we were able to identify a simple bi-dimensional relation that can be used to aid filtering potentially bogus light curves in future studies. We provide a complete list of objects with potential scientific application so they can be further scrutinised by the community. These results confirm the importance of combining automatic machine learning algorithms with domain knowledge in the construction of recommendation systems for astronomy. Our code is publicly available.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Quantifying the Impact of LSST u -band Survey Strategy on Photometric Redshift Estimation and the Detection of Lyman-break Galaxies

The Vera C. Rubin Observatory will conduct the Legacy Survey of Space and Time (LSST), promising to discover billions of galaxies out to redshift 7, using six photometric bands (ugrizy) spanning the near-ultraviolet to the near-infrared. The exact number of and quality of information about these galaxies will depend on survey depth in these six bands, which in turn depends on the LSST survey strategy, i.e., how often and how long to expose in each band. u-band depth is especially important for photometric redshift (photo-z) estimation and for detection of high-redshift Lyman-break galaxies (LBGs). In this paper, we use a simulated galaxy catalog and an analytic model for the LBG population to study how recent updates and proposed changes to Rubin’s u-band throughput and LSST survey strategy impact photo-z accuracy and LBG detection. We find that proposed variations in u-band strategy have a small impact on photo-z accuracy for z < 1.5 galaxies, but the outlier fraction, scatter, and bias for higher-redshift galaxies vary by up to 50%, depending on the survey strategy considered. The number of u-band dropout LBGs at z ∼ 3 is also highly sensitive to the u-band depth, varying by up to 500%, while the number of griz-band dropouts is only modestly affected. Under the new u-band strategy recommended by the Rubin Survey Cadence Optimization Committee, we predict u-band dropout number densities of 110 deg −2 (3200 deg −2 ) in year 1 (10) of LSST. We discuss the implications of these results for LSST cosmology.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Regional and Temporal Variability of Atmospheric River Seasonality: Influences of Detection Algorithms and Moisture Transport Dynamics

Abstract Understanding the regional and temporal variability of atmospheric river (AR) seasonality is crucial for preparedness and mitigation of extreme events. While ARs were thought to peak in winter, recent research shows they exhibit region‐specific seasonality and are heavily influenced by the chosen detection algorithm. This study examines the link between the year‐to‐year consistency of peak‐AR activity to the presence of a dominant seasonal pattern, considering both location and algorithm choice. Regions are categorized by their temporal characteristics: consistent patterns (e.g., East Asia), patterns with occasional outliers (e.g., British Columbia coast), and regions lacking a clear dominant peak season (e.g., South Atlantic, parts of Australia). Hence, not all regions display a consistent seasonal cycle of AR activity. This study quantifies the extent to which a region experiences a dominant peak season of AR activity (or lacks one) and offers insights to enhance decision‐making in water management, natural hazard preparedness, and forecasting. Furthermore, given our finding that detection algorithms influence the peak season of AR activity, we also examine two diagnostic variables representative of moisture transport to corroborate our results. Integrated vapor transport, which captures meridional and zonal moisture transport, and Moist Wave Activity, representing moisture intrusions from lower to higher latitudes, are examined. Our analysis indicates that inconsistencies in the seasonal cycle of AR activity are not solely due to discrepancies in detection algorithms but also arise from changes in moisture transport. Plain Language Summary Atmospheric rivers (ARs) are critical weather phenomena that can cause extreme events like heavy rainfall and flooding. Understanding when and where ARs are most likely to occur throughout the year is essential for preparing and responding to these events. Traditionally, ARs were thought to peak in winter, but recent studies show this varies by region. Our study helps address the challenge decision‐makers face in anticipating and preparing for AR events by providing insights into the consistency of peak seasonal patterns across different areas. Some regions, like East Asia, and the British Columbia coast, show a consistent peak season, while others, like the South Atlantic and parts of Australia, have significant year‐to‐year variations, making it hard to identify a dominant season. To better understand these changes over time, the study also examines how moisture moves in the atmosphere, using Integrated Vapor Transport (which looks at moisture movement in various directions) and Moist Wave Activity (which tracks moisture shifts from lower to higher latitudes). The findings suggest that inconsistencies in AR patterns are due not only to detection methods but also due to changes in moisture transport. Key Points The peak season of atmospheric river activity can change depending on the year in some areas Interannual variations in the peak season can make identifying a dominant season challenging for some regions Frequent shifts in peak season across years reflect inconsistencies tied to detection algorithms and to underlying dynamics

Kamnani, Diya↗
DOE OSTI2021

Automatic Drift Correction through Nonlinear Sensing

For successful design and operation of advanced monitoring and control systems, engineers rely on high quality sensor signals that are simultaneously accurate, representative, voluminous, and timely. Unfortunately, sensor faults are common and lead to short-lived symptoms, such as outliers and spikes as well as long-lived symptoms, such as sensor drift. Sensor drift belongs to the category of incipient faults. These are particularly challenging to detect, diagnose, and correct as the time scales of these faults are typically longer than the time scales of the system dynamics that are of interest. Moreover, if sensor drift occurs as a result of exposure to measured medium, then it is likely that multiple sensors will exhibit similar drift rates, thus challenging fault management strategies based on redundancy. In this contribution, we present a first method that can handle this unique challenge.

Chowdhury, Dhruba↗