Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “statistical feature extraction”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

71 records · Page 4

FORTRAN programs to process Magsat data for lithospheric, external field, and residual core components

The FORTRAN programs supplied in this document provide a complete processing package for statistically extracting residual core, external field and lithospheric components in Magsat observations. To process the individual passes: (1) orbits are separated into dawn and dusk local times and by altitude, (2) passes are selected based on the variance of the magnetic field observations after a least-squares fit of the core field is removed from each pass over the study area, and (3) spatially adjacent passes are processed with a Fourier correlation coefficient filter to separate coherent and non-coherent features between neighboring tracks. In the second state of map processing: (1) data from the passes are normalized to a common altitude and gridded into dawn and dusk maps with least squares collocation, (2) dawn and dusk maps are correlated with a Fourier correlation efficient filter to separate coherent and non-coherent features; the coherent features are averaged to produce a total field grid, (3) total field grids from all altitudes are continued to a common altitude, correlation filtered for coherent anomaly features, and subsequently averaged to produce the final total field grid for the study region, and (4) the total field map is differentially reduced to the pole.

Alsdorf, Douglas E.↗

A Study of Feature Extraction Using Divergence Analysis of Texture Features

An empirical study of texture analysis for feature extraction and classification of high spatial resolution remotely sensed imagery (10 meters) is presented in terms of specific land cover types. The principal method examined is the use of spatial gray tone dependence (SGTD). The SGTD method reduces the gray levels within a moving window into a two-dimensional spatial gray tone dependence matrix which can be interpreted as a probability matrix of gray tone pairs. Haralick et al (1973) used a number of information theory measures to extract texture features from these matrices, including angular second moment (inertia), correlation, entropy, homogeneity, and energy. The derivation of the SGTD matrix is a function of: (1) the number of gray tones in an image; (2) the angle along which the frequency of SGTD is calculated; (3) the size of the moving window; and (4) the distance between gray tone pairs. The first three parameters were varied and tested on a 10 meter resolution panchromatic image of Maryville, Tennessee using the five SGTD measures. A transformed divergence measure was used to determine the statistical separability between four land cover categories forest, new residential, old residential, and industrial for each variation in texture parameters.

Hallada, W. A.↗

Speech Acquisition and Automatic Speech Recognition for Integrated Spacesuit Audio Systems

A voice-command human-machine interface system has been developed for spacesuit extravehicular activity (EVA) missions. A multichannel acoustic signal processing method has been created for distant speech acquisition in noisy and reverberant environments. This technology reduces noise by exploiting differences in the statistical nature of signal (i.e., speech) and noise that exists in the spatial and temporal domains. As a result, the automatic speech recognition (ASR) accuracy can be improved to the level at which crewmembers would find the speech interface useful. The developed speech human/machine interface will enable both crewmember usability and operational efficiency. It can enjoy a fast rate of data/text entry, small overall size, and can be lightweight. In addition, this design will free the hands and eyes of a suited crewmember. The system components and steps include beam forming/multi-channel noise reduction, single-channel noise reduction, speech feature extraction, feature transformation and normalization, feature compression, model adaption, ASR HMM (Hidden Markov Model) training, and ASR decoding. A state-of-the-art phoneme recognizer can obtain an accuracy rate of 65 percent when the training and testing data are free of noise. When it is used in spacesuits, the rate drops to about 33 percent. With the developed microphone array speech-processing technologies, the performance is improved and the phoneme recognition accuracy rate rises to 44 percent. The recognizer can be further improved by combining the microphone array and HMM model adaptation techniques and using speech samples collected from inside spacesuits. In addition, arithmetic complexity models for the major HMMbased ASR components were developed. They can help real-time ASR system designers select proper tasks when in the face of constraints in computational resources.

Huang, Yiteng↗

Algorithms for Spectral Decomposition with Applications to Optical Plume Anomaly Detection

The analysis of spectral signals for features that represent physical phenomenon is ubiquitous in the science and engineering communities. There are two main approaches that can be taken to extract relevant features from these high-dimensional data streams. The first set of approaches relies on extracting features using a physics-based paradigm where the underlying physical mechanism that generates the spectra is used to infer the most important features in the data stream. We focus on a complementary methodology that uses a data-driven technique that is informed by the underlying physics but also has the ability to adapt to unmodeled system attributes and dynamics. We discuss the following four algorithms: Spectral Decomposition Algorithm (SDA), Non-Negative Matrix Factorization (NMF), Independent Component Analysis (ICA) and Principal Components Analysis (PCA) and compare their performance on a spectral emulator which we use to generate artificial data with known statistical properties. This spectral emulator mimics the real-world phenomena arising from the plume of the space shuttle main engine and can be used to validate the results that arise from various spectral decomposition algorithms and is very useful for situations where real-world systems have very low probabilities of fault or failure. Our results indicate that methods like SDA and NMF provide a straightforward way of incorporating prior physical knowledge while NMF with a tuning mechanism can give superior performance on some tests. We demonstrate these algorithms to detect potential system-health issues on data from a spectral emulator with tunable health parameters.

Srivastava, Askok N.↗

Active Missions and the VxOs with THEMIS as an Example

The Virtual Observatories (VxOs) provide a host of services to data producers and researchers. They help data producers to describe their data in standard Space Physics Archive Search and Extract (SPASE) terms that enable scientists to understand data products from a wide range of missions. They offer search interfaces based on specified criteria that help researchers discover conjunctions, prominent events, and intervals of interest. In this talk, we show how VMO services can be used with Time History of Events and Macroscale Interactions during Substorms (THEMIS) observations to identify magnetotail intervals marked by high speed flows, enhanced densities, or high temperatures. We present statistical surveys of when and where these phenomena occur. We then show how the VMO services can be used to identify events in which two or more THEMIS spacecraft observe specified features for more detailed analysis. We conclude by discussing the current limitations of VMO tools and outline plans for the future.

Sibeck, D. G.↗

Classifying Agnostic Biosignatures using Raman, VNIR, and Elemental Data

How can we use our current wealth of terrestrial data, encompassing biogenic and abiogenic systems, to determine the distinguishing properties of life? SCOBI (Statistical Classification of Biosignature Information) uses machine learning techniques to algorithmically identify combinations of measurements that are “indicative of life”. A set of ~1000 observations, comprising elemental abundance, isotopic fractionation, VNIR reflectance, and (in progress) Raman spectra, have been assembled from existing literature and databases. The observations cover systems classified as “indicative alive” (e.g., cells, vegetation), “indicative non-alive” (e.g., fossils, teeth), “mixed indicative” (e.g., soil, pond water), or “non-indicative” (e.g., rocks, meteorites). VNIR data was preprocessed by linear interpolation from 400-2100 nm and smoothed with a Savitzky-Golay filter. To limit the amount of Earth-biochemistry-specific (non-agnostic) information included, the first five spectral features extracted were number of peaks, number of troughs, mean reflectance, mean peak width, and broadest peak width. To help further emphasize agnostic biosignatures, Earth-specific features such as chlorophylls have been manually flagged so that feature importance with and without them can be compared. Classifiers including k-nearest neighbors (KNN), Gaussian Naïve Bayes (GNB), logistic regression (LR), random forest (RF), and support vector machine (SVM) were implemented, as was a combination voting classifier. Performance metrics included false positive rates, false negative rates, and AUC with 50-50 test/train splits (Monte Carlo simulations). Key takeaways from this stage, prior to the inclusion of Raman spectra, are (1) the overall success rate of 0.933 AUC was most heavily influenced by the elemental abundance data; and (2) VNIR reflectance had the lowest classification performance with 0.52 AUC (58% of objects correctly classified). The next steps are to complete integration of Raman spectral data and to improve the approach to pre-processing and feature extraction for both types of spectral data, such as automated baseline removal, whole spectrum matching, and dimensionality reduction.

Biosignatures↗

An IDL-based analysis package for COBE and other skycube-formatted astronomical data

UIMAGE is a data analysis package written in IDL for the Cosmic Background Explorer (COBE) project. COBE has extraordinarily stringent accuracy requirements: 1 percent mid-infrared absolute photometry, 0.01 percent submillimeter absolute spectrometry, and 0.0001 percent submillimeter relative photometry. Thus, many of the transformations and image enhancements common to analysis of large data sets must be done with special care. UIMAGE is unusual in this sense in that it performs as many of its operations as possible on the data in its native format and projection, which in the case of COBE is the quadrilateralized sphereical cube ('skycube'). That is, after reprojecting the data, e.g., onto an Aitoff map, the user who performs an operation such as taking a crosscut or extracting data from a pixel is transparently acting upon the skycube data from which the projection was made, thereby preserving the accuracy of the result. Current plans call for formatting external data bases such as CO maps into the skycube format with a high-accuracy transformation, thereby allowing Guest Investigators to use UIMAGE for direct comparison of the COBE maps with those at other wavelengths from other instruments. It is completely menu-driven so that its use requires no knowledge of IDL. Its functionality includes I/O from the COBE archives, FITS files, and IDL save sets as well as standard analysis operations such as smoothing, reprojection, zooming, statistics of areas, spectral analysis, etc. One of UIMAGE's more advanced and attractive features is its terminal independence. Most of the operations (e.g., menu-item selection or pixel selection) that are driven by the mouse on an X-windows terminal are also available using arrow keys and keyboard entry (e.g., pixel coordinates) on VT200 and Tektronix-class terminals. Even limited grey scales of images are available this way. Obviously, image processing is very limited on this type of terminal, but it is nonetheless surprising how much analysis can be done on that medium. Such flexibility has the virtue of expanding the user community to those who must work remotely on non-image terminals, e.g., via modem.

Ewing, J. A.↗

Use of high-dimensional spectral data to evaluate organic matter, reflectance relationships in soils

Recent breakthroughs in remote sensing technology have led to the development of a spaceborne high spectral resolution imaging sensor, HIRIS, to be launched in the mid-1990s for observation of earth surface features. The effects of organic carbon content on soil reflectance over the spectral range of HIRIS, and to examine the contributions of humic and fulvic acid fractions to soil reflectance was evaluated. Organic matter from four Indiana agricultural soils was extracted, fractionated, and purified, and six individual components of each soil were isolated and prepared for spectral analysis. The four soils, ranging in organic carbon content from 0.99 percent, represented various combinations of genetic parameters such as parent material, age, drainage, and native vegetation. An experimental procedure was developed to measure reflectance of very small soil and organic component samples in the laboratory, simulating the spectral coverage and resolution of the HIRIS sensor. Reflectance in 210 narrow (10 nm) bands was measured using the CARY 17D spectrophotometer over the 400 to 2500 nm wavelength range. Reflectance data were analyzed statistically to determine the regions of the reflective spectrum which provided useful information about soil organic matter content and composition. Wavebands providing significant information about soil organic carbon content were located in all three major regions of the reflective spectrum: visible, near infrared, and middle infrared. The purified humic acid fractions of the four soils were separable in six bands in the 1600 to 2400 nm range, suggesting that longwave middle infrared reflectance may be useful as a non-destructive laboratory technique for humic acid characterization.

Henderson, T. L.↗

Climatological Processing of Radar Data for the TRMM Ground Validation Program

The Tropical Rainfall Measuring Mission (TRMM) satellite was successfully launched in November, 1997. The main purpose of TRMM is to sample tropical rainfall using the first active spaceborne precipitation radar. To validate TRMM satellite observations, a comprehensive Ground Validation (GV) Program has been implemented. The primary goal of TRMM GV is to provide basic validation of satellite-derived precipitation measurements over monthly climatologies for the following primary sites: Melbourne, FL; Houston, TX; Darwin, Australia; and Kwajalein Atoll, RMI. As part of the TRMM GV effort, research analysts at NASA Goddard Space Flight Center (GSFC) generate standardized TRMM GV products using quality-controlled ground-based radar data from the four primary GV sites as input. This presentation will provide an overview of the TRMM GV climatological processing system. A description of the data flow between the primary GV sites, NASA GSFC, and the TRMM Science and Data Information System (TSDIS) will be presented. The radar quality control algorithm, which features eight adjustable height and reflectivity parameters, and its effect on monthly rainfall maps will be described. The methodology used to create monthly, gauge-adjusted rainfall products for each primary site will also be summarized. The standardized monthly rainfall products are developed in discrete, modular steps with distinct intermediate products. These developmental steps include: (1) extracting radar data over the locations of rain gauges, (2) merging rain gauge and radar data in time and space with user-defined options, (3) automated quality control of radar and gauge merged data by tracking accumulations from each instrument, and (4) deriving Z-R relationships from the quality-controlled merged data over monthly time scales. A summary of recently reprocessed official GV rainfall products available for TRMM science users will be presented. Updated basic standardized product results and trends involving monthly accumulation, Z-R relationship, and gauge statistics for each primary GV site will be also displayed.

Kulie, Mark↗

Algorithmic Classification of Raman Spectra Biosignatures: Improving Life Detection Confidence

“Agnostic” biosignatures – indicators of life (or the absence of life), independent of a particular biochemistry – are increasingly considered a high standard for life detection. The Ladder of Life Detection (2018) called for investigating how combinations of independent and different potential biosignatures affect confidence. To address this gap, statistical classification of elemental abundances, isotopic fractionation, and reflectance spectroscopy (VNIR) has been implemented. Raman spectroscopy, highly desirable due to its wide availability, has the potential to improve this predictive power. This work implemented biosignature classification algorithms on Raman data alone, in preparation for combination with the other data types. Raman spectroscopy data was collected from published databases and papers as part of a manually curated dataset of “indicative” and “non-indicative of life” samples. These currently include 61 non-indicative samples (meteorites, magnetite); 3 indicative living samples (bacteria); 20 indicative non-living samples (chalk, bone); and 12 indicative mixed (with non-indicative material) samples (soil, microbial mats). Laboratory work is ongoing to characterize additional samples, particularly a greater breadth of mixed systems. Spectra were interpolated, filtered with the Savitzsky-Golay filter, and de-noised. For a preliminary examination, agnostic features were manually extracted including mean intensity, number of peaks, and mean peak width. Different peak prominences and filtering polynomials were used to refine features. Classification algorithms were implemented: k-nearest neighbors (KNN), logistic regression (LR), linear support vector machines (SVM), random forest (RF), Gaussian naïve bayes (GNB). Lastly, Monte Carlo simulations on 1,000 50%-train-test-splits were used to validate classification performance and feature significance. The preliminary feature set achieved its highest AUC of 0.52 with LR, with no strongly discriminatory features. Work to improve feature extraction, such as through deep learning with back propagation, is planned. In future work, the Raman data will be combined with the other data types, and potentially new data types such as enantiomeric excess. This project was partially supported through the NASA Ames Project EXcellence (APEX) incubator program.

Astrobiology↗

Estimations of ABL fluxes and other turbulence parameters from Doppler lidar data

Techniques for extraction boundary layer parameters from measurements of a short-pulse CO2 Doppler lidar are described. The measurements are those collected during the First International Satellites Land Surface Climatology Project (ISLSCP) Field Experiment (FIFE). By continuously operating the lidar for about an hour, stable statistics of the radial velocities can be extracted. Assuming that the turbulence is horizontally homogeneous, the mean wind, its standard deviations, and the momentum fluxes were estimated. Spectral analysis of the radial velocities is also performed from which, by examining the amplitude of the power spectrum at the inertial range, the kinetic energy dissipation was deduced. Finally, using the statistical form of the Navier-Stokes equations, the surface heat flux is derived as the residual balance between the vertical gradient of the third moment of the vertical velocity and the kinetic energy dissipation. Combining many measurements would normally reduce the error provided that, it is unbiased and uncorrelated. The nature of some of the algorithms however, is such that, biased and correlated errors may be generated even though the raw measurements are not. Data processing procedures were developed that eliminate bias and minimize error correlation. Once bias and error correlations are accounted for, the large sample size is shown to reduce the errors substantially. The principal features of the derived turbulence statistics for two case studied are presented.

Gal-Chen, Tzvi↗

Intelligent System Development Using a Rough Sets Methodology

The purpose of this research was to examine the potential of the rough sets technique for developing intelligent models of complex systems from limited information. Rough sets a simple but promising technology to extract easily understood rules from data. The rough set methodology has been shown to perform well when used with a large set of exemplars, but its performance with sparse data sets is less certain. The difficulty is that rules will be developed based on just a few examples, each of which might have a large amount of noise associated with them. The question then becomes, what is the probability of a useful rule being developed from such limited information? One nice feature of rough sets is that in unusual situations, the technique can give an answer of 'I don't know'. That is, if a case arises that is different from the cases the rough set rules were developed on, the methodology can recognize this and alert human operators of it. It can also be trained to do this when the desired action is unknown because conflicting examples apply to the same set of inputs. This summer's project was to look at combining rough set theory with statistical theory to develop confidence limits in rules developed by rough sets. Often it is important not to make a certain type of mistake (e.g., false positives or false negatives), so the rules must be biased toward preventing a catastrophic error, rather than giving the most likely course of action. A method to determine the best course of action in the light of such constraints was examined. The resulting technique was tested with files containing electrical power line 'signatures' from the space shuttle and with decompression sickness data.

Anderson, Gray T.↗

The Multiplatform Precipitation Feature (MPF) Database: Synthesizing Satellite and Ground-Based Precipitation and Lightning Datasets for Convective Studies

NASA’s Lightning Imaging Sensor (LIS) and the Global Precipitation Measurement (GPM) mission have contributed a wealth of data toward global lightning and precipitation studies, respectively. Combining lightning and precipitation datasets leverages their unique insights into deep convective processes that inform about characteristics of convection and its intensity. Recent efforts to synthesize the LIS and GPM datasets prepare the opportunity for unprecedented large-scale, value-added multiplatform analyses of convection. This data synthesis proof-of-concept study elaborates on the creation of a database of reflectivity-based multiplatform precipitation features (MPFs) that capture a combination of information extracted from spatiotemporally coincident lightning and precipitation data within individual storm features. The space-based GPM Dual-frequency Precipitation Radar (DPR) provides a record of precipitation data, while the GPM Validation Network (VN) additionally incorporates ground-based polarimetric Doppler radar data to provide microphysical and kinematic context to DPR data. The LIS instrument onboard the International Space Station has contributed lightning observations since 2017. MPFs encapsulating information from these datasets are created from isolated regions of filtered, smoothed DPR reflectivity data to which ellipses are fit. Each MPF includes feature location, size, and eccentricity information as well as summary reflectivity characteristics. They also include summaries of precipitation microphysics and derived three-dimensional wind available from ground-based radar data. LIS data provides standard lightning characteristics such as flash count and density to each MPF as well as other informative metrics such as flash area and radiance. Each MPF file includes information about the original data from which the MPF and its characteristics were determined, allowing end-user reconstruction of the ellipse and deeper “level I” analysis of captured data. This database of VN-LIS MPFs enables broad statistical analysis of the relationships between the microphysical, kinematic, and electrical properties of convection. Preliminary results from a demonstration of the database will be described as well as ongoing efforts and avenues for future work.

Lightning↗

Cosmic Complexity

What explains the extraordinary complexity of the observed universe, on all scales from quarks to the accelerating universe? My favorite explanation (which I certainty did not invent) ls that the fundamental laws of physics produce natural instability, energy flows, and chaos. Some call the result the Life Force, some note that the Earth is a living system itself (Gaia, a "tough bitch" according to Margulis), and some conclude that the observed complexity requires a supernatural explanation (of which we have many). But my dad was a statistician (of dairy cows) and he told me about cells and genes and evolution and chance when I was very small. So a scientist must look for me explanation of how nature's laws and statistics brought us into conscious existence. And how is that seemll"!gly Improbable events are actually happening a!1 the time? Well, the physicists have countless examples of natural instability, in which energy is released to power change from simplicity to complexity. One of the most common to see is that cooling water vapor below the freezing point produces snowflakes, no two alike, and all complex and beautiful. We see it often so we are not amazed. But physlc!sts have observed so many kinds of these changes from one structure to another (we call them phase transitions) that the Nobel Prize in 1992 could be awarded for understanding the mathematics of their common features. Now for a few examples of how the laws of nature produce the instabilities that lead to our own existence. First, the Big Bang (what an insufficient name!) apparently came from an instability, in which the "false vacuum" eventually decayed into the ordinary vacuum we have today, plus the most fundamental particles we know, the quarks and leptons. So the universe as a whole started with an instability. Then, a great expansion and cooling happened, and the loose quarks, finding themselves unstable too, bound themselves together into today's less elementary particles like protons and neutrons, liberating a little energy and creating complexity. Then, the expanding universe cooled some more, and neutrons and protons, no longer kept apart by immense temperatures, found themselves unstable and formed helium nuclei. Then, a little more cooling, and atomic nuclei and electrons were no longer kept apart, and the universe became transparent. Then a little more cooling, and the next instability began: gravitation pulled matter together across cosmic distances to form stars and galaxies. This instability is described as a "negative heat capadty" in which extracting energy from a gravitating system makes it hotter -- clearly the 2nd law of thermodynamics does not apply here! (This is the physicist's part of the answer to e e cummings' question: what is the wonder that's keeping the stars apart?) Then, the next instability is that hydrogen and helium nuclei can fuse together to release energy and make stars burn for billions of years. And then at the end of the fuel source, stars become unstable and explode and liberate the chemical elements back into space. And because of that, on planets like Earth, sustained energy flows support the development of additional instabilities and all kinds of complex patterns. Gravitational instability pulls the densest materials into the core of the Earth, leaving a thin skin of water and air, and makes the interior churn incessantly as heat flows outwards. And the heat from the sun, received mostly near the equator and flowing towards the poles, supports the complex atmospheric and oceanic circulations. And because or that, the physical Earth is full of natural chemical laboratories, concentrating elements here, mixing them there, raising and lowering temperatures, ceaselessly experimenting with uncountable events where new instabilities can arise. At least one of them was the new experiment called life. Now that we know that there are at least as many planets as there are stars, it is hard to imagine that nature's ceasess experimentation would not be able to produce life elsewhere -- but we don't know for sure. And life went on to cause new Instabilities, constantly evolving, with living things in an extraordinary range of environments, changing the global environment, with boom-and-bust cycles. with predators for every kInd of prey, with criminals for every possible crime, with governments to prevent them, and instabilities of the governments themselves. One of the instabilities Is that humans demand new weapons and new products of all sort, leading to serious investments in science and technology. So the natural/human world of competition and combat is structured to lead to advanced weaponry and cell phones. So here we are In 2012, with people writing essays and wondering whether their descendents will be artificial life forms travelling back into space. And, pondering what are the origins of those forces of nature that give rise to everything. Verllnde has argued that gravitation, the one force that has so far resisted our efforts at a Quantum description, is not even a fundamental force, but is itself it a statistical force, like osmosis. What an amazing turn of events! But after all I've just said, I should not be surprised a bit.

Mather, John C.↗

Tropical Pacific moisture variability

The objectives are to describe synoptic scale variability of moisture over the tropical Pacific Ocean and the systems leading to this variability; implement satellite analysis procedures in support of this effort, and to incorporate additional satellite information into operational analysis forecast systems at the National Meteorological Center (NMC). Composite satellite radiance patterns describe features detectable well before the development of synoptic scale tropical plumes. These typical features were extracted from historical files of Tiros Operational Vertical Sounder (TOVS) radiance observations for a pair of tropical plumes which developed during January 1989. Signals were inserted into the NMC operational medium range forecast model and a suite of model integrations were conducted. Many of the 48 h model errors of the historical forecasts were eliminated by the inclusion of more complete satellite observations. Three studies in satellite radiance analysis progressed. An analysis which blended TOVS moisture channels, OLR observations and European Center for Medium Weather Forecasts (ECMWF) model analysis to generate fields of total precipitable water comparable to those estimated from Scanning Multichannel Microwave Radiometer (SMMR) mu-wave observations. This study demonstrated that a 10 y climatology of precipitable water over the oceans is feasible, using available infrared observations (OLR and TOVS) and model analysis (ECMWF, NMC or similar quality). The estimates are sensitive to model quality and the estimating model must be updated with operational model changes. Coe developed a set of tropical plume and ITCZ composites from TOVS observations, and from NMC and ECMWF analyses which had been passed through a radiative transfer model to simulate TOVS radiances. The composites have been completed as well as many statistical diagnostics of individual TOVS channels. Analysis of the computations is commencing. Chung has initiated a study of the differences between TOVS observed vapor structure during El Nino Southern Oscillation (ENSO) (1983) and non-ENSO (1984) years. Preliminary diagnosis demonstrates gross moisture changes between warm and cold sea surface temperature episodes.

Mcguirk, James P.↗

On the Response of the Special Sensor Microwave/Imager to the Marine Environment: Implications for Atmospheric Parameter Retrievals

A reasonably rigorous basis for understanding and extracting the physical information content of Special Sensor Microwave/Imager (SSM/I) satellite images of the marine environment is provided. To this end, a comprehensive algebraic parameterization is developed for the response of the SSM/I to a set of nine atmospheric and ocean surface parameters. The brightness temperature model includes a closed-form approximation to microwave radiative transfer in a non-scattering atmosphere and fitted models for surface emission and scattering based on geometric optics calculations for the roughened sea surface. The combined model is empirically tuned using suitable sets of SSM/I data and coincident surface observations. The brightness temperature model is then used to examine the sensitivity of the SSM/I to realistic variations in the scene being observed and to evaluate the theoretical maximum precision of global SSM/I retrievals of integrated water vapor, integrated cloud liquid water, and surface wind speed. A general minimum-variance method for optimally retrieving geophysical parameters from multichannel brightness temperature measurements is outlined, and several global statistical constraints of the type required by this method are computed. Finally, a unified set of efficient statistical and semi-physical algorithms is presented for obtaining fields of surface wind speed, integrated water vapor, cloud liquid water, and precipitation from SSM/I brightness temperature data. Features include: a semi-physical method for retrieving integrated cloud liquid water at 15 km resolution and with rms errors as small as approximately 0.02 kg/sq m; a 3-channel statistical algorithm for integrated water vapor which was constructed so as to have improved linear response to water vapor and reduced sensitivity to precipitation; and two complementary indices of precipitation activity (based on 37 GHz attenuation and 85 GHz scattering, respectively), each of which are relatively insensitive to variations in other environmental parameters.

Petty, Grant W.↗

The IPAC Image Subtraction and Discovery Pipeline for the Intermediate Palomar Transient Factory

We describe the near real-time transient-source discovery engine for the intermediate Palomar Transient Factory (iPTF), currently in operations at the Infrared Processing and Analysis Center (IPAC), Caltech. We coin this system the IPAC/iPTF Discovery Engine (or IDE). We review the algorithms used for PSF-matching, image subtraction, detection, photometry, and machine-learned (ML) vetting of extracted transient candidates. We also review the performance of our ML classifier. For a limiting signal-to-noise ratio of 4 in relatively unconfused regions, bogus candidates from processing artifacts and imperfect image subtractions outnumber real transients by approximately equal to 10:1. This can be considerably higher for image data with inaccurate astrometric and/or PSF-matching solutions. Despite this occasionally high contamination rate, the ML classifier is able to identify real transients with an efficiency (or completeness) of approximately equal to 97% for a maximum tolerable false-positive rate of 1% when classifying raw candidates. All subtraction-image metrics, source features, ML probability-based real-bogus scores, contextual metadata from other surveys, and possible associations with known Solar System objects are stored in a relational database for retrieval by the various science working groups. We review our efforts in mitigating false-positives and our experience in optimizing the overall system in response to the multitude of science projects underway with iPTF.

methods: analytical – methods: data analysis –↗