SEARCH · Engineering Papers

Results for “large data sets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14

All sources NASA NTRS DOE OSTI

NASA NTRS1990

Intercomparison between microwave and infrared observations of precipitable water

The possible utility of observations by the TIROS-N Vertical Sounder system as a data base for a global climatology of atmospheric precipitable water is assessed. A comparison is described between the atmospheric precipitable water content (PWC) derived from IR observations by the High-Resolution Infrared Radiation Sounder (HIRS/2) of the TOVS system using the eigenvector regression technique and derived from microwave observations by the Special Sensor Microwave/Imager (SSM/I). It is suggested that the difference between the two data sets can largely be attributed to the eigenvector method used to retrieve PWC from the HIRS/2 observations. It is shown that in the regions of subsidence associated with the air motion around the subtropical zone of high pressure, the HIRS/2 tends to overestimate the PWC.

Tjemkes, Stephen A.↗

NASA NTRS1993

Electronic Catalog Of Extragalactic Objects

NASA/IPAC Extragalactic Database (NED) is publicly accessible computerized catalog of published information about extragalactic observations. Developed to accommodate increasingly large sets of data from surveys, exponentially growing literature, and trend among astronomers to take multispectral approach to astrophysical problems. Accessible to researchers and librarians.

Helou, George↗

NASA NTRS1992

A Southern Hemisphere radar meteor orbit survey

A meteor radar system has been operated on a routine basis near Christchurch, New Zealand, to determine the orbits of Earth-impacting interplanetary dust and meteoroids. The system sensitivity is +13 visual magnitude, corresponding to approximately 100 micron sized meteoroids. With an orbital precision of 2 degrees in angular elements and 10 percent in orbital energy (1/a), the operation yields an average of 1500 orbits daily with a total to date in excess of 10(exp 5). The use of pc's and automated data reduction permit the large orbital data sets we collect to be routinely reduced. Some illustrative examples are presented of the signal formats/processing and the results of data reduction, giving the individual orbital elements and hence the overall distributions. Current studies include the distribution of dust in the inner solar system; the influx of meteoroids associated with near-Earth asteroids; and the orbital structure existing in comet-produced streams.

Baggaley, W. Jack↗

NASA NTRS1992

Automating sky object classification in astronomical survey images

We describe the application of machine classification techniques to the development of an automated tool for the reduction of a large scientific data set. The 2nd Palomer Observatory Sky Survey is nearly completed. This survey provides comprehensive coverage of the northern celestial hemisphere in the form of photographic plates. The plates are being transformed into digitized images whose quality will probably not be surpassed in the next ten to twenty years. The images are expected to contain on the order of 10(exp 7) galaxies and 10(exp 8) stars. Astronomers wish to determine which of these sky objects belong to various classes of galaxies and stars. The size of this data set precludes manual analysis. Our approach is to develop a software system which integrates the functions of independently developed techniques for image processing and data classification. Digitized sky images are passed through image processing routines to identify sky objects and to extract a set of features for each object. These routines are used to help select a useful set of attributes for classifying sky objects. Then GID3* and O-BTree, two inductive learning techniques, learn classification decision trees from examples. These classifiers will be used to process the rest of the data. This paper gives an overview of the machine learning techniques used, describes the details of our specific application, and reports the initial encouraging results. The results indicate that our approach is well-suited to the problem. The primary benefits of the approach are increased data reduction throughput and consistency of classification. The classification rules which are the product of the inductive learning techniques will form an object, examinable basis for classifying sky objects. A final, not to be underestimated benefit is that astronomers will be freed from the tedium of an intensely visual task to pursue more challenging analysis and interpretation problems based on automatically cataloged data.

Fayyad, Usama M.↗

NASA NTRS1993

Using machine learning techniques to automate sky survey catalog generation

We describe the application of machine classification techniques to the development of an automated tool for the reduction of a large scientific data set. The 2nd Palomar Observatory Sky Survey provides comprehensive photographic coverage of the northern celestial hemisphere. The photographic plates are being digitized into images containing on the order of 10(exp 7) galaxies and 10(exp 8) stars. Since the size of this data set precludes manual analysis and classification of objects, our approach is to develop a software system which integrates independently developed techniques for image processing and data classification. Image processing routines are applied to identify and measure features of sky objects. Selected features are used to determine the classification of each object. GID3* and O-BTree, two inductive learning techniques, are used to automatically learn classification decision trees from examples. We describe the techniques used, the details of our specific application, and the initial encouraging results which indicate that our approach is well-suited to the problem. The benefits of the approach are increased data reduction throughput, consistency of classification, and the automated derivation of classification rules that will form an objective, examinable basis for classifying sky objects. Furthermore, astronomers will be freed from the tedium of an intensely visual task to pursue more challenging analysis and interpretation problems given automatically cataloged data.

Fayyad, Usama M.↗

NASA NTRS1994

RAVE: Rapid Visualization Environment

Visualization is used in the process of analyzing large, multidimensional data sets. However, the selection and creation of visualizations that are appropriate for the characteristics of a particular data set and the satisfaction of the analyst's goals is difficult. The process consists of three tasks that are performed iteratively: generate, test, and refine. The performance of these tasks requires the utilization of several types of domain knowledge that data analysts do not often have. Existing visualization systems and frameworks do not adequately support the performance of these tasks. In this paper we present the RApid Visualization Environment (RAVE), a knowledge-based system that interfaces with commercial visualization frameworks and assists a data analyst in quickly and easily generating, testing, and refining visualizations. RAVE was used for the visualization of in situ measurement data captured by spacecraft.

Klumpar, D. M.↗

NASA NTRS1994

Spacecraft Data Simulator for the test of level zero processing systems

The Microelectronic Systems Branch (MSB) at Goddard Space Flight Center (GSFC) has developed a Spacecraft Data Simulator (SDS) to support the development, test, and verification of prototype and production Level Zero Processing (LZP) systems. Based on a disk array system, the SDS is capable of generating large test data sets up to 5 Gigabytes and outputting serial test data at rates up to 80 Mbps. The SDS supports data formats including NASA Communication (Nascom) blocks, Consultative Committee for Space Data System (CCSDS) Version 1 & 2 frames and packets, and all the Advanced Orbiting Systems (AOS) services. The capability to simulate both sequential and non-sequential time-ordered downlink data streams with errors and gaps is crucial to test LZP systems. This paper describes the system architecture, hardware and software designs, and test data designs. Examples of test data designs are included to illustrate the application of the SDS.

Shi, Jeff↗

NASA NTRS1993

Lithologic discrimination and alteration mapping from AVIRIS Data, Socorro, New Mexico

Geologic maps are, by their very nature, interpretive documents. In contrasts, images prepared from AVIRIS data can be used as uninterpreted, and thus unbiased, geologic maps. We are having significant success applying AVIRIS data in this non-quantitative manner to geologic problems. Much of our success has come from the power of the Linked Windows Interactive Data System. LinkWinds is a visual data analysis and exploration system under development at JPL which is designed to rapidly and interactively investigate large multivariate data sets. In this paper, we present information on the analysis technique, and preliminary results from research on potassium metasomatism, a distinctive and structurally significant type of alteration associated with crustal extension.

Beratan, K. K.↗

NASA NTRS1992

Tropospheric chemical models

The differences in atmospheric composition over the globe and the short- and long-term variations in this composition are the net effect of several atmospheric and biospheric processes: biospheric emissions, atmospheric circulation, atmospheric chemical transformations and finally deposition back to the surface. Accurate and realistic atmospheric chemistry and circulation models are essential to interpret the observed global distributions and trends of atmospheric species in terms of these underlying processes. Comparisons between model predictions and observations test current understanding of these processes and models used in conjunction with inverse methods allow deductions of the rates of these processes from the observations. With the planned inclusion of at least CO and CH4 observations on the Earth Observing System (EOS) satellites, together with the large global data set expected from in situ observations under the International Global Atmospheric Chemistry (IGAC) Project, the further development of global three-dimensional high-resolution atmospheric chemistry and circulation models in order to interpret this new data is a high-priority endeavor.

Prinn, R. G.↗

NASA NTRS1996

Steady Capillary Driven Flow

A steady capillary driven flow is developed for a liquid index in a circular tube which is partially coated with a surface modifier to produce a discontinuous wetting condition from one side of the tube to the other. The bulk flow is novel in that it is truly steady, and controlled solely by the physics associated with dynamic wetting. The influence of gravity on the flow is minimized through the use of small diameter tubes approximately O(1 mm) tested horizontally in a laboratory and larger tubes approximately O(10 mm) tested in the low gravity environment of a drop tower. Average steady velocities are predicted and compared against a large experimental data set which includes the effects of tube dimensions and fluid properties. The sensitivity of the velocity to surface cleanliness is dramatic and the advantages of experimentation in a microgravity environment are discussed.

Weislogel, Mark M.↗

NASA NTRS1995

Visual interface for space and terrestrial analysis

The management of large geophysical and celestial data bases is now, more than ever, the most critical path to timely data analysis. With today's large volume data sets from multiple satellite missions, analysts face the task of defining useful data bases from which data and metadata (information about data) can be extracted readily in a meaningful way. Visualization, following an object-oriented design, is a fundamental method of organizing and handling data. Humans, by nature, easily accept pictorial representations of data. Therefore graphically oriented user interfaces are appealing, as long as they remain simple to produce and use. The Visual Interface for Space and Terrestrial Analysis (VISTA) system, currently under development at the Naval Research Laboratory's Backgrounds Data Center (BDC), has been designed with these goals in mind. Its graphical user interface (GUI) allows the user to perform queries, visualization, and analysis of atmospheric and celestial backgrounds data.

Dombrowski, Edmund G.↗

NASA NTRS1992

A Framework for the Design of Effective Graphics for Scientific Visualization

This proposal presents a visualization framework, based on a data model, that supports the production of effective graphics for scientific visualization. Visual representations are effective only if they augment comprehension of the increasing amounts of data being generated by modern computer simulations. These representations are created by taking into account the goals and capabilities of the scientist, the type of data to be displayed, and software and hardware considerations. This framework is embodied in an assistant-based visualization system to guide the scientist in the visualization process. This will improve the quality of the visualizations and decrease the time the scientist is required to spend in generating the visualizations. I intend to prove that such a framework will create a more productive environment for tile analysis and interpretation of large, complex data sets.

Miceli, Kristina D.↗

NASA NTRS1998

The detection of clouds, aerosols and marine atmospheric boundary layer characteristics from simulated GLAS data

Scheduled for launch in 2001 as part of NASA's Earth Observing System (EOS), the Geoscience Laser Altimeter System (GLAS) will provide continuous laser sounding of the earth's atmosphere from space for the first time. From its polar orbit about 600 km above the surface, GLAS will employ a 40 Hz solid state laser operating at 1064 nm to measure topography to an accuracy of 10 cm. Simultaneously, the atmospheric channels (1064 and 532 nm) of GLAS will provide profiles of atmospheric backscatter from 40 km to the ground with 75 meter vertical resolution (Spinhirne and Palm, 1996). These measurements will give scientists an unprecedented global data set on the vertical structure of clouds and aerosols which will greatly aid research efforts aimed at understanding their effects on climate and their role in climate change (Hartman, 1994). To better understand and predict the performance of the GLAS atmospheric channels, a computer model was developed to simulate the type of signal that the instrument would likely produce. The model uses aircraft lidar data and provides realistic simulated GLAS data sets over large areas spanning a wide range of atmospheric conditions. These simulated GLAS datasets are invaluable for designing and testing algorithms for the retrieval of parameters such as cloud and aerosol layer height, optical depth and extinction cross section. This work is currently proceeding and in this paper we will present results of the cloud and aerosol detection algorithm with emphasis on the detection of Marine Atmospheric Boundary Layer (MABL) aerosol. In addition, we use a recently developed technique to ascertain the feasability of estimating MABL moisture and temperature structure from spaceborne systems such as GLAS.

Palm, Stephen P.↗

NASA NTRS1999

Optimized System Identification

In system identification, one usually cares most about finding a model whose outputs are as close as possible to the true system outputs when the same input is applied to both. However, most system identification algorithms do not minimize this output error. Often they minimize model equation error instead, as in typical least-squares fits using a finite-difference model, and it is seen here that this distinction is significant. Here, we develop a set of system identification algorithms that minimize output error for multi-input/multi-output and multi-input/single-output systems. This is done with sequential quadratic programming iterations on the nonlinear least-squares problems, with an eigendecomposition to handle indefinite second partials. This optimization minimizes a nonlinear function of many variables, and hence can converge to local minima. To handle this problem, we start the iterations from the OKID (Observer/Kalman Identification) algorithm result. Not only has OKID proved very effective in practice, it minimizes an output error of an observer which has the property that as the data set gets large, it converges to minimizing the criterion of interest here. Hence, it is a particularly good starting point for the nonlinear iterations here. Examples show that the methods developed here eliminate the bias that is often observed using any system identification methods of either over-estimating or under-estimating the damping of vibration modes in lightly damped structures.

Juang, Jer-Nan↗

NASA NTRS2002

VEXT: A Virtual Observatory Exploration Toolkit

This final report consists of two main parts. The first is taken from a paper by the PiCA (Pittsburgh Computational Astrostatistics) Group which describes our ongoing work in fast computation of n-point correlation functions. We present here a new algorithm for the fast computation of N-point correlation functions in large astronomical data sets. The algorithm is based on kd-trees which are decorated with cached sufficient statistics thus allowing for orders of magnitude speed-ups over the naive non-tree-based implementation of correlation functions. We further discuss the use of controlled approximations within the computation which allows for further acceleration. In summary, our algorithm now makes it possible to compute exact, all-pairs, measurements of the two, three and four-point correlation functions for cosmological data sets like the Sloan Digital Sky Survey and the next generation of Cosmic Microwave Background experiments. The second part summarizes the progress made by the PiCA Group in this area through the AISR grant.

Schneider, Jeff↗

NASA NTRS2004

Visual data mining for quantized spatial data

In previous papers we've shown how a well known data compression algorithm called Entropy-constrained Vector Quantization ( can be modified to reduce the size and complexity of very large, satellite data sets. In this paper, we descuss how to visualize and understand the content of such reduced data sets.

cluster analysis↗

NASA NTRS2006

Recent HARVIST Results: Classifying Crops from Remote Sensing Data

In this paper, we report on recent results from the Heterogeneous Agricultural Research Via Interactive, Scalable Technology (HARVIST) project. HARVIST seeks to provide the tools and scalability required to enable practicioners to analyze large, diverse data sets that may come from different data sources. We have focused on agricultural applications, and our current results demonstrate the ability of the system to train a crop type classifier that operates on orbital remote sensing images. We find that this classifier can label crops with an accuracy of 82%, comparable to other published results.

support vector machines↗

NASA NTRS2007

The 'Weekend Effect' in Tropospheric NO2 Seen from the Ozone Monitoring Instrument

The Ozone Monitoring Instrument has gathered daily global data on NO2 and other atmospheric trace gases since its launch on the EOS Aura satellite in 2004. The large accumulated data set makes it possible to monitor changes of both meteorological and anthropogenic origin in tropospheric NOz amounts. In particular, averages on time scales on the order of a year show a distinct 'weekend effect' in NO2 variation, with smaller NO2 amounts seen on Saturday and/or Sunday than on the remaining weekdays. Using the OMI NO2 Standard Product (SP), we examine this effect in relation to geopolitical boundaries and investigate implications for identifying sources. We also use the SP data to find evidence for other short-term anthropogenic changes in NO2 emissions over heavily polluted regions including the United States, Europe and China.

Bucsela, E.↗