Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “large data sets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12

An Altimeter Wind Speed Model Using Both Radar Backscatter and Significant Wave Height

The development and validation of a new altimeter wind speed model will be presented. This algorithm provides a direct mapping of TOPEX-measured backscatter and significant wave height to 10 m wind speed. A large scatterometer/altimeter crossover data set was assembled to develop the routine and several large ancillary data sets have been assembled for validation purposes. Validation results suggest that this two input routine provides marginal, yet measurable improvements over the standard single-parameter MCW algorithm.

Vandemark, D.↗

On the Use of Mahalanobis Distance in Particle Image Velocimetry Post-Processing

Particle Image Velocimetry (PIV) is a method of flow measurement that has become increasingly popular as an experimental tool. New technology has made high-speed and higher-dimension (stereoscopic, tomographic, etc) methods available to an ever-growing population of researchers. These advanced methods can provide significantly more data than traditional low-speed planar PIV, but these larger data sets also require more resources to process and store. A major time sink in the post-processing of large PIV data sets is the identification and rejection of “bad” or ”spurious” vectors that survive an initial processing step in commercial software, which attempts to identify spurious vectors on an image-by-image basis and not with respect to repeated trials. The Mahalanobis distance, an almost 100-year-old statistical function, was determined to be well suited for this task for its computational efficiency and higher-dimensional nature. This work includes a summary of the Mahalanobis distance and validation of its usefulness as a tool for outlier rejection in the post-processing of PIV data.

Experimental Methods↗

The Viability of Trajectory Analysis for Diagnosing Dynamical and Chemical Influences on Ozone Concentrations in the UTLS

The viability of trajectory analysis for diagnosing the interplay between chemistry and dynamics is investigated by comparing ozone mixing ratios modelled using air-parcel pathways to values observed along flight tracks during ATTREX (Airborne Tropical TRopopause EXperiment). Trajectories are initiated at the locations of ozone observations and tracked backward in time to their sources at termini of backward trajectories. The modelled values of ozone utilize 3-dimensional analysis fields from WACCM (Whole Atmosphere Community Climate Model) (a chemical-climate model with dynamical fields nudged towards MERRA (Modern-Era Retrospective Analysis and Research Applications) reanalysis) and ERA-interim (product of ECMWF - the European Centre for Medium-Range Weather Forecasts) to determine source mixing ratios with chemical production and loss terms derived from the ozone chemistry used in WACCM. A statistical base of modelled ozone is constructed with 6 trajectory platforms (adiabatic, diabatic, and kinematic forced by ERA-interim and MERRA), two chemical models (WACCM chemistry and no chemistry), and 4 trajectory lengths (5, 10, 20, and 30 days). Linear regression is employed to separate systematic errors from random errors and to characterize the impact of source mixing ratios, path length, vertical motion, and chemistry on modelled ozone errors. Errors in the analysis ozone fields are large, if not dominant, contributors to model error. Random errors are particularly large for point-by-point comparisons, however averaging over 800 km (75 minutes) flight segments substantially reduces random error and exposes systematic errors. Of the two analysis ozone data sets, WACCM, which incorporates detailed chemistry, provides the smaller systematic errors while ERA-interim, which has crude chemistry but assimilates observational data, has the smaller random errors. Of the different trajectory platforms, adiabatic calculations produce the smaller random errors (irrespective of the use of chemistry) but both vertical motion and chemistry are required to optimally reduce systematic errors. These results suggest that meaningful analysis of dynamical and chemical interactions that control ozone mixing ratios are viable on spatial scales larger than a few reanalysis grid spaces, that errors in the analyzed ozone data sets are large but not prohibitively so, and that vertical velocities and heating rates from reanalysis data, while problematic, contain useful information [on the ozone concentrations in the UTLS (Upper Troposphere/Lower Stratosphere)].

ozone↗

Fractals and Spatial Methods for Mining Remote Sensing Imagery

The rapid increase in digital remote sensing and GIS data raises a critical problem -- how can such an enormous amount of data be handled and analyzed so that useful information can be derived quickly? Efficient handling and analysis of large spatial data sets is central to environmental research, particularly in global change studies that employ time series. Advances in large-scale environmental monitoring and modeling require not only high-quality data, but also reliable tools to analyze the various types of data. A major difficulty facing geographers and environmental scientists in environmental assessment and monitoring is that spatial analytical tools are not easily accessible. Although many spatial techniques have been described recently in the literature, they are typically presented in an analytical form and are difficult to transform to a numerical algorithm. Moreover, these spatial techniques are not necessarily designed for remote sensing and GIS applications, and research must be conducted to examine their applicability and effectiveness in different types of environmental applications. This poses a chicken-and-egg problem: on one hand we need more research to examine the usability of the newer techniques and tools, yet on the other hand, this type of research is difficult to conduct if the tools to be explored are not accessible. Another problem that is fundamental to environmental research are issues related to spatial scale. The scale issue is especially acute in the context of global change studies because of the need to integrate remote-sensing and other spatial data that are collected at different scales and resolutions. Extrapolation of results across broad spatial scales remains the most difficult problem in global environmental research. There is a need for basic characterization of the effects of scale on image data, and the techniques used to measure these effects must be developed and implemented to allow for a multiple scale assessment of the data before any useful process-oriented modeling involving scale-dependent data can be conducted. Through the support of research grants from NASA, we have developed a software module called ICAMS (Image Characterization And Modeling System) to address the need to develop innovative spatial techniques and make them available to the broader scientific communities. ICAMS provides new spatial techniques, such as fractal analysis, geostatistical functions, and multiscale analysis that are not easily available in commercial GIS/image processing software. By bundling newer spatial methods in a user-friendly software module, researchers can begin to test and experiment with the new spatial analysis methods and they can gauge scale effects using a variety of remote sensing imagery. In the following, we describe briefly the development of ICAMS and present application examples.

Lam, Nina↗

Addition of First-Person Visualization Capabilities to the Glenn Research Center Communication Analysis Suite (GCAS)

The ability to visualize large sets of data is important to giving an understanding of abstract ideas to large groups. With the ability to visualize the terrain of lunar surfaces in three dimensions from both an orbital and first-person view, the capabilities of the Glenn Research Center Communication Analysis Suite (GCAS) will have expanded to allow specific landing sites to be seen more realistically. GCAS will utilize GeoTIFF-based data to visualize the lunar terrain.

ECMAScript↗

Data indexing techniques for the EUVE all-sky survey

This poster describes techniques developed for manipulating large full-sky data sets for the Extreme Ultraviolet Explorer project. The authors have adapted the quatrilateralized cubic sphere indexing algorithm to allow us to efficiently store and process several types of large data sets, such as full-sky maps of photon counts, exposure time, and count rates. A variation of this scheme is used to index sparser data such as individual photon events and viewing times for selected areas of the sky, which are eventually used to create EUVE source catalogs.

Lewis, J.↗

Comparison of Monthly Mean Cloud Fraction and Cloud Optical depth Determined from Surface Cloud Radar, TOVS, AVHRR, and MODIS over Barrow, Alaska

A one year comparison is made of mean monthly values of cloud fraction and cloud optical depth over Barrow, Alaska (71 deg 19.378 min North, 156 deg 36.934 min West) between 35 GHz radar-based retrievals, the TOVS Pathfinder Path-P product, the AVHRR APP-X product, and a MODIS based cloud retrieval product from the CERES-Team. The data sets represent largely disparate spatial and temporal scales, however, in this paper, the focus is to provide a preliminary analysis of how the mean monthly values derived from these different data sets compare, and determine how they can best be used separately, and in combination to provide reliable estimates of long-term trends of changing cloud properties. The radar and satellite data sets described here incorporate Arctic specific modifications that account for cloud detection challenges specific to the Arctic environment. The year 2000 was chosen for this initial comparison because the cloud radar data was particularly continuous and reliable that year, and all of the satellite retrievals of interest were also available for the year 2000. Cloud fraction was chosen as a comparison variable as accurate detection of cloud is the primary product that is necessary for any other cloud property retrievals. Cloud optical depth was additionally selected as it is likely the single cloud property that is most closely correlated to cloud influences on surface radiation budgets.

Uttal, Taneil↗

Problems in merging Earth sensing satellite data sets

Satellite remote sensing systems provide a tremendous source of data flow to the Earth science community. These systems provide scientists with data of types and on a scale previously unattainable. Looking forward to the capabilities of Space Station and the Earth Observing System (EOS), the full realization of the potential of satellite remote sensing will be handicapped by inadequate information systems. There is a growing emphasis in Earth science research to ask questions which are multidisciplinary in nature and global in scale. Many of these research projects emphasize the interactions of the land surface, the atmosphere, and the oceans through various physical mechanisms. Conducting this research requires large and complex data sets and teams of multidisciplinary scientists, often working at remote locations. A review of the problems of merging these large volumes of data into spatially referenced and manageable data sets is presented.

Smith, Paul H.↗

Data-link alternatives for the NASA pilot data systems

There are several preliminary, or pilot, efforts for developing data systems for supporting NASA science research. The Pilot Data Systems consist of the Pilot Land Data System (PLDS), the Pilot Ocean Data System (PODS), the Pilot Climate Data System (PCDS), and the Pilot Planetary Data System (PPDS). The PLDS is likely to have the greatest communication needs of the various pilots. Communications issues and requirements are examined in the context of the mature PLDS as it may exist by 1990. The PLDS is seen as a distributed processing system linking resources at a number of NASA research centers and outside universities. Large image data sets, including LANDSAT scenes, are a major data type to be moved along the PLDS communications network. The unusually large size of these data files requires examining new technologies that may allow efficient and affordable communication at rates of up to 60 megabits per second. Potentially useful developments include portable satellite ground stations, and time division multiple access for sharing high-speed satellite links. Further pooling of communications for the various data systems into a single network would reduce costs and may make economic justification of access to the required data rates possible.

Likens, W. C.↗

Visual Data Comm: A Tool for Visualizing Data Communication in the Multi Sector Planner Study

Data comm is a new technology proposed in future air transport system as a potential tool to provide comprehensive data connectivity. It is a key enabler to manage 4D trajectory digitally, potentially resulting in improved flight times and increased throughput. Future concepts with data comm integration have been tested in a number of human-in-the-loop studies but analyzing the results has proven to be particularly challenging because future traffic environment in which data comm is fully enabled has assumed high traffic density, resulting in data set with large amount of information. This paper describes the motivation, design, current and potential future application of Visual Data Comm (VDC), a tool for visualizing data developed in Java using Processing library which is a tool package designed for interactive visualization programming. This paper includes an example of an application of VDC on data pertaining to the most recent Multi Sector Planner study, conducted at NASA s Airspace Operations Laboratory in 2009, in which VDC was used to visualize and interpret data comm activities

Lee, Hwasoo Eric↗

Simultaneous Retrieval of Temperature, Water Vapor and Ozone Atmospheric Profiles from IASI: Compression, De-noising, First Guess Retrieval and Inversion Algorithms

A fast temperature water vapor and ozone atmospheric profile retrieval algorithm is developed for the high spectral resolution Infrared Atmospheric Sounding Interferometer (IASI) space-borne instrument. Compression and de-noising of IASI observations are performed using Principal Component Analysis. This preprocessing methodology also allows, for a fast pattern recognition in a climatological data set to obtain a first guess. Then, a neural network using first guess information is developed to retrieve simultaneously temperature, water vapor and ozone atmospheric profiles. The performance of the resulting fast and accurate inverse model is evaluated with a large diversified data set of radiosondes atmospheres including rare events.

Aires, F.↗

Autoclass: An automatic classification system

The task of inferring a set of classes and class descriptions most likely to explain a given data set can be placed on a firm theoretical foundation using Bayesian statistics. Within this framework, and using various mathematical and algorithmic approximations, the AutoClass System searches for the most probable classifications, automatically choosing the number of classes and complexity of class descriptions. A simpler version of AutoClass has been applied to many large real data sets, has discovered new independently-verified phenomena, and has been released as a robust software package. Recent extensions allow attributes to be selectively correlated within particular classes, and allow classes to inherit, or share, model parameters through a class hierarchy. The mathematical foundations of AutoClass are summarized.

Stutz, John↗

Bayesian classification theory

The task of inferring a set of classes and class descriptions most likely to explain a given data set can be placed on a firm theoretical foundation using Bayesian statistics. Within this framework and using various mathematical and algorithmic approximations, the AutoClass system searches for the most probable classifications, automatically choosing the number of classes and complexity of class descriptions. A simpler version of AutoClass has been applied to many large real data sets, has discovered new independently-verified phenomena, and has been released as a robust software package. Recent extensions allow attributes to be selectively correlated within particular classes, and allow classes to inherit or share model parameters though a class hierarchy. We summarize the mathematical foundations of AutoClass.

Hanson, Robin↗

Software for Managing Parametric Studies

The Information Power Grid Virtual Laboratory (ILab) is a Practical Extraction and Reporting Language (PERL) graphical-user-interface computer program that generates shell scripts to facilitate parametric studies performed on the Grid. (The Grid denotes a worldwide network of supercomputers used for scientific and engineering computations involving data sets too large to fit on desktop computers.) Heretofore, parametric studies on the Grid have been impeded by the need to create control language scripts and edit input data files painstaking tasks that are necessary for managing multiple jobs on multiple computers. ILab reflects an object-oriented approach to automation of these tasks: All data and operations are organized into packages in order to accelerate development and debugging. A container or document object in ILab, called an experiment, contains all the information (data and file paths) necessary to define a complex series of repeated, sequenced, and/or branching processes. For convenience and to enable reuse, this object is serialized to and from disk storage. At run time, the current ILab experiment is used to generate required input files and shell scripts, create directories, copy data files, and then both initiate and monitor the execution of all computational processes.

Yarrow, Maurice↗

A Gaussian Process Enhancement to Linear Parameter Varying Models

Simulation and analysis for modern engineering systems now routinely requires the merging of multiple disciplines, physical-domains, time-scales, and data sets — all at ever increasing levels. These capabilities are especially needed in the domain of Advanced Air Mobility, where rapidly emerging vehicle designs are significantly more complex, while having to be both cost-effective and safe. To meet these engineering challenges, machine learning methods are an attractive option for merging models and data across multiple areas while providing uncertainty quantification and maintaining computational efficiency. This paper examines the use of Gaussian process machine learning to generalize and enhance the commonly used class of quasi-Linear Parameter Varying models for fast full-envelope simulation while also supporting control system design and analysis with model uncertainty. Gaussian process machine learning is selected because it: can fuse multiple data sets, enables an easy trade-off between data fitting and smoothing, provides model uncertainty quantification, scales well with increasing complexity, and does not generally require starting from a large training data set. To demonstrate the benefits of the approach, a robust stability analysis with Gaussian process uncertainty is shown for a NASA reference design of an electric quad-rotor air-taxi concept vehicle with motor parameter uncertainty.

Gaussian Process↗

SAVS: A Space Analysis and Visualization System

We propose to develop, test, demonstrate, and deliver to NASA a powerful and versatile data acquisition, manipulation, analysis and visualization system which will enhance scientific capabilities in the display and interpretation of diverse and distributed data within an integrated user-friendly environment. Our approach exploits existing technologies and combines three major elements into an easy-to-use interactive package: (1) innovative visualization software; (2) advanced database techniques; and (3) a rich set of mathematical and image processing tools. Visualization capabilities will include one-, two-, and three-dimensional displays, along with animation, compression, warping and slicing functions. Analysis tools will include generic mathematical and statistical techniques along with the ability to use large scale models for interactive interpretation of large volume data sets. Our system will be implemented on Sun and DEC UNIX workstations and on the Stardent Graphics Super computer. Our final deliverable will include complete documentation and a NASA/NSF-CDAW/SUNDIAL campaign demonstration.

Szuszczewicz, Edward P.↗

SAVS: A Space Analysis and Visualization System

We propose to develop, test, demonstrate, and deliver to NASA a powerful and versatile data acquisition, manipulation, analysis and visualization system which will enhance scientific capabilities in the display and interpretation of diverse and distributed data within an integrated user-friendly environment. Our approach exploits existing technologies and combines three major elements into an easy-to-use interactive package: (1) innovative visualization software; (2) advanced database techniques; and (3) a rich set of mathematical and image processing tools. Visualization capabilities will include one-, two-, and three-dimensinal displays, along with animation, compression, warping and slicing functions. Analysis tools will include generic mathematical and statistical techniques along with the ability to use large scale models for interactive interpretation of large volume data sets. Our system will be implemented on Sun and EC UNIX workstations and on the Stardent Graphics Supercomputer. Our final deliverable will include complete documentation and a NASA/NSF-CDAW/SUNDIAL campaign demonstration.

Szuszczewicz, Edward P.↗

A multiscale numerical study into the cascade of kinetic energy leading to severe local storms

The cascade of kinetic energy from macro- through mesoscales is studied on the basis of a nested grid system used to solve a set of nonlinear differential equations. The kinetic energy cascade and the concentration of vorticity through the hydrodynamic spectrum provide a means for predicting the location and intensity of severe weather from large-scale data sets. A mechanism described by the surface pressure tendency equation proves to be important in explaining how initial middle-tropospheric mass-momentum imbalances alter the low-level pressure field.

Paine, D. A.↗