Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “large data sets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
NASA NTRS2020

Efficient Active Learning for New Domains

The promise of active learning is to reduce the number of labeled examples required by supervised machine learning algorithms. The largest potential benefits lie in entirely new domains, for which no labeled examples yet exist. Yet to date, most active learning studies are retroactive and demonstrate the benefits that could have been gained if active learning had been used. What are the barriers to true adoption and utilization of active learning? We focus on two: (1) the cold start or class discovery problem, in which active learning methods may struggle to make progress with zero labeled examples, and (2) the cost of having the classifier in the loop to select the next example to be labeled. We assess different active learning approaches in the context of these two barriers and conclude with recommendations for how to employ active learning in new domains. As an example, we report on the use of active learning on a large, novel data set of Mars surface images.

Lu, Steven↗
NASA NTRS2023

Machine Intelligence for Radiation Science: Summary of the Radiation Research Society 67th Annual Meeting Symposium

The era of high-throughput techniques created big data in the medical field and research disciplines. Machine intelligence (MI) approaches can overcome critical limitations on how those large-scale data sets are processed, analyzed, and interpreted. The 67 th Annual Meeting of the Radiation Research Society featured a symposium on MI approaches to highlight recent advancements in the radiation sciences and their clinical applications. This article summarizes three of those presentations regarding recent developments for metadata processing and ontological formalization, data mining for radiation outcomes in pediatric oncology, and imaging in lung cancer.

radiation↗
NASA NTRSDate not supplied

Hydrological Data at the NASA GES DISC: Current Capabilities and New Opportunities

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is one of twelve NASA Earth science data centers that document, process, archive and distribute data from Earth observation missions and projects. GES DISC maintains an archive of several hydrology datasets, including the Land Data Assimilation Systems (LDAS) and the Gravity Recovery and Climate Experiment (GRACE) Data Assimilation for Drought Monitoring (GRACE-DA-DM) data products. These datasets include model output of heat fluxes, rain, snow, soil temperature, soil moisture, and runoff; and observational forcing data, including surface pressure, temperature, precipitation, downward shortwave and longwave radiation, humidity, and wind. The temporal resolution of the hydrology data at GES DISC ranges from hourly to monthly, and spatial resolutions range from 0.1° to 1.0°. The GES DISC provides services which enable users to aggregate, temporally and spatially subset, regrid, and visualize archived data including the GES DISC Subsetter, Hydrology Data Rods, and the Geospatial Interactive Online Visualization and Analysis Infrastructure (GIOVANNI). The Hydrology Data Rods service optimally reorganizes large hydrological data sets as extended time series, providing more efficient access for the hydrological community. The time series data (aka “data rods”) were integrated into hydrology community tools, such as the Data Rods Explorer on HydroShare. Furthermore, the GES DISC is in the process of migrating its data and services to the cloud. Hydrological data available at the GES DISC are now available in the Amazon Web Services (AWS) cloud (us-west-2 region) providing users Direct S3 data access and the capability for cloud computing operations. In this presentation, the hydrology data products and services currently available at the GES DISC will be summarized. Also discussed are the migration to the cloud, user support through this transition, and the status of migrating the data rods service to the cloud.

Ashley Heath↗
NASA NTRS2011

Large Terrain Modeling and Visualization for Planets

Physics-based simulations are actively used in the design, testing, and operations phases of surface and near-surface planetary space missions. One of the challenges in realtime simulations is the ability to handle large multi-resolution terrain data sets within models as well as for visualization. In this paper, we describe special techniques that we have developed for visualization, paging, and data storage for dealing with these large data sets. The visualization technique uses a real-time GPU-based continuous level-of-detail technique that delivers multiple frames a second performance even for planetary scale terrain model sizes.

digital elevation map↗
NASA NTRS2012

Investigation into Cloud Computing for More Robust Automated Bulk Image Geoprocessing

Geospatial resource assessments frequently require timely geospatial data processing that involves large multivariate remote sensing data sets. In particular, for disasters, response requires rapid access to large data volumes, substantial storage space and high performance processing capability. The processing and distribution of this data into usable information products requires a processing pipeline that can efficiently manage the required storage, computing utilities, and data handling requirements. In recent years, with the availability of cloud computing technology, cloud processing platforms have made available a powerful new computing infrastructure resource that can meet this need. To assess the utility of this resource, this project investigates cloud computing platforms for bulk, automated geoprocessing capabilities with respect to data handling and application development requirements. This presentation is of work being conducted by Applied Sciences Program Office at NASA-Stennis Space Center. A prototypical set of image manipulation and transformation processes that incorporate sample Unmanned Airborne System data were developed to create value-added products and tested for implementation on the "cloud". This project outlines the steps involved in creating and testing of open source software developed process code on a local prototype platform, and then transitioning this code with associated environment requirements into an analogous, but memory and processor enhanced cloud platform. A data processing cloud was used to store both standard digital camera panchromatic and multi-band image data, which were subsequently subjected to standard image processing functions such as NDVI (Normalized Difference Vegetation Index), NDMI (Normalized Difference Moisture Index), band stacking, reprojection, and other similar type data processes. Cloud infrastructure service providers were evaluated by taking these locally tested processing functions, and then applying them to a given cloud-enabled infrastructure to assesses and compare environment setup options and enabled technologies. This project reviews findings that were observed when cloud platforms were evaluated for bulk geoprocessing capabilities based on data handling and application development requirements.

Brown, Richard B.↗
NASA NTRS2001

Estimation of comprehensive forest variable sets from multiparameter SAR data over a large area with diverse species

Polarimetric and multifrequency data from the NASA/JPL airborne synthetic aperture radar (AIRSAR) have been used in a multi-tier estimation algorithm to calculate a comprehensive set of forest canopy properties including branch layer moisture and thickness, trunk density, trunk water content and diameter, trunk height, and subcanapy soil moisture. The estimation algorithm takes advantage of species-specific allometric relations, and is applied to a 100Km x 100Km area in the Canadian boreal region containing many different vegetation species types. The results show very good agreement with ground measurements taken at several focused and auxiliary study sites. This paper expands on the results reported in [1] and applies the algorithm on the regional scale.

variable estimation forestry multiparameter SAR↗
NASA NTRS2004

Solving Large Problems Quickly: Progress in 2001-2003

This document describes the progress we have made and the lessons we have learned in 2001 through 2003 under the NASA grant entitled "Solving Important Problems Faster". The long-term goal of this research is to accelerate large, irregular scientific applications which have enormous data sets and which are difficult to parallelize. To accomplish this goal, we are exploring two complementary techniques: (i) using compiler-inserted prefetching to automatically hide the I/O latency of accessing these large data sets from disk; and (ii) using thread-level data speculation to enable the optimistic parallelization of applications despite uncertainty as to whether data dependences exist between the resulting threads which would normally make them unsafe to execute in parallel. Overall, we made significant progress in 2001 through 2003, and the project has gone well.

Mowry, Todd C.↗
NASA NTRS1990

Studying marine stratus with large eddy simulation

Data sets from field experiments over the stratocumulus regime may include complications from larger scale variations, decoupled cloud layers, diurnal cycle, or entrainment instability, etc. On top of the already complicated turbulence-radiation-condensation processes within the cloud-topped boundary layer (CTBL), these complexities may sometimes make interpretation of the data sets difficult. To study these processes, a better understanding is needed of the basic processes involved in the prototype CTBL. For example, is cloud top radiative cooling the primary source of the turbulent kinetic energy (TKE) within the CTBL. Historically, laboratory measurements have played an important role in addressing the turbulence problems. The CTBL is a turbulent field which is probably impossible to generate in laboratories. Large eddy simulation (LES) is an alternative way of 'measuring' the turbulent structure under controlled environments, which allows the systematic examination of the basic physical processes involved. However, there are problems with the LES approach for the CTBL. The LES data need to be consistent with the observed data. The LES approach is discussed, and results are given which provide some insights into the simulated turbulent flow field. Problems with this approach for the CTBL and information from the FIRE experiment needed to justify the LES results are discussed.

Moeng, Chin-Hoh↗
NASA NTRS2003

Program for User-Friendly Management of Input and Output Data Sets

A computer program manages large, hierarchical sets of input and output (I/O) parameters (typically, sequences of alphanumeric data) involved in computational simulations in a variety of technological disciplines. This program represents sets of parameters as structures coded in object-oriented but otherwise standard American National Standards Institute C language. Each structure contains a group of I/O parameters that make sense as a unit in the simulation program with which this program is used. The addition of options and/or elements to sets of parameters amounts to the addition of new elements to data structures. By association of child data generated in response to a particular user input, a hierarchical ordering of input parameters can be achieved. Associated with child data structures are the creation and description mechanisms within the parent data structures. Child data structures can spawn further child data structures. In this program, the creation and representation of a sequence of data structures is effected by one line of code that looks for children of a sequence of structures until there are no more children to be found. A linked list of structures is created dynamically and is completely represented in the data structures themselves. Such hierarchical data presentation can guide users through otherwise complex setup procedures and it can be integrated within a variety of graphical representations.

Klimeck, Gerhard↗
NASA NTRS2024

SOFIA/HAWC+ Far-Infrared Polarimetric Large-area CMZ Exploration Survey. III. Full Survey Data Set

We present the second data release (DR2) of the Far-InfraREd Polarimetric Large-Area CMZ Exploration (FIREPLACE) survey. This survey utilized the Stratospheric Observatory for Infrared Astronomy High-resolution Airborne Wideband Camera plus instrument at 214 μm (E band) at a resolution of 19"6 to observe thermal polarized dust emission throughout the Central Molecular Zone (CMZ). DR2 consists of observations obtained in 2022 covering the region of the CMZ extending from the Brick to the Sgr C molecular clouds (corresponding to a 1° × 0°.75 region of the sky). We combine DR2 with the first FIREPLACE data release (DR1) to obtain full coverage of the CMZ (a 1°.5 × 0°.75 region of the sky). After applying total and polarized intensity significance cuts on the full FIREPLACE data set, we obtain ∼64,000 Nyquist-sampled polarization pseudovectors. The distribution of polarization pseudovectors confirms a bimodal distribution in the CMZ magnetic field orientations, recovering field components that are oriented predominantly parallel or perpendicular to the Galactic plane. This distribution of orientations is similar to what was observed in DR1 and other studies. We also inspect the magnetic fields toward a set of prominent CMZ molecular clouds (the Brick, Three Little Pigs, 50 and 20 km s −1 clouds, circumnuclear disk, CO 0.02-0.02, and Sgr C), revealing spatially varying magnetic fields having orientations that generally trace the total intensity morphologies of the clouds. We find evidence that compression from stellar winds and shear from tidal forces are prominent mechanisms influencing the structure of the magnetic fields.

Molecular clouds↗
NASA NTRS1992

On the use of stochastic process-based methods for the analysis of hyperspectral data

Further development in remote sensing technology requires refinement of information system design aspects, i.e., the ability to specify precisely the data to collect and the means to extract increasing amounts of information from the increasingly rich and complex data stream created. One of the principal directions of advance is that data from much larger numbers of spectral bands can be collected, but with significantly increased signal-to-noise ratio. The theory of stochastic or random processes may be applied to the modeling of second-order variations. A multispectral data set with a large number of spectral bands is analyzed using standard pattern recognition techniques. The data were classified using first a single spectral feature, then two, and continuing on with greater and greater numbers of features. Three different classification schemes are used: a standard maximum likelihood Gaussian scheme; the same approach with the mean values of all classes adjusted to be the same; and the use of a minimum distance to means scheme such that mean differences are used.

Landgrebe, David A.↗
NASA NTRS2005

Impact of CAMEX-4 Data Sets for Hurricane Forecasts using a Global Model

This study explores the impact on hurricane data assimilation and forecasts from the use of dropsondes and remote-sensed moisture profiles from the airborne Lidar Atmospheric Sensing Experiment (LASE) system. We show that the use of these additional data sets, above those from the conventional world weather watch, has a positive impact on hurricane predictions. The forecast tracks and intensity from the experiments show a marked improvement compared to the control experiment where such data sets were excluded. A study of the moisture budget in these hurricanes showed enhanced evaporation and precipitation over the storm area. This resulted in these data sets making a large impact on the estimate of mass convergence and moisture fluxes, which were much smaller in the control runs. Overall this study points to the importance of high vertical resolution humidity data sets for improved model results. We note that the forecast impact from the moisture profiling data sets for some of the storms is even larger than the impact from the use of dropwindsonde based winds.

Kamineni, Rupa↗
NASA NTRS2015

Simulations of Cloud-Radiation Interaction Using Large-Scale Forcing Derived from the CINDY/DYNAMO Northern Sounding Array

The recently completed CINDY/DYNAMO field campaign observed two Madden-Julian oscillation (MJO) events in the equatorial Indian Ocean from October to December 2011. Prior work has indicated that the moist static energy anomalies in these events grew and were sustained to a significant extent by radiative feedbacks. We present here a study of radiative fluxes and clouds in a set of cloud-resolving simulations of these MJO events. The simulations are driven by the large-scale forcing data set derived from the DYNAMO northern sounding array observations, and carried out in a doubly periodic domain using the Weather Research and Forecasting (WRF) model. Simulated cloud properties and radiative fluxes are compared to those derived from the S-PolKa radar and satellite observations. To accommodate the uncertainty in simulated cloud microphysics, a number of single-moment (1M) and double-moment (2M) microphysical schemes in the WRF model are tested. The 1M schemes tend to underestimate radiative flux anomalies in the active phases of the MJO events, while the 2M schemes perform better, but can overestimate radiative flux anomalies. All the tested microphysics schemes exhibit biases in the shapes of the histograms of radiative fluxes and radar reflectivity. Histograms of radiative fluxes and brightness temperature indicate that radiative biases are not evenly distributed; the most significant bias occurs in rainy areas with OLR less than 150 W/ cu sq in the 2M schemes. Analysis of simulated radar reflectivities indicates that this radiative flux uncertainty is closely related to the simulated stratiform cloud coverage. Single-moment schemes underestimate stratiform cloudiness by a factor of 2, whereas 2M schemes simulate much more stratiform cloud.

Indian Ocean↗
NASA NTRS1988

Error dynamics and relative orbital uncertainty of the Magellan spacecraft

Relative error is discussed in light of a proposed orbiter mission to Venus. It is hypothesized that the relative (position) error of the orbiting spacecraft between disjoint data sets will be large because of residual uncertainties of the absolute estimates and their decorrelation caused by succeeding estimates. A method is proposed to limit relative error by inducing correlations between the estimates on successive data arcs. These correlations are induced by overlapping the data arcs. Covariance analyses of the data sets are undertaken at two distinct mapping times, yielding the relative error between those mapping times. The technique is applied to a mosaicking problem on a specific day in the mission.

Haw, R. J.↗
NASA NTRS2001

A strategy for compression and analysis of massive geophysical data sets

This paper describes a method for summaraizing data in a way that approximately preserves high-resolution data structure while reducing data volume and maintaining global integrity of very large, remote sensing data sets. The method is under development for one of Terra's instruments, the Multi-angle Imaging SpectroRadiometer (MISR).

geophysical data sets compression↗
NASA NTRS2006

Status and Plans for the WCRP/GEWEX Global Precipitation Climatology Project (GPCP)

Status and plans for GPCP are presented along with scientific findings from the current data set. Global and large regional rainfall variations and possible long-term changes are examined using the 26-year (1979-2004) monthly dataset from the Global Precipitation Climatology Project (GPCP). One emphasis is to discriminate among the variations due to ENSO, volcanic events and possible long-term changes. Although the global change of precipitation in the data set is near zero, the data set does indicate an upward trend (0.13 mm/day/25yr) and a downward trend (-0.06 mm/day/25yr) over tropical oceans and lands (25S-25N), respectively. This corresponds to a 4% increase (ocean) and 2% decrease (land) during this time period. Simple techniques are derived to attempt to eliminate variations due to ENSO and major volcanic eruptions in the Tropics. Using only annual values two "volcano years" are determined by examining ocean-land coupled variations in precipitation related to ENSO and other phenomena. The outlier years coincide with Pinatubo and El Chicon eruptions. The ENSO signal is reduced by deriving mean ocean and land values for El Nino, La Nina and neutral conditions based on Nino 3.4 SST and normalizing the annual ocean and land precipitation to the neutral set of cases. The impact of the two major volcanic eruptions over the past 25 years is estimated to be about a 5% reduction in tropical rainfall. The modified data set (with ENSO and volcano effect at least partially removed) retains the same approximate linear change slopes over the data set period, but with reduced variance leading to significance tests with results in the 90-95% range. Intercomparisons between the GPCP, SSM/I (1988-2004), and TRMM (1998-2004) satellite rainfall products and alternate gauge analyses over land are made to attempt to increase or decrease confidence in the changes seen in the GPCP analysis.

Adkerm Robert F.↗
NASA NTRS2013

Mapping Impervious Surfaces Globally at 30m Resolution Using Global Land Survey Data

Impervious surfaces, mainly artificial structures and roads, cover less than 1% of the world's land surface (1.3% over USA). Regardless of the relatively small coverage, impervious surfaces have a significant impact on the environment. They are the main source of the urban heat island effect, and affect not only the energy balance, but also hydrology and carbon cycling, and both land and aquatic ecosystem services. In the last several decades, the pace of converting natural land surface to impervious surfaces has increased. Quantitatively monitoring the growth of impervious surface expansion and associated urbanization has become a priority topic across both the physical and social sciences. The recent availability of consistent, global scale data sets at 30m resolution such as the Global Land Survey from the Landsat satellites provides an unprecedented opportunity to map global impervious cover and urbanization at this resolution for the first time, with unprecedented detail and accuracy. Moreover, the spatial resolution of Landsat is absolutely essential to accurately resolve urban targets such a buildings, roads and parking lots. With long term GLS data now available for the 1975, 1990, 2000, 2005 and 2010 time periods, the land cover/use changes due to urbanization can now be quantified at this spatial scale as well. In the Global Land Survey - Imperviousness Mapping Project (GLS-IMP), we are producing the first global 30 m spatial resolution impervious cover data set. We have processed the GLS 2010 data set to surface reflectance (8500+ TM and ETM+ scenes) and are using a supervised classification method using a regression tree to produce continental scale impervious cover data sets. A very large set of accurate training samples is the key to the supervised classifications and is being derived through the interpretation of high spatial resolution (approx. 2 m or less) commercial satellite data (Quickbird and Worldview2) available to us through the unclassified archive of the National Geospatial Intelligence Agency (NGA). For each continental area several million training pixels are derived by analysts using image segmentation algorithms and tools and then aggregated to the 30m resolution of Landsat. Here we will discuss the production/testing of this massive data set for Europe, North and South America and Africa, including assessments of the 2010 surface reflectance data. This type of analysis is only possible because of the availability of long term 30m data sets from GLS and shows much promise for integration of Landsat 8 data in the future.

global land survey↗
NASA NTRS1985

Dynamics explorer guest investigator

The use of Dynamics Explorer (DE) data sets to model the auroral inputs for the time dependent ionospheric model (TDIM) is reported. The modelling requires DE-1 SAI images and simultaneous DE-2 LAPI particle data. The data sets allow the large scale relative auroral variations and local absolute energy flexes and characteristics energies to be defined. The images enabled global scale auroral modelling with 12 min. time resolution and the LAPI data presented a detailed energy flux and characteristic energy calibration of the image model. The auroral model is used as an input to the TDIM and studies ionospheric storms.

Sojka, J. J.↗