Wgiss-54-11: Metrics for Cloud-Based Data Use
Explore the source record for details and available documents.
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Explore the source record for details and available documents.
No abstract provided
No abstract provided
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Opportunistic managed access to local in-network compute resources can improve the performance of distributed applications and reduce the dependence on shared network resources. Instead of backhauling application data to a centralized cloud data center for processing, networked services may be adaptively and continuously dispersed into shared compute resources that are closer to the source of need. While this approach has several benefits, support for mission-aware access to computation is often an afterthought, and is implemented as a brittle extension over traditional load-balancer solutions.In this work, we investigate the design of two priority-aware resource allocation strategies and two load-balancing dispatching strategies as first class citizens in an open-source dispersed computing middleware. We present a control theoretic analysis of these load-balancing primitives to identify weaknesses and strengths in our design, and recommend future directions. In parallel, we prototype two priority-aware allocation algorithms to validate our priority predictions. In initial experiments our prototype shows substantial gains in processing prioritized load. Finally, we make our source-code and experimental configurations open source.
A global survey of cloud particle size variations can provide crucial constraints on how cloud processes determine cloud liquid water contents and their variation with temperature, and further, may indicate the magnitude of aerosol effects on clouds. A method, based on a complete radiative transfer model for Advanced Very High Resolution Radiometer (AVHRR)-measured radiances, is described for retrieving cloud particle radii in liquid water clouds from satellite data currently available from the International Satellite Cloud Climatology Project. Results of sensitivity tests and validation studies provide error estimates. AVHRR data from NOAA-9 and NOAA-10 have been analyzed for January, April, July and October in 1987 and 1988. The results of this first survey reveal systematic continental and maritime differences and hemispheric contrasts that are indicative of the effects of associated aerosol concentration differences: cloud droplet radii in continental water clouds are about 2-3 micrometers smaller than in marine clouds, and droplet radii are about 1 micrometer smaller in marine clouds of the Northern Hemisphere than in the Southern Hemisphere. The height dependencies of cloud droplet radii in continental and marine clouds are also consistent with differences in the vertical profiles of aerosol concentration. Significant seasonal and diurnal variations of effective droplet radii are also observed, particularly at lower latitudes. Variations of the relationship between cloud optical thickness and droplet radii may indicate variations in cloud microphysical regimes.
Cloud computing is an approach and tool to support "Big Data".
Digitized optical detectors for cloud data processing onboard satellite using Fourier analysis
Cloud properties are critical for understanding the Earth’s radiation budget and cloud feedbacks. At NASA Langley Research Center, the Satellite ClOud and Radiative Property retrieval System (SatCORPS) provides real-time and historical analyses of clouds derived from Geostationary satellite (GEOsat) data for weather and climate applications. For the Clouds and the Earth’s Radiant Energy System (CERES) program, the global constellation of GEOsats has been analyzed since 2000 to help characterize and account for the diurnal cycle of clouds and their radiative impacts in the CERES climate data record. Obtaining consistent cloud properties over the GEOsat data record during the CERES era is a major objective but a significant challenge considering the diversity of imaging capabilities deployed during that time. The GEOsat data analysis approach for the current CERES Edition-4 (Ed4) data products was focused on accuracy and consistency with MODIS by employing as much spectral information as possible from each satellite. However, the inconsistent use of spectral information across GEOsats led to marked discontinuities in the spatial and temporal record of cloud properties that had to be accounted for post facto in downstream CERES processing. This paper reports progress in developing a new GEOsat analysis system for the next CERES edition (Ed5) that has potential to improve cross-platform consistency and continuity. In this approach, the spectral channel complement is limited to just 3-channels during daytime, ~0.65 µm (VIS), ~3.9 µm (NIR), and ~10.8 µm (IR), common to nearly all of the satellites in the record. At night, a 2-channel approach is taken with the NIR and IR, and ~6.7 µm bands that includes a machine learning approach for optically thick cloud properties. A tradeoff is the potential for reduced accuracy particularly using data from the more advanced satellites that have more spectral channels (e.g. SEVIRI, AHI and ABI) that are known to help improve thin cirrus detection, cloud-aerosol discrimination and estimates in other difficult conditions that challenge cloud remote sensing. The new continuity approach is applied to one month of global GEOSat data for each year of the CERES record since 2000 and compared with the Ed4 GEO and MODIS cloud property time series in order to evaluate the level of improved consistency in the GEOsat record and to assess the accuracy impacts. Cloud fraction will also be assessed with CALIPSO data. Outstanding issues and challenges will be discussed. The results are expected to guide future work needed to develop a more robust GEOsat cloud data record for CERES.
The effect of sensor spatial resolution on satellite-derived estimates of cloud fractional coverage is quantified on the basis of Landsat satellite radiance data. Cloud fraction is found to depend on cloud algorithm as much as it depends on sensor spatial resolution. Even for 28.5-m spatial resolution data, large cloud fraction differences exist between algorithms. Satellite cloud retrieval algorithms depend strongly on sensor spatial resolution and/or on the optical depth of the cloud field. When present, spatial resolution effects are small (less than 0.01) for pixel diameter less than 1/4 km and are large for pixel diameter larger than 1 km. The International Satellite Cloud Climatology Project bispectral threshold gives an increase in cloud fraction of 0.11 as spatial resolution degrades from 20 m to 8 km. The spatial coherence algorithm underestimates boundary layer cloud fraction by 0.18. The use of functional box counting and an assumption of fractal scale invariance overestimates the dependence of cloud fraction for spatial scales below 1 km.
The spatial and temporal variability of sea ice concentrations derived from passive microwave data is studied in conjunction with co-registered high resolution infrared and visible satellite data. Cloud free infrared and visible data provide surface temperature and large scale surface characteristics, respectively, that can be used to better understand regional and seasonal fluctuations in ice concentrations. Results from correlation analysis of ice concentration versus surface temperature data show the intuitively expected negative relationship but the strength in the relationship is unexpectedly very strong. In the Antarctic, the correlation is consistently very high spatially when yearly anomalies are used, and not so high in some areas when seasonal anomalies are used, especially during spring and summer. In the monthly anomalies, the correlation is also good, especially in dynamically active regions. The expanse in the anomalies in surface temperature are shown to go way beyond the sea ice regions into the open ocean and continental areas, suggesting strong atmospheric forcing. Weak correlations are normally found in highly consolidated areas, where large changes in temperature do not cause large changes in ice concentration on a short term, and in open ocean polynya areas, where the change in ice concentration may be cause by melt from the underside of the ice. In the Arctic, strong correlations between surface temperature and ice concentration are evident for all seasons except during the summer. In the summer, factors such as meltponding, surface wetness, and ice breakup, as detected by high resolution visible data, contributes to larger uncertainties in the determination of ice concentration and the lack of good correlation of the variables.
The Astromaterials Acquisition & Curation Office at NASA's Johnson Space Center (JSC) is the designated facility for curating all of NASA's extraterrestrial samples. Today, the suite of collections includes the lunar samples from the Apollo missions, cosmic dust particles falling into the Earth's atmosphere, meteorites collected in Antarctica, comet and interstellar dust particles from the Stardust mission, asteroid particles from Japan's Hayabusa mission, solar wind atoms collected during the Genesis mission, and space‐exposed hardware from several missions. To support planetary science research on these samples, JSC's Astromaterials Curation Office hosts NASA's Astromaterials Curation digital repository and data access portal [http://curator.jsc.nasa.gov/], providing descriptions of the missions and collections, and critical information about each individual sample. Our office is designing and implementing several informatics initiatives to better serve the planetary research community. First, we are re‐hosting the basic database framework by consolidating legacy databases for individual collections and providing a uniform access point for information (descriptions, imagery, classification) on all of our samples. Second, we continue to upgrade and host digital compendia that summarize and highlight published findings on the samples (e.g., lunar samples, meteorites from Mars). We host high resolution imagery of samples as it becomes available, including newly scanned images of historical prints from the Apollo missions. Finally we are creating plans to collect and provide new data, including 3D imagery, point cloud data, micro CT data, and external links to other data sets on selected samples. Together, these individual efforts will provide unprecedented digital access to NASA's Astromaterials, enabling preservation of the samples through more specific and targeted requests, and supporting new planetary science research and collaborations on the samples.
Total atmospheric radiation calculations from cloud data using actinometric instruments
In the year 2001, NASA will launch the satellite TRIANA that will be the first Earth observing mission to provide a continuous, full disk view of the sunlit Earth. As a part of the HPCC Program at NASA GSFC, we have started a project whose objectives are to develop and implement a 3D cloud data assimilation system, by combining TRIANA measurements with model simulation, and to produce accurate statistics of global cloud coverage as an important element of the Earth's climate. For simulation of the atmosphere within this project we are using the NCEP/NOAA operational Eta model. In order to compare TRIANA and the Eta model data on approximately the same grid without significant downscaling, the Eta model will be integrated at a resolution of about 15 km. The integration domain (from -70 to +70 deg in latitude and 150 deg in longitude) will cover most of the sunlit Earth disc and will continuously rotate around the globe following TRIANA. The cloud data assimilation is supposed to run and produce 3D clouds on a near real-time basis. Such a numerical setup and integration design is very ambitious and computationally demanding. Thus, though the Eta model code has been very carefully developed and its computational efficiency has been systematically polished during the years of operational implementation at NCEP, the current MPI version may still have problems with memory and efficiency for the TRIANA simulations. Within this work, we optimize a parallel version of the Eta model code on a Cray T3E and a network of PCs (theHIVE) in order to improve its overall efficiency. Our optimization procedure consists of introducing dynamically allocated arrays to reduce the size of static memory, and optimizing on a single processor by splitting loops to limit the number of streams. All the presented results are derived using an integration domain centered at the equator, with a size of 60 x 60 deg, and with horizontal resolutions of 1/2 and 1/3 deg, respectively. In accompanying charts we report the elapsed time, the speedup and the Mflops as a function of the number of processors for the non-optimized version of the code on the T3E and theHIVE. The large amount of communication required for model integration explains its poor performance on theHIVE. Our initial implementation of the dynamic memory allocation has contributed to about 12% reduction of memory but has introduced a 3% overhead in computing time. This overhead was removed by performing loop splitting in some of the high demanding subroutines. When the Eta code is fully optimized in order to meet the memory requirement for TRIANA simulations, a non-negligeable overhead may appear that may seriously affect the efficiency of the code. To alleviate this problem, we are considering implementation of a new algorithm for the horizontal advection that is computationally less expensive, and also a new approach for marching in time.
A cryogenic, 50 liter volume Planetary Cloud Simulation Chamber has been constructed to permit the laboratory study of the cloud compositions which are likely to be found in the atmospheres of the outer planets. On the basis of available data, clouds composed of water ice, carbon dioxide, and liquid and solid ammonia and methane, both pure and in various mixtures, have been generated. Cloud microphysical observations have been permitted through the use of a cloud particle slide injector and photomicrography. Viewports in the lower chamber have enabled the collection of cloud backscattering data using 633 and 838 nm laser light, including linear depolarization ratios and complete Stokes parameterization. The considerable technological difficulties associated with the collection of angular scattering patterns within the chamber, however, could not be completely overcome.