Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “native analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

File Specification for GEOS-5 FP (Forward Processing)

The GEOS-5 FP Atmospheric Data Assimilation System (GEOS-5 ADAS) uses an analysis developed jointly with NOAA's National Centers for Environmental Prediction (NCEP), which allows the Global Modeling and Assimilation Office (GMAO) to take advantage of the developments at NCEP and the Joint Center for Satellite Data Assimilation (JCSDA). The GEOS-5 AGCM uses the finite-volume dynamics (Lin, 2004) integrated with various physics packages (e.g, Bacmeister et al., 2006), under the Earth System Modeling Framework (ESMF) including the Catchment Land Surface Model (CLSM) (e.g., Koster et al., 2000). The GSI analysis is a three-dimensional variational (3DVar) analysis applied in grid-point space to facilitate the implementation of anisotropic, inhomogeneous covariances (e.g., Wu et al., 2002; Derber et al., 2003). The GSI implementation for GEOS-5 FP incorporates a set of recursive filters that produce approximately Gaussian smoothing kernels and isotropic correlation functions. The GEOS-5 ADAS is documented in Rienecker et al. (2008). More recent updates to the model are presented in Molod et al. (2011). The GEOS-5 system actively assimilates roughly 2 × 10(exp 6) observations for each analysis, including about 7.5 × 10(exp 5) AIRS radiance data. The input stream is roughly twice this volume, but because of the large volume, the data are thinned commensurate with the analysis grid to reduce the computational burden. Data are also rejected from the analysis through quality control procedures designed to detect, for example, the presence of cloud. To minimize the spurious periodic perturbations of the analysis, GEOS-5 FP uses the Incremental Analysis Update (IAU) technique developed by Bloom et al. (1996). More details of this procedure are given in Appendix A. The assimilation is performed at a horizontal resolution of 0.3125-degree longitude by 0.25- degree latitude and at 72 levels, extending to 0.01 hPa. All products are generated at the native resolution of the horizontal grid. The majority of data products are time-averaged, but four instantaneous products are also available. Hourly data intervals are used for two-dimensional products, while 3-hourly intervals are used for three-dimensional products. These may be on the model's native 72-layer vertical grid or at 42 pressure surfaces extending to 0.1 hPa. This document describes the gridded output files produced by the GMAO near real-time operational FP, using the most recent version of the GEOS-5 assimilation system. Additional details about variables listed in this file specification can be found in a separate document, the GEOS-5 File Specification Variable Definition Glossary. Documentation about the current access methods for products described in this document can be found on the GMAO products page: http://gmao.gsfc.nasa.gov/products/.

GSI↗

Ellipsometric studies of sputtered silicon nitride on n-GaAs

Ellipsometry, which is the study of optical systems through their effect on polarized light, is a nondestructive technique offering extremely high sensitivity and in situ operation. One of the intriguing possibilities is to make use of the high sensitivity of ellipsometry for studying the very thin interface between an insulating film and a semiconductor substrate. Researchers have been especially interested in native or anodic oxides. In a recent study, Aspnes and Theeten used ellipsometry to study the interface between silicon and its thermally grown oxide, SiO2. These authors were able to conclude that the interface consisted of a 7 +/- 2 A graded region of atomically mixed silicon and oxygen. In this paper we report both analysis and experimental results for the system: sputtered silicon nitride on (100) n-type gallium arsenide.

Peng, Y. K.↗

Energy balance components from six sites in a native prairie

The surface-flux stations investigated are of similar design and operation, but are located on different slopes and aspects and provided data with significant variations. The largest difference between sites is noticed in the data for soil heat-flux density, and the general variability is theorized to be related to treatment and locations. The number of surface-flux stations required for the prairie topography can be inferred from the results of the data analysis.

Fritschen, Leo J.↗

Metadata Evaluation and Improvement: Evolving Analysis and Reporting

ESIP Community members create and manage a large collection of environmental datasets that span multiple decades, the entire globe, and many parts of the solar system. Metadata are critical for discovering, accessing, using and understanding these data effectively and ESIP community members have successfully created large collections of metadata describing these data. As part of the White House Big Earth Data Initiative (BEDI), ESDIS has developed a suite of tools for evaluating these metadata in native dialects with respect to recommendations from many organizations. We will describe those tools and demonstrate evolving techniques for sharing results with data providers.

metadata recommendations↗

An Examination of Issues Related to a Europa Subsurface Component for the JIMO Mission

The Galileo Europa data set served to revolutionize our view of Europa. In particular the strong evidence of a large, cold, salty Ocean beneath 5-30 km of ice has profoundly altered the significance of Europa in our thinking, especially of context of habitability in the solar system. While much remains to be learned from spacecraft observations of several sorts, there are significant questions answerable only by in-situ techniques; these relate to the formation of Europa, the nature of its ocean, and the prospects for life in its ocean, sediments, and ice. We feel that wide-ranging discussion of an in-situ subsurface mission to Europa, as part of JIMO, should proceed. The science objective of the mission is to characterize the icy shell of Europa to resolve its provenance, estimate the composition of brine of the Europa ocean, and search for evidence of Earth-like life. Probably anyone would agree that an in-situ mission to Europa would be of great value, but he or she would also immediately take the position that such a mission is utterly impractical. We take the position here of defining the least complex mission that can nonetheless justify its cost and to argue that such a mission is realistic enough that it should be seriously considered. Our mission thinking has been: 1) Soft landing. A soft lander is required on a site sufficiently flat to offer a stable platform; no further site selectivity is required. 2) Subsurface exploration. The Europa subsurface must be examined. Surficial processes on Europa arguably have exposed the upper 200 m of shell to chemical effects from the Jovian radiation belts as well as cometary infall, etc; to examine native ice we must descend below that point to, for discussion, 300 m. At that depth we argue that the ice is characteristic of ice at depth and possibly is effectively sea ice. 3) Science data. A few simple measurements at various depths and at 300 m constitute a scientifically successful mission. Measurements would include analysis of meltwater for a few inorganic ions and amino acids and an optical examination of the borehole wall. 4) Communication. Transmission of data to an orbiter is essential, but we will constrain the landed mission to a daily communication over a few days. 5) Subsurface access. Drilling to 300 m is a significant challenge; it can be addressed by several means: Thermal Probe (Cryobot) which permits water to refreeze above the vehicle. This is our tentative choice with plutonium as the fuel to generate thermal energy for drilling and electrical power for operations. Open Hole Drill, a thermal system in which the meltwater is removed for greater thermal efficiency. Meltwater removal on Europa is both a complexity and a risk, but analysis is improved. Mechanical Drilling in which cutting or grinding generates ice chips which are removed. This is too complex at Europa temperatures. The measurement objectives for the mission will be: Obj. 1: Determine the concentration of simple inorganic salts in the Europa Ice Shell and, by extrapolation, of the ocean. These data will also validate spaceborne sensors. Obj. 2: Determine the nature and abundance of amino acids in the ice such that cometary infall material in the upper ice can be compared to material at depth. Obj. 3: Optically examine the ice to resolve inclusion structure, particulate content, and stratification. Access to 300 m depth is a significant if not audacious plan; we are aware that this has not been done on any planetary body. Our approach is the use of a plutonium heat source; to overcome Europa's surface temperature and to melt ice a significant amount of plutonium is needed, and significant shielding and other protective steps will be required. The quantity of plutonium is a key concern. The mission will require subsurface collection and processing of samples for in situ analysis, calling for a miniature, high pressure micro-sampling system designed to meet needs of instruments that require low presses for operation. The inlet system itself collects a micro-sample in the external high pressure environment, then transfers it into a protected low pressure environment for analysis.

Carsey, F. D.↗

Python-Based Scientific Analysis and Visualization of Precipitation Systems at NASA Marshall Space Flight Center

At NASA Marshall Space Flight Center (MSFC), Python is used several different ways to analyze and visualize precipitating weather systems. A number of different Python‐based software packages have been developed, which are available to the larger scientific community. The approach in all these packages is to utilize pre‐existing Python modules as well as to be object‐oriented and scalable. The first package that will be described and demonstrated is the Python Advanced Microwave Precipitation Radiometer (AMPR) Data Toolkit, or PyAMPR for short. PyAMPR reads geolocated brightness temperature data from any flight of the AMPR airborne instrument over its 25‐year history into a common data structure suitable for user‐defined analyses. It features rapid, simplified (i.e., one line of code) production of quick‐look imagery, including Google Earth overlays, swath plots of individual channels, and strip charts showing multiple channels at once. These plotting routines are also capable of significant customization for detailed, publication‐ready figures. Deconvolution of the polarization‐varying channels to static horizontally and vertically polarized scenes is also available. Examples will be given of PyAMPR's contribution toward real‐time AMPR data display during the Integrated Precipitation and Hydrology Experiment (IPHEx), which took place in the Carolinas during May‐June 2014. The second software package is the Marshall Multi‐Radar/Multi‐Sensor (MRMS) Mosaic Python Toolkit, or MMM‐Py for short. MMM‐Py was designed to read, analyze, and display three‐dimensional national mosaicked reflectivity data produced by the NOAA National Severe Storms Laboratory (NSSL). MMM‐Py can read MRMS mosaics from either their unique binary format or their converted NetCDF format. It can also read and properly interpret the current mosaic design (4 regional tiles) as well as mosaics produced prior to late July 2013 (8 tiles). MMM‐Py can easily stitch multiple tiles together to provide a larger regional or national picture of precipitating weather systems. Composites, horizontal and vertical crosssections, and combinations thereof are easily displayed using as little as one line of code. MMM‐Py can also write to the native MRMS binary format, and sub‐sectioning of tiles (or multiple stitched tiles) is anticipated to be in place by the time of this meeting. Thus, MMM‐Py also can be used to power the creation of custom mosaics for targeted regional studies. Overlays of other data (e.g., lightning observations) are easily accomplished. Demonstrations of MMM‐Py, including the creation of animations, will be shown. Finally, Marshall has done significant work to interface Python‐based analysis routines with the U.S. Department of Energy's Py‐ART software package for radar data ingest, processing, and analysis. One example of this is the Python Turbulence Detection Algorithm (PyTDA), an MSFC‐based implementation of the National Center for Atmospheric Research (NCAR) Turbulence Detection Algorithm (NTDA) for the purposes of convective‐scale analysis, situational awareness, and forensic meteorology. PyTDA exploits Py‐ART's radar data ingest routines and data model to rapidly produce aviation‐relevant turbulence estimates from Doppler radar data. Work toward processing speed optimization and better integration within the Py‐ART framework will be highlighted. Python‐based analysis within the Py‐ART framework is also being done for new research related to intercomparison of ground‐based radar data with satellite estimates of ocean winds, as well as research on the electrification of pyrocumulus clouds.

Lang, Timothy J.↗

GammaCore: The Compton Observatory research environment

The Compton Observatory Science Support Center (COSSC) is developing a coherent analysis environment for the analysis of Compton and other gamma-ray astronomy data. This environment, GammaCore, allows the astronomer to access the data analysis systems developed at the Principal Investigator (PI) sites for the four Compton Observatory instruments. In addition users have access to standard astronomical tools such as IRAF, IDL, and XANADU. The user interface of GammaCore is the AGCL (AnswerGarden Command Language), developed at the AXAF Science Center. The parameter interface supported by the AGCL allows GammaCore to access all PI software systems in a uniform fashion. These systems are quite different, having been developed independently on heterogeneous systems without much concern for general portability. The data kibitzer concept, where a window running in a specific PI environment is controlled by the AGCL, has been used extensively. Users can choose to view what is going on in the native environment, to use the window to control PI software directly, or to ignore the PI systems entirely and to work only through the homogeneous AGCL interface. Software developed at the COSSC is also integrated within GammaCore. Extensive facilities for conversions of PI data formats to and from FITS have been developed. Access to the Compton data archive and catalogs will also be completely integrated with the GammaCore. Users can retrieve any publicly available Compton data. This paper examines the issues that have arisen in attempting to meld these widely diverse systems. The advantages and limitations of the parameter interface and the kibitzer are discussed along with issues of data portability, documentation, and the feasibility of multi-instrument analysis. Limited capabilities are now available within GammaCore with significant enhancements planned over the coming year. An implementation including all PI systems will be available within that time. Instructions on how to access GammaCore and how to get more information are given.

Mcglynn, T. A.↗

International Satellite Cloud Climatology Project (ISCCP) Stage DX Pixel Level Cloud Product - Revised Algorithm in Native (NAT) Format (ISCCP_DX_NAT)

Since 1983 an international group of institutions has collected and analyzed satellite radiance measurements from up to five geostationary and two polar orbiting satellites to infer the global distribution of cloud properties and their diurnal, seasonal and interannual variations. The primary focus of the first phase of the project (1983-1995) was the elucidation of the role of clouds in the radiation budget (top of the atmosphere and surface). In the second phase of the project (1995 onwards) the analysis also concerns improving understanding of clouds in the global hydrological cycle. [Location=TROPOSPHERE] [Temporal_Coverage: Start_Date=1983-07-01; Stop_Date=] [Spatial_Coverage: Southernmost_Latitude=-90; Northernmost_Latitude=90; Westernmost_Longitude=-180; Easternmost_Longitude=180] [Data_Resolution: Latitude_Resolution=30 Km; Longitude_Resolution=30 Km; Temporal_Resolution=3 Hourly].

CLOUD ICE↗

International Satellite Cloud Climatology Project (ISCCP) Stage B3 Reduced Radiances in Native (NAT) Format (ISCCP_B3_NAT)

Since 1983 an international group of institutions has collected and analyzed satellite radiance measurements from up to five geostationary and two polar orbiting satellites to infer the global distribution of cloud properties and their diurnal, seasonal and interannual variations. The primary focus of the first phase of the project (1983-1995) was the elucidation of the role of clouds in the radiation budget (top of the atmosphere and surface). In the second phase of the project (1995 onwards) the analysis also concerns improving understanding of clouds in the global hydrological cycle. [Location=TROPOSPHERE] [Temporal_Coverage: Start_Date=1983-07-01; Stop_Date=] [Spatial_Coverage: Southernmost_Latitude=-90; Northernmost_Latitude=90; Westernmost_Longitude=-180; Easternmost_Longitude=180] [Data_Resolution: Latitude_Resolution=30 Km; Longitude_Resolution=30 Km; Temporal_Resolution=3 Hourly].

CLOUD PARTICLE SIZE↗

International Satellite Cloud Climatology Project (ISCCP) Ice Snow Product in Native (NAT) Format (ISCCP_ICESNOW_NAT)

Since 1983 an international group of institutions has collected and analyzed satellite radiance measurements from up to five geostationary and two polar orbiting satellites to infer the global distribution of cloud properties and their diurnal, seasonal and interannual variations. The primary focus of the first phase of the project (1983-1995) was the elucidation of the role of clouds in the radiation budget (top of the atmosphere and surface). In the second phase of the project (1995 onwards) the analysis also concerns improving understanding of clouds in the global hydrological cycle. [Location=GLOBAL] [Temporal_Coverage: Start_Date=1983-07-01; Stop_Date=] [Spatial_Coverage: Southernmost_Latitude=-90; Northernmost_Latitude=90; Westernmost_Longitude=-180; Easternmost_Longitude=180] [Data_Resolution: Latitude_Resolution=112 Km; Longitude_Resolution=112 Km; Temporal_Resolution=5-day].

SNOW COVER↗

International Satellite Cloud Climatology Project (ISCCP) TIROS Operational Vertical Sounder (TOVS) Product in Native (NAT) Format (ISCCP_TOVS_NAT)

Since 1983 an international group of institutions has collected and analyzed satellite radiance measurements from up to five geostationary and two polar orbiting satellites to infer the global distribution of cloud properties and their diurnal, seasonal and interannual variations. The primary focus of the first phase of the project (1983-1995) was the elucidation of the role of clouds in the radiation budget (top of the atmosphere and surface). In the second phase of the project (1995 onwards) the analysis also concerns improving understanding of clouds in the global hydrological cycle. [Location=GLOBAL] [Temporal_Coverage: Start_Date=1983-07-01; Stop_Date=] [Spatial_Coverage: Southernmost_Latitude=-90; Northernmost_Latitude=90; Westernmost_Longitude=-180; Easternmost_Longitude=180] [Data_Resolution: Latitude_Resolution=280 degree; Longitude_Resolution=280 degree; Temporal_Resolution=Daily - < Monthly].

PRECIPITABLE WATER PROFILES↗

International Satellite Cloud Climatology Project (ISCCP) Stage D1 3-Hourly Cloud Product - Revised Algorithm in Native (NAT) Format (ISCCP_D1_NAT)

Since 1983 an international group of institutions has collected and analyzed satellite radiance measurements from up to five geostationary and two polar orbiting satellites to infer the global distribution of cloud properties and their diurnal, seasonal and interannual variations. The primary focus of the first phase of the project (1983-1995) was the elucidation of the role of clouds in the radiation budget (top of the atmosphere and surface). In the second phase of the project (1995 onwards) the analysis also concerns improving understanding of clouds in the global hydrological cycle. [Location=TROPOSPHERE] [Temporal_Coverage: Start_Date=1983-07-01; Stop_Date=] [Spatial_Coverage: Southernmost_Latitude=-90; Northernmost_Latitude=90; Westernmost_Longitude=-180; Easternmost_Longitude=180] [Data_Resolution: Latitude_Resolution=280 Km; Longitude_Resolution=280 Km; Temporal_Resolution=3 Hourly].

GRID DATA↗

NASA GES DISC DAAC Satellite Data for GIS

NASA's Goddard Earth Science (GES) Data and Information Services Center (DISC) Distributed Active Archive Center (DAAC) makes available a large and continually growing collection of spatially continuous global satellite observations of environmental parameters. These products include those from the MODIS (Moderate Resolution Imaging Spectroradiometer) on both Terra and Aqua platforms, and the Tropical Rainfall Measuring Mission (TRMM). These data products are well suited for use within Geographic Information Systems (GIS), as both backdrops to cartographic products as well as spatial analysis. However, data format, file size, and other issues have limited their widespread use by traditional GIS users. To address these data usability issues, the GES DISC DAAC recently updated tools and improved documentation of conversion procedures. In addition, the GES DISC DAAC has also been working with a major GIS software vendor to incorporate the ability to read the native Hierarchial Data Format (HDF), the format in which most of the NASA data is stored. The result is the enabling of GIS users to realize the benefit of GES DISC DAAC data without a substantial expenditure in resources to incorporate these data into their GIS. Several documents regarding the potential uses of GES DISC DAAC satellite data in GIS have recently been created. These show the combinations of concurrent data from different satellite products with traditional GIS vector products for given geographic areas. These map products include satellite imagery of Hurricane Isabel and the California wildfires, and can be viewed at http://daac.gsfc.nasa.gov/MODIS/GIS/.

Nickless, Darryl↗

International Satellite Cloud Climatology Project (ISCCP) Stage D2 Monthly Cloud Product - Revised Algorithm in Native (NAT) Data Format (ISCCP_D2_NAT)

Since 1983 an international group of institutions has collected and analyzed satellite radiance measurements from up to five geostationary and two polar orbiting satellites to infer the global distribution of cloud properties and their diurnal, seasonal and interannual variations. The primary focus of the first phase of the project (1983-1995) was the elucidation of the role of clouds in the radiation budget (top of the atmosphere and surface). In the second phase of the project (1995 onwards) the analysis also concerns improving understanding of clouds in the global hydrological cycle. [Location=TROPOSPHERE] [Temporal_Coverage: Start_Date=1983-07-01; Stop_Date=] [Spatial_Coverage: Southernmost_Latitude=-90; Northernmost_Latitude=90; Westernmost_Longitude=-180; Easternmost_Longitude=180] [Data_Resolution: Latitude_Resolution=280 Km; Longitude_Resolution=280 Km; Temporal_Resolution=Monthly].

CLOUD OPTICAL THICKNESS↗

Electron Spin Resonance (ESR) studies of returned comet nucleus samples

The most important objective of the Comet Nucleus Sample Returm Mission is to return samples which could reflect formation conditions and evolutionary processes in the early solar nebula. It is expected that the returned samples will consist of fine-grained silicate materials mixed with ices composed of simple molecules such as H2O, NH3, CH4 as well as organics and/or more complex compounds. Because of the exposure to ionizing radiation from cosmic-ray, gamma-ray, and solar wind protons at low temperature, free radicals are expected to be formed and trapped in the solid ice matrices. The kind of trapped radical species together with their concentration and thermal stability can be used as a dosimeter as well as a geothermometer to determine thermal and radiation histories as well as outgassing and other possible alternation effects since the nucleus material was formed. Since free radicals that are known to contain unpaired electrons are all paramagnetic in nature, they can be readily detected and characterized in their native form by the Electron Spin Resonance (ESR) method. In fact, ESR has been shown to be a non-destructive, highly sensitive tool for the detection and characterization of paramagnetic, ferromagnetic, and radiation damage centers in terrestrial and extraterrestrial geological samples. The potential use of ESR as an effective method in the study of returned comet nucleus samples, in particular, in the analysis of fine-grained solid state icy samples is discussed.

Tsay, Fun-Dow↗

Advances in PSP Testing in LaRC High Reynolds Number Facilities

The use of luminescent coatings for the global measurement of surface aerodynamic properties at full-flight Reynolds numbers has been ongoing at the NASA Langley Research Center since the late 1990s, beginning with Temperature Sensitive Paint (TSP) for boundary layer analysis at the 0.3-m Transonic Cryogenics Tunnel. Since then, significant work has been made to extend these measurements to pressure using Pressure Sensitive Paint (PSP) as well as develop systems for application in larger scale, full-flight Reynolds number facilities, such as the National Transonic Facility (NTF). Recently, a system for the measurement of time-resolved pressure using recently developed unsteady PSP (uPSP) formulations has been designed and implemented in the Transonic Dynamics Tunnel (TDT). The use of PSP (or uPSP) in either of these large scale, full-fight Reynolds number wind tunnels is complicated by the fact that both facilities operate best in oxygen deficient environments; however, the PSP technique relies on the presence of oxygen to work. The NTF is typically operated in cryogenic conditions, which is achieved using liquid nitrogen. In this facility, temperatures can reach as low as 116 K (-250 °F) with nominal oxygen concentrations of less than 50 ppm. This has resulted in the development of specialized PSP formulations that can operate in the cryogenic environment with the introduction of low amounts of oxygen (typically less than 2000 ppm) for PSP response. Likewise, to achieve full-flight Reynolds number conditions in the TDT, the atmosphere in the tunnel is replaced with a “heavy gas” of R-134A (1,1,1,2-tetrafluoroethane), which also has minimal oxygen native in the flow. The use of uPSP in this case will also depend on the introduction of trace amounts of oxygen, with the precise concentrations to be determined in an upcoming test. This presentation will describe some recent advancements that have been made for PSP measurements in both facilities. For the NTF system, there has been significant development of enhanced lighting for use at cryogenic conditions, as well as improvements in the application efficiency of the PSP. Furthermore, there are efforts underway to incorporate advanced data analysis techniques to acquire additional surface aerodynamic properties using the PSP technique that can yield not only improved experimental efficiency, but also provide data needed for next generation vehicle design and development. For the uPSP system in TDT, significant efforts to improve the data transfer rates to a high-performance computing environment for analysis are underway and performance and initial results from the system in the heavy-gas environment will be presented.

Pressure Sensitive Paint↗

TRMM Common Microphysics Products: A Tool for Evaluating Spaceborne Precipitation Retrieval Algorithms

A customized product for analysis of microphysics data collected from aircraft during field campaigns in support of the TRMM program is described. These Common Microphysics Products (CMP's) are designed to aid in evaluation of TRMM spaceborne precipitation retrieval algorithms. Information needed for this purpose (e.g., particle size spectra and habit, liquid and ice water content) was derived using a common processing strategy on the wide variety of microphysical instruments and raw native data formats employed in the field campaigns. The CMP's are organized into an ASCII structure to allow easy access to the data for those less familiar with and without the tools to accomplish microphysical data processing. Detailed examples of the CMP show its potential and some of its limitations. This approach may be a first step toward developing a generalized microphysics format and an associated community-oriented, non-proprietary software package for microphysics data processing, initiatives that would likely broaden community access to and use of microphysics datasets.

Kingsmill, David E.↗

A Spatiotemporal Indexing Approach for Efficient Processing of Big Array-Based Climate Data with MapReduce

Climate observations and model simulations are producing vast amounts of array-based spatiotemporal data. Efficient processing of these data is essential for assessing global challenges such as climate change, natural disasters, and diseases. This is challenging not only because of the large data volume, but also because of the intrinsic high-dimensional nature of geoscience data. To tackle this challenge, we propose a spatiotemporal indexing approach to efficiently manage and process big climate data with MapReduce in a highly scalable environment. Using this approach, big climate data are directly stored in a Hadoop Distributed File System in its original, native file format. A spatiotemporal index is built to bridge the logical array-based data model and the physical data layout, which enables fast data retrieval when performing spatiotemporal queries. Based on the index, a data-partitioning algorithm is applied to enable MapReduce to achieve high data locality, as well as balancing the workload. The proposed indexing approach is evaluated using the National Aeronautics and Space Administration (NASA) Modern-Era Retrospective Analysis for Research and Applications (MERRA) climate reanalysis dataset. The experimental results show that the index can significantly accelerate querying and processing (10 speedup compared to the baseline test using the same computing cluster), while keeping the index-to-data ratio small (0.0328). The applicability of the indexing approach is demonstrated by a climate anomaly detection deployed on a NASA Hadoop cluster. This approach is also able to support efficient processing of general array-based spatiotemporal data in various geoscience domains without special configuration on a Hadoop cluster.

big data↗