Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Earth Science Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
NASA NTRS2019

Collecting and Processing Earth Science Data Metrics at NASA ESDIS

Since the launch of Terra satellite in 1999, the number of Earth Science remote sensing data products created and distributed by NASA's Earth Observing System (EOS) Data and Information System (EOSDIS) has increased from a few hundred to nearly ten thousand. NASA's Earth Science Data and Information System (ESDIS) Metrics System (EMS) collects metrics on data ingest, archive, and distribution by its Distributed Active Archive Centers (DAACs) and the Science Investigator-led Systems (SIPS), known as Data Providers. These metrics are critical in helping NASA management as well as data producers in resource planning and gaining a wide range of knowledge of data users and data usage.EMS receives flat files, or log files of data archive, ingest, and distribution either in their raw format, such as Apache web logs, or text files of log records formatted by the Data Providers. Tens of millions of records are processed each day to extract metrics on data products, user information, distribution protocols and services, and so on. The metrics are then made available to designated parties.This presentation provides an overview of the EMS processing workflow and improvement efforts made in recent years to handle ever-increasing number of data records and new metrics requirements, discusses several key steps including mapping log records to data products and identifying user communities along with geo-distribution, and demonstrates typical metrics capabilities produced by the EMS system. Challenges and potential approaches to improve the system are also discussed.

Pan, Jianfu↗
NASA NTRS2018

NASA GIBS and Worldview: Visualizing NASA's Earth Science Data for All to Explore

For more than 20 years, the NASA Earth Observing System (EOS) has operated dozens of remote sensing satellites collecting nearly 15 Petabytes of data that span thousands of science parameters. Within these observations are keys the Earth Scientists have used to unlock many discoveries that we now understand about our planet. Also contained within these observations are a myriad of opportunities for learning and education. The challenge is making them accessible to educators and students in intuitive and simple ways so that effort can be spent on lesson enrichment and not overcoming technical hurdles. The NASA Global Imagery Browse Services (GIBS) system and NASA Worldview website provide a unique view into EOS data through daily full resolution visualizations of hundreds of earth science parameters. For many of these parameters, visualizations are available within hours of acquisition from the satellite. For others, visualizations are available for the entire mission of the satellite. Accompanying the visualizations are visual aids such as color legends, place names, and orbit tracks. By using these visualizations, educators and students can observe natural phenomena that enrich a scientific education.

Cechini, Matthew↗
NASA NTRS2019

NASA GIBS and Worldview: Visualizing NASA's Earth Science Data for All to Explore

For more than 20 years, the NASA Earth Observing System (EOS) has operated dozens of remote sensing satellites collecting nearly 15 Petabytes of data that span thousands of science parameters. Within these observations are keys the Earth Scientists have used to unlock many discoveries that we now understand about our planet. Also contained within these observations are a myriad of opportunities for learning and education. The challenge is making them accessible to educators and students in intuitive and simple ways so that effort can be spent on lesson enrichment and not overcoming technical hurdles.The NASA Global Imagery Browse Services (GIBS) system and NASA Worldview interactive mapping site provide a unique view into EOS data through daily full resolution visualizations of hundreds of Earth science parameters. For many of these parameters, visualizations are available within hours of acquisition from the satellite. For others, visualizations are available for the entire mission of the satellite. Accompanying the visualizations are visual aids such as color legends, place names, and orbit tracks. By using these visualizations, educators and students can observe natural phenomena that enrich a scientific education.This presentation will provide an overview of the visualizations available in NASA GIBS and Worldview and how they are accessed. Specific attention will be given to the newer capabilities and accomplishments, including: Support for geostationary sub-daily visualizations, Enhanced support for vector-based visualizations, Improved Worldview tour and snapshot capabilities, New imagery products across a growing set of scientific areas.

Cechini, Matthew↗
NASA NTRS2016

Using Cloud-Based Storage Technologies for Earth Science Data

Cloud based infrastructure may offer several key benefits of scalability, built in redundancy and reduced total cost of ownership as compared with a traditional data center approach. However, most of the tools and software systems developed for NASA data repositories were not developed with a cloud based infrastructure in mind and do not fully take advantage of commonly available cloud-based technologies. Object storage services are provided through all the leading public (Amazon Web Service, Microsoft Azure, Google Cloud, etc.) and private (Open Stack) clouds, and may provide a more cost-effective means of storing large data collections online. We describe a system that utilizes object storage rather than traditional file system based storage to vend earth science data. The system described is not only cost effective, but shows superior performance for running many different analytics tasks in the cloud. To enable compatibility with existing tools and applications, we outline client libraries that are API compatible with existing libraries for HDF5 and NetCDF4. Performance of the system is demonstrated using clouds services running on Amazon Web Services.

Data↗
NASA NTRS2005

Giovanni: A System for Rapid Access, Visualization and Analysis of Earth Science Data Online

Collecting data and understanding data structures traditionally are the first steps that a user must take, before the core investigation can begin. This is a time-consuming and challenging task, especially when science objectives require users to deal with large multi-sensor data that are usually in different formats and internal structures. The Goddard Earth Sciences Data and Information Services Center (GES DISC) has created the GES DISC Interactive Online Visualization and ANalysis Infrastructure, Giovanni, to enable Web-based visualization and analysis of satellite remotely sensed meteorological, oceanographic, and hydrologic data sets, without users having to download data. The current operational Giovanni interfaces provide the capability to process a number of important satellite measurements, such as (1) ozone and other trace gases from TOMS, OMI, HALOE, and MLS; (2) air temperature, water vapor, and geopotential height from AIRS; (3) aerosols from MODIS TerrdAqua, and GOCART model; (4) precipitation from TRMM and ground measurements; (5) chlorophyll and other ocean color products from SeaWiFS and MODIS Aqua; and (6) sea surface temperature from MODIS Aqua. Depending on the input data structure, the system provides simple statistical analysis and creates time-averaged area plot, area-averaged time series, animations, Hovmoller latitude vs. time and longitude vs. time plots, as well as vertical profiles. The inter-comparison interfaces allow a user to compare observations from different instruments, to conduct anomaly analysis, and to study basic relationships between physical parameters. Giovanni handles data with different temporal and spatial resolutions and, thus, enables both regional and global long-term climate research and short-term special events investigation, as well as data validations and assessments. Because of its simplicity of usage, Giovanni is powerful and versatile, able to assist a wide range of users, from the discipline scientists conducting preliminary research in various fields, to students in the classroom learning about weather, climate, and other natural phenomena. Giovanni can be accessed from: http://disc. esfc.nasa.gov/techlab/giovanni/index.shtml

Shen, S.↗
NASA NTRS2010

Reducing the Volume of NASA Earth-Science Data

A computer program reduces data generated by NASA Earth-science missions into representative clusters characterized by centroids and membership information, thereby reducing the large volume of data to a level more amenable to analysis. The program effects an autonomous data-reduction/clustering process to produce a representative distribution and joint relationships of the data, without assuming a specific type of distribution and relationship and without resorting to domain-specific knowledge about the data. The program implements a combination of a data-reduction algorithm known as the entropy-constrained vector quantization (ECVQ) and an optimization algorithm known as the differential evolution (DE). The combination of algorithms generates the Pareto front of clustering solutions that presents the compromise between the quality of the reduced data and the degree of reduction. Similar prior data-reduction computer programs utilize only a clustering algorithm, the parameters of which are tuned manually by users. In the present program, autonomous optimization of the parameters by means of the DE supplants the manual tuning of the parameters. Thus, the program determines the best set of clustering solutions without human intervention.

Lee, Seungwon↗
NASA NTRSDate not supplied

LAADS DAAC Migrates to the Cloud: Lessons Learned from Communicating About Earth Science Data on the Cloud

The Level-1 and Atmosphere Archive Distribution System (LAADS) Distributed Active Archive Center (DAAC) is migrating data to the cloud. As one of twelve DAACS supported by NASA’s Earth Science Data and Information System (ESDIS), LAADS is using moving away from on premise data storage facilities to migrating to Amazon Web Services, where the massive archive of data from the Moderate Imaging Spectroradiometer (MODIS) and the Visible Infrared Imaging Radiometer Suite (VIIRS) will be available for download and post-processing transformations online. The migration is happening in three phases and LAADS is concluding its beta testing period. This poster shows the lessons learned from communicating with a select group of users about how to effectively educate data users on using data in the cloud.

Tassia Owen↗
NASA NTRS1987

Animated computer graphics models of space and earth sciences data generated via the massively parallel processor

The capability was developed of rapidly producing visual representations of large, complex, multi-dimensional space and earth sciences data sets via the implementation of computer graphics modeling techniques on the Massively Parallel Processor (MPP) by employing techniques recently developed for typically non-scientific applications. Such capabilities can provide a new and valuable tool for the understanding of complex scientific data, and a new application of parallel computing via the MPP. A prototype system with such capabilities was developed and integrated into the National Space Science Data Center's (NSSDC) Pilot Climate Data System (PCDS) data-independent environment for computer graphics data display to provide easy access to users. While developing these capabilities, several problems had to be solved independently of the actual use of the MPP, all of which are outlined.

Treinish, Lloyd A.↗
NASA NTRS2012

Earth-Science Data Co-Locating Tool

This software is used to locate Earth-science satellite data and climate-model analysis outputs in space and time. This enables the direct comparison of any set of data with different spatial and temporal resolutions. It is written in three separate modules that are clearly separated for their functionality and interface with other modules. This enables a fast development of supporting any new data set. In this updated version of the tool, several new front ends are developed for new products. This software finds co-locatable data pairs for given sets of data products and creates new data products that share the same spatial and temporal coordinates. This facilitates the direct comparison between the two heterogeneous datasets and the comprehensive and synergistic use of the datasets.

Lee, Seungwon↗
NASA NTRS1993

Applications of wavelet-based compression to multidimensional Earth science data

A data compression algorithm involving vector quantization (VQ) and the discrete wavelet transform (DWT) is applied to two different types of multidimensional digital earth-science data. The algorithms (WVQ) is optimized for each particular application through an optimization procedure that assigns VQ parameters to the wavelet transform subbands subject to constraints on compression ratio and encoding complexity. Preliminary results of compressing global ocean model data generated on a Thinking Machines CM-200 supercomputer are presented. The WVQ scheme is used in both a predictive and nonpredictive mode. Parameters generated by the optimization algorithm are reported, as are signal-to-noise (SNR) measurements of actual quantized data. The problem of extrapolating hydrodynamic variables across the continental landmasses in order to compute the DWT on a rectangular grid is discussed. Results are also presented for compressing Landsat TM 7-band data using the WVQ scheme. The formulation of the optimization problem is presented along with SNR measurements of actual quantized data. Postprocessing applications are considered in which the seven spectral bands are clustered into 256 clusters using a k-means algorithm and analyzed using the Los Alamos multispectral data analysis program, SPECTRUM, both before and after being compressed using the WVQ program.

Bradley, Jonathan N.↗
NASA NTRS2015

Earth Science Data Fusion with Event Building Approach

Objectives of the NASA Information And Data System (NAIADS) project are to develop a prototype of a conceptually new middleware framework to modernize and significantly improve efficiency of the Earth Science data fusion, big data processing and analytics. The key components of the NAIADS include: Service Oriented Architecture (SOA) multi-lingual framework, multi-sensor coincident data Predictor, fast into-memory data Staging, multi-sensor data-Event Builder, complete data-Event streaming (a work flow with minimized IO), on-line data processing control and analytics services. The NAIADS project is leveraging CLARA framework, developed in Jefferson Lab, and integrated with the ZeroMQ messaging library. The science services are prototyped and incorporated into the system. Merging the SCIAMACHY Level-1 observations and MODIS/Terra Level-2 (Clouds and Aerosols) data products, and ECMWF re- analysis will be used for NAIADS demonstration and performance tests in compute Cloud and Cluster environments.

Lukashin, C.↗
NASA NTRS2019

NASA GIBS and Worldview: Leveraging FOSS for NASA Earth Science Data Visualizations

The NASA Global Imagery Browse Services (GIBS) and Worldview interactive mapping site leverage scientific and community best practices, open source software, and public standards to provide a scalable, compliant, and authoritative source for NASA Earth Observing System (EOS) Earth science data visualizations. GIBS and Worldview allow end users to easily and quickly interact with more than 800 full resolution pre-generated raster- and vector-based visualizations. This interactive discovery approach relies on visual observation and identification of phenomena that are not as simply identified otherwise. This eLightning presentation will exhibit the broad set of capabilities and visualization layers made possible through the GIBS and Worldview open source software. Specific dependencies on, and contributions to, open source software will be highlighted. Additionally, opportunities for future improvements for better interoperability and reuse through open source software will be discussed.

NASA Global Imagery Browse Services (GIBS)↗
NASA NTRS2019

A Unified Level of Service Model for NASA Earth Science Data Stewardship

During the past year, the Interagency Implementation and Concepts Team (IMPACT) reviewed existing service models in use at various NASA data centers in an effort to produce a unified, cohesive, and comprehensive Level of Service Model for all of NASA Distributed Active Archive Centers (DAACs). NASA DAACs are responsible for ensuring NASA Earth Science data are accurately and securely ingested, distributed, supported, and preserved. The term “Service" as used here refers to the spectrum of data management activities and outputs provided by DAACs in support of the data cared fo by each data cente. The unified Level-of- Service (LoS) model described in this presentation utilizes both the NASA-defined data product category and the data processing level to easily identify an appropriate level-of-service to be applied to a data product throughout the full data life cycle. This LoS model is to be used by DAACs when appraising incoming data in order to determine the appropriate and required services to provide. The LoS model utilizes a 3-level system in which services build upon previous levels and thereby require greater commitment and effort both on the part of the DAAC personnel and the data producer at the highest level. The LoS model description also contains examples of ways to communicate with data producers and data users what services can be expected, thereby bringing more consistent user experiences across the enterprise. In this presentation, we will outline the features of the LoS model and describe how it relates to the FAIR data practices and the NOAA Maturity Matrix model.

Smith, Deborah↗
NASA NTRS2010

Tracking Provenance of Earth Science Data

Tremendous volumes of data have been captured, archived and analyzed. Sensors, algorithms and processing systems for transforming and analyzing the data are evolving over time. Web Portals and Services can create transient data sets on-demand. Data are transferred from organization to organization with additional transformations at every stage. Provenance in this context refers to the source of data and a record of the process that led to its current state. It encompasses the documentation of a variety of artifacts related to particular data. Provenance is important for understanding and using scientific datasets, and critical for independent confirmation of scientific results. Managing provenance throughout scientific data processing has gained interest lately and there are a variety of approaches. Large scale scientific datasets consisting of thousands to millions of individual data files and processes offer particular challenges. This paper uses the analogy of art history provenance to explore some of the concerns of applying provenance tracking to earth science data. It also illustrates some of the provenance issues with examples drawn from the Ozone Monitoring Instrument (OMI) Data Processing System (OMIDAPS) run at NASA's Goddard Space Flight Center by the first author.

Tilmes, Curt↗
NASA NTRS2003

ESML for Earth Science Data Sets and Analysis

The primary objective of this research project was to transition ESML from design to application. The resulting schema and prototype software will foster community acceptance for the Define once, use anywhere concept central to ESML. Supporting goals include: 1) Refinement of the ESML schema and software libraries in cooperation with the user community; 2) Application of the ESML schema and software to a variety of Earth science data sets and analysis tools; 3) Development of supporting prototype software for enhanced ease of use; 4) Cooperation with standards bodies in order to assure ESML is aligned with related metadata standards as appropriate; and 5) Widespread publication of the ESML approach, schema, and software.

Graves, Sara↗
NASA NTRS2002

Grids for Dummies: Featuring Earth Science Data Mining Application

This viewgraph presentation discusses the concept and advantages of linking computers together into data grids, an emerging technology for managing information across institutions, and potential users of data grids. The logistics of access to a grid, including the use of the World Wide Web to access grids, and security concerns are also discussed. The potential usefulness of data grids to the earth science community is also discussed, as well as the Global Grid Forum, and other efforts to establish standards for data grids.

Hinke, Thomas H.↗
NASA NTRS2017

An Innovative Infrastructure with a Universal Geo-Spatiotemporal Data Representation Supporting Cost-Effective Integration of Diverse Earth Science Data

The SpatioTemporal Adaptive Resolution Encoding (STARE) is a unifying scheme encoding geospatial and temporal information for organizing data on scalable computing/storage resources, minimizing expensive data transfers. STARE provides a compact representation that turns set-logic functions into integer operations, e.g. conditional sub-setting, taking into account representative spatiotemporal resolutions of the data in the datasets. STARE geo-spatiotemporally aligns data placements of diverse data on massive parallel resources to maximize performance. Automating important scientific functions (e.g. regridding) and computational functions (e.g. data placement) allows scientists to focus on domain-specific questions instead of expending their efforts and expertise on data processing. With STARE-enabled automation, SciDB (Scientific Database) plus STARE provides a database interface, reducing costly data preparation, increasing the volume and variety of interoperable data, and easing result sharing. Using SciDB plus STARE as part of an integrated analysis infrastructure dramatically eases combining diametrically different datasets.

Rilee, Michael Lee↗
NASA NTRS2019

Evaluation of MODIS and VIIRS Cloud-Gap-Filled Snow-Cover Products for Production of an Earth Science Data Record

MODerate resolution Imaging Spectroradiometer (MODIS) cryosphere products have been available since 2000 – following the 1999 launch of the Terra MODIS and the 2002 launch of the Aqua MODIS – and include global snow-cover extent (SCE) (swath, daily, and 8 d composites) at 500 m and ∼5 km spatial resolutions. These products are used extensively in hydrological modeling and climate studies. Reprocessing of the complete snow-cover data record, from Collection 5 (C5) to Collection 6 (C6) and Collection 6.1 (C6.1), has provided improvements in the MODIS product suite. Suomi National Polar-orbiting Partnership (S-NPP) Visible Infrared Imaging Radiometer Suite (VIIRS) Collection 1 (C1) snow-cover products at a 375 m spatial resolution have been available since 2011 and are currently being reprocessed for Collection 2 (C2). Both the MODIS C6.1 and the VIIRS C2 products will be available for download from the National Snow and Ice Data Center beginning in early 2020 with the complete time series available in 2020. To address the need for a cloud-reduced or cloud-free daily SCE product for both MODIS and VIIRS, a daily cloud-gap-filled (CGF) snow-cover algorithm was developed for MODIS C6.1 and VIIRS C2 processing. MOD10A1F (Terra) and MYD10A1F (Aqua) are daily, 500 m resolution CGF SCE map products from MODIS. VNP10A1F is the daily, 375 m resolution CGF SCE map product from VIIRS. These CGF products include quality-assurance data such as cloud-persistence statistics showing the age of the observation in each pixel. The objective of this paper is to introduce the new MODIS and VIIRS standard CGF daily SCE products and to provide a preliminary evaluation of uncertainties in the gap-filling methodology so that the products can be used as the basis for a moderate-resolution Earth science data record (ESDR) of SCE. Time series of the MODIS and VIIRS CGF products have been developed and evaluated at selected study sites in the US and southern Canada. Observed differences, although small, are largely attributed to cloud masking and differences in the time of day of image acquisition. A nearly 3-month time-series comparison of Terra MODIS and S-NPP VIIRS CGF snow-cover maps for a large study area covering all or parts of 11 states in the western US and part of southwestern Canada reveals excellent correspondence between the Terra MODIS and S-NPP VIIRS products, with a mean difference of 11 070 sqkm, which is ∼0.45 % of the study area. According to our preliminary validation of the Terra and Aqua MODIS CGF SCE products in the western US study area, we found higher accuracy of the Terra product compared with the Aqua product. The MODIS CGF SCE data record beginning in 2000 has been extended into the VIIRS era, which should last at least through the early 2030s.

Hall, Dorothy K.↗