Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “earth data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

NASA's Earth Observing Data and Information System - Near-Term Challenges

NASA's Earth Observing System Data and Information System (EOSDIS) has been a central component of the NASA Earth observation program since the 1990's. EOSDIS manages data covering a wide range of Earth science disciplines including cryosphere, land cover change, polar processes, field campaigns, ocean surface, digital elevation, atmosphere dynamics and composition, and inter-disciplinary research, and many others. One of the key components of EOSDIS is a set of twelve discipline-based Distributed Active Archive Centers (DAACs) distributed across the United States. Managed by NASA's Earth Science Data and Information System (ESDIS) Project at Goddard Space Flight Center, these DAACs serve over 3 million users globally. The ESDIS Project provides the infrastructure support for EOSDIS, which includes other components such as the Science Investigator-led Processing systems (SIPS), common metadata and metrics management systems, specialized network systems, standards management, and centralized support for use of commercial cloud capabilities. Given the long-term requirements, and the rapid pace of information technology and changing expectations of the user community, EOSDIS has evolved continually over the past three decades. However, many challenges remain. Challenges addressed in this paper include: growing volume and variety, achieving consistency across a diverse set of data producers, managing information about a large number of datasets, migration to a cloud computing environment, optimizing data discovery and access, incorporating user feedback from a diverse community, keeping metadata updated as data collections grow and age, and ensuring that all the content needed for understanding datasets by future users is identified and preserved.

Remote Sensing↗

Overview of NASA's Earth Science Data Systems

For over the last 15 years, NASA's Earth Science Enterprise (ESE) has devoted a tremendous effort to design and build the Earth Observing System (EOS) Data and Information System (EOSDIS) to acquire, process, archive and distribute the data of the EOS series of satellites and other ESE missions and field programs. The development of EOSDIS began with an early prototype to support NASA data from heritage missions and progressed through a formal development process to today's system that supports the data from multiple missions including Landsat 7, Terra, Aqua, SORCE and ICESat. The system is deployed at multiple Distributed Active Archive Centers (DAACs) and its current holdings are approximately 4.5 petabytes. The current set of unique users requesting EOS data and information products exceeds 2 million. While EOSDIS has been the centerpiece of NASA's Earth Science Data Systems, other initiatives have augmented the services of EOSDIS and have impacted its evolution and the future directions of data systems within the ESE. ESDIS had an active prototyping effort and has continued to be involved in the activities of the Earth Science Technology Office (ESTO). In response to concerns from the science community that EOSDIS was too large and monolithic, the ESE initiated the Earth Science Information Partners (ESP) Federation Experiment that funded a series of projects to develop specialized products and services to support Earth science research and applications. Last year, the enterprise made 41 awards to successful proposals to the Research, Education and Applications Solutions Network (REASON) Cooperative Agreement Notice to continue and extend the ESP activity. The ESE has also sponsored a formulation activity called the Strategy for the Evolution of ESE Data Systems (SEEDS) to develop approaches and decision support processes for the management of the collection of data system and service providers of the enterprise. Throughout the development of its earth science data systems, NASA has had an active collaboration with a number of interagency and international partners. One of the mechanisms that has been extremely helpful in initiating and promoting this collaboration has been NASA's participation in the Committee on Earth Observation Satellites (CEOS) and its Working Group on Information Systems and Services (WGISS). The CEOS members, working together, have implemented an International Directory Network that enables users to locate collections of earth science data held by the international community and an International Catalog System to search and order specific data products. CEOS WGISS has also promoted the international interest in the Open GIS Consortium s specifications that further advance the access and use of geospatial data and the interoperation of GTS components. These are just a few highlights of the benefits that member agencies gain from CEOS participation.

McDonald, Kenneth↗

NASA's Global Change Master Directory: Discover and Access Earth Science Data Sets, Related Data Services, and Climate Diagnostics

NASA's Global Change Master Directory provides the scientific community with the ability to discover, access, and use Earth science data, data-related services, and climate diagnostics worldwide. The GCMD offers descriptions of Earth science data sets using the Directory Interchange Format (DIF) metadata standard; Earth science related data services are described using the Service Entry Resource Format (SERF); and climate visualizations are described using the Climate Diagnostic (CD) standard. The DIF, SERF and CD standards each capture data attributes used to determine whether a data set, service, or climate visualization is relevant to a user's needs. Metadata fields include: title, summary, science keywords, service keywords, data center, data set citation, personnel, instrument, platform, quality, related URL, temporal and spatial coverage, data resolution and distribution information. In addition, nine valuable sets of controlled vocabularies have been developed to assist users in normalizing the search for data descriptions. An update to the GCMD's search functionality is planned to further capitalize on the controlled vocabularies during database queries. By implementing a dynamic keyword "tree", users will have the ability to search for data sets by combining keywords in new ways. This will allow users to conduct more relevant and efficient database searches to support the free exchange and re-use of Earth science data. http://gcmd.nasa.gov/

Aleman, Alicia↗

Data Democratization: Challenges and Opportunities

Democratizing Earth data is one of the challenges many organizations around the world face in order to maximize the use of their Earth data for research, applications, education, and societal benefits. For example, at the NASA Goddard Earth Sciences (GES) Data and Information Services Center (DISC), over 1600 global and regional datasets in several NASA Earth science focus areas, including atmospheric composition, water and energy cycles, and climate variability, are archived and distributed to the public. Giovanni, the Geospatial Interactive Online Visualization and Analysis Infrastructure, was developed by GES DISC to facilitate data access and exploration, especially for novice users of Earth science. With Giovanni, users can analyze and visualize over 2000 Earth science variables (e.g., precipitation, aerosol, surface wind) without downloading data, software, the expert understanding of data formats and structures, and coding skills, lowering the barrier to data analysis/comparison by preprocessing and accessing to the data. Results of data analysis and visualization can be accessed in several popular formats (e.g., NetCDF, CSV). As a result of Giovanni's efforts, more than 3000 referral papers have been published in various fields. In spite of this, Giovanni is still difficult to use for some users. For instance, if one searches for "precipitation," it will return over 150 related variables. The question is, which one to use? Furthermore, variables from different data providers (e.g., satellites and models) are named differently with different units, further confusing users, especially those outside the communities. Data democratization is complex and multifaceted. Challenges include service and data discovery, user experiences, visualization, data quality, trustworthiness, and more. In this presentation, we will examine Giovanni as an example of challenges and opportunities in developing data democratization services.

data democratization↗

Federated Cloud Challenges in NASA's Earth Science Data Systems (Why So Difficult?)

NASA is presented with a number of opportunities and challenges in federating its Earth Science Data Systems in the burgeoning world of cloud computing. Cloud hosting of Earth Science data provides a new way of bringing data together, at least from a virtual location sense, and is one of the main motives for NASA to host data there. However, NASA is also faced with a Big Data Variety challenge, brought on by the variety of the EO datasets in its archives. This diversity requires many diverse science archives to service the different science communities. As a result, nearly every major function in the Earth Observing System Data and Information System (EOSDIS) must also be federated across its data centers. This pattern is repeated with many of the outside agencies and organizations that EOSDIS federates with, such as the Committee for Earth Observing Satellites, leading to pioneering work on "deep federation" in a joint project with the European Space Agency to develop a Multi-Mission Algorithm and Analysis Platform.

Lynnes, Christopher↗

Intelligent Systems Technologies to Assist in Utilization of Earth Observation Data

With the launch of several Earth observing satellites over the last decade, we are now in a data rich environment. From NASA's Earth Observing System (EOS) satellites alone, we are accumulating more than 3 TB per day of raw data and derived geophysical parameters. The data products are being distributed to a large user community comprising scientific researchers, educators and operational government agencies. Notable progress has been made in the last decade in facilitating access to data. However, to realize the full potential of the growing archives of valuable scientific data, further progress is necessary in the transformation of data into information, and information into knowledge that can be used in particular applications. Sponsored by NASA s Intelligent Systems Project within the Computing, Information and Communication Technology (CICT) Program, a conceptual architecture study has been conducted to examine ideas to improve data utilization through the addition of intelligence into the archives in the context of an overall knowledge building system. Potential Intelligent Archive concepts include: 1) Mining archived data holdings using Intelligent Data Understanding algorithms to improve metadata to facilitate data access and usability; 2) Building intelligence about transformations on data, information, knowledge, and accompanying services involved in a scientific enterprise; 3) Recognizing the value of results, indexing and formatting them for easy access, and delivering them to concerned individuals; 4) Interacting as a cooperative node in a web of distributed systems to perform knowledge building (i.e., the transformations from data to information to knowledge) instead of just data pipelining; and 5) Being aware of other nodes in the knowledge building system, participating in open systems interfaces and protocols for virtualization, and collaborative interoperability. This paper presents some of these concepts and identifies issues to be addressed by research in future intelligent systems technology.

Ramapriyan, Hampapuram K.↗

NASA's Earth Science Data Systems Standards Process Experiences

NASA has impaneled several internal working groups to provide recommendations to NASA management on ways to evolve and improve Earth Science Data Systems. One of these working groups is the Standards Process Group (SPC). The SPG is drawn from NASA-funded Earth Science Data Systems stakeholders, and it directs a process of community review and evaluation of proposed NASA standards. The working group's goal is to promote interoperability and interuse of NASA Earth Science data through broader use of standards that have proven implementation and operational benefit to NASA Earth science by facilitating the NASA management endorsement of proposed standards. The SPC now has two years of experience with this approach to identification of standards. We will discuss real examples of the different types of candidate standards that have been proposed to NASA's Standards Process Group such as OPeNDAP's Data Access Protocol, the Hierarchical Data Format, and Open Geospatial Consortium's Web Map Server. Each of the three types of proposals requires a different sort of criteria for understanding the broad concepts of "proven implementation" and "operational benefit" in the context of NASA Earth Science data systems. We will discuss how our Standards Process has evolved with our experiences with the three candidate standards.

Ullman, Richard E.↗

Enabling Analytics in the Cloud for Earth Science Data

The purpose of this workshop was to hold interactive discussions where providers, users, and other stakeholders could explore the convergence of three main elements in the rapidly developing world of technology: Big Data, Cloud Computing, and Analytics, [for earth science data].

Analytics↗

Saturn gravity results obtained from Pioneer 11 tracking data and earth-based Saturn satellite data

Improved gravity coefficients for Saturn, its satellites and rings are calculated on the basis of a combination of Pioneer 11 spacecraft Doppler tracking data and earth-based determinations of Saturn natural satellite apse and node rates. Solutions are first obtained separately from the coherent Doppler tracking data obtained for the interval from August 20 to September 4, surrounding the time of closest approach, with the effects of solar plasma on radio signal propagation taken into account, and from secular rates for Mimas, Enceladus, Tethys, Dione, Rhea and Titan determined from astrometric data by Kozai (1957, 1976) and Garcia (1972). Combination of the data by the use of the Pioneer solution and corresponding unadjusted covariance matrix as a priori information for a secular rate analysis results in values for the total ring mass of essentially zero at a standard error level of 1.7 x 10 to the -6th Saturn masses, a ratio of solar mass to that of the Saturn system of 3498.09 + or - 0.22, masses of Rhea, Titan and Iapetus of 4.0 + or - 0.9, 238.8 + or - 3, and 3.4 + or - 1.3 x 10 to the -6th Saturn masses, respectively, and second and fourth zonal harmonics of 16,479 + or - 18 and -937 + or - 38, respectively. The harmonic coefficients are noted to be important as boundary conditions in the modeling of the Saturn interior.

Null, G. W.↗

A Combination of Earth Orientation Data: SPACE93

A combination of independently-determined Earth orientation data has been generated from space-geodetic observations spanning 1976-1993. The approach taken is based upon a Kalman filter that was developed at the Jet Propulsion Laboratory (JPL) for just such a purpose.

lunar laser ranging↗

The Advantages of Synergy-Quantitative Earth Science Data Visualization and Analysis with Giovanni, Panoply, and Excel

The NASA Giovanni data analysis system provides a multitude of basic analysis capabilities for numerous Earth science data products which are available in the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) archive, as well as for additional selected data products provided by other NASA Distributed Active Archive Center (DAAC) archives. In Giovanni, users can easily generate time-averaged data maps, area-averaged time-series, Latitude-Time and Longitude-Time Hovmoeller diagrams, correlation maps, accumulation maps, and map animations (22 analysis options are available in total). While ASCII text output is available for time-series plots, it is not included as an option for data maps. In order to provide a quantitative, easy-to-use numerical output in ASCII text form, the NetCDF file output from a Giovanni visualization is downloaded and then opened with the free NASA visualization software package Panoply. Panoply provides the capability of translating the Giovanni file into comma-separated-variable (CSV) output. Panoply also provides additional visualization options, including the facile calculation of difference maps and quasi-anomaly maps using Giovanni output files. The CSV files from Panoply can then be imported into an Excel spreadsheet, where an Excel macro converts the CSV files. The output consists of latitude-longitude-data value triads in text form for maps, and either longitude-time-data value or latitude-time-data value triads in text form for Hovmoeller diagrams. This presentation will explicate the basic procedure for the conversion, and then provide several examples where the procedure is applied to Giovanni output from different analysis options.

Earth science↗

NASA Earth Science Data Rescue Efforts

Historically, at the end of a NASA mission, earth and space science data were stored at NASA's National Space Science Data Center (NSSDC). The original data archive consisted of both magnetic tapes and film media. As data storage technology improved, data from later missions were stored on disks and platters and higher capacity magnetic media for online accessibility. To conserve physical space at NASA archive sites and to meet disaster recovery guidelines, historical data originally stored on magnetic tapes and film were moved to the Federal Archives and Record Center (FRC) as a temporary holding area until its long-term value was determined by NASA. All records at the FRC are controlled by the NASA Records Retention Schedule (NRRS) which determines the disposal date for each record. On that date, responsible NASA parties are notified that all scheduled records should be reviewed and assessed to determine if they continue to hold significant historical, scientific or administrative value. For Earth Science data records being held at FRC, the Earth Science Data and Information System (ESDIS) Project office is the party responsible for making the value assessment that determines which records warrant preservation and which are ready for proper disposal according to NASA guidelines. Once the data's long-term value is determined, ESDIS takes definitive steps to preserve this data for future discovery and access. Deteriorating media containing historic data of value are recalled from FRC and brought back to ESDIS. Through a tedious, laborious process, digital data are recovered and restored to modern formats with improved metadata and documentation to aid discovery. The restored digital products are then incorporated into our modern online archive, and made immediately accessible to the public. In this paper, we will discuss how we identify data-at-risk, ways to minimize data loss, how we plan for recovery, how we delegate recovery activities to our archive facilities, and how we make recovered data more accessible.

Data Systems; Social and Information sciences↗

Giovanni in the Cloud: Earth Science Data Exploration in Amazon Web Services

Giovanni is an exploration tool at the NASA Goddard Earth Sciences Data Information Services Center (GES DISC), providing 22 analysis and visualization services for over 1600 Earth Science data variables. Owing to its popularity, Giovanni has experienced a consistent growth in overall demand, with periodic usage spikes attributed to trainings by education organizations, extensive data analysis in response to natural disasters, preparations for science meetings, etc. Furthermore, the new generation of spaceborne sensors and high resolution models have resulted in an exponential growth in data volume with data distributed across the traditional boundaries of data centers. Seamless exploration of data (without users having to worry about data center boundaries) has been a key recommendation of the GES DISC User Working Group. These factors have required new strategies for delivering acceptable performance. The cloud-based Giovanni, built on Amazon Web Services (AWS), evaluates (1) AWS native solutions to provide a scalable, serverless architecture; (2) open standards for data storage in the Cloud; (3) a cost model for operations; and (4) end-user performance. Our preliminary findings indicate that the use of serverless architecture has a potential to significantly reduce development and operational cost of Giovanni. The combination of using AWS managed services, storage of data in open standards, and schema-on-read data access strategy simplifies data access and analytics, in addition to making data more accessible to the end users of Giovanni through popular programming languages.

Giovanni↗

Automating the Processing of Earth Observation Data

NASA s vision for Earth science is to build a "sensor web": an adaptive array of heterogeneous satellites and other sensors that will track important events, such as storms, and provide real-time information about the state of the Earth to a wide variety of customers. Achieving this vision will require automation not only in the scheduling of the observations but also in the processing of the resulting data. To address this need, we are developing a planner-based agent to automatically generate and execute data-flow programs to produce the requested data products.

Golden, Keith↗

Earth Science Data Analytics: Preparing for Extracting Knowledge from Information

Data analytics is the process of examining large amounts of data of a variety of types to uncover hidden patterns, unknown correlations and other useful information. Data analytics is a broad term that includes data analysis, as well as an understanding of the cognitive processes an analyst uses to understand problems and explore data in meaningful ways. Analytics also include data extraction, transformation, and reduction, utilizing specific tools, techniques, and methods. Turning to data science, definitions of data science sound very similar to those of data analytics (which leads to a lot of the confusion between the two). But the skills needed for both, co-analyzing large amounts of heterogeneous data, understanding and utilizing relevant tools and techniques, and subject matter expertise, although similar, serve different purposes. Data Analytics takes on a practitioners approach to applying expertise and skills to solve issues and gain subject knowledge. Data Science, is more theoretical (research in itself) in nature, providing strategic actionable insights and new innovative methodologies. Earth Science Data Analytics (ESDA) is the process of examining, preparing, reducing, and analyzing large amounts of spatial (multi-dimensional), temporal, or spectral data using a variety of data types to uncover patterns, correlations and other information, to better understand our Earth. The large variety of datasets (temporal spatial differences, data types, formats, etc.) invite the need for data analytics skills that understand the science domain, and data preparation, reduction, and analysis techniques, from a practitioners point of view. The application of these skills to ESDA is the focus of this presentation. The Earth Science Information Partners (ESIP) Federation Earth Science Data Analytics (ESDA) Cluster was created in recognition of the practical need to facilitate the co-analysis of large amounts of data and information for Earth science. Thus, from a to advance science point of view: On the continuum of ever evolving data management systems, we need to understand and develop ways that allow for the variety of data relationships to be examined, and information to be manipulated, such that knowledge can be enhanced, to facilitate science. Recognizing the importance and potential impacts of the unlimited ways to co-analyze heterogeneous datasets, now and especially in the future, one of the objectives of the ESDA cluster is to facilitate the preparation of individuals to understand and apply needed skills to Earth science data analytics. Pinpointing and communicating the needed skills and expertise is new, and not easy. Information technology is just beginning to provide the tools for advancing the analysis of heterogeneous datasets in a big way, thus, providing opportunity to discover unobvious scientific relationships, previously invisible to the science eye. And it is not easy It takes individuals, or teams of individuals, with just the right combination of skills to understand the data and develop the methods to glean knowledge out of data and information. In addition, whereas definitions of data science and big data are (more or less) available (summarized in Reference 5), Earth science data analytics is virtually ignored in the literature, (barring a few excellent sources).

data analytics↗

Making Connections: Where STEM Learning and Earth Science Data Services Meet

STEM (Science, Technology, Engineering, Mathematics) learning is most effective when students are encouraged to see the connections between science, technology and real world problems. Helping to make these connections has become an increasingly important aspect of Earth Science data research. The Global Hydrology Resource Center (GHRC), one of NASA's 12 EOSDIS (Earth Observing System Data Information System) data centers, has developed a new type of documentation called the micro article to facilitate making connections between data and Earth science research problems.

Micro articles↗