SEARCH · Engineering Papers

Results for “Earth Observing System Data Information System”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21

All sources NASA NTRS DOE OSTI

NASA NTRSDate not supplied

Infusion of AI/ML Technology into Operational NASA Data Systems

NASA has been developing a variety of Artificial Intelligence / Machine Learning technologies related to Earth Observations. In most cases, the full value of such a technology is realized when it is infused into an operational system. NASA’s Earth Science Data Systems program has been formulating repeatable methods to execute technology infusion. These efforts include the Advancing Collaborative Connections for Earth System Science (ACCESS) program, a Technology Infusion Playbook, and an assemblage of working groups investigating methods for infusion collaboration, community development, and capacity building. ESDS has also been executing a pathfinder activity to infuse a machine-learning-driven recommender of science keywords for Earth Observation datasets, which is intended to be used for metadata curation in the Earth Observation System Data and Information System.

C Lynnes↗

NASA NTRSDate not supplied

Automated Collection of Scientific Publications Linked to NASA Earth Science Datasets

NASA's Earth Observing System Data and Information System (EOSDIS) began dataset Digital Object Identifier (DOI) registration in 2012. The number of dataset DOIs registered as of January of 2023 exceeds 11,000. As the research community becomes aware of the importance of sharing data through Open Science and optimizing data reuse through Findability, Accessibility, Interoperability, and Reuse (FAIR) data management principles, datasets are increasingly being cited in scientific publications. When datasets are cited explicitly by DOI within published works, automated methods can be developed for collecting these published works from a variety of bibliometric sources. The coverage of the sources varies, so each source can collect citations that are only available within it. Using major citation databases such as Scopus and Web of Science, the Google Scholar search engine, the CrossRef Open Citation Index, and the dataset DOI registry DataCite, we present an automated workflow for dataset citation collection. By harvesting citations automatically, a citation library is created explicitly linking EOSDIS datasets to publications that cite them. Using Zotero, a free and open-source citation manager, we demonstrate how to access and browse this library by the tags indicating bibliometric sources, dataset DOI, and the dataset archive center. We also demonstrate temporary trends in the number of publications harvested from bibliometric sources.

Infometrics↗

NASA NTRSDate not supplied

Utilizing Google Scholar as a Bibliographic Resource for Publications Search

This study focuses on the automated search for publication citations for the Earth Observing System Data and Information System (EOSDIS) datasets. The research investigates the feasibility of using automated search methods to gather published works from various bibliometric databases. A comparison is presented, highlighting the differences in citation counts obtained from Google Scholar compared to established bibliographic databases. The study also introduces a methodology and an open-source tool for getting publication citations from Google Scholar, utilizing dataset DOIs and keyword searches. The findings contribute to understanding the reliability and effectiveness of Google Scholar as a source for dataset citation retrieval and provide researchers with a valuable resource for obtaining comprehensive citation data.

Infometrics↗

NASA NTRS1989

Initial Scientific Assessment of the EOS Data and Information System (EOSDIS)

Crucial to the success of the Earth Observing System (Eos) is the Eos Data and Information System (EosDIS). The goals of Eos depend not only on its instruments and science investigations, but also on how well EosDlS helps scientists integrate reliable, large-scale data sets of geophysical and biological measurements made from Eos data, and on how successfully Eos scientists interact with other investigations in Earth System Science. Current progress in the use of remote sensing for science is hampered by requirements that the scientist understand in detail the instrument, the electromagnetic properties of the surface, and a suite of arcane tape formats, and by the immaturity of some of the techniques for estimating geophysical and biological variables from remote sensing data. These shortcomings must be transcended if remote sensing data are to be used by a much wider population of scientists who study environmental change at regional and global scales.

Source record↗

NASA NTRS1992

The EOS Data and Information System (EOSDIS)

The Earth Observing System (EOS) is the centerpiece of NASA's Mission to Planet Earth initiative. It is a pivotal part of the U.S. Global Change Research Program and hence of the international effort to understand global change and the increasing demands of human activity. EOS consists of a space-based observing system, a Data and Information System (EOSDIS) and a scientific research program. The space component consists of two series of polar-orbiting spacecraft, the first scheduled for launch in 1998, that will collect data for 15 years. EOS is completing its conceptual design phase and is preparing to enter the design phase with the selection and construction of the instruments for the first platform. EOSDIS will allow researchers to quickly and easily access data about the Earth system. Development of EOSDIS has already begun; it will support research and analysis with existing data. Geophysical and biological products will be created from the satellite data to be used to a broad range of the scientific community. NASA has also committed to providing smaller missions - called Earth Probes - dedicated to near-term observations of specific Earth processes. The scientific research program was initiated in 1990, with funding for 28 interdisciplinary teams, to begin development of models that will use EOS data and define the data requirements from the instruments, nine facility instruments and their science teams, 23 instrument investigations and definition studies for a synthetic-aperture radar.

Dozier, J.↗

NASA NTRS2000

EOS Reference Handbook 1999: A Guide to NASA's Earth Science Enterprise and the Earth Observing System

The content of this handbook includes Earth Science Enterprise; The Earth Observing System; EOS Data and Information System (EOSDIS); Data and Information Policy; Pathfinder Data Sets; Earth Science Information Partners and the Working Prototype-Federation; EOS Data Quality: Calibration and Validation; Education Programs; International Cooperation; Interagency Coordination; Mission Elements; EOS Instruments; EOS Interdisciplinary Science Investigations; and Points-of-Contact.

King, M. D.↗

NASA NTRS2004

Overview of NASA's Earth Science Data Systems

For over the last 15 years, NASA's Earth Science Enterprise (ESE) has devoted a tremendous effort to design and build the Earth Observing System (EOS) Data and Information System (EOSDIS) to acquire, process, archive and distribute the data of the EOS series of satellites and other ESE missions and field programs. The development of EOSDIS began with an early prototype to support NASA data from heritage missions and progressed through a formal development process to today's system that supports the data from multiple missions including Landsat 7, Terra, Aqua, SORCE and ICESat. The system is deployed at multiple Distributed Active Archive Centers (DAACs) and its current holdings are approximately 4.5 petabytes. The current set of unique users requesting EOS data and information products exceeds 2 million. While EOSDIS has been the centerpiece of NASA's Earth Science Data Systems, other initiatives have augmented the services of EOSDIS and have impacted its evolution and the future directions of data systems within the ESE. ESDIS had an active prototyping effort and has continued to be involved in the activities of the Earth Science Technology Office (ESTO). In response to concerns from the science community that EOSDIS was too large and monolithic, the ESE initiated the Earth Science Information Partners (ESP) Federation Experiment that funded a series of projects to develop specialized products and services to support Earth science research and applications. Last year, the enterprise made 41 awards to successful proposals to the Research, Education and Applications Solutions Network (REASON) Cooperative Agreement Notice to continue and extend the ESP activity. The ESE has also sponsored a formulation activity called the Strategy for the Evolution of ESE Data Systems (SEEDS) to develop approaches and decision support processes for the management of the collection of data system and service providers of the enterprise. Throughout the development of its earth science data systems, NASA has had an active collaboration with a number of interagency and international partners. One of the mechanisms that has been extremely helpful in initiating and promoting this collaboration has been NASA's participation in the Committee on Earth Observation Satellites (CEOS) and its Working Group on Information Systems and Services (WGISS). The CEOS members, working together, have implemented an International Directory Network that enables users to locate collections of earth science data held by the international community and an International Catalog System to search and order specific data products. CEOS WGISS has also promoted the international interest in the Open GIS Consortium s specifications that further advance the access and use of geospatial data and the interoperation of GTS components. These are just a few highlights of the benefits that member agencies gain from CEOS participation.

McDonald, Kenneth↗

NASA NTRS2019

Collecting and Processing Earth Science Data Metrics at NASA ESDIS

Since the launch of Terra satellite in 1999, the number of Earth Science remote sensing data products created and distributed by NASA's Earth Observing System (EOS) Data and Information System (EOSDIS) has increased from a few hundred to nearly ten thousand. NASA's Earth Science Data and Information System (ESDIS) Metrics System (EMS) collects metrics on data ingest, archive, and distribution by its Distributed Active Archive Centers (DAACs) and the Science Investigator-led Systems (SIPS), known as Data Providers. These metrics are critical in helping NASA management as well as data producers in resource planning and gaining a wide range of knowledge of data users and data usage.EMS receives flat files, or log files of data archive, ingest, and distribution either in their raw format, such as Apache web logs, or text files of log records formatted by the Data Providers. Tens of millions of records are processed each day to extract metrics on data products, user information, distribution protocols and services, and so on. The metrics are then made available to designated parties.This presentation provides an overview of the EMS processing workflow and improvement efforts made in recent years to handle ever-increasing number of data records and new metrics requirements, discusses several key steps including mapping log records to data products and identifying user communities along with geo-distribution, and demonstrates typical metrics capabilities produced by the EMS system. Challenges and potential approaches to improve the system are also discussed.

Pan, Jianfu↗

NASA NTRS2019

NASA EOSDIS Data Usage Metrics- Insight and Assessment

NASA's Earth Science Data and Information System (ESDIS) Project collects Earth science data usage metrics on a daily basis through the ESDIS Metrics System (EMS). This includes metrics on distribution of data products, users, data volumes, and number of files, which are key parameters in evaluating system-level performance of any of the Distributed Active Archive Centers (DAACs) encompassed by NASA's Earth Observing System (EOS) Data and Information System (EOSDIS). EOSDIS data usage metrics illustrate the benefits of making NASA data openly available to the public and show a rapid growth in data distribution to a worldwide user community. In fact, each year since 2014 the EOSDIS has distributed over one billion data files of products from EOS satellite, airborne, and in situ observations. An assessment of the long-term trends of data usage metrics and user characterization provides insights into data usability.This study will focus on describing the EMS as a metrics collection tool and will provide a comprehensive analysis of EOSDIS data usage metrics over the last 10 years. This study will also characterize the product distribution metrics by various tools and services, such as Giovanni, the Open-source Project for a Network Data Access Protocol (OPeNDAP), and subsets, to address how these tools/services have extended the usage of data in the EOSDIS collection. Data usage patterns based on discipline and study area will further assist in understanding how EOSDIS data user needs have evolved over time. Results from this study will provide useful information for the DAACs that can help them improve the functionality of their tools and services as well as more efficiently allocate the resources necessary for enhanced access and availability of their data products. Knowledge of these metrics may also benefit user discovery of data in the EOSDIS collection, promote research collaboration, and stimulate new ideas from work and research conducted using specific datasets and data collections.

Kafle, Durga N.↗

NASA NTRS2019

Using Big Data Technologies with Earth Science Data in HDF5: HDF5 Scalable Solutions

HDF5 (Hierarchical Data Format 5) is open-source, high-performance software that consists of an abstract data model, library, and fileformat used for storing and managing extremely large and/or complex data collections. NASA Earth Observing System (EOS) Data and Information Systems use HDF5 as an archival format to store remote sensing data from EOS satellites. HDF5 is also used to store other types of Geoscience and Strophysical data, e.g., seismic data and data from Low-Frequency Array (LOFAR) radio telescopes. Data stored in HDF5 has reached tens of petabytes and is growing at an accelerated rate.With the growing amout of HDF5 Earth Science data to analyze and process, scientists need to adopt big data technologies including new storage paradigms such as cloud and object storage. To run models and perform data analysis they also need to utilizied efficient and diverse ways to access data, from high-performance computing's (HPC) Message Passing Interface (MPI) I/O and deep memory hierarchies (DMH) to non-HPC frameworks such as Apache Hadoop, Spark, and Drill. The HDF Group continually works to enable usage of big data technologies in HDF software.

Knox, Larry↗

NASA NTRS1994

A statistical inference approach for the retrieval of the atmospheric ozone profile from simulated satellite measurements of solar backscattered ultraviolet radiation

NASA's Mission to Planet Earth (MTPE) will address important interdisciplinary and environmental issues such as global warming, ozone depletion, deforestation, acid rain, and the like with its long term satellite observations of the Earth and with its comprehensive Data and Information System. Extensive sets of satellite observations supporting MTPE will be provided by the Earth Observing System (EOS), while more specific process related observations will be provided by smaller Earth Probes. MTPE will use data from ground and airborne scientific investigations to supplement and validate the global observations obtained from satellite imagery, while the EOS satellites will support interdisciplinary research and model development. This is important for understanding the processes that control the global environment and for improving the prediction of events. In this paper we illustrate the potential for powerful artificial intelligence (AI) techniques when used in the analysis of the formidable problems that exist in the NASA Earth Science programs and of those to be encountered in the future MTPE and EOS programs. These techniques, based on the logical and probabilistic reasoning aspects of plausible inference, strongly emphasize the synergetic relation between data and information. As such, they are ideally suited for the analysis of the massive data streams to be provided by both MTPE and EOS. To demonstrate this, we address both the satellite imagery and model enhancement issues for the problem of ozone profile retrieval through a method based on plausible scientific inferencing. Since in the retrieval problem, the atmospheric ozone profile that is consistent with a given set of measured radiances may not be unique, an optimum statistical method is used to estimate a 'best' profile solution from the radiances and from additional a priori information.

Bonavito, N. L.↗

NASA NTRS2009

An Intelligent Archive Testbed Incorporating Data Mining

Many significant advances have occurred during the last two decades in remote sensing instrumentation, computation, storage, and communication technology. A series of Earth observing satellites have been launched by U.S. and international agencies and have been operating and collecting global data on a regular basis. These advances have created a data rich environment for scientific research and applications. NASA s Earth Observing System (EOS) Data and Information System (EOSDIS) has been operational since August 1994 with support for pre-EOS data. Currently, EOSDIS supports all the EOS missions including Terra (1999), Aqua (2002), ICESat (2002) and Aura (2004). EOSDIS has been effectively capturing, processing and archiving several terabytes of standard data products each day. It has also been distributing these data products at a rate of several terabytes per day to a diverse and globally distributed user community (Ramapriyan et al. 2009). There are other NASA-sponsored data system activities including measurement-based systems such as the Ocean Data Processing System and the Precipitation Processing system, and several projects under the Research, Education and Applications Solutions Network (REASoN), Making Earth Science Data Records for Use in Research Environments (MEaSUREs), and the Advancing Collaborative Connections for Earth-Sun System Science (ACCESS) programs. Together, these activities provide a rich set of resources constituting a value chain for users to obtain data at various levels ranging from raw radiances to interdisciplinary model outputs. The result has been a significant leap in our understanding of the Earth systems that all humans depend on for their enjoyment, livelihood, and survival. The trend in the community today is towards many distributed sets of providers of data and services. Despite this, visions for the future include users being able to locate, fuse and utilize data with location transparency and high degree of interoperability, and being able to convert data to information and usable knowledge in an efficient, convenient manner, aided significantly by automation (Ramapriyan et al. 2004; NASA 2005). We can look upon the distributed provider environment with capabilities to convert data to information and to knowledge as an Intelligent Archive in the Context of a Knowledge Building system (IA-KBS). Some of the key capabilities of an IA-KBS are: Virtual Product Generation, Significant Event Detection, Automated Data Quality Assessment, Large-Scale Data Mining, Dynamic Feedback Loop, and Data Discovery and Efficient Requesting (Ramapriyan et al. 2004).

Ramapriyan, H.↗

NASA NTRS1994

Panel to review EOSDIS plans

Formed in Jan. 1992, the Panel to Review EOSDIS Plans was charged with advising NASA on its plans for developing the Earth Observing System (EOS) Data and Information System (EOSDIS). Specifically, the panel was asked to do the following: assess the validity of the engineering and technical underpinnings of the EOSDIS; assess its potential value to scientific users; suggest how technical risk can be minimized; and assess whether current plans are sufficiently resilient to be adaptable to changing technology and requirements such as budget environments, data volumes, new users, and new databases. The panel completed an interim report (Addendum A) and transmitted it to NASA and other interested parties in the government on 9 Apr. 1992. Because of a delay in NASA's plans to select the contractor for EOSDIS, the panel was not able to complete its review of the program according to the original government request. With the issuance of a letter report (Addendum B) on 28 Sep. 1992, the panel became inactive until such time as NASA could release the details of the contractor's proposed architecture, schedule, and costs for developing EOSDIS. In early 1993, NASA awarded the contract for the EOSDIS Core System (ECS). On 20 Apr. 1993, NASA asked the panel to reconvene to do the following: ( 1) complete its review of NASA's approach to the EOSDIS architecture and implementation; (2) appraise NASA's responses to the panel's previous recommendations; and (3) review the planning for EOSDIS in the context of NASA's role in the Global Change Data and Information System (GCDIS) implementation plan. To respond to the NASA charge, the panel met three times in 1993 including sessions with NASA officials and the EOSDIS contractor. In addition, several of the panel members visited individual Distributed Active Archive Centers (DAAC's) to obtain additional views of EOSDIS. The panel has now obtained substantial information on the EOSDIS budget, contractor work program, and current baseline architecture that was not previously available, due to procurement restrictions. This report presents the panel's findings and recommendations based on this additional information.

Source record↗

NASA NTRS2004

The Development of Two Science Investigator-led Processing Systems (SIPS) for NASA's Earth Observation System (EOS)

In 2001, NASA Goddard Space Flight Center's Laboratory for Terrestrial Physics started the construction of a science Investigator-led Processing System (SIPS) for processing data from the Ozone Monitoring Instrument (OMI) which will launch on the Aura platform in mid 2004. The Ozone Monitoring Instrument (OMI) is a contribution of the Netherlands Agency for Aerospace Programs (NIVR) in collaboration with the Finnish Meteorological Institute (FMI) to the Earth Observing System (EOS) Aura mission. It will continue the Total Ozone Monitoring System (TOMS) record for total ozone and other atmospheric parameters related to ozone chemistry and climate. OMI measurements will be highly synergistic with the other instruments on the EOS Aura platform. The LTP previously developed the Moderate Resolution Imaging Spectrometer (MODIS) Data Processing System (MODAPS), which has been in full operations since the launches of the Terra and Aqua spacecrafts in December, 1999 and May, 2002 respectively. During that time, it has continually evolved to better support the needs of the MODIS team. We now run multiple instances of the system managing faster than real time reprocessings of the data as well as continuing forward processing. The new OMI Data Processing System (OMIDAPS) was adapted from the MODAPS. It will ingest raw data from the satellite ground station and process it to produce calibrated, geolocated higher level data products. These data products will be transmitted to the Goddard Distributed Active Archive Center (GDAAC) instance of the Earth Observing System (EOS) Data and Information System (EOSDIS) for long term archive and distribution to the public. The OMIDAPS will also provide data distribution to the OMI Science Team for quality assessment, algorithm improvement, calibration, etc. We have taken advantage of lessons learned from the MODIS experience and software already developed for MODIS. We made some changes in the hardware system organization, database and software to adapt the system for OMI. We replaced the fundamental database system, Sybase, with an Open Source RDBMS called PostgreSQL, and based the entire OMIDAPS on a cluster of Linux based commodity computers rather than the large SGI servers that MODAPS uses. Rather than relying on a central I/O server host, the new system distributes its data archive among multiple server hosts in the cluster. OMI is also customizing the graphical user interfaces and reporting structure to more closely meet the needs of the OMI Science Team. Prior to 2003, simulated OMI data and the science algorithms were not ready for production testing. We initially constructed a prototype system and tested using a 25 year dataset of Total Ozone Mapping Spectrometer (TOMS) and Solar Backscatter Ultraviolet Instrument (SBUV) data. This prototype system provided a platform to support the adaptation of the algorithms for OMI, and provided reprocessing of the historical data aiding in its analysis. In a recent reanalysis of the TOMS data, the OMIDAPS processed 108,000 full orbits of data through 4 processing steps per orbit, producing about 800,000 files (400 GiB) of level 2 and greater data files. More recently we have installed two instances of the OMIDAPS for integration and testing of OM1 science processes as they get delivered from the Science Team. A Test instance of the OMIDAPS has also supported a series of "Interface Confidence Tests" (ICTs) and End-to-End Ground System tests to ensure the launch readiness of the system. This paper will discuss the high-level hardware, software, and database organization of the OMIDAPS and how it builds on the MODAPS heritage system. It will also provide an overview of the testing and implementation of the production OMIDAPS.

Tilmes, Curt↗

NASA NTRS2004

A Low-Cost Clustered Archive Approach for Storing Remote Sensing Data

As part of NASA's Earth Observing System (EOS) Data and Information System (EOSDIS), the Moderate Resolution Imaging Spectrometer (MODIS) Data Processing System (MODAPS) is now processing data from two instruments on the EOS flag ship spacecraft Terra and Aqua. Between the two, MODAPS is generating over 1 Terrabyte of data per day and has surpassed 3 Petabytes of total data. The bulk of the data is stored near-line in StorageTek Powerhorn tape jukeboxes. Accessing data that has been moved to tape involves submitting an order, scheduling the tape, and waiting for the data to become available. I am developing a low cost clustered archive that could enable storing a very large amount of data such as the MODIS data described above in an organized fashion on a cluster of commodity hardware using low cost SATA hard drives such that the files are directly available online. The system takes full advantage of Open Source software, using the GNULinux operating system, PostgreSQL relational database and the Apache HTTP Server. This poster session will depict my approach, the interface to the archive, a brief discussion of the internals of the system and some performance numbers from my prototyping. I will also describe various costs and benefits of this approach versus the traditional large tape jukebox approach currently in use in the EOSDIS.

Tilmes, Curt↗

NASA NTRS2010

NASA's Earth Science Data Systems - Lessons Learned and Future Directions

In order to meet the increasing demand for Earth Science data, NASA has significantly improved the Earth Science Data Systems over the last two decades. This improvement is reviewed in this slide presentation. Many Earth Science disciplines have been able to access the data that is held in the Earth Observing System (EOS) Data and Information System (EOSDIS) at the Distributed Active Archive Centers (DAACs) that forms the core of the data system.

Ramapriyan, Hampapuram K.↗

NASA NTRS2014

SERVIR Presentation for the Hyperwall at the ESRI International Users Conference

From space, we can view our planet in new ways. As a joint initiative of NASA and USAID working in partnership with leading regional organizations, SERVIR connects space to village by helping people in developing countries use that view to gain knowledge and insights about their environments. We teach regional decision-makers to use Earth observation satellite data, Geographic Information Systems, and predictive models for addressing issues such as water and land use, natural disasters, agricultural problems, biodiversity conservation, and more to improve the lives, livelihoods, safety, and future of the people in their countries.

Irwin, Dan↗

NASA NTRS2020

Reaching Broad Audiences from a Large Agency Setting

NASA’s missions, engineering accomplishments, and scientific findings have inspired generations and advanced our understanding of the world we live in. NASA’s Earth science data are acquired by various sources, including satellites, aircraft, and field measurements. Captured data, their by-products, and their visual representations developed by research teams become available within few hours after satellite overpass or processing through a variety of NASA’s imaging, mapping services, and portals. Such online services as the Global Imagery Browse Services (GIBS), Worldview, LANCE, and LAADS DAAC are freely and openly available thanks to NASA’s Earth-Observing Satellite Data and Information Systems (EOSDIS). These services provide access to products created over the last 30 years, support a broad range of users from the scientific community to the general public, and cover a multitude of applications such as basic and applied scientific research, natural hazard and disaster monitoring, and social and educational outreach. In order to illustrate the significance of the overall work, the visualization products, and the broad range of users, we present three case studies: NASA’s Black Marble Product Suite, the Global Imagery Browse Services (GIBS) and Worldview, and the scientific visualization production process to communicate results to the scientific community and the general public.

Helen-Nicole Kostis↗