Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “science data systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20
All sourcesNASA NTRSDOE OSTI
NASA NTRS1997

Case Study of an Object-Oriented System: EOSDIS

Mission to Planet Earth (MTPE) is a long-term NASA research mission to study the processes leading to global climate change. The Earth Observing System (EOS) is a NASA campaign of satellite observatories that are a major component of MTPE. The EOS Data and Information System (EOSDIS) is another component of MTPE that will provide the Earth science community with easy, affordable, and reliable access to Earth science data. EOSDIS is a distributed system, with major facilities at six Distributed Active Archive Centers (DAACS) located throughout the United States. The EOSDIS software architecture is being designed to receive, process, and archive several terabytes of science data on a daily basis. Thousands of science users and perhaps several hundred thousands of non-science users are expected to access the system. While there are many segments in EOSDIS (e.g., flight operations, network) this case study discusses the development of the science data processing segment (SDPS). We briefly review the architecture of the system, the goals of the SDPS, and the development progress to date. This study highlights key software development challenges, experiences integrating COTS, and the difficulties of managing a complex system development effort.

Behnke, Jeanne↗
NASA NTRS2001

Gridded Hourly Text Products: A TRMM Data Reduction Approach

The quantity of precipitation data from satellite-based observations is a blessing and a curse. The sheer volume of the data makes it difficult for many researchers to use in targeted applications. This volume increases further as algorithm improvements lead to the reprocessing of mission data. In addition to the overall volume of data, the size and format complexity of orbital granules contribute to the difficulty in using all the available data. Finally, the number of different instruments available to measure rainfall and related parameters further contributes to the volume concerns. In summary, we have an embarrassment of riches. The science team of the Tropical Rainfall Measuring Mission (TRMM) recognized this dilemma and has developed a strategy to address it. The TRMM Science Data and Information System (TSDIS) produces, at the direction of the Joint TRMM Science Team, a number of instantaneous rainfall products. The TRMM Microwave Imager (TMI), the Precipitation Radar and a Combined TMI/PR are the key "instruments" used in this production. Each of these products contains an entire orbit of data. The algorithm code computes not just rain rates but a large number of other physical parameters as well as information needed for monitoring algorithm performance. That makes these products very large. For example, a single orbit of TMI rain rate product is 99 MB, a single orbit of the combined product yields a granule that is 158 MB, while the 80 vertical levels of rain information from the PR yields an orbital product of 253 MB. These are large products that are often difficult for science users to electronically transfer to their sites especially if they want a large period of time. Level 3 gridded products are much smaller, but their 5 or 30 day temporal resolution is insufficient for many researchers. In addition, TRMM standard products are produced in the HDF format. While a large number of user-friendly tools are available to hide the details of the format (including a toolkit developed at TSDIS for the TRMM science team), many potential users shy away

Stocker, Erich↗
NASA NTRS1998

The Planetary Data System - A Case Study in the Development and Management of Meta-Data for a Scientific Digital Library

The Planetary Data System (PDS) is an active science data archive managed by scientists for NASA's planetary science community. With the advent of the World Wide Web the majority of the archive has been placed on-line as a science digital libraty for access by scientists, the educational community, and the general public.

Data System Meta-data scientific digital library↗
NASA NTRS1987

Integration of multi-discipline data processing for earth observing systems

The first steps taken to ensure the controlled evolution of existing facilities toward greater interoperability and sharing of resources among NASA-supported earth science and applications data systems (ESADS) are described. Recommendations made by the various panels during the 1987 ESADS Workshop are presented. The panels were concerned with directories and catalogs, data archives, data manipulation software, computational facilities, data storage media, database management, and networking. Consideration was also given to the tracking and tuning of overall development and management coordination issues.

Kahn, Ralph↗
DOE OSTI2024

A High-level Design for Bidirectional Data Streaming to High-Performance Computing Systems from External Science Facilities

Cutting-edge science is increasingly data-driven due to the emergence of scientific machine learning models that can guide scientists toward fruitful areas of exploration. Experimental science facilities such as light and neutron sources, particle colliders, and radio astronomy telescopes are also producing raw measurement data at rates that exceed available data storage and computing capacity at those facilities. As a result, scientific workflows are being developed that concurrently couple experiments at science facilities with high-performance computing (HPC) facilities to enable analysis of experimental data while the experiment is ongoing, and where analysis results are potentially fed back to the experiment for use in experimental control and/or steering in a time-sensitive manner. Our goal is to design, prototype, and deploy a new capability for the Oak Ridge Leadership Computing Facility (OLCF) that enables such workflows through support for bidirectional, memory-based streaming of data from external experiments into and out of OLCF HPC systems. This high-level design document describes the related work and motivating use cases that inform our understanding of the technical requirements for this capability, and describes a proposed architectural solution that meets these requirements and our plans for demonstrating the capability.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2017

The Earth Observing System (EOS)

The Earth Observing System is a US National Aeronautics and Space Administration (NASA) program consisting of a science segment, a data system, and a space segment made up of a series of polar-orbiting and mid-inclination satellites for long-term monitoring of the Earth as an integrated system, including observations of the land surface, biosphere, atmosphere, cryosphere, and oceans. Initially conceived in the mid-late 1980s, it was implemented as a series of large "flagship" missions and smaller focused satellites, often in partnership with instruments and sometimes spacecraft from other nations. In 1984 NASA formed several instrument panels to develop concepts for “facility instruments” that would provide comprehensive support to the scientific and applications’ communities, each with a supporting science team competitively selected via peer-review. An Announcement of Opportunity was released in 1988 and proposals were solicited for three types of proposals: (i) instrument investigations that would include the provision of instrumentation for flight on a polar platform, including non-Earth science payloads requiring polar orbits (PI instruments), (ii) research facility instrument team member and team leader investigations for facility instruments, and (iii) interdisciplinary science investigations to provide data analysis and modeling, preparing to use data acquired from the space platforms. The initial selections made in 1989 included support for 24 instrument investigations, 6 research facility instrument investigations, and 28 interdisciplinary investigations. Although considerable changes were made over the years in the structure and orbital configuration of the various instruments and platforms, a total of 10 different missions were eventually flown, as shown in Fig. 1. These included three multiinstrument flagship platforms known as Terra (launched in 1999), Aqua (launched in 2002), and Aura (launched in 2004), where the Terra mission emphasis was primarily on land processes, Aqua on the hydrological cycle, and Aura on atmospheric chemistry. In addition to these flagship missions the EOS missions included Landsat 7 (land cover classification and change), QuikScat (wind speed and direction over the oceans), Jason-1 (ocean surface topography and sea level), ACRIMSAT (total solar irradiance (TSI)), SAGE III (stratospheric aerosol and ozone levels), ICESat (ice, cloud, and land elevation), and SORCE (Solar Radiation and Climate Experiment; total and spectral solar irradiance). In addition EOS supported CERES (an Earth radiation budget sensor) and LIS (a lightning sensor), that flew as part of the NASA/JAXA Tropical Rainfall Measuring Mission (1997), and SeaWinds (wind speed and direction), that flew as part of the short-lived ADEOS II (Midori II, 2002) mission of JAXA (Japan Aerospace Exploration Agency). The EOS contributions are too numerous to present in detail in this article, but can be viewed for convenience as a means of obtaining global measurements (with appropriate characteristics) of the 24 variables and variable groups listed in Table 1. This table was created by the EOS community in the late 1990s as a way to succinctly indicate which instruments on the various EOS platforms were expected to contribute to each of the specified variables in primary (bold), secondary (bold italics), or contributing (roman) roles. All of the science data products produced by these space missions are available without charge throughout the world, and are thus highly valued for their many contributions to applications such as air quality, fires, land cover change, and the state of the oceans, as well as for science and monitoring of the state of the Earth system and its changes over time. These and other international satellites have played a crucial role in understanding and documenting global change, including (i) global surface temperature and atmospheric temperature and water vapor profiles, (ii) glacial retreat, (iii) sea ice extent and change, (iv) solar radiation into and out of the Earth-atmosphere system, (v) atmospheric aerosol and cloud properties, (vi) sources and sinks of

ICESat↗
NASA NTRS1993

Support of an Active Science Project by a Large Information System: Lessons for the EOS Era

The ability of large information systems to support the changing data requirements of active science projects is being tested in a NASA collaborative study. This paper briefly profiles both the active science project and the large information system involved in this effort and offers some observations about the effectiveness of the project support. This is followed by lessons that are important for those participating in large information systems that need to support active science projects or that make available the valuable data produced by these projects. We learned in this work that it is difficult for a large information system focused on long term data management to satisfy the requirements of an on-going science project. For example, in order to provide the best service, it is important for all information system staff to keep focused on the needs and constraints of the scientists in the development of appropriate services. If the lessons learned in this and other science support experiences are not applied by those involved with large information systems of the EOS (Earth Observing System) era, then the final data products produced by future science projects may not be robust or of high quality, thereby making the conduct of the project science less efficacious and reducing the value of these unique suites of data for future research.

Angelici, Gary L.↗
NASA NTRS1996

Implementation of the ASTER Science Standard Data Product Requirements in the EOSDIS System

The Advanced Spaceborne Thermal Emmission and Reflection Radiometer (ASTER) is designed to provide a high-resolution map of the Earth in both visible, near-infrared, and thermal spectral regions of the electromagnetic spectrum. The ASTER Science Team has developed standard data product algorithms to permit the estimation of surface radiances and reflectance values, to calculate surface temperatures both over water and land, to provide a color enhanced product with a high degree of surface discriminability, in addition to other functions. The ASTER Product Generation System (PGS) Team is implementing these requirements within the constraints of the EOSDIS system, using a rapid development methodology that emphasizes open lines of communication in a team approach using concurrent engineering techniques. This paper describes the critical elements in the development and integration of the ASTER PGS system.

ASTER↗
NASA NTRS2018

NASA's Earth Observing Data and Information System - Near-Term Challenges

NASA's Earth Observing System Data and Information System (EOSDIS) has been a central component of the NASA Earth observation program since the 1990's. EOSDIS manages data covering a wide range of Earth science disciplines including cryosphere, land cover change, polar processes, field campaigns, ocean surface, digital elevation, atmosphere dynamics and composition, and inter-disciplinary research, and many others. One of the key components of EOSDIS is a set of twelve discipline-based Distributed Active Archive Centers (DAACs) distributed across the United States. Managed by NASA's Earth Science Data and Information System (ESDIS) Project at Goddard Space Flight Center, these DAACs serve over 3 million users globally. The ESDIS Project provides the infrastructure support for EOSDIS, which includes other components such as the Science Investigator-led Processing systems (SIPS), common metadata and metrics management systems, specialized network systems, standards management, and centralized support for use of commercial cloud capabilities. Given the long-term requirements, and the rapid pace of information technology and changing expectations of the user community, EOSDIS has evolved continually over the past three decades. However, many challenges remain. Challenges addressed in this paper include: growing volume and variety, achieving consistency across a diverse set of data producers, managing information about a large number of datasets, migration to a cloud computing environment, optimizing data discovery and access, incorporating user feedback from a diverse community, keeping metadata updated as data collections grow and age, and ensuring that all the content needed for understanding datasets by future users is identified and preserved.

Remote Sensing↗
NASA NTRS2005

EOSDIS Petabyte Archives: Tenth Anniversary

One of the world s largest scientific data systems, NASA'S Earth Observing System Data and Information System (EOSDIS) has stored over three petabytes of earth science data in a geographically distributed mass storage system. Design for this system began in the early 1990s and included a presentation of the design of the mass storage system at this conference in 1995. Many changes have occurred in the ten years since that presentation, much of it performed while the system was operational. In its first operational year (2000), the EOSDIS system had increased NASA's collection of earth science data holdings eight-fold. Today, EOSDIS collects over 7,000 gigabytes of data per week, almost 60 times more than the Hubble Space Telescope. This load represents major challenges for ingest into the mass storage system, as well as for timely and balanced data distribution out of the mass storage system. This paper discusses the evolution of the EOSDIS focusing primarily on the mass storage system component of the archive. We present the lessons that were learned over the years and some directions that we are taking for the future.

Behnke, Jeanne↗
NASA NTRS2006

Engineering the LISA Project: Systems Engineering Challenges

The Laser Interferometer Space Antenna (LISA) is a joint NASA/ESA mission to detect and measure gravitational waves with periods from 1 s to 10000 s. The systems engineering challenges of developing a giant interferometer, 5 million kilometers on a side, an: numerous. Some of the key challenges are presented in this paper. The organizational challenges imposed by sharing the engineering function between three centers (ESA ESTEC, NASA GSFC, and JPL) across nine time zones are addressed. The issues and approaches to allocation of the acceleration noise and measurement sensitivity budget terms across a traditionally decomposed system are discussed. Additionally, using LISA to detect gravitational waves for the first time presents significant data analysis challenges, many of which drive the project system design. The approach to understanding the implications of science data analysis on the system is also addressed.

systems engineering↗
NASA NTRS2016

Benchmark Comparison of Cloud Analytics Methods Applied to Earth Observations

Cloud computing has the potential to bring high performance computing capabilities to the average science researcher. However, in order to take full advantage of cloud capabilities, the science data used in the analysis must often be reorganized. This typically involves sharding the data across multiple nodes to enable relatively fine-grained parallelism. This can be either via cloud-based file systems or cloud-enabled databases such as Cassandra, Rasdaman or SciDB. Since storing an extra copy of data leads to increased cost and data management complexity, NASA is interested in determining the benefits and costs of various cloud analytics methods for real Earth Observation cases. Accordingly, NASA's Earth Science Technology Office and Earth Science Data and Information Systems project have teamed with cloud analytics practitioners to run a benchmark comparison on cloud analytics methods using the same input data and analysis algorithms. We have particularly looked at analysis algorithms that work over long time series, because these are particularly intractable for many Earth Observation datasets which typically store data with one or just a few time steps per file. This post will present side-by-side cost and performance results for several common Earth observation analysis operations.

science data management↗
NASA NTRS2017

NASA's Earth Observing System Data and Information System (EOSDIS)

EOSDIS is a data system created by NASA to manage its collection of Earth Science data. This presentation is a brief description of the data system provided to the general user community. The presentation reviews the data types, management and software development techniques in use to organize the system.

science data management↗
NASA NTRS2009

Integrating and Visualizing Tropical Cyclone Data Using the Real Time Mission Monitor

The Real Time Mission Monitor (RTMM) is a visualization and information system that fuses multiple Earth science data sources, to enable real time decision-making for airborne and ground validation experiments. Developed at the NASA Marshall Space Flight Center, RTMM is a situational awareness, decision-support system that integrates satellite imagery, radar, surface and airborne instrument data sets, model output parameters, lightning location observations, aircraft navigation data, soundings, and other applicable Earth science data sets. The integration and delivery of this information is made possible using data acquisition systems, network communication links, network server resources, and visualizations through the Google Earth virtual globe application. RTMM is extremely valuable for optimizing individual Earth science airborne field experiments. Flight planners, scientists, and managers appreciate the contributions that RTMM makes to their flight projects. A broad spectrum of interdisciplinary scientists used RTMM during field campaigns including the hurricane-focused 2006 NASA African Monsoon Multidisciplinary Analyses (NAMMA), 2007 NOAA-NASA Aerosonde Hurricane Noel flight, 2007 Tropical Composition, Cloud, and Climate Coupling (TC4), plus a soil moisture (SMAP-VEX) and two arctic research experiments (ARCTAS) in 2008. Improving and evolving RTMM is a continuous process. RTMM recently integrated the Waypoint Planning Tool, a Java-based application that enables aircraft mission scientists to easily develop a pre-mission flight plan through an interactive point-and-click interface. Individual flight legs are automatically calculated "on the fly". The resultant flight plan is then immediately posted to the Google Earth-based RTMM for interested scientists to view the planned flight track and subsequently compare it to the actual real time flight progress. We are planning additional capabilities to RTMM including collaborations with the Jet Propulsion Laboratory in the joint development of a Tropical Cyclone Integrated Data Exchange and Analysis System (TC IDEAS) which will serve as a web portal for access to tropical cyclone data, visualizations and model output.

Goodman, H. Michael↗
NASA NTRS2019

NASA GIBS and Worldview: Leveraging FOSS for NASA Earth Science Data Visualizations

The NASA Global Imagery Browse Services (GIBS) and Worldview interactive mapping site leverage scientific and community best practices, open source software, and public standards to provide a scalable, compliant, and authoritative source for NASA Earth Observing System (EOS) Earth science data visualizations. GIBS and Worldview allow end users to easily and quickly interact with more than 800 full resolution pre-generated raster- and vector-based visualizations. This interactive discovery approach relies on visual observation and identification of phenomena that are not as simply identified otherwise. This eLightning presentation will exhibit the broad set of capabilities and visualization layers made possible through the GIBS and Worldview open source software. Specific dependencies on, and contributions to, open source software will be highlighted. Additionally, opportunities for future improvements for better interoperability and reuse through open source software will be discussed.

NASA Global Imagery Browse Services (GIBS)↗