Engineering PapersSearch

SEARCH · Engineering Papers

Results for “Science Data Processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

The design of a petabyte archive and distribution system for the NASA ECS project

The NASA EOS Data and Information System (EOSDIS) Core System (ECS) will contain one of the largest data management systems ever built - the ECS Science and Data Processing System (SDPS). SDPS is designed to support long term Global Change Research by acquiring, producing, and storing earth science data, and by providing efficient means for accessing and manipulating that data. The first two releases of SDPS, Release A and Release B, will be operational in 1997 and 1998, respectively. Release B will be deployed at eight Distributed Active Archiving Centers (DAAC's). Individual DAAC's will archive different collections of earth science data, and will vary in archive capacity. The storage and management of these data collections is the responsibility of the SDPS Data Server subsystem. It is anticipated that by the year 2001, the Data Server subsystem at the Goddard DAAC must support a near-line data storage capacity of one petabyte. The development of SDPS is a system integration effort in which COTS products will be used in favor of custom components in very possible way. Some software and hardware capabilities required to meet ECS data volume and storage management requirements beyond 1999 are not yet supported by available COTS products. The ECS project will not undertake major custom development efforts to provide these capabilities. Instead, SDPS and its Data Server subsystem are designed to support initial implementations with current products, and provide an evolutionary framework that facilitates the introduction of advanced COTS products as they become available. This paper provides a high-level description of the Data Server subsystem design from a COTS integration standpoint, and discussed some of the major issues driving the design. The paper focuses on features of the design that will make the system scalable and adaptable to changing technologies.

Caulk, Parris M.

A Multi-mission Event-Driven Component-Based System for Support of Flight Software Development, ATLO, and Operations first used by the Mars Science Laboratory (MSL) Project

This paper details an architectural description of the Mission Data Processing and Control System (MPCS), an event-driven, multi-mission ground data processing components providing uplink, downlink, and data management capabilities which will support the Mars Science Laboratory (MSL) project as its first target mission. MPCS is developed based on a set of small reusable components, implemented in Java, each designed with a specific function and well-defined interfaces. An industry standard messaging bus is used to transfer information among system components. Components generate standard messages which are used to capture system information, as well as triggers to support the event-driven architecture of the system. Event-driven systems are highly desirable for processing high-rate telemetry (science and engineering) data, and for supporting automation for many mission operations processes.

Mission Data Processing and Control System (MPCS)

Enhanced Preparation for Intelligent Cybermanufacturing Systems (EPICS)

Opportunities exist for realizing transformative advances in productivity and reductions in energy footprint through ubiquitous sensing in manufacturing environments. Enhanced Preparation for Intelligent Cybermanufacturing Systems (EPICS) is a 21-month (4 academic semesters, plus one summer) experience for graduate students that focuses on scaling the knowledge, understanding and leadership skills in the cyber manufacturing area. Masters students (8/year, 32 total) complete 2-year projects on industrially-driven project topics, rotating to internships in summer semester to work on scoping and implementation at project partners. Students complete academic training in embedded systems, process modeling, data science, and cloud-based systems design. Their projects are targeted toward sensor retrofit, process monitoring, root cause analysis, and sensor fusion.

Advanced Manufacturing

An interfaces approach to TES ground data system processing design with the Science Investigator-led Processing System (SIPS)

Developing production-quality software to process the large volumes of scientific data is the responsibility of the TES Ground Data System, which is being developed at the Jet Propulsion Laboratory together with support contractor Raytheon/ITSS. The large data volume and processing requirements of the TES pose significant challenges to the design.

TES TES SIPS

Advancing User Supports with a Structured How-To Knowledge Base for Earth Science Data

It is a challenge to access and process fast growing Earth science data from satellites and numerical models, which may be archived in very different data format and structures. NASA data centers, managed by the Earth Observing System Data and Information System (EOSDIS), have developed a rich and diverse set of data services and tools with features intended to simplify finding, downloading, and working with these data. Although most data services and tools have user guides, many users still experience difficulties with accessing or reading data due to varying levels of familiarity with data services, tools, and/or formats. A type of structured online document, data recipe, were created in beginning 2013 by Goddard Earth Science Data and Information Services Center (GES DISC). A data recipe is the How-To document created by using the fixed template, containing step-by-step instructions with screenshots and examples of accessing and working with real data. The recipes has been found to be very helpful, especially to first-time-users of particular data services, tools, or data products. Online traffic to the data recipe pages is significant to some recipes. In 2014, the NASA Earth Science Data System Working Group (ESDSWG) for data recipes was established, aimed to initiate an EOSDIS-wide campaign for leveraging the distributed knowledge within EOSDIS and its user communities regarding their respective services and tools. The ESDSWG data recipe group started with inventory and analysis of existing EOSDIS-wide online help documents, and provided recommendations and guidelines and for writing and grouping data recipes. This presentation will overview activities of creating How-To documents at GES DISC and ESDSWG. We encourage feedback and contribution from users for improving the data How-To knowledge base.

how-to

GES DISC Data Recipes in Jupyter Notebooks

The Earth Science Data and Information System (ESDIS) Project manages twelve Distributed Active Archive Centers (DAACs) which are geographically dispersed across the United States. The DAACs are responsible for ingesting, processing, archiving, and distributing Earth science data produced from various sources (satellites, aircraft, field measurements, etc.). In response to projections of an exponential increase in data production, there has been a recent effort to prototype various DAAC activities in the cloud computing environment. This, in turn, led to the creation of an initiative, called the Cloud Analysis Toolkit to Enable Earth Science (CATEES), to develop a Python software package in order to transition Earth science data processing to the cloud. This project, in particular, supports CATEES and has two primary goals. One, to transition data recipes created by the Goddard Earth Science Data and Information Service Center (GES DISC) into an interactive and educational environment using JupyterNotebooks. Two, to acclimate Earth scientists to cloud computing. To accomplish these goals, we create JupyterNotebooks to compartmentalize the different steps of data analysis and help users obtain and parse data from the command line. We also develop a Docker container, comprised of Jupyter Notebooks, Python dependencies, and command line tools, and configure it into an easy-to-deploy package. The end result is an end-to-end product that simulates the use case of end users working in the cloud computing environment.

discoverability

Autonomous Sciencecraft Experiment (ASE) Test Operations in 2003

NASA has identified the development of an autonomously operating spacecraft as a necessity for an expanded program of missions exploring the Solar System. The Autonomous Sciencecraft Experiment (ASE) has been selected for flight demonstration by NASA s New Millennium Program (NMP) as part of the Space Technology 6 (ST6) mission. ASE is scheduled to fly on the US Air Force Research Laboratory (AFRL) Techsat-21 constellation in 2006. Tech- Sat-21 consists of three satellites flying in a variable-geometry formation in Earth orbit. Each satellite is equipped with X-band Synthetic Aperture Radar, yielding high spatial resolution images (approx. 3 m) of the Earth s surface. The constellation will fly at an altitude of 550 km, in a 35.4 inclination circular orbit, yielding exact repeat-track observations every 13 days. Prior to full deployment, elements of the versatile ASE spacecraft command and control software, image formation software and science processing software will be utilized and tested on two very different platforms in 2003: AirSAR and EO-1 (described below). Advantages of Autonomous Operations: ASE will demonstrate advanced autonomous science data acquisition, processing, and product downlink prioritization, as well as autonomous spacecraft command and control, and fault detection. The advantages of spacecraft autonomy are to future missions include: (a) making the best use of reduced downlink; (b) the overcoming of communication delays through decisionmaking in situ, enabling fast reaction to dynamic events; (c) an increase of science content per byte of returned data; and (d) an avoidance of return of null (no-change/no feature) datasets: if there is no change detectable between two scenes of the same target, there is no need to return the second dataset.

Chien, S.

NASA's Earth Science Flight Program Meets the Challenges of Today and Tomorrow

NASA's Earth science flight program is a dynamic undertaking that consists of a large fleet of operating satellites, an array of satellite and instrument projects in various stages of development, a robust airborne science program, and a massive data archiving and distribution system. Each element of the flight program is complex and present unique challenges. NASA builds upon its successes and learns from its setbacks to manage this evolving portfolio to meet NASA's Earth science objectives. NASA fleet of 16 operating missions provide a wide range of scientific measurements made from dedicated Earth science satellites and from instruments mounted to the International Space Station. For operational missions, the program must address issues such as an aging satellites operating well beyond their prime mission, constellation flying, and collision avoidance with other spacecraft and orbital debris. Projects in development are divided into two broad categories: systematic missions and pathfinders. The Earth Systematic Missions (ESM) include a broad range of multi-disciplinary Earth-observing research satellite missions aimed at understanding the Earth system and its response to natural and human-induced forces and changes. Understanding these forces will help determine how to predict future changes, and how to mitigate or adapt to these changes. The Earth System Science Pathfinder (ESSP) program provides frequent, regular, competitively selected Earth science research opportunities that accommodate new and emerging scientific priorities and measurement capabilities. This results in a series of relatively low-cost, small-sized investigations and missions. Principal investigators whose scientific objectives support a variety of studies lead these missions, including studies of the atmosphere, oceans, land surface, polar ice regions, or solid Earth. This portfolio of missions and investigations provides opportunity for investment in innovative Earth science that enhances NASA's capability for better understanding the current state of the Earth system. ESM and ESSP projects often involve partnerships with other US agencies and/or international organizations. This adds to the complexity of mission development, but allows for a greater scientific return on NASA's investments. The Earth Science Airborne Science Program provides manned and unmanned aircraft systems that further science and advance the use of satellite data. NASA uses these assets worldwide in campaigns to investigate extreme weather events, observe Earth system processes, obtain data for Earth science modeling activities, and calibrate instruments flying aboard Earth science spacecraft. The Airborne Science Program has six dedicated aircraft and access to many other platforms. The Earth Science Multi-Mission Operations program acquires, preserves, and distributes observational data from operating spacecraft to support Earth Science research focus areas. The Earth Observing System Data and Information System (EOSDIS), which has been in operations since 1994, primarily accomplishes this. EOSDIS acquires, processes, archives, and distributes Earth Science data and information products. The archiving of NASA Earth Science information happens at eight Distributed Active Archive Centers (DAACs) and four disciplinary data centers located across the United States. The DAACs specialize by topic area, and make their data available to researchers around the world. The DAACs currently house over 9 petabytes of data, growing at a rate of 6.4 terabytes per day. NASA's current Earth Science portfolio is responsive to the National Research Council (NRC) 2007 Earth Science Decadal Survey and well as the 2010 NASA Response to President Obama's Climate Plan. As the program evolves into the future it will leverage the lessons learned from the current missions in operations and development, and plan for adjustments to future objectives in response to the anticipated 2017 NRC Decadal Survey.

Flight Program

OCI Geolocation Evaluation and Refinement Using Landsat Control Points

The Plankton, Aerosol, Cloud, and ocean Ecosystem (PACE) mission is NASA’s next investment in ocean biology, clouds, and aerosol data records. A key feature of PACE is the inclusion of an advanced satellite radiometer, Ocean Color Instrument (OCI), a global mapping radiometer that combines multispectral and hyperspectral remote sensing. The geolocation processing is performed for OCI using spacecraft navigation data and an instrument geometry model. To evaluate the geolocation accuracy for OCI and develop refinements to the processing methods, control point matching using Landsat data has been implemented as a step in the operational processing of OCI data at the Science Data Segment. This processing provides between 200 and 300 high-quality matchups per day with good global and geometric distribution, allowing rapid evaluation of the OCI geolocation accuracy. The results provided an early indication of the overall quality of the geolocation processing and of specific aspects needing improvement. A standard set of granules was identified to support rapid implementation and testing of geolocation refinements, and this approach has been highly successful in improving the geolocation processing accuracy to meet the science requirements. The evaluation will continue throughout the mission to ensure the ongoing accuracy of geolocation. This paper describes the control point matching methodology, the approach to development of the geolocation processing refinements, and the recent results.

PACE

OCI Geolocation Evaluation and Refinement Using Landsat Control Points

The Plankton, Aerosol, Cloud, and ocean Ecosystem (PACE) mission is NASA’s next investment in ocean biology, clouds, and aerosol data records. A key feature of PACE is the inclusion of an advanced satellite radiometer, Ocean Color Instrument (OCI), a global mapping radiometer that combines multispectral and hyperspectral remote sensing. The geolocation processing is performed for OCI using spacecraft navigation data and an instrument geometry model. To evaluate the geolocation accuracy for OCI and develop refinements to the processing methods, control point matching using Landsat data has been implemented as a step in the operational processing of OCI data at the Science Data Segment. This processing provides between 200 and 300 high-quality matchups per day with good global and geometric distribution, allowing rapid evaluation of the OCI geolocation accuracy. The results provided an early indication of the overall quality of the geolocation processing and of specific aspects needing improvement. A standard set of granules was identified to support rapid implementation and testing of geolocation refinements, and this approach has been highly successful in improving the geolocation processing accuracy to meet the science requirements. The evaluation will continue throughout the mission to ensure the ongoing accuracy of geolocation. This paper describes the control point matching methodology, the approach to development of the geolocation processing refinements, and the recent results.

PACE

Integrated Science Investigation of the Sun (ISIS): Design of the Energetic Particle Investigation

The Integrated Science Investigation of the Sun (ISIS) is a complete science investigation on the Solar Probe Plus (SPP) mission, which flies to within nine solar radii of the Sun's surface. ISIS comprises a two-instrument suite to measure energetic particles over a very broad energy range, as well as coordinated management, science operations, data processing, and scientific analysis. Together, ISIS observations allow us to explore the mechanisms of energetic particles dynamics, including their: (1) Origins-defining the seed populations and physical conditions necessary for energetic particle acceleration; (2) Acceleration-determining the roles of shocks, reconnection, waves, and turbulence in accelerating energetic particles; and (3) Transport-revealing how energetic particles propagate from the corona out into the heliosphere. The two ISIS Energetic Particle Instruments measure lower (EPI-Lo) and higher (EPI-Hi) energy particles. EPI-Lo measures ions and ion composition from approx. 20 keV/nucleon-15 MeV total energy and electrons from approx.25-1000 keV. EPI-Hi measures ions from approx. 1-200 MeV/nucleon and electrons from approx. 0.5-6 MeV. EPI-Lo comprises 80 tiny apertures with fields-of-view (FOVs) that sample over nearly a complete hemisphere, while EPI-Hi combines three telescopes that together provide five large-FOV apertures. ISIS observes continuously inside of 0.25 AU with a high data collection rate and burst data (EPI-Lo) coordinated with the rest of the SPP payload; outside of 0.25 AU, ISIS runs in low-rate science mode whenever feasible to capture as complete a record as possible of the solar energetic particle environment and provide calibration and continuity for measurements closer in to the Sun. The ISIS Science Operations Center plans and executes commanding, receives and analyzes all ISIS data, and coordinates science observations and analyses with the rest of the SPP science investigations. Together, ISIS' unique observations on SPP will enable the discovery, untangling, and understanding of the important physical processes that govern energetic particles in the innermost regions of our heliosphere, for the first time. This paper summarizes the ISIS investigation at the time of the SPP mission Preliminary Design Review in January 2014.

Coronal Mass Ejections

Integrated tools and techniques applied to the TES ground data system

The author of this paper will dicuss the selection of CASE tools, a decision making process, requirements tracking and a review mechanism that leads to a highly integrated approach to software development that must deal with the constant pressure to change software requirements and design that is associated with research and development.

science data processing TES

Apollo experience report: Apollo lunar surface experiments package data processing system

Apollo Program experience in the processing of scientific data from the Apollo lunar surface experiments package, in which computers and associated hardware and software were used, is summarized. The facility developed for the preprocessing of the lunar science data is described, as are several computer facilities and programs used by the Principal Investigators. The handling, processing, and analyzing of lunar science data and the interface with the Principal Investigators are discussed. Pertinent problems that arose in the development of the data processing schemes are discussed so that future programs may benefit from the solutions to the problems. The evolution of the data processing techniques for lunar science data related to recommendations for future programs of this type.

Eason, R. L.

Aquarius's Instrument Science Data System (ISDS) Automated to Acquire, Process, Trend Data and Produce Radiometric System Assessment Reports

The Aquarius Radiometer, a subsystem of the Aquarius Instrument required a data acquisition ground system to support calibration and radiometer performance assessment. To support calibration and compose performance assessments, we developed an automated system which uploaded raw data to a ftp server and saved raw and processed data to a database. This paper details the overall functionalities of the Aquarius Instrument Science Data System (ISDS) and the individual electrical ground support equipment (EGSE) which produced data files that were infused into the ISDS. Real time EGSEs include an ICDS Simulator, Calibration GSE, Labview controlled power supply, and a chamber data acquisition system. ICDS Simulator serves as a test conductor primary workstation, collecting radiometer housekeeping (HK) and science data and passing commands and HK telemetry collection request to the radiometer. Calibration GSE (Radiometer Active Test Source) provides source choice from multiple targets for the radiometer external calibration. Power Supply GSE, controlled by labview, provides real time voltage and current monitoring of the radiometer. And finally the chamber data acquisition system produces data reflecting chamber vacuum pressure, thermistor temperatures, AVG and watts. Each GSE system produce text based data files every two to six minutes and automatically copies the data files to the Central Archiver PC. The Archiver PC stores the data files, schedules automated uploads of these files to an external FTP server, and accepts request to copy all data files to the ISDS for offline data processing and analysis. Aquarius Radiometer ISDS contains PHP and MATLab programs to parse, process and save all data to a MySQL database. Analysis tools (MATLab programs) in the ISDS system are capable of displaying radiometer science, telemetry and auxiliary data in near real time as well as performing data analysis and producing automated performance assessment reports of the Aquarius Radiometer.

Source record

Agile Multi-Scale Decompositions for Automatic Image Registration

In recent works, the first and third authors developed an automatic image registration algorithm based on a multiscale hybrid image decomposition with anisotropic shearlets and isotropic wavelets. This prototype showed strong performance, improving robustness over registration with wavelets alone. However, this method imposed a strict hierarchy on the order in which shearlet and wavelet features were used in the registration process, and also involved an unintegrated mixture of MATLAB and C code. In this paper, we introduce a more agile model for generating features, in which a flexible and user-guided mix of shearlet and wavelet features are computed. Compared to the previous prototype, this method introduces a flexibility to the order in which shearlet and wavelet features are used in the registration process. Moreover, the present algorithm is now fully coded in C, making it more efficient and portable than the MATLAB and C prototype. We demonstrate the versatility and computational efficiency of this approach by performing registration experiments with the fully-integrated C algorithm. In particular, meaningful timing studies can now be performed, to give a concrete analysis of the computational costs of the flexible feature extraction. Examples of synthetically warped and real multi-modal images are analyzed.

Science Data Processing

Agile Multi-Scale Decompositions for Automatic Image Registration

In recent works, the first and third authors developed an automatic image registration algorithm based on a multiscale hybrid image decomposition with anisotropic shearlets and isotropic wavelets. This prototype showed strong performance, improving robustness over registration with wavelets alone. However, this method imposed a strict hierarchy on the order in which shearlet and wavelet features were used in the registration process, and also involved an unintegrated mixture of MATLAB and C code. In this paper, we introduce a more agile model for generating features, in which a flexible and user-guided mix of shearlet and wavelet features are computed. Compared to the previous prototype, this method introduces a flexibility to the order in which shearlet and wavelet features are used in the registration process. Moreover, the present algorithm is now fully coded in C, making it more efficient and portable than the MATLAB and C prototype. We demonstrate the versatility and computational efficiency of this approach by performing registration experiments with the fully-integrated C algorithm. In particular, meaningful timing studies can now be performed, to give a concrete analysis of the computational costs of the flexible feature extraction. Examples of synthetically warped and real multi-modal images are analyzed.

Science Data Processing

Case Study of an Object-Oriented System: EOSDIS

Mission to Planet Earth (MTPE) is a long-term NASA research mission to study the processes leading to global climate change. The Earth Observing System (EOS) is a NASA campaign of satellite observatories that are a major component of MTPE. The EOS Data and Information System (EOSDIS) is another component of MTPE that will provide the Earth science community with easy, affordable, and reliable access to Earth science data. EOSDIS is a distributed system, with major facilities at six Distributed Active Archive Centers (DAACS) located throughout the United States. The EOSDIS software architecture is being designed to receive, process, and archive several terabytes of science data on a daily basis. Thousands of science users and perhaps several hundred thousands of non-science users are expected to access the system. While there are many segments in EOSDIS (e.g., flight operations, network) this case study discusses the development of the science data processing segment (SDPS). We briefly review the architecture of the system, the goals of the SDPS, and the development progress to date. This study highlights key software development challenges, experiences integrating COTS, and the difficulties of managing a complex system development effort.

Behnke, Jeanne