SEARCH · Engineering Papers

Results for “archival processes”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

All sources NASA NTRS DOE OSTI

NASA NTRS1997

International Ultraviolet Explorer Final Archive

CSC processed IUE images through the Final Archive Data Processing System. Raw images were obtained from both NDADS and the IUEGTC optical disk platters for processing on the Alpha cluster, and from the IUEGTC optical disk platters for DECstation processing. Input parameters were obtained from the IUE database. Backup tapes of data to send to VILSPA were routinely made on the Alpha cluster. IPC handled more than 263 requests for priority NEWSIPS processing during the contract. Staff members also answered various questions and requests for information and sent copies of IUE documents to requesters. CSC implemented new processing capabilities into the NEWSIPS processing systems as they became available. In addition, steps were taken to improve efficiency and throughput whenever possible. The node TORTE was reconfigured as the I/O server for Alpha processing in May. The number of Alpha nodes used for the NEWSIPS processing queue was increased to a maximum of six in measured fashion in order to understand the dependence of throughput on the number of nodes and to be able to recognize when a point of diminishing returns was reached. With Project approval, generation of the VD FITS files was dropped in July. This action not only saved processing time but, even more significantly, also reduced the archive storage media requirements, and the time required to perform the archiving, drastically. The throughput of images verified through CDIVS and processed through NEWSIPS for the contract period is summarized below. The number of images of a given dispersion type and camera that were processed in any given month reflects several factors, including the availability of the required NEWSIPS software system, the availability of the corresponding required calibrations (e.g., the LWR high-dispersion ripple correction and absolute calibration), and the occurrence of reprocessing efforts such as that conducted to incorporate the updated SWP sensitivity-degradation correction in May.

Source record↗

NASA NTRS2017

Lessons Learned While Exploring Cloud-Native Architectures for NASA EOSDIS Applications and Systems

NASA's Earth Observing System (EOS) is a coordinated series of satellites for long term global observations. NASA's Earth Observing System Data and Information System (EOSDIS) is a multi-petabyte-scale archive of environmental data that supports global climate change research by providing end-to-end services from EOS instrument data collection to science data processing to full access to EOS and other earth science data. On a daily basis, the EOSDIS ingests, processes, archives and distributes over 3 terabytes of data from NASA's Earth Science missions representing over 6000 data products ranging from various types of science disciplines. EOSDIS has continually evolved to improve the discoverability, accessibility, and usability of high-impact NASA data spanning the multi-petabyte-scale archive of Earth science data products. Reviewed and approved by Chris Lynnes.

Cumulus↗

NASA NTRS1991

Version 0 EOSDIS - An overview

Attention is given to NASA's Earth Observing System Data and Information System (EOSDIS), which is to be a single, distributed but internally consistent evolutionary system to support the planning and execution of EOS data acquisitions and to process, archive, and distribute EOS data products and selected non-EOS data to enable interdisciplinary studies of the earth. V0 EOSDIS, a logical step in this evolutionary process, is to address both technical and managerial challenges. Technical challenges include developing a multidiscipline, distributed system for searching and ordering data in a heterogeneous environment, and standardizing data formats and distribution techniques among differing communities and organizations. Managerial challenges include establishing and maintaining a structure consisting of geographically distributed entities such that cooperative development is carried out effectively despite organizational differences, maintaining interactions with the scientific community to ensure its close involvement despite its size and diversity, and keeping the expectations for V0 consistent with its schedules and resources.

Ramapriyan, H. K.↗

NASA NTRS2016

Restoration of Apollo Data by the Lunar Data Project/PDS Lunar Data Node: An Update

The Apollo 11, 12, and 14 through 17 missions orbited and landed on the Moon, carrying scientific instruments that returned data from all phases of the missions, included long-lived Apollo Lunar Surface Experiments Packages (ALSEPs) deployed by the astronauts on the lunar surface. Much of these data were never archived, and some of the archived data were on media and in formats that are outmoded, or were deposited with little or no useful documentation to aid outside users. This is particularly true of the ALSEP data returned autonomously for many years after the Apollo missions ended. The purpose of the Lunar Data Project and the Planetary Data System (PDS) Lunar Data Node is to take data collections already archived at the NASA Space Science Data Coordinated Archive (NSSDCA) and prepare them for archiving through PDS, and to locate lunar data that were never archived, bring them into NSSDCA, and then archive them through PDS. Preparing these data for archiving involves reading the data from the original media, be it magnetic tape, microfilm, microfiche, or hard-copy document, converting the outmoded, often binary, formats when necessary, putting them into a standard digital form accepted by PDS, collecting the necessary ancillary data and documentation (metadata) to ensure that the data are usable and well-described, summarizing the metadata in documentation to be included in the data set, adding other information such as references, mission and instrument descriptions, contact information, and related documentation, and packaging the results in a PDS-compliant data set. The data set is then validated and reviewed by a group of external scientists as part of the PDS final archive process. We present a status report on some of the data sets that we are processing.

Lunar↗

NASA NTRS2006

Cassini Archive Tracking System

The Cassini Archive Tracking System (CATS) is a computer program that enables tracking of scientific data transfers from originators to the Planetary Data System (PDS) archives. Without CATS, there is no systematic means of locating products in the archive process or ensuring their completeness. By keeping a database of transfer communications and status, CATS enables the Cassini Project and the PDS to efficiently and accurately report on archive status. More importantly, problem areas are easily identified through customized reports that can be generated on the fly from any Web-enabled computer. A Web-browser interface and clearly defined authorization scheme provide safe distributed access to the system, where users can perform functions such as create customized reports, record a transfer, and respond to a transfer. CATS ensures that Cassini provides complete science archives to the PDS on schedule and that those archives are available to the science community by the PDS. The three-tier architecture is loosely coupled and designed for simple adaptation to multimission use. Written in the Java programming language, it is portable and can be run on any Java-enabled Web server.

Conner, Diane↗

NASA NTRS2004

Overview of NASA's Earth Science Data Systems

For over the last 15 years, NASA's Earth Science Enterprise (ESE) has devoted a tremendous effort to design and build the Earth Observing System (EOS) Data and Information System (EOSDIS) to acquire, process, archive and distribute the data of the EOS series of satellites and other ESE missions and field programs. The development of EOSDIS began with an early prototype to support NASA data from heritage missions and progressed through a formal development process to today's system that supports the data from multiple missions including Landsat 7, Terra, Aqua, SORCE and ICESat. The system is deployed at multiple Distributed Active Archive Centers (DAACs) and its current holdings are approximately 4.5 petabytes. The current set of unique users requesting EOS data and information products exceeds 2 million. While EOSDIS has been the centerpiece of NASA's Earth Science Data Systems, other initiatives have augmented the services of EOSDIS and have impacted its evolution and the future directions of data systems within the ESE. ESDIS had an active prototyping effort and has continued to be involved in the activities of the Earth Science Technology Office (ESTO). In response to concerns from the science community that EOSDIS was too large and monolithic, the ESE initiated the Earth Science Information Partners (ESP) Federation Experiment that funded a series of projects to develop specialized products and services to support Earth science research and applications. Last year, the enterprise made 41 awards to successful proposals to the Research, Education and Applications Solutions Network (REASON) Cooperative Agreement Notice to continue and extend the ESP activity. The ESE has also sponsored a formulation activity called the Strategy for the Evolution of ESE Data Systems (SEEDS) to develop approaches and decision support processes for the management of the collection of data system and service providers of the enterprise. Throughout the development of its earth science data systems, NASA has had an active collaboration with a number of interagency and international partners. One of the mechanisms that has been extremely helpful in initiating and promoting this collaboration has been NASA's participation in the Committee on Earth Observation Satellites (CEOS) and its Working Group on Information Systems and Services (WGISS). The CEOS members, working together, have implemented an International Directory Network that enables users to locate collections of earth science data held by the international community and an International Catalog System to search and order specific data products. CEOS WGISS has also promoted the international interest in the Open GIS Consortium s specifications that further advance the access and use of geospatial data and the interoperation of GTS components. These are just a few highlights of the benefits that member agencies gain from CEOS participation.

McDonald, Kenneth↗

NASA NTRSDate not supplied

Improving Data Discovery, Analysis, and Visualizations With Cloud-Based User Services

The Global Hydrometeorology Resource Center (GHRC) Distributed Active Archive Center (DAAC) is one of 12 DAACs managed by the United States National Aeronautics and Space Administration (NASA) Earth Science Data and Information System (ESDIS) project [1]. GHRC and the other DAACs are designed to process, archive, document, and distribute NASA Earth-observing data, ranging from satellite missions to field campaigns [2]. A major goal of the DAACs is to enable science with these data. Science enabling can be difficult as datasets can be very large, use multiple formats, come from numerous platforms, and require three-dimensional visualization. GHRC is using its expertise with cloud-based technologies to develop open source and open science tools to empower users to explore, coincidentally visualize, and analyze multiple datasets. Being open source, the user community can develop visualizations for their own datasets. This presentation will expand on this objective and highlight the capabilities available to the international community now.

GHRC↗

NASA NTRS2006

Characterizing the LANDSAT Global Long-Term Data Record

The effects of global climate change are fast becoming politically, sociologically, and personally important: increasing storm frequency and intensity, lengthening cycles of drought and flood, expanding desertification and soil salinization. A vital asset in the analysis of climate change on a global basis is the 34-year record of Landsat imagery. In recognition of its increasing importance, a detailed analysis of the Landsat observation coverage within the US archive was commissioned. Results to date indicate some unexpected gaps in the US-held archive. Fortunately, throughout the Landsat program, data have been downlinked routinely to International Cooperator (IC) ground stations for archival, processing, and distribution. These IC data could be combined with the current US holdings to build a nearly global, annual observation record over this 34-year period. Today, we have inadequate information as to which scenes are available from which IC archives. Our best estimate is that there are over four million digital scenes in the IC archives, compared with the nearly two million scenes held in the US archive. This vast pool of Landsat observations needs to be accurately documented, via metadata, to determine the existence of complementary scenes and to characterize the potential scope of the global Landsat observation record. Of course, knowing the extent and completeness of the data record is but the first step. It will be necessary to assure that the data record is easy to use, internally consistent in terms of calibration and data format, and fully accessible in order to fully realize its potential.

Arvidson, T.↗

NASA NTRS2011

Automated X-ray and Optical Analysis of the Virtual Observatory and Grid Computing

We are developing a system to combine the Web Enabled Source Identification with X-Matching (WESIX) web service, which emphasizes source detection on optical images,with the XAssist program that automates the analysis of X-ray data. XAssist is continuously processing archival X-ray data in several pipelines. We have established a workflow in which FITS images and/or (in the case of X ray data) an X-ray field can be input to WESIX. Intelligent services return available data (if requested fields have been processed) or submit job requests to a queue to be performed asynchronously. These services will be available via web services (for non-interactive use by Virtual Observatory portals and applications) and through web applications (written in the Django web application framework). We are adding web services for specific XAssist functionality such as determining .the exposure and limiting flux for a given position on the sky and extracting spectra and images for a given region. We are improving the queuing system in XAssist to allow for "watch lists" to be specified by users, and when X-ray fields in a user's watch list become publicly available they will be automatically added to the queue. XAssist is being expanded to be used as a survey planning 1001 when coupled with simulation software, including functionality for NuStar, eRosita, IXO, and the Wide Field Xray Telescope (WFXT), as part of an end to end simulation/analysis system. We are also investigating the possibility of a dedicated iPhone/iPad app for querying pipeline data, requesting processing, and administrative job control.

Ptak, A.↗

NASA NTRS1989

The Alaska SAR processor - Operations and control

The Alaska SAR (synthetic-aperture radar) Facility (ASF) will be capable of receiving, processing, archiving, and producing a variety of SAR image products from three satellite-borne SARs: E-ERS-1 (ESA), J-ERS-1 (NASDA) and Radarsat (Canada). Crucial to the success of the ASF is the Alaska SAR processor (ASP), which will be capable of processing over 200 100-km x 100-km (Seasat-like) frames per day from the raw SAR data, at a ground resolution of about 30 m x 30 m. The processed imagery is of high geometric and radiometric accuracy, and is geolocated to within 500 m. Special-purpose hardware has been designed to execute a SAR processing algorithm to achieve this performance. This hardware is currently undergoing acceptance testing for delivery to the University of Alaska. Particular attention has been devoted to making the operations semi-automated and to providing a friendly operator interface via a computer workstation. The operations and control of the Alaska SAR processor are described.

Carande, Richard E.↗

NASA NTRSDate not supplied

Software Development for SonicSonde Instrumentation Suite

The SonicSonde is an innovative weather instrumentation suite for boundary layer sampling. Software development included creating a graphical user interface (GUI) to receive, process, archive, and display data in real-time. Targeted at the Windows operating system but with multi-platform extensibility in mind, the GUI was developed using Qt 5.14 in C++. A key design consideration was to allow for tailored displays and analysis while also permitting additional sensors to be added to the platform. As such, the application relies heavily on flexible interfaces and multithreading processing for capturing, modelling, and plotting data. The object-oriented nature of Qt made it ideal for this purpose. The application supports input through live serial streams and static .csv or .dat files, custom meteorological graphs, and user-dictated panel views and output intervals. Additionally, the GUI displays real-time information about the state of the instrument. The versatility of the SonicSonde will allow meteorologists and researchers to perform or better support a variety of aeronautic and atmospheric missions.

meterology↗

NASA NTRS2008

An Ontology-Based Archive Information Model for the Planetary Science Community

The Planetary Data System (PDS) information model is a mature but complex model that has been used to capture over 30 years of planetary science data for the PDS archive. As the de-facto information model for the planetary science data archive, it is being adopted by the International Planetary Data Alliance (IPDA) as their archive data standard. However, after seventeen years of evolutionary change the model needs refinement. First a formal specification is needed to explicitly capture the model in a commonly accepted data engineering notation. Second, the core and essential elements of the model need to be identified to help simplify the overall archive process. A team of PDS technical staff members have captured the PDS information model in an ontology modeling tool. Using the resulting knowledge-base, work continues to identify the core elements, identify problems and issues, and then test proposed modifications to the model. The final deliverables of this work will include specifications for the next generation PDS information model and the initial set of IPDA archive data standards. Having the information model captured in an ontology modeling tool also makes the model suitable for use by Semantic Web applications.

Information Model↗

NASA NTRS2004

The Lifecycle of NASA's Earth Science Enterprise Data Resources

A major endeavor of NASA's Earth Science Enterprise (ESE) is to acquire, process, archive and distribute data from Earth observing satellites in support of a broad set of science research and applications in the U. S. and abroad. NASA policy directives specifically call for the agency to collect, announce, disseminate and archive all scientific and technical data resulting from NASA and NASA-funded research. During the active life of the satellite missions, while the data products are being created, validated and refined, a number of NASA organizations have the responsibility for data and information system functions. Following the completion of the missions, the responsibility for the long-term stewardship of the ocean and atmospheric, and land process data products transitions to the National Oceanic and Atmospheric Administration (NOAA) and the U.S. Geological Survey (USGS), respectively. Ensuring that long-term satellite data be preserved to support global climate change studies and other research topics and applications presents some major challenges to NASA and its partners. Over the last several years, with the launch and operation of the EOS satellites and the acquisition and production of an unprecedented volume of Earth science data, the importance of addressing these challenges has been elevated. The lifecycle of NASA's Earth science data has been the subject of several agency and interagency studies and reports and has implications and effects on agency charters, policies and budgets and on their data system's requirements, implementation plans and schedules. While much remains to be done, considerable progress has been made in understanding and addressing the data lifecycle issues.

McDonald, Kenneth R.↗

NASA NTRS2018

Improving AIRS NRT Imagery for the Enhancement of Visualization Services in LANCE

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) has been the home of processing, archiving, and distribution services for data from the Atmospheric Infrared Sounder (AIRS) mission since its launch in 2002. AIRS provides data enabling global observations of the atmospheric state. The GES DISC provides service to both AIRS standard products and Near Real-time (NRT) products. The AIRS NRT product is one important element in the Land, Atmosphere Near real-time Capability for EOS (LANCE). The LANCE processing of the AIRS NRT product and the generation of the imagery are performed at the GES DISC. The AIRS NRT imagery are generated by mosaicking and mapping the available AIRS 6-minute retrieval granules to a global cylindrical projection. The images are constantly refreshed when new granules are produced. The AIRS NRT Viewer and LANCE Worldview provide visualization services to online users for AIRS NRT imagery. The imagery include atmospheric temperature, humidity, precipitation, Dust Score, CO, and SO2. The AIRS Applications Development Team at NASA JPL developed a new orbit-based algorithm and software to improve the AIRS NRT imagery. The GES DISC is collaborating with the AIRS Applications Development Team for the implementation of the new algorithm and software. The improvements include image quality, new color palettes, and variable changes. In this presentation, we will detail the improvements and demonstrate visualization of the new imagery.

Ding, Feng↗

NASA NTRSDate not supplied

Hydrological Data at the NASA GES DISC: Current Capabilities and New Opportunities

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is one of twelve NASA Earth science data centers that document, process, archive and distribute data from Earth observation missions and projects. GES DISC maintains an archive of several hydrology datasets, including the Land Data Assimilation Systems (LDAS) and the Gravity Recovery and Climate Experiment (GRACE) Data Assimilation for Drought Monitoring (GRACE-DA-DM) data products. These datasets include model output of heat fluxes, rain, snow, soil temperature, soil moisture, and runoff; and observational forcing data, including surface pressure, temperature, precipitation, downward shortwave and longwave radiation, humidity, and wind. The temporal resolution of the hydrology data at GES DISC ranges from hourly to monthly, and spatial resolutions range from 0.1° to 1.0°. The GES DISC provides services which enable users to aggregate, temporally and spatially subset, regrid, and visualize archived data including the GES DISC Subsetter, Hydrology Data Rods, and the Geospatial Interactive Online Visualization and Analysis Infrastructure (GIOVANNI). The Hydrology Data Rods service optimally reorganizes large hydrological data sets as extended time series, providing more efficient access for the hydrological community. The time series data (aka “data rods”) were integrated into hydrology community tools, such as the Data Rods Explorer on HydroShare. Furthermore, the GES DISC is in the process of migrating its data and services to the cloud. Hydrological data available at the GES DISC are now available in the Amazon Web Services (AWS) cloud (us-west-2 region) providing users Direct S3 data access and the capability for cloud computing operations. In this presentation, the hydrology data products and services currently available at the GES DISC will be summarized. Also discussed are the migration to the cloud, user support through this transition, and the status of migrating the data rods service to the cloud.

Ashley Heath↗

NASA NTRS2014

Temperature and Humidity Profiles in the TqJoint Data Group of AIRS Version 6 Product for the Climate Model Evaluation

The Atmospheric Infrared Sounder (AIRS) mission is entering its 13th year of global observations of the atmospheric state, including temperature and humidity profiles, outgoing long-wave radiation, cloud properties, and trace gases. Thus AIRS data have been widely used, among other things, for short-term climate research and observational component for model evaluation. One instance is the fifth phase of the Coupled Model Intercomparison Project (CMIP5) which uses AIRS version 5 data in the climate model evaluation. The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is the home of processing, archiving, and distribution services for data from the AIRS mission. The GES DISC, in collaboration with the AIRS Project, released data from the version 6 algorithm in early 2013. The new algorithm represents a significant improvement over previous versions in terms of greater stability, yield, and quality of products. The ongoing Earth System Grid for next generation climate model research project, a collaborative effort of GES DISC and NASA JPL, will bring temperature and humidity profiles from AIRS version 6. The AIRS version 6 product adds a new "TqJoint" data group, which contains data for a common set of observations across water vapor and temperature at all atmospheric levels and is suitable for climate process studies. How different may the monthly temperature and humidity profiles in "TqJoint" group be from the "Standard" group where temperature and water vapor are not always valid at the same time? This study aims to answer the question by comprehensively comparing the temperature and humidity profiles from the "TqJoint" group and the "Standard" group. The comparison includes mean differences at different levels globally and over land and ocean. We are also working on examining the sampling differences between the "TqJoint" and "Standard" group using MERRA data.

climate model evaluation↗

NASA NTRS2018

Toward Long-Term Aquatic Science Products from Heritage Landsat Missions

This paper aims at generating a long-term consistent record of Landsat-derived remote sensing reflectance (Rrs) products, which are central for producing downstream aquatic science products (e.g., concentrations of total suspended solids). The products are derived from Landsat-5 and Landsat-7 observations leading to Landsat-8 era to enable retrospective analyses of inland and nearshore coastal waters. In doing so, the data processing was built into the SeaWiFS Data Analysis System (SeaDAS) followed by vicariously calibrating Landsat-7 and -5 data using reference in situ measurements and near-concurrent ocean color products, respectively. The derived Rrs products are then validated using (a) matchups using the Aerosol Robotic Network (AERONET) data measured by in situ radiometers, i.e., AERONET-OC, and (b) ocean color products at select sites in North America. Following the vicarious calibration adjustments, it is found that the overall biases in Rrs products are significantly reduced. The root-mean-square errors (RMSE), however, indicate noticeable uncertainties due to random and systematic noise. Long-term (since 1984) seasonal Rrs composites over 12 coastal and inland systems are further evaluated to explore the utility of Landsat archive processed via SeaDAS. With all the qualitative and quantitative assessments, it is concluded that with careful algorithm developments, it is possible to discern natural variability in historic water quality conditions using heritage Landsat missions. This requires the changes in Rrs exceed maximum expected uncertainties, i.e., 0.0015 [1/sr], estimated from mean RMSEs associated with the matchups and intercomparison analyses. It is also anticipated that Landsat-5 products will be less susceptible to uncertainties in turbid waters with Rrs(660) > 0.004 [1/sr], which is equivalent of ~1.2% reflectance. Overall, end-users may utilize heritage Rrs products with "fitness-for-purpose" concept in mind, i.e., products could be valuable for one application but may not be viable for another. Further research should be dedicated to enhancing atmospheric correction to account for non-negligible near-infrared reflectance in CDOM-rich and extremely turbid waters.

Pahlevan, Nima↗

NASA NTRSDate not supplied

Atmospheric Infrared Sounder Version 7 Near-Real-Time Product and Imagery Released by NASA GES DISC

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) has been the home of data processing, archive, and distribution services for data from the Atmospheric Infrared Sounder (AIRS) mission since its launch in 2002. The GES DISC provides service to both AIRS routine and Near Real-Time (NRT) products. The AIRS NRT products are an important element in the Land, Atmosphere Near real-time Capability for EOS (LANCE). In collaboration with AIRS Project, the GES DISC has just released products from the Version 7 algorithm. The new version algorithm provides significant improvements over the previous version. The most substantial advances are: improved consistency between day and night water vapor; improved total column ozone and temperature; improved infrared-only (IR-only) retrievals, especially in high latitude regions; an improved Stochastic Cloud Clearing Neural Network used as a first guess at the initial value in the iterative retrieval process; and removal of ambiguity in surface classification in the IR-only retrieval algorithm. In addition, the GES DISC produces AIRS NRT imagery. The AIRS NRT imagery are generated by mosaicking and mapping the available AIRS 6-minute retrieval granules to a global projection. The images are constantly refreshed when new granules are produced. The AIRS NRT Viewer and LANCE Worldview provide visualization services to online users for AIRS NRT imagery. The data products used to generate this imagery include atmospheric temperature, humidity, precipitation, cloud, Dust Score, CO, and SO2. In this presentation, we will demonstrate visualization of the AIRS NRT imagery from the new Version 7, and demonstrate some improvements over the previous version. Progress on improving the AIRS NRT imagery, a collaboration project with the AIRS Applications Development Team at NASA Jet Propulsion Laboratory (JPL), will also be presented.

Feng Ding↗