SEARCH · Engineering Papers

Results for “Science Data Processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19

All sources NASA NTRS DOE OSTI

NASA NTRS2014

An Analysis of Earth Science Data Analytics Use Cases

The increase in the number and volume, and sources, of globally available Earth science data measurements and datasets have afforded Earth scientists and applications researchers unprecedented opportunities to study our Earth in ever more sophisticated ways. In fact, the NASA Earth Observing System Data Information System (EOSDIS) archives have doubled from 2007 to 2014, to 9.1 PB (Ramapriyan, 2009; and https:earthdata.nasa.govaboutsystem-- performance). In addition, other US agency, international programs, field experiments, ground stations, and citizen scientists provide a plethora of additional sources for studying Earth. Co--analyzing huge amounts of heterogeneous data to glean out unobvious information is a daunting task. Earth science data analytics (ESDA) is the process of examining large amounts of data of a variety of types to uncover hidden patterns, unknown correlations and other useful information. It can include Data Preparation, Data Reduction, and Data Analysis. Through work associated with the Earth Science Information Partners (ESIP) Federation, a collection of Earth science data analytics use cases have been collected and analyzed for the purpose of extracting the types of Earth science data analytics employed, and requirements for data analytics tools and techniques yet to be implemented, based on use case needs. ESIP generated use case template, ESDA use cases, use case types, and preliminary use case analysis (this is a work in progress) will be presented.

data analytics↗

NASA NTRS1995

Upper Atmosphere Research Satellite Validation Workshop III: Temperature and Constituents Validation

The Upper Atmosphere Research Satellite (UARS) was launched in September 1991. Since that time data have been retrieved continuously from the various instruments on the UARS spacecraft. These data have been processed by the respective instrument science teams and subsequently archived in the UARS Central Data Handling Facility (CDHF) at the NASA Goddard Space Flight Center, Greenbelt, Maryland. This report contains the proceedings from one of the three workshops held to evaluate the progress in validating UARS constituents and temperature data and to document the quality of that data. The first workshop was held in Oxford, England, in March 1992, five and one-half months after UARS launch. The second workshop was held in Boulder, Colorado in October 1992. Since launch, the various data have undergone numerous revisions. In many instances these revisions are a result of data problems identified during the validation workshops. Thus, the formal validation effort is a continually ongoing process.

Grose, William L.↗

NASA NTRS2017

Restoration of the Apollo 15 Heat Flow Experiment Data from 1975 to 1977

The Apollo 15 Heat Flow Experiment (HFE) was conducted from July 1971 through January 1977. Two heat flow probes were deployed roughly 8.5 meters apart. Probe 1 and Probe 2 penetrated to 1.4-meters and 1-meter depths into the lunar regolith, respectively. Temperatures at different depths and the surface were logged with 7.25-minute intervals and transmitted to Earth. At the conclusion of the experiment, only data obtained from July 1971 through December 1974 were processed and archived at the National Space Science Data Center (NSSDC) by the principal investigator of the experiment, Marcus Langseth of Columbia University. Langseth died in 1997. It is not known what happened to the HFE data tapes he used. Current researchers have strong interests in re-examining the HFE data for the full duration of the experiment. We have recovered and processed large portions of the Apollo 15 HFE data from 1975 through 1977 by assembling data and metadata from various sources.

HFE↗

NASA NTRS2015

NASA'S Earth Science Data Stewardship Activities

NASA has been collecting Earth observation data for over 50 years using instruments on board satellites, aircraft and ground-based systems. With the inception of the Earth Observing System (EOS) Program in 1990, NASA established the Earth Science Data and Information System (ESDIS) Project and initiated development of the Earth Observing System Data and Information System (EOSDIS). A set of Distributed Active Archive Centers (DAACs) was established at locations based on science discipline expertise. Today, EOSDIS consists of 12 DAACs and 12 Science Investigator-led Processing Systems (SIPS), processing data from the EOS missions, as well as the Suomi National Polar Orbiting Partnership mission, and other satellite and airborne missions. The DAACs archive and distribute the vast majority of data from NASA’s Earth science missions, with data holdings exceeding 12 petabytes The data held by EOSDIS are available to all users consistent with NASA’s free and open data policy, which has been in effect since 1990. The EOSDIS archives consist of raw instrument data counts (level 0 data), as well as higher level standard products (e.g., geophysical parameters, products mapped to standard spatio-temporal grids, results of Earth system models using multi-instrument observations, and long time series of Earth System Data Records resulting from multiple satellite observations of a given type of phenomenon). EOSDIS data stewardship responsibilities include ensuring that the data and information content are reliable, of high quality, easily accessible, and usable for as long as they are considered to be of value.

metadata↗

NASA NTRSDate not supplied

Open-Source Science-Driven Development of the Science Data System (SDS) for Earth System Observatory (ESO) Atmospheric Missions

The NASA Earth System Observatory (ESO) atmospheric missions will provide space-based and suborbital observations of collocated cloud, dynamic, precipitation and aerosol processing leading to improved weather, air quality, and climate predictions. The Science Data System (SDS) will deploy the adaptive processing system (APS) developed within the Cloud to manage the research and operational processing of ESO atmospheric mission orbital and suborbital sensors and curate these data for near real-time and collection reprocessing and transfer them to a NASA Distributed Active Archive Center (DAAC) for long-term storage and distribution. Further, the SDS will follow guidelines provided by NASA Earth Science Data Systems (ESDS) program including standard conventions for data file formats, naming, and metadata to improve data interoperability, interpretability, usability, discovery, provenance, and spatiotemporal representativeness. The SDS follows NASA’s commitment to Open-Source Science (OSS) including the sharing of data, software, and knowledge in an open and timely manner. Each of the SDS system components will be developed with open-source concepts including components of APS itself as well as ESO atmospheric mission algorithms. This presentation describes the framework of the SDS and its integral part in facilitating OSS within the ESO atmospheric missions.

David M. Giles↗

NASA NTRS2011

Provenance Challenges for Earth Science Dataset Publication

Modern science is increasingly dependent on computational analysis of very large data sets. Organizing, referencing, publishing those data has become a complex problem. Published research that depends on such data often fails to cite the data in sufficient detail to allow an independent scientist to reproduce the original experiments and analyses. This paper explores some of the challenges related to data identification, equivalence and reproducibility in the domain of data intensive scientific processing. It will use the example of Earth Science satellite data, but the challenges also apply to other domains.

Tilmes, Curt↗

NASA NTRS2022

Terra and Aqua MODIS Collection 7 Level 1B Algorithm

MODIS continues to be an important instrument for NASA’s Earth Observing System (EOS). Terra and Aqua MODIS have produced more than 22 and 20 years of global datasets that have significantly helped scientists better understand the Earth’s systems respectively. The MODIS Level-1B (L1B) algorithms use the uncalibrated, geolocated Earth scene observations as input and convert the instrument response into calibrated reflectance and radiance, which are used to generate the downstream science products. The sustained calibration and characterization activities undertaken by the MODIS Characterization Support Team have resulted in several upgrades to the L1B algorithms in order to maintain accurate calibration in the data products. In this paper, we present an overview of the L1B algorithm designated as Collection 7. Various algorithm enhancements both in the reflective bands and thermal bands characterization, are currently under science testing and evaluation. Once applied in data processing (projected in early 2023), they are expected to manifest in improved science products, both in terms of radiometric accuracy and long-term stability.

MODIS↗

NASA NTRS1993

Tensoral: A system for post-processing turbulence simulation data

Many computer simulations in engineering and science -- and especially in computational fluid dynamics (CFD) -- produce huge quantities of numerical data. These data are often so large as to make even relatively simple post-processing of this data unwieldy. The data, once computed and quality-assured, is most likely analyzed by only a few people. As a result, much useful numerical data is under-utilized. Since future state-of-the-art simulations will produce even larger datasets, will use more complex flow geometries, and will be performed on more complex supercomputers, data management issues will become increasingly cumbersome. My goal is to provide software which will automate the present and future task of managing and post-processing large turbulence datasets. My research has focused on the development of these software tools -- specifically, through the development of a very high-level language called 'Tensoral'. The ultimate goal of Tensoral is to convert high-level mathematical expressions (tensor algebra, calculus, and statistics) into efficient low-level programs which numerically calculate these expressions given simulation datasets. This approach to the database and post-processing problem has several advantages. Using Tensoral the numerical and data management details of a simulation are shielded from the concerns of the end user. This shielding is carried out without sacrificing post-processor efficiency and robustness. Another advantage of Tensoral is that its very high-level nature lends itself to portability across a wide variety of computing (and supercomputing) platforms. This is especially important considering the rapidity of changes in supercomputing hardware.

Dresselhaus, Eliot↗

NASA NTRS2016

Astro-H Data Analysis, Processing and Archive

Astro-H (Hitomi) is an X-ray Gamma-ray mission led by Japan with international participation, launched on February 17, 2016. The payload consists of four different instruments (SXS, SXI, HXI and SGD) that operate simultaneously to cover the energy range from 0.3 keV up to 600 keV. This paper presents the analysis software and the data processing pipeline created to calibrate and analyze the Hitomi science data along with the plan for the archive and user support.These activities have been a collaborative effort shared between scientists and software engineers working in several institutes in Japan and USA.

Archive↗

NASA NTRS2018

Astro-H/Hitomi Data Analysis, Processing, and Archive

Astro-H is the x-ray/gamma-ray mission led by Japan with international participation, launched on February 17, 2016. Soon after launch, Astro-H was renamed Hitomi. The payload consists of four different instruments (SXS, SXI, HXI, and SGD) that operate simultaneously to cover the energy range from 0.3 keV up to 600 keV. On March 27, 2016, JAXA lost contact with the satellite and, on April 28, they announced the cessation of the efforts to restore mission operations. Hitomi collected about one months worth of data with its instruments. This paper presents the analysis software and the data processing pipeline created to calibrate and analyze the Hitomi science data, along with the plan for the archive. These activities have been a collaborative effort shared between scientists and software engineers working in several institutes in Japan and United States.

Angelini, Lorella↗

NASA NTRSDate not supplied

An Update on the Hydrological Land Surface Data and Services at NASA GES DISC

"The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is one of twelve NASA Earth Observing System data centers that document, process, archive and distribute data from Earth science missions and related projects. The GES DISC hosts many hydrological land surface data products including North American Land Data Assimilation System (NLDAS), Global Land Data Assimilation System (GLDAS), the Famine Early Warning Systems Network (FEWS NET) Land Data Assimilation System (FLDAS), and assimilated-Gravity Recovery and Climate Experiment (GRACE). The Land Surface Model (LSM) products contain model output, including heat fluxes, rain, snow, soil temperature, soil moisture and runoff; and observational forcing data, including surface pressure, temperature, precipitation, downward shortwave and longwave radiation, humidity, and wind. The temporal resolution of the hydrology data at the GES DISC ranges from hourly to monthly and the spatial resolution 0.1 degree to 1.0 degree. The variability of Earth’s water and energy cycles can be characterized with these spatially and temporally consistent, and quality controlled hydrological land surface data. These data could support ongoing modeling activities, and help improve our understanding of land-surface-atmosphere interactions and their impact on climate. Over the years, new data is added, and older data is reprocessed and updated. This presentation provides a summary table of the hydrological data and discusses recent updates to the data and data services at the GES DISC. New data products include (1) GRACE Global Version 3.0 drought indicators and (2) FLDAS monthly global and Central Asia daily data. Reprocessed and updated products include (1) NLDAS Version 2.0 and (2) GRACE United States Version 4.0 drought indicators. Lastly, this presentation will discuss the opportunities for accessing some of our hydrology data from the cloud. As the hydrology land surface data is moved to the cloud, users can expect faster service for longer time series, notably the hydrology data rods time series service."

Ashley Heath↗

NASA NTRSDate not supplied

How GES DISC is Opening Doors to Open Science

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is one of 12 NASA Earth Observing System data centers that document, process, archive, and distribute data from Earth science missions and related projects. GES DISC hosts numerous Earth Science products through various data formats and supports their access with several programming and website tools. The User Needs team at GES DISC is responsible for producing tutorials and resources to teach these tools, collecting and engaging with user feedback, and collaborating with other NASA data centers to enable science creation for everyone. This poster presentation outlines how GES DISC, and the User Needs team, are committed to enabling open science.

Christopher Battisto↗

NASA NTRSDate not supplied

How GES DISC is Opening Doors to Open Science

Christopher Battisto↗

NASA NTRS1998

Scale in Remote Sensing and GIS: An Advancement in Methods Towards a Science of Scale

The term "scale", both in space and time, is central to remote sensing and Geographic Information Systems (GIS). The emergence and widespread use of GIS technologies, including remote sensing, has generated significant interest in addressing scale as a generic topic, and in the development and implementation of techniques for dealing explicitly with the vicissitudes of scale as a multidisciplinary issue. As science becomes more complex and utilizes databases that are capable of performing complex space-time data analyses, it becomes paramount that we develop the tools and techniques needed to operate at multiple scales, to work with data whose scales are not necessarily ideal, and to produce results that can be aggregated or disaggregated ways that suit the decision-making process. Contemporary science is constantly coping with compromises, and the data available for a particular study rarely fit perfectly with the scales at which the processes being investigated operate, or the scales that policy-makers require to make sound, rational decisions. This presentation discusses some of the problems associated with scale as related to remote sensing and GIS, and describes some of the questions that need to be addressed in approaching the development of a multidisciplinary "science of scale". Techniques for dealing with multiple scaled data that have been developed or explored recently are described as a means for recognizing scale as a generic issue, along with associated theory and tools that can be of simultaneous value to a large number of disciplines. These can be used to seek answers to a host of interrelated questions in the interest of providing a formal structure for the management and manipulation of scale and its universality as a key concept from a multidisciplinary perspective.

Quattrochi, D. A.↗

NASA NTRS1998

Scale in Remote Sensing and GIS: An Advancement in Methods Towards a Science of Scale

The term "scale", both in space and time, is central to remote sensing and geographic information systems (GIS). The emergence and widespread use of GIS technologies, including remote sensing, has generated significant interest in addressing scale as a generic topic, and in the development and implementation of techniques for dealing explicitly with the vicissitudes of scale as a multidisciplinary issue. As science becomes more complex and utilizes databases that are capable of performing complex space-time data analyses, it becomes paramount that we develop the tools and techniques needed to operate at multiple scales, to work with data whose scales are not necessarily ideal, and to produce results that can be aggregated or disaggregated in ways that suit the decision-making process. Contemporary science is constantly coping with compromises, and the data available for a particular study rarely fit perfectly with the scales at which the processes being investigated operate, or the scales that policy-makers require to make sound, rational decisions. This presentation discusses some of the problems associated with scale as related to remote sensing and GIS, and describes some of the questions that need to be addressed in approaching the development of a multidisciplinary "science of scale". Techniques for dealing with multiple scaled data that have been developed or explored recently are described as a means for recognizing scale as a generic issue, along with associated theory and tools that can be of simultaneous value to a large number of disciplines. These can be used to seek answers to a host of interrelated questions in the interest of providing a formal structure for the management and manipulation of scale and its universality as a key concept from a multidisciplinary perspective.

Quattrochi, Dale A.↗

NASA NTRS2000

KDD Services at the Goddard Earth Sciences Distributed Active Archive Center

NASA's Goddard Earth Sciences Distributed Active Archive Center (GES DAAC) processes, stores and distributes earth science data from a variety of remote sensing satellites. End users of the data range from instrument scientists to global change and climate researchers to federal agencies and foreign governments. Many of these users apply data mining techniques to large volumes of data (up to 1 TB) received from the GES DAAC. However, rapid advances in processing power are enabling increases in data processing that are outpacing tape drive performance and network capacity. As a result, the proportion of data that can be distributed to users continues to decrease. As mitigation, we are migrating more data mining and mining preparation activities into the data center in order to reduce the data volume that needs to be distributed and to offer the users a more useful and manageable product. This migration of activities faces a number of technical and human-factor challenges. As data reduction and mining algorithms are normally quite specific to the user's research needs, the user's algorithm must be integrated virtually unchanged into the archive environment. Also, the archive itself is busy with everyday data archive and distribution activities and cannot be dedicated to, or even impacted by, the mining activities. Therefore, we schedule KDD 'campaigns' (similar to reprocessing campaigns), during which we schedule a wholesale retrieval of specific data products, offering users the opportunity to extract information from the data being retrieved during the campaign.

Lynnes, Christopher↗

NASA NTRS2020

Back to the Future: Surveying the Northern Hemisphere and Reprocessing the Southern TESS Data Set

TESS launched 18 April 2018 to conduct a two-year, near all-sky survey for at least 50 small, nearby exoplanets for which masses can be ascertained and whose atmospheres can be characterized by ground- and space-based follow-on observations. TESS has completed its survey of the southern hemisphere and begun its survey of the northern hemisphere, identifying >1000 candidate exoplanets and unveiling a plethora of exciting non-exoplanet astrophysics results, such as asteroseismology, asteroids, and supernova. The TESS Science Processing Operations Center (SPOC) processes the data downlinked every two weeks to generate a range of data products hosted at the Mikulski Archive for Space Telescopes (MAST). For each sector (~1 month) of observations, the SPOC calibrates the image data for both 30-min Full Frame Images (FFIs) and up to 20,000 pre-selected 2-min target star postage stamps. Data products for the 2-min targets include simple aperture photometry and systematic error-corrected flux time series. The SPOC also conducts searches for transiting exoplanets in the 2-min data for each sector and generates Data Validation time series and associated reports for each transit-like feature identified in the search. Multi-sector searches for exoplanets are conducted periodically to discover longer period planets, including those in the James Webb Continuous Viewing Zone (CVZ), which are observed for up to one year. Starting with Sector 8, scattered light from the Earth and Moon contaminated significant portions of the data in each orbit. We have developed algorithms for automated identification of the scattered light features at the individual target level. Previously, data for all stars on a CCD affected by scattered light were manually excluded. The automated flagging will allow us to retain significantly more data for stars that are not affected by the scattered light even though it is occurring elsewhere on the CCD. We also discuss enhancements to the SPOC pipeline and the newly available FFI light curves. The TESS Mission is funded by NASA's Science Mission Directorate as an Astrophysics Explorer Mission.

Jenkins, Jon M.↗

NASA NTRS2019

NASA's Fire Information for Resource Management System (FIRMS): Near Real-Time Global Fire Monitoring Using Data from MODIS and VIIRS

NASA's Fire Information for Resource Management System (FIRMS) provides near-real time active fire / hotspot products from MODIS and VIIRS to users in over 160 countries. The goal of FIRMS is to meet the needs of natural resource and protected area managers that face considerable challenges in obtaining timely satellite-derived information on fires burning within and around their management area. FIRMS has been reliably providing active fire / hotspot data in easy to use formats since its inception in 2006. Fire information is provided through a web map interface, email alerts, a web mapping service, and a range of downloadable files (SHP, CSV, KML and JSON). FIRMS data are used directly by end users and by brokers who take the data and add value to it before re-distributing it.FIRMS was initially developed by the University of Maryland in 2006; it was funded by the United Nations FAO and NASA's Applied sciences program under a NASA ROSES call. In 2012 FIRMS became integrated in to NASA's Land Atmosphere Near real-time Capability for EOS (LANCE); a virtual system that leverages NASAs existing science processing capabilities to deliver NRT data from ten instruments within 3 hours of satellite overpass. This presentation will describe the FIRMS system, provide an overview of the system, briefly describe some of the known applications and describe plans to further integrate the data in to NASA's Global Imagery Browse Services (GIBS) public mapping services and Worldview website.

active fire/hotspots↗