Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Web Services”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12

GC31G-1182: Opennex, a Private-Public Partnership in Support of the National Climate Assessment

The NASA Earth Exchange (NEX) is a collaborative computing platform that has been developed with the objective of bringing scientists together with the software tools, massive global datasets, and supercomputing resources necessary to accelerate research in Earth systems science and global change. NEX is funded as an enabling tool for sustaining the national climate assessment. Over the past five years, researchers have used the NEX platform and produced a number of data sets highly relevant to the National Climate Assessment. These include high-resolution climate projections using different downscaling techniques and trends in historical climate from satellite data. To enable a broader community in exploiting the above datasets, the NEX team partnered with public cloud providers to create the OpenNEX platform. OpenNEX provides ready access to NEX data holdings on a number of public cloud platforms along with pertinent analysis tools and workflows in the form of Machine Images and Docker Containers, lectures and tutorials by experts. We will showcase some of the applications of OpenNEX data and tools by the community on Amazon Web Services, Google Cloud and the NEX Sandbox.

datasets↗

A Relevancy Algorithm for Curating Earth Science Data Around Phenomenon

Earth science data are being collected for various science needs and applications, processed using different algorithms at multiple resolutions and coverages, and then archived at different archiving centers for distribution and stewardship causing difficulty in data discovery. Curation, which typically occurs in museums, art galleries, and libraries, is traditionally defined as the process of collecting and organizing information around a common subject matter or a topic of interest. Curating data sets around topics or areas of interest addresses some of the data discovery needs in the field of Earth science, especially for unanticipated users of data. This paper describes a methodology to automate search and selection of data around specific phenomena. Different components of the methodology including the assumptions, the process, and the relevancy ranking algorithm are described. The paper makes two unique contributions to improving data search and discovery capabilities. First, the paper describes a novel methodology developed for automatically curating data around a topic using Earthscience metadata records. Second, the methodology has been implemented as a standalone web service that is utilized to augment search and usability of data in a variety of tools.

earth science phenomena↗

GEOS-5 During ORACLES: Status Update

In this talk we summarize the GEOS-5 capabilities to be deployed during the ORACLES 2016 Campaign. We describe model configuration, data products and web services available. We also discuss the measurement and flight requirements for the GEOS-5 Team.

Cloud↗

Enhancement of Mutual Discovery, Search, and Access of Data for Users of NASA and GEOSS-Cataloged Data Systems

An ongoing NASA-funded Data Rods (time series) project has demonstrated the removal of a longstanding barrier to accessing NASA data (i.e., accessing archived time-step array data as point-time series) for selected variables of the North American and Global Land Data Assimilation Systems (NLDAS and GLDAS, respectively) and other NASA data sets. Data rods are pre-generated or generated on-the-fly (OTF), leveraging the NASA Simple Subset Wizard (SSW), a gateway to NASA data centers. Data rods Web services are accessible through the CUAHSI Hydrologic Information System (HIS) and the Goddard Earth Sciences Data and Information Services Center (GES DISC) but are not easily discoverable by users of other non-NASA data systems. An ongoing GEOSS Water Services project aims to develop a distributed, global registry of water data, map, and modeling services cataloged using the standards and procedures of the Open Geospatial Consortium and the World Meteorological Organization. Preliminary work has shown GEOSS can be leveraged to help provide access to data rods. A new NASA-funded project is extending this early work.

data access↗

Hosting Hyrax in the Cloud While Preserving the User Experience

Cloud object stores enable data providers to reduce costs and provide uniform data servers across organizational divisions. This talk will address how new capabilities provided by cloud computing can be used with existing software so that users get the benefits provided by cloud technology (access to data at lower cost, e.g.) without disrupting their existing workflow(s). We will address ways to reduce response latency often associated with Cloud object stores and ways to simulate the hierarchy of a POSIX file system using a key-value object store. This work is based an on-going NASA EED2 project to enhance the Hyrax data server for operation in the Amazon Web Services Cloud.

Hyrax server↗

Geonex: A NASA-NOAA Collaboration for Producing Land Surface Products from Geostationary Sensors Using Cloud Computing

The latest generation of geostationary satellites carry sensors such as the Advanced Baseline Imager (GOES-16/17) and the Advanced Himawari Imager (Himawari-8/9) that closely mimic the spatial and spectral characteristics of MODIS and VIIRS, useful for monitoring land surface conditions. The NASA Earth Exchange (NEX) team at Ames Research Center has embarked on a collaborative effort among scientists from NASA and NOAA exploring the feasibility of producing operational land surface products similar to those from MODIS/VIIRS. The team built a processing pipeline called GEONEX that is capable of converting raw geostationary data into routine products of Fires, surface reflectances, vegetation indices, LAI/FPAR, ET and GPP/NPP using algorithms adapted from both NASA/EOS and NOAA/GOES-R programs. The GEONEX pipeline has been deployed on Amazon Web Services cloud platform and it currently leverages near-realtime geostationary data hosted in AWS public datasets under a NOAA-AWS agreement.Initial analyses of various products from ABI/AHI sensors suggest that they are comparable to those from MODIS in representing the spatio-temporal dynamics of land conditions. Cloud computing offers a variety of options for deploying the GEONEX pipeline including choice CPUs, storage media, and automation. We estimate the cost of deploying GEONEX to be $400 - 750 a month for processing data (every 30 minutes) and producing products over the conterminous US. For products such as Fire, latency can be as little as 10 minutes from the time of data acquisition.

Geostationary↗

GEONEX: Webpage to Display NASA-NOAA Collaboration of Producing Land Surface Products from Geostationary Sensors

The latest generation of geostationary satellites carry sensors such as the Advanced Baseline Imager (GOES-16/17) and the Advanced Himawari Imager (Himawari-8/9) that closely mimic the spatial and spectral characteristics of MODIS and VIIRS, useful for monitoring land surface conditions. The NASA Earth Exchange (NEX) team at Ames Research Center has embarked on a collaborative effort among scientists from NASA and NOAA exploring the feasibility of producing operational land surface products similar to those from MODIS/VIIRS. The team built a processing pipeline called GEONEX that is capable of converting raw geostationary data into routine products of Fires, surface reflectances, vegetation indices, LAI/FPAR, ET and GPP/NPP using algorithms adapted from both NASA/EOS and NOAA/GOES-R programs. The GEONEX pipeline has been deployed on Amazon Web Services cloud platform and it currently leverages near-realtime geostationary data hosted in AWS public datasets under a NOAA-AWS agreement.In order to better introduce the GEONEX products to the science community, we set up a simple webpage (www.geonex.org) to describe the background and the motivation of the project, the algorithms used in deriving the products, and user manuals to the data files. We will also update the status of the data processing, in particular the near-real-time products, on the website and provide links (in text or json files) to the latest datasets.

geostationary↗

The SPASE Data Model: A Metadata Standard for Registering, Finding, Accessing, and Using Heliophysics Data Obtained from Observations and Modeling

The Space Physics Archive Search and Extract Consortium has developed and implemented the SPASE Data Model that provides a common language for registering a wide range of Heliophysics data and other products. The Data Model enables discovery and access tools such that any researcher can obtain data easily, thereby facilitating research, including on space weather. The Data Model includes descriptions of Simulation Models and Numerical Output, pioneered by the Integrated Medium for Planetary Exploration (IMPEx) group in Europe, and subsequently adopted by the Community Coordinated Modeling Center (CCMC). The SPASE group intends to register all relevant Heliophysics data resources, including space-, ground-, and model-based. Substantial progress has been made, especially for space-based observational data and associated observatories, instruments, and display data. Legacy product registrations and access go back more than 50 years. Real-time data will be included. The National Aeronautics and Space Administration (NASA) portion of the SPASE group has funding that assures continuity in the upkeep of the Data Model and aids with adding new products. Tools are being developed for making and editing data descriptions. Digital Object Identifiers (DOIs) for Data Products can now be included in the descriptions. The data access that SPASE facilitates is becoming more uniform, and work is progressing on Web Service access via a standard Application Programming Interface. The SPASE Data Model is stable; changes over the past 9 years were additions of terms and capabilities that are backward compatible. This paper provides a summary of the history, structure, use, and future of the SPASE Data Model.

Roberts, D. Aaron↗

Geonex: Land Surface Monitoring from a New Generation of Geostationary Sensors

The latest generation of geostationary satellites carry sensors such as the Advanced Baseline Imager (GOES-16/17) and the Advanced Himawari Imager (Himawari-8/9) that closely mimic the spatial and spectral characteristics of MODIS and VIIRS, useful for monitoring land surface conditions. The NASA Earth Exchange (NEX) team at Ames Research Center has embarked on a collaborative effort among scientists from NASA and NOAA exploring the feasibility of producing operational land surface products similar to those from MODIS/VIIRS. The team built a processing pipeline called GEONEX that is capable of converting raw geostationary data into routine products of Fires, surface reflectances, vegetation indices, LAI/FPAR, ET and GPP/NPP using algorithms adapted from both NASA/EOS and NOAA/GOES-R programs. The GEONEX pipeline has been deployed on Amazon Web Services cloud platform and it currently leverages near-realtime geostationary data hosted in AWS public datasets under a NOAA-AWS agreement. Initial analyses of various products from ABI/AHI sensors suggest that they are comparable to those from MODIS in representing the spatio-temporal dynamics of land conditions. Cloud computing offers a variety of options for deploying the GEONEX pipeline including choice CPUs, storage media, and automation. By making the GEONEX pipeline available on the cloud, we hope to engage a broad community of Earth scientists from around the world in utilizing this new source of data for Earth monitoring.

Nemani, Ramakrishna R.↗

Earth Observations from Geostationary Satellites

The latest generation of geostationary satellites carry sensors such as the Advanced Baseline Imager (GOES-16/17) and the Advanced Himawari Imager (Himawari-8/9) that closely mimic the spatial and spectral characteristics of MODIS and VIIRS, useful for monitoring land surface conditions. The NASA Earth Exchange (NEX) team at Ames Research Center has embarked on a collaborative effort among scientists from NASA and NOAA exploring the feasibility of producing operational land surface products similar to those from MODIS/VIIRS. The team built a processing pipeline called GeoNEX that is capable of converting raw geostationary data into routine products of Fires, surface reflectances, vegetation indices, LAI/FPAR, ET and GPP/NPP using algorithms adapted from both NASA/EOS and NOAA/GOES-R programs. The GeoNEX pipeline has been deployed on Amazon Web Services cloud platform and it currently leverages near-realtime geostationary data hosted in AWS public datasets under a NOAA-AWS agreement. Initial analyses of various products from ABI/AHI sensors suggest that they are comparable to those from MODIS in representing the spatio-temporal dynamics of land conditions. Cloud computing offers a variety of options for deploying the GeoNEX pipeline including choice CPUs, storage media, and automation. By making the GEONEX pipeline available on the cloud, we hope to engage a broad community of Earth scientists from around the world in utilizing this new source of data for Earth monitoring.

Earth↗

GeoNEX: Land Monitoring from a New Generation of Geostationary Sensors

The latest generation of geostationary satellites carry sensors such as the Advanced Baseline Imager (GOES-16/17) and the Advanced Himawari Imager (Himawari-8/9) that closely mimic the spatial and spectral characteristics of MODIS and VIIRS, useful for monitoring land surface conditions. The NASA Earth Exchange (NEX) team at Ames Research Center has embarked on a collaborative effort among scientists from NASA and NOAA exploring the feasibility of producing operational land surface products similar to those from MODIS/VIIRS. The team built a processing pipeline called GEONEX that is capable of converting raw geostationary data into routine products of Fires, surface reflectances, vegetation indices, LAI/FPAR, ET and GPP/NPP using algorithms adapted from both NASA/EOS and NOAA/GOES-R programs. The GEONEX pipeline has been deployed on Amazon Web Services cloud platform and it currently leverages near-realtime geostationary data hosted in AWS public datasets under a NOAA-AWS agreement. Initial analyses of various products from ABI/AHI sensors suggest that they are comparable to those from MODIS in representing the spatio-temporal dynamics of land conditions. Cloud computing offers a variety of options for deploying the GEONEX pipeline including choice CPUs, storage media, and automation. By making the GEONEX pipeline available on the cloud, we hope to engage a broad community of Earth scientists from around the world in utilizing this new source of data for Earth monitoring.

GeoNEX↗

Forming Aggregations using Virtual Sharding: Lessons Learned from Simple Scalable Storage (S3)

Data aggregation is the ability to combine separate datasets to form a single new logical dataset provides users with a powerful abstraction. The advantage of an aggregate dataset is that the users are freed from having to understand, and incorporate into their workflow, knowledge about the (ad hoc) organization of the constituent datasets. However, aggregating large numbers of files can be computationally complex with data server systems performing many repetitive operations. As part of the authors work on subsetting data stored on Amazon Web Service (AWS) Simple Storage Service (S3), we developed technology to read portions of otherwise monolithic data files. This enables the formation of virtual shards for user in subsetting data stored in HDF5 (hierarchical data format, version 5) files. This same tool can be used to form aggregations that combine data stored in many HDF5 files when those files are stored on S3. The nature of the virtual sharding and the algorithm that exploits it for subsetting is such that it can also be used for aggregation with the need for many of the repetitive operations required by the per file aggregation techniques. We will present timing information that demonstrates the flexibility of this approach. However, the lessons learned is that while this is a useful result in and of itself, these very same techniques can be applied in other contexts where data are stored in services and on media other than S3. For example, this same technique can be applied to data stored on spinning disk. Pushing the envelope for S3 forced a reexamination of our data access techniques which lead to unexpected positive benefits.

Gallagher, James↗

TPSAS-NF1676L-32099-DND

This talk provides a short overview of the new POWER web site that provides data parameters to energy sector societal benefit areas from NASA research projects. The NASA research projects include the GEWEX Surface Radiation Budget project, the CERES Fast Shortwave and Longwave radiative flux project (FLASHFlux) and NASA Goddard's Global Modeling and Assimilation Office (GMAO) MERRA-2 atmospheric reanalysis. Data products are customized to the energy industry and make products available from the daily time scale to long-term climatological averages. The web services allow uses to choose multiple parameters and obtain the data in multiple file formats. This short talk is designed to be delivered at the NASA booth area at the 2018 Fall American Geophysical Union conference.

Paul W Stackhouse, Jr.↗

OpenNEX: An open collaboration platform for the earth science community

Satellite data from the past several decades provide the most consistent record of land-surface processes that form the basis for scientific assessments of the impacts of climate variations and changes on the environment and human social-economical activities. During this time, scientific research on the characterization and assessment of environmental changes had tended to focus on large-scale land-surface changes with significant social-economic impacts. Increasingly, attention is shifting toward changes that occur more locally and that most directly relate to the everyday life of the majority of the population. In addition, there has been needs to develop management and policy decision support systems that are based on local environmental information. Almost at the same time, the advancement in sensor technology has allowed us to collect an unprecedented volume of environmental data. These data must be curated and analyzed to extract useful information for research and decision support purposes. Established in 2013 and funded by NASA, the Open NASA Earth eXchange (OpenNEX; https://opennex.org/ , Jia et al., 2019 ) project partnered with Amazon Web Services (AWS) to make available a large amount of Earth observing data, modeling results, and analysis tools on the AWS. OpenNEX provides researchers, developers, educators, and ordinary users with easy access to an integrated Earth science computational and data platform, enabling citizen scientists and application developers to realize the full value of NASA data assets and software tools. To encourage the public's engagement in this project, NASA ran virtual workshops and prize competitions. The virtual workshops provided online lectures and tutorials about how prominent scientists used the data in their research and the tutorials gave examples how to use the tools to interrogate the data in the Amazon cloud. Finally, the prize competitions allowed much wider participation in the OpenNEX project and enable testing the non-traditional projects and out-of-box ideas. OpenNEX has continued to evolve and mature. Here, we highlight new features and functionalities available to the community.

Jian Zhang↗

NASA biological and physical sciences databases: who’s the FAIRest of them all?

Conceptual models are a key part of the foundation of scientific study. Scientific data discovery and retrieval are often inaccurate and incomplete because these models are not sufficiently well-incorporated into data retrieval systems. Systems often don’t provide the necessary tools to those producing scientific data to fully and unambiguously annotate them and the result is consumers of the data cannot find them efficiently. The capability of data archives to provide these tools to link data to underlying conceptual models is one of dimensions of the recently developed “FAIR” principles (https://www.go-fair.org/fair-principles/ ), and is key to many automated processes being able to operate on these data, particularly analytics involving artificial intelligence. We used an open-source web service to measure the FAIR compliance of the three data archives operated by NASA for the biological and physical sciences: the Life Sciences Data Archive, the Physical Sciences Informatics database, and GeneLab. The service ingests references to data sets in these archives, and then executes domain-non-specific examinations of these data and metadata that test compliance to the FAIR principles. Of the 22 metrics tested, GeneLab passed 11 (50%), and PSI and LSDA each passed 7 (32%). These data were gathered using only one representative data set from each archive and we anticipate variability in results as we continue to apply these metrics to other data. A preliminary study of the failure traces for each metric suggests there is a wide range of effort and complexity in the enhancements required for each system to elevate FAIR compliance, and this is the subject of continued investigation. This information has been and will likely continue to be important information in planning these enhancements, with the goal of increased readiness of the data for automated processes.

database↗

Advances to a Global Agroclimatology Solar Insolation and Meteorological Parameter Data Base: Improved Solar Irradiance up to Hourly Temporal Resolution

A primary objective of NASA’s Prediction of Worldwide Energy Resource (POWER) project is to facilitate the use of NASA Earth Science data holdings within the energy, agricultural, and architectural industries. To this end daily averaged solar data from several NASA projects and metrological data from a NASA assimilation model have long been reformatted and via a user friendly web based data portal (https://power.larc.nasa.gov) at the native resolution of each data products. Potential users can access solar and metrological data in a column formatted DSSAT ASCII format by entering single site specific coordinates or from an area by entering the appropriate area coordinates. Upgrades to the POWER data portal have been implemented that result in a complete upgrade of the base solar insolation data products. From the years 1984 through 2000, a new version of the NASA/GEWEX Surface Radiation Budget (SRB) and Clouds is utilized that reduces the RMS relative to surface measurements. Additionally, Clouds and the Earth’s Radiant Energy System (CERES) SYNoptic 1x1 Degree (SYN1Deg) data products are utilized starting from January 1, 2001 through 3 months of real-time. The CERES Fast Longwave and SHortwave radiative Fluxes (FLASHFlux) is still used to provided daily data spanning from the end of SYN1Deg to within 7 days of real-time. Meteorological parameters now are taken from the NASA Modern Era Retrospective-analysis for Research and Applications (MERRA-2) data set which provides higher resolution data products (hourly and 0.5 x 625 degree) covering the entire globe. Besides updating the solar and meteorological data products, this new version features new data products such as photosynthetically active radiation (PAR), more cloud information, estimates of soil temperatures and improved options for long-term climatological data requests. More than 36+ years daily time slices are included in the combined solar and meteorological data sets. However, perhaps the most important innovation of the POWER GIS-enabled Web Services is the provision of hourly solar and meteorological data products beginning in Jan 1, 2001. The hourly values will enable more detailed modeling and crop analysis that incorporates the diurnal variability of these parameters. The new capability is made possible by utilizing both the CERES SYN1Deg and MERRA-2 data products that include parameters at these resolutions. An assessment of both the meteorological and the solar irradiance data are based upon comparisons with globally distributed surface observations. The meteorological observations from the National Center for Environmental Information’s “Integrated Surface Database” as well as the “Global Summary of the Day” (GSOD) data files. The estimates of solar insolation are compared to the Baseline Solar Radiation Network (BSRN) and other high quality surface measurement networks. Results from the uncertainty assessments demonstrates that the NASA’s meteorological and solar irradiance data can represent a viable alternative to surface observations, particularly in data sparse regions of the world.

solar irradiance↗

Results from the ASTERIA CubeSat Extended Mission Experiments

Over the past two years, JPL has used the ASTERIA (Arcsecond Space Telescope Enabling Research In Astrophysics) CubeSat as an in-flight test platform during extended missions. ASTERIA successfully completed its prime mission in early 2018, and continued to operate in low Earth orbit (LEO) for an additional twenty months. This paper describes demonstrations that were performed on the spacecraft and on the ground-based testbed during the extended mission. These demonstrations fall into three categories: Autonomy technology maturation, hardware characterization, and science discovery. Autonomy technology maturation supported three development efforts. The first shifted the spacecraft commanding paradigm from time-based sequences to Task Networks (tasknets), which allow simpler commanding and more robust onboard execution. The second demonstrated onboard orbit determination in Low Earth Orbit (LEO) without GPS. This activity used a fully-independent means of spacecraft orbit determination for Earth orbiters using only passive imaging. The third technology provided in situ hardware health state estimation using a model-based reasoning technique. These three technologies were demonstrated either in flight or on the testbed individually, and then were combined to demonstrate the capability to perform autonomous navigation on board without ground intervention, even in the presence of anomalies. Hardware characterization involved both onboard and ground-based activities. On board, nonstandard attitude control modes were commanded to characterize the spacecraft pointing jitter as a function of target brightness, reaction wheel speed, controller gain, and the number of guide stars. The results provide insights into the contribution of jitter to the ASTERIA photometry and inform the feasibility of future astrophysics small satellite missions for which jitter control is an enabling technology. On the ground, the ASTERIA Operations Team coordinated with Amazon Web Services (AWS) to configure their new ground stations to communicate with ASTERIA to prove out their viability. ASTERIA used AWS ground stations for nominal operations for the last four months of the mission. Finally, ASTERIA continued to perform exoplanet science as the spacecraft was well-suited to execute long-term monitoring of stars such as alpha Centauri to search for small transiting planets. The science team also imaged a number of interesting objects including a comet, an asteroid, cities at night, and the moon, and coordinated with other projects on Targets of Opportunity for follow-up confirmations and co-observations. Throughout the prime and the extended missions, the ASTERIA spacecraft proved to be a mighty platform that “will go into history as an innovative milestone.”[1 - Zurbuchen]

Doran, Patrick↗

Characterizing Wildfires in Western US.: A Cloud-based Case Study for Interdisciplinary Research using NASA Resources

This presentation will demonstrate a case study of interdisciplinary research done in the Amazon Web Services (AWS) cloud platform, in addition to in the local machine. We conduct data analysis next to data by leveraging various cloud-based data in NASA Earthdata Cloud, which are distributed by different missions/NASA Distributed Active Archive Centers (DAACs), and cloud computing resources at NASA. For instance, we directly access multiple datasets stored in the AWS Simple Storage Service (S3) buckets using a Python Jupyter notebook through a JupyterHub interface hosted in AWS (without having to download data), and conduct data analysis next to data in the cloud. We will also show how to share the research results following Open Source policy. This case study characterizes the change in wildfire events in the western United States during the past 20 years. In particular, we focus on the wildfires in California in 2021, one of the most severe wildfire years occurring in the most recent 20 years in California. We will analyze the possible causes of wildfires, such as drought conditions and climate variability, and examine the impacts of wildfires on air quality and atmospheric composition, and on land cover. We will examine the data distributed by the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC), including aerosols and meteorological data from the NASA Modern-Era Retrospective analysis for Research and Applications version 2 (MERRA-2), precipitation from the Global Precipitation Measurement (GPM) and Global Precipitation Climate Project (GPCP), and aerosol index from Ozone Monitoring Instrument (OMI). We also utilize the data distributed by the Physical Oceanography (PO) DAAC, such as Sea Surface Temperature (SST) data from the Group for High Resolution Sea Surface Temperature (GHRSST), and the data distributed by Land Processes (LP) DAAC, such as Normalized Difference Vegetation Index (NDVI).

Xiaohua Pan↗