Engineering PapersSearch

SEARCH · Engineering Papers

Results for “aws”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

GeoNEX: A Cloud Gateway for Near Real-time Processing of Geostationary Satellite Products

The emergence of a new generation of geostationary satellite sensors provides land andatmosphere monitoring capabilities similar to MODIS and VIIRS with far greater temporal resolution (5-15 minutes). However, processing such large volume, highly dynamic datasets requires computing capabilities that (1) better support data access and knowledge discovery for scientists; (2) provide resources to enable real-time processing for emergency response (wildfire, smoke, dust, etc.); and (3) provide reliable and scalable services for the broader user community. This paper presents an implementation of GeoNEX (Geostationary NASA-NOAA Earth Exchange) services that integrate scientific algorithms with Amazon Web Services (AWS) to provide near realtime monitoring (~5 minute latency) capability in a hybrid cloud-computing environment. It offers a user-friendly, manageable and extendable interface and benefits from the scalability provided by Amazon Web Services. Four use cases are presented to illustrate how to (1) search and access geostationary data; (2) configure computing infrastructure to enable near real-time processing; (3) disseminate and utilize research results, visualizations, and animations to concurrent users; and (4) use a Jupyter Notebook-like interface for data exploration and rapid prototyping. As an example of (3), the Wildfire Automated Biomass Burning Algorithm (WF_ABBA) was implemented on GOES-16 and -17 data to produce an active fire map every 5 minutes over the conterminous US. Details of the implementation strategies, architectures, and challenges of the use cases are discussed.

GeoNEX

NASA Earthdata Cloud

This presentation is an overview OF EOSDIS and motivation for commercial cloud and will discuss components of the Earthdata cloud, AWS accounts metric, account structure architecture and cloud resource and cost model.

McInerney, Mark A.

The CEOS Data Cube Portal: A User-Friendly, Open Source Software Solution for the Distribution, Exploration, Analysis, and Visualization of Analysis Ready Data

There is an urgent need to increase the capacity of developing countries to take part in the study and monitoring of their environments through remote sensing and space-based Earth observation technologies. The Open Data Cube (ODC) provides a mechanism for efficient storage and a powerful framework for processing and analyzing satellite data. While this is ideal for scientific research, the expansive feature space can also be daunting for end-users and decision-makers who simply require a solution which provides easy exploration, analysis, and visualization of Analysis Ready Data (ARD). Utilizing innovative web-design and a modular architecture, the Committee on Earth Observation Satellites (CEOS) has created a web-based user interface (UI) which harnesses the power of the ODC yet provides a simple and familiar user experience: the CEOS Data Cube (CDC). This paper presents an overview of the CDC architecture and the salient features of the UI. In order to provide adaptability, flexibility, scalability, and robustness, we leverage widely-adopted and well-supported technologies such as the Django web framework and the AWS Cloud platform. The fully-customizable source code of the UI is available at our public repository. Interested parties can download the source and build their own UIs. The UI empowers users by providing features that assist with streamlining data preparation, data processing, data visualization, and sub-setting ARD products in order to achieve a wide variety of Earth imaging objectives through an easy to use web interface.

User Interface

Lessons from Earth Bioaerosol Sampling for Venus Habitability & Biosignature Science

Venus’s clouds are often discussed as a potential habitat. Several high-priority science goals – cloud aerosol composition, internal radiative flux, and aerosol residence time and circulation models – will improve our understanding of Venus in an astrobiology context. Lessons from Earth’s aerobiosphere can inform these requirements and interpretation of results. A stable Venus aerobiosphere would need microorganisms to reproduce faster than they settle out due to gravity. This is a joint constraint of the potential nutrients and energy influx, bioenergetic costs such as desiccation and radiation damage, and aerosol dynamics. Though life cycles involving S- and Fe-based redox metabolism have been proposed, at an optimistic estimate of 75% H2SO4, Venus aerosol water activity (aw) is still ~0.02, far below the observed microbial growth limit of ~0.6. Long-term desiccation with brief spurts of repair, growth and reproduction in response to transient water influx (perhaps volcanism) is the most likely model for Earth-like life on Venus – a ‘desert bloom’ scenario. In Earth’s troposphere, clouds can carry 103 – 105 cells/mL, some metabolically active. A better analogue may be Earth’s stratospheric sulfate aerosol layer: supercooled sulfuric acid aerosols (acid weight fraction 0.6 – 0.85, 0.1 – 1 µm diameter, 0.1 – 1 cm-3) with little water activity, long residence times, high UV radiation, and only sporadic influx from surface particle sources. Viable cells isolated from this region are rare (~102 cells/m3), though ‘hot spots’ at tropospheric mixing occur. Cells are primarily in inactive forms such as spores, and it is unclear if any are associated with sulfate aerosols or simply co-located. Given the heterogeneities in Venus clouds and the model of a sparse, largely dormant ecosystem, a single transect on descent would most likely pass through a low-water, inactive region, missing potential signs of habitability or biosignatures. Even a passive aerial platform (balloon) may end up following a single air mass over its lifetime. A better strategy for Venus would target an aerial region with some upwelling from surface sources, and take multiple transects separated in time and space. Earth’s stratospheric bioaerosol population could provide a starting point for estimating the required lower volume and detection limits.

Earth

Lessons from Earth Aerobiology for Venus Astrobiology

Venus’s clouds have often suggested as a possible habitat. The constraints governing putative airborne life in such a habitat in turn inform priorities and strategies for remote and in situ exploration, methods by which resulting biosignatures might be detected, exoplanet habitability assessment, and planetary protection concerns. Lessons drawn from studying Earth’s aerobiosphere can help improve this understanding. There are altitude ranges within Venus’s clouds in which temperature, pressure, particle size, and radiation appear to be within the limits of microbial life on Earth, and life cycles involving S- and Fe-based redox metabolism have been proposed. However, given the lack of a habitable surface reservoir, a long-term stable Venus aerobiosphere would require that the reproduction rate of the airborne microbes be faster than the settling rate of airborne microbes due to gravity, or eventually the population would be depleted; put another way, the mean generation time would need to exceed the mean residence time. This creates a joint constraint of aerosol dynamics, potential nutrient availability and energy influx, and bioenergetic costs such as desiccation and radiation damage. Even at an optimistic estimate of 75% H2SO4, Venus aerosol water activity (aw) is still ~0.02, far below the observed microbial growth limit of ~0.6. Long-term desiccation with brief spurts of repair and growth in response to transient water influx, such as from volcanism, is the most likely model for Earth-like life on Venus – a ‘desert bloom’ scenario. Several high-priority science goals – cloud aerosol composition, internal radiative flux, and aerosol residence time and circulation models – thus will also improve our understanding of Venus in an astrobiology context. In Earth’s troposphere, warm water clouds can carry 103 – 105 cells/mL, some metabolically active. However, Earth’s stratospheric sulfate aerosol layer may be a better analogue: supercooled sulfuric acidaerosols (acid weight fraction 0.6 – 0.85, 0.1 – 1 μm diameter, 0.1 – 1 cm-3) with little water activity, long residence times, high UV radiation, and only sporadic influx from surface particle sources. Though ‘hot spots’ can occur associated with tropospheric mixing, viable cells in stratospheric samples are rare (~102 cells/m3), and primarily inactive forms such as spores. It is not yet clear whether such bioaerosols are associated with sulfate aerosols or simply co-located, and reproduction in situ has not yet been observed. In this model of a sparse, largely dormant Venus ecosystem, a single transect on descent is likely to pass through a low-water, inactive region, missing potential signs of habitability or biosignatures. A targeted strategy would sample through an aerial region with some upwelling from surface sources, andtake multiple transects separated in time and space. This is compatible with other in situ science goals seeking to understand the dynamics and heterogeneities of Venus’s clouds.

Lower atmosphere

Surface melting over the Greenland ice sheet from enhanced resolution passive microwave brightness temperatures (1979–2019)

Surface melting is a major component of the Greenland ice sheet (GrIS) surface mass balance, affecting sea level rise through direct runoff and the modulation on ice dynamics and hydrological processes, supraglacially, englacially and subglacially. Passive microwave (PMW) brightness temperature observations are of paramount importance in studying the spatial and temporal evolution of surface melting in view of their long temporal coverage (1979–to date) and high temporal resolution (daily). However, a major limitation of PMW datasets has been the relatively coarse spatial resolution, being historically of the order of tens of kilometres. Here, we use a newly released passive microwave dataset (37 GHz, horizontal polarization) made available through the NASA MeASUREs program to study the spatiotemporal evolution of surface melting over the GrIS at an enhanced spatial resolution of 3.125 Km. We assess the outputs of different detection algorithms through data collected by Automatic Weather Stations (AWS) and the outputs of the MAR regional climate model. We found that surface melting is well captured using a dynamic algorithm based on the outputs of MEMLS model, capable to detect sporadic and persistent melting. Our results indicate that, during the reference period 1979–2019 (1988–2019), surface melting over the GrIS increased in terms of both duration, up to ~4.5 (2.9) days per decade, and extension, up to 6.9 % (3.6 %) of the GrIS surface extent per decade, according to the MEMLS algorithm. Furthermore, the melting season has started up to ~4 (2.5) days earlier and ended ~7 (3.9) days later per decade. We also explored the information content of the enhanced resolution dataset with respect to the one at 25 km and MAR outputs through a semi-variogram approach. We found that the enhanced product is more sensitive to local scale processes, hence confirming the potential interest of this new enhanced product for studying surface melting over Greenland at a higher spatial resolution than the historical products and monitor its impact on sea level rise. This offers the opportunity to improve our understanding of the processes driving melting, to validate modelled melt extent at high resolution and potentially to assimilate this data in climate models.

Surface melting

Evaluation of Aqua MODIS and S-NPP VIIRS Thermal Emissive Bands Calibration Stability Using Dome-C

Establishing the calibration consistency between satellite measurements is an essential step in the implementation of a long-term global monitoring plan, which often leads to sensor calibration improvements. The Suomi National Polar-Orbiting Partnership (S-NPP) Visible Infrared Imaging Radiometer Suite (VIIRS) is a polar-orbiting Earth remote sensing instrument built with a strong Moderate Resolution Imaging Spectroradiometer (MODIS) heritage. The center wavelengths of all VIIRS thermal emissive bands (TEBs) match well with those of MODIS with the exception of VIIRS TEB M15 (10.7μm). Previous work from the MODIS Characterization Support Team (MCST) at the NASA/GSFC used specific Earth surface targets to track the long-term consistency,stability, and relative bias between the two MODIS instruments onboard the Terra and Aqua satellites. Using similar methodologies, this paper evaluates the TEB calibration consistency between the MODIS instruments and S-NPP VIIRS over Dome Concordia (Dome-C). The Dome-C site is uniformly snow-covered and the atmospheric effects are small in the surrounding area. Near-surface air temperature measurements from an Automatic Weather Station (AWS) are used as a reference to track each sensor’s calibration stability and determine the relative bias between the MODIS and VIIRS instruments. The results of this study provide a quantitative assessment of the S-NPP VIIRS TEB mission-long performance.

MODIS

Entwine Point Tiles for 3D Visualization and Querying of ICESat-2

Point Cloud data from non-optical sensors present challenges in scientific computing in both volume of data and files, even for cloud services environments. As part of the Multi-Mission Algorithm and Analysis Platform (MAAP), a joint open science platform for global biomass modelling, we’ve developed a cloud optimized workflow for using ATL08 (ICESat-2) data as a point cloud. For MAAP, the ATL08 data product is published as Entwine Point Tiles (EPT), allowing users to visualize and query the full extent of this collection interactively without pre-downloading, or preprocessing. The EPT format is a cloud-optimized point cloud data format which re-organizes points into a cloud friendly spatially indexed data structure. MAAP uses AWS S3 to store these point clouds and serves them over OGC specified APIs, 3DTiles for visualization, and WFS for querying. This workflow allows for interactive 3D visualizations in a web browser, including notebook environments and facilitates on the fly subsetting for interactive data exploration, all of which can be applied to other similar sensors.

Alex Mandel

Applying the Cognitive Space Gateway to Swarm Topologies

NASA's future vision for interplanetary networking includes a lunar network, Cube Satellite (CubeSat) constellations, and deep space robotic missions, comprising what could be viewed as a network of networks. Delay-tolerant networking (DTN) architecture and protocols provide a standard network layer among these varying scenarios and mitigate many challenges of the space environment, such as long delays, unplanned service interruptions, and asymmetric links. The Cognitive Space Gateway (CSG) is a routing method in a DTN architecture that uses spiking neural networks as the learning element to optimize routing decisions in a complex environment. This work aims to further develop cognitive networking technologies in several critical areas, including DTN, the CSG algorithm, CubeSat swarm topologies, and cloud services. To test the algorithm in a realistic scenario, the emulated network topology is based on a CubeSat swarm. The swarm may function as a mesh of nodes or as a hub-and-spoke network. An emulation environment will be built upon a commercial cloud service, such as Amazon Web Services (AWS) Elastic Compute Cloud. The cloud environment may enable a flexible, lower maintenance approach versus a multi-hop network based in a physical laboratory. The cloud platform will provide a secure environment allowing for collaboration among government and academic entities.

Ricardo Lent

NASA Leveraging Commercial Communication Ground Stations for Small Satellites

The Space Communications and Navigation (SCaN) program at NASA has reorganized its operations portfolio into two networks: the Deep Space Network and the new Near Space Network (NSN). With this reorganization, NASA can begin transforming to 100% direct-to-Earth commercial communications services for missions in the near-Earth region. NASA’s leveraging of commercial direct-to-Earth ground stations offers several benefits for the small satellite community, including lower cost, greater coverage, and increased technology infusion. In the fall of 2020, SCaN announced their intention to rely primarily on industry-provided communications services for missions close to Earth by 2030. Commercial services are one way to infuse new technology into the ground station network without requiring an investment from NASA. Digital Video Broadcast, Satellite Second Generation (DVB-S2) is one example of a current technology. When combined with variable coding and modulation (VCM), the system automatically optimizes the data rate based on signal performance, significantly increasing total downlink data volume without an increase in the spacecraft effective isotropic radiated power (EIRP). There are several commercial service providers, including Amazon Web Service (AWS) Ground Station (AGS) and the KSATLITE ground stations that support SmallSat missions using DVB-S2 waveforms for downlinks. This paper identifies some commercial off-the-shelf (COTS) CubeSat/SmallSat DVB-S2 X-band and Ka-band radios. Overall, NASA’s increased dependence on commercial direct-to-Earth ground stations is a significant benefit for the small satellite community.

Space Communications

Applying the Cognitive Space Gateway to Swarm Topologies

NASA’s future vision for interplanetary networking includes a lunar network, Cube Satellite (CubeSat) constellations, and deep space robotic missions, comprising what could be viewed as a network of networks. Delay-tolerant networking (DTN) architecture and protocols provide a standard network layer among these varying scenarios and mitigate many challenges of the space environment, such as long delays, unplanned service interruptions, and asymmetric links. The Cognitive Space Gateway (CSG) is a routing method in a DTN architecture that uses spiking neural networks as the learning element to optimize outing decisions in a complex environment. This work aims to further develop cognitive networking technologies in several critical areas, including DTN, the CSG algorithm, SmallSat swarm topologies, and cloud services. The CSG algorithm is tested in a realistic scenario in which the emulated network topology is based on a SmallSat swarm. The emulation environment will be built upon a commercial cloud service, such as Amazon Web Services (AWS) Elastic Compute Cloud. This work investigates the ability of such a platform to enable a flexible, lower maintenance approach to creating a multihop network outside of a physical laboratory. The cloud platform will provide a secure environment allowing for collaboration among government and academic entities.

Ricardo Lent

Connecting Users and Applications with Po.daac Hosted GHRSST Data

The 80+ GHRSST public datasets represent a rich resource for sea surface temperature research and applications given their time series length, resolution, spatial coverage, varying measurement types and processing levels, and availability in the full spectrum of PO.DAAC tools and services ecosystem. The PO.DAAC has created a publicly accessible recipe suite for the user community to perform straightforward yet powerful computations on GHRSST data using python recipes, Jupyter notebooks, R, Matlab, and the NCO programming language. These recipes include numerical computations for regional and global SST trends, anomaly derivations, EOF analysis, climate signal reproduction, and ocean phenology. For example, one recipe reproduces a famous SST based warming figure from the Fourth National Climate Assessment (USA) while another focuses on quantifying the regional changes in ocean SST phenology. Most are python-based while some contain hybrid calls and leverage the NCO programming interface too. All are available on the PO.DAAC user forum (https://podaac.jpl.nasa.gov/forum/) and/or via the open source NASA GitHub repository (https://github.com/nasa/podaac_tools_and_services). Several are available in the Jupyter notebook framework including podaacypy (https://github.com/nasa/podaacpy), a recipe for GHRSST granule metadata discovery and application, and more recently a Jupyter notebook developed to support data analysis and visualization of a cloud-based Zarr formatted Level 4 MUR dataset in the AWS Open Data Registry. Throughout the summer of 2020, the PO.DAAC intends to add and migrate more of its numerical recipes to the Jupyter notebook framework and publish them on its open source GitHub repository.

Gentemann, Chelle

AAM National Campaign Tech Talk: Data Pipeline Familiarization

NASA's AWS-based Data Pipeline allows real-time data submission and ingestion, with immediate monitoring of data rate, coverage and ingestion quality. Problems are immediately discovered and can be corrected with agility by both Partners and NASA during the a simulation or flight event​.

Data Pipeline

Cloud-Based Demodulation and Data Distribution of a Satellite Downlink

Ground station networks connected to the cloud allow space missions to have global communications coverage without operating their own infrastructure. In this work, we describe the communications architecture for the TechEdSat-13 mission, which performed the first in-space characterization of a neuromorphic processor. The mission utilizes a commercial provider for S-band downlinks. A suite of cloud services and open-source software such as GNU Radio are leveraged to demodulate signals received by an AWS ground station during passes with TechEdSat-13 and store recovered data. Once a pass is scheduled, the entire process takes place without human intervention. On-orbit results the past year of operations are presented, demonstrating the advantages of this approach over traditional operator-owned ground stations. Use of software-defined radio makes possible custom signal processing. The homogeneity of apertures and their interfaces to the cloud simplifies scaling across many sites. This abundance of candidate links lays the groundwork for intelligent scheduling agents to optimize pass selection across several factors, automatically recover from failed contacts, and gather metrics to learn from past performance.

cloud demodulation

The NASA Merra-2 Reanalysis Products: Data and Tools Used for Aerosol and Air Quality Studies

The NASA Modern-Era Retrospective analysis for Research and Applications Version 2 (MERRA-2) is atmospheric reanalysis data spanning 1980 to present. It has been produced by the NASA Global Modeling and Assimilation Office (GMAO) and is distributed by the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC). MERRA-2 data includes 100 collections of Earth system variables, mainly from the atmospheric model, such as aerosol fields and meteorological fields, radiation fields, and aerosol fields, guided by the assimilation of as many as six million observations every six hours. MERRA-2 has been one of the most popular datasets from NASA and is widely used in interdisciplinary research and applications, with increasing numbers of new users. For example, at least 7000 users accessed MERRA-2 data at GES DISC in the year 2021, ~1000 more users than in the year 2020. In this presentation, we will introduce the MERRA-2 datasets associated with aerosol and air quality studies and use a wildfire case study to demonstrate the data tools developed at GES DISC to analyze and visualize MERRA-2 data, such as Giovanni and the level 3 and level 4 subsetter, and Jupyter Python notebook. We will also update the status of cloud migration of the MERRA-2 data to Amazon Web Services (AWS).

Xiaohua Pan

Enhancing Autonomous Satellite Communication Systems with Weather-Aware Scheduling and Reconfiguration

NASA currently provides communication support to over 100 satellite missions, and the agency is driving developments in Ka-band communications and network management automation to support additional future missions. At Ka-band frequencies, rain can degrade a communication link by more than 10 dB, which may be mitigated by agile scheduling and data rate control. We present a weather forecasting module for Ka-band communications that is intended to be used in an autonomous network management service that employs machine-to-machine scheduling systems for dynamic user access opportunities. Our forecasting module (NIMBUS) runs on AWS Cloud, consumes the freely and publicly available NOAA MRMS precipitation rate dataset (1km x 1km x 2-min), produces 30-minute Nowcasts using the pySTEPS algorithm, and publishes high level ground station specific link quality predictions. We evaluate two potential NIMBUS outputs, a binary classifier that predicts rain attenuation greater than 3 dB and a rain attenuation estimator, and we backtest these outputs using one year of power measurement data collected from observations of the geostationary ANIK F2 satellite’s Ka-band beacon. We report, with a 30-minute lead time, a binary classifier accuracy of 84% and an estimator RMSE of 1.67 dB. Additionally, we discuss how the NIMBUS module could be incorporated into a user-initiated service framework to enable weather-aware scheduling and reconfiguration. Operating a static link budget with minimal link margin can increase network operation costs by creating additional scheduling tasks due to failed packets, which may require human intervention and/or lead to inefficient asset utilization. The proposed system aims to increase throughput by reducing link margin, while mitigating increases in network operations costs by leveraging autonomous machine-to-machine scheduling, shortened prediction lead times, and advances in precipitation Nowcasting.

satellite communications

Hydrological Data at the NASA GES DISC: Current Capabilities and New Opportunities

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is one of twelve NASA Earth science data centers that document, process, archive and distribute data from Earth observation missions and projects. GES DISC maintains an archive of several hydrology datasets, including the Land Data Assimilation Systems (LDAS) and the Gravity Recovery and Climate Experiment (GRACE) Data Assimilation for Drought Monitoring (GRACE-DA-DM) data products. These datasets include model output of heat fluxes, rain, snow, soil temperature, soil moisture, and runoff; and observational forcing data, including surface pressure, temperature, precipitation, downward shortwave and longwave radiation, humidity, and wind. The temporal resolution of the hydrology data at GES DISC ranges from hourly to monthly, and spatial resolutions range from 0.1° to 1.0°. The GES DISC provides services which enable users to aggregate, temporally and spatially subset, regrid, and visualize archived data including the GES DISC Subsetter, Hydrology Data Rods, and the Geospatial Interactive Online Visualization and Analysis Infrastructure (GIOVANNI). The Hydrology Data Rods service optimally reorganizes large hydrological data sets as extended time series, providing more efficient access for the hydrological community. The time series data (aka “data rods”) were integrated into hydrology community tools, such as the Data Rods Explorer on HydroShare. Furthermore, the GES DISC is in the process of migrating its data and services to the cloud. Hydrological data available at the GES DISC are now available in the Amazon Web Services (AWS) cloud (us-west-2 region) providing users Direct S3 data access and the capability for cloud computing operations. In this presentation, the hydrology data products and services currently available at the GES DISC will be summarized. Also discussed are the migration to the cloud, user support through this transition, and the status of migrating the data rods service to the cloud.

Ashley Heath

Improving Access to the GEOS Composition Forecast Model with API Development and Ingestion into Google Earth Engine

The GEOS Composition Forecast (GEOS-CF) model produces forecast and historical estimates of atmospheric composition and meteorology fields, which provide useful insight into air quality issues and events. In a year for which Canadian wildfires created adverse air quality conditions in the eastern United States, access to model fields such as PM2.5 are in high demand. The GEOS-CF team at the NASA Global Modeling and Assimilation Office (GMAO) first developed in-house solutions to improve data access via the CF API, and recently partnered with Google to ingest a collated set of model diagnostics into the Google Earth Engine (GEE) data repository. GEOS-CF model output is also being ingested into AWS storage. Creating these various open access points to GEOS-CF model diagnostics provides the public with an opportunity to easily interact with air quality information. Users are able to use a temporally consistent global grid of air quality fields in machine learning applications, mapping tools, and data informatics. Hosting GEOS-CF forecasts and the historical timeseries of these chemistry and meteorology fields in GEE allows users to create dynamic JavaScript-based air quality applications in the GEE code editor. GEOS-CF users can also access the model output via the GEE Python application programming interface (API), making it easy to perform various analyses with Python. This presentation will show two examples of accessing the GEOS-CF model through GEE. The first is an example application made in the GEE code editor which allows users to view time series plots and downscaled maps of surface level NO2. The second example exhibits using the GEE Python API to create a machine learning model to temporally gap-fill between air quality observations. These examples are an introduction to the many possible benefits of having open access to the GEOS-CF model through multiple platforms.

Callum Wayman