Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “AWS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16

Evaluation of MODIS and Sentinel-3 SLSTR Thermal Emissive Bands Calibration Consistency Using Dome C

Over the years, data from different satellites has provided invaluable information about Earth's atmosphere, land and oceans. The thermal emissive bands (TEB) on the Moderate Resolution Imaging Spectroradiometer (MODIS) are comprised of 16 spectral bands with wavelengths ranging from 3.7 to 14.4 μm. MODIS TEB are calibrated on orbit on a scan-by-scan basis using an on-board blackbody (BB). Sentinel-3 Sea and Land Surface Temperature Radiometer (SLSTR), launched on 16 February 2016, has 11 spectral bands with wavelengths from 0.55 to 12 μm. In this study, we compare the observed brightness temperature from MODIS bands 31 and 32 and SLSTR bands S8 and S9 over Dome C using a 20 × 20 km region of interest (ROI) centered at (75.102 °S,123.395 °E). A total of 2989 scenes for Terra, 2963 for Aqua and 1961 for SLSTR from November, 2016 to January 2018 are analyzed. The relative bias between MODIS and SLSTR is evaluated using the near-surface temperature measurements from an Automatic Weather Station (AWS).

Shrestha, Ashish↗

NASA Earthdata Cloud

Briefing is to highlight an overview of the NASA Earthdate Cloud with more specific details on the "Governance of Cloud Accounts", "Shared Services", Cloud ADA controls, in scope of this AWS public sector session.

Computer Operations and Hardware↗

Unraveling the Internal Magnetic Field Structure of the Earth-Directed Interplanetary Coronal Mass Ejections During 1995-2015

The magnetic field configurations associated with interplanetary coronal mass ejections (ICMEs) are the in situ manifestations of the entrained magnetic structure associated with coronal mass ejections (CMEs). We present a comprehensive study of the internal magnetic field configurations of ICMEs observed at 1 AU by the Wind mission during 1995-2015. The goal is to unravel the internal magnetic structure associated with the ICMEs and establish the signatures that validate a flux-rope structure. We examine the expected magnetic field signatures by simulating spacecraft trajectories within a simple flux rope, i.e., with circular–cylindrical (CC) helical magnetic field geometry. By comparing the synthetic configurations with the 353 ICME in situ observations, we find that only 152 events ( Fr ) display the clear signatures of an expected axial-symmetric flux rope. Two more populations exhibit possible signatures of flux rope; 58 cases ( F− ) display a small rotation ( <90° ) of the magnetic field direction, interpreted as a large separation of the spacecraft from the center, and, 62 cases ( F+ ) exhibit larger rotations, possibly arising from more complex configuration. The categories, Cx (14%) and E events (9%), reveal signatures of complexity possibly related with evolutionary processes. We then reconstruct the flux ropes assuming CC geometry. We examine the orientation and geometrical properties during the solar activity levels at the end of Solar Cycle 22 (SC22), SC23 and part of SC24. The orientation exhibits solar cycle trends and follow the heliospheric current sheet orientation. We confirm previous studies that found a Hale cycle dependence of the poloidal field reversal. By comparing our results with the occurrence of CMEs with large angular width ( AW>60∘ ) we find a broad correlation suggesting that such events are highly inclined CMEs. The solar cycle distribution of bipolar vs. unipolar Bz configuration confirms that the CMEs may remove solar cycle magnetic field and helicity.

Nieves-Chinchilla, Teresa↗

Forming Aggregations using Virtual Sharding: Lessons Learned from Simple Scalable Storage (S3)

Data aggregation is the ability to combine separate datasets to form a single new logical dataset provides users with a powerful abstraction. The advantage of an aggregate dataset is that the users are freed from having to understand, and incorporate into their workflow, knowledge about the (ad hoc) organization of the constituent datasets. However, aggregating large numbers of files can be computationally complex with data server systems performing many repetitive operations. As part of the authors work on subsetting data stored on Amazon Web Service (AWS) Simple Storage Service (S3), we developed technology to read portions of otherwise monolithic data files. This enables the formation of virtual shards for user in subsetting data stored in HDF5 (hierarchical data format, version 5) files. This same tool can be used to form aggregations that combine data stored in many HDF5 files when those files are stored on S3. The nature of the virtual sharding and the algorithm that exploits it for subsetting is such that it can also be used for aggregation with the need for many of the repetitive operations required by the per file aggregation techniques. We will present timing information that demonstrates the flexibility of this approach. However, the lessons learned is that while this is a useful result in and of itself, these very same techniques can be applied in other contexts where data are stored in services and on media other than S3. For example, this same technique can be applied to data stored on spinning disk. Pushing the envelope for S3 forced a reexamination of our data access techniques which lead to unexpected positive benefits.

Gallagher, James↗

GeoNEX: A Cloud Gateway for Near Real-time Processing of Geostationary Satellite Products

The emergence of a new generation of geostationary satellite sensors provides land andatmosphere monitoring capabilities similar to MODIS and VIIRS with far greater temporal resolution (5-15 minutes). However, processing such large volume, highly dynamic datasets requires computing capabilities that (1) better support data access and knowledge discovery for scientists; (2) provide resources to enable real-time processing for emergency response (wildfire, smoke, dust, etc.); and (3) provide reliable and scalable services for the broader user community. This paper presents an implementation of GeoNEX (Geostationary NASA-NOAA Earth Exchange) services that integrate scientific algorithms with Amazon Web Services (AWS) to provide near realtime monitoring (~5 minute latency) capability in a hybrid cloud-computing environment. It offers a user-friendly, manageable and extendable interface and benefits from the scalability provided by Amazon Web Services. Four use cases are presented to illustrate how to (1) search and access geostationary data; (2) configure computing infrastructure to enable near real-time processing; (3) disseminate and utilize research results, visualizations, and animations to concurrent users; and (4) use a Jupyter Notebook-like interface for data exploration and rapid prototyping. As an example of (3), the Wildfire Automated Biomass Burning Algorithm (WF_ABBA) was implemented on GOES-16 and -17 data to produce an active fire map every 5 minutes over the conterminous US. Details of the implementation strategies, architectures, and challenges of the use cases are discussed.

GeoNEX↗

NASA Earthdata Cloud

This presentation is an overview OF EOSDIS and motivation for commercial cloud and will discuss components of the Earthdata cloud, AWS accounts metric, account structure architecture and cloud resource and cost model.

McInerney, Mark A.↗

The CEOS Data Cube Portal: A User-Friendly, Open Source Software Solution for the Distribution, Exploration, Analysis, and Visualization of Analysis Ready Data

There is an urgent need to increase the capacity of developing countries to take part in the study and monitoring of their environments through remote sensing and space-based Earth observation technologies. The Open Data Cube (ODC) provides a mechanism for efficient storage and a powerful framework for processing and analyzing satellite data. While this is ideal for scientific research, the expansive feature space can also be daunting for end-users and decision-makers who simply require a solution which provides easy exploration, analysis, and visualization of Analysis Ready Data (ARD). Utilizing innovative web-design and a modular architecture, the Committee on Earth Observation Satellites (CEOS) has created a web-based user interface (UI) which harnesses the power of the ODC yet provides a simple and familiar user experience: the CEOS Data Cube (CDC). This paper presents an overview of the CDC architecture and the salient features of the UI. In order to provide adaptability, flexibility, scalability, and robustness, we leverage widely-adopted and well-supported technologies such as the Django web framework and the AWS Cloud platform. The fully-customizable source code of the UI is available at our public repository. Interested parties can download the source and build their own UIs. The UI empowers users by providing features that assist with streamlining data preparation, data processing, data visualization, and sub-setting ARD products in order to achieve a wide variety of Earth imaging objectives through an easy to use web interface.

User Interface↗

Lessons from Earth Bioaerosol Sampling for Venus Habitability & Biosignature Science

Venus’s clouds are often discussed as a potential habitat. Several high-priority science goals – cloud aerosol composition, internal radiative flux, and aerosol residence time and circulation models – will improve our understanding of Venus in an astrobiology context. Lessons from Earth’s aerobiosphere can inform these requirements and interpretation of results. A stable Venus aerobiosphere would need microorganisms to reproduce faster than they settle out due to gravity. This is a joint constraint of the potential nutrients and energy influx, bioenergetic costs such as desiccation and radiation damage, and aerosol dynamics. Though life cycles involving S- and Fe-based redox metabolism have been proposed, at an optimistic estimate of 75% H2SO4, Venus aerosol water activity (aw) is still ~0.02, far below the observed microbial growth limit of ~0.6. Long-term desiccation with brief spurts of repair, growth and reproduction in response to transient water influx (perhaps volcanism) is the most likely model for Earth-like life on Venus – a ‘desert bloom’ scenario. In Earth’s troposphere, clouds can carry 103 – 105 cells/mL, some metabolically active. A better analogue may be Earth’s stratospheric sulfate aerosol layer: supercooled sulfuric acid aerosols (acid weight fraction 0.6 – 0.85, 0.1 – 1 µm diameter, 0.1 – 1 cm-3) with little water activity, long residence times, high UV radiation, and only sporadic influx from surface particle sources. Viable cells isolated from this region are rare (~102 cells/m3), though ‘hot spots’ at tropospheric mixing occur. Cells are primarily in inactive forms such as spores, and it is unclear if any are associated with sulfate aerosols or simply co-located. Given the heterogeneities in Venus clouds and the model of a sparse, largely dormant ecosystem, a single transect on descent would most likely pass through a low-water, inactive region, missing potential signs of habitability or biosignatures. Even a passive aerial platform (balloon) may end up following a single air mass over its lifetime. A better strategy for Venus would target an aerial region with some upwelling from surface sources, and take multiple transects separated in time and space. Earth’s stratospheric bioaerosol population could provide a starting point for estimating the required lower volume and detection limits.

Earth↗

Lessons from Earth Aerobiology for Venus Astrobiology

Venus’s clouds have often suggested as a possible habitat. The constraints governing putative airborne life in such a habitat in turn inform priorities and strategies for remote and in situ exploration, methods by which resulting biosignatures might be detected, exoplanet habitability assessment, and planetary protection concerns. Lessons drawn from studying Earth’s aerobiosphere can help improve this understanding. There are altitude ranges within Venus’s clouds in which temperature, pressure, particle size, and radiation appear to be within the limits of microbial life on Earth, and life cycles involving S- and Fe-based redox metabolism have been proposed. However, given the lack of a habitable surface reservoir, a long-term stable Venus aerobiosphere would require that the reproduction rate of the airborne microbes be faster than the settling rate of airborne microbes due to gravity, or eventually the population would be depleted; put another way, the mean generation time would need to exceed the mean residence time. This creates a joint constraint of aerosol dynamics, potential nutrient availability and energy influx, and bioenergetic costs such as desiccation and radiation damage. Even at an optimistic estimate of 75% H2SO4, Venus aerosol water activity (aw) is still ~0.02, far below the observed microbial growth limit of ~0.6. Long-term desiccation with brief spurts of repair and growth in response to transient water influx, such as from volcanism, is the most likely model for Earth-like life on Venus – a ‘desert bloom’ scenario. Several high-priority science goals – cloud aerosol composition, internal radiative flux, and aerosol residence time and circulation models – thus will also improve our understanding of Venus in an astrobiology context. In Earth’s troposphere, warm water clouds can carry 103 – 105 cells/mL, some metabolically active. However, Earth’s stratospheric sulfate aerosol layer may be a better analogue: supercooled sulfuric acidaerosols (acid weight fraction 0.6 – 0.85, 0.1 – 1 μm diameter, 0.1 – 1 cm-3) with little water activity, long residence times, high UV radiation, and only sporadic influx from surface particle sources. Though ‘hot spots’ can occur associated with tropospheric mixing, viable cells in stratospheric samples are rare (~102 cells/m3), and primarily inactive forms such as spores. It is not yet clear whether such bioaerosols are associated with sulfate aerosols or simply co-located, and reproduction in situ has not yet been observed. In this model of a sparse, largely dormant Venus ecosystem, a single transect on descent is likely to pass through a low-water, inactive region, missing potential signs of habitability or biosignatures. A targeted strategy would sample through an aerial region with some upwelling from surface sources, andtake multiple transects separated in time and space. This is compatible with other in situ science goals seeking to understand the dynamics and heterogeneities of Venus’s clouds.

Lower atmosphere↗

Surface melting over the Greenland ice sheet from enhanced resolution passive microwave brightness temperatures (1979–2019)

Surface melting is a major component of the Greenland ice sheet (GrIS) surface mass balance, affecting sea level rise through direct runoff and the modulation on ice dynamics and hydrological processes, supraglacially, englacially and subglacially. Passive microwave (PMW) brightness temperature observations are of paramount importance in studying the spatial and temporal evolution of surface melting in view of their long temporal coverage (1979–to date) and high temporal resolution (daily). However, a major limitation of PMW datasets has been the relatively coarse spatial resolution, being historically of the order of tens of kilometres. Here, we use a newly released passive microwave dataset (37 GHz, horizontal polarization) made available through the NASA MeASUREs program to study the spatiotemporal evolution of surface melting over the GrIS at an enhanced spatial resolution of 3.125 Km. We assess the outputs of different detection algorithms through data collected by Automatic Weather Stations (AWS) and the outputs of the MAR regional climate model. We found that surface melting is well captured using a dynamic algorithm based on the outputs of MEMLS model, capable to detect sporadic and persistent melting. Our results indicate that, during the reference period 1979–2019 (1988–2019), surface melting over the GrIS increased in terms of both duration, up to ~4.5 (2.9) days per decade, and extension, up to 6.9 % (3.6 %) of the GrIS surface extent per decade, according to the MEMLS algorithm. Furthermore, the melting season has started up to ~4 (2.5) days earlier and ended ~7 (3.9) days later per decade. We also explored the information content of the enhanced resolution dataset with respect to the one at 25 km and MAR outputs through a semi-variogram approach. We found that the enhanced product is more sensitive to local scale processes, hence confirming the potential interest of this new enhanced product for studying surface melting over Greenland at a higher spatial resolution than the historical products and monitor its impact on sea level rise. This offers the opportunity to improve our understanding of the processes driving melting, to validate modelled melt extent at high resolution and potentially to assimilate this data in climate models.

Surface melting↗

Evaluation of Aqua MODIS and S-NPP VIIRS Thermal Emissive Bands Calibration Stability Using Dome-C

Establishing the calibration consistency between satellite measurements is an essential step in the implementation of a long-term global monitoring plan, which often leads to sensor calibration improvements. The Suomi National Polar-Orbiting Partnership (S-NPP) Visible Infrared Imaging Radiometer Suite (VIIRS) is a polar-orbiting Earth remote sensing instrument built with a strong Moderate Resolution Imaging Spectroradiometer (MODIS) heritage. The center wavelengths of all VIIRS thermal emissive bands (TEBs) match well with those of MODIS with the exception of VIIRS TEB M15 (10.7μm). Previous work from the MODIS Characterization Support Team (MCST) at the NASA/GSFC used specific Earth surface targets to track the long-term consistency,stability, and relative bias between the two MODIS instruments onboard the Terra and Aqua satellites. Using similar methodologies, this paper evaluates the TEB calibration consistency between the MODIS instruments and S-NPP VIIRS over Dome Concordia (Dome-C). The Dome-C site is uniformly snow-covered and the atmospheric effects are small in the surrounding area. Near-surface air temperature measurements from an Automatic Weather Station (AWS) are used as a reference to track each sensor’s calibration stability and determine the relative bias between the MODIS and VIIRS instruments. The results of this study provide a quantitative assessment of the S-NPP VIIRS TEB mission-long performance.

MODIS↗

Entwine Point Tiles for 3D Visualization and Querying of ICESat-2

Point Cloud data from non-optical sensors present challenges in scientific computing in both volume of data and files, even for cloud services environments. As part of the Multi-Mission Algorithm and Analysis Platform (MAAP), a joint open science platform for global biomass modelling, we’ve developed a cloud optimized workflow for using ATL08 (ICESat-2) data as a point cloud. For MAAP, the ATL08 data product is published as Entwine Point Tiles (EPT), allowing users to visualize and query the full extent of this collection interactively without pre-downloading, or preprocessing. The EPT format is a cloud-optimized point cloud data format which re-organizes points into a cloud friendly spatially indexed data structure. MAAP uses AWS S3 to store these point clouds and serves them over OGC specified APIs, 3DTiles for visualization, and WFS for querying. This workflow allows for interactive 3D visualizations in a web browser, including notebook environments and facilitates on the fly subsetting for interactive data exploration, all of which can be applied to other similar sensors.

Alex Mandel↗

Applying the Cognitive Space Gateway to Swarm Topologies

NASA's future vision for interplanetary networking includes a lunar network, Cube Satellite (CubeSat) constellations, and deep space robotic missions, comprising what could be viewed as a network of networks. Delay-tolerant networking (DTN) architecture and protocols provide a standard network layer among these varying scenarios and mitigate many challenges of the space environment, such as long delays, unplanned service interruptions, and asymmetric links. The Cognitive Space Gateway (CSG) is a routing method in a DTN architecture that uses spiking neural networks as the learning element to optimize routing decisions in a complex environment. This work aims to further develop cognitive networking technologies in several critical areas, including DTN, the CSG algorithm, CubeSat swarm topologies, and cloud services. To test the algorithm in a realistic scenario, the emulated network topology is based on a CubeSat swarm. The swarm may function as a mesh of nodes or as a hub-and-spoke network. An emulation environment will be built upon a commercial cloud service, such as Amazon Web Services (AWS) Elastic Compute Cloud. The cloud environment may enable a flexible, lower maintenance approach versus a multi-hop network based in a physical laboratory. The cloud platform will provide a secure environment allowing for collaboration among government and academic entities.

Ricardo Lent↗

NASA Leveraging Commercial Communication Ground Stations for Small Satellites

The Space Communications and Navigation (SCaN) program at NASA has reorganized its operations portfolio into two networks: the Deep Space Network and the new Near Space Network (NSN). With this reorganization, NASA can begin transforming to 100% direct-to-Earth commercial communications services for missions in the near-Earth region. NASA’s leveraging of commercial direct-to-Earth ground stations offers several benefits for the small satellite community, including lower cost, greater coverage, and increased technology infusion. In the fall of 2020, SCaN announced their intention to rely primarily on industry-provided communications services for missions close to Earth by 2030. Commercial services are one way to infuse new technology into the ground station network without requiring an investment from NASA. Digital Video Broadcast, Satellite Second Generation (DVB-S2) is one example of a current technology. When combined with variable coding and modulation (VCM), the system automatically optimizes the data rate based on signal performance, significantly increasing total downlink data volume without an increase in the spacecraft effective isotropic radiated power (EIRP). There are several commercial service providers, including Amazon Web Service (AWS) Ground Station (AGS) and the KSATLITE ground stations that support SmallSat missions using DVB-S2 waveforms for downlinks. This paper identifies some commercial off-the-shelf (COTS) CubeSat/SmallSat DVB-S2 X-band and Ka-band radios. Overall, NASA’s increased dependence on commercial direct-to-Earth ground stations is a significant benefit for the small satellite community.

Space Communications↗

Applying the Cognitive Space Gateway to Swarm Topologies

NASA’s future vision for interplanetary networking includes a lunar network, Cube Satellite (CubeSat) constellations, and deep space robotic missions, comprising what could be viewed as a network of networks. Delay-tolerant networking (DTN) architecture and protocols provide a standard network layer among these varying scenarios and mitigate many challenges of the space environment, such as long delays, unplanned service interruptions, and asymmetric links. The Cognitive Space Gateway (CSG) is a routing method in a DTN architecture that uses spiking neural networks as the learning element to optimize outing decisions in a complex environment. This work aims to further develop cognitive networking technologies in several critical areas, including DTN, the CSG algorithm, SmallSat swarm topologies, and cloud services. The CSG algorithm is tested in a realistic scenario in which the emulated network topology is based on a SmallSat swarm. The emulation environment will be built upon a commercial cloud service, such as Amazon Web Services (AWS) Elastic Compute Cloud. This work investigates the ability of such a platform to enable a flexible, lower maintenance approach to creating a multihop network outside of a physical laboratory. The cloud platform will provide a secure environment allowing for collaboration among government and academic entities.

Ricardo Lent↗

Connecting Users and Applications with Po.daac Hosted GHRSST Data

The 80+ GHRSST public datasets represent a rich resource for sea surface temperature research and applications given their time series length, resolution, spatial coverage, varying measurement types and processing levels, and availability in the full spectrum of PO.DAAC tools and services ecosystem. The PO.DAAC has created a publicly accessible recipe suite for the user community to perform straightforward yet powerful computations on GHRSST data using python recipes, Jupyter notebooks, R, Matlab, and the NCO programming language. These recipes include numerical computations for regional and global SST trends, anomaly derivations, EOF analysis, climate signal reproduction, and ocean phenology. For example, one recipe reproduces a famous SST based warming figure from the Fourth National Climate Assessment (USA) while another focuses on quantifying the regional changes in ocean SST phenology. Most are python-based while some contain hybrid calls and leverage the NCO programming interface too. All are available on the PO.DAAC user forum (https://podaac.jpl.nasa.gov/forum/) and/or via the open source NASA GitHub repository (https://github.com/nasa/podaac_tools_and_services). Several are available in the Jupyter notebook framework including podaacypy (https://github.com/nasa/podaacpy), a recipe for GHRSST granule metadata discovery and application, and more recently a Jupyter notebook developed to support data analysis and visualization of a cloud-based Zarr formatted Level 4 MUR dataset in the AWS Open Data Registry. Throughout the summer of 2020, the PO.DAAC intends to add and migrate more of its numerical recipes to the Jupyter notebook framework and publish them on its open source GitHub repository.

Gentemann, Chelle↗

AAM National Campaign Tech Talk: Data Pipeline Familiarization

NASA's AWS-based Data Pipeline allows real-time data submission and ingestion, with immediate monitoring of data rate, coverage and ingestion quality. Problems are immediately discovered and can be corrected with agility by both Partners and NASA during the a simulation or flight event​.

Data Pipeline↗

Cloud-Based Demodulation and Data Distribution of a Satellite Downlink

Ground station networks connected to the cloud allow space missions to have global communications coverage without operating their own infrastructure. In this work, we describe the communications architecture for the TechEdSat-13 mission, which performed the first in-space characterization of a neuromorphic processor. The mission utilizes a commercial provider for S-band downlinks. A suite of cloud services and open-source software such as GNU Radio are leveraged to demodulate signals received by an AWS ground station during passes with TechEdSat-13 and store recovered data. Once a pass is scheduled, the entire process takes place without human intervention. On-orbit results the past year of operations are presented, demonstrating the advantages of this approach over traditional operator-owned ground stations. Use of software-defined radio makes possible custom signal processing. The homogeneity of apertures and their interfaces to the cloud simplifies scaling across many sites. This abundance of candidate links lays the groundwork for intelligent scheduling agents to optimize pass selection across several factors, automatically recover from failed contacts, and gather metrics to learn from past performance.

cloud demodulation↗