Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “aws”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
NASA NTRS2017

Task 28: Web Accessible APIs in the Cloud Trade Study

This study explored three candidate architectures for serving NASA Earth Science Hierarchical Data Format Version 5 (HDF5) data via Hyrax running on Amazon Web Services (AWS). We studied the cost and performance for each architecture using several representative Use-Cases. The objectives of the project are: Conduct a trade study to identify one or more high performance integrated solutions for storing and retrieving NASA HDF5 and Network Common Data Format Version 4 (netCDF4) data in a cloud (web object store) environment. The target environment is Amazon Web Services (AWS) Simple Storage Service (S3).Conduct needed level of software development to properly evaluate solutions in the trade study and to obtain required benchmarking metrics for input into government decision of potential follow-on prototyping. Develop a cloud cost model for the preferred data storage solution (or solutions) that accounts for different granulation and aggregation schemes as well as cost and performance trades.

cost model↗
NASA NTRS2017

Use of Schema on Read in Earth Science Data Archives

Traditionally, NASA Earth Science data archives have file-based storage using proprietary data file formats, such as HDF and HDF-EOS, which are optimized to support fast and efficient storage of spaceborne and model data as they are generated. The use of file-based storage essentially imposes an indexing strategy based on data dimensions. In most cases, NASA Earth Science data uses time as the primary index, leading to poor performance in accessing data in spatial dimensions. For example, producing a time series for a single spatial grid cell involves accessing a large number of data files. With exponential growth in data volume due to the ever-increasing spatial and temporal resolution of the data, using file-based archives poses significant performance and cost barriers to data discovery and access. Storing and disseminating data in proprietary data formats imposes an additional access barrier for users outside the mainstream research community. At the NASA Goddard Earth Sciences Data Information Services Center (GES DISC), we have evaluated applying the schema-on-read principle to data access and distribution. We used Apache Parquet to store geospatial data, and have exposed data through Amazon Web Services (AWS) Athena, AWS Simple Storage Service (S3), and Apache Spark. Using the schema-on-read approach allows customization of indexing spatially or temporally to suit the data access pattern. The storage of data in open formats such as Apache Parquet has widespread support in popular programming languages. A wide range of solutions for handling big data lowers the access barrier for all users. This presentation will discuss formats used for data storage, frameworks with This presentation will discuss formats used for data storage, frameworks with support for schema-on-read used for data access, and common use cases covering data usage patterns seen in a geospatial data archive.

cloud applications↗
NASA NTRS2018

Geonex: A NASA-NOAA Collaboration for Producing Land Surface Products from Geostationary Sensors Using Cloud Computing

The latest generation of geostationary satellites carry sensors such as the Advanced Baseline Imager (GOES-16/17) and the Advanced Himawari Imager (Himawari-8/9) that closely mimic the spatial and spectral characteristics of MODIS and VIIRS, useful for monitoring land surface conditions. The NASA Earth Exchange (NEX) team at Ames Research Center has embarked on a collaborative effort among scientists from NASA and NOAA exploring the feasibility of producing operational land surface products similar to those from MODIS/VIIRS. The team built a processing pipeline called GEONEX that is capable of converting raw geostationary data into routine products of Fires, surface reflectances, vegetation indices, LAI/FPAR, ET and GPP/NPP using algorithms adapted from both NASA/EOS and NOAA/GOES-R programs. The GEONEX pipeline has been deployed on Amazon Web Services cloud platform and it currently leverages near-realtime geostationary data hosted in AWS public datasets under a NOAA-AWS agreement.Initial analyses of various products from ABI/AHI sensors suggest that they are comparable to those from MODIS in representing the spatio-temporal dynamics of land conditions. Cloud computing offers a variety of options for deploying the GEONEX pipeline including choice CPUs, storage media, and automation. We estimate the cost of deploying GEONEX to be $400 - 750 a month for processing data (every 30 minutes) and producing products over the conterminous US. For products such as Fire, latency can be as little as 10 minutes from the time of data acquisition.

Geostationary↗
NASA NTRS2018

GEONEX: Webpage to Display NASA-NOAA Collaboration of Producing Land Surface Products from Geostationary Sensors

The latest generation of geostationary satellites carry sensors such as the Advanced Baseline Imager (GOES-16/17) and the Advanced Himawari Imager (Himawari-8/9) that closely mimic the spatial and spectral characteristics of MODIS and VIIRS, useful for monitoring land surface conditions. The NASA Earth Exchange (NEX) team at Ames Research Center has embarked on a collaborative effort among scientists from NASA and NOAA exploring the feasibility of producing operational land surface products similar to those from MODIS/VIIRS. The team built a processing pipeline called GEONEX that is capable of converting raw geostationary data into routine products of Fires, surface reflectances, vegetation indices, LAI/FPAR, ET and GPP/NPP using algorithms adapted from both NASA/EOS and NOAA/GOES-R programs. The GEONEX pipeline has been deployed on Amazon Web Services cloud platform and it currently leverages near-realtime geostationary data hosted in AWS public datasets under a NOAA-AWS agreement.In order to better introduce the GEONEX products to the science community, we set up a simple webpage (www.geonex.org) to describe the background and the motivation of the project, the algorithms used in deriving the products, and user manuals to the data files. We will also update the status of the data processing, in particular the near-real-time products, on the website and provide links (in text or json files) to the latest datasets.

geostationary↗
NASA NTRS2019

Geonex: Land Surface Monitoring from a New Generation of Geostationary Sensors

The latest generation of geostationary satellites carry sensors such as the Advanced Baseline Imager (GOES-16/17) and the Advanced Himawari Imager (Himawari-8/9) that closely mimic the spatial and spectral characteristics of MODIS and VIIRS, useful for monitoring land surface conditions. The NASA Earth Exchange (NEX) team at Ames Research Center has embarked on a collaborative effort among scientists from NASA and NOAA exploring the feasibility of producing operational land surface products similar to those from MODIS/VIIRS. The team built a processing pipeline called GEONEX that is capable of converting raw geostationary data into routine products of Fires, surface reflectances, vegetation indices, LAI/FPAR, ET and GPP/NPP using algorithms adapted from both NASA/EOS and NOAA/GOES-R programs. The GEONEX pipeline has been deployed on Amazon Web Services cloud platform and it currently leverages near-realtime geostationary data hosted in AWS public datasets under a NOAA-AWS agreement. Initial analyses of various products from ABI/AHI sensors suggest that they are comparable to those from MODIS in representing the spatio-temporal dynamics of land conditions. Cloud computing offers a variety of options for deploying the GEONEX pipeline including choice CPUs, storage media, and automation. By making the GEONEX pipeline available on the cloud, we hope to engage a broad community of Earth scientists from around the world in utilizing this new source of data for Earth monitoring.

Nemani, Ramakrishna R.↗
NASA NTRS2019

Earth Observations from Geostationary Satellites

The latest generation of geostationary satellites carry sensors such as the Advanced Baseline Imager (GOES-16/17) and the Advanced Himawari Imager (Himawari-8/9) that closely mimic the spatial and spectral characteristics of MODIS and VIIRS, useful for monitoring land surface conditions. The NASA Earth Exchange (NEX) team at Ames Research Center has embarked on a collaborative effort among scientists from NASA and NOAA exploring the feasibility of producing operational land surface products similar to those from MODIS/VIIRS. The team built a processing pipeline called GeoNEX that is capable of converting raw geostationary data into routine products of Fires, surface reflectances, vegetation indices, LAI/FPAR, ET and GPP/NPP using algorithms adapted from both NASA/EOS and NOAA/GOES-R programs. The GeoNEX pipeline has been deployed on Amazon Web Services cloud platform and it currently leverages near-realtime geostationary data hosted in AWS public datasets under a NOAA-AWS agreement. Initial analyses of various products from ABI/AHI sensors suggest that they are comparable to those from MODIS in representing the spatio-temporal dynamics of land conditions. Cloud computing offers a variety of options for deploying the GeoNEX pipeline including choice CPUs, storage media, and automation. By making the GEONEX pipeline available on the cloud, we hope to engage a broad community of Earth scientists from around the world in utilizing this new source of data for Earth monitoring.

Earth↗
NASA NTRS2019

GeoNEX: Land Monitoring from a New Generation of Geostationary Sensors

The latest generation of geostationary satellites carry sensors such as the Advanced Baseline Imager (GOES-16/17) and the Advanced Himawari Imager (Himawari-8/9) that closely mimic the spatial and spectral characteristics of MODIS and VIIRS, useful for monitoring land surface conditions. The NASA Earth Exchange (NEX) team at Ames Research Center has embarked on a collaborative effort among scientists from NASA and NOAA exploring the feasibility of producing operational land surface products similar to those from MODIS/VIIRS. The team built a processing pipeline called GEONEX that is capable of converting raw geostationary data into routine products of Fires, surface reflectances, vegetation indices, LAI/FPAR, ET and GPP/NPP using algorithms adapted from both NASA/EOS and NOAA/GOES-R programs. The GEONEX pipeline has been deployed on Amazon Web Services cloud platform and it currently leverages near-realtime geostationary data hosted in AWS public datasets under a NOAA-AWS agreement. Initial analyses of various products from ABI/AHI sensors suggest that they are comparable to those from MODIS in representing the spatio-temporal dynamics of land conditions. Cloud computing offers a variety of options for deploying the GEONEX pipeline including choice CPUs, storage media, and automation. By making the GEONEX pipeline available on the cloud, we hope to engage a broad community of Earth scientists from around the world in utilizing this new source of data for Earth monitoring.

GeoNEX↗
NASA NTRS2019

Cloud Giovanni: Reining in Costs and Improving Performance with Analytical Data Stores Using Scalable Serverless Architecture

Giovanni is the Geospatial Interactive Online Visualization ANd aNalysis Infrastructure developed at NASA GES DISC which provides a simple and intuitive way to visualize, analyze, and access vast amounts of Earth science data. It receives large number of user requests each day for a variety of analysis and visualization services, which leads to the big data challenge of serving gradually increasing large data volumes with diverse statistical algorithms. We hereby propose a multi-dimensional accumulation method which provides fast and cost-efficient cloud analysis for diverse services including both area averaging and time averaging. This method involves the weighted volume integration over multiple variable dimensions (time and space), and is implemented in AWS using Athena providing serverless and highly scalable data analysis. Compared to the standard method, this approach dramatically reduced the computational time by order of magnitude with a minimal AWS cost incurred. For example, for a benchmark of 10-year area averaging over the 1x1 degree daily variable, the computational time was reduced from minutes to seconds, and the Athena cost is only $5 for 100,000 requests.

Zhang, Hailiang↗
NASA NTRSDate not supplied

OpenNEX: An open collaboration platform for the earth science community

Satellite data from the past several decades provide the most consistent record of land-surface processes that form the basis for scientific assessments of the impacts of climate variations and changes on the environment and human social-economical activities. During this time, scientific research on the characterization and assessment of environmental changes had tended to focus on large-scale land-surface changes with significant social-economic impacts. Increasingly, attention is shifting toward changes that occur more locally and that most directly relate to the everyday life of the majority of the population. In addition, there has been needs to develop management and policy decision support systems that are based on local environmental information. Almost at the same time, the advancement in sensor technology has allowed us to collect an unprecedented volume of environmental data. These data must be curated and analyzed to extract useful information for research and decision support purposes. Established in 2013 and funded by NASA, the Open NASA Earth eXchange (OpenNEX; https://opennex.org/ , Jia et al., 2019 ) project partnered with Amazon Web Services (AWS) to make available a large amount of Earth observing data, modeling results, and analysis tools on the AWS. OpenNEX provides researchers, developers, educators, and ordinary users with easy access to an integrated Earth science computational and data platform, enabling citizen scientists and application developers to realize the full value of NASA data assets and software tools. To encourage the public's engagement in this project, NASA ran virtual workshops and prize competitions. The virtual workshops provided online lectures and tutorials about how prominent scientists used the data in their research and the tutorials gave examples how to use the tools to interrogate the data in the Amazon cloud. Finally, the prize competitions allowed much wider participation in the OpenNEX project and enable testing the non-traditional projects and out-of-box ideas. OpenNEX has continued to evolve and mature. Here, we highlight new features and functionalities available to the community.

Jian Zhang↗
NASA NTRSDate not supplied

Building Collaborative Proving Grounds for R2O2R – NASA/CCMC - NOAA/Space Weather Prediction Testbed (SWPT) Partnership

In 2016 and 2019, Executive Orders were signed by the President to prepare the Nation for Space Weather Events. At the forefront of being able to achieve this goal is the necessity to accelerate and enhance R2O2R. To drive this initiative, the Space Weather R2O2R Framework, initially led by NOAA and NASA, was developed. At the very center of the framework is the Space Weather Proving Grounds effort that will allow various institutions to collaborate, share, and advance their space weather modeling and simulation capabilities. The first of these Proving Grounds is the Architecture for Collaborative Evaluation (ACE) environment shared between the Community Coordinated Modeling Center (CCMC) at NASA and the Space Weather Prediction Testbed (SWPT) at NOAA. Since 2019, the ACE environment has been set up on the AWS cloud where it utilizes AWS cloud services and Docker containerization technology.

T. Tsui↗
NASA NTRS2021

Results from the ASTERIA CubeSat Extended Mission Experiments

Over the past two years, JPL has used the ASTERIA (Arcsecond Space Telescope Enabling Research In Astrophysics) CubeSat as an in-flight test platform during extended missions. ASTERIA successfully completed its prime mission in early 2018, and continued to operate in low Earth orbit (LEO) for an additional twenty months. This paper describes demonstrations that were performed on the spacecraft and on the ground-based testbed during the extended mission. These demonstrations fall into three categories: Autonomy technology maturation, hardware characterization, and science discovery. Autonomy technology maturation supported three development efforts. The first shifted the spacecraft commanding paradigm from time-based sequences to Task Networks (tasknets), which allow simpler commanding and more robust onboard execution. The second demonstrated onboard orbit determination in Low Earth Orbit (LEO) without GPS. This activity used a fully-independent means of spacecraft orbit determination for Earth orbiters using only passive imaging. The third technology provided in situ hardware health state estimation using a model-based reasoning technique. These three technologies were demonstrated either in flight or on the testbed individually, and then were combined to demonstrate the capability to perform autonomous navigation on board without ground intervention, even in the presence of anomalies. Hardware characterization involved both onboard and ground-based activities. On board, nonstandard attitude control modes were commanded to characterize the spacecraft pointing jitter as a function of target brightness, reaction wheel speed, controller gain, and the number of guide stars. The results provide insights into the contribution of jitter to the ASTERIA photometry and inform the feasibility of future astrophysics small satellite missions for which jitter control is an enabling technology. On the ground, the ASTERIA Operations Team coordinated with Amazon Web Services (AWS) to configure their new ground stations to communicate with ASTERIA to prove out their viability. ASTERIA used AWS ground stations for nominal operations for the last four months of the mission. Finally, ASTERIA continued to perform exoplanet science as the spacecraft was well-suited to execute long-term monitoring of stars such as alpha Centauri to search for small transiting planets. The science team also imaged a number of interesting objects including a comet, an asteroid, cities at night, and the moon, and coordinated with other projects on Targets of Opportunity for follow-up confirmations and co-observations. Throughout the prime and the extended missions, the ASTERIA spacecraft proved to be a mighty platform that “will go into history as an innovative milestone.”[1 - Zurbuchen]

Doran, Patrick↗
NASA NTRSDate not supplied

Design and Characterization of New 90 GHz Detectors for the Cosmology Large Angular Scale Surveyor (CLASS)

The Cosmology Large Angular Scale Surveyor (CLASS) is a polarization-sensitive telescope array located at an altitude of 5,200 m in the Chilean Atacama Desert. CLASS is designed to measure “E-mode” (even parity) and “B-mode” (odd parity) polarization patterns in the Cosmic Microwave Background (CMB) over large angular scales with the aim of improving our understanding of inflation, reionization, and dark matter. CLASS is currently observing with three telescopes covering four frequency bands: one at 40 GHz (Q); one at 90 GHz (W1); and one dichroic system at 150/220 GHz (G). In these proceedings, we discuss the updated design and in-lab characterization of new 90 GHz detectors. The new detectors include design changes to the transition-edge sensor (TES) bolometer architecture, which aim to improve stability and optical efficiency. We assembled and tested four new detector wafers, to replace four modules of the W1 focal plane. These detectors were installed into the W1 telescope, and will achieve first light in the austral winter of 2022. We present electrothermal parameters and bandpass measurements from in-lab dark and optical testing. From in-lab dark tests, we also measure a median NEP of 12.3 aW √ s across all four wafers about the CLASS signal band, which is below the expected photon NEP of 32 aW √ s from the field. We therefore expect the new detectors to be photon noise limited."

Carolina Nunez↗
NASA NTRSDate not supplied

Spacecraft-Initiated Scheduling of Commercial Communications Services

We propose a software framework enabling a spacecraft in Earth orbit to schedule its own access to space communications services via ground stations and relay satellites. All operations are automated, without a human in the loop, allowing for highly responsive service to meet time-varying needs. We describe a modular software architecture allowing the framework to interface with the scheduling systems of several providers - both government and commercial. Specifically, we present the results of an experiment with Amazon Web Services (AWS) Ground Station, an operational commercial service provider. Proposed software executed scheduling of a ground station antenna with minimal lead time (less than 20 minutes before the start of a pass). During the contact AWS cloud infrastructure was used to demodulate, process, and distribute data products from an orbiting satellite. We believe this work demonstrates the framework's ability to interface with additional providers and potential to integrate more complex decision-making in the future to schedule optimal contacts.

Scheduling↗
NASA NTRSDate not supplied

Parallel Plate Capacitor Aluminum KIDs for Future Far-Infrared Space-Based Observatories

Future space-based far-infrared astrophysical observatories will require exquisitely sensitive detectors consistent with the low optical backgrounds. The PRobe far-Infrared Mission for Astrophysics (PRIMA) will deploy arrays of thousands of superconducting kinetic inductance detectors (KIDs) sensitive to radiation between 25 and 265 μm. Here, we present laboratory characterization of prototype, 25 – 80 μm wavelength, low-volume, aluminum KIDs designed for the low-background environment expected with PRIMA. A compact parallel plate capacitor is used to minimize the detector footprint and suppress TLS noise. A novel resonant absorber is designed to enhance response in the band of interest. We present noise and optical efficiency measurements of these detectors taken with a low background cryostat and a cryogenic blackbody. A microlens-hybridized KID array is found to be photon noise limited down to about 50 aW with a limiting detector NEP of about 6.5×10 −19 W/Hz 1/2 . A fit to an NEP model shows that our optical system is well characterized and understood down to 50 aW. We discuss future plans for low-volume aluminum KID array development as well as the testbeds used for these measurements.

Far-Infrared↗
DOE OSTI2024

Hydrogen and its Vital Role in a Clean Energy Future

Large-scale, low -cost hydrogen production can enable an economically competitive, secure, and environmentally beneficial future energy system across multiple sectors. Furthermore, clean hydrogen can address specific sectors that are hard to decarbonize (e.g., heavy-duty trucking, load-following electricity, iron, steel, and cement) and can help the U.S. meet the net zero carbon goal by 2050. To achieve this goal, tens of millions of metric tons of clean, reliable, and affordable hydrogen will be needed annually1. In 2021, the Hydrogen Energy Earthshot was launched, and its goal is to reduce the cost of clean hydrogen to $1 per $1 kilogram in 1 decade (1 1 1) 2. One very promising pathway for large-scale hydrogen production is water splitting. Water splitting technologies range from commercial technologies such as electrolyzers to approaches that are at a much earlier stage of development, such as photoelectrochemical (PEC) and thermochemical (TCH) processes. All these water splitting pathways offer diverse benefits in energy storage, grid services, and cross-sector emissions reductions while taking advantage of the diverse domestic resources. However, critical materials-, component- and system-level challenges must be addressed to improve efficiency and durability and reduce cost. To address these barriers and move these promising and high impact technologies forward, the HydroGEN Advanced Water Splitting Materials (AWSM) and the H2 from the Next-generation of Electrolyzers of Water (H2NEW) consortia were formed and supported by the Department of Energy (DOE) EERE Hydrogen and Fuel Cell Technologies Office (HFTO). HydroGEN (https://www.energy.gov/eere/h2awsm/) consortium, established in 2016, is an Energy Materials Network (EMN) that aims to accelerate the materials R&D of low technology readiness level (TRL) advanced water splitting (AWS) technologies. The consortium comprises five core national laboratories and focuses on four early-stage AWS pathways: alkaline exchange membrane (AEM) electrolysis, proton conducting solid oxide electrolysis (p-SOEC), photoelectrochemical, and thermochemical water splitting. Liquid alkaline and PEM electrolyzers are already commercial and significant advancements in oxygen conducting solid oxide electrolysis cells (o-SOECs) have been realized. Yet, these systems are still too expensive and not sufficiently durable for wide-scale commercialization. To enable high-volume manufacturing of affordable, durable, efficient electrolyzers, H2NEW (https://h2new.energy.gov/), another multi-lab consortium, was established in 2020. This comprehensive, concerted effort is focused on overcoming barriers related to components and materials integration and scale-up to achieve performance, durability, with an initial focus to achieve $2/kg H2 by 2026.

AEM↗
DOE OSTI2024

Computer Vision on Edge Devices for the Short Term Prediction of Cloud Cover

Edge Computing and IoT are important pieces of today's technological landscape. Here, we build a low-cost IoT sensor for sky imaging and program it using AWS GreenGrass, one of the leading IoT platforms. We demonstrate remote reprogramming of this device to load software that predicts sun shading events through the linear advection method, which is a baseline algorithm that can be used to benchmark algorithmic improvements in future work. Some future directions for sky imaging research are enumerated.

14 SOLAR ENERGY↗
NASA NTRSDate not supplied

Creating Satellite Data Products in the Cloud: SatCORPS Global Cloud Composites

Real time satellite observations and real time derived cloud products are becoming an important tool for both science as well as business ventures. The SatCORPS group leverages public and private cloud-based sources of satellite observations to create its Global Cloud Composite product in near real-time. This dataset allows others access to cloud information that can be accessed directly from the cloud. In this work, we describe the software algorithms and software infrastructure that we have created to create and distribute this product through our hybrid cloud and on-premises system that leverages the strengths and weaknesses of each platform. We will also describe the GCC product itself in terms of the scientific parameters available, resolution and temporal availability. Finally, we will also describe our web-based tools including our Geographic Information System that can be used for visualization and analysis. The products from the processing can be geospatially-service enabled as ArcGIS Image Services and Open Geospatial Consortium (OGC) Web Mapping/Coverage Services for visualization and analysis via a web mapping application and common geospatial software. The SatCORPS Global Composite Cloud product provides sophisticated global cloud products with very low latency which we see that as filling a rapidly growing need in the research, modelling and business community. We present a detailed description of the SatCORPS GCC product in terms of the capabilities and research benefits as well as the delivery architecture in AWS. We take the discussion further and describe the web tools that the SatCORPS group has developed that allow users to access and use the GIS data that it creates.

Global Cloud Composite SatCORPS SMCE AWS↗
DOE OSTI2025

Using the ATLAS experiment software on heterogeneous resources

With the large dataset expected from 2030 onwards by the HL-LHC at CERN, the ATLAS experiment is reaching the limits of the current data processing model in terms of traditional CPU resources based on x86_64 architectures and an extensive program for software upgrades towards the HL-LHC has been set up. The ARM CPU architecture is becoming a competitive and energy efficient alternative. Accelerators like GPUs are available in any recent HPC. In the past years ATLAS has successfully ported its full data processing and simulation software framework Athena to ARM and has invested significant effort in porting parts of the reconstruction and simulation algorithms to GPUs. We report on the successful usage of the ATLAS experiment offline and online software framework Athena on ARM and GPUs through the PanDA workflow management system at various WLCG sites. Furthermore we report on performance optimizations of the builds for ARM CPUs and the GPU integration efforts. We will discuss performance comparisons of different ARM and x86_64 architectures on WLCG resources and Cloud compute providers like GCP and AWS using ATLAS productions workflows as used in the Hep-Score23 benchmark suite.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗