SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Files needed to reproduce figures and extended data in "Spin squeezing in an ensemble of nitrogen-vacancy centres in diamond". Unzip the data file and run the Jupyter notebooks, which contain multiple figures. Julia 1.9.4 and Python 3 are required.
Engine icing researchers at the NASA Glenn Research Center use the Escort data acquisition system in the Propulsion Systems Laboratory (PSL) to generate and collect a tremendous amount of data every day. Currently these researchers spend countless hours processing and formatting their data, selecting important variables, and plotting relationships between variables, all by hand, generally analyzing data in a spreadsheet-style program (such as Microsoft Excel). Though spreadsheet-style analysis is familiar and intuitive to many, processing data in spreadsheets is often unreproducible and small mistakes are easily overlooked. Spreadsheet-style analysis is also time inefficient. The same formatting, processing, and plotting procedure has to be repeated for every dataset, which leads to researchers performing the same tedious data munging process over and over instead of making discoveries within their data. This paper documents a data analysis tool written in Python hosted in a Jupyter notebook that vastly simplifies the analysis process. From the file path of any folder containing time series datasets, this tool batch loads every dataset in the folder, processes the datasets in parallel, and ingests them into a widget where users can search for and interactively plot subsets of columns in a number of ways with a click of a button, easily and intuitively comparing their data and discovering interesting dynamics. Furthermore, comparing variables across data sets and integrating video data (while extremely difficult with spreadsheet-style programs) is quite simplified in this tool. This tool has also gathered interest outside the engine icing branch, and will be used by researchers across NASA Glenn Research Center. This project exemplifies the enormous benefit of automating data processing, analysis, and visualization, and will help researchers move from raw data to insight in a much smaller time frame.
In this talk, we will discuss our work for testing OPeNDAP client access of data stored in the Amazon S3 cloud storage using a set of common analysis tools including Panoply, Jupyter Notebooks with Python xarray, NCO command line tool package, ArcGIS, and GDAL. We will also discuss our ongoing work on improving performance in Hyrax aggregation functionality.
The emergence of a new generation of geostationary satellite sensors provides land andatmosphere monitoring capabilities similar to MODIS and VIIRS with far greater temporal resolution (5-15 minutes). However, processing such large volume, highly dynamic datasets requires computing capabilities that (1) better support data access and knowledge discovery for scientists; (2) provide resources to enable real-time processing for emergency response (wildfire, smoke, dust, etc.); and (3) provide reliable and scalable services for the broader user community. This paper presents an implementation of GeoNEX (Geostationary NASA-NOAA Earth Exchange) services that integrate scientific algorithms with Amazon Web Services (AWS) to provide near realtime monitoring (~5 minute latency) capability in a hybrid cloud-computing environment. It offers a user-friendly, manageable and extendable interface and benefits from the scalability provided by Amazon Web Services. Four use cases are presented to illustrate how to (1) search and access geostationary data; (2) configure computing infrastructure to enable near real-time processing; (3) disseminate and utilize research results, visualizations, and animations to concurrent users; and (4) use a Jupyter Notebook-like interface for data exploration and rapid prototyping. As an example of (3), the Wildfire Automated Biomass Burning Algorithm (WF_ABBA) was implemented on GOES-16 and -17 data to produce an active fire map every 5 minutes over the conterminous US. Details of the implementation strategies, architectures, and challenges of the use cases are discussed.
The Committee on Earth Observation Satellites (CEOS) System Engineering Office (SEO) has supported the Open Data Cube (ODC) initiative to provide a data architecture solution that has value to its global users and increases the impact of EO satellite data. ODC is an open-source platform for processing satellite data. We have developed software products and tools around the core ODC that would help users perform machine learning on EO satellite data. The recent United Nations (UN) Sustainable Development Agenda provides a shared blueprint for peace and prosperity for people and for the planet, considering our current situation and helping to create a plan. The core of this agenda is a set of seventeen Sustainable Development Goals (SDGs), which represent an urgent call for action by all countries - both developed and developing - in a global partnership. The CEOS SEO team has recently developed and released a set of innovative Jupyter notebooks addressing UN SDGs 6.6.1 (spatial extents of water-related ecosystems), 11.3.1 (ratio of land consumption rate to population growth rate), and 15.3.1 (proportion of land that is degraded over total land area). These notebooks empower users by providing features that will assist with streamlining analysis ready data retrieval, processing, and visualization. We have recently incorporated several machine learning techniques in these notebooks. In this paper, we present the lessons learned from our experience on classifying land using supervised and unsupervised machine learning techniques using ODC framework for UN SDGs. We identify the current limitations of ODC to seamlessly support machine learning techniques. We propose features that would help machine learning, specifically within the ODC framework. We propose a thematic indexing/loading of data for both unsupervised learning as well as data annotation/labeling pipeline. Currently, ODC supports machine learning by separating data-management from the analysis process. It works as a mechanism to load cubes of data. ODC does not natively support features that are vital in machine learning such as validation splits, fair/balanced sampling, establishing load size constraints, etc. We believe that our proposed features will empower users by providing features that bring machine learning techniques closed to ODC. Enhancements to ODC to better accommodate machine learning techniques can assist in fulfilling UN SDGs such as 6.3.2, 6.4.2, 6.6.1, 11.3.1, 14.1.1, 15.1.1, 15.3.1, and 15.4.2.
The Atmospheric Science Data Center (ASDC), Goddard Earth Sciences Data and Information Systems Center (GES DISC), Socioeconomic Data and Applications Center (SEDAC), Oak Ridge National Laboratory (ORNL), Land Processes DAAC, Alaska Satellite Facility (ASF), as well as NASA Global Imagery Browse Service (GIBS), Earth Science Data Systems (ESDS) Geographic Information Systems Team (EGIST), Earthdata Content Delivery Team, ArcGIS Online Governance Team, and the Systematic Data Transformation ACCESS team have come together to establish the ArcGIS DAAC Collaboration. This coalition of participants will demonstrate their use of the ArcGIS Enterprise to support Earth science research, applied science, and outreach using Earth Observing System Data and Information System (EOSDIS) data. This includes the use of web services to fuse data products across space and time through services (e.g. ArcGIS Image Services, Open Geospatial Consortium (OGC) Web Coverage Service (WCS), and OGC Web Mapping Services (WMS)) for analysis in Jupyter notebooks, desktop tools, and web based applications.
The LaRC Smart Lab applications are a series of software tools to greatly enhance researcher efficiency by streamlining and automating workflows. Python scripts and applications are increasingly being used in scientific workflows, including for instrument control and data processing. Interactive Python scripting environments such as Jupyter Lab provide powerful tools for using Python. In some use cases, the development of standalone applications with dedicated graphical user interfaces (GUIs) can enhance the utility of the code and open it up to more users, including non-programmers. Here, we describe a GUI based optical micrometer data visualization application developed as part of the LaRC SmartLab project. We highlight its use in visualizing experimental data and briefly discuss its implementation to give pointers to programmers who wish develop work based on this application's or similar co de.
Thermodynamics has been fundamental to the interpretation of geologic data and modeling of geologic systems for decades. However, more recent advancements in computational capabilities and a marked increase in researchers’ accessibility to computing tools has outpaced the functionality and extensibility of currently available modeling tools. Here we present VESIcal (Volatile Equilibria and Saturation Identification calculator): the first comprehensive modeling tool for H 2 O, CO 2 , and mixed (H 2 O-CO 2 ) solubility in silicate melts that: a) allows users access to seven commonly used models, plus easy inter-comparison between models; b) provides universal functionality for all models (e.g., functions for calculating saturation pressures, degassing paths, etc.); c) can process large datasets (1,000’s of samples) automatically; d) can output computed data into an excel spreadsheet for simple post-modeling analysis; e) integrates advanced plotting capabilities directly within the tool; and f) provides all of these within the framework of a python library, making the tool extensible by the user and allowing any of the model functions to be incorporated into any other code capable of calling python. The tool is presented within this manuscript, which is a Jupyter notebook containing worked examples accessible to python users with a range of skill levels. The basic functions of VESIcal can also be access via a web app (https://vesical.anvil.app). The VESIcal python library is open-source and available for download at https://github.com/kaylai/VESIcal.
GeneLab for Colleges and Universities (GL4U) will provide space biology-relevant training in bioinformatics to the next generation of scientists through direct and indirect approaches. The GeneLab (GL) team will host two annual data processing bootcamps, one for college-level students (direct) and one for college educators (indirect – Training of Trainers), in which participants learn to analyze space-relevant omics data hosted on GL. The first bootcamp took place in early June 2021 with about 30 SJSU undergraduate students and covered space biology-specific lectures and hands-on instruction using Jupyter Notebooks (JNs) for RNA sequence (RNAseq) data analysis. All training materials including the enclosed files listed below will be made publicly available on GitHub. RNAseq Bootcamp Lectures (attached in combined file): Introduction to NASA, Space Biology, GeneLab, and the Command Line: NASA_GL_CL_Intro_FINAL.pdf - DRAFT from initial submission NASA_SB_GL_CL_Intro_FULL.pdf - FINAL version presented during the bootcamp - only minor edits from the draft version RNAseq and Data Processing Overview: RNAseq_Overview_FINAL.pdf - DRAFT from initial submission RNAseq_Overview_FULL.pdf - FINAL version presented during the bootcamp - only minor edits from the draft version Overview of the Statistics Used for RNAseq Data Analysis: SJSU_Statistics_Intro_Lecture_FINAL.pdf - DRAFT from initial submission Statistics_Overview_FULL.pdf - FINAL version presented during the bootcamp - only minor edits from the draft version Completed JNs in HTML format (attached in combined file): Unix_Intro_JN_06-2021_completed.html R_Intro_JN_06-2021_completed.html RNAseq_fastq_to_counts_JN_06-2021_completed.html RNAseq_DGE_JN_06-2021_completed.html RNAseq Bootcamp Recordings (attached): GL4U_RNAseq_Bootcamp_June_2021_Pilot_Day1_Part_1_of_5.mp4 GL4U_RNAseq_Bootcamp_June_2021_Pilot_Day1_Part_2_of_5.mp4 GL4U_RNAseq_Bootcamp_June_2021_Pilot_Day1_Part_3_of_5.mp4 *There were issues with the part 4 recording so that is not available GL4U_RNAseq_Bootcamp_June_2021_Pilot_Day1_Part_5_of_5.mp4 GL4U_RNAseq_Bootcamp_June_2021_Pilot_Day2_Part_1_of_3.mp4 GL4U_RNAseq_Bootcamp_June_2021_Pilot_Day2_Part_2_of_3.mp4 GL4U_RNAseq_Bootcamp_June_2021_Pilot_Day2_Part_3_of_3.mp4 GL4U_RNAseq_Bootcamp_June_2021_Pilot_Day3_Part_1_of_4.mp4 GL4U_RNAseq_Bootcamp_June_2021_Pilot_Day3_Part_2_of_4.mp4 GL4U_RNAseq_Bootcamp_June_2021_Pilot_Day3_Part_3_of_4.mp4 GL4U_RNAseq_Bootcamp_June_2021_Pilot_Day3_Part_4_of_4.mp4 GL4U_RNAseq_Bootcamp_June_2021_Pilot_Day4_Part_1_of_4.mp4 GL4U_RNAseq_Bootcamp_June_2021_Pilot_Day4_Part_2_of_4.mp4 GL4U_RNAseq_Bootcamp_June_2021_Pilot_Day4_Part_3_of_4.mp4 GL4U_RNAseq_Bootcamp_June_2021_Pilot_Day4_Part_4_of_4.mp4 GL4U_RNAseq_Bootcamp_June_2021_Pilot_Day5_Part_1_of_4.mp4 GL4U_RNAseq_Bootcamp_June_2021_Pilot_Day5_Part_2_of_4.mp4 GL4U_RNAseq_Bootcamp_June_2021_Pilot_Day5_Part_3_of_4.mp4 GL4U_RNAseq_Bootcamp_June_2021_Pilot_Day5_Part_4_of_4.mp4
Spaceflight modifies gene expression in every organism examined to date, including humans. Understanding how these gene expression changes affect physiology is crucial for the development of countermeasures to enable long-duration manned missions. NASA’s GeneLab project provides researchers open access to multi-omics data, including genetic and gene expression data, from spaceflight experiments that can be mined to understand the effects of spaceflight on biological systems. To ensure new knowledge generation through data re-use, it is important to maximize the number of scientists who utilize GeneLab data. Training students on the GeneLab platform is the best way to create long-term adopters of this NASA database and its tools. Turning students into future instructors and advocates will also accelerate the dissemination of these data and tools to the broader scientific community. Therefore, in collaboration with the GeneLab Educational Working Group (EWG), GeneLab has created GeneLab for Colleges and Universities (GL4U). GL4U provides space biology-relevant training in bioinformatics to the next generation of scientists through direct and indirect approaches. The GeneLab team plans to host two annual data processing bootcamps, one for college-level students (direct) and one for college educators (indirect – training of trainers), in which participants learn to analyze GeneLab’s space-relevant omics data. During the bootcamp, educators will receive materials and training to enable them to run the bootcamp at their home institutions or alternatively to adapt the content to implement within existing courses, thereby extending the reach of this initiative. The GL4U direct training pilot program was conducted in June 2021 in collaboration with USRA and San Jose State University (SJSU). During the pilot, SJSU students participated in a week-long bootcamp consisting of space biology-specific lectures and hands-on instruction using Jupyter Notebooks to analyze RNA sequence data. This pilot demonstrates the capacity of GL4U for training young scientists and encouraging data re-use.
Two recent large data releases for the Atacama Cosmology Telescope (ACT), called DR4 and DR5, are available for public access. These data include temperature and polarization maps that cover nearly half the sky at arcminute resolution in three frequency bands; lensing maps and component-separated maps covering ∼2100 deg 2 of sky; derived power spectra and cosmological likelihoods; a catalog of over 4000 galaxy clusters; and supporting ancillary products including beam functions and masks. The data and products are described in a suite of ACT papers; here we provide a summary. In order to facilitate ease of access to these data, we present a set of Jupyter IPython notebooks developed to introduce users to DR4, DR5, and the tools needed to analyze these data. The data products (excluding simulations) and the set of notebooks are publicly available on the NASA Legacy Archive for Microwave Background Data Analysis; simulation products are available on the National Energy Research Scientific Computing Center.
The Modern Era Retrospective-analysis for Research and Applications, Version 2 (MERRA-2) is the global atmospheric data reanalysis for the satellite era produced by NASA’s Global Modeling and Assimilation Office (GMAO), using the Goddard Earth Observing System Model (GEOS)version 5.12.4. The data are officially distributed by the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC). MERRA-2 data have been widely used by the Earth sciences and application community. Since MERRA-2 data were released in early 2016, the number of registered data users has grown steadily from 1,252 in 2016 to 6477 in 2020. By the end of October 2021, ~16 petabytes (over 360 million files) of data have been distributed to more than 18,900 users. Searching in Google Scholar (https://scholar.google.com/), we have found over 7,000 articles, published between January 2017 and May 2021, involving the use ofMERRA-2 data. The figure shows the numbers for various application areas in which theMERRA-2 data have been used, covering almost all of the application areas defined in NASA Applied Sciences (http://appliedsciences.nasa.gov). The largest number of articles are found in disaster research, with the subcategories ordered in flood, wildfires, hurricanes and cyclones, and other forms of severe weather. In this presentation, we will discuss the preliminary findings from a review of the selected literature that uses MERRA-2 data in applied sciences. The current analytic and interoperable data services at GES DISC are listed, such as the on-the-fly subset and analysis service, NASA Giovanni; THREDDS Data Server(TDS); and Python Jupyter notebooks. In addition, we will introduce two new services for supporting the open sciences: My Dashboard and Related Publications.
NASA’s GeneLab project provides researchers open access to space-relevant experiment multi-omics data that can be mined to understand the effects of spaceflight on biological systems. To maximize the number of scientists who understand and utilize GeneLab data and data processing pipelines, GeneLab has created GeneLab for Colleges and Universities (GL4U). GL4U provides space biology-relevant training in bioinformatics to the next generation of scientists through direct and indirect approaches. The GeneLab team plans to host two annual data processing bootcamps, one for college-level students (direct) and one for college educators (indirect – training of trainers), in which participants learn to analyze GeneLab’s space-relevant omics data. The GL4U direct training pilot program was conducted in June 2021. During the pilot, students participated in a week-long bootcamp consisting of space biology-specific lectures and hands-on instruction using Jupyter Notebooks to analyze RNA sequence data. This pilot demonstrated the capacity of GL4U for training young scientists and encouraging data re-use. During the educator pilot, scheduled for June 2022, educators will receive materials and training to enable them to run the bootcamp at their home institutions or alternatively to adapt the content to implement within existing courses, thereby extending the reach of this initiative.
This workshop will cover the basics of using the fmdtools package for the simulation of hazardous scenarios for resilience simulation. The fmdtools simulation package is an open-source python toolkit for simulating the dynamic response of a system to internal and externally-driven hazardous scenarios, including faults and environmental conditions, that can be used to analyze the risks related to these hazards. Prior to the development of fmdtools, researchers had to either adapt an (often limited) propriety toolkit or develop their own design/simulation/analysis codes to develop their models of hazardous events, a significant technical burden to both (1) leveraging resilience modeling methodologies and (2) extending these methodologies with their own contributions. The fmdtools package provides a number of model constructs and simulation and analysis methods to enable the designer to focus to solely on their modeling case-study while still enabling a significant degree of model expressiveness and adaptability via Python-based model definition. This tutorial will present the setup of fmdtools and a high-level overview of its use, as well as some simple examples for understanding how to leverage its modelling, simulation, and analysis capabilities. Familiarity with jupyter notebook and basic python will be assumed.
NASA’s GeneLab project provides researchers open access to space-relevant experiment multi-omics data that can be mined to understand the effects of spaceflight on biological systems. To maximize the number of scientists who understand and utilize GeneLab data and data processing pipelines, GeneLab has created GeneLab for Colleges and Universities (GL4U). GL4U provides space biology-relevant training in bioinformatics to the next generation of scientists through direct and indirect approaches. The GeneLab team plans to host two annual data processing bootcamps, one for college-level students (direct) and one for college educators (indirect – training of trainers), in which participants learn to analyze GeneLab’s space-relevant omics data. The GL4U direct training pilot program was conducted in June 2021. During the pilot, students participated in a week-long bootcamp consisting of space biology-specific lectures and hands-on instruction using Jupyter Notebooks to analyze RNA sequence data. This pilot demonstrated the capacity of GL4U for training young scientists and encouraging data re-use. In June 2022, GL4U partnered with Jet Propulsion Laboratory’s (JPL) Planetary Protection Center of Excellence to conduct the indirect training pilot program by training educators at historically black colleges and universities (HBCUs) and minority serving institutions (MSIs). During the educator pilot, participants received materials, training, and will be provided the necessary compute resources to enable them to run the bootcamp at their home institutions or alternatively to adapt the content to implement within existing courses, thereby extending the reach of this initiative. The GL4U training program provides undergraduate students from underrepresented groups the opportunity to learn about NASA and Space Biology, and to enhance their career prospects by gaining hands-on experience analyzing omics data, a skillset that is highly applicable and marketable in the life sciences. Pre- and post-bootcamp surveys were completed by all participants and show the overwhelming success of the bootcamps.
NASA’s GeneLab project provides researchers open access to space-relevant experiment multi-omics data that can be mined to understand the effects of spaceflight on biological systems. To maximize the number of scientists who understand and utilize GeneLab data and data processing pipelines, GeneLab has created GeneLab for Colleges and Universities (GL4U). GL4U provides space biology-relevant training in bioinformatics to the next generation of scientists through direct and indirect approaches. The GeneLab team plans to host two annual data processing bootcamps, one for college-level students (direct) and one for college educators (indirect – training of trainers), in which participants learn to analyze GeneLab’s space-relevant omics data. The GL4U direct training pilot program was conducted in June 2021. During the pilot, students participated in a week-long bootcamp consisting of space biology-specific lectures and hands-on instruction using Jupyter Notebooks to analyze RNA sequence data. This pilot demonstrated the capacity of GL4U for training young scientists and encouraging data re-use. During the educator pilot, scheduled for June 2022, educators will receive materials and training to enable them to run the bootcamp at their home institutions or alternatively to adapt the content to implement within existing courses, thereby extending the reach of this initiative.
This presentation will demonstrate a case study of interdisciplinary research done in the Amazon Web Services (AWS) cloud platform, in addition to in the local machine. We conduct data analysis next to data by leveraging various cloud-based data in NASA Earthdata Cloud, which are distributed by different missions/NASA Distributed Active Archive Centers (DAACs), and cloud computing resources at NASA. For instance, we directly access multiple datasets stored in the AWS Simple Storage Service (S3) buckets using a Python Jupyter notebook through a JupyterHub interface hosted in AWS (without having to download data), and conduct data analysis next to data in the cloud. We will also show how to share the research results following Open Source policy. This case study characterizes the change in wildfire events in the western United States during the past 20 years. In particular, we focus on the wildfires in California in 2021, one of the most severe wildfire years occurring in the most recent 20 years in California. We will analyze the possible causes of wildfires, such as drought conditions and climate variability, and examine the impacts of wildfires on air quality and atmospheric composition, and on land cover. We will examine the data distributed by the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC), including aerosols and meteorological data from the NASA Modern-Era Retrospective analysis for Research and Applications version 2 (MERRA-2), precipitation from the Global Precipitation Measurement (GPM) and Global Precipitation Climate Project (GPCP), and aerosol index from Ozone Monitoring Instrument (OMI). We also utilize the data distributed by the Physical Oceanography (PO) DAAC, such as Sea Surface Temperature (SST) data from the Group for High Resolution Sea Surface Temperature (GHRSST), and the data distributed by Land Processes (LP) DAAC, such as Normalized Difference Vegetation Index (NDVI).
The NASA Modern-Era Retrospective analysis for Research and Applications Version 2 (MERRA-2) is atmospheric reanalysis data spanning 1980 to present. It has been produced by the NASA Global Modeling and Assimilation Office (GMAO) and is distributed by the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC). MERRA-2 data includes 100 collections of Earth system variables, mainly from the atmospheric model, such as aerosol fields and meteorological fields, radiation fields, and aerosol fields, guided by the assimilation of as many as six million observations every six hours. MERRA-2 has been one of the most popular datasets from NASA and is widely used in interdisciplinary research and applications, with increasing numbers of new users. For example, at least 7000 users accessed MERRA-2 data at GES DISC in the year 2021, ~1000 more users than in the year 2020. In this presentation, we will introduce the MERRA-2 datasets associated with aerosol and air quality studies and use a wildfire case study to demonstrate the data tools developed at GES DISC to analyze and visualize MERRA-2 data, such as Giovanni and the level 3 and level 4 subsetter, and Jupyter Python notebook. We will also update the status of cloud migration of the MERRA-2 data to Amazon Web Services (AWS).