Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data science”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,459 records · Page 81

Characterizing Aerosols over Southeast Asia using the AERONET Data Synergy Tool

Biomass burning, urban pollution and dust aerosols have significant impacts on the radiative forcing of the atmosphere over Asia. In order to better quanti@ these aerosol characteristics, the Aerosol Robotic Network (AERONET) has established over 200 sites worldwide with an emphasis in recent years on the Asian continent - specifically Southeast Asia. A total of approximately 15 AERONET sun photometer instruments have been deployed to China, India, Pakistan, Thailand, and Vietnam. Sun photometer spectral aerosol optical depth measurements as well as microphysical and optical aerosol retrievals over Southeast Asia will be analyzed and discussed with supporting ground-based instrument, satellite, and model data sets, which are freely available via the AERONET Data Synergy tool at the AERONET web site (http://aeronet.gsfc.nasa.gov). This web-based data tool provides access to groundbased (AERONET and MPLNET), satellite (MODIS, SeaWiFS, TOMS, and OMI) and model (GOCART and back trajectory analyses) databases via one web portal. Future development of the AERONET Data Synergy Tool will include the expansion of current data sets as well as the implementation of other Earth Science data sets pertinent to advancing aerosol research.

Giles, David M.↗

The NASA Soil Moisture Active Passive (SMAP) Mission - Algorithm and Cal/Val Activities and Synergies with SMOS and Other L-Band Missions

NASA's Soil Moisture Active Passive (SMAP) mission, planned for launch in late 2014, has as its key measurement objective the frequent, global mapping of near-surface soil moisture and its freeze-thaw state. SMAP soil moisture and freeze/thaw measurements at 10 km and 3 km resolutions respectively, would enable significantly improved estimates of water, energy and carbon transfers between the land and atmosphere. Soil moisture control of these fluxes is a key factor in the performance of atmospheric models used for weather forecasts and climate projections Soil moisture measurements are also of great importance in assessing floods and for monitoring drought. In addition, observations of soil moisture and freeze/thaw timing over the boreal latitudes can help reduce uncertainties in quantifying the global carbon balance. The SMAP measurement concept utilizes an L-band radar and radiometer sharing a rotating 6-meter mesh reflector antenna. The SMAP radiometer and radar flight hardware and ground processing designs are incorporating approaches to identify and mitigate potential terrestrial radio frequency interference (RFI). The radar and radiometer instruments are planned to operate in a 680 km polar orbit, viewing the surface at a constant 40-degree incidence angle with a 1000-km swath width, providing 3-day global coverage. Data from the instruments would yield global maps of soil moisture and freeze/thaw state to be provided at 10 km and 3 km resolutions respectively, every two to three days. Plans are to provide also a radiometer-only soil moisture product at 40-km spatial resolution. This product and the underlying brightness temperatures have characteristics similar to those provided by the Soil Moisture and Ocean Salinity (SMOS) mission. As a result, there are unique opportunities for common data product development and continuity between the two missions. SMAP also has commonalities with other satellite missions having L-band radiometer and/or radar sensors applicable to soil moisture measurement, such as Aquarius, SAO COM, and ALOS-2. The algorithms and data products for SMAP are being developed in the SMAP Science Data System (SDS) Testbed. The algorithms are developed and evaluated in the SDS Testbed using simulated SMAP observations as well as observational data from current airborne and spaceborne L-band sensors including SMOS. The SMAP project is developing a Calibration and Validation (Cal/Val) Plan that is designed to support algorithm development (pre-launch) and data product validation (post-launch). A key component of the Cal/Val Plan is the identification, characterization, and instrumentation of sites that can be used to calibrate and validate the sensor data (Level I) and derived geophysical products (Level 2 and higher). In this presentation we report on the development status of the SMAP data product algorithms, and the planning and implementation of the SMAP Cal/Val program. Several components of the SMAP algorithm development and Cal/Val plans have commonality with those of SMOS, and for this reason there are shared activities and resources that can be utilized between the missions, including in situ networks, ancillary data sets, and long-term monitoring sites.

Njoku, Eni↗

Temperature and Humidity Profiles in the TqJoint Data Group of AIRS Version 6 Product for the Climate Model Evaluation

The Atmospheric Infrared Sounder (AIRS) mission is entering its 13th year of global observations of the atmospheric state, including temperature and humidity profiles, outgoing long-wave radiation, cloud properties, and trace gases. Thus AIRS data have been widely used, among other things, for short-term climate research and observational component for model evaluation. One instance is the fifth phase of the Coupled Model Intercomparison Project (CMIP5) which uses AIRS version 5 data in the climate model evaluation. The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is the home of processing, archiving, and distribution services for data from the AIRS mission. The GES DISC, in collaboration with the AIRS Project, released data from the version 6 algorithm in early 2013. The new algorithm represents a significant improvement over previous versions in terms of greater stability, yield, and quality of products. The ongoing Earth System Grid for next generation climate model research project, a collaborative effort of GES DISC and NASA JPL, will bring temperature and humidity profiles from AIRS version 6. The AIRS version 6 product adds a new "TqJoint" data group, which contains data for a common set of observations across water vapor and temperature at all atmospheric levels and is suitable for climate process studies. How different may the monthly temperature and humidity profiles in "TqJoint" group be from the "Standard" group where temperature and water vapor are not always valid at the same time? This study aims to answer the question by comprehensively comparing the temperature and humidity profiles from the "TqJoint" group and the "Standard" group. The comparison includes mean differences at different levels globally and over land and ocean. We are also working on examining the sampling differences between the "TqJoint" and "Standard" group using MERRA data.

climate model evaluation↗

Analyzing and Visualizing Precipitation and Soil Moisture in ArcGIS

Precipitation and soil moisture are among the most important parameters in many land GIS (Geographic Information System) research and applications. These data are available globally from NASA GES DISC (Goddard Earth Science Data and Information Services Center) in GIS-ready format at 10-kilometer spatial resolution and 24-hour or less temporal resolutions. In this presentation, well demonstrate how rainfall and soil moisture data are used in ArcGIS to analyze and visualize spatiotemporal patterns of droughts and their impacts on natural vegetation and agriculture in different parts of the world.

precipitation↗

Evolution in Metadata Quality: Common Metadata Repository's Role in NASA Curation Efforts

Metadata Quality is one of the chief drivers of discovery and use of NASA EOSDIS (Earth Observing System Data and Information System) data. Issues with metadata such as lack of completeness, inconsistency, and use of legacy terms directly hinder data use. As the central metadata repository for NASA Earth Science data, the Common Metadata Repository (CMR) has a responsibility to its users to ensure the quality of CMR search results. This poster covers how we use humanizers, a technique for dealing with the symptoms of metadata issues, as well as our plans for future metadata validation enhancements. The CMR currently indexes 35K collections and 300M granules.

metadata quality↗

UUI: Reusable Spatial Data Services in Unified User Interface at NASA GES DISC

Unified User Interface (UUI) is a next-generation operational data access tool that has been developed at Goddard Earth Sciences Data and Information Services Center(GES DISC) to provide a simple, unified, and intuitive one-stop shop experience for the key data services available at GES DISC, including subsetting (Simple Subset Wizard -SSW), granule file search (Mirador), plotting (Giovanni), and other legacy spatial data services. UUI has been built based on a flexible infrastructure of reusable web services self-contained building blocks that can easily be plugged into spatial applications, including third-party clients or services, to easily enable new functionality as new datasets and services become available. In this presentation, we will discuss our experience in designing UUI services based on open industry standards. We will also explain how the resulting framework can be used for a rapid development, deployment, and integration of spatial data services, facilitating efficient access and dissemination of spatial data sets.

EOSDIS↗

Exploring and Analyzing Climate Variations Online by Using NASA MERRA-2 Data at GES DISC

NASA Giovanni (Goddard Interactive Online Visualization ANd aNalysis Infrastructure) (http:giovanni.sci.gsfc.nasa.govgiovanni) is a web-based data visualization and analysis system developed by the Goddard Earth Sciences Data and Information Services Center (GES DISC). Current data analysis functions include Lat-Lon map, time series, scatter plot, correlation map, difference, cross-section, vertical profile, and animation etc. The system enables basic statistical analysis and comparisons of multiple variables. This web-based tool facilitates data discovery, exploration and analysis of large amount of global and regional remote sensing and model data sets from a number of NASA data centers. Long term global assimilated atmospheric, land, and ocean data have been integrated into the system that enables quick exploration and analysis of climate data without downloading, preprocessing, and learning data. Example data include climate reanalysis data from NASA Modern-Era Retrospective analysis for Research and Applications, Version 2 (MERRA-2) which provides data beginning in 1980 to present; land data from NASA Global Land Data Assimilation System (GLDAS), which assimilates data from 1948 to 2012; as well as ocean biological data from NASA Ocean Biogeochemical Model (NOBM), which provides data from 1998 to 2012. This presentation, using surface air temperature, precipitation, ozone, and aerosol, etc. from MERRA-2, demonstrates climate variation analysis with Giovanni at selected regions.

knowledge base↗

Astro-H Data Analysis, Processing and Archive

Astro-H (Hitomi) is an X-ray Gamma-ray mission led by Japan with international participation, launched on February 17, 2016. The payload consists of four different instruments (SXS, SXI, HXI and SGD) that operate simultaneously to cover the energy range from 0.3 keV up to 600 keV. This paper presents the analysis software and the data processing pipeline created to calibrate and analyze the Hitomi science data along with the plan for the archive and user support.These activities have been a collaborative effort shared between scientists and software engineers working in several institutes in Japan and USA.

Archive↗

Astro-H/Hitomi Data Analysis, Processing, and Archive

Astro-H is the x-ray/gamma-ray mission led by Japan with international participation, launched on February 17, 2016. Soon after launch, Astro-H was renamed Hitomi. The payload consists of four different instruments (SXS, SXI, HXI, and SGD) that operate simultaneously to cover the energy range from 0.3 keV up to 600 keV. On March 27, 2016, JAXA lost contact with the satellite and, on April 28, they announced the cessation of the efforts to restore mission operations. Hitomi collected about one months worth of data with its instruments. This paper presents the analysis software and the data processing pipeline created to calibrate and analyze the Hitomi science data, along with the plan for the archive. These activities have been a collaborative effort shared between scientists and software engineers working in several institutes in Japan and United States.

Angelini, Lorella↗

Simplifying Satellite and Ground Data Validation with Level-2 Subsetting

We demonstrate that scientists can simplify their satellite data validation workflow with the use of NASA Godddard Earth Sciences Data and Information Services Center (GES DISC) subsetting services. We perform a sample validation of Aura ozone products collocated with ground-based ozone measurements using subsetting services to trim satellite data to only the relevant user-defined variables and spatio-temporal region. Because the subsetting service automatically returns only relevant data granules that adhere to a set of user-defined coincidence criteria, user workload is greatly reduced. Moreover, the resultant data files are substantially smaller than full data granules due to the subsetting service further culling the data to the relevant geospatio-temporal coincidence criteria, user-defined variables, and user-defined dimensions of variables. This decreases data download throughput and file storage requirements. The validation presented here quantifies the time and file size savings that can be achieved by utilizing subsetting services within the satellite data validation workflow.

Johnson, James↗

Arm DevSummit Keynote: Environment for Data Engineering in Virtual Reality: Ethical Considerations

Currently, the US Government is going through a large-scale data Transformation effort, where the GSA playbook is guiding all agencies to make their processes data-driven with the help of emerging technologies. To address concerns about algorithm sharing, AI adoption, and vendor lock-in, the NASA Langley Research Center Digital Transformation Group has developed the Environment for Data Engineering in Virtual Reality (EnDEVR), a data science ecosystem that allows users to command and investigate customizable data analyses from a VR environment. The system has been evaluated in the Oculus Rift S and Quest environments, two popular VR systems powered by the ARM architecture. Current and future development will employ several AI capabilities to guide research and automation within the environment and to facilitate algorithm sharing. In this talk, we will discuss the results of an initial ethical investigation and recommended considerations for the use of AI within the system.

Artificial Intelligence↗

pyQuARC: Open Source Library for Earth Observation Metadata Quality Assessment

Metadata quality is essential to effective data discovery and has become increasingly vital as more Earth Science data sets become available. The Common Metadata Repository (CMR) hosts metadata describing NASA’s Earth Observation data products, which are archived across 12 Distributed Active Archive Centers (DAACs). The Analysis and Review of CMR (ARC) Team, located at Marshall Space Flight Center, conducts metadata quality assessments to ensure that these data products are discoverable, accessible, and usable. To achieve these goals, the ARC team has developed a metadata quality assessment framework to evaluate metadata completeness, correctness, and consistency. ARC uses a combination of manual and automated methods to assess these three components and identify areas of improvement; the team then collaborates with the DAACs to resolve any findings. To streamline this process, ARC is currently developing a host of scripts, known as pyQuARC, to automate metadata quality assessments as much as possible. pyQuARC is an open source library for Earth Observation Metadata Quality Assessment, and the tool utilizes ARC’s metadata quality assessment framework to make basic validation checks, pinpoint inconsistencies between dataset-level (i.e. collection) and file-level (i.e. granule) metadata, and identify opportunities for more descriptive and robust information. Since pyQuARC is also customizable, other users can make modifications as needed, and future metadata standards can also be implemented. Once pyQuARC is fully developed, it will support multiple schema types to serve the broader EOSDIS metadata community. This presentation will provide an overview of pyQuARC and its process of development while showcasing the tool’s valuable features and uses.

Jenny Wood↗

Squeezing Every Last 'Bit' of Information from Enceladus Mass Spectrometry

Potential opportunities to return to Enceladus in Discovery and Flagship class missions inspire development of next-generation instruments and creative approaches to sample collection, sample analysis, and data analysis and transmission strategies. Mass spectrometers (MS) are ideally suited to future Enceladus missions due to their analytical power in identifying a range of molecular and ionic compositions – including complex organics – and potentially astrobiologically-important features such as isotope ratios, chirality, and enantiomeric excess. However, long communication delays from Enceladus and limited bandwidth limits the data transmission from these higher-data-volume instruments, likely delaying mission-related response to new data. We explore the utility of data science and machine learning (ML) on isotope ratio (IR)MS data collected from laboratory analogs of Enceladus to: 1) process data quickly for rapid ground-based analyses, 2) understand if compositional and biosignature information could be extracted from IRMS data, and 3) evaluate whether onboard ML techniques could improve sample analysis, cadence, and transmission prioritization. Laboratory analogs analyzed isotopes of volatile CO2 that interacted with seawaters of varying composition, and include both abiotic and biotic (microbially-influenced) experiments. Enceladus’s alkaline oceans promote speciation of carbon into multiple forms (e.g., H2CO3 / CO2, HCO3-, and CO32-), each of which could be isotopically fractionated by abiotic or biotic reactions. Large (>2‰) changes in carbon isotopes (δ13C) are observed from some biotic experiments inoculated with complex microbial ecosystems relative to the abiotic seawaters. ML training and classification suggests that microbial samples can be distinguished from abiotic samples, yet that a broad range of microbial experiments are necessary to train ML models to cover a range of complexities including disequilibria, and isotopic and compositional fractionation.

geochemistry↗

Optimizing a Small RNAseq Analysis Pipeline for NASA GeneLab Using Open-Source Tools and Libraries

Small RNA sequencing (small RNAseq) is a powerful tool for studying the regulation of gene expression in various organisms. Small RNAseq has been leveraged in space biology research to study how expression of small RNAs, e.g. micro RNAs (miRNAs), small interfering RNAs (siRNAs), and piwi-interacting RNAs (piRNAs), change upon exposure to the space environment. NASA GeneLab currently hosts small RNAseq raw data derived from space-relevant experiments on the Open Science Data Repository (OSDR). To maximize the accessibility of these data to the scientific community, in addition to hosting raw data, which is only interpretable by bioinformaticians, GeneLab plans to process all small RNAseq datasets and make those processed data available to the scientific community via the OSDR. In this study, we present the development of the GeneLab standardized pipeline for processing small RNAseq datasets. Using human, plant, and synthetic small RNAseq datasets, we interrogate various open-source software and publicly available databases to evaluate their accuracy and reproducibility in each step of the pipeline. For quality control and adapter detection and trimming, we evaluated TrimGalore!, FASTX, SeqKit, and DNApi methods to optimize alignment to reference genomes. We compared BWA, Bowtie, and Bowtie2 to determine the optimal alignment tool. For each alignment tool we also assessed various reference databases, including Ensembl reference genomes and different types of small RNA reference databases, including genome, hairpin, and miRNA references from the miRbase and MirGeneDB databases. To quantify the aligned data, we compared SAMtools, HTSeq, and RSEM for counting alignment events from each alignment tool used. Finally, we evaluated various tools, including DESeq2 and EdgeR, for data normalization and subsequent differential expression analysis. We will present the results from our comparative analyses for each pipeline step and propose a consensus pipeline for processing small RNAseq data derived from various organisms exposed to the space environment.

SmallRNAseq, NASA GeneLab, quality control, adapte↗

GLAST (FERMI) Data-Processing Pipeline

The Data Processing Pipeline ('Pipeline') has been developed for the Gamma-Ray Large Area Space Telescope (GLAST) which launched June 11, 2008. It generically processes graphs of dependent tasks, maintaining a full record of its state, history and data products. The Pipeline is used to automatically process the data down-linked from the satellite and to deliver science products to the GLAST collaboration and the Science Support Center and has been in continuous use since launch with great success. The pipeline handles up to 2000 concurrent jobs and in reconstructing science data produces approximately 750GB of data products using 1/2 CPU-year of processing time per day.

99 GENERAL AND MISCELLANEOUS↗

AIRS-Only Product in Giovanni for Exploring Up-to-Date AIRS Observation and Comparing with AIRS+AMSU Product

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) has been the home of processing, archiving, and distribution services for the Atmospheric Infrared Sounder (AIRS) mission since its launch in 2002 for global observations of the atmospheric state. Giovanni, a Web-based application developed by the GES DISC, provides a simple and intuitive way to visualize, analyze, and access vast amounts of Earth science remote sensing data without having to download the data. Most important AIRS variables, including temperature and humidity profiles, outgoing longwave radiation, cloud properties, and trace gases, are available in Giovanni. AIRS is an instrument suite comprised of a hyperspectral infrared instrument (AIRS) and two multichannel microwave instruments, the Advanced Microwave Sounding Unit (AMSU) and the Humidity Sounder for Brazil (HSB). As HSB ceased operation in the very early stages of the AIRS mission, the AIRS project operates two parallel retrieval algorithms: one using both IR and MW measurements (AIRS+AMSU) and the other using only IR measurements (AIRS-only), which covers most of the mission duration. The AIRS+AMSU product is better quality, and the variables in Giovanni are from this product. However, generation of the AIRS+AMSU product has been suspended since the AMSU instrument anomaly occurred in late September 2016. To continue exploring up-to-date AIRS observations, the same set of variables from the AIRS-only product have been added to Giovanni by the GES DSIC. This will also support comparison of AIRS-only with AIRS+AMSU retrievals. In this presentation, we demonstrate the visualization of the AIRS-only product and plots/statistics of comparison with AIRS+AMSU product using Giovanni.

AIRS↗

Prospects of federated machine learning in fluid dynamics

Physics-based models have been mainstream in fluid dynamics for developing predictive models. In recent years, machine learning has offered a renaissance to the fluid community due to the rapid developments in data science, processing units, neural network based technologies, and sensor adaptations. So far in many applications in fluid dynamics, machine learning approaches have been mostly focused on a standard process that requires centralizing the training data on a designated machine or in a data center. In this article, we present a federated machine learning approach that enables localized clients to collaboratively learn an aggregated and shared predictive model while keeping all the training data on each edge device. We demonstrate the feasibility and prospects of such a decentralized learning approach with an effort to forge a deep learning surrogate model for reconstructing spatiotemporal fields. Our results indicate that federated machine learning might be a viable tool for designing highly accurate predictive decentralized digital twins relevant to fluid dynamics.

36 MATERIALS SCIENCE↗

Distributed Active Archive Center

The Goddard Space Flight Center Version 0 Distributed Active Archive Center (GSFC V0 DAAC) is being developed to enhance and improve scientific research and productivity by consolidating access to remote sensor earth science data in the pre-EOS time frame. In cooperation with scientists from the science labs at GSFC, other NASA facilities, universities, and other government agencies, the DAAC will support data acquisition, validation, archive and distribution. The DAAC is being developed in response to EOSDIS Project Functional Requirements as well as from requirements originating from individual science projects such as SeaWiFS, Meteor3/TOMS2, AVHRR Pathfinder, TOVS Pathfinder, and UARS. The GSFC V0 DAAC has begun operational support for the AVHRR Pathfinder (as of April, 1993), TOVS Pathfinder (as of July, 1993) and the UARS (September, 1993) Projects, and is preparing to provide operational support for SeaWiFS (August, 1994) data. The GSFC V0 DAAC has also incorporated the existing data, services, and functionality of the DAAC/Climate, DAAC/Land, and the Coastal Zone Color Scanner (CZCS) Systems.

Bodden, Lee↗