Developing Resources for Users to Discover, Access, and Use GES DISC Data and Services in the Cloud
Explore the source record for details and available documents.
Binita KC↗
SEARCH · Engineering Papers
Results for “data discover”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
ESIP Information Quality Cluster (IQC)
The Information Quality Cluster (IQC) within the Federation of Earth Science Information Partners (ESIP) was initially formed in 2011 and has evolved significantly over time. The current objectives of the IQC are to: 1. Actively evaluate community data quality best practices and standards; 2. Improve capture, description, discovery, and usability of information about data quality in Earth science data products; 3. Ensure producers of data products are aware of standards and best practices for conveying data quality, and data providers distributors intermediaries establish, improve and evolve mechanisms to assist users in discovering and understanding data quality information; and 4. Consistently provide guidance to data managers and stewards on how best to implement data quality standards and best practices to ensure and improve maturity of their data products. The activities of the IQC include: 1. Identification of additional needs for consistently capturing, describing, and conveying quality information through use case studies with broad and diverse applications; 2. Establishing and providing community-wide guidance on roles and responsibilities of key players and stakeholders including users and management; 3. Prototyping of conveying quality information to users in a more consistent, transparent, and digestible manner; 4. Establishing a baseline of standards and best practices for data quality; 5. Evaluating recommendations from NASA's DQWG in a broader context and proposing possible implementations; and 6. Engaging data providers, data managers, and data user communities as resources to improve our standards and best practices. Following the principles of openness of the ESIP Federation, IQC invites all individuals interested in improving capture, description, discovery, and usability of information about data quality in Earth science data products to participate in its activities.
data products↗
The X-Ray Variable Sky
A novel approach to search for rapid time variability of every source, whether persistent or transient, within the entire ROSAT detector field has been developed. Semi-automated software statistically tests the photon arrival time event list for source variability and reports significant (greater than 5 sigma) variable enhancement in the photon count rate on time scales ranging from 5-500 seconds. The computational algorithms have been thoroughly tested, and are able to discriminate between false detections induced by instrumental effects, and real X-ray variable sources (XRVs); when carefully scrutinized by the PI. The analysis has proven to be successful, with both periodic and transient XRVs having been found. Some of these are entirely new discoveries, while others have been detected in previous X-ray missions, yet their nature remained elusive since source identification and analysis had yet to be performed. On average, one new (unidentified) XRV is found per ROSAT PSPC field analyzed. In some cases the XRVs show only a transient signature (a burst type event) with no persistent X-ray or other known (optical) counterpart. The ambitious goal of this project was to process as much of the ROSAT PSPC archive data as feasibly possible to discover XRVs and analyze this data.
Stringfellow, Guy↗
Monitoring of the tidal dynamics of the Dutch Waddensea by SIR-B
The potential of LANDSAT data, covering the entire tidal flats at a certain, known, tidal situation, was assessed. It was discovered that the data cannot be used for systematic survey because of the long interval between subsequent passes, weather conditions often interfere with recording, and of the lack of correlation between passes and the tidal situation. The objective is to overcome the problems by using: (1) the synoptic view obtained by SIR-B, which has the potential of surveying large areas of the flats simultaneously; (2) the all-weather capability of the microwave system; (3) the recording during consecutive days, which results in a straightforeward correlation with the tidal cycle and the picturing of different tidal stages; and (4) the multiangle incidence of SIR-B to analyze the bottom configuration of submerged parts of the flats. The use of a weather independent monitoring device, such as radar, an improvement in the monitoring technique of tidal coastal areas.
Koopmans, B. N.↗
Final technical report for DE-SC0022255: Discovering Physically Meaningful Structures from Climate Extreme Data
The past two decades have witnessed natural disasters and extreme weather events that affect millions of people. At the same time, the data volume from high-resolution climate models, satellite, in-situ and ground-based measurements have substantially increased to petabyte scales. These new and readily accessible datasets create the previously missing pipeline required for scientific machine learning (ML) and therefore new opportunities for improved understanding and prediction capability of climate extreme events. This project developed a deep latent variable model framework to discover physically meaningful hidden structures from high-dimensional, spatiotemporal climate extreme data.
97 MATHEMATICS AND COMPUTING↗
MODIS 3 Km Aerosol Product: Applications over Land in an Urban/suburban Region
MODerate resolution Imaging Spectroradiometer (MODIS) instruments aboard the Terra and Aqua satellites have provided a rich dataset of aerosol information at a 10 km spatial scale. Although originally intended for climate applications, the air quality community quickly became interested in using the MODIS aerosol data. However, 10 km resolution is not sufficient to resolve local scale aerosol features. With this in mind, MODIS Collection 6 is including a global aerosol product with a 3 km resolution. Here, we evaluate the 3 km product over the Baltimore/Washington D.C., USA, corridor during the summer of 2011, by comparing with spatially dense data collected as part of the DISCOVER-AQ campaign these data were measured by the NASA Langley Research Center airborne High Spectral Resolution Lidar (HSRL) and a network of 44 sun photometers (SP) spaced approximately 10 km apart. The HSRL instrument shows that AOD can vary by up to 0.2 within a single 10 km MODIS pixel, meaning that higher resolution satellite retrievals may help to characterize aerosol spatial distributions in this region. Different techniques for validating a high-resolution aerosol product against SP measurements are considered. Although the 10 km product is more statistically reliable than the 3 km product, the 3 km product still performs acceptably, with more than two-thirds of MODIS/SP collocations falling within the expected error envelope with high correlation (R > 0.90). The 3 km product can better resolve aerosol gradients and retrieve closer to clouds and shorelines than the 10 km product, but tends to show more significant noise especially in urban areas. This urban degradation is quantified using ancillary land cover data. Overall, we show that the MODIS 3 km product adds new information to the existing set of satellite derived aerosol products and validates well over the region, but due to noise and problems in urban areas, should be treated with some degree of caution.
MODIS↗
Faults Discovery By Using Mined Data
Fault discovery in the complex systems consist of model based reasoning, fault tree analysis, rule based inference methods, and other approaches. Model based reasoning builds models for the systems either by mathematic formulations or by experiment model. Fault Tree Analysis shows the possible causes of a system malfunction by enumerating the suspect components and their respective failure modes that may have induced the problem. The rule based inference build the model based on the expert knowledge. Those models and methods have one thing in common; they have presumed some prior-conditions. Complex systems often use fault trees to analyze the faults. Fault diagnosis, when error occurs, is performed by engineers and analysts performing extensive examination of all data gathered during the mission. International Space Station (ISS) control center operates on the data feedback from the system and decisions are made based on threshold values by using fault trees. Since those decision-making tasks are safety critical and must be done promptly, the engineers who manually analyze the data are facing time challenge. To automate this process, this paper present an approach that uses decision trees to discover fault from data in real-time and capture the contents of fault trees as the initial state of the trees.
Lee, Charles↗
Lifetimes and Occurrence Rates of Dark Vortices on Neptune from 25 Years of Hubble Space Telescope Images
We scoured the full set of blue-wavelength Hubble Space Telescope images of Neptune, finding one additional dark spot in new Hubble data beyond those discovered in 1989, 1994, 1996, and 2015. We report the complete disappearance of the SDS-2015 dark spot, using new Hubble data taken on 2018 September 9–10, as part of the Outer Planet Atmospheres Legacy (OPAL) program. Overall, dark spots in the full Hubble data set have lifetimes of at least one to two years, and no more than six years. We modeled a set of dark spots randomly distributed in time over the latitude range on Neptune that is visible from Earth, finding that the cadence of archival Hubble images would have detected about 70% of these spots if their lifetimes are only one year, or about 85%–95% of simulated spots with lifetimes of two or more years. Based on the Hubble data set, we conclude that dark spots have average occurrence rates of one dark spot every four to six years. Many numerical models to date have simulated much shorter vortex lifetimes, so our findings provide constraints that may lead to improved understanding of Neptune’s wind field, stratification, and humidity.
Hsu, Andrew I.↗
Examples of EOS Variables as compared to the UMM-Var Data Model
In effort to provide EOSDIS clients a way to discover and use variable data from different providers, a Unified Metadata Model for Variables is being created. This presentation gives an overview of the model and use cases we are handling.
CMR↗
MAGSAT anomaly profiles of the eastern Indian Ocean
Ground tracks from SEASAT were used in an effort to develop qualititative relationships between the gravity field and MAGSAT magnetic anomalies in the eastern Indian Ocean. Investigation of data quality led to analyses of the average value (over 80 vector data points, or approximately 36 km intervals) and of the standard deviation of this average, as a mean of identifying noisy portions of the data. It was discovered that the plots of the average value minus the individual (measured) point value are most useful for identifying noisy areas and data spikes. Spectrum analysis using edited (spikes removed) data show that the noise floor is less than 1 nT and the slope of the spectrum in the region of wavelengths between 1200 km and 250 km is approximately -3. Consequently the estimated resolution limit improved from approximately 360 km to approximately 250 km.
Sailor, R. V.↗
Parametric-Studies and Data-Plotting Modules for the SOAP
"Parametric Studies" and "Data Table Plot View" are the names of software modules in the Satellite Orbit Analysis Program (SOAP). Parametric Studies enables parameterization of as many as three satellite or ground-station attributes across a range of values and computes the average, minimum, and maximum of a specified metric, the revisit time, or 21 other functions at each point in the parameter space. This computation produces a one-, two-, or three-dimensional table of data representing statistical results across the parameter space. Inasmuch as the output of a parametric study in three dimensions can be a very large data set, visualization is a paramount means of discovering trends in the data (see figure). Data Table Plot View enables visualization of the data table created by Parametric Studies or by another data source: this module quickly generates a display of the data in the form of a rotatable three-dimensional-appearing plot, making it unnecessary to load the SOAP output data into a separate plotting program. The rotatable three-dimensionalappearing plot makes it easy to determine which points in the parameter space are most desirable. Both modules provide intuitive user interfaces for ease of use.
Source record↗
OMI/TROPOMI Data Support from GES DISC
The NASA GES DISC has been the official archive center for data from the Ozone Monitoring Instrument (OMI) aboard NASA's Aura mission since 2004. In recent years, the GES DISC has been evolving and improving its data management and services in order to promote its data to be easily discovered and accessed, as well as to facilitate data "interoperability". This presentation will summarize metrics collected of OMI data usage and GES DISC data services. In order to support the recently released Sentinel-5P/TROPOMI data from ESA at the NASA GES DISC, a new End User License Agreement (EULA) has been implemented for users accessing these data from the GES DISC.
Johnson, J.↗
Finding Your TEMPO: An Introduction to the Mission, Products, and Data Services for Air Quality Observations over North America
NASA's Tropospheric Emissions: Monitoring of Pollution (TEMPO) mission is the first space-based instrument to monitor major air pollutants across the North American continent every daylight hour at high spatial resolution. TEMPO is an ultraviolet and visible spectrometer that sits on a commercial satellite in a geostationary orbit about 22,000 miles above Earth's equator. This vantage point enables TEMPO to monitor daily variations in ozone, nitrogen dioxide, and other key elements of air pollution from the Atlantic to the Pacific, and from Mexico City and the Yucatan Peninsula to the Canadian oil sands. This webinar will provide an overview of the TEMPO mission and its data products and will show you how to discover and access TEMPO data products using NASA's Earthdata Search. This includes finding documentation, performing searches and filtering, using subsetting/concatenation services in Earthdata Search, and utilizing the Earthdata Forum.
Caroline Nowlan↗
In Situ Thermal Inspection of Automated Fiber Placement Operations for Tow and Ply Defect Detection
The advent of Automated Fiber Placement (AFP) systems have aided the rapid manufacturing of composite aerospace structures. One of the challenges that AFP systems pose is the uniformity of the deposited prepreg tape layers, which complicates detection of laps, gaps, overlaps and twists. The current detection method used in industry involves halting fabrication and performing a time consuming, visual inspection of each tape layer. Typical AFP systems use a quartz lamp to heat the base layer to make the surface tacky as it deposits another tape layer. The innovation proposed in this paper is to use the preheated base layer as a through-transmission heat source for inspecting the newly added tape layer in situ using a thermographic camera mounted on to the AFP hardware. Such a system would not only increase manufacturing throughput by reducing inspection times, but it would also aid in process development for new structural designs or material systems by providing data on as-built parts. To this end, a small thermal camera was mounted onto an AFP robotic research platform at NASA, and thermal data was collected during typical and experimental layup operations. The data was post processed to reveal defects such as tow overlap/gap, wrinkling, and peel-up. Defects that would have been impossible to detect visually were also discovered in the data, such as poor/loss of adhesion between plies and the effects of vacuum debulking. This paper will cover the results of our experiments, and the plans for future versions of this inspection system.
Juarez, Peter D.↗
Final Technical Report
This grant supported research into uses of the 'Hubble Space Telescope' Guide Star Selection System data base for important astrophysical problems ancillary to HST observations. The initial focus was to use the data to survey the frequency of wide binary stars at high galactic latitude, as a probe of local dark matter. In the course of this work, it was also discovered that the data base could be used for a purpose never envisaged by its designers, namely, detection of proper motions in faint stars. Some further evaluation of this capability supported by the grant led to the discovery of an entirely new class of stellar object, namely the dwarf carbon star. Publications supported by this grant, including both papers in refereed journals, and also two Ph.D. thesis dissertations, are described briefly.
Margon, Bruce↗
Finding Atmospheric Composition (AC) Metadata
The Atmospheric Composition Portal (ACP) is an aggregator and curator of information related to remotely sensed atmospheric composition data and analysis. It uses existing tools and technologies and, where needed, enhances those capabilities to provide interoperable access, tools, and contextual guidance for scientists and value-adding organizations using remotely sensed atmospheric composition data. The initial focus is on Essential Climate Variables identified by the Global Climate Observing System CH4, CO, CO2, NO2, O3, SO2 and aerosols. This poster addresses our efforts in building the ACP Data Table, an interface to help discover and understand remotely sensed data that are related to atmospheric composition science and applications. We harvested GCMD, CWIC, GEOSS metadata catalogs using machine to machine technologies - OpenSearch, Web Services. We also manually investigated the plethora of CEOS data providers portals and other catalogs where that data might be aggregated. This poster is our experience of the excellence, variety, and challenges we encountered.Conclusions:1.The significant benefits that the major catalogs provide are their machine to machine tools like OpenSearch and Web Services rather than any GUI usability improvements due to the large amount of data in their catalog.2.There is a trend at the large catalogs towards simulating small data provider portals through advanced services. 3.Populating metadata catalogs using ISO19115 is too complex for users to do in a consistent way, difficult to parse visually or with XML libraries, and too complex for Java XML binders like CASTOR.4.The ability to search for Ids first and then for data (GCMD and ECHO) is better for machine to machine operations rather than the timeouts experienced when returning the entire metadata entry at once. 5.Metadata harvest and export activities between the major catalogs has led to a significant amount of duplication. (This is currently being addressed) 6.Most (if not all) Earth science atmospheric composition data providers store a reference to their data at GCMD.
metadata search↗
What is LAADS DAAC? How to Download Data from NASA's Source for Cloud and Aerosol Data
The Level-1 and Atmosphere Archive Distribution System (LAADS) Distributed Active Archive Center (DAAC) primarily archives and distributes data on clouds and aerosols in Earth’s atmosphere in addition to providing Level 0 and 1 Moderate Imaging Radiometer Suite (MODIS), Visible Infrared Imaging Radiometer Suite (VIIRS) and airborne data as well as data from the European Space Agency’s Sea and Land Surface Temperature Radiometer (SLSTR) and the Ocean and Land Colour Instrument (OLCI). Secondarily, LAADS DAAC serves as a backup source for MODIS and VIIRS land products. As one of twelve DAACS supported by NASA’s Earth Science Data and Information System (ESDIS), LAADS DAAC provides a gateway to data through multiple tools. The newest, View Data, allows users to preview data sets before downloading or applying any post-processing transformations. Learn about how to access LAADS DAAC data, get up-to-date information about LAADS DAAC's migration to hosting data in the cloud, and discover how to use the View Data tool.
Tassia Owen↗
Discovering methylated DNA motifs in bacterial nanopore sequencing data with MIJAMP
Abstract Bacterial DNA methylation is involved in diverse cellular functions, including modulation of gene expression, DNA repair, and restriction–modification systems for defense against viruses and other foreign DNA. Restriction systems hinder efforts to engineer organisms to produce fuels and chemicals from waste and renewable feedstocks by degrading DNA during transformation. Methylome analysis allows identification of motifs within a bacterial chromosome that may be targeted by native restriction enzymes. Further expression of the corresponding methyltransferases in Escherichia coli allows plasmid DNA to be protected from restriction in the target organism, thereby drastically enhancing transformation efficiency. Nanopore sequencing can detect methylated bases, but software is needed to transform modified base coordinates into methylated motifs. Here, we develop MIJAMP (MIJAMP Is Just A MethylBED Parser), a software package that was developed to discover methylated motifs from the output of ONT’s Modkit or other data in the methylBED format. MIJAMP employs a human-driven refinement strategy that empirically validates all motifs against genome-wide methylation data, thus eliminating incorrect motifs. MIJAMP also reports methylation data on specific, user-defined motifs. Using MIJAMP, we determined the methylated motifs both in a control strain (wild-type E. coli) and in Synecococcus sp. strain PCC7002, laying the foundation for improved transformation in this organism. MIJAMP is available at https://code.ornl.gov/alexander-public/mijamp/. One Sentence Summary: Here we describe software written to discover DNA methylation motifs from nanopore sequencing data.
59 BASIC BIOLOGICAL SCIENCES↗