Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Time Series Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
NASA NTRS2022

Cloud Mask Intercomparison eXercise (CMIX): An evaluation of cloud masking algorithms for Landsat 8 and Sentinel-2

Cloud cover is a major limiting factor in exploiting time-series data acquired by optical spaceborne remote sensing sensors. Multiple methods have been developed to address the problem of cloud detection in satellite imagery and a number of cloud masking algorithms have been developed for optical sensors but very few studies have carried out quantitative intercomparison of state-of-the-art methods in this domain. This paper summarizes results of the first Cloud Masking Intercomparison eXercise (CMIX) conducted within the Committee Earth Observation Satellites (CEOS) Working Group on Calibration & Validation (WGCV). CEOS is the forum for space agency coordination and cooperation on Earth observations, with activities organized under working groups. CMIX, as one such activity, is an international collaborative effort aimed at intercomparing cloud detection algorithms for moderate-spatial resolution (10–30 m) spaceborne optical sensors. The focus of CMIX is on open and free imagery acquired by the Landsat 8 (NASA/USGS) and Sentinel-2 (ESA) missions. Ten algorithms developed by nine teams from fourteen different organizations representing universities, research centers and industry, as well as space agencies (CNES, ESA, DLR, and NASA), are evaluated within the CMIX. Those algorithms vary in their approach and concepts utilized which were based on various spectral properties, spatial and temporal features, as well as machine learning methods. Algorithm outputs are evaluated against existing reference cloud mask datasets. Those datasets vary in sampling methods, geographical distribution, sample unit (points, polygons, full image labels), and generation approaches (experts, machine learning, sky images). Overall, the performance of algorithms varied depending on the reference dataset, which can be attributed to differences in how the reference datasets were produced. The algorithms were in good agreement for thick cloud detection, which were opaque and had lower uncertainties in their identification, in contrast to thin/semi-transparent clouds detection. Not only did CMIX allow identification of strengths and weaknesses of existing algorithms and potential areas of improvements, but also the problems associated with the existing reference datasets. The paper concludes with recommendations on generating new reference datasets, metrics, and an analysis framework to be further exploited and additional input datasets to be considered by future CMIX activities.

Sergii Skakun↗
NASA NTRSDate not supplied

On Dynamical Impacts on Stratospheric Water Vapor Trends Above Boulder, Colorado

Long-term stratospheric water vapor (SWV) trends above Boulder, CO, (40°N, 255°E) are generally positive and slightly larger than those for zonal average SWV. This study finds a slight enhancement for SWV at 31.6 hPa above Boulder from middle February through March 1979, as seen in the satellite dataset from the LIMS instrument. That excess of SWV correlates well with air in the polar vortex that underwent an elongation and splitting in middle February following a final stratospheric warming event. Zonal wave-2 events tend to occur in late winter, but not every year. Such aperiodic variations in data time series of SWV may affect the fitting of its linear trend term.

Ellis Remsberg↗
NASA NTRSDate not supplied

On Dynamical Impacts on Stratospheric Water Vapor Trends Above Boulder, Colorado

Long-term stratospheric water vapor (SWV) trends above Boulder, CO, (40°N, 255°E) are generally positive and slightly larger than those for zonal average SWV. This study finds a slight enhancement for SWV at 31.6 hPa above Boulder from middle February through March 1979, as seen in the satellite dataset from the LIMS instrument. That excess of SWV correlates well with air in the polar vortex that underwent an elongation and splitting in middle February following a final stratospheric warming event. Zonal wave-2 events tend to occur in late winter, but not every year. Such aperiodic variations in data time series of SWV may affect the fitting of its linear trend term.

Ellis Remsberg↗
NASA NTRS2005

Development and Testing of Data Mining Algorithms for Earth Observation

The new algorithms developed under this project included a principled procedure for classification of objects, events or circumstances according to a target variable when a very large number of potential predictor variables is available but the number of cases that can be used for training a classifier is relatively small. These "high dimensional" problems require finding a minimal set of variables -called the Markov Blanket-- sufficient for predicting the value of the target variable. An algorithm, the Markov Blanket Fan Search, was developed, implemented and tested on both simulated and real data in conjunction with a graphical model classifier, which was also implemented. Another algorithm developed and implemented in TETRAD IV for time series elaborated on work by C. Granger and N. Swanson, which in turn exploited some of our earlier work. The algorithms in question learn a linear time series model from data. Given such a time series, the simultaneous residual covariances, after factoring out time dependencies, may provide information about causal processes that occur more rapidly than the time series representation allow, so called simultaneous or contemporaneous causal processes. Working with A. Monetta, a graduate student from Italy, we produced the correct statistics for estimating the contemporaneous causal structure from time series data using the TETRAD IV suite of algorithms. Two economists, David Bessler and Kevin Hoover, have independently published applications using TETRAD style algorithms to the same purpose. These implementations and algorithmic developments were separately used in two kinds of studies of climate data: Short time series of geographically proximate climate variables predicting agricultural effects in California, and longer duration climate measurements of temperature teleconnections.

Glymour, Clark↗
NASA NTRS2015

Application of Data Cubes for Improving Detection of Water Cycle Extreme Events

As part of an ongoing NASA-funded project to remove a longstanding barrier to accessing NASA data (i.e., accessing archived time-step array data as point-time series), for the hydrology and other point-time series-oriented communities, "data cubes" are created from which time series files (aka "data rods") are generated on-the-fly and made available as Web services from the Goddard Earth Sciences Data and Information Services Center (GES DISC). Data cubes are data as archived rearranged into spatio-temporal matrices, which allow for easy access to the data, both spatially and temporally. A data cube is a specific case of the general optimal strategy of reorganizing data to match the desired means of access. The gain from such reorganization is greater the larger the data set. As a use case of our project, we are leveraging existing software to explore the application of the data cubes concept to machine learning, for the purpose of detecting water cycle extreme events, a specific case of anomaly detection, requiring time series data. We investigate the use of support vector machines (SVM) for anomaly classification. We show an example of detection of water cycle extreme events, using data from the Tropical Rainfall Measuring Mission (TRMM).

water cycle extreme events↗
NASA NTRSDate not supplied

Building a Bilingual Google Earth Engine Dashboard to Increase Accessibility to Long-term Time Series Remote Sensing Data for Monitoring Saline System Changes in Chile’s Atacama Desert

Saline systems, consisting of salt flats, ponds, and marshes, provide vital water resources to wildlife and communities in northern Chile’s Atacama Desert, one of the driest regions in the world. Mining is extensive in the Atacama, which contains 30% of the world’s lithium reserves and is abundant in potassium and boron. The groundwater that feeds into salt marshes and ponds is extracted in large volumes for mining operations, limiting the availability of water for ecosystems. However, identifying long-term and large-scale environmental impacts from local lithium mining on the saline systems is limited by region inaccessibility and terrain variability. Open access satellite imagery and cloud computing technology has made studying Atacama saline systems feasible and allowed for collaboration across different agencies and countries. The NASA DEVELOP Program partnered with Chile’s la Universidad de La Serena and Servicio Nacional de Geología y Minería (SERNAGEOMIN) to create the Saline Analysis Tool (SalT) in Google Earth Engine (GEE). SalT is used to analyze the extent and distribution of remote saline systems in the Atacama from 1986 to the present day. The tool filters Landsat 5 Thematic Mapper (TM) and Landsat 8 Operational Land Imager (OLI) data from GEE’s data catalog and creates a single composite image per year for analysis. Additional output analyses include land cover classification, Normalized Difference Vegetation Index (NDVI) and Normalized Difference Water Index (NDWI) raster images that can be displayed on the map interface or exported. The tool can also generate time-lapse videos and charts displaying NDVI, NDWI, and land cover over time. A key feature of the tool is the use of a bilingual graphical user interface to make analysis accessible and customizable to different users’ needs—SalT provides options to select an analysis area, analysis time period, and outputs to display or export. The tool also incorporates new Earth observations as they are added to GEE’s catalog. The ability to easily visualize and analyze long-term remote sensing imagery will enable SERNAGEOMIN and la Universidad de la Serena to continually monitor changes in these saline systems and inform future land management policy.

NASA DEVELOP↗
NASA NTRS2003

Compendium of NASA Data Base for the Global Tropospheric Experiment's Transport and Chemical Evolution Over the Pacific (TRACE-P): DC-8 - Volume 1

This report provides a compendium of NASA aircraft data that are available from NASA's Global Tropospheric Experiment's (GTE) Transport and Chemical Evolution over the Pacific (TRACE-P) Mission. The broad goal of TRACE-P was to characterize the transit and evolution of the Asian outflow over the western Pacific. Conducted from February 24 through April 10, 2001, TRACE-P integrated airborne, satellite- and ground-based observations, as well as forecasts from aerosol and chemistry models. The format of this compendium utilizes data plots (time series) of selected data acquired aboard the NASA/Dryden DC-8 (vol. 1) and NASA/Wallops P-3B (vol. 2) aircraft during TRACE-P. The purpose of this document is to provide a representation of aircraft data that are available in archived format via NASA Langley s Distributed Active Archive Center (DAAC) and through the GTE Project Office archive. The data format is not intended to support original research/analyses, but to assist the reader in identifying data that are of interest.

Kleb, Mary M.↗
NASA NTRS2003

Compendium of NASA Data Base for the Global Tropospheric Experiment's Transport and Chemical Evolution Over the Pacific (TRACE-P): P-3B - Volume 2

This report provides a compendium of NASA aircraft data that are available from NASA's Global Tropospheric Experiment's (GTE) Transport and Chemical Evolution over the Pacific (TRACE-P) Mission. The broad goal of TRACE-P was to characterize the transit and evolution of the Asian outflow over the western Pacific. Conducted from February 24 through April 10, 2001, TRACE-P integrated airborne, satellite- and ground based observations, as well as forecasts from aerosol and chemistry models. The format of this compendium utilizes data plots (time series) of selected data acquired aboard the NASA/Dryden DC-8 (vol. 1) and NASA/Wallops P-3B (vol. 2) aircraft during TRACE-P. The purpose of this document is to provide a representation of aircraft data that are available in archived format via NASA Langley's Distributed Active Archive Center (DAAC) and through the GTE Project Office archive. The data format is not intended to support original research/analyses, but to assist the reader in identifying data that are of interest.

Kleb, Mary M.↗
NASA NTRS2015

Between a Map and a Data Rod

A Digital Divide has long stood between how NASA and other satellite-derived data are typically archived (time-step arrays or maps) and how hydrology and other point-time series oriented communities prefer to access those data. In essence, the desired method of data access is orthogonal to the way the data are archived. Our approach to bridging the Divide is part of a larger NASA-supported data rods project to enhance access to and use of NASA and other data by the Consortium of Universities for the Advancement of Hydrologic Science, Inc. (CUAHSI) Hydrologic Information System (HIS) and the larger hydrology community. Our main objective was to determine a way to reorganize data that is optimal for these communities. Two related objectives were to optimally reorganize data in a way that (1) is operational and fits in and leverages the existing Goddard Earth Sciences Data and Information Services Center (GES DISC) operational environment and (2) addresses the scaling up of data sets available as time series from those archived at the GES DISC to potentially include those from other Earth Observing System Data and Information System (EOSDIS) data archives. Through several prototype efforts and lessons learned, we arrived at a non-database solution that satisfied our objectivesconstraints. We describe, in this presentation, how we implemented the operational production of pre-generated data rods and, considering the tradeoffs between length of time series (or number of time steps), resources needed, and performance, how we implemented the operational production of on-the-fly (virtual) data rods. For the virtual data rods, we leveraged a number of existing resources, including the NASA Giovanni Cache and NetCDF Operators (NCO) and used data cubes processed in parallel. Our current benchmark performance for virtual generation of data rods is about a years worth of time series for hourly data (9,000 time steps) in 90 seconds. Our approach is a specific implementation of the general optimal strategy of reorganizing data to match the desired means of access. Results from our project have already significantly extended NASA data to the large and important hydrology user community that has been, heretofore, mostly unable to easily access and use NASA data.

machine learning↗
NASA NTRS2014

Enhancement of Mutual Discovery, Search, and Access of Data for Users of NASA and GEOSS-Cataloged Data Systems

An ongoing NASA-funded Data Rods (time series) project has demonstrated the removal of a longstanding barrier to accessing NASA data (i.e., accessing archived time-step array data as point-time series) for selected variables of the North American and Global Land Data Assimilation Systems (NLDAS and GLDAS, respectively) and other NASA data sets. Data rods are pre-generated or generated on-the-fly (OTF), leveraging the NASA Simple Subset Wizard (SSW), a gateway to NASA data centers. Data rods Web services are accessible through the CUAHSI Hydrologic Information System (HIS) and the Goddard Earth Sciences Data and Information Services Center (GES DISC) but are not easily discoverable by users of other non-NASA data systems. An ongoing GEOSS Water Services project aims to develop a distributed, global registry of water data, map, and modeling services cataloged using the standards and procedures of the Open Geospatial Consortium and the World Meteorological Organization. Preliminary work has shown GEOSS can be leveraged to help provide access to data rods. A new NASA-funded project is extending this early work.

data access↗
NASA NTRS1994

Consistency in long-term observations of oceans and ice from space

Satellite remote sensing has provided powerful tools for constructing long-term records of observations of many of Earth's geophysical processes. Geophysicists studying climage change would like to construct time series of data which are as long, accurate, and consistent as possible. While technological advances continue to improve the quality of observations from space, they also present consistency problems. The scanning multichannel microwave radiometer (SMMR), which operated from October 1978 to August 1987, and the special sensor microwave imager (SSMI), which overlapped with the SMMR for nearly two months in 1987, have provided microwave radiances which have been used to monitor long-term changes in sea ice concentration and snow accumulation, among other things. Inspection has shown, however, that brightness temperatures from the two sensors over the same polar firn-covered scene can differ by as much as 14 K. Calibration corrections for polar firn have been derived; in this paper, we present calculations addressing the physics of the small frequency and viewing angle differences between the SMMR and the SSMI on microwave emission from polar scenes containing free water. We focus here only on surface emission. We use a surface scattering model and study the effects of surface roughness, snow wetness, snow density, and young sea ice concentration. We also compare data from Antarctica with our modeled results. We find that for most scenes, surface roughness dominates over the effects of frequency and viewing angle differences, but that for open long-term times series, data must be calibrated and constructed at the geophysical product level rather than at the level of measured radiances.

Zabel, I. H. H.↗
NASA NTRS2016

Nasa Space Flight Instruments: Cost Time Trends

Are NASA’s space flight instruments becoming cheaper or more expensive as time marches forward? After analyzing the costs of hundreds of instruments launched over the last 30 years, the short answer to this question is no… and yes. This paper gives a visual analysis of the cost time trends for various NASA space flight instrument types, such as optical, particles detectors, fields detectors and microwave instruments. In addition to the statistical approaches utilized, such as significance tests, cluster analysis and principle components analysis (PCA), we will also discuss the intangibles which are likely at play, including technological progress, NASA policy and the luck of the draw associated with mission manifests. This analysis was performed as the main driver for the NASA Instrument Cost Model (NICM) recent cost estimating model redesign. Started in 2004, the first version of NICM was based off of instruments launched from 1985-2005, or 20 years’ worth of data. As NICM hit its 10-year anniversary, we wanted to know: should NICM continue to only use the most recent 20 years’ worth of data (1995-2015)? Are instruments becoming cheaper or more expensive as time marches forward? There is evidence in favor of a drop in the median dollar-per-kg value across some instrument types, but little in others. Whereas further research is needed to substantiate, Particles and Optical-Planetary instrument types show moderate to strong evidence of a downward trend in dollar-per-kg. Further research is required to study the nature of this trend (shift, taper, cyclic, etc.). Little evidence for a similar downward trend was detected for Fields or Microwave instruments, or Optical instruments on Earth Orbiting spacecraft. We presented evidence in favor of a drop in the median dollar-per-kg value for Particles and Optical-Planetary instrument types. While similar evidence was weak at best for Fields and Microwave instruments. We can speculate as to the causes for this effect, but we are also equipped to begin to rule out, or at least prioritize, some of the suspected drivers. We observed, for Particles and Optical-Planetary instruments, that perhaps a launch manifest effect was playing part of the role in the observed decrease in dollar-per-kg over the years, noting that the more flagship class missions, which have more money to spend on their instruments, were seen in the earlier years in our data, versus the later years which were dominated by less expensive class missions. However, if this were a dominating driver, would we not have seen the downward trend in the Fields and Microwave instruments as well, which were drawn from that same launch manifest? The fact that we did not observe this helps us rule out the launch manifest effect, and other drivers, such as advances in technology, that seem to be more likely suspects. In that case, however, why would technology advances be helping the Particles and Optical-Planetary instruments only? Why would it not be impacting Optical-Earth Orbiting instruments? Further suspects were looked at as well and ruled out, such as the “Faster, Better, Cheaper”era of NASA development which did not seem to actually impact trends by instrument type on a dollar-perkg scale. VI. Future Work A. Time Series Detailed Statistical Assessment The analysis discussed above sets the foundation for a more rigorous time series analysis of the data. Time series analysis will further explore evidence to-date of time trends for the instrument types which showed the strongest indicators for a decrease in dollar-per-kg: Optical (Planetary) and Particles instruments. More than providing evidence and top-level significance tests, time series analysis would help elucidate what kind of trend that exists in the data, their significance and allow statistically based forecasting (see Figure 10)

Mrozinksi, Joseph↗
NASA NTRS1996

Statistical Challenges in Modern Astronomy II

Wavelets are the central idea of a broad framework for thinking about displaying, and analyzing data - not just a new specialized technique. They are localized basis functions that provide an extremely flexible and efficient representation of time series. Current developments in the theory and application of wavelets are yielding powerful multiresolution techniques for the analysis of time series and image data. Wavelet methods can effect noise removal. characterization of stochastic behavior, depiction of power spectrum time-variations, and data compression - for time series data, image data, and data of higher dimensions. I review published applications of wavelets in astronomy.

Scargle, Jeffrey D.↗
NASA NTRS2008

Post-Flight Estimation of Motion of Space Structures: Part 1

A computer program estimates the relative positions and orientations of two space structures from data on the angular positions and distances of fiducial objects on one structure as measured by a target tracking electronic camera and laser range finders on another structure. The program is written specifically for determining the relative alignments of two antennas, connected by a long truss, deployed in outer space from a space shuttle. The program is based partly on transformations among the various coordinate systems involved in the measurements and on a nonlinear mathematical model of vibrations of the truss. The program implements a Kalman filter that blends the measurement data with data from the model. Using time series of measurement data from the tracking camera and range finders, the program generates time series of data on the relative position and orientation of the antennas. A similar program described in a prior NASA Tech Briefs article was used onboard for monitoring the structures during flight. The present program is more precise and designed for use on Earth in post-flight processing of the measurement data to enable correction, for antenna motions, of scientific data acquired by use of the antennas.

Brugarolas, Paul↗
NASA NTRS2004

Vegetation Effects on Soil Moisture Estimation

Several successful algorithms have been developed to estimate soil moisture of bare surfaces. We previously reported a new algorithm using the tilted Bragg approximation. However, these algorithms are only applicable to bare surfaces. When vegetation is present, soil moisture is typically underestimated by bare surface algorithms. In order to derive soil moisture under vegetation, we have to understand the complex scattering process due to vegetation. Our main interest is to retrieve the global soil moisture information using Hydros L-band polarimetric radar data. The Hydros mission will provide the first global view of land soil moisture using L-band radar and radiometer. The unique characteristics of the Hydros data are the availability of the low resolution soil moisture information from radiometer data and the continuous time series radar data collected at the same incidence angle. In this paper, we will examine a potential inversion algorithm to retrieve soil moisture under vegetation canopies using Hydros L-band polarimetric radar data.

soil moisture↗