Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data migration”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
NASA NTRS1998

Technology Assessment of High Capacity Data Storage Systems: Can We Avoid a Data Survivability Crisis?

This technology assessment of long-term high capacity data storage systems identifies an emerging crisis of severe proportions related to preserving important historical data in science, healthcare, manufacturing, finance and other fields. For the last 50 years, the information revolution, which has engulfed all major institutions of modem society, centered itself on data-their collection, storage, retrieval, transmission, analysis and presentation. The transformation of long term historical data records into information concepts, according to Drucker, is the next stage in this revolution towards building the new information based scientific and business foundations. For this to occur, data survivability, reliability and evolvability of long term storage media and systems pose formidable technological challenges. Unlike the Y2K problem, where the clock is ticking and a crisis is set to go off at a specific time, large capacity data storage repositories face a crisis similar to the social security system in that the seriousness of the problem emerges after a decade or two. The essence of the storage crisis is as follows: since it could take a decade to migrate a peta-byte of data to a new media for preservation, and the life expectancy of the storage media itself is only a decade, then it may not be possible to complete the transfer before an irrecoverable data loss occurs. Over the last two decades, a number of anecdotal crises have occurred where vital scientific and business data were lost or would have been lost if not for major expenditures of resources and funds to save this data, much like what is happening today to solve the Y2K problem. A pr-ime example was the joint NASA/NSF/NOAA effort to rescue eight years worth of TOVS/AVHRR data from an obsolete system, which otherwise would have not resulted in the valuable 20-year long satellite record of global warming. Current storage systems solutions to long-term data survivability rest on scalable architectures having parallel paths for data migration.

Halem, M.↗
NASA NTRSDate not supplied

The Future of NASA Earth Science in the Commercial Cloud: Challenges and Opportunities

NASA produces a large volume and variety of data products that are used every day to support research, decision making, and education. The widespread use of NASA’s Earth Science data is enabled by NASA’s Earth Science Data System (ESDS) program, which oversees the archiving and distribution of these data and invests in the development of new data systems and tools. However, NASA’s current approach to Earth Science data distribution — based on distributed institutional archives with individual on-premises high-performance computing capabilities — faces some significant challenges, including massive increases in data volume from upcoming missions, a greater need for transdisciplinary science that synthesizes many different kinds of observations, and a push to make science more open, inclusive, and accessible. To address these challenges, NASA is aggressively migrating its Earth Science data and related tools and services into the commercial cloud. Migration of data into the commercial cloud can significantly improve NASA’s existing data system capabilities by (1) providing more flexible options for storage and compute (including rapid, as-needed access to state-of-the-art capabilities); (2) by centralizing and standardizing data access, which gives all of NASA’s institutional data centers access to all of each other’s datasets; and (3) by facilitating “analysis-in-place”, whereby users can bring their own computational workflows and tools to the data rather than having to maintain their own copies of NASA datasets. However, migration to the commercial cloud also poses some significant challenges, including (1) managing costs under a “pay-as-you-go” model; (2) incompatibility with existing tools and data formats with object-based storage and network access; (3) vendor lock-in; (4) challenges with data access for workflows that mix on-premise and cloud computing; and (5) standardization for highly diverse data as is present in NASA’s data archive. I conclude with two examples of recent NASA activities showcasing capabilities enabled by the commercial cloud: An interactive analysis and development platform for analyzing airborne imaging spectroscopy data, and a new collection of tools and services for data discovery, analysis, publication, and data-driven storytelling (Visualization, Exploration, and Data Analysis, VEDA).

Alexey N Shiklomanov↗
NASA NTRS2019

Diurnally Migrating Lunar Water: Evidence from Ultraviolet Data

Data from the Lunar Reconnaissance Orbiter Lyman Alpha Mapping Project and Diviner are consistent with surface water on the Moon varying in abundance with both terrain type and local time/temperature. A thermal desorption model including latitudinally varying desorption activation energy reproduces the observations. We interpret the observed variability in spectral slopes as water molecules in the uppermost lunar regolith (<1% of a monolayer) thermally adsorbing and desorbing from grains depending upon the local temperature and availability of chemisorption sites. The Lyman Alpha Mapping Project data also demonstrate that in the Earth's magnetotail, where the solar wind source of protons is absent, a decrease in H2O on the surface is not observed. This rules out a steady state process involving aprompt solar wind source and favors a migration mechanism for the distribution of adsorbed water onthe Moon.

Diurnally↗
NASA NTRSDate not supplied

What is LAADS DAAC? How to Download Data from NASA's Source for Cloud and Aerosol Data

The Level-1 and Atmosphere Archive Distribution System (LAADS) Distributed Active Archive Center (DAAC) primarily archives and distributes data on clouds and aerosols in Earth’s atmosphere in addition to providing Level 0 and 1 Moderate Imaging Radiometer Suite (MODIS), Visible Infrared Imaging Radiometer Suite (VIIRS) and airborne data as well as data from the European Space Agency’s Sea and Land Surface Temperature Radiometer (SLSTR) and the Ocean and Land Colour Instrument (OLCI). Secondarily, LAADS DAAC serves as a backup source for MODIS and VIIRS land products. As one of twelve DAACS supported by NASA’s Earth Science Data and Information System (ESDIS), LAADS DAAC provides a gateway to data through multiple tools. The newest, View Data, allows users to preview data sets before downloading or applying any post-processing transformations. Learn about how to access LAADS DAAC data, get up-to-date information about LAADS DAAC's migration to hosting data in the cloud, and discover how to use the View Data tool.

Tassia Owen↗
NASA NTRS1998

Technology Assessment of High Capacity Data Storage Systems: Can We Avoid a Data Survivability Crisis

The density of digital storage media in our information-intensive society increases by a factor of four every three years, while the rate at which this data can be migrated to viable long-term storage has been increasing by a factor of only four every nine years. Meanwhile, older data stored on increasingly obsolete media, are at considerable risk. When the systems for which the media were designed are no longer serviced by their manufacturers (many of whom are out of business), the data will no longer be accessible. In some cases, older media suffer from a physical breakdown of components - tapes simply lose their magnetic properties after a long time in storage. The scale of the crisis is compatible to that facing the Social Security System. Greater financial and intellectual resources to the development and refinement of new storage media and migration technologies in order to preserve as much data as possible.

Halem, M.↗
NASA NTRS1981

Methods of editing cloud and atmospheric layer affected pixels from satellite data

The location and migration of cloud, land and water features were examined in spectral space (reflective VIS vs. emissive IR). Daytime HCMM data showed two distinct types of cloud affected pixels in the south Texas test area. High altitude cirrus and/or cirrostratus and "subvisible cirrus" (SCi) reflected the same or only slightly more than land features. In the emissive band, the digital counts ranged from 1 to over 75 and overlapped land features. Pixels consisting of cumulus clouds, or of mixed cumulus and landscape, clustered in a different area of spectral space than the high altitude cloud pixels. Cumulus affected pixels were more reflective than land and water pixels. In August the high altitude clouds and SCi were more emissive than similar clouds were in July. Four-channel TIROS-N data were examined with the objective of developing a multispectral screening technique for removing SCi contaminated data.

Nixon, P. R.↗
NASA NTRS2012

The NASA Tournament Laboratory (NTL): Improving Data Access at PDS while Spreading Joy and Engaging Students through 16 Micro-Contests

NASA PDS hosts terabytes of valuable data from hundreds of data sources and spans decades of research. Data is stored on flat-file systems regulated through careful meta dictionaries. PDS's data is available to the public through its website which supports data searches through drill-down navigation. While the system returns data quickly, result sets in response to identical input differ depending on the drill-down path a user follows. To correct this Issue, to allow custom searching, and to improve general accessibility, PDS sought to create a new data structure and API, and to use them to build applications that are a joy to use and showcase the value of the data to students, teachers and citizens. PDS engaged TopCoder and Harvard Business School through the NTL to pursue these objectives in a pilot effort. Scope was limited to Small Bodies Node data. NTL analyzed data, proposed a solution, and implemented it through a series of micro-contests. Contest focused on different segments of the problem; conceptualization, architectural design, implementation, testing, etc. To demonstrate the utility of the completed solution, NTL developed web-based and mobile applications that can compare targets, regardless of mission. To further explore the potential of the solution NTL hosted "Mash-up" challenges that integrated the API with other publically available assets, to produce consumer and teaching applications, including an Augmented Reality iPad tool. Two contests were also posted to middle and high school students via the NoNameSite.com platform, and as a result of these contests, PDS/SBN has initiated a Facebook program. These contests defined and implemented a data warehouse with the necessary migration tools to transform legacy data, produced a public web interface for the new search, developed a public API, and produced four mobile applications that we expect to appeal to users both within and, without the academic community.

LaMora, Andy↗
NASA NTRS1984

Impact of IPAD on CAD/CAM database university research

IPAD program has provided direction, focus and software products which impacted on CAD/CAM data base research and follow-on research. The relationship of IPAD to the research projects which involve the storage of geometric data in common data ase facilities such as data base machines, the exchange of data between heterogeneous data bases, the development of IGES processors, the migration of lrge CAD/CAM data base management systems to noncompatible hosts, and the value of RIM as a research tool is described.

Leach, L. M.↗
NASA NTRS2007

Alaska at the Crossroads of Migration: Space Based Ornithology

Understanding bird migration on a global scale is one of the most compelling and challenging problems of modern biology with major implications for human health and conservation biology. Revolutionary advances in remote sensing now provide us with near real-time measurements of atmospheric and land surface conditions at high spatial resolution over entire continents. We use spatially-explicit, individual based bird migration models driven by numerical weather prediction models of atmospheric conditions, dynamic habitat suitability maps derived from remotely sensed land surface conditions, biophysiological models, and biological field data to simulate migration routes, timing, energy budgets, and survival of individual birds and populations. Long-distance migratory birds travel annually between breeding grounds in Alaska and wintering grounds in Latin Amierica. Approximately 25% of these species are potential vectors of Avian Influenza. Alaska is at the crossroads of Asian and New World migratory flyways and is likely to be a point of introduction of Asian H5N1 AI into the western hemisphere. If/when an infected bird is detected, a pressing question will be where was this bird several days ago, and where is it likely to go after it was released from the survey site? Answers to such questions will increase effectiveness of AI surveillance and mitigation measures. From a conservation perspective, Alaska's diverse landscape provides breeding sites for many migrants, and climatic and land surface changes along migratory flyways in the western hemisphere may reduce bird survival and physical condition upon arrival at Alaskan breeding territories, success and migrant populations.

Deppe, Jill↗
NASA NTRS2020

Resolution of Lava Tubes with Ground Penetrating Radar: The TubeX project

Remote sensing surveys of the Moon and Mars show evidence of lava tubes, which are potential safe havens for human crews and their equipment. Ground penetrating radar (GPR) can be used to map tubes because the void/rock interface at tube ceilings and floors strongly reflects radar pulses. We have tested the capacity of GPR to sense lava tube geometry at Lava Beds National Monument (LBNM) in California, USA. GPR and detailed LiDAR data are presented for two tubes: Skull Cave, with a few meters of overburden, diameter ~10-20 m, and a rubbly floor; and Valentine Cave, with similarly thin overburden, diameter ~1-3 meters, and a flatter smoother floor. On both caves GPR clearly resolves the ceiling and permits good estimates of the cave width as validated with LiDAR data. Where GPR fails, the primary cause is inferred to be strong out-of-plane effects due to complex 3D geometries. Recovery of the floor position requires migrating the GPR data with a 2D velocity model, as signal velocity is faster in void space. We find that floor position is recoverable in caves whose voids are taller than the radar wavelength (~3 m in this study). Forward modeling assuming planetary parameters suggests GPR should be similarly successful on the Moon or Mars.

S. Esmaeili↗
NASA NTRS2011

Performance Comparison of Wireless Sensor Network Standard Protocols in an Aerospace Environment: ISA100.11a and ZigBee Pro

Standards-based wireless sensor network (WSN) protocols are promising candidates for spacecraft avionics systems, offering unprecedented instrumentation flexibility and expandability. Ensuring reliable data transport is key, however, when migrating from wired to wireless data gathering systems. In this paper, we conduct a rigorous laboratory analysis of the relative performances of the ZigBee Pro and ISA100.11a protocols in a representative crewed aerospace environment. Since both operate in the 2.4 GHz radio frequency (RF) band shared by systems such as Wi-Fi, they are subject at times to potentially debilitating RF interference. We compare goodput (application-level throughput) achievable by both under varying levels of 802.11g Wi-Fi traffic. We conclude that while the simpler, more inexpensive ZigBee Pro protocol performs well under moderate levels of interference, the more complex and costly ISA100.11a protocol is needed to ensure reliable data delivery under heavier interference. This paper represents the first published, rigorous analysis of WSN protocols in an aerospace environment that we are aware of and the first published head-to-head comparison of ZigBee Pro and ISA100.11a.

Wagner, Raymond S.↗
NASA NTRS1986

Measurements of hydrogen and helium isotopes in Galactic cosmic rays from 1978 through 1984

The propagation of light cosmic rays is examined using measurements of the relative abundances of the isotopes H-1, H-2 and He-3 and He-4 made with the ISEE 3 instrumentation. It is believed that cosmic ray particles experience spallation in traveling through the interstellar medium, thereby producing the isotopes examined in the present study. The isotopic ratios are therefore expected to yield data on outward migrating particles, which lose energy while moving toward extragalactic space, i.e., the 'leaky box approximation'. The energy ranges covered are 26-138 MeV/nucleon for H-1 and He-4, 24-89 MeV/nucleon for H-2 and 43-146 MeV/nucleon for He-3. Solar activity ranged from minimum to maximum over the observational period. Details of the experimental strategies, instrumentation features and calibration techniques employed with the particle detectors are provided. Histograms were geneated of the energies attributable to each particle track and mass ratios of the various isotopes were calculated over the measured energy ranges. Account was taken of errors introduced by solar modulation, and an escape path length of 5.6-7.8 g/sq cm was estimated for particle propagation through the Galaxy. The projected path length agrees with previous estimates based on data from heavier cosmic ray nuclei.

Kroeger, R.↗
NASA NTRS1987

High-speed inlet research program and supporting analyses

A Mach 5 cruise aircraft was studied in a joint program effort. The propulsion system chosen for this aircraft was an over-under turbojet/ramjet system. The ramjet portion of the inlet is to be tested in NASA Lewis' 10 x 10 SWT. Goals of the test program are to obtain performance data and bleed requirements, and also to obtain analysis code validation data. Supporting analysis of the inlet using a three-dimensional Navier-Stokes code (PEPSIS) indicates that sidewall shock/boundary layer interactions cause large separated regions in the corners underneath the cowl. Such separations generally lead to inlet unstart, and are thus a major concern. As a result of the analysis, additional bleed regions were added to the inlet model sidewalls and cowl to control separations in the corners. A two-dimensional analysis incorporating bleed on the ramp is also presented. Supporting experiments for the Mach 5 programs were conducted in the Lewis' 1 x 1 SWT. A small-scale model representing the inlet geometry up to the ramp shoulder and cowl lip was tested to verify the accelerator plate test technique and to obtain data on flow migration in the ramp and sidewall boundary layers. Another study explored several ramp bleed configurations to control boundary layer separations in that region. Design of a two-dimensional Mach 5 cruise inlet represents several major challenges including multimode operation and dual flow, high temperatures, and three-dimensional airflow effects.

Coltrin, Robert E.↗
NASA NTRSDate not supplied

NASA’s Global Imagery Browse Services and Worldview Migrate to Mapbox Vector Tiles: The Drawbacks and Advantages

NASA’s Global Imagery Browse Services (GIBS) is in the process of migrating their vector data from raster tiles to Mapbox Vector Tiles (MVTs). MVTs are becoming ubiquitous in the world of web mapping. Serving data in place of pixels for each feature presents numerous new visualization opportunities as well as various new challenges to overcome. This presentation will cover some of the hardships these teams encountered, the techniques used to overcome those barriers, and it will give you a glimpse at how the user experience was enhanced.

Benjamin King↗
NASA NTRS2018

Earthdata Cloud Analytics Project

This presentation describes a nascent project in NASA to develop a framework to support end-user analytics of NASA's Earth science data in the cloud. The chief benefit of migrating EOSDIS (Earth Observation System Data and Information Systems) data to the cloud is to position the data next to enormous computing capacity to allow end users to process data at scale. The Earthdata Cloud Analytics project will user a service-based approach to facilitate the infusion of evolving analytics technology and the integration with non-NASA analytics or other complementary functionality at other agencies and in other nations.

data services↗
NASA NTRS2018

Generalizing a Data Analysis Pipeline in the Cloud to Handle Diverse Use Cases in NASA's EOSDIS

NASA's Earth Observing System Data and Information System (EOSDIS) is tasked with archiving and distributing Earth Observation data across a range of disciplines, including atmospheric science, oceanography, land processes, natural hazards, solar radiance and even socioeconomic aspects relating to the environment. Driven by rapidly rising data volumes, EOSDIS is migrating to a cloud computing based archive over the next few years. Although this simplifies data management somewhat, the main aim is to provide the data in an environment where end users can bring their analysis to the data rather than attempting to download and manage ever-increasing volumes. To that end, a cloud-based analysis platform is being constructed to enable data transformations, analyses and visualization without egressing the data from the cloud. In this endeavor, we expect a wide variety of users, algorithms and use cases. Consequently, the architecture of this cloud analytics platform is expressly designed to be based on open services, thus fostering an ecosystem that enables the efficient combination of common components with data-specific or analysis-specific components. Reviewed and approved by Andrew Mitchell, ESDIS project manager.

Cloud computing↗
NASA NTRSDate not supplied

Pyroscopegridding: Geo-Leo Aerosol Data Fusion (an Open-Source Package)

The retrieval of aerosol optical depths (AODs) from sun-synchronous polar orbiting satellites, such as MODISs, VIIRSs, OMI, TROPOMI, etc., has been widely adopted as a method for obtaining information regarding particulate matter (PM) and related atmospheric processes. However, the advent of recently launched geostationary satellites, such as GOES-16/17/18, Himawari-8/9, and Meteosat Third Generation (MTG), has led to an increased temporal resolution of AOD observations (order of 10 minutes), resulting in typically one or more images per hour during daylight hours, compared to the once-per-day observations obtained from LEO satellites. By integrating these observations, the diurnal cycle of global AOD can be characterized at local, regional, and global scales. The scientific community is still examining the novel data from geostationary satellite observations and evaluating methods for effectively merging these observations with differing spatial and temporal resolutions. This presents a significant ""Big Data"" challenge, encompassing not only data storage, but also data discoverability, accessibility, and migration within cloud computing environments. This study presents our attempts at fusing Level 2 aerosol data from six satellites, three of which are geostationary (GOES-16/17 and Himawari-8) and three of which are polar orbiting (TERRA/MODIS, AQUA/MODIS, and SNPP-VIIRS), using the Dark Target aerosol retrieval algorithm. The ability to fuse remote sensing products on demand into desired temporal and spatial domains empowers researchers and practitioners to more efficiently work with satellite and sensor data. It is our hope that through making our open-source package and accompanying functionality available, the scientific community will have improved access to aerosol data processing resources.

Jennifer Wei↗
NASA NTRS1984

A SAR correlation algorithm which accommodates large-range migration

This paper presents an algorithm for azimuth correlation of synthetic aperture radar (SAR) data with extraordinarily large-range migration which cannot be accommodated by the existing frequency domain interpolation approach used in current Seasat SAR processing. A mathematical model is provided for the SAR range-correlated point-target response both in the spatial and frequency domains. A simple and efficient processing algorithm derived from the exact two-dimensional correlation algorithm is given. This algorithm enables azimuth correlation by two cascaded one-dimensional correlation steps. The first step is to transform the range-correlated SAR data into the frequency domain in azimuth, followed by a newly developed range convolution filter to correct the range-dispersed spectrum of the range-correlated point-target response (RCPTR). The second step is the frequency-domain range migration correction approach for the azimuth compression. Simulation results provided here indicate that this processing algorithm yields a satisfactory compressed impulse response for SAR data with large-range migration whereas previous methods were subject to a broadening of the impulse response along range.

Jin, M. Y.↗