Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data cube”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRS2019

MERRA-2 Data and Analytic Services at NASA GES DISC for Climate Extremes Study

NASA's climate reanalysis datasets from the Modern Era Retrospective-analysis for Research and Applications, Version 2 (MERRA-2) contains numerous long-term atmosphere, land, and ocean data products from 1980-present. MERRA-2 datasets, such as precipitation, soil moisture, and temperature, have been used widely to study extreme events. The native archived MERRA-2 data files are day-file (hourly time interval) and month-file, containing up to 125 parameters in one file. Due to the large number of data files and volumes, it is challenging for users, especially the applications research community, to handle the original hourly data files for long time periods to analyze extreme events. In this presentation, we review MERRA-2 data for studies of extreme conditions, and demonstrate analytic services at the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC). One of the current operational services, 'subsetter', allows users to download only specific data of interest, i.e. data selected by parameter, region, and time period. New services are under development that will provide more 'on-the-fly' statistical calculations when downloading data; improve efficiency when accessing long time-series data. We will provide additional "How-to" resources that include step-by-step instructions on data access and usage. We have tested restructuring of day-files in an optimized data cube, which has significantly improved system performance for accessing long time-series. Overall performance is associated with cube size and structure, data compression method, and how the data are accessed. The optimized data cube structure will enable better online analytic services for statistical analysis and extreme events mining. To demonstrate the service, we use an extreme drought associated with the anomalous 2016 monsoon over southern Asia. This prototype time-series service may be augmented in the cloud infrastructure in the future.

data access↗
NASA NTRS2017

Investigating Access Performance of Long Time Series with Restructured Big Model Data

Data sets generated by models are substantially increasing in volume, due to increases in spatial and temporal resolution, and the number of output variables. Many users wish to download subsetted data in preferred data formats and structures, as it is getting increasingly difficult to handle the original full-size data files. For example, application research users such as those involved with wind or solar energy, or extreme weather events are likely only interested in daily or hourly model data at a single point (or for a small area) for a long time period, and prefer to have the data downloaded in a single file. With native model file structures, such as hourly data from NASA Modern-Era Retrospective analysis for Research and Applications Version-2 (MERRA-2), it may take over 10 hours for the extraction of parameters-of-interest at a single point for 30 years. The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is exploring methods to address this particular user need. One approach is to create value-added data by reconstructing the data files. Taking MERRA-2 data as an example, we have tested converting hourly data from one-day-per-file into different data cubes, such as one-month, or one-year. Performance is compared for reading local data files and accessing data through interoperable services, such as OPeNDAP. Results show that, compared to the original file structure, the new data cubes offer much better performance for accessing long time series. We have noticed that performance is associated with the cube size and structure, the compression method, and how the data are accessed. An optimized data cube structure will not only improve data access, but also may enable better online analysis services

reanalysis↗
DOE OSTI2022

Pan-sharpening for microscopy

Techniques for generating full-spatial resolution, full spectral resolution image(s) from a 3D spectral-data cube for any spectral value within a given spectral range are provided without requiring the acquisition of all full-spatial resolution, full spectral resolution data by an instrument. The 3D spectral-data cube is generated from a limited number of full-spatial resolution, sparse spectral resolution data and a sparse-spatial resolution, full-spectral resolution data of the same area of the sample. The use of the 3D spectral-data cube reduces the data acquisition time.

Ovchinnikova, Olga S.↗
NASA NTRS2012

Systems, computer-implemented methods, and tangible computer-readable storage media for wide-field interferometry

Disclosed herein are systems, computer-implemented methods, and tangible computer-readable storage media for wide field imaging interferometry. The method includes for each point in a two dimensional detector array over a field of view of an image: gathering a first interferogram from a first detector and a second interferogram from a second detector, modulating a path-length for a signal from an image associated with the first interferogram in the first detector, overlaying first data from the modulated first detector and second data from the second detector, and tracking the modulating at every point in a two dimensional detector array comprising the first detector and the second detector over a field of view for the image. The method then generates a wide-field data cube based on the overlaid first data and second data for each point. The method can generate an image from the wide-field data cube.

Lyon, Richard G.↗
NASA NTRS2013

PRSEUS Pressure Cube Test Data and Response

NASA s Environmentally Responsible Aviation (ERA) Program is examining the hybrid wing body (HWB) aircraft, among others, in an effort to increase the fuel efficiency of commercial aircraft. The HWB design combines features of a flying wing with features of conventional transport aircraft, and has the advantage of simultaneously increasing both fuel efficiency and payload. Recent years have seen an increased focus on the structural performance of the HWB. The key structural challenge of a HWB airframe is the ability to create a cost and weight efficient, non-circular, pressurized shell. Conventional round fuselage sections react cabin pressure by hoop tension. However, the structural configuration of the HWB subjects the majority of the structural panels to bi-axial, in-plane loads in addition to the internal cabin pressure, which requires more thorough examination and analysis than conventional transport aircraft components having traditional and less complex load paths. To address this issue, while keeping structural weights low, extensive use of advanced composite materials is made. This report presents the test data and preliminary conclusions for a pressurized cube test article that utilizes Boeing's Pultruded Rod Stitched Efficient Unitized Structure (PRSEUS), and which is part of the building block approach used for HWB development.

Lovejoy, Andrew E.↗
NASA NTRS2019

TPSAS-NF1676L-32235-DND

This presentation will provide a summary of the Open Data Cube initiative. In addition, a training manual will be provided to support Open Data Cube training for GEO and the Senegal government.

Brian Killough↗
NASA NTRS2019

TPSAS-NF1676L-32249-DND

This presentation will provide a summary of the Open Data Cube initiative. In addition, a training manual will be provided to support Open Data Cube training for the GEO Secretariat.

Brian Killough↗
NASA NTRS2019

TPSAS-NF1676L-32493-DND

The Committee on Earth Observation Satellites (CEOS) System Engineering Office (SEO) has supported the Open Data Cube (ODC) initiative to provide a data architecture solution that has value to its global users and increases the impact of EO satellite data. ODC is an open-source platform for processing satellite data. We have developed software products and tools around the core ODC that would help users perform machine learning on EO satellite data. The recent United Nations (UN) Sustainable Development Agenda provides a shared blueprint for peace and prosperity for people and for the planet, considering our current situation and helping to create a plan. The core of this agenda is a set of seventeen Sustainable Development Goals (SDGs), which represent an urgent call for action by all countries - both developed and developing - in a global partnership. The CEOS SEO team has recently developed and released a set of innovative Jupyter notebooks addressing UN SDGs 6.6.1 (spatial extents of water-related ecosystems), 11.3.1 (ratio of land consumption rate to population growth rate), and 15.3.1 (proportion of land that is degraded over total land area). These notebooks empower users by providing features that will assist with streamlining analysis ready data retrieval, processing, and visualization. We have recently incorporated several machine learning techniques in these notebooks. In this paper, we present the lessons learned from our experience on classifying land using supervised and unsupervised machine learning techniques using ODC framework for UN SDGs. We identify the current limitations of ODC to seamlessly support machine learning techniques. We propose features that would help machine learning, specifically within the ODC framework. We propose a thematic indexing/loading of data for both unsupervised learning as well as data annotation/labeling pipeline. Currently, ODC supports machine learning by separating data-management from the analysis process. It works as a mechanism to load cubes of data. ODC does not natively support features that are vital in machine learning such as validation splits, fair/balanced sampling, establishing load size constraints, etc. We believe that our proposed features will empower users by providing features that bring machine learning techniques closed to ODC. Enhancements to ODC to better accommodate machine learning techniques can assist in fulfilling UN SDGs such as 6.3.2, 6.4.2, 6.6.1, 11.3.1, 14.1.1, 15.1.1, 15.3.1, and 15.4.2.

Syed R Rizvi↗
DOE OSTI2022

SDSS-IV MaNGA: pyPipe3D Analysis Release for 10,000 Galaxies

We present here the analysis performed using the pyPipe3D pipeline for the final MaNGA data set included in the Sloan Digital Sky Survey data release 17. This data set comprises more than 10,000 individual data cubes, being the integral field spectroscopic (IFS) galaxy survey with the largest number of galaxies. pyPipe3D processes the IFS data cubes to extract spatially resolved spectroscopic properties of both the stellar population and the ionized gas emission lines. A brief summary of the properties of the sample and the characteristics of the analyzed data are included. The article provides details of: (i) the analysis performed; (ii) a description of the pipeline; (iii) the adopted stellar population library; (iv) the morphological and photometric analysis; (v) the adopted data model for the spatially resolved properties derived; and (vi) the individual integrated and characteristic galaxy properties included in the final catalog. Comparisons with the results from a previous version of the pipeline for earlier data releases and from other tools using this data set are included. A practical example of how to use the full data set and the final catalog illustrates how to handle the delivered product. Our full analysis can be accessed and downloaded from our web page.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

An iterative method to deblend AGN-Host contributions for Integral Field spectroscopic observations

ABSTRACT We present a new iterative deblending method to separate the host galaxy (HG) and their Active Galactic Nuclei (AGNs) emission with the use of Integral Field spectroscopic (IFS) data. The method decomposes the resolved HG emission from the unresolved AGN emission by modelling the two-dimensional surface brightness (SB) profile of the point-spread function (PSF) and the two-dimensional SB HG continuum simultaneously per each monochromatic slide. Our method does not require any prior information about the observed SB profile or a detailed fitting of the PSF, making it ideal for the automatic analysis of large galaxy samples. In this work, we test the quality of our method, its advantages, and its disadvantages. We test our method by using a set of IFS mock data cubes to quantify the reliability of our deblending process and further compare our method with the qdblend3d analysis tool. Furthermore, we applied our method to three data cubes selected from the MaNGA survey according to the dominance of either its HG or its AGN. We show that our deblending method is capable of disengaging the bright, non-resolved AGN emission from the HG continuum and its narrow emission lines. However, the decoupling depends on how well the IFS spatially resolves the PSF, and on the relative flux intensity of the HG-AGN. Therefore, the method is ideal for disentangling the bright-flux contribution from AGN-dominated spectra.

Ibarra-Medel, H. (ORCID:0000000297906313)↗
NASA NTRS2000

The 3-Dimensional Inner and Outer Structure of Ejecta Around Eta Carinae as Detected by the STIS

The HST/STIS instrument was used successfully to perform a complete mapping of the Homunculus nebula at two wavelength ranges including H-alpha and H-beta with a spectral resolving power of about 5000 and a spatial resolution of 0.1". The individual spectra were merged to synthesize three-dimensional data cubes that contain a set of images of Eta Car with spatial resolution of 0.10 to 0.251, sliced at velocity increment of 10 -- 30 km/s. For the first time this unique method allows us to diagnose the origin of intrinsic narrow emission structure of the nebula with high spatial and velocity resolution. Our initial analysis revealed the inner emission structure appeared to trace an elongated bipolar shell (possibly other shells as well) with a scale size of an arcsecond (i.e., "little homunculus in the Homunculus"). Furthermore, the mapping data cube revealed that the "fan" or "paddle" -- often referred as the source of peculiar blue-shifted intrinsic emissions including the Strontium cloud -- is not the source of intrinsic emissions. The fan is not even a part of the equatorial disk, but is spatially separated from the peculiar emission structure. Indeed we suggest that the fan is a surface of the Northwest lobe, possibly revealed by a blowout of the equatorial disk. We will use a number of visualization techniques (tomographic animations and simple 3-D models) to show these structures. These new results have strong impact upon future numerical modelings of the Homunculus nebula and of understanding of the evolution of the ejecta powered by the central source(s).

Ishibashi, Kazunori↗
NASA NTRS2015

Between a Map and a Data Rod

A Digital Divide has long stood between how NASA and other satellite-derived data are typically archived (time-step arrays or maps) and how hydrology and other point-time series oriented communities prefer to access those data. In essence, the desired method of data access is orthogonal to the way the data are archived. Our approach to bridging the Divide is part of a larger NASA-supported data rods project to enhance access to and use of NASA and other data by the Consortium of Universities for the Advancement of Hydrologic Science, Inc. (CUAHSI) Hydrologic Information System (HIS) and the larger hydrology community. Our main objective was to determine a way to reorganize data that is optimal for these communities. Two related objectives were to optimally reorganize data in a way that (1) is operational and fits in and leverages the existing Goddard Earth Sciences Data and Information Services Center (GES DISC) operational environment and (2) addresses the scaling up of data sets available as time series from those archived at the GES DISC to potentially include those from other Earth Observing System Data and Information System (EOSDIS) data archives. Through several prototype efforts and lessons learned, we arrived at a non-database solution that satisfied our objectivesconstraints. We describe, in this presentation, how we implemented the operational production of pre-generated data rods and, considering the tradeoffs between length of time series (or number of time steps), resources needed, and performance, how we implemented the operational production of on-the-fly (virtual) data rods. For the virtual data rods, we leveraged a number of existing resources, including the NASA Giovanni Cache and NetCDF Operators (NCO) and used data cubes processed in parallel. Our current benchmark performance for virtual generation of data rods is about a years worth of time series for hourly data (9,000 time steps) in 90 seconds. Our approach is a specific implementation of the general optimal strategy of reorganizing data to match the desired means of access. Results from our project have already significantly extended NASA data to the large and important hydrology user community that has been, heretofore, mostly unable to easily access and use NASA data.

machine learning↗
DOE OSTI2023

A Workflow for Accelerating Multimodal Data Collection for Electrodeposited Films

Abstract Future machine learning strategies for materials process optimization will likely replace human capital-intensive artisan research with autonomous and/or accelerated approaches. Such automation enables accelerated multimodal characterization that simultaneously minimizes human errors, lowers costs, enhances statistical sampling, and allows scientists to allocate their time to critical thinking instead of repetitive manual tasks. Previous acceleration efforts to synthesize and evaluate materials have often employed elaborate robotic self-driving laboratories or used specialized strategies that are difficult to generalize. Herein we describe an implemented workflow for accelerating the multimodal characterization of a combinatorial set of 915 electroplated Ni and Ni–Fe thin films resulting in a data cube with over 160,000 individual data files. Our acceleration strategies do not require manufacturing-scale resources and are thus amenable to typical materials research facilities in academic, government, or commercial laboratories. The workflow demonstrated the acceleration of six characterization modalities: optical microscopy, laser profilometry, X-ray diffraction, X-ray fluorescence, nanoindentation, and tribological (friction and wear) testing, each with speedup factors ranging from 13–46x. In addition, automated data upload to a repository using FAIR data principles was accelerated by 64x.

36 MATERIALS SCIENCE↗
NASA NTRS2006

The Wide-Field Imaging Interferometry Testbed: Recent Results

We present recent results from the Wide-Field Imaging Interferometry Testbed (WIIT). The data acquired with the WIIT is "double Fourier" data, including both spatial and spectral information within each data cube. We have been working with this data, and starting to develop algorithms, implementations, and techniques for reducing this data. Such algorithms and tools are of great importance for a number of proposed future missions, including the Space Infrared Interferometric Telescope (SPIRIT), the Submillimeter Probe of the Evolution of Cosmic Structure (SPECS), and the Terrestrial Planet Finder Interferometer (TPF-I)/Darwin. Recent results are discussed and future study directions are described.

Rinehart, Stephen↗
NASA NTRS1994

Determining structure in molecular clouds

We descibe an automatic, objective routine for analyzing the clumpy structure in a spectral line position-position-velocity data cube. The algorithm works by first contouring the data at a multiple of the rms noise of the observations, then searches for peaks of emission which locate the clumps, and then follows them down to lower intensities. No a proiri clump profile is assumed. By creating simulated data, we test the performance of the algorithm and show that a contour map most accurately depicts internal structure at a contouring interval equal to twice the rms noise of the map. Blending of clump emission leads to small errors in mass and size determinations and in severe cases can result in a number of clumps being misidentified as a single unit, flattening the measured clump mass spectrum. The algorithm is applied to two real data sets as an example of its use. The Rosette molecular cloud is a 'typical' star-forming cloud, but in the Maddalena molecular cloud high-mass star formation is completely absent. Comparison of the two clump lists generated by the algorithm show that on a one-to-one basis the clumps in the star-forming cloud have higher peak temperatures, higher average densities, and are more gravitationally bound than in the non-star-forming cloud. Collective properties of the clumps, such as temperature-size-line-width-mass relations appear very similar, however. Contrary to the initial results reported in a previous paper (Williams & Blitz 1993), we find that the current, more thoroughly tes ted analysis finds no significant difference in the clump mass spectrum of the two clouds.

Williams, Jonathan P.↗
NASA NTRS2013

Interstellar and Ejecta Dust in the Cas A Supernova Remnant

The ejecta of the Cas A supernova remnant has a complex morphology, consisting of dense fast-moving line emitting knots and diffuse X-ray emitting regions that have encountered the reverse shock, as well as more slowly expanding, unshocked regions of the ejecta. Using the Spitzer 5-35 micron IRS data cube, and Herschel 70, 100, and 160 micron PACS data, we decompose the infrared emission from the remnant into distinct spectral components associated with the different regions of the ejecta. Such decomposition allows the association of different dust species with ejecta layers that underwent distinct nuclear burning histories, and determination of the dust heating mechanisms. Our decomposition identified three characteristic dust spectra. The first, most luminous one, exhibits strong emission features at approx. 9 and 21 micron, and a weaker 12 micron feature, and is closely associated with the ejecta knots that have strong [Ar II] 6.99 micron and [Ar III] 8.99 micron emission lines. The dust features can be reproduced by magnesium silicate grains with relatively low MgO-to-SiO2 ratios. A second, very different dust spectrum that has no indication of any silicate features, is best fit by Al2O3 dust and is found in association with ejecta having strong [Ne II] 12.8 micron and [Ne III] 15.6 micron emission lines. A third characteristic dust spectrum shows features that best matched by magnesium silicates with relatively high MgO-to-SiO2 ratio. This dust is primarily associated with the X-ray emitting shocked ejecta and the shocked interstellar/circumstellar material. All three spectral components include an additional featureless cold dust component of unknown composition. Colder dust of indeterminate composition is associated with [Si II] 34.8 micron emission from the interior of the SNR, where the reverse shock has not yet swept up and heated the ejecta. The dust mass giving rise to the warm dust component is about approx. 0.1solar M. However, most of the dust mass is associated with the unidentified cold dust component. Its mass could be anywhere between 0.1 and 1 solar M, and is primarily limited by the mass of refractory elements in the ejecta. Given the large uncertainty in the dust mass, the question of whether supernovae can produce enough dust to account for ISM dust masses in the local and high-z universe remains largely unresolved.

Interstellar↗