Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,621 records · Page 90

Loads and Deformations of Buckled Rectangular Plates

The nonlinear large-deflection equations of von Karman for plates are converted into a set of linear equations by expanding the displacements Into a power series in terms of an arbitrary parameter. The postbuckling behavior of simply supported rectangular plates subjected to longitudinal compression and subject to a uniform temperature rise is investigated in detail by solving the first few of the equations. Experimental data are presented for the compression problem. Comparisons are made for total shortening and local strains and deflections which indicate good agreement between experimental and theoretical results.

Stein, Manuel↗

Efficient Encoding and Rendering of Time-Varying Volume Data

Visualization of time-varying volumetric data sets, which may be obtained from numerical simulations or sensing instruments, provides scientists insights into the detailed dynamics of the phenomenon under study. This paper describes a coherent solution based on quantization, coupled with octree and difference encoding for visualizing time-varying volumetric data. Quantization is used to attain voxel-level compression and may have a significant influence on the performance of the subsequent encoding and visualization steps. Octree encoding is used for spatial domain compression, and difference encoding for temporal domain compression. In essence, neighboring voxels may be fused into macro voxels if they have similar values, and subtrees at consecutive time steps may be merged if they are identical. The software rendering process is tailored according to the tree structures and the volume visualization process. With the tree representation, selective rendering may be performed very efficiently. Additionally, the I/O costs are reduced. With these combined savings, a higher level of user interactivity is achieved. We have studied a variety of time-varying volume datasets, performed encoding based on data statistics, and optimized the rendering calculations wherever possible. Preliminary tests on workstations have shown in many cases tremendous reduction by as high as 90% in both storage space and inter-frame delay.

CODING↗

Generation of a suite of 3D computer-generated breast phantoms from a limited set of human subject data

Purpose: The authors previously reported on a three-dimensional computer-generated breast phantom, based on empirical human image data, including a realistic finite-element based compression model that was capable of simulating multimodality imaging data. The computerized breast phantoms are a hybrid of two phantom generation techniques, combining empirical breast CT (bCT) data with flexible computer graphics techniques. However, to date, these phantoms have been based on single human subjects. In this paper, the authors report on a new method to generate multiple phantoms, simulating additional subjects from the limited set of original dedicated breast CT data. The authors developed an image morphing technique to construct new phantoms by gradually transitioning between two human subject datasets, with the potential to generate hundreds of additional pseudoindependent phantoms from the limited bCT cases. The authors conducted a preliminary subjective assessment with a limited number of observers (n= 4) to illustrate how realistic the simulated images generated with the pseudoindependent phantoms appeared. Methods: Several mesh-based geometric transformations were developed to generate distorted breast datasets from the original human subject data. Segmented bCT data from two different human subjects were used as the 'base' and 'target' for morphing. Several combinations of transformations were applied to morph between the 'base' and 'target' datasets such as changing the breast shape, rotating the glandular data, and changing the distribution of the glandular tissue. Following the morphing, regions of skin and fat were assigned to the morphed dataset in order to appropriately assign mechanical properties during the compression simulation. The resulting morphed breast was compressed using a finite element algorithm and simulated mammograms were generated using techniques described previously. Sixty-two simulated mammograms, generated from morphing three human subject datasets, were used in a preliminary observer evaluation where four board certified breast radiologists with varying amounts of experience ranked the level of realism (from 1 ='fake' to 10 ='real') of the simulated images. Results: The morphing technique was able to successfully generate new and unique morphed datasets from the original human subject data. The radiologists evaluated the realism of simulated mammograms generated from the morphed and unmorphed human subject datasets and scored the realism with an average ranking of 5.87 {+-} 1.99, confirming that overall the phantom image datasets appeared more 'real' than 'fake.' Moreover, there was not a significant difference (p > 0.1) between the realism of the unmorphed datasets (6.0 {+-} 1.95) compared to the morphed datasets (5.86 {+-} 1.99). Three of the four observers had overall average rankings of 6.89 {+-} 0.89, 6.9 {+-} 1.24, 6.76 {+-} 1.22, whereas the fourth observer ranked them noticeably lower at 2.94 {+-} 0.7. Conclusions: This work presents a technique that can be used to generate a suite of realistic computerized breast phantoms from a limited number of human subjects. This suite of flexible breast phantoms can be used for multimodality imaging research to provide a known truth while concurrently producing realistic simulated imaging data.

60 APPLIED LIFE SCIENCES↗

Uncertainty Visualization of Critical Points of 2D Scalar Fields for Parametric and Nonparametric Probabilistic Models

This paper presents a novel end-to-end framework for closed-form computation and visualization of critical point uncertainty in 2D uncertain scalar fields. Critical points are fundamental topological descriptors used in the visualization and analysis of scalar fields. The uncertainty inherent in data (e.g., observational and experimental data, approximations in simulations, and compression), however, creates uncertainty regarding critical point positions. Uncertainty in critical point positions, therefore, cannot be ignored, given their impact on downstream data analysis tasks. Here, in this work, we study uncertainty in critical points as a function of uncertainty in data modeled with probability distributions. Although Monte Carlo (MC) sampling techniques have been used in prior studies to quantify critical point uncertainty, they are often expensive and are infrequently used in production-quality visualization software. We, therefore, propose a new end-to-end framework to address these challenges that comprises a threefold contribution. First, we derive the critical point uncertainty in closed form, which is more accurate and efficient than the conventional MC sampling methods. Specifically, we provide the closed-form and semianalytical (a mix of closed-form and MC methods) solutions for parametric (e.g., uniform, Epanechnikov) and nonparametric models (e.g., histograms) with finite support. Second, we accelerate critical point probability computations using a parallel implementation with the VTK-m library, which is platform portable. Finally, we demonstrate the integration of our implementation with the ParaView software system to demonstrate near-real-time results for real datasets.

97 MATHEMATICS AND COMPUTING↗

Processing and Archiving Camera Data Effectively for Operando Neutron Measurement of Metal Additive Manufacturing

As the name suggests, The Operando Neutron Measurement of Metal Additive Manufacturing project conducted by ORNL’s Manufacturing Demonstration Facility (MDF) is experimenting with advanced additive metal manufacturing techniques while analyzing the process using the Spallation Neutron Source’s (SNS) beamline. As part of this experiment, the MDF is seeking to employ 2 XIMEA visible light cameras and a single infrared camera to analyze and correct manufacturing in real-time. The MDF requires a solution for capturing the high-resolution data feed from the cameras with compression while preserving enough detail for their software to detect and correct errors in real time. Our solution was to develop a Robot Operating System (ROS) driver to feed the camera data into ROS. From ROS, the feed is compressed and temporarily stored locally to a stripped 4 NVMe SSD RAID array. Post-experiment, the data is transitioned to long-term storage for archival purposes.

36 MATERIALS SCIENCE↗

Data and Code for: Observation-constrained agroecosystem model inversion reveals continental-scale variation of winter wheat traits

This repository contains the simulation outputs and processing scripts associated with the study of winter wheat traits across the United States, utilizing the Ecosys agroecosystem model. The dataset includes model results for both rainfed and irrigated winter wheat systems, supporting the findings presented in the manuscript titled "Observation-constrained agroecosystem model inversion reveals continental-scale variation of winter wheat traits." Data includes the original Ecosys simulation outputs (archived in .db format within the compressed .zip files) and extracted analysis data (stored in .pkl files for efficient processing). Python code for data processing and figure generation is provided in a Jupyter notebook. External Observational Datasets should refer to the following official repositories for the input and validation data used in this study. The eddy covariance data from the AmeriFlux network (https://ameriflux.lbl.gov/). Climate-forcing data of NLDAS-2 from NASA LDAS (https://ldas.gsfc.nasa.gov/nldas/nldas-2-forcing-data). Soil data from the Gridded Soil Survey Geographic Database (gSSURGO), available at (https://www.nrcs.usda.gov/resources/data-and-reports/gridded-soil-survey-geographic-gssurgo-database). Crop yields, planting and harvest dates from the USDA public databases (https://quickstats.nass.usda.gov/; https://webapp.rma.usda.gov/apps/actuarialinformationbrowser/CropCriteria.aspx). Satellite-derived SLOPE GPP data from ORNL DAAC (https://daac.ornl.gov/cgi-bin/dsviewer.pl?ds_id=1786). Land use and crop progress information from the USDA Crop Data Layer and Crop Progress and Condition Gridded Layers (https://www.nass.usda.gov/Research_and_Science/). The Ecosys model code is available online at https://github.com/jinyun1tang/ECOSYS.

Wheat↗

High Performance Access to Archival Data Stored in HDF4 and HDF5 on Cloud Object Stores Without Reformatting the Files

Cloud computing offers numerous advantages for users of extensive Earth science data collections. These benefits encompass direct online access to data files and granules from any location, scalable access supporting parallel computing workflows, and flexible computing tools enabling innovative experimentation with processing techniques. However, older archival file formats designed for distinct computing systems hinder efficient access to decade-long time-series data when compared to data stored in modern cloud-optimized formats like Web Object Stores (WOS), exemplified by Amazon Web Services’ Simple Storage Service (S3). We describe DMR++ (Dataset Metadata Response plus plus), a technology facilitating efficient access to HDF5 (Hierarchical Data Format, version 5) and HDF4 files stored on WOS systems without requiring data reformatting. DMR++ achieves performance comparable to technologies like Zarr while preserving the original file structure, a substantial benefit considering the vast quantity of archival files held by organizations such as NASA. Moreover, DMR++ typically outperforms cloud-optimized versions of HDF5. Essentially an XML (Extensible Markup Language) document usually stored alongside the described data, DMR++ can also be generated on-the-fly but is generally created during data staging to the WOS. Archival files that use HDF4/5 often store large arrays of numerical data. The data in these files is often compressed, typically reducing their size by a factor of four or more. To achieve efficient access to portions of those arrays, they are 'chunked' into smaller sub-arrays, each individually compressed. The chunk size is a compromise, where spinning disks can efficiently access data in smaller chunks while S3 favors larger chunks. A simple optimization of aggregating smaller chunks that are stored adjacently, transferring them in a single access and then individually decompressing them will improve performance. NASA data pose an additional challenge: special Application Programmer Interface (API) libraries are often needed to compute some variables. These libraries are incompatible with WOS environments. Our solution involves storing computed values in the DMR++ document or a companion file, making them accessible like other variables and eliminating the need for specialized APIs. We outline specific optimizations for both satellite grid and swath data stored in HDF4-EOS2 (Earth Observing System).

James Gallagher↗

A petabyte size electronic library using the N-Gram memory engine

A model library containing petabytes of data is proposed by Triada, Ltd., Ann Arbor, Michigan. The library uses the newly patented N-Gram Memory Engine (Neurex), for storage, compression, and retrieval. Neurex splits data into two parts: a hierarchical network of associative memories that store 'information' from data and a permutation operator that preserves sequence. Neurex is expected to offer four advantages in mass storage systems. Neurex representations are dense, fully reversible, hence less expensive to store. Neurex becomes exponentially more stable with increasing data flow; thus its contents and the inverting algorithm may be mass produced for low cost distribution. Only a small permutation operator would be recalled from the library to recover data. Neurex may be enhanced to recall patterns using a partial pattern. Neurex nodes are measures of their pattern. Researchers might use nodes in statistical models to avoid costly sorting and counting procedures. Neurex subsumes a theory of learning and memory that the author believes extends information theory. Its first axiom is a symmetry principle: learning creates memory and memory evidences learning. The theory treats an information store that evolves from a null state to stationarity. A Neurex extracts information data without a priori knowledge; i.e., unlike neural networks, neither feedback nor training is required. The model consists of an energetically conservative field of uniformly distributed events with variable spatial and temporal scale, and an observer walking randomly through this field. A bank of band limited transducers (an 'eye'), each transducer in a bank being tuned to a sub-band, outputs signals upon registering events. Output signals are 'observed' by another transducer bank (a mid-brain), except the band limit of the second bank is narrower than the band limit of the first bank. The banks are arrayed as n 'levels' or 'time domains, td.' The banks are the hierarchical network (a cortex) and transducers are (associative) memories. A model Neurex was built and studied. Data were 50 MB to 10 GB samples of text, data base, and images: black/white, grey scale, and high resolution in several spectral bands. Memories at td, S(m(sub td)), were plotted against outputs of memories at td-1. S(m(sub td)) was Boltzman distributed, and memory frequencies exhibited self-organized criticality (SOC); i.e., 'l/f(sup beta)' after long exposures to data. Whereas output signals from level n may be encoded with B(sub output) = O(-log(2)f(sup beta)) bits, and input data encoded with B(sub input) = O((S(td)/S(td-1))(sup n)), B(sup output)/B(sub input) is much less than 1 always, the Neurex determines a canonical code for data and it is a lossless data compressor. Further tests are underway to confirm these results with more data types and larger samples.

Bugajski, Joseph M.↗

Development of a simultaneous Hugoniot and temperature measurement for preheated-metal shock experiments: Melting temperatures of Ta at pressures of 100 GPa

Equations of state of metals are important issues in earth science and planetary science. A major limitation of them is the lack of experimental data for determining pressure-volume and temperature of shocked metal simultaneously. By measuring them in a single experiment, a major source of systematic error is eliminated in determining from which shock pressure release pressure originates. Hence, a non-contact fast optical method was developed and demonstrated to simultaneously measure a Hugoniot pressure-volume (P{sub H}-V{sub H}) point and interfacial temperature T{sub R} on the release of Hugoniot pressure (P{sub R}) for preheated metals up to 1000 K. Experimental details in our investigation are (i) a Ni-Cr resistance coil field placed around the metal specimen to generate a controllable and stable heating source, (ii) a fiber-optic probe with an optical lens coupling system and optical pyrometer with ns time resolution to carry out non-contact fast optical measurements for determining P{sub H}-V{sub H} and T{sub R}. The shock response of preheated tantalum (Ta) at 773 K was investigated in our work. Measured data for shock velocity versus particle velocity at an initial state of room temperature was in agreement with previous shock compression results, while the measured shock data between 248 and 307 GPa initially heated to 773 K were below the Hugoniot evaluation from its off-Hugoniot states. Obtained interfacial temperatures on release of Hugoniot pressures (100-170 GPa) were in agreement with shock-melting points at initial ambient condition and ab initio calculations of melting curve. It indicates a good consistency for shock melting data of Ta at different initial temperatures. Our combined diagnostics for Hugoniot and temperature provides an important approach for studying EOS and the temperature effect of shocked metals. In particular, our measured melting temperatures of Ta address the current controversy about the difference by more than a factor of 2 between the melting temperatures measured under shock and those measured in a laser-heated diamond anvil cell at {approx}100 GPa.

36 MATERIALS SCIENCE↗

Quantifying the hierarchy of structural and mechanical length scales in granular systems

Continuum modeling of granular media is made possible by the existence of a length scale at and above which grain-resolved properties can be meaningfully homogenized. Progress has been made in identifying such length scales relevant to local structural properties such as porosity. However, a systematic analysis of scales above which different mechanical properties can be homogenized has yet to emerge. Here, X-ray tomography and 3D X-ray diffraction data are examined to identify such length scales. The data was obtained in-situ in compressed granular materials with rigid and flexible confinement. The experimental data are supplemented with validated discrete element simulations which examine different system sizes and different boundary conditions. Overall, our study reveals a hierarchy in the length scales of granular solids, with lengths governing structural variables being the shortest, lengths of stress variables being intermediate, and lengths of energy dissipation being the longest. All structural and mechanical length scales obey a power law based on the theory of Geostatistics, implying that the length scales can be found by analyzing samples significantly smaller than the length scales themselves. The length scales are also found to be sensitive to boundary conditions, implying that they are extrinsic features of granular media.

36 MATERIALS SCIENCE↗

Fallon FORGE: Analogue Outcrop Samples

Compilation of results for mechanical and fluid flow properties of analogue outcrop samples - experimental data for compressional and shear wave velocities, tensile strengths, and compressive strengths. Outcrop location and sample orientation data are documented in a separate csv file.

15 GEOTHERMAL ENERGY↗

The Solar Maximum observatory

The successful retrieval and repair of the Solar Maximum Mission (SMM) satellite by Shuttle astronauts in April 1984 permitted continuance of solar flare observations that began in 1980. The SMM carries a soft X ray polychromator, gamma ray, UV and hard X ray imaging spectrometers, a coronagraph/polarimeter and particle counters. The data gathered thus far indicated that electrical potentials of 25 MeV develop in flares within 2 sec of onset. X ray data show that flares are composed of compressed magnetic loops that have come too close together. Other data have been taken on mass ejection, impacts of electron beams and conduction fronts with the chromosphere and changes in the solar radiant flux due to sunspots.

Rust, D. M.↗

A Survey on Error-Bounded Lossy Compression for Scientific Datasets

Error-bounded lossy compression has been effective in significantly reducing the data storage/transfer burden while preserving the reconstructed data fidelity very well. Many error-bounded lossy compressors have been developed for a wide range of parallel and distributed use cases for years. They are designed with distinct compression models and principles, such that each of them features particular pros and cons. In this article, we provide a comprehensive survey of emerging error-bounded lossy compression techniques. The key contribution is fourfold. (1) We summarize a novel taxonomy of lossy compression into six classic models. (2) We provide a comprehensive survey of 10 commonly used compression components/modules. (3) We summarized pros and cons of 47 state-of-the-art lossy compressors and present how state-of-the-art compressors are designed based on different compression techniques. (4) We discuss how customized compressors are designed for specific scientific applications and use-cases. We believe this survey is useful to multiple communities including scientific applications, high-performance computing, lossy compression, and big data.

Error-Bounded Lossy Compression↗

Upper bound on compression ratio for run-length encoding.

Run-length encoding is a scheme which has been used to remove redundancy from video signals. Based upon a first-order Markov model of video data, an upper bound on compression ratio is found for run-length encoding. It is shown that at best run-length encoding will give a compression ratio which is approximately 60% of the maximum attainable value. This percentage is fairly independent of the particular picture involved.

Gray, K. G.↗

Lifetime Extension Report: Progress on the SAVY-4000 Lifetime Extension Program

The 3-year accelerated aging study of the SAVY-4000 O-ring shows very little evidence of significant degradation to samples subjected to aggressive elevated temperature and radiation conditions. Whole container thermal aging studies followed by helium leakage testing and compression set measurements were used to establish an estimate for a failure criterion for O-ring compression set of ≥65 %. The whole container aging studies further show that the air flow and efficiency functions of the filter do not degrade significantly after thermal aging. However, the degradation of the water-resistant function leads to water penetration failure after four months at 210°C, but does not cause failure after 10 months at 120°C (130°C is the maximum operating temperature for the PTFE membrane). The thermal aging data for O-ring compression set do not meet the assumptions of standard time-temperature superposition analysis for accelerated aging studies. Instead, the data suggest that multiple degradation mechanisms are operative, with a reversible mechanism operative at low aging temperatures and an irreversible mechanism dominating at high aging temperatures. To distinguish between these mechanisms, we have measured compression set after allowing the sample to physically relax, thereby minimizing the effect of the reversible mechanism. The resulting data were analyzed using two distinct mathematical methods to obtain a lifetime estimate based on chemical degradation alone. Both methods support a lifetime estimate of greater than 150 years at 80°C. Although the role of the reversible mechanism is not fully understood, it is clear that the contribution to the total compression set is small in comparison to that due to the chemical degradation mechanism. To better understand the chemical degradation mechanism, thermally aged O-ring samples have been characterized by Fourier Transform Infrared (FTIR), Electron Paramagnetic Resonance (EPR), Gel Permeation Chromatography (GPC), and Differential Scanning Calorimetry (DSC). These experiments detect no significant O-ring degradation below 80°C. Furthermore, durometer measurements indicate that there is no significant change in O-ring hardness at all aging conditions examined. Therefore, our current conservative lifetime estimate for the O-ring and the filter is 10 years at 80°C. In FY17, we will continue to probe the chemical degradation mechanism using oxygen consumption measurements under accelerated aging conditions to reveal temperatures at which oxidation occurs, along with any differences in oxidation rate at the low vs. high aging temperatures. We will also refine the failure criteria and finalize the radiation/thermal synergistic studies to determine a final design lifetime.

42 ENGINEERING↗

GPU-based Image Compression for Efficient Compositing in Distributed Rendering Applications

Visualizations of large-scale data sets are often created on graphics clusters that distribute the rendering task amongst many processes. When using real-time GPU-based graphics algorithms, the most time-consuming aspect of distributed rendering is typically the com-positing phase - combining all partial images from each rendering process into the final visualization. Compo siting requires image data to be copied off the GPU and sent over a network to other processes. While compression has been utilized in existing distributed rendering compositors to reduce the data being sent over the network, this compression tends to occur after the raw images are transferred from the GPU to main memory. In this paper, we present work that leverages OpenGL / CUDA interoperability to compress raw images on the GPU prior to transferring the data to main memory. This approach can significantly reduce the device-to-host data transfer time, thus enabling more efficient compositing of images generated by distributed rendering applications.

Lipinksi, Riley↗

Analysis of AIS Data of the Recluse Oil Field, Recluse, Wyoming

Airborne Imaging Spectrometer (AIS) data were flown over the Recluse, Wyoming oil field on September 9, 1984. Processing software was developed at Earth Satellite Corporation (EarthSat) for interactive analysis of the AIS data. EarthSat's AIS processing capabilities include destriping, solar irradiance corrections, residual calculations, geometric resampling, equal energy normalization, interactive spectral classifications and a variety of compressive algorithms to reduce the data to 8-bit format with a minimum of information loss. The in-house photolab facilities of EarthSat can routinely produce high-quality color renditions of the enhanced AIS data. A total of 80 lithologic samples were collected under the AIS flight lines. Correlation (within the atmospheric windows) between the laboratory and the AIS spectra of sample sites was generally poor. Reasonable correlation was only possible in large, freshly plowed fields. Mixed pixels and contrast between the natural and sample's surfaces were believed responsible for the poor correlation. Finally, a drift of approximately three channels was observed in the diffraction grating position within the 1.8 to 2.1 micron quadrant.

Dykstra, J. D.↗

Investigating Access Performance of Long Time Series with Restructured Big Model Data

Data sets generated by models are substantially increasing in volume, due to increases in spatial and temporal resolution, and the number of output variables. Many users wish to download subsetted data in preferred data formats and structures, as it is getting increasingly difficult to handle the original full-size data files. For example, application research users such as those involved with wind or solar energy, or extreme weather events are likely only interested in daily or hourly model data at a single point (or for a small area) for a long time period, and prefer to have the data downloaded in a single file. With native model file structures, such as hourly data from NASA Modern-Era Retrospective analysis for Research and Applications Version-2 (MERRA-2), it may take over 10 hours for the extraction of parameters-of-interest at a single point for 30 years. The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is exploring methods to address this particular user need. One approach is to create value-added data by reconstructing the data files. Taking MERRA-2 data as an example, we have tested converting hourly data from one-day-per-file into different data cubes, such as one-month, or one-year. Performance is compared for reading local data files and accessing data through interoperable services, such as OPeNDAP. Results show that, compared to the original file structure, the new data cubes offer much better performance for accessing long time series. We have noticed that performance is associated with the cube size and structure, the compression method, and how the data are accessed. An optimized data cube structure will not only improve data access, but also may enable better online analysis services

reanalysis↗