Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data compression (Computer science)”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Fast Radiative Transfer Model and Retrieval Algorithm Development for Satellite Remote Sensing Applications

The radiative transfer model (RTM) has a wide range of applications in satellite remote sensing and atmospheric radiation studies. For example, it can be used as a forward model for an inversion algorithm and a satellite data assimilation system, or as a L1 data simulator for pre-launch end-to-end satellite sensor performance studies. However, millions of line-by-line (LBL) radiative transfer calculations at fine monochromatic frequencies are needed in order to properly calculate spectral contributions of water vapor and trace gases in the atmosphere in infrared and solar spectral regions. Therefore, fast, and accurate radiative transfer models are needed. A Principal Component-based radiative transfer model (PCRTM) was developed at NASA Langley to fulfil this need. The PCRTM can simulate the top-of-atmosphere (TOA) radiance or reflectance spectra from 250 nm to 2000 micrometers with several orders of magnitude faster speed as compared to a LBL RTM. It is also extremely accurate compared to LBL RTM benchmarks. The PCRTM model has been developed for hyperspectral sensors such as AIRS, CrIS, IASI, NAST-I, SHIS, CPF, TEMPO, EMIT, OMI, and SCIAMACHY. By using the PCRTM as forward model for an inversion algorithm, one can reduce the data dimension significantly while maintaining original information content by compressing the TOA radiance spectrum into PC-scores. The PCRTM can directly compute the PC-scores and their derivatives with respect to retrieved parameters. Examples of using various PCRTM inversion algorithms to retrieve atmospheric temperature, water vapor, and trace gas profiles, as well as cloud and surface properties from satellite hyperspectral remote sensors will be given. Some of the algorithms have been transitioned to NASA's Goddard Earth Sciences Data and Information Services Center (GES DISC) for public access of high-quality L2 and L3 data.

Xu Liu↗

Galileo mission planning for Low Gain Antenna based operations

The Galileo mission operations concept is undergoing substantial redesign, necessitated by the deployment failure of the High Gain Antenna, while the spacecraft is on its way to Jupiter. The new design applies state-of-the-art technology and processes to increase the telemetry rate available through the Low Gain Antenna and to increase the information density of the telemetry. This paper describes the mission planning process being developed as part of this redesign. Principal topics include a brief description of the new mission concept and anticipated science return (these have been covered more extensively in earlier papers), identification of key drivers on the mission planning process, a description of the process and its implementation schedule, a discussion of the application of automated mission planning tool to the process, and a status report on mission planning work to date. Galileo enhancements include extensive reprogramming of on-board computers and substantial hard ware and software upgrades for the Deep Space Network (DSN). The principal mode of operation will be onboard recording of science data followed by extended playback periods. A variety of techniques will be used to compress and edit the data both before recording and during playback. A highly-compressed real-time science data stream will also be important. The telemetry rate will be increased using advanced coding techniques and advanced receivers. Galileo mission planning for orbital operations now involves partitioning of several scarce resources. Particularly difficult are division of the telemetry among the many users (eleven instruments, radio science, engineering monitoring, and navigation) and allocation of space on the tape recorder at each of the ten satellite encounters. The planning process is complicated by uncertainty in forecast performance of the DSN modifications and the non-deterministic nature of the new data compression schemes. Key mission planning steps include quantifying resource or capabilities to be allocated, prioritizing science observations and estimating resource needs for each, working inter-and intra-orbit trades of these resources among the Project elements, and planning real-time science activity. The first major mission planning activity, a high level, orbit-by-orbit allocation of resources among science objectives, has already been completed; and results are illustrated in the paper. To make efficient use of limited resources, Galileo mission planning will rely on automated mission planning tools capable of dealing with interactions among time-varying downlink capability, real-time science and engineering data transmission, and playback of recorded data. A new generic mission planning tool is being adapted for this purpose.

Gershman, R.↗

A multiresolution adaptive wavelet method for nonlinear partial differential equations

We report the multiscale complexity of modern problems in computational science and engineering can prohibit the use of traditional numerical methods in multi-dimensional simulations. Therefore, novel algorithms are required in these situations to solve partial differential equations (PDEs) with features evolving on a wide range of spatial and temporal scales. To meet these challenges, we present a multiresolution wavelet algorithm to solve PDEs with significant data compression and explicit error control. We discretize in space by projecting fields and spatial derivative operators onto wavelet basis functions. We provide error estimates for the wavelet representation of fields and their derivatives. Then, our estimates are used to construct a sparse multiresolution discretization which guarantees the prescribed accuracy. Additionally, we embed a predictor-corrector procedure within the temporal integration to dynamically adapt the computational grid and maintain the accuracy of the solution of the PDE as it evolves. We present examples to highlight the accuracy and adaptivity of our approach.

97 MATHEMATICS AND COMPUTING↗

QuadConv: Quadrature-based convolutions with applications to non-uniform PDE data compression

We present a new convolution layer for deep learning architectures which we call QuadConv — an approximation to continuous convolution via quadrature. Our operator is developed explicitly for use on non-uniform, mesh-based data, and accomplishes this by learning a continuous kernel that can be sampled at arbitrary locations. Moreover, the construction of our operator admits an efficient implementation which we detail and construct. As an experimental validation of our operator, we consider the task of compressing partial differential equation (PDE) simulation data from fixed meshes. Here, we show that QuadConv can match the performance of standard discrete convolutions on uniform grid data by comparing a QuadConv autoencoder (QCAE) to a standard convolutional autoencoder (CAE). Further, we show that the QCAE can maintain this accuracy even on non-uniform data. In both cases, QuadConv also outperforms alternative unstructured convolution methods such as graph convolution.

Compression↗

The role of pre-existing heterogeneities in materials under shock and spall

There has been a challenge for many decades to understand how heterogeneities influence the behavior of materials under shock loading, eventually leading to spall formation and failure. Experimental, analytical, and computational techniques have matured to the point where systematic studies of materials with complex microstructures under shock loading and the associated failure mechanisms are feasible. This is enabled by more accurate diagnostics as well as characterization methods. As interest in complex materials grows, understanding and predicting the role of heterogeneities in determining the dynamic behavior becomes crucial. Early computational studies, hydrocodes, in particular, historically preclude any irregularities in the form of defects and impurities in the material microstructure for the sake of simplification and to retain the hydrodynamic conservation equations. Contemporary computational methods, notably molecular dynamics simulations, can overcome this limitation by incorporating inhomogeneities albeit at a much lower length and time scale. This review discusses literature that has focused on investigating the role of various imperfections in the shock and spall behavior, emphasizing mainly heterogeneities such as second-phase particles, inclusions, and voids under both shock compression and release. Pre-existing defects are found in most engineering materials, ranging from thermodynamically necessary vacancies, to interstitial and dislocation, to microstructural features such as inclusions, second phase particles, voids, grain boundaries, and triple junctions. This literature review explores the interaction of these heterogeneities under shock loading during compression and release. Systematic characterization of material heterogeneities before and after shock loading, along with direct measurements of Hugoniot elastic limit and spall strength, allows for more generalized theories to be formulated. Further, continuous improvement toward time-resolved, in situ experimental data strengthens the ability to elucidate upon results gathered from simulations and analytical models, thus improving the overall ability to understand and predict how materials behave under dynamic loading.

36 MATERIALS SCIENCE↗

2015 IC Report: w14_biosurveillance [Slides]

Our primary use of the institutional computing clusters has been for Wikipedia-based disease surveillance and forecasting work. We've expanded breadth: the number of diseases and locations that can be surveilled using internet data. We've expanded forecasting: surprisingly little work towards forecasting disease spread with quantified uncertainty. We've expanded big data processing techniques: the Wikipedia access logs are 5TB of compressed text; we're working on efficient ways to process and store time series. Our work received significant media coverage and was highlighted as LANL's best science story of 2014.

59 BASIC BIOLOGICAL SCIENCES↗

Shock compression of fused silica: An impedance matching standard

The properties of silica (SiO 2 ) at extreme conditions have important applications for planetary processes and for high pressure research. Here, we report the results of 125 plate impact shock compression experiments on fused silica spanning 200–1100 GPa using the Z machine at Sandia National Laboratories. Additionally, we present a complementary set of density functional theory based molecular dynamics calculations based on an amorphous reference state that extend the Hugoniot to 2500 GPa. We find good agreement between the Z data, extant laser driven shock compression experiment data, and computational results over most of the pressure range. With these results, fused silica can be used as a new impedance matching standard for shock compression experiments.

54 ENVIRONMENTAL SCIENCES↗

Exacution: Enhancing Scientific Data Management for Exascale

As we continue toward exascale, scientific data volume is continuing to scale and becoming more burdensome to manage. In this paper, we lay out opportunities to enhance state of the art data management techniques. We emphasize well-principled data compression, and using it to achieve progressive refinement. This can both accelerate I/O and afford the user increased flexibility when she interacts with the data. The formulation naturally maps onto enabling partitioning of the progressively improving-quality representations of a data quantity into different media-type destinations, to keep the highest priority information as close as possible to the computation, and take advantage of deepening memory/storage hierarchies in ways not previously possible. Careful monitoring is requisite to our vision, not only to verify that compression has not eliminated salient features in the data, but also to better understand the performance of massively parallel scientific applications. Increased mathematical rigor would be ideal,to help bring compression on a better-understood theoretical footing, closer to the relevant scientific theory, more aware of constraints imposed by the science, and more tightly error-controlled. Throughout, we highlight pathfinding research we have begun exploring related these topics, and comment toward future work that will be needed.

97 MATHEMATICS AND COMPUTING↗

ISABELA for effective in situ compression of scientific data: ISABELA FOR EFFECTIVE IN-SITU REDUCTION OF SPATIO-TEMPORAL DATA

Exploding dataset sizes from extreme-scale scientific simulations necessitates efficient data management and reduction schemes to mitigate I/O costs. With the discrepancy between I/O bandwidth and computational power, scientists are forced to capture data infrequently, thereby making data collection an inherently lossy process. Although data compression can be an effective solution, the random nature of real-valued scientific datasets renders lossless compression routines ineffective. These techniques also impose significant overhead during decompression, making them unsuitable for data analysis and visualization, which require repeated data access. To address this problem, we propose an effective method for In situ Sort-And-B-spline Error-bounded Lossy Abatement (ISABELA) of scientific data that is widely regarded as effectively incompressible. With ISABELA, we apply a pre-conditioner to seemingly random and noisy data along spatial resolution to achieve an accurate fitting model that guarantees a ≥0.99 correlation with the original data. We further take advantage of temporal patterns in scientific data to compress data b ≈ 85%, while introducing only a negligible overhead on simulations in terms of runtime. ISABELA significantly outperforms existing lossy compression methods, such as wavelet compression, in terms of data reduction and accuracy. We extend upon our previous paper by additionally building a communication-free, scalable parallel storage framework on top of ISABELA-compressed data that is ideally suited for extreme-scale analytical processing. The basis for our storage framework is an inherently local decompression method (it need not decode the entire data), which allows for random access decompression and low-overhead task division that can be exploited over heterogeneous architectures. Furthermore, analytical operations such as correlation and query processing run quickly and accurately over data in the compressed space. Copyright © 2012 John Wiley & Sons, Ltd.

Computer Science↗

The cold equation of state of tantalum

In high-pressure isentropic compression experiments (ICE), the pressure is dominated by the cold curve. In order to obtain an accurate semi-empirical cold curve for Ta, we calculate the thermal pressure from ab initio phonon and electronic excitation spectra. The cold curve is then inferred from ultrasonic and shock data. Our empirical cold pressure is compared to density functional calculations and found to be closer to GGA results at low pressure and to approach LDA at high pressure.

36 MATERIALS SCIENCE↗

LiveCheckHSI: a Hardware/Software Co-verification Tool for Hyperspectral Imaging Systems with Embedded System-on-Chip Instrument Avionics

The emergent technology of system-on-chip (SoC) devices promises lighter, smaller, cheaper, and more capable and reliable space electronic systems that could help to unveil some of the most treasured secrets in our universe. This technology is an improvement over the technology that is currently used in space applications, which lags behind state-of-the-art commercial-off-the-shelf (COTS) equipment by several generations. SoC technology integrates all computational power required by next-generation space exploration science instruments onto a single chip. This paper describes hardware/software co-verification tools for the Xilinx Zynq-based control and data handling system that have been developed at the Jet Propulsion Laboratory (JPL) for visible-infrared imaging spectrometers. The system acquires and compresses images in real-time, in addition to programming the spectrometer (frame rate, exposure time), focus step motor, and heaters and reporting telemetry.

Dolman, David↗

Extending Skel to Support the Development and Optimization of Next Generation I/O Systems

As the memory and storage hierarchy get deeper and more complex, it is important to have new benchmarks and evaluation tools that allow us to explore the emerging middleware solutions to use this hierarchy. Skel is a tool aimed at automating and refining this process of studying HPC I/O performance. It works by generating application I/O kernel/benchmarks as determined by a domain-specific model. This paper provides some techniques for extending Skel to address new situations and to answer new research questions. For example, we document use cases as diverse as using Skel to troubleshoot I/O performance issues for remote users, refining an I/O system model, and facilitating the development and testing of a mechanism for runtime monitoring and performance analytics. We also discuss data oriented extensions to Skel to support the study of compression techniques for Exascale scientific data management.

97 MATHEMATICS AND COMPUTING↗

R-Adaptivity to Enable Compression of Elementary Computations in Extreme-Scale Finite Element Simulators

Modern computing systems are capable of exascale calculations, which are revolutionizing the development and application of high-fidelity numerical models in computational science and engineering. While these systems continue to grow in processing power, the available system memory has not increased commensurately, and electrical power consumption continues to grow. A predominant approach to limit the memory usage in large-scale applications is to exploit the abundant processing power and continually recompute many low-level simulation quantities, rather than storing them. However, this approach can adversely impact the throughput of the simulation and diminish the benefits of modern computing architectures. We present three novel contributions to reduce the memory burden while maintaining, and sometimes improving, performance in simulations based on finite element discretizations. The first contribution develops dictionary-based data compression schemes that detect and exploit the structure of the discretization, due to redundancies across the finite element mesh. While these schemes are shown to reduce memory requirements by more than 99% on meshes with large numbers of identical mesh cells, there are applications where this structure does not exist. The second contribution leverages a recently developed augmented Lagrangian optimization algorithm to enable r-adaptivity for meshes with the goal of enhancing the redundancies in the mesh. The third contribution extends these methods to patch-based linear solvers and preconditioners by compressing local matrices. Numerical results demonstrate the effectiveness of the proposed methods to detect, enhance and exploit mesh structure on a suite of examples inspired by large-scale applications.

97 MATHEMATICS AND COMPUTING↗

Optimizing Management of Persistent Data Structures in High-Performance Analytics

Large-scale data analytics workflows ingest massive input data into various data structures, including graphs and key-value datastores. These data structures undergo multiple transformations and computations and are typically reused in incremental and iterative analytics workflows. Persisting in-memory views of these data structures enables reusing them beyond the scope of a single program run while avoiding repetitive raw data ingestion overheads. Memory-mapped I/O enables persisting in-memory data structures without data serialization and deserialization overheads. However, memory-mapped I/O lacks the key feature of persisting consistent snapshots of these data structures for incremental ingestion and processing. The obstacles to efficient virtual memory snapshots using memory-mapped I/O include background writebacks outside the application’s control, and the significantly high storage footprint of such snapshots. To address these limitations, we present Privateer, a memory and storage management tool that enables storage-efficient virtual memory snapshotting while also optimizing snapshot I/O performance. Here, we integrated Privateer into Metall, a state-of-the-art persistent memory allocator for C++, and the Lightning Memory-Mapped Database (LMDB), a widely-used key-value datastore in data analytics and machine learning. Privateer optimized application performance by 1.22× when storing data structure snapshots to node-local storage, and up to 16.7× when storing snapshots to a parallel file system. Privateer also optimizes storage efficiency of incremental data structure snapshots by up to 11× using data deduplication and compression.

Computer science↗

Automated phase segmentation and quantification of high-resolution TEM image for alloy design

In the alloy design and development process, a wealth of atomically resolved structural high-resolution transmission electron microscopy (HRTEM) images are produced. Identifying the different nano-precipitate phases and tracking their evolution under various compositions and during manufacturing or post-processing requires hundreds of HRTEM images and thousands of precipitates. The nanoscopic phase information labeling and analysis purely relies on humans are prohibitively costly and time-consuming, sometimes not reliable because of the lack of authoritative knowledge. Here, in this work, we develop a novel unsupervised machine learning approach coupled with adaptive computer vision techniques with features in the Fourier space to automatically determine the number of phases and segment/quantify the phases with nanoscale resolution, allowing for quantitative correlation between nanostructure formation, processing and functional properties. To automate the phase extraction/quantification and ascertain its applicability, we have applied the developed framework to the HRTEM images from several alloy systems, processing conditions, image magnifications, and phase types and morphologies (precipitates, nano-twins, stacking faults, crystalline matrix, and amorphous structures) for verification. This study paves the road for compression, visualization, and translation of raw image structural data into physically relevant information in real-time with minimal human supervision. It shows the promise of enabling high-throughput materials characterization for the acceleration of alloy manufacturing and design.

36 MATERIALS SCIENCE↗

LDRD 226360 Final Project Report: Simulated X-ray Diffraction and Machine Learning for Optimizing Dynamic Experiment Analysis

This report is the final documentation for the one-year LDRD project 226360: Simulated X-ray Diffraction and Machine Learning for Optimizing Dynamic Experiment Analysis. As Sandia has successfully developed in-house X-ray diffraction tools for study of atomic structure in experiments, it has become increasingly important to develop computational analysis methods to support these experiments. When dynamically compressed lattices and orientations are not known a priori, the identification requires a cumbersome and sometimes intractable search of possible final states. These final states can include phase transition, deformation and mixed/evolving states. Our work consists of three parts: (1) development of an XRD simulation tool and use of traditional data science methods to match XRD patterns to experiments; (2) development of ML-based models capable of decomposing and identifying the lattice and orientation components of multicomponent experimental diffraction patterns; and (3) conducting experiments which showcase these new analysis tools in the study of phase transition mechanisms. Our target material has been cadmium sulfide, which exhibits complex orientation-dependent phase transformation mechanisms. In our current one-year LDRD, we have begun the analysis of high-quality c-axis CdS diffraction data from DCS and Thor experiments, which had until recently eluded orientation identification.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Transform coding for space applications

Data compression coding requirements for aerospace applications differ somewhat from the compression requirements for entertainment systems. On the one hand, entertainment applications are bit rate driven with the goal of getting the best quality possible with a given bandwidth. Science applications are quality driven with the goal of getting the lowest bit rate for a given level of reconstruction quality. In the past, the required quality level has been nothing less than perfect allowing only the use of lossless compression methods (if that). With the advent of better, faster, cheaper missions, an opportunity has arisen for lossy data compression methods to find a use in science applications as requirements for perfect quality reconstruction runs into cost constraints. This paper presents a review of the data compression problem from the space application perspective. Transform coding techniques are described and some simple, integer transforms are presented. The application of these transforms to space-based data compression problems is discussed. Integer transforms have an advantage over conventional transforms in computational complexity. Space applications are different from broadcast or entertainment in that it is desirable to have a simple encoder (in space) and tolerate a more complicated decoder (on the ground) rather than vice versa. Energy compaction with new transforms are compared with the Walsh-Hadamard (WHT), Discrete Cosine (DCT), and Integer Cosine (ICT) transforms.

Glover, Daniel↗

Understanding the Effects of Compression and Constraints on Water Uptake of Fuel-Cell Membranes

Accurate characterization of polymer-electrolyte fuel cells (PEFCs) requires understanding the impact of mechanical and electrochemical loads on cell components. An essential aspect of this relationship is the effect of compression on the polymer membrane?s water-uptake behavior and transport properties. However, there is limited information on the impact of physical constraints on membrane properties. In this paper, we investigate both theoretically and experimentally how the water uptake of Nafion membrane changes under external compression loads. The swelling of a compressed membrane is modeled by modifying the swelling pressure in the polymer backbone which relies on the changes in the microscopic volume of the polymer. The model successfully predicts the water content of the compressed membrane measured through in-situ swelling-compression tests and neutron imaging. The results show that external mechanical loads could reduce the water content and conductivity of the membrane, especially at lower temperatures, higher humidities, and in liquid water. The modeling framework and experimental data provide valuable insight for the swelling and conductivity of constrained and compressed membranes, which are of interest in electrochemical devices such as batteries and fuel cells.

36 MATERIALS SCIENCE↗