Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24

Variable Precision Computing

This report summarizes the activities and major accomplishments of the Variable Precision Computing Strategic Initiative project. The overarching goal of this project was to initiate and promote a new paradigm in High Performance Computing (HPC) that would fundamentally change how we represent and make use of representations of real numbers in finite precision and that would establish Lawrence Livermore National Laboratory (LLNL) as a leader in precision-related research for HPC. We pursued three integrated and concurrent research thrusts: new, more dynamic data representations to address data motion and capacity limitations; improved mixed precision algorithms to accelerate computation and to leverage new hardware; and new tools to help developers reason about precision and to automate code transformations. Within these thrust areas, we made significant advances in the use of compressed array data types in numerical calculations, in the theoretical justification, in improved performance, and in the prospects of hardware implementation; in the development of metrics to significantly reduce the storage needs of molecular dynamics simulations; in the development and analysis of efficient mixed-precision algorithms for time-dependent partial differential equations and for orthogonal factorization; and in the capability of tools that can analyze sensitivity of computed results to choices in precision and that can make automated code transformations based on such analysis. We established that there are definite opportunities to realize significant improvements in HPC above and beyond the 2x limits of traditional mixed precision using the technologies we have developed. The project produced numerous papers, presentations, and posters in important venues as well as open-source software contributions to the community, and many of the new capabilities have initiated or are being leveraged in ongoing projects.

97 MATHEMATICS AND COMPUTING↗

FY08 LDRD Final Report LOCAL: Locality-Optimizing Caching Algorithms and Layouts

This project investigated layout and compression techniques for large, unstructured simulation data to reduce bandwidth requirements and latency in simulation I/O and subsequent post-processing, e.g. data analysis and visualization. The main goal was to eliminate the data-transfer bottleneck - for example, from disk to memory and from central processing unit to graphics processing unit - through coherent data access and by trading underutilized compute power for effective bandwidth and storage. This was accomplished by (1) designing algorithms that both enforce and exploit compactness and locality in unstructured data, and (2) adapting offline computations to a novel stream processing framework that supports pipelining and low-latency sequential access to compressed data. This report summarizes the techniques developed and results achieved, and includes references to publications that elaborate on the technical details of these methods.

99 GENERAL AND MISCELLANEOUS↗

Shockless compression and release behavior of beryllium to 110 GPa

A magnetohydrodynamic loading technique was used to shocklessly compress beryllium to peak longitudinal stresses of 19–110 GPa and, subsequently, unload in order to determine both the compressive response and also the shear stress supported upon release. Loading strain rates were on the order of 10{sup 6 }s{sup −1}, while the unloading rates were nearly constant at 3 × 10{sup 5 }s{sup −1}. Velocimetry was used to monitor the ramp and release behavior of a beryllium/lithium fluoride window interface. After applying window corrections to infer in situ beryllium velocities, a Lagrangian analysis was employed to determine the material response. The Lagrangian wavespeed-particle velocity response is integrated to generate the stress-strain path, average change in shear stress over the elastic unloading, and estimates of the shear modulus at peak compression. These data are used to infer the pressure dependence of the flow strength at the unloading rate. Comparisons to several strength models reveal good agreement to 45 GPa, but the data indicate 20%–30% higher strength near 100 GPa.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Detection of Coherent Structures in Extreme-Scale Simulations

The analysis of coherent structures is a common problem in many scientific domains ranging from astrophysics to combustion, fusion, and materials science. The data from three-dimensional simulations are analyzed to detect the structures, extract statistics on them, and track them over time to gain insights into the phenomenon being modeled. This analysis is typically done off-line, using data that have been written out by the simulations. However, the move towards extreme scale architectures, with multi-core processors and graphical processing units, will affect how such analysis is done as it is unlikely that the systems will support the I/O bandwidth required for off-line analysis. Moving the analysis in-situ is a solution only if we know a priori what analysis will be done, as well as the algorithms used and their parameter settings. Even then, we need to ensure that this will not substantially increase the memory requirements or the data movement as the former will be limited and the latter will be expensive. In the Exa-DM project, a collaboration between Lawrence Livermore National Laboratory and University of Minnesota, we are exploring ways in which we can address the conflicting demands of coherent structure analysis of simulation data and the architecture of modern parallel systems, while enabling scientific discovery at the exascale. In this paper, we describe our work in two areas: the in situ implementation of an existing algorithm for coherent structure analysis and the use of graph-based techniques to efficiently compress the data.

79 ASTRONOMY AND ASTROPHYSICS↗

Optimization of SAMtools sorting using OpenMP tasks

SAMtools is a widely-used genomics application for post-processing high-throughput sequence alignment data. Such sequence alignment data are commonly sorted to make downstream analysis more efficient. However, this sorting process itself can be computationally- and I/O-intensive: high-throughput sequence alignment files in the de facto standard binary alignment/map (BAM) format can be many gigabytes in size, and may need to be decompressed before sorting and compressed afterwards. As a result, BAM-file sorting can be a bottleneck in genomics workflows. Here, this paper describes a case study on the performance analysis and optimization of SAMtools for sorting large BAM files. OpenMP task parallelism and memory optimization techniques resulted in a speedup of 5.9X versus the upstream SAMtools 1.3.1 for an internal (in-memory) sort of 24.6 GiB of compressed BAM data (102.6 GiB uncompressed) with 32 processor cores, while a 1.98X speedup was achieved for an external (out-of-core) sort of a 271.4 GiB BAM file.

97 MATHEMATICS AND COMPUTING↗

Hugoniot Models for Na and LiF from LEOS

In this document, we provide the Hugoniot for sodium from two models: LEOS table L110 and Lynx table 110. We also provide the Hugoniot for lithium fluoride from LEOS (L2240) and Lynx (2240). The Hugoniot pressures are supplied for temperatures between 338.0 and 1.16×10 9 Kelvin and densities between 0.968 and 11.5 g/cc. These LEOS models were developed by the quotidian EOS methodology, which is a widely used and robust method for producing tabular EOS data. Tables list the model data for LEOS 110, Lynx 110, LEOS 2240, and Lynx 2240. The Lynx models follow the same methodology as the LEOS models; however, the Purgatorio average-atom DFT code was used to compute the electron thermal part of the EOS. The models for Lynx are only listed at high compression due to known issues with the Lynx library at lower pressures.

36 MATERIALS SCIENCE↗

Custom NEON AOP reflectance mosaics and maps of shade masks, canopy water content

This mosaic of visible to shortwave infrared (VSWIR) data was derived from the assignable asset NEON AOP radiance data that was collected by LBNL’s Watershed Function SFA during the summer of 2018 (DOI: 10.15485/1617204). This atmospheric correction was completed to take into account site-specific terrain variability in the 334 km2 survey area centered around Crested Butte, CO. The atmospheric correction was completed using ACORN atmospheric correction software executed on 200 x 200 pixel kernels rather than line by line in order to capture local flight altitude conditions. Manual cloud delineation removed any small shaded areas that occurred within the data collection areas. Mosaics were developed based on preference for days that were in close timing proximity to the coincident ground campaign and with mosaicing criteria that minimized the angle between the sun and the sensor to retrieve the most consistent reflectance (min_phase_refl_tiled). Finally, we generated shade masks based on the geometry between the sun angle at time of flight, the ground surface, and the sensor. Here we provide the orthomosaiced estimated reflectance data (min_phase_refl_tiled.tif), canopy water content (wtrl), estimated atmospheric water vapor (wtrv), the observational data (obs) for this particular mosaic, the estimated visibility (vis), shade masks derived from the digital surface elevation model (shade) and the digital terrain model (shade_tch) added to the canopy height model (DOI: 10.15485/1617203), and a wavelength metadata file. Each image file is provided as a GeoTiff with internal tiling and LZW compression. These data can also be found on Google Earth Engine for extraction, download, and analysis of smaller extents (Google Earth Engine data is linked in external link table).

2018 NEON and 2025 CHESS Campaigns↗

Spatiotemporally Adaptive Compression for Scientific Dataset with Feature Preservation – A Case Study on Simulation Data with Extreme Climate Events Analysis

Scientific discoveries are increasingly constrained by limited storage space and I/O capacities. For time-series simulations and experiments, their data often need to be decimated over timesteps to accommodate storage and I/O limitations. In this paper, we propose a technique that addresses storage costs while improving post-analysis accuracy through spatiotemporal adaptive, error-controlled lossy compression. We investigate the trade-off between data precision and temporal output rates, revealing that reducing data precision and increasing timestep frequency lead to more accurate analysis outcomes. Additionally, we integrate spatiotemporal feature detection with data compression and demonstrate that performing adaptive error-bounded compression in higher dimensional space enables greater compression ratios, leveraging the error propagation theory of a transformation-based compressor. To evaluate our approach, we conduct experiments using the well-known E3SM climate simulation code and apply our method to compress variables used for cyclone tracking. Our results show a significant reduction in storage size while enhancing the quality of cyclone tracking analysis, both quantitatively and qualitatively, in comparison to the prevalent timestep decimation approach. Compared to three state-of-the-art lossy compressors lacking feature preservation capabilities, our adaptive compression framework improves perfectly matched cases in TC tracking by 26.4-51.3% at medium compression ratios and by 77.3-571.1% at large compression ratios, with a merely 5–11% computational overhead.

Gong, Qian↗

Principal Component Analysis of Spectroscopic Imaging Data in Scanning Probe Microscopy

The approach for data analysis in band excitation family of scanning probe microscopies based on principal component analysis (PCA) is explored. PCA utilizes the similarity between spectra within the image to select the relevant response components. For small signal variations within the image, the PCA components coincide with the results of deconvolution using simple harmonic oscillator model. For strong signal variations, the PCA allows effective approach to rapidly process, de-noise and compress the data. The extension of PCA for correlation function analysis is demonstrated. The prospects of PCA as a universal tool for data analysis and representation in multidimensional SPMs are discussed.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Direct prediction of inelastic neutron scattering spectra from the crystal structure*

Abstract Inelastic neutron scattering (INS) is a powerful technique to study vibrational dynamics of materials with several unique advantages. However, analysis and interpretation of INS spectra often require advanced modeling that needs specialized computing resources and relevant expertise. This difficulty is compounded by the limited experimental resources available to perform INS measurements. In this work, we develop a machine-learning based predictive framework which is capable of directly predicting both one-dimensional INS spectra and two-dimensional INS spectra with additional momentum resolution. By integrating symmetry-aware neural networks with autoencoders, and using a large scale synthetic INS database, high-dimensional spectral data are compressed into a latent-space representation, and a high-quality spectra prediction is achieved by using only atomic coordinates as input. Our work offers an efficient approach to predict complex multi-dimensional neutron spectra directly from simple input; it allows for improved efficiency in using the limited INS measurement resources, and sheds light on building structure-property relationships in a variety of on-the-fly experimental data analysis scenarios.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Cascade of transitions in twisted and non-twisted graphene layers within the van Hove scenario

Abstract Motivated by measurements of compressibility and STM spectra in twisted bilayer graphene, we analyze the pattern of symmetry breaking for itinerant fermions near a van Hove singularity. Making use of an approximate SU(4) symmetry of the Landau functional, we show that the structure of the spin/isospin order parameter changes with increasing filling via a cascade of transitions. We compute the feedback from different spin/isospin orders on fermions and argue that each order splits the initially 4-fold degenerate van Hove peak in a particular fashion, consistent with the STM data and compressibility measurements, providing a unified interpretation of the cascade of transitions in twisted bilayer graphene. Our results follow from a generic analysis of an SU(4)-symmetric Landau functional and are valid beyond a specific underlying fermionic model. We argue that an analogous van Hove scenario explains the cascade of phase transitions in non-twisted Bernal bilayer and rhombohedral trilayer graphene.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Mesoscale Numerical Characterization of Kevlar and Carbon–Kevlar Hybrid Plain-Woven Fabric Compression Behavior

A mesoscale finite element model (FEM) of the plain-woven fabric representative elementary volume (REV/unit cell) is developed. The developed FEM is used to predict the nonlinear compaction behavior of monolithic and intraply hybrid fabric made of high-strength carbon and Kevlar fibers. For geometrical modeling of the fabric REV, TexGen, an open-source geometrical modeling software (developed at Nottingham University), is utilized, and for mechanical characterization, FE simulation tool Abaqus{sup ®} is used. Geometrical nonlinearity of the fabric, including periodic boundary conditions and yarn interactions, is incorporated in the REV FE model. Material nonlinearity caused by transversely isotropic behavior of fibrous yarns is implemented through the user material subroutine form in FE simulation along with local orientation assigned to each yarn in the REV. Material properties of fibers are extracted from the literature and supplier data. The compression behavior of monolithic Kevlar and hybrid carbon–Kevlar (carbon warp and Kevlar weft yarns, C–K) fabric is predicted using the proposed FE model. FEM-predicted results were found to have good agreement with the experimentally obtained ones. Results depict that C–K hybrid fabrics have improved compression strength. Hybridization effect is governed by yarns mutual interaction, yarn area at crossovers, and transverse in-plane properties of the interyarn hybridized fabric. The proposed FE model can be used to predict the mechanical behavior of woven fabrics with different weaving patterns, unit cell geometry, material properties, and loading conditions.

36 MATERIALS SCIENCE↗

Analysis and compression of six-dimensional gyrokinetic datasets using higher order singular value decomposition

Higher order singular value decomposition (HOSVD) is explored as a tool for analyzing and compressing gyrokinetic data. An efficient numerical implementation of an HOSVD algorithm is described. HOSVD is used to analyze the full six-dimensional (three spatial, two velocity space, and time dimensions) gyrocenter distribution function from gyrokinetic simulations of ion temperature gradient, electron temperature gradient, and trapped electron mode driven turbulence. The HOSVD eigenvalues for the velocity space coordinates decay very rapidly, indicating that only a few structures in velocity space can capture the most important dynamics. In almost all of the cases studied, HOSVD extracts parallel velocity space structures which are very similar to orthogonal polynomials. HOSVD is also used to compress gyrokinetic datasets, an application in which it is shown to significantly outperform the more commonly used singular value decomposition. It is shown that the effectiveness of the HOSVD compression improves as the dimensionality of the dataset increases. (C) 2012 Elsevier Inc. All rights reserved.

equation↗

Understanding the Hierarchy of Dense Subgraphs in Stationary and Temporally Varying Setting

Graphs are widely used to model relationships in a wide variety of domains such as sociology, bioinformatics, infrastructure, the WWW, to name a few. One of the key observations is that while real-world graphs are often globally sparse, they are locally dense. In other words, the average degree is often quite small (say at most 10 in a million vertex graph), but vertex neighborhoods are often dense. Finding dense subgraphs is a critical aspect of graph mining It has been used for finding communities and spam link farms in web graphs, graph visualization, real-time story identification, DNA motif detection in biological networks, finding correlated genes, epilepsy prediction, finding price value motifs in financial data, graph compression, distance query indexing, and increasing the throughput of social networking site servers. However, most standard formulations of this problem (like clique, quasi-clique, k-densest subgraph) are NP-hard. Furthermore, current dense subgraph finding algorithms usually optimize some objective, and only find a few such subgraphs without providing any structural relations, whereas the goal is rarely to find the "true optimum," but to identify many (if not all) dense substructures, understand their distribution in the graph, and ideally determine relationships among them. In this project, we first aim to devise algorithms and provide 3 implementations with nice visualizations to find the hierarchy between dense subgraphs, and then understand the structure of the hierarchy to gain more insight on the hidden patterns in real-world networks. Another important aspects in graph analysis is the temporal nature of networks. Networks evolve over time and in many applications data arrives at a high velocity, and thus it is important to design algorithms that can process data efficiently. We report three main results towards identifying dense structures in large evolving graphs. First, we will show how the hierarchical connectedness structure can be maintained efficiently, where connectedness is defined by increasing levels of connectivity strength. Next, we present dense structure can be identified in bipartite graphs without building projection graphs. And finally, we present a new method for peeling algorithms This new approach avoids sequential nature of peeling algorithms and is amenable to parallelization, which is crucial for processing high velocity data.

97 MATHEMATICS AND COMPUTING↗

ROOT I/O compression improvements for HEP analysis

We overview recent changes in the ROOT I/O system, increasing performance and enhancing it and improving its interaction with other data analysis ecosystems. Both the newly introduced compression algorithms, the much faster bulk I/O data path, and a few additional techniques have the potential to significantly to improve experiment's software performance. The need for efficient lossless data compression has grown significantly as the amount of HEP data collected, transmitted, and stored has dramatically increased during the LHC era. While compression reduces storage space and, potentially, I/O bandwidth usage, it should not be applied blindly: there are significant trade-offs between the increased CPU cost for reading and writing files and the reduce storage space.

Shadura, Oksana↗

Poisson’s ratio of eTPU molded bead foams in compression via in situ synchrotron X-ray microtomography

In situ synchrotron X-ray microtomography was used to characterize the bulk deformation behavior by computing the Poisson's ratio of expanded thermoplastic polyurethane (eTPU) molded bead foams used in footwear midsole during compression. Quantitative data on morphological characteristics were obtained using an iterative image processing workflow. Image correlation on the 4D datasets using DVC was performed to calculate the volumetric and axial strain to estimate the Poisson ratio. Strain maps from DVC showed the influence of variability in ligament thickness distribution on the global mechanical behavior exhibited which dominated the response seen in these bead foams. Lastly, our results showed a strong correlation between Poisson ratio and distribution of ligament thickness in foams.

36 MATERIALS SCIENCE↗

Autoignition behavior of unsaturated hydrocarbons in the low and high temperature regions

In this work, numerical and experimental techniques are used to investigate the effect of the position of the double bond on the ignition properties of pentene and hexene linear isomers. A wide-range kinetic model for the oxidation of C{sub 5}-C{sub 6} linear alkenes has been developed. Literature rapid compression machine data were used to validate the model at low temperatures and new shock tube experiments were performed in order to assess the behavior of the considered alkenes in the high temperature region. Some interesting inversions in the relative reactivity of the isomers were detected. The model successfully reproduced the measured behavior and allowed to explain the reason of these reactivity changes. The information gathered will be applied to the development of the kinetic mechanisms of larger unsaturated surrogate components.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Parallel Tensor Compression for Large-Scale Scientific Data.

As parallel computing trends towards the exascale, scientific data produced by high-fidelity simulations are growing increasingly massive. For instance, a simulation on a three-dimensional spatial grid with 512 points per dimension that tracks 64 variables per grid point for 128 time steps yields 8 TB of data. By viewing the data as a dense five way tensor, we can compute a Tucker decomposition to find inherent low-dimensional multilinear structure, achieving compression ratios of up to 10000 on real-world data sets with negligible loss in accuracy. So that we can operate on such massive data, we present the first-ever distributed memory parallel implementation for the Tucker decomposition, whose key computations correspond to parallel linear algebra operations, albeit with nonstandard data layouts. Our approach specifies a data distribution for tensors that avoids any tensor data redistribution, either locally or in parallel. We provide accompanying analysis of the computation and communication costs of the algorithms. To demonstrate the compression and accuracy of the method, we apply our approach to real-world data sets from combustion science simulations. We also provide detailed performance results, including parallel performance in both weak and strong scaling experiments.

59 BASIC BIOLOGICAL SCIENCES↗