Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,099 records · Page 61
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Application of the van der Pauw method for electrical conductivity measurements at high temperatures using an insulating compressing ring

High quality data on the high temperature electrical properties of ceramics, particularly oxides, is of great value for material selection, design, and modeling for a broad range of emerging applications. Utilizing the mismatch in the coefficient of thermal expansion between two materials, a purely mechanical method for establishing electrical contact in the van der Pauw geometry to measure the bulk resistivity of ceramic disks at high temperatures is presented. Measurements of a reference material, 20 mol. % Gd-doped cerium oxide, are presented up to 1000 °C. The viability of electrical measurements up to a maximum temperature of 1600 °C is also considered. Measurements are performed using multiple techniques and compared to literature values finding excellent agreement. Finally, the approach described in this work enables the van der Pauw method to be applied to many ceramic materials over a wide range of temperatures and environments.

47 OTHER INSTRUMENTATION↗
NASA NTRS1992

The dynamic cusp

Results of a study of satellite and ground-based data acquired near the dayside cusp during a period when the IMF was inferred to have a strong northward component are presented. The presence of Pc 1 bursts, Pc 4-5 pulsations, and a tailward traveling twin vortex pattern of ionospheric convection suggests that the magnetosphere may have been temporarily compressed. Magnetic field data acquired at a synchronous altitude from GOES 5 and on the ground from Huanacayo support this suggestion. Plasma with ion dispersion characteristics associated with a cusp during southward IMF was detected by Viking over a 3.5-deg range of latitude. The presence of standing Alfven waves and ring current ions suggests that this 'cusplike' plasma was observed on closed geomagnetic field lines. It is suggested that the magnetosphere, during a northward IMF, is temporarily compressed by a solar wind pressure enhancement that produces the Pc 1 bursts, Pc 4-5 pulsations, and ionospheric vortices.

Potemra, T. A.↗
DOE OSTI2021

Technical Report: Accessor Realization

As we have seen the arithmetic power of hardware architectures growing much faster than the memory bandwidth, there is an urgent need to rethink the way we operate, communicate, and store data. A promising strategy for overcoming the “memory wall” is the decoupling of memory precision from arithmetic precision as proposed in Anzt et al.. Such a decoupling of memory precision and arithmetic precision requires a “memory accessor” that handles the conversion between formats (resp. compression of the data) on the fly in registers. For such a memory accessor to be useful in practice, it has to combine multiple characteristics.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Scalable Incremental Checkpointing using GPU-Accelerated De-Duplication

Writing large amounts of data concurrently to stable storage is a typical I/O pattern of many HPC workflows. This pattern introduces high I/O overheads and results in increased storage space utilization especially for workflows that need to capture the evolution of data structures with high frequency as checkpoints. In this context, many applications, such as graph pattern matching, perform sparse updates to large data structures between checkpoints. For these applications, incremental checkpointing techniques that save only the differences from one checkpoint to another can dramatically reduce the checkpoint sizes, I/O bottlenecks, and storage space utilization. However, such techniques are not without challenges: it is non-trivial to transparently determine what data has changed since a previous checkpoint and assemble the differences in a compact fashion that does not result in excessive metadata. State-of-art data reduction techniques (e.g., compression and de-duplication) have significant limitations when applied to modern HPC applications that leverage GPUs: slow at detecting the differences, generate a large amount of metadata to keep track of the differences, and ignore crucial spatiotemporal checkpoint data redundancy. This paper addresses these challenges by proposing a Merkle tree-based incremental checkpointing method to exploit GPUs' high memory bandwidth and massive parallelism. Experimental results at scale show a significant reduction of the I/O overhead and space utilization of checkpointing compared with state-of-the-art incremental checkpointing and compression techniques.

Tan, Nigel↗
DOE OSTI2020

A data-driven approach to sampling matrix selection for compressive sensing

Sampling is a fundamental aspect of any implementation of compressive sensing. Typically, the choice of sampling method is guided by the reconstruction basis. However, this approach can be problematic with respect to certain hardware constraints and is not responsive to domain-specific context. We propose a method for defining an order for a sampling basis that is optimal with respect to capturing variance in data, thus allowing for meaningful sensing at any desired level of compression. We focus on the Walsh-Hadamard sampling basis for its relevance to hardware constraints, but our approach applies to any sampling basis of interest. We illustrate the effectiveness of our method on the Physical Sciences Inc. Fabry-Perot interferometer sensor multispectral dataset, the Johns Hopkins Applied Physics Lab FTIR-based longwave infrared sensor hyperspectral dataset, and a Colorado State University Swiss Ranger depth image dataset. The spectral datasets consist of simulant experiments, including releases of chemicals such as GAA and SF6. We combine our sampling and reconstruction with the adaptive coherence estimator (ACE) and bulk coherence for chemical detection and we incorporate an algorithmic threshold for ACE values to determine the presence or absence of a chemical. We compare results across sampling methods in this context. We have successful chemical detection at a compression rate of 90%. For all three datasets, we compare our sampling approach to standard orderings of sampling basis such as random, sequency, and an analog of sequency that we term `frequency.' In one instance, the peak signal to noise ratio was improved by over 30% across a test set of depth images.

compressive sensing, hyperspectral imaging, Walsh-↗
NASA NTRS1997

Motion video compression system with neural network having winner-take-all function

A motion video data system includes a compression system, including an image compressor, an image decompressor correlative to the image compressor having an input connected to an output of the image compressor, a feedback summing node having one input connected to an output of the image decompressor, a picture memory having an input connected to an output of the feedback summing node, apparatus for comparing an image stored in the picture memory with a received input image and deducing therefrom pixels having differences between the stored image and the received image and for retrieving from the picture memory a partial image including the pixels only and applying the partial image to another input of the feedback summing node, whereby to produce at the output of the feedback summing node an updated decompressed image, a subtraction node having one input connected to received the received image and another input connected to receive the partial image so as to generate a difference image, the image compressor having an input connected to receive the difference image whereby to produce a compressed difference image at the output of the image compressor.

Fang, Wai-Chi↗
NASA NTRS1994

Numerical Simulation of Supersonic Compression Corners and Hypersonic Inlet Flows Using the RPLUS2D Code

A two-dimensional computational code, PRLUS2D, which was developed for the reactive propulsive flows of ramjets and scramjets, was validated for two-dimensional shock-wave/turbulent-boundary-layer interactions. The problem of compression corners at supersonic speeds was solved using the RPLUS2D code. To validate the RPLUS2D code for hypersonic speeds, it was applied to a realistic hypersonic inlet geometry. Both the Baldwin-Lomax and the Chien two-equation turbulence models were used. Computational results showed that the RPLUS2D code compared very well with experimentally obtained data for supersonic compression corner flows, except in the case of large separated flows resulting from the interactions between the shock wave and turbulent boundary layer. The computational results compared well with the experiment results in a hypersonic NASA P8 inlet case, with the Chien two-equation turbulence model performing better than the Baldwin-Lomax model.

Kapoor, Kamlesh↗
NASA NTRS2011

Analysis of Compression Algorithm in Ground Collision Avoidance Systems (Auto-GCAS)

Automatic Ground Collision Avoidance Systems (Auto-GCAS) utilizes Digital Terrain Elevation Data (DTED) stored onboard a plane to determine potential recovery maneuvers. Because of the current limitations of computer hardware on military airplanes such as the F-22 and F-35, the DTED must be compressed through a lossy technique called binary-tree tip-tilt. The purpose of this study is to determine the accuracy of the compressed data with respect to the original DTED. This study is mainly interested in the magnitude of the error between the two as well as the overall distribution of the errors throughout the DTED. By understanding how the errors of the compression technique are affected by various factors (topography, density of sampling points, sub-sampling techniques, etc.), modifications can be made to the compression technique resulting in better accuracy. This, in turn, would minimize unnecessary activation of A-GCAS during flight as well as maximizing its contribution to fighter safety.

Schmalz, Tyler↗
NASA NTRS1993

TOPEX/POSEIDON GPS Demonstration Experiment Data

Global Positioning Satellite (GPS) measurements obtained from the on-board TOPEX/POSEIDON demonstration flight receiver and a global network of ground receivers are available from the GPS Data Handling Facility (GDHF) archive. Ground receiver observations are derived from a six station subset of the GPS global tracking network. The six TOPEX ground stations yielded 95 percent of the expected data during the six month demonstration phase from November, 1992 to April, 1993. For the same period, 98 percent of the available P-code tracking from the TOPEX flight receiver was obtained. Raw carrier phase and P-code pseudorange measurements are provided for the ground and flight receivers along with an additional file of calibrated and compressed flight receiver data...

Guinn, J. R.↗
DOE OSTI2023

CODAS Data from Oliktok Point, Beaufort Sea, Alaska

Cryosphere/Ocean Distributed Acoustic Sensing (CODAS) data collected from the Beaufort Sea, Alaska, using ~37.4 km of dark telecommunications fiber located at Oliktok Point, Alaska. Data were collected with a Silixa iDAS, using 10 m gauge length, 2 m spatial resolution, and 1000 Hz sample rate. Provided here are the DAS-recorded time series for the rapid refreeze event described in Baker & Abbott (2022) (see link below). This covers a date range of 2021-11-10 15:00 UTC to 2021-11-11 17:00 UTC. Data have been decimated to 100 Hz and 20 m (i.e., every 10th channel for 1831 channels, total), as used in Baker & Abbott (2022). Data have been extracted from raw format into 1-hour long .sac* files and organized into directories by channel number, spanning channels 100 to 18400. Time series units are nano-strainrate (nm/m/s). For distribution, data have been compressed into .zip files containing all time series files for 100 channels. *For information on the Seismic Analysis Code (SAC) file format: https://seiscode.iris.washington.edu/projects/sac

16 TIDAL AND WAVE POWER↗
NASA NTRS2013

Human Factors Evaluation of Conflict Detection Tool for Terminal Area

A conflict detection and resolution tool, Terminal-area Tactical Separation-Assured Flight Environment (T-TSAFE), is being developed to improve the timeliness and accuracy of alerts and reduce the false alert rate observed with the currently deployed technology. The legacy system in use today, Conflict Alert, relies primarily on a dead reckoning algorithm, whereas T-TSAFE uses intent information to augment dead reckoning. In previous experiments, T-TSAFE was found to reduce the rate of false alerts and increase time between the alert to the controller and a loss of separation over the legacy system. In the present study, T-TSAFE was tested under two meteorological conditions, 1) all aircraft operated under instrument flight regimen, and 2) some aircraft operated under mixed operating conditions. The tool was used to visually alert controllers to predicted Losses of separation throughout the terminal airspace, and show compression errors, on final approach. The performance of T-TSAFE on final approach was compared with Automated Terminal Proximity Alert (ATPA), a tool recently deployed by the FAA. Results show that controllers did not report differences in workload or situational awareness between the T-TSAFE and ATPA cones but did prefer T-TSAFE features over ATPA functionality. T-TSAFE will provide one tool that shows alerts in the data blocks and compression errors via cones on the final approach, implementing all tactical conflict detection and alerting via one tool in TRACON airspace.

Verma, Savita Arora↗
DOE OSTI2025

Optimizing Management of Persistent Data Structures in High-Performance Analytics

Large-scale data analytics workflows ingest massive input data into various data structures, including graphs and key-value datastores. These data structures undergo multiple transformations and computations and are typically reused in incremental and iterative analytics workflows. Persisting in-memory views of these data structures enables reusing them beyond the scope of a single program run while avoiding repetitive raw data ingestion overheads. Memory-mapped I/O enables persisting in-memory data structures without data serialization and deserialization overheads. However, memory-mapped I/O lacks the key feature of persisting consistent snapshots of these data structures for incremental ingestion and processing. The obstacles to efficient virtual memory snapshots using memory-mapped I/O include background writebacks outside the application’s control, and the significantly high storage footprint of such snapshots. To address these limitations, we present Privateer, a memory and storage management tool that enables storage-efficient virtual memory snapshotting while also optimizing snapshot I/O performance. Here, we integrated Privateer into Metall, a state-of-the-art persistent memory allocator for C++, and the Lightning Memory-Mapped Database (LMDB), a widely-used key-value datastore in data analytics and machine learning. Privateer optimized application performance by 1.22× when storing data structure snapshots to node-local storage, and up to 16.7× when storing snapshots to a parallel file system. Privateer also optimizes storage efficiency of incremental data structure snapshots by up to 11× using data deduplication and compression.

Computer science↗
DOE OSTI2025

A surprising proliferation of detwinning in β -tin at extreme loading rates

Integrating data from dynamic compression experiments of condensed matter across three national laboratories has led to insight and quantitative calibration of materials strength over decades of loading rate. For many materials, a single strength model (such as PTW) is sufficient to capture the flow-stress strain rate relationship which is monotonic. Here, we show here that β -tin, a tetragonal metal, exhibits dramatic deviations from this behavior. Naive fitting to a single PTW model is insufficient to capture the behavior; indeed, such resulting inferred flow stress versus strain exhibits a non-monotonic behavior. We suggest a resolution to this by proposing that in β -tin there are important Bauschinger effects arising from favorable conditions for twinning and detwinning. A simple yield surface model when paired with PTW hardening captures the experimental data.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Evaluation of Graph Analytics Frameworks Using the GAP Benchmark Suite

The analysis of connected data is an increasingly important application in high-performance computing. Such analyses can reveal fraudulent patterns in financial transactions, optimize telecommunications networks, predict information flow in social networks, etc. However, the landscape of graph analytics is highly diverse. Graph algorithms stress processor architectures differently, and no one graph can represent all topologies. Consequently, no single approach or framework is expected to be optimal for all graph analytics problems. To help make sense of this diverse landscape, we evaluated four approaches to graph analytics: GraphBLAS, Galois, BGL17, GraphIt; and compare them against hand-tuned implementations that take advantage of hardware features on our test platform. Graph- BLAS formulates graph analytics as sparse linear algebra. Galois provides syntactic constructs for data parallelism over irregular data structures. BGL17 is a generic C++ template library for implementing graph algorithms. GraphIt provides a domain- specific language to describe and optimize graph algorithms. We use the GAP Benchmark Suite to establish baseline performance and guide the side-by-side evaluation of each framework. GAP consists of 30 tests: six graph analytics algorithms (breadth- first search, single-source shortest path, PageRank, betweenness centrality, connected components, and triangle counting) run on five graphs, each with different topological characteristics (e.g., high diameter, skewed degree distribution, high average degree). High-performance reference implementations are included for each benchmark algorithm. Because a graph can be loaded into memory a number of ways (e.g., flat file on disk, compressed sparse format, data frames, retrieved from SQL or NoSQL databases), our evaluation focused on computational performance rather than I/O. Our results show the relative strengths of each framework.

Graph algorithms, Benchmarking, shared-memory prog↗
NASA NTRS2017

Hierarchical Data Formats (HDF) Update

In this presentation, we will talk about the latest releases of HDF4 and HDF5 software and tools, new features available in HDF5, and roadmap for the HDF software. We will also solicit feedback from the users of HDF data and HDF application developers on new features and new tools. The talk will cover: Difference between 1.8 and 1.10 releases and how and when to move to the latest release Features of the recent HDF5 1.8.19, 1.10.1 and HDF 4.2.13 Overview of HDF View 3.0 and other enhancements to tools Supported compilers and systems Open discussion of new requirements and wish list of the HDF features Compression library for interoperability with h5py and Pandas and better floating-point data compression.

HDFView↗
DOE OSTI2025

High-performance data format for scientific data storage and analysis

Here, in this article, we present the High-Performance Output (HiPO) data format developed at Jefferson Laboratory for storing and analyzing data from Nuclear Physics experiments. The format was designed to efficiently store large amounts of experimental data, utilizing modern fast compression algorithms. The purpose of this development was to provide organized data in the output, facilitating access to relevant information within the large data files. The HiPO data format has features that are suited for storing raw detector data, reconstruction data, and the final physics analysis data efficiently, eliminating the need to do data conversions through the lifecycle of experimental data. The HiPO data format is implemented in C++ and JAVA, and provides bindings to FORTRAN, Python, and Julia, providing users with the choice of data analysis frameworks to use. In this paper, we will present the general design and functionalities of the HiPO library and compare the performance of the library with more established data formats used in data analysis in High Energy and Nuclear Physics (such as ROOT and Parquete). In columnar data analysis, HiPO surpasses established data formats in performance and can be effectively applied to data analysis in other scientific fields.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
NASA NTRS1993

A browse facility for Earth science remote sensing data: Center director's discretionary fund final report

An image data visual browse facility is developed for a UNIX platform using the X Windows 11 system. It allows one to visually examine reduced resolution image data to determine which data are applicable for further research. Links with a relational data base manager then allow one to extract not only the full resolution image data, but any other ancillary data related to the case study. Various techniques are examined for compression of the image data in order to reduce data storage requirements and time necessary to transmit the data on the internet. Data used were from the WetNet project.

Meyer, P. J.↗