Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 757 records · Page 42
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Expanding Access to Science Participation: A FAIR Framework for Petascale Data Visualization and Analytics

The massive data generated by scientists daily serve as both a major catalyst for new discoveries and innovations, as well as a significant roadblock that restricts access to the data. Here, our paper introduces a new approach to removing Big Data barriers and democratizing access to petascale data for the broader scientific community. Our novel data fabric abstraction layer allows user-friendly querying of scientific information while hiding the complexities of dealing with file systems or cloud services. We enable FAIR (Findable, Accessible, Interoperable, and Reusable) access to datasets such as NASA’s petascale climate datasets. Our paper presents an approach to managing, visualizing, and analyzing petabytes of data within a browser on equipment ranging from the top NASA supercomputer to commodity hardware like a laptop. Our novel data fabric abstraction utilizes state-of-the art progressive compression algorithms and machine-learning insights to power scalable visualization dashboards for petascale data. The result provides users with the ability to identify extreme events or trends dynamically, expanding access to scientific data and further enabling discoveries. We validate our approach by improving the ability of climate scientists to visually explore their data via three fully interactive dashboards. We further validate our approach by deploying the dashboards and simplified training materials in the classroom at a minority-serving institution. These dashboards, released in simplified form to the general public, contribute significantly to a broader push to democratize the access and use of climate data.

Computer science↗
DOE OSTI2014

Transparent in Situ Data Transformations in ADIOS

Though an abundance of novel "data transformation" technologies have been developed (such as compression, level-of-detail, layout optimization, and indexing), there remains a notable gap in the adoption of such services by scientific applications. In response, we develop an in situ data transformation framework in the ADIOS I/O middleware with a "plug in" interface, thus greatly simplifying both the deployment and use of data transform services in scientific applications. Our approach ensures user-transparency, runtime-configurability, compatibility with existing I/O optimizations, and the potential for exploiting read-optimizing transforms (such as level-of-detail) to achieve I/O reduction. We demonstrate use of our framework with the QLG simulation at up to 8,192 cores on the leadership-class Titan supercomputer, showing negligible overhead. We also explore the read performance implications of data transforms with respect to parameters such as chunk size, access pattern, and the "opacity" of different transform methods including compression and level-of-detail.

Computer Science↗
DOE OSTI2024

Pathway to Decarbonization Through Industrial Energy Efficiency: Micro and Macro Perspectives from Compressed Air Usage

Abstract Energy audits directly provided the industrial sector with reduced energy costs and avoided emissions. Still, they also lead to far-reaching indirect and induced local, regional, and national benefits. This paper aims to present the techno-economic-environmental analysis to achieve decarbonization through implementing industrial energy efficiency at micro and macro levels. An integrated techno-economic-environmental methodology is developed. Case studies of micro-level carbon reduction efforts through industrial energy efficiency technologies are presented. The broader macroeconomic and environmental effects of technology on society are analyzed using data from 206 energy audits of industrial compressed air systems conducted over 13 years. The impacts show that energy-efficient improvements lead to direct cost savings for manufacturers, boost economic activity across sectors, and affect carbon dioxide emissions both short-term and long-term in the region. Given their extensive benefits, energy audits significantly influence policymaking. We devised a methodology to link micro-level energy audit data with macroeconomic and environmental analyses to quantify these cascading benefits. The economic scenario analysis shows that $228 M has been saved from direct industrial energy savings from implementing all compressed air recommendations in the studied periods and the region. In addition, the investment made through manufacturers would create 2,025 jobs and $383 M annually, cascading regional economic impacts. The environmental analysis shows that the regional manufacturers have directly avoided about 2.8 M metric tons of carbon dioxide emissions.

Engineering↗
DOE OSTI2019

SPRUCE Vegetation Community Survey Data from SPRUCE Experimental Plots, 2014-2020

This dataset provides vegetation survey data collected across vegetation community plots within the 10 chambered treatment plots (6, 19, 11, 20, 4, 13, 8, 16, 10, 17) and 3 unchambered ambient plots (7, 14, 21) from the Spruce and Peatland Responses Under Changing Environments (SPRUCE) experimental site within the Marcell Experimental Forest in northern Minnesota, USA. Collections occurred from 2014-2020 (2014-04-27 to 2020-08-06). The vegetation community plots, three in each experimental plot, were established in 2014 and designated to be undisturbed and reserved for long-term sampling. Vegetation survey data were collected at the peak of each growing season by placing a 1x2 meter rectangular grid over each community plot. The 1x2 meter grid is subset into 50 20x20 cm grid cells. Observations are of the presence of each species in each grid cell for each subplot. That’s a total of 39 subplots across the 13 experimental plots. From 2018 to 2020, shrub stem density was counted and percent shrub cover was estimated in five selected grid cells in each vegetation community plot. Also included are photographs of each community plot taken at the time of data collection for each measurement year. These data provide insight into the responses of plant community composition to warming and increased carbon dioxide. These data were use in analyses in McPartland et al (2018; 2020). This dataset contains three data files in comma-separate values (*.csv) format and seven compressed (*.zip) files containing 877 JPEG (*.jpg) files. Additional metadata are provided: a data dictionary and a file-level metadata file in comma-separate values (.csv) format and a user guide in PDF (*.pdf) format.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

FedCSpc: A Cross-Silo Federated Learning System With Error-Bounded Lossy Parameter Compression

Cross-Silo federated learning is widely used for scaling deep neural network (DNN) training over data silos from different locations worldwide while guaranteeing data privacy. Communication has been identified as the main bottleneck when training large-scale models due to large-volume model parameters and gradient transmission across public networks with limited bandwidth. Most previous works focus on gradient compression, while limited work tries to compress parameters that can not be ignored and extremely affect communication performance during the training. Here, to bridge this gap, we propose FedCSpc: an efficient cross-silo federated learning system with an XAI-driven adaptive parameter compression strategy for large-scale model training. Our work substantially differs from existing gradient compression techniques due to the distinct data features of gradient and parameter. The key contributions of this paper are fourfold. (1) Our designed FedCSpc proposes to compress the parameter during the training using the state-of-the-art error-bounded lossy compressor – SZ3. (2) We develop an adaptive compression error bound adjustment algorithm to guarantee the model accuracy effectively. (3) We exploit an efficient approach to utilize the idle CPU resources of clients to compress the parameters. (4) We perform a comprehensive evaluation with a wide range of models and benchmarks on a GPU cluster with 65 GPUs. Results show that FedCSpc can achieve the same model accuracy as FedAvg while reducing the data volume of parameters and gradients in communication by up to 7.39× and 288×, respectively. With 32 clients on a 4 Gb size model, FedCSpc significantly outperforms FedAvg in wall-clock time in the emulated WAN environment (at the bandwidth of 1 Gbps or lower without loss of generality).

SZ3↗
DOE OSTI2019

Dynamic Mode Decomposition for Compressive System Identification

Dynamic mode decomposition has emerged as a leading technique to identify spatiotemporal coherent structures from high-dimensional data, benefiting from a strong connection to nonlinear dynamical systems via the Koopman operator. In this work, we integrate and unify two recent innovations that extend DMD to systems with actuation [Proctor et al., 2016] and systems with heavily subsampled measurements [Brunton et al., 2015]. When combined, these methods yield a novel framework for compressive system identification [code is publicly available at: https://github.com/zhbai/cDMDc]. It is possible to identify a low-order model from limited input-output data and reconstruct the associated full-state dynamic modes with compressed sensing, adding interpretability to the state of the reduced-order model. Moreover, when full-state data is available, it is possible to dramatically accelerate downstream computations by first compressing the data. We demonstrate this unified framework on two model systems, investigating the effects of sensor noise, different types of measurements (e.g., point sensors, Gaussian random projections, etc.), compression ratios, and different choices of actuation (e.g., localized, broadband, etc.). In the first example, we explore this architecture on a test system with known low-rank dynamics and an artificially inflated state dimension. The second example consists of a real-world engineering application given by the fluid flow past a pitching airfoil at low Reynolds number. This example provides a challenging and realistic test-case for the proposed method, and results demonstrate that the dominant coherent structures are well characterized despite actuation and heavily subsampled data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

Method to measure orientation of discontinuous fiber embedded in the polymer matrix from computerized tomography scan data

Usage of discontinuous glass fibers in injection and compression molded resin components is rapidly increasing to improve their mechanical properties. Since added fiber contributes to more strength along the fiber direction compared to transverse direction, the mechanical properties of such components strongly depend on the fiber orientation. Thus it is important to estimate the fiber orientation distribution in such materials. Here, we are presenting recently developed method to estimate fiber orientation from 3D image of fibers generated using micro CT scan. Typically the 3D images from CT scan are very large and it becomes difficult to separate each fiber and extract end point information. In this paper a novel method to address this challenge is presented. The micro-CT images were broken into finite volume, reducing data size, and then each fiber was reduced to its own center line, further reducing the data size, using Mimics Innovation Suite (Materialise NV). These 3D centerlines were then used to quantify the 2nd order orientation tensor. Our results from the proposed method are compared with the measurements using well established industry standard approach called method of ellipses for validation. The key challenges in estimating the fiber orientation are identified and future improvements are proposed.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Localized material compression to correct distortion in wire arc additive manufacturing

Wire Arc Additive Manufacturing (WAAM) is an advanced manufacturing technology which utilizes welding systems to generate three dimensional geometries in a layer-by-layer fashion. Distortion or warping of a print substrate and WAAM components due to thermally induced residual stresses is an ongoing challenge limiting the widespread adoption of WAAM technologies for producing components. In this manuscript, a novel approach is described to address thermal distortion in deposited components by applying lateral compressions along the length of the deposited material. To demonstrate this method, a series of single-track walls were printed and compressed at evenly spaced intervals using a modified hydraulic cutter tool. The jaws of the tool were modified to compress material rather than to shear it. A mathematical model was developed to relate the curvature of the deposited material to the volume of compression required to eliminate this distortion. Validation of this model was performed using 3D scan data to compare the change in wall curvature induced by compression to the volume of the applied compressions. Substrate deflection was also compared against a control wall, and implementation of wall compression reduced maximum deflections by 93% across a series of four depositions and subsequent compressions. Wall cross sections were also analyzed to determine the impact of compression on material hardness and grain structure. The results demonstrate that successively placed lateral compressions can effectively control and potentially eliminate bending distortion in printed parts. This methodology can be further developed to form a robust model for correction of thermally-induced distortion in WAAM components.

Additive manufacturing↗
DOE OSTI2021

4D Imaging of ZnO-Coated Nanoporous Al 2 O 3 Aerogels by Chemically Sensitive Ptychographic Tomography: Implications for Designer Catalysts

The 3D chemical structure (4D spectromicroscopy) of nanoporous Al 2 O 3 aerogels coated with ZnO by atomic layer deposition (ALD) was studied by multienergy scanning transmission X-ray microscopy. These materials are representative of a class of designer catalysts in which the nanoporous support is prepared separately from the active catalyst material, which is subsequently introduced by ALD, thereby allowing independent optimization of the morphology, chemistry, and spatial distribution of the support and catalyst. The samples studied were prepared by Ga ion and Xe plasma focused ion beam (FIB) milling as well as drop casting from water suspension. Zn L and Al K edge spectra of six samples with three different ZnO loadings were measured to investigate how loading and different sample preparation methods affect the 3D distribution of the ZnO and Al 2 O 3 . Scanning transmission X-ray microscopy (STXM) and ptychographic imaging at two energies each at the Zn L 3 and Al K edge were measured. The ptychography data were analyzed by using the SHARP reconstruction code to generate quantitative 2D chemical maps of the Al 2 O 3 and the ZnO. The STXM and ptychography maps were then measured at a sequence of tilt angles, covering up to 160° of rotation. The 3D structure of the ZnO and Al 2 O 3 was derived from the tilt series data by tomographic reconstruction using a compressed sensing algorithm. A twodimensional spatial resolution (half-period) of 6 nm, measured by Fourier ring correlation, and a 3D spatial resolution (half-period) of 9 nm, measured by Fourier shell correlation, were achieved when using the COSMIC beamline at the Advanced Light Source (ALS). The results show that for all of the ZnO loadings studied there is nonuniform coverage of the ZnO on the Al 2 O 3 aerogel framework. In addition, we found that both FIB methods create sample artifacts, although the distortion was less with Xe plasma than Ga ion FIB.

36 MATERIALS SCIENCE↗
DOE OSTI2013

Pressure-induced densification in GeO{sub 2} glass: A transmission x-ray microscopy study

Nanoscale transmission x-ray microscopy measurements have been performed to determine the effect of pressure (P) on the volume (V) change in GeO{sub 2} glass up to 38.5 GPa. The P-V data show a continuous increase upon compression, indicating that the density-driven structural transformation is a gradual process. Over the pressure range studied, a transition is observed at approximately 10–13 GPa, where the material displays distinct compression behaviors. The pressure-induced densification that involves the coordination number change has been discussed. Using this newly developed high-pressure imaging technique with tens of nanometer resolution, we have provided a direct and unequivocal way for measuring density of amorphous materials to much higher pressures with accuracy rivaling x-ray diffraction of crystalline solids.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2014

Fixed-rate compressed floating-point arrays

ZFP is a library for lossy compression of single- and double-precision floating-point data. One of the unique features of ZFP is its support for fixed-rate compression, which enables random read and write access at the granularity of small blocks of values. Using a C++ interface, this allows declaring compressed arrays (1D, 2D, and 3D arrays are supported) that through operator overloading can be treated just like conventional, uncompressed arrays, but which allow the user to specify the exact number of bits to allocate to the array. ZFP also has variable-rate fixed-precision and fixed-accuracy modes, which allow the user to specify a tolerance on the relative or absolute error.

ECP↗
DOE OSTI2019

bokeh streaming [SWR 19-08]

Simple examples of bokeh streaming data. The plots are intended to be generic and able to infer some of the plot data from the structure of the data sent. The receiver accepts zlib compressed pickled python dictionaries.

Simpson, Jeffery↗
DOE OSTI2019

Energy Storage Technology and Cost Characterization Report

This report defines and evaluates cost and performance parameters of six battery energy storage technologies (BESS) (lithium-ion batteries, lead-acid batteries, redox flow batteries, sodium-sulfur batteries, sodium metal halide batteries, and zinc-hybrid cathode batteries) and four non-BESS storage technologies (pumped storage hydropower, flywheels, compressed air energy storage, and ultracapacitors). Data for combustion turbines are also presented. Cost information was procured for the most recent year for which data were available based on an extensive literature review, conversations with vendors and stakeholders, and summaries of actual costs provided from specific projects at sites across the United States. Detailed cost and performance estimates were presented for 2018 and projected out to 2025.

25 ENERGY STORAGE↗
DOE OSTI2018

Deformation and Fracture in Complex-Shaped Energetic Particles

A new technical basis on the mechanics of energetic materials at the individual particle scale has been developed. Despite these particles being in most of our Sandia non-nuclear explosive components, we have historically lacked any understanding of particle behavior. Through the novel application of nanoidentation methods to single crystal films and single particles of energetic materials with complex shapes, discovery data has been collected elucidating phenomena of particle strength, elastic and plastic deformation, and fracture. This work specifically developed the experimental techniques and analysis methodologies to distill data into relationships suitable for future integration into particle level simulations of particle reassembly. This project utilized experimental facilities at CINT and the Explosive Components Facility to perform ex-situ and in-situ nanoidentation experiments with simultaneous scanning electron microscope (SEM) imaging. Data collected by an applied axial compressive load in either force-control or displacement-control was well represented by Hertzian contact theory for linear elastic materials. Particle fracture phenomenology was effectively modeled by an empirical damage model.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

Orange County Transportation Authority Fuel Cell Electric Bus Progress Report (Feb-Jul 2020) [Slides]

This report presents early results from a deployment of fuel cell electric buses (FCEBs) operated by Orange County Transportation Authority (OCTA) in Southern California. The ten FCEBs, produced by New Flyer, feature an electric drive propulsion system powered by a Ballard fuel cell system. The project team is collaborating with the U.S. Department of Energy (DOE) and DOE's National Renewable Energy Laboratory (NREL) to evaluate the buses in revenue service. The goal of this evaluation is to compare the FCEB performance to that of conventional technology and to track progress over time toward meeting the technical targets set by DOE and the Department of Transportation (DOT). The FCEBs were delivered beginning in late-2019, and were placed in service on February 9, 2020. The data period covers February 2020 through July 2020. NREL collects data on ten 2016 model year compressed natural gas (CNG) buses as a baseline comparison at OCTA.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2021

SunLine Transit Agency Fuel Cell Electric Bus Progress Report (Data Period Focus Jan-Jun 2020) [Slides]

This report presents early results from a deployment of fuel cell electric buses (FCEBs) operated by SunLine Transit Agency in the Coachella Valley area of California. The five FCEBs, produced by New Flyer, feature an electric drive propulsion system powered by a Ballard fuel cell system. The project team is collaborating with the U.S. Department of Energy (DOE) and DOE's National Renewable Energy Laboratory (NREL) to evaluate the buses in revenue service. The goal of this evaluation is to compare the FCEB performance to that of conventional technology and to track progress over time toward meeting the technical targets set by DOE and the Department of Transportation (DOT). The FCEBs were delivered beginning in mid-2019. The data period covers January 2020 through July 2020. NREL collects data on five 2019 model year compressed natural gas (CNG) buses as a baseline comparison at SunLine. These new CNG buses were phased into service beginning in April 2020.

25 ENERGY STORAGE↗
DOE OSTI2022

Orange County Transportation Authority Fuel Cell Electric Bus Progress Report - Data Period Focus: Feb. 2020 through Jul. 2021 [Slides]

This report presents early results from a deployment of fuel cell electric buses (FCEBs) operated by Orange County Transportation Authority (OCTA) in Southern California. The ten FCEBs, produced by New Flyer, feature an electric drive propulsion system powered by a Ballard fuel cell system. The project team is collaborating with the U.S. Department of Energy (DOE) and DOE's National Renewable Energy Laboratory (NREL) to evaluate the buses in revenue service. The goal of this evaluation is to compare the FCEB performance to that of conventional technology and to track progress over time toward meeting the technical targets set by DOE and the Department of Transportation (DOT). The FCEBs were delivered beginning in late-2019, and were placed in service on February 9, 2020. The data period covers February 2020 through July 2021. NREL collects data on ten 2016 model year compressed natural gas (CNG) buses as a baseline comparison at OCTA.

30 DIRECT ENERGY CONVERSION↗