SEARCH · Engineering Papers

Results for “data compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

All sources NASA NTRS DOE OSTI

DOE OSTI2025

Fiducial-cosmology-dependent systematics for the DESI 2024 BAO analysis

When measuring the Baryon Acoustic Oscillations (BAO) scale from galaxy surveys, one typically assumes a fiducial cosmology when converting redshift measurements into comoving distances and also when defining input parameters for the reconstruction algorithm. A parameterised template for the model to be fitted is also created based on a (possibly different) fiducial cosmology. This model reliance can be considered a form of data compression, and the data is then analysed allowing that the true answer is different from the fiducial cosmology assumed. In this study, we evaluate the impact of the fiducial cosmology assumed in the BAO analysis of the Dark Energy Spectroscopic Instrument (DESI) survey Data Release 1 (DR1) on the final measurements in DESI 2024 III. We utilise a suite of mock galaxy catalogues with survey realism that mirrors the DESI DR1 tracers: the bright galaxy sample (BGS), the luminous red galaxies (LRG), the emission line galaxies (ELG) and the quasars (QSO), spanning a redshift range from 0.1 to 2.1. We compare the four secondary AbacusSummit cosmologies against DESI's fiducial cosmology (Planck 2018). The secondary cosmologies explored include a lower cold dark matter density, a thawing dark energy universe, a higher number of effective species, and a lower amplitude of matter clustering. The mocks are processed through the BAO pipeline by consistently iterating the grid, template, and reconstruction reference cosmologies. We determine a conservative systematic contribution to the error of 0.1% for both the isotropic and anisotropic dilation parameters αiso and αAP. We then directly test the impact of the fiducial cosmology on DESI DR1 data.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2016

Damaris: Addressing performance variability in data management for post-petascale simulations

With exascale computing on the horizon, reducing performance variability in data management tasks (storage, visualization, analysis, etc.) is becoming a key challenge in sustaining high performance. Here, this variability significantly impacts the overall application performance at scale and its predictability over time. In this article, we present Damaris, a system that leverages dedicated cores in multicore nodes to offload data management tasks, including I/O, data compression, scheduling of data movements, in situ analysis, and visualization. We evaluate Damaris with the CM1 atmospheric simulation and the Nek5000 computational fluid dynamic simulation on four platforms, including NICS’s Kraken and NCSA’s Blue Waters. Our results show that (1) Damaris fully hides the I/O variability as well as all I/O-related costs, thus making simulation performance predictable; (2) it increases the sustained write throughput by a factor of up to 15 compared with standard I/O approaches; (3) it allows almost perfect scalability of the simulation up to over 9,000 cores, as opposed to state-of-the-art approaches that fail to scale; and (4) it enables a seamless connection to the VisIt visualization software to perform in situ analysis and visualization in a way that impacts neither the performance of the simulation nor its variability. In addition, we extended our implementation of Damaris to also support the use of dedicated nodes and conducted a thorough comparison of the two approaches—dedicated cores and dedicated nodes—for I/O tasks with the aforementioned applications.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Deep learning for electron and scanning probe microscopy: From materials design to atomic fabrication

Machine learning and artificial intelligence (ML/AI) are rapidly becoming an indispensable part of physics research, with applications ranging from theory and materials prediction to high-throughput data analysis. In parallel, the recent successes in applying ML/AI methods for autonomous systems from robotics through self-driving cars to organic and inorganic synthesis are generating enthusiasm for the potential of these techniques to enable automated and autonomous experiment in imaging. Here, we discuss recent progress in application of machine learning methods in scanning transmission electron microscopy and scanning probe microscopy, from applications such as data compression and exploratory data analysis to physics learning to atomic fabrication.

36 MATERIALS SCIENCE↗

DOE OSTI2024

DLSIA: Deep Learning for Scientific Image Analysis

DLSIA (Deep Learning for Scientific Image Analysis) is a Python-based machine learning library that empowers scientists and researchers across diverse scientific domains with a range of customizable convolutional neural network (CNN) architectures for a wide variety of tasks in image analysis to be used in downstream data processing. DLSIA features easy-to-use architectures, such as autoencoders, tunable U-Nets and parameter-lean mixed-scale dense networks (MSDNets). Additionally, this article introduces sparse mixed-scale networks (SMSNets), generated using random graphs, sparse connections and dilated convolutions connecting different length scales. For verification, several DLSIA-instantiated networks and training scripts are employed in multiple applications, including inpainting for X-ray scattering data using U-Nets and MSDNets, segmenting 3D fibers in X-ray tomographic reconstructions of concrete using an ensemble of SMSNets, and leveraging autoencoder latent spaces for data compression and clustering. As experimental data continue to grow in scale and complexity, DLSIA provides accessible CNN construction and abstracts CNN complexities, allowing scientists to tailor their machine learning approaches, accelerate discoveries, foster interdisciplinary collaboration and advance research in scientific image analysis.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2016

Fourier band-power E/B-mode estimators for cosmic shear

We introduce new Fourier band-power estimators for cosmic shear data analysis and E/B-mode separation. We consider both the case where one performs E/B-mode separation and the case where one does not. The resulting estimators have several nice properties which make them ideal for cosmic shear data analysis. First, they can be written as linear combinations of the binned cosmic shear correlation functions. Secondly, they account for the survey window function in real-space. Thirdly, they are unbiased by shape noise since they do not use correlation function data at zero separation. Fourthly, the band-power window functions in Fourier space are compact and largely non-oscillatory. Fifthly, they can be used to construct band-power estimators with very efficient data compression properties. In particular, we find that all of the information on the parameters Ωm, σ8 and ns in the shear correlation functions in the range of ~10–400 arcmin for single tomographic bin can be compressed into only three band-power estimates. Finally, we can achieve these rates of data compression while excluding small-scale information where the modelling of the shear correlation functions and power spectra is very difficult. Given these desirable properties, these estimators will be very useful for cosmic shear data analysis.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2020

Compression behavior of microcrystalline cellulose spheres: Single particle compression and confined bulk compression across regimes

Particle characteristics can drastically influence the process-structure-property-performance aspects of granular materials in compression. In this study, we aim to computationally simulate the mechanical processes of stress redistribution in compacts including the kinematics of particle rearrangement during densification and particle deformation leading to fragmentation. Confined compression experiments are conducted with three sets of commercial microcrystalline cellulose particles nearly spherical in shape with different mean particle size. Experimentally measured compression curves from tall powder columns are fitted with the Kenkre et al. (J. of American Chemical Society, Vol. 79, No. 12) model. This model provides a basis to derive several common two-parameter literature models and as a framework to incorporate statistical representations of critical particle behaviors. We focus on the low-stress compression data and the model comparisons typically not discussed in the literature. Furthermore, additional single particle compressions report fracture strength with particle size for comparison to the apparent particle strength extracted from bulk compression data.

42 ENGINEERING↗

DOE OSTI2019

SWinzip v. 2.0

SWinzip is a Matlab and C++ library for scientific lossy data compression and reconstruction using compressed sensing and tree-wavelets transforms. These methods are known for their large compression and usefulness in data analytics such as features extraction. Compressed sensing and wavelets methods rely heavily on sparse and dense linear algebra operations implemented through the Boost codes in our library. SWinzip accommodates data represented on both regular grids (e.g. image data) and point-clouds (e.g. unstructured meshes). In SWinzip v2.0, there is more focus on wavelet compression and the following features are added: - The matrix-free (direct) wavelet transform is implemented in both C++ and Matlab. A threshold-based compression can now be used. Examples for distributed compression are also added. - Python implementation of Alpert wavelet transforms - Field data comparison using wavelets in Matlab with global and local metrics

Solloum, Maher↗

DOE OSTI2020

Machine Learning for Memory Reduction in the Implicit Monte Carlo Simulations of Thermal Radiative Transfer [Slides]

Project Goal: Use parametric Machine Learning methods in order to reduce memory requirements at checkpointing & restarting in the IMC simulations of Thermal Radiative Transfer using: Expectation Maximization and Weighted Gaussian Mixture Model-based approach for `particle-data compression', introduced in Plasma Physics to model Maxwellian particle distributions by Luis Chacon and Guangye Chen; Expectation Maximization with Weighted Hyper-Erlang Model in order to compress isotropic IMC particle data in the frequency domain; and Expectation Maximization and von Mises-Fisher Mixture Model for compression of anisotropic IMC particle data in the angular domain (work-in-progress).

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2016

SWinzip v. 1.0

Salloum, Maher↗

DOE OSTI2025

Variational autoencoders for at-source data reduction and anomaly detection in high energy particle detectors

Detectors in next-generation high-energy physics experiments face several daunting requirements, such as high data rates, damaging radiation exposure, and stringent constraints on power, space, and latency. To address these challenges, machine learning in readout electronics can be leveraged for smart detector designs, enabling intelligent inference and data reduction at-source. Variational autoencoders (VAEs) offer a variety of benefits for front-end readout; an on-sensor encoder can perform efficient lossy data compression while simultaneously providing a latent space representation that can be used for anomaly detection. Results are presented from low-latency and resource-efficient VAEs for front-end data processing in a futuristic silicon pixel detector. Encoder-based data compression is found to preserve good performance of off-detector analysis while significantly reducing the off-detector data rate as compared to a similarly sized data filtering approach. Furthermore, the latent space information is found to be a useful discriminator in the context of real-time sensor defect monitoring. Together, these results highlight the multifaceted utility of autoencoder-based front-end readout schemes and motivate their consideration in future detector designs.

47 OTHER INSTRUMENTATION↗

DOE OSTI2024

Data-Driven Compression of Electron-Phonon Interactions

First-principles calculations of electron interactions in materials have seen rapid progress in recent years, with electron-phonon ( e − ph ) interactions being a prime example. However, these techniques use large matrices encoding the interactions on dense momentum grids, which reduces computational efficiency and obscures interpretability. For e − ph interactions, existing interpolation techniques leverage locality in real space, but the high dimensionality of the data remains a bottleneck to balance cost and accuracy. Here we show an efficient way to compress e − ph interactions based on singular value decomposition (SVD), a widely used matrix and image compression technique. Leveraging (un)constrained SVD methods, we accurately predict material properties related to e − ph interactions—including charge mobility, spin relaxation times, band renormalization, and superconducting critical temperature—while using only a small fraction (1%–2%) of the interaction data. These findings unveil the hidden low-dimensional nature of e − ph interactions. Furthermore, they accelerate state-of-the-art first-principles e − ph calculations by about 2 orders of magnitude without sacrificing accuracy. Our Pareto-optimal parametrization of e − ph interactions can be readily generalized to electron-electron and electron-defect interactions, as well as to other couplings, advancing quantitative studies of condensed matter. Published by the American Physical Society 2024

Physics↗

DOE OSTI2014

Experimental verification of the high pressure crystal structures in NH{sub 3}BH{sub 3}

A detailed high-pressure study on NH{sub 3}BH{sub 3} has been carried out using in situ synchrotron X-ray diffraction (XRD) and Raman scattering with a diamond anvil cell up to 20 and 33 GPa, respectively. The Rietveld refinement based on the XRD pattern and analysis of Raman data indicate two first-order phase transitions from the ambient pressure I4 mm structure (α-NH{sub 3}BH{sub 3}) to a high pressure Cmc2{sub 1} phase (β-NH{sub 3}BH{sub 3}) at 2.14 GPa, and further into a monoclinic P2{sub 1} (Z = 2) phase (γ-NH{sub 3}BH{sub 3}) at 9.67 GPa. Fitting the measured volumetric compression data to the third order Birch-Murnaghan equation of state reveals a bulk modulus of B{sub 0} = 9.9 ± 0.5 and 17.0 ± 3.0 GPa (with fixed B{sub 0}{sup ′} = 4) for the β-NH{sub 3}BH{sub 3} below and above 5 GPa, respectively. Still, with the splitting of the NBH rock mode in Raman experiment, it is concluded that a second-order isostructural phase transition occurs at 5 GPa. By analyzing the dihydrogen bonding framework, the origin of the isostructural phase transition is attributed to the number of dihydrogen bondings per molecule in the Cmc2{sub 1} phase increasing from 12 to 14 at 5 GPa.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2022

Femtosecond diffraction studies of the sodium chloride phase diagram under laser shock compression

The phase diagram of sodium chloride (NaCl) under laser shock compression has been studied at Linac Coherent Light Source (LCLS) at the x-ray free-electron laser facility. Both solid–solid (B1 → B2) and solid–liquid (B2 → liquid) transitions have been observed along the Hugoniot over nanosecond time scales. By combining structural measurements through in situ x-ray diffraction, pressure determination through velocimetry, and a thermal equation-of-state, the shock-compressed data are used to constrain the phase diagram of NaCl. Transformation into the B2 phase is found to occur at 28(2) GPa, and B2–liquid coexistence is observed between 54(4) and 66(6) GPa, with near full melt at 66(6) GPa. Late-time pressure release from an initial shocked B2-state results in a B2 → B1 back transformation. Our results show agreement with previous static compression data, suggesting that the time scale for melting is very rapid and that equilibrium states in NaCl are being accessed over nanosecond time scales. Finally, a multiphase equation-of-state description of NaCl incorporated into a one-dimensional hydrocode is used to interpret pressure and temperature evolution over these rapid time scales.

36 MATERIALS SCIENCE↗

DOE OSTI2026

Dark Energy Survey Year 3 results: optimized $w$CDM simulation-based inference with weak lensing map-level hybrid statistics

We present cosmological constraints from the Dark Energy Survey Year 3 (DES Y3) weak lensing data using hierarchical hybrid statistics within a Bayesian simulation-based inference framework that is based on the Gower Street simulations. To maximize the precision of the inference, we have developed a new, information-theory based, data compression of the weak lensing maps to just seven highly informative summary statistics. The hybrid scheme exploits the high information content of the power spectrum, compressing both the power spectrum and neural-based summaries that are designed to extract further information. Our simulation-based approach enables principled forward modelling of all major sources of systematic uncertainty and survey properties into realistic mock observations, including the survey mask, photometric redshift uncertainties, intrinsic galaxy alignments, multiplicative shear calibration bias, source galaxy clustering, non-Gaussian shape noise, and non-linear structure formation. The summary statistics are then used in a Bayesian simulation-based inference pipeline. The inference is validated through coverage tests and checks for robustness against baryonic feedback. Assuming a $w$CDM cosmology, our analysis yields $S_8 = 0.808 \pm 0.017$, $Ω_{\rm m} = 0.325 \pm 0.024$, and $w < -0.766$ (marginalized posterior 68 per cent credible intervals). This rigorous combination of information theory, physics- and neural network-based extreme data compression, and principled Bayesian analysis improves the figure of merit for $(Ω_{\rm m}, S_8, w)$ by 60 per cent over the previous state-of-the-art, and by almost a factor of 3 over two-point analyses of the same data. They are the most precise joint constraints on $(Ω_{\rm m}, S_8, w)$ from weak gravitational lensing data alone of any survey to date. We intend to apply this analysis to the more recent DES Y6 data.

Williamson, J. [University Coll. London]↗

DOE OSTI2019

Tensor-decomposition techniques for ab initio nuclear structure calculations: From chiral nuclear potentials to ground-state energies

The computational resources needed to generate the ab initio solution of the nuclear many-body problem for increasing mass number and/or accuracy necessitates innovative developments to improve upon (i) the storage of many-body operators and (ii) the scaling of many-body methods used to evaluate nuclear observables. The storing and efficient handling of many-body operators with high particle ranks is currently one of the major bottlenecks limiting the applicability range of ab initio studies with respect to mass number and accuracy. Recently, the application of tensor decomposition techniques to many-body tensors has proven highly beneficial to reduce the computational cost of ab initio calculations in quantum chemistry and solid-state physics. The impact of applying state-of-the-art tensor factorization techniques to modern nuclear Hamiltonians derived from chiral effective field theory is investigated. Subsequently, the error induced by the tensor decomposition of the input Hamiltonian on ground-state energies of closed-shell nuclei calculated via second-order many-body perturbation theory is benchmarked. The first proof-of-principles application of tensor-decomposition techniques to the nuclear Hamiltonian is performed. Two different tensor formats are investigated by systematically benchmarking the approximation error on matrix elements stored in various bases of interest. The analysis is achieved while including normal-ordered three-nucleon interactions that are nowadays used as input to the most advanced ab initio calculations in medium-mass nuclei. With the aid of the factorized Hamiltonian, the second-order perturbative correction to ground-state energies is decomposed and the scaling properties of the underlying tensor network are discussed. The employed tensor formats are found to lead to an efficient data compression of two-body matrix elements of the nuclear Hamiltonian. In particular, the sophisticated tensor hypercontraction scheme yields low tensor ranks with respect to both harmonic-oscillator and Hartree-Fock single-particle bases. It is found that the tensor rank depends on the two-body total angular momentum J for which one performs the decomposition, which is itself directly related to the sparsity of the corresponding tensor. Furthermore, including normal-ordered two-body contributions originating from three-body interactions does not compromise the efficient data compression. Ultimately, the use of factorized matrix elements authorizes controlled approximations of the exact second-order ground-state energy corrections. In particular, a small enough error is obtained from low-rank factorizations in He4,O16, and Ca40. It is presently demonstrated that tensor-decomposition techniques can be efficiently applied to systematically approximate the nuclear many-body Hamiltonian in terms of lower-rank tensors. Beyond the input Hamiltonian, tensor-decomposition techniques can be envisioned to scale down the cost of state-of-the-art nonperturbative many-body methods in order to extend ab initio studies to (i) higher precisions, (ii) larger masses, and (iii) nuclei of doubly open-shell character.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2020

Compression behavior of dense H 2 -He mixtures up to 160 GPa

We have studied the compression behavior of H 2 -He mixtures in comparison with pure H 2 and He using powder synchrotron x-ray diffraction and present the pressure-volume (PV) compression data of H 2 -He mixtures to 160 GPa. The results indicate that both H 2 and He in H 2 -He mixtures remain in hcp to the maximum pressure studied, yet develop a substantial level of lattice distortion in the (100) plane, most profound in He-rich solids and below 66 GPa. The measured PV data also indicate softening of He (or H 2 )-rich lattice upon increasing the level of the guest H 2 (or He) concentration. We suggest that the observed softening and lattice distortion are due to a substitutional incorporation of H2 (guest) molecules into the basal plane of hcp-He (host) lattice and, thereby, reflect the miscibility between H 2 and He in H 2 -He mixtures. Interestingly, solid He exhibits a lesser degree of preferred orientation in H2-He mixtures than in pure He, likely due to the presence of solid H 2 disturbing the crystalline ordering of He-rich solids. Finally, the present PV compression data of H 2 -rich and He-rich solids to 160 GPa deviate from those of pure H 2 and pure He above ~70 and 45 GPa respectively, providing new constraints for development of the EOS for H 2 -He mixtures for planetary models.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

DOE OSTI2025

GRUMDN: A Multi-Task Model for Predicting Human Patterns-of-Life from Stay Transition Data

Understanding human patterns-of-life (PoL) is essential towards ensuring safe and secure indoor facility environment as well as outdoor urban environment. Prediction of human movement in between places of interest is vital in understanding human PoL. Movement between spaces maybe represented and detected in one of the two forms: 1) trajectories: locations measured at regular time intervals by mobile sensors, bluetooth or GPS sensors; or 2) stay transitions: semantic PoI (points of interest) and stay duration data measurable by eventbased sensors that collect data when a check-in or check-out event is detected. Stay transition data provides a more compressed data format compared to trajectories data, especially in situations with longer stay durations, while preserving the information necessary for PoL analysis. Now as introduced briefly in the paper, our deployed end application (Digital Twin of a facility with non-player characters, besides the interactive user in virtual reality) needed a well-performing and validated AI/ML model for simulating high quality stay transitions behavior. In this study we thus primarily present our findings with developing and validating that model, which is a multi-task neural network for stay transition prediction. The neural network consists of two heads, for corresponding two tasks of stay category prediction and stay duration prediction. We evaluated gated recurrent units and multi-layer perceptrons of varying network sizes for stay category prediction; while mixture density networks, noisy generator-only networks, and generative adversarial networks of varying network sizes for stay duration prediction. We have then evaluated four multi-task models, constructed by combining these specialized models, on their ability to predict stay transition data. We tested our models on datasets from two different cases: 1) a simulation-generated dataset of indoor movement within the HFIR (high flux isotope reactor) nuclear reactor facility at Oak Ridge National Laboratory (ORNL); and 2) the GeoLife human mobility dataset of outdoor urban movement available in literature. Our results indicate that GRUMDN, which combines gated recurrent units (GRU) for stay category prediction task, and mixture density networks (MDN) for stay duration prediction task, did overall outperform other multitask models and the current state-of-the-art.

Gunaratne, Chathika [ORNL] (ORCID:0000000225088745↗

DOE OSTI2023

HQ-Sim: High-performance State Vector Simulation of Quantum Circuits on Heterogeneous HPC Systems

Quantum circuit simulations are applied in more and more circum- stances as the quantum computing community becomes broader. It helps researchers to evaluate quantum algorithms and relieve the burden of limited quantum computing resources. However, most of the state-of-the-art quantum simulators utilize either CPU or GPU to store and calculate the state vector, which results in resource starvation. Moreover, the maximum number of qubits supported by the simulator is bounded by the memory, since the memory utilization increases exponentially with the number of qubits. In this study, we leverage Heterogeneous computing to utilize both CPU and GPU to store and update state vectors. We also integrate lossy data compression to reduce memory requirements. Specifically, we develop a heterogeneous framework that has a dynamic scheduler to fully utilize the computing resources. We apply lossy compression to chunked state vector to make the maximum number of qubits higher than the regular simulators, the compression also benefits the data movement between CPU and GPU.

Zhang, Boyuan↗