Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Information theory entropy”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Thermodynamics of signal transduction systems and fluctuation theorem in a signal cascade

Biochemical chain reactions are signal transduction cascades that can transmit biological information about the intracellular environment. In this study, we modelled a chain reaction as a code string for applying information theory. Herein, we assumed that cell signal transduction selects a strategy to maximize the transduced signal per signal event duration. To investigate the same, we calculated the information transmission capacity of the reaction chain by maximizing the average entropy production rate per reaction time, indicating the idea of the entropy coding method. Moreover, we defined a signal cascade trajectory. Subsequently, we found that the logarithm of the forward and reverse transition ratio per reaction time is equal to the entropy production rate, which derives the form of the fluctuation theorem in signal transduction. Our findings suggest the application of information entropy theory for analysing signal transduction.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

ET-AL: Entropy-targeted active learning for bias mitigation in materials data

Growing materials data and data-driven informatics drastically promote the discovery and design of materials. While there are significant advancements in data-driven models, the quality of data resources is less studied despite its huge impact on model performance. In this work, we focus on data bias arising from uneven coverage of materials families in existing knowledge. Observing different diversities among crystal systems in common materials databases, we propose an information entropy-based metric for measuring this bias. To mitigate the bias, we develop an entropy-targeted active learning (ET-AL) framework, which guides the acquisition of new data to improve the diversity of underrepresented crystal systems. We demonstrate the capability of ET-AL for bias mitigation and the resulting improvement in downstream machine learning models. This approach is broadly applicable to data-driven materials discovery, including autonomous data acquisition and dataset trimming to reduce bias, as well as data-driven informatics in other scientific domains.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Method of information entropy for convergence assessment of molecular dynamics simulations

The lack of a reliable method to evaluate the convergence of molecular dynamics simulations has contributed to discrepancies in different areas of molecular dynamics. Here, the method of information entropy is introduced to molecular dynamics for stationarity assessment. The Shannon information entropy formalism is used to monitor the convergence of the atom motion to a steady state in a continuous spatial domain and is also used to assess the stationarity of calculated multidimensional fields such as the temperature field in a discrete spatial domain. It is demonstrated in this work that monitoring the information entropy of the atom position matrix provides a clear indicator of reaching steady state in radiation damage simulations, non-equilibrium molecular dynamics thermal conductivity computations, and simulations of Poiseuille and Couette flow in nanochannels. A main advantage of the present technique is that it is non-local and relies on fundamental quantities available in all molecular dynamics simulations. Unlike monitoring average temperature, the technique is applicable to simulations that conserve total energy such as reverse non-equilibrium molecular dynamics thermal conductivity computations and to simulations where energy dissipates through a boundary as in radiation damage simulations. The method is applied to simulations of iron using the Tersoff/ZBL splined potential, silicon using the Stillinger–Weber potential, and to Lennard–Jones fluid. Its applicability to both solids and fluids shows that the technique has potential for generalization to other areas in molecular dynamics.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2026

Complexity analysis of a CT injection experiment on BRB

In this work, we use Jensen–Shannon complexity and permutation entropy to analyze the magnetic field fluctuations of an astrophysically scaled plasma experiment. The experiment was intended to emulate an interplanetary coronal mass ejection event in the lab, recreating the major sections seen in satellite data. We also use a technique called “delay,” in which we use select elements, skipping one or more data points at a time, in our time series data to obtain Jensen–Shannon complexity as a function of frequency and investigate the frequency of maximized complexity. We then compare the delay frequencies to other frequencies in the plasma. We found that the frequencies for maximum complexity do not correspond to the frequencies investigated, implying that other physical mechanisms lead to an increase in complexity at these frequencies.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2026

Structural and compositional complexities of hierarchical self-assembly: A hypergraph approach

Programmable self-assembly enables the construction of complex molecular, supramolecular, and crystalline architectures from well-designed building blocks. In this work, we introduce a hypergraph-based formalism, Blocks & Bonds (B&B), which generalizes classical chemical graph theory by incorporating directed and multicolored interactions, internal symmetries, and hierarchical organization. Within this framework, we develop the Structure Code (SC), a compact and versatile language for describing self-assembled architectures. We define a Kolmogorov-style structural complexity as the total information content of SC, obtained through its tokenization and Shannon information assignment. Complementing this encoding-based measure, we introduce a much simpler quantity, the compositional complexity, which depends only on the number and cumulative usage of block and bond types in the construction set. A central result of this work is a strong empirical correlation between the token-based structural complexity and the compositional complexity across all examined systems. Owing to this agreement, the compositional complexity emerges as the most practical and broadly applicable measure: it is easy to compute, requires no explicit encoding, and yet closely tracks the actual information content of structurally diverse architectures. Applications to molecular systems (ethylene glycol and glucose), DNA-origami lattices, and crystalline assemblies show that B&B hypergraphs provide a unified, scalable, and information-efficient representation of structural organization, naturally capturing symmetry, modularity, and stereochemistry. This framework establishes a quantitative foundation for complexity-aware classification and inverse design of programmable matter.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Maximizing efficiency of dataset compression for machine learning potentials with information theory

Machine learning interatomic potentials (MLIPs) balance high accuracy and lower costs compared to density functional theory calculations, but their performance often depends on the size and diversity of training datasets. Large datasets improve model accuracy and generalization but are computationally expensive to produce and train on, while smaller datasets risk discarding rare but important atomic environments and compromising MLIP accuracy/reliability. Here, we develop an information-theoretical framework to quantify the efficiency of dataset compression methods and propose an algorithm that maximizes this efficiency. By framing atomistic dataset compression as an instance of the minimum set cover (MSC) problem over atom-centered environments, our method identifies the smallest subset of structures that contains as much information as possible from the original dataset while pruning redundant information. The approach is extensively demonstrated on the GAP-20 and TM23 datasets and validated on 64 varied datasets from the ColabFit repository. Across all cases, MSC consistently retains outliers, preserves dataset diversity, and reproduces the long-tail distributions of forces even at high compression rates, outperforming other subsampling methods. Furthermore, MLIPs trained on MSC-compressed datasets exhibit reduced error for out-of-distribution data even in low-data regimes. We explain these results using an outlier analysis and show that such quantitative conclusions could not be achieved with conventional dimensionality reduction methods. The algorithm is implemented in the open-source QUESTS package and can be used for several tasks in atomistic modeling, from data subsampling, outlier detection, and training improved MLIPs at a lower cost.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Vibrational Entropy of Crystalline Solids from Covariance of Atomic Displacements

The vibrational entropy of a solid at finite temperature is investigated from the perspective of information theory. Ab initio molecular dynamics (AIMD) simulations generate ensembles of atomic configurations at finite temperature from which we obtain the N-body distribution of atomic displacements, ρN. We calculate the information-theoretic entropy from the expectation value of lnρN. At a first level of approximation, treating individual atomic displacements independently, our method may be applied using Debye–Waller B-factors, allowing diffraction experiments to obtain an upper bound on the thermodynamic entropy. At the next level of approximation we correct the overestimation through inclusion of displacement covariances. We apply this approach to elemental body-centered cubic sodium and face-centered cubic aluminum, showing good agreement with experimental values above the Debye temperatures of the metals. Below the Debye temperatures, we extract an effective vibrational density of states from eigenvalues of the covariance matrix, and then evaluate the entropy quantum mechanically, again yielding good agreement with experiment down to low temperatures. Our method readily generalizes to complex solids, as we demonstrate for a high entropy alloy. Further, our method applies in cases where the quasiharmonic approximation fails, as we demonstrate by calculating the HCP/BCC transition in Ti.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2025

Entropy of the Quantum–Classical Interface: A Potential Metric for Security

Hybrid quantum–classical systems are emerging as key platforms in quantum computing, sensing, and communication technologies, but the quantum–classical interface (QCI)—the boundary enabling these systems—introduces unique and largely unexplored security vulnerabilities. This position paper proposes using entropy-based metrics to monitor and enhance security, specifically at the QCI. We present a theoretical security outline that leverages well-established information-theoretic entropy measures, such as Shannon entropy, von Neumann entropy, and quantum relative entropy, to detect anomalous behaviors and potential breaches at the QCI. By linking entropy fluctuations to scenarios of practical relevance—including quantum key distribution, quantum sensing, and hybrid control systems—we promote the potential value and applicability of entropy-based security monitoring. While explicitly acknowledging practical limitations and theoretical assumptions, we argue that entropy-based metrics provide a complementary approach to existing security methods, inviting further empirical studies and theoretical refinements that can strengthen future quantum technologies.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Entropy and its Relationship with Statistics

The purpose of our report is to discuss the notion of entropy and its relationship with statistics. Our goal is to provide a manner in which you can think about entropy, its central role within information theory and relationship with statistics. We review various relationships between information theory and statistics—nearly all are well-known but unfortunately are often not recognized. Entropy quantities the "average amount of surprise" in a random variable and lies at the heart of information theory, which studies the transmission, processing, extraction, and utilization of information. For us, data is information. What is the distinction between information theory and statistics? Information theorists work with probability distributions. Instead, statisticians work with samples. In so many words, information theory using samples is the practice of statistics.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Deforestation reshapes land-surface energy-flux partitioning

Land-use and land-cover change significantly modify local land-surface characteristics and water/energy exchanges, which can lead to atmospheric circulation and regional climate changes. In particular, deforestation accounts for a large portion of global land-use changes, which transforms forests into other land cover types, such as croplands and grazing lands. Many previous efforts have focused on observing and modeling land–atmosphere–water/energy fluxes to investigate land–atmosphere coupling induced by deforestation. However, interpreting land–atmosphere–water/energy-flux responses to deforestation is often complicated by the concurrent impacts from shifts in land-surface properties versus background atmospheric forcings. In this study, we used 29 paired FLUXNET sites, to improve understanding of how deforested land surfaces drive changes in surface-energy-flux partitioning. Each paired sites included an intact forested and non-forested site that had similar background climate. We employed transfer entropy, a method based on information theory, to diagnose directional controls between coupling variables, and identify nonlinear cause–effect relationships. Transfer entropy is a powerful tool to detective causal relationships in nonlinear and asynchronous systems. The paired eddy covariance flux measurements showed consistent and strong information flows from vegetation activity (gross primary productivity (GPP)) and physical climate (e.g. shortwave radiation, air temperature) to evaporative fraction (EF) over both non-forested and forested land surfaces. More importantly, the information transfers from radiation, precipitation, and GPP to EF were significantly reduced at non-forested sites, compared to forested sites. We then applied these observationally constrained metrics as benchmarks to evaluate the Energy Exascale Earth System Model (E3SM) land model (ELM). ELM predicted vegetation controls on EF relatively well, but underpredicted climate factors on EF, indicating model deficiencies in describing the relationships between atmospheric state and surface fluxes. Moreover, changes in controls on surface energy flux partitioning due to deforestation were not detected in the model. We highlight the need for benchmarking model simulated surface-energy fluxes and the corresponding causal relationships against those of observations, to improve our understanding of model predictability on how deforestation reshapes land surface energy fluxes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2019

On the General Definition of the Production of Entropy in Open Markov Quantum Systems

We discuss the question of the general definition of the production of entropy per unit time for a quantum system governed by the Lindblad equation. The difficulty is as follows: in order to determine the total production of entropy, one must know the entropy flow from the system into the environment. This requires additional information on the environment and on its interaction with the system. The Lindblad equation for the reduced density matrix of the system does not contain such information. Therefore, the following question arises: What minimum additional information about the environment must be added to the Lindblad equation in order to find the flow of entropy into the environment and the total production of entropy? To answer this question, we use the concept of a complementary quantum channel known from the the quantum information theory. We also prove a theorem on the nonnegativity of production of entropy, and, under certain assumptions, the adiabatic and nonadiabatic contribution to it.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Exploring Hydrologic Model Process Connectivity at the Continental Scale Through an Information Theory Approach

Exploring water fluxes between hydrological model (HM) components is essential to assess and improve model realism. Many classical metrics for HM diagnosis rely solely on streamflow and hence provide limited insights into model performance across processes. This study applies an information theory measure known as “transfer entropy” (TE) to systematically quantify the transfer of information among major HM components. To test and demonstrate the benefits of TE, we use the Framework for Understanding Structural Errors (FUSE) model to mimic and compare four commonly used HM structures, VIC, PRMS, SACRAMENTO, and TOPMODEL, across 671 catchments spanning a variety of hydrologic regimes in the conterminous United States. We explore connections between HM components and catchment landscape characteristics (e.g., climate, topography, soil, and vegetation) and characterize their nonlinear associations using distance correlation and Spearman correlation coefficients. Our results indicate that while the information transferred from precipitation to runoff is similar across model structures (likely as a result of calibration), the information transferred among other components can vary significantly from a FUSE structure to another. We find that aridity, precipitation duration and frequency, snow fraction, mean elevation, forest area, and leaf area index are often significantly associated with TE between the main HM components. Here, we propose that the presence of meaningful nonlinear associations can be used to diagnose process representation in HMs. Our results highlight the necessity to enhance the conventional streamflow-only calibration approach for a more realistic representation of water dynamics in the models.

large-sample hydrology↗
DOE OSTI2021

Revisiting Huffman Coding: Toward Extreme Performance on Modern GPU Architectures

Today's high-performance computing (HPC) applications are producing vast volumes of data, which are challenging to store and transfer efficiently during the execution, such that data compression is becoming a critical technique to mitigate the storage burden and data movement cost. Huffman coding is arguably the most efficient Entropy coding algorithm in information theory, such that it could be found as a fundamental step in many modern compression algorithms such as DEFLATE. On the other hand, today's HPC applications are more and more relying on the accelerators such as GPU on supercomputers, while Huffman encoding suffers from low throughput on GPUs, resulting in a significant bottleneck in the entire data processing. In this paper, we propose and implement an efficient Huffman encoding approach based on modern GPU architectures, which addresses two key challenges: (1) how to parallelize the entire Huffman encoding algorithm, including codebook construction, and (2) how to fully utilize the high memory-bandwidth feature of modern GPU architectures. The detailed contribution is fourfold. (1) We develop an efficient parallel codebook construction on GPUs that scales effectively with the number of input symbols. (2) We propose a novel reduction based encoding scheme that can efficiently merge the codewords on GPUs. (3) We optimize the overall GPU performance by leveraging the state-of-the-art CUDA APIs such as Cooperative Groups. (4) We evaluate our Huffman encoder thoroughly using six real-world application datasets on two advanced GPUs and compare with our implemented multi-threaded Huffman encoder. Experiments show that our solution can improve the encoding throughput by up to 5.0x and 6.8x on NVIDIA RTX 5000 and V100, respectively, over the state-of-the-art GPU Huffman encoder, and by up to 3.3x over the multi-thread encoder on two 28-core Xeon Platinum 8280 CPUs.

Tian, Jiannan↗
DOE OSTI2021

Revisiting Huffman Coding: Toward Extreme Performance on Modern GPU Architectures

Today's high-performance computing (HPC) applications are producing vast volumes of data, which are challenging to store and transfer efficiently during the execution, such that data compression is becoming a critical technique to mitigate the storage burden and data movement cost. Huffman coding is arguably the most efficient Entropy coding algorithm in information theory, such that it could be found as a fundamental step in many modern compression algorithms such as DEFLATE. On the other hand, today's HPC applications are more and more relying on the accelerators such as GPU on supercomputers, while Huffman encoding suffers from low throughput on GPUs, resulting in a significant bottleneck in the entire data processing. In this paper, we propose and implement an efficient Huffman encoding approach based on modern GPU architectures, which addresses two key challenges: (1) how to parallelize the entire Huffman encoding algorithm, including codebook construction, and (2) how to fully utilize the high memory-bandwidth feature of modern GPU architectures. The detailed contribution is fourfold. (1) We develop an efficient parallel codebook construction on GPUs that scales effectively with the number of input symbols. (2) We propose a novel reduction based encoding scheme that can efficiently merge the codewords on GPUs. (3) We optimize the overall GPU performance by leveraging the state-of-the-art CUDA APIs such as Cooperative Groups. (4) We evaluate our Huffman encoder thoroughly using six real-world application datasets on two advanced GPUs and compare with our implemented multithreaded Huffman encoder. Experiments show that our solution can improve the encoding throughput by up to 5.0× and 6.8× on NVIDIA RTX 5000 and V100, respectively, over the state-of-the-art GPU Huffman encoder, and by up to 3.3× over the multithread encoder on two 28-core Xeon Platinum 8280 CPUs.

Tian, Jiannan↗
DOE OSTI2024

Entropy removal of medical diagnostics

Shannon entropy is a core concept in machine learning and information theory, particularly in decision tree modeling. To date, no studies have extensively and quantitatively applied Shannon entropy in a systematic way to quantify the entropy of clinical situations using diagnostic variables (true and false positives and negatives, respectively). Decision tree representations of medical decision-making tools can be generated using diagnostic variables found in literature and entropy removal can be calculated for these tools. This concept of clinical entropy removal has significant potential for further use to bring forth healthcare innovation, such as quantifying the impact of clinical guidelines and value of care and applications to Emergency Medicine scenarios where diagnostic accuracy in a limited time window is paramount. This analysis was done for 623 diagnostic tools and provided unique insights into their utility. For studies that provided detailed data on medical decision-making algorithms, bootstrapped datasets were generated from source data to perform comprehensive machine learning analysis on these algorithms and their constituent steps, which revealed a novel and thorough evaluation of medical diagnostic algorithms.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Transient anisotropic kernel for probabilistic learning on manifolds

PLoM (Probabilistic Learning on Manifolds) is a method introduced in 2016 for handling small training datasets by projecting an Itô equation from a stochastic dissipative Hamiltonian dynamical system, acting as the MCMC generator, for which the KDE-estimated probability measure with the training dataset is the invariant measure. PLoM performs a projection on a reduced-order vector basis related to the training dataset, using the diffusion maps (DMAPS) basis constructed with a time-independent isotropic kernel. In this paper, we propose a new ISDE projection vector basis built from a transient anisotropic kernel, providing an alternative to the DMAPS basis to improve statistical surrogates for stochastic manifolds with heterogeneous data. The construction ensures that for times near the initial time, the DMAPS basis coincides with the transient basis. For larger times, the differences between the two bases are characterized by the angle of their spanned vector subspaces. The optimal instant yielding the optimal transient basis is determined using an estimation of mutual information from Information Theory, which is normalized by the entropy estimation to account for the effects of the number of realizations used in the estimations. Consequently, this new vector basis better represents statistical dependencies in the learned probability measure for any dimension. Three applications with varying levels of statistical complexity and data heterogeneity validate the proposed theory, showing that the transient anisotropic kernel improves the learned probability measure.

Diffusion maps↗
DOE OSTI2022

Misanthropic entropy and renormalization as a communication channel

A central physical question is the extent to which infrared (IR) observations are sufficient to reconstruct a candidate ultraviolet (UV) completion. We recast this question as a problem of communication, with messages encoded in field configurations of the UV being transmitted to the IR degrees of freedom via a noisy channel specified by renormalization group (RG) flow, with noise generated by coarse graining/decimation. We present an explicit formulation of these considerations in terms of lattice field theory, where we show that the “misanthropic entropy” — the mutual information obtained from decimating neighbors — encodes the extent to which information is lost in marginalizing over/tracing out UV degrees of freedom. Our considerations apply both to statistical field theories as well as density matrix renormalization of quantum systems, where in the quantum case, the statistical field theory analysis amounts to a leading-order approximation. As a concrete example, we focus on the case of the 2D Ising model, where we show that the misanthropic entropy detects the onset of the phase transition at the Ising model critical point.

Physics↗