Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “information entropy”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
NASA NTRS1989

Simple proof of the concavity of the entropy power with respect to Gaussian noise

A very simple proof of M. H. Costa's result that the entropy power of Xt = X + N (O, tI) is concave in t, is derived as an immediate consequence of an inequality concerning Fisher information. This relationship between Fisher information and entropy is found to be useful for proving the central limit theorem. Thus, one who seeks new entropy inequalities should try first to find new inequalities about Fisher information, or at least to exploit the existing ones in new ways.

Dembo, Amir↗
NASA NTRS1996

Algorithm For A Self-Growing Neural Network

CID3 algorithm simulates self-growing neural network. Constructs decision trees equivalent to hidden layers of neural network. Based on ID3 algorithm, which dynamically generates decision tree while minimizing entropy of information. CID3 algorithm generates feedforward neural network by use of either crisp or fuzzy measure of entropy.

Cios, Krzysztof J.↗
DOE OSTI2025

Environmental Controls on Water Vapor Deuterium Excess in the Coastal Boundary Layer: An Information Theory Perspective

We use information theory to quantify the environmental controls on water vapor deuterium excess (D-excess) in coastal Southern California from June 2023 through February 2024. Using Shannon entropy, mutual information (MI), and joint mutual information, metrics that capture both linear and nonlinear relationships, we identify the most informative variables and variable combinations governing D-excess across contrasting marine and continental regimes. Relative humidity with respect to sea surface temperature (RHS) is consistently the strongest individual predictor, explaining up to 27% of D-excess variability during marine conditions but only 10% in continental air masses. The Relative humidity(RHS) + sea surface temperature (SST) combination demonstrates synergistic effects, where their joint influence (explaining up to 36% of D-excess variability) exceeds what either variable achieves individually, confirming their coupled influence on deuterium excess. Wind direction complements RHS most effectively during continental conditions. The best three-variable combination (RHS + SST + Planetary Boundary Layer height) explains 38% of D-excess variability in marine air, while no combination exceeds 20% explanatory power during continental periods. Information theory shows that heteroscedasticity in D-excess relationships indicates regime shifts in controlling processes and quantifies fundamental constraints on predictor variables: some environmental factors like surface pressure or water vapor flux contain insufficient information content to explain D-excess variability regardless of their physical relevance. These results highlight the different predictability limits between marine and continental regimes, challenging the adequacy of linear models and providing a rigorous framework for quantifying the information content of isotope-climate relationships with implications for both modern and paleoclimate applications.

information theory↗
NASA NTRS1984

Discrimination of coherent features in turbulent boundary layers by the entropy method

Entropy in information theory is defined as the expected or mean value of the measure of the amount of self-information contained in the ith point of a distribution series x sub i, based on its probability of occurrence p(x sub i). If p(x sub i) is the probability of the ith state of the system in probability space, then the entropy, E(X) = - sigma p(x sub i) logp (x sub i), is a measure of the disorder in the system. Based on this concept, a method was devised which sought to minimize the entropy in a time series in order to construct the signature of the most coherent motions. The constrained minimization was performed using a Lagrange multiplier approach which resulted in the solution of a simultaneous set of non-linear coupled equations to obtain the coherent time series. The application of the method to space-time data taken by a rake of sensors in the near-wall region of a turbulent boundary layer was presented. The results yielded coherent velocity motions made up of locally decelerated or accelerated fluid having a streamwise scale of approximately 100 nu/u(tau), which is in qualitative agreement with the results from other less objective discrimination methods.

Corke, T. C.↗
DOE OSTI2025

Entropy-based feature selection for capturing impacts in Earth system models with abrupt forcing

This paper presents the development of a new entropy-based feature selection method for identifying and quantifying impacts. Here, impacts are defined as statistically significant differences in spatio-temporal fields when comparing datasets with and without an external forcing in an Earth system model. Temporal feature selection is performed by first computing the cross-fuzzy entropy to quantify similarity of patterns between two datasets and then applying changepoint detection to identify regions of statistically constant entropy. The method is used to capture temperate north surface cooling from a 9-member simulation ensemble of the Mt. Pinatubo volcanic eruption, which injected 10 Tg of SO 2 into the stratosphere. The results estimate a mean difference decrease in near surface air temperature of -0.560 K with a 99% confidence interval between -0.864 K and -0.257 K between April and November of 1992, one year following the eruption. A sensitivity analysis with decreasing SO 2 injection revealed that the impact is statistically significant at 5 Tg but not at 3 Tg. Using identified features, a dependency graph model based on a 9-day lag had significantly fewer nodes than a graph based on monthly means. Furthermore, this demonstrates our method’s ability to perform dimension reduction while still uncovering source-to-impact pathways.

Changepoint detection↗
DOE OSTI2025

Machine learning informed rational design of high entropy double perovskite oxide universal air/steam electrodes for solid oxide electrochemical cells

Due to their high efficiency and versatility, solid oxide electrochemical cells (SOCs) are poised to play a significant role in future energy conversion and storage applications. In recent years, SOCs have bifurcated into two distinct categories: traditional oxygen-ion conducting SOCs that typically operate from ∼650—850 °C and the more recent proton-conducting ceramic (PCC) SOCs that typically operate from ∼400—650 °C. Current performance and lifetime of both oxygen-ion conducting SOCs and PCCs is primarily limited by the air/steam electrode, which facilitates the oxygen reduction reaction (ORR) during fuel cell operation and must also facilitate the oxygen evolution reaction (OER) during electrolysis operation. Here, we present a newly designed high-entropy double perovskite oxide suitable as a universal ORR/OER electrode for both oxygen-ion conducting SOCs and PCCs. Machine learning methods are applied to identify chemical descriptors for highly catalytic high-entropy double perovskite oxides (AA’B 2 O 6 ) across a large compositional space. Based on the machine-learning guidance, we ultimately converge on Ba 0.9 Cs 0.1 (Ca 0.2 Gd 0.2 La 0.2 Pr 0.2 Sr 0.2 )Co 1.5 Fe 0.5 O 6 (CsBaHEO) as a universal air/steam electrode. Structure stabilization is accomplished by an equimolar five-cation high-entropy composition on the A’-site, while cesium substitution on the A-site enhances the electrical conductivity and leads to a higher oxygen vacancy concentration. This material exhibits versatility and high performance in reversible oxygen-ion SOCs, reversible PCCs, and also large-scale tubular PCCs. For example, the CsBaHEO-based PCC reaches 1018 mW∙cm −2 at 600°C, while a large-scale tubular PCC using CsBaHEO for electrolysis achieves a hydrogen production rate of 21.314 ML∙min −1 at 600 °C.

Cell↗
DOE OSTI2020

Machine-learning informed prediction of high-entropy solid solution formation: Beyond the Hume-Rothery rules

The empirical rules for the prediction of solid solution formation proposed so far in the literature usually have very compromised predictability. Some rules with seemingly good predictability were, however, tested using small data sets. Based on an unprecedented large dataset containing 1252 multicomponent alloys, machine-learning methods showed that the formation of solid solutions can be very accurately predicted (93%). The machine-learning results help identify the most important features, such as molar volume, bulk modulus, and melting temperature. As such a new thermodynamics-based rule was developed to predict solid–solution alloys. The new rule is nonetheless slightly less accurate (73%) but has roots in the physical nature of the problem. The new rule is employed to predict solid solutions existing in the three blocks, each of which consists of 9 elements. The predictions encompass face-centered cubic (FCC), body-centered cubic (BCC), and hexagonal closest packed (HCP) structures in a high throughput manner. The validity of the prediction is further confirmed by CALculations of PHAse Diagram (CALPHAD) calculations with high consistency (94%). Since the new thermodynamics-based rule employs only elemental properties, applicability in screening for solid solution high-entropy alloys is straightforward and efficient.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Entanglement phase transitions in random stabilizer tensor networks

Here we explore a class of random tensor network models with “stabilizer” local tensors which we name random stabilizer tensor networks (RSTNs). For RSTNs defined on a two-dimensional square lattice, we perform extensive numerical studies of entanglement phase transitions between volume-law and area-law entangled phases of the one-dimensional boundary states. These transitions occur when either (a) the bond dimension D of the constituent tensors is varied or (b) the tensor network is subject to random breaking of bulk bonds, implemented by forced measurements. In the absence of broken bonds, we find that the RSTN supports a volume-law entangled boundary state with bond dimension D ≥ 3 where D is a prime number, and an area-law entangled boundary state for D = 2. Upon breaking bonds at random in the bulk with probability p, there exists a critical measurement rate p c for each D ≥ 3 above which the boundary state becomes area-law entangled. To explore the conformal invariance at these entanglement transitions for different prime D, we consider tensor networks on a finite rectangular geometry with a variety of boundary conditions, and extract universal operator scaling dimensions via extensive numerical calculations of the entanglement entropy, mutual information, and mutual negativity at their respective critical points. Our results at large D approach known universal data of percolation conformal field theory, while showing clear discrepancies at smaller D, suggesting a distinct entanglement transition universality class for each prime D. We further study universal entanglement properties in the volume-law phase and demonstrate quantitative agreement with the recently proposed description in terms of a directed polymer in a random environment.

36 MATERIALS SCIENCE↗
DOE OSTI2025

FedEFsz: Fair Cross-Silo Federated Learning System With Error-Bounded Lossy Compression

Cross-Silo federated learning systems have been identified as an efficient approach to scaling DNN training across geographically-distributed data silos to preserve the privacy of the training data. Communication efficiency and fairness are two major issues that need to be both satisfied when federated learning systems are deployed in practice. Simultaneously guaranteeing both of them, however, is exceptionally difficult because simply combining communication reduction and fairness optimization approaches often causes non-converged training or drastic accuracy degradation. Here, to bridge this gap, we propose FedEFsz. On the one hand, it integrates the state-of-the-art error-bounded lossy compressor SZ3 into cross-silo federated learning systems to significantly reduce communication traffic during the training. On the other hand, it achieves a high fairness (i.e., rather consistent model accuracy and performance across different clients) through a carefully designed heuristic algorithm that can tune the error-bound of SZ3 for different clients during the training. Extensive experimental results based on a GPU cluster with 65 GPU cards show that FedEFsz improves the fairness across different benchmarks by up to 60.88% and meanwhile reduces the communication traffic by up to 315×.

Cross-Silo Federated Learning Systems↗
NASA NTRS2004

The Generalization of Mutual Information as the Information between a Set of Variables: The Information Correlation Function Hierarchy and the Information Structure of Multi-Agent Systems

The topic of this paper is a hierarchy of information-like functions, here named the information correlation functions, where each function of the hierarchy may be thought of as the information between the variables it depends upon. The information correlation functions are particularly suited to the description of the emergence of complex behaviors due to many- body or many-agent processes. They are particularly well suited to the quantification of the decomposition of the information carried among a set of variables or agents, and its subsets. In more graphical language, they provide the information theoretic basis for understanding the synergistic and non-synergistic components of a system, and as such should serve as a forceful toolkit for the analysis of the complexity structure of complex many agent systems. The information correlation functions are the natural generalization to an arbitrary number of sets of variables of the sequence starting with the entropy function (one set of variables) and the mutual information function (two sets). We start by describing the traditional measures of information (entropy) and mutual information.

Wolf, David R.↗
NASA NTRS1991

An information theory approach for evaluating earth radiation budget (ERB) measurements - Nonuniform sampling of diurnal longwave flux variations

Satellite measurements are subject to a wide range of uncertainties due to their temporal, spatial, and directional sampling characteristics. An information-theory approach is suggested to examine the nonuniform temporal sampling of ERB measurements. The information (i.e., its entropy or uncertainty) before and after the measurements is determined, and information gain (IG) is defined as a reduction in the uncertainties involved. A stochastic model for the diurnal outgoing flux variations that affect the ERB is developed. Using Gaussian distributions for the a priori and measured radiant exitance fields, the IG is obtained by computing the a posteriori covariance. The IG for the monthly outgoing flux measurements is examined for different orbital parameters and orbital tracks, using the Earth Observing System orbital parameters as specific examples. Variations in IG due to changes in the orbit's inclination angle and the initial ascending node local time are investigated.

Halyo, Nesim↗
NASA NTRS1994

A robot control formalism based on an information quality concept

A relevance measure based on Jaynes maximum entropy principle is introduced. Information quality is the conjunction of accuracy and relevance. The formalism based on information quality is developed for one-agent applications. The robot requires a well defined working environment where properties of each object must be accurately specified.

Ekman, A.↗
DOE OSTI2023

Thermodynamics of signal transduction systems and fluctuation theorem in a signal cascade

Biochemical chain reactions are signal transduction cascades that can transmit biological information about the intracellular environment. In this study, we modelled a chain reaction as a code string for applying information theory. Herein, we assumed that cell signal transduction selects a strategy to maximize the transduced signal per signal event duration. To investigate the same, we calculated the information transmission capacity of the reaction chain by maximizing the average entropy production rate per reaction time, indicating the idea of the entropy coding method. Moreover, we defined a signal cascade trajectory. Subsequently, we found that the logarithm of the forward and reverse transition ratio per reaction time is equal to the entropy production rate, which derives the form of the fluctuation theorem in signal transduction. Our findings suggest the application of information entropy theory for analysing signal transduction.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Theory, observations, and simulations of kinetic entropy in a magnetotail electron diffusion region

In this work, we examine velocity-space kinetic entropy, a spatially local measure of entropy for systems out of thermal equilibrium, during an encounter of an electron diffusion region at a magnetic reconnection site in Earth’s magnetotail by the Magnetospheric Multiscale (MMS) mission. We start by generalizing the theory of kinetic entropy to the case of non-uniform velocity space grids and transforming the equations into spheri- cal energy coordinates useful to experimental plasma detectors. The theory is then applied to MMS data and compared to particle-in-cell simulations of reconnection. We demonstrate that the entropy-based non-Maxwellianity measure from the MMS data is of sufficiently high precision to reliably identify non-Maxwellian distributions and therefore the measurements when kinetic effects are most significant. By comparing two different non-Maxwellian measures, we show that total entropy density suffers from “information loss” because it lacks a dependence on the velocity space grid, and so has lost information about how well a distribution function is resolved. Local velocity-space kinetic entropy density recovers this information. We quantify information loss and argue that the considerations needed to minimize it are crucial for instruments designed to measure distribution functions in situ.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Quantum Extremal Surfaces and the Holographic Entropy Cone

Quantum states with geometric duals are known to satisfy a stricter set of entropy inequalities than those obeyed by general quantum systems. The set of allowed entropies derived using the Ryu-Takayanagi (RT) formula defines the Holographic Entropy Cone (HEC). These inequalities are no longer satisfied once general quantum corrections are included by employing the Quantum Extremal Surface (QES) prescription. Nevertheless, the structure of the QES formula allows for a controlled study of how quantum contributions from bulk entropies interplay with HEC inequalities. In this paper, we initiate an exploration of this problem by relating bulk entropy constraints to boundary entropy inequalities. In particular, we show that requiring the bulk entropies to satisfy the HEC implies that the boundary entropies also satisfy the HEC. Further, we also show that requiring the bulk entropies to obey monogamy of mutual information (MMI) implies the boundary entropies also obey MMI.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

Channel capacity: limitation of entropy rate during cell signal transduction

Cellular signal transduction is mediated through chain activation reactions involving signalling molecules. Information thermodynamics may be a promising approach for quantifying signal transduction. A cell signal is transduced through the signal molecule activation as a step of a signal transduction cascade. In each step, two types of entropy can be defined. First, the signal transduction was defined using thermodynamic entropy production during the chemical activation of signal molecules. Second, the chronology was described as a code string, and the logarithm of the number of signal code strings corresponded to the information entropy. Finally, the total entropy during signal transduction was determined by summing the two entropies. When maximising the total entropy, the total entropy rate per signalling molecule concentration was found to be consistent independently of each transduction step (i.e., signal molecule types). Consequently, the speed limit of the entropy rate can be calculated, providing the capacity, which is an essential property of the signal transduction cascade. The theory introduced here is expected to provide a basic framework for understanding new methods of discovering signal cascades and intracellular signal transduction.

59 BASIC BIOLOGICAL SCIENCES↗