SEARCH · Engineering Papers

Results for “information entropy”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

All sources NASA NTRS DOE OSTI

DOE OSTI2025

Bayesian Entropy Neural Networks for physics-aware prediction

This article addresses the need for deep learning models to integrate well-defined constraints into their outputs, driven by their application in surrogate models, learning with limited data and partial information, and scenarios requiring flexible model behavior to incorporate non-data sample information. We introduce Bayesian Entropy Neural Networks (BENN), a framework grounded in Maximum Entropy (MaxEnt) principles, designed to impose constraints on Bayesian Neural Network (BNN) predictions. BENN is capable of constraining not only the predicted values but also their derivatives and variances, ensuring a more robust and reliable model output. To achieve simultaneous uncertainty quantification and constraint satisfaction, we employ the method of multipliers approach. This allows for the concurrent estimation of neural network parameters and the Lagrangian multipliers associated with the constraints. Our experiments, spanning diverse applications such as beam deflection modeling and microstructure generation, demonstrate the effectiveness of BENN. The results highlight significant improvements over traditional BNNs and showcase competitive performance relative to contemporary constrained deep learning methods.

14 SOLAR ENERGY↗

DOE OSTI2025

Entropy-Assisted Quality Pattern Identification in Finance

Short-term patterns in financial time series form the cornerstone of many algorithmic trading strategies, yet extracting these patterns reliably from noisy market data remains a formidable challenge. In this paper, we propose an entropy-assisted framework for identifying high-quality, non-overlapping patterns that exhibit consistent behavior over time. We ground our approach in the premise that historical patterns, when accurately clustered and pruned, can yield substantial predictive power for short-term price movements. To achieve this, we incorporate an entropy-based measure as a proxy for information gain: patterns that lead to high one-sided movements in historical data yet retain low local entropy are more “informative” in signaling future market direction. Compared to conventional clustering techniques such as K-means and Gaussian Mixture Models (GMMs), which often yield biased or unbalanced groupings, our approach emphasizes balance over a forced visual boundary, ensuring that quality patterns are not lost due to over-segmentation. By emphasizing both predictive purity (low local entropy) and historical profitability, our method achieves a balanced representation of Buy and Sell patterns, making it better suited for short-term algorithmic trading strategies. This paper offers an in-depth illustration of our entropy-assisted framework through two case studies on Gold vs. USD and GBPUSD. While these examples demonstrate the method’s potential for extracting high-quality patterns, they do not constitute an exhaustive survey of all possible asset classes.

Physics↗

DOE OSTI2023

Temperature Uncertainty Modeling with Proxy Structural Data as Geostatistical Constraints for Well Siting: An Example Applied to Granite Springs Valley, NV, USA

Utilizing existing temperature and structural information around Granite Springs Valley, Nevada, we build 3D stochastic temperature models with the aim of evaluating the 3D uncertainty of temperature and choosing between candidate exploration well locations . The data used to support the modeling are measured temperatures and structural proxies from 3D geologic modeling, the latter considered "secondary" data. Two stochastic geostatistical techniques are explored for incorporating the structural proxies: cosimulation and local varying mean. With both the cosimulation and local varying mean methods, many equally likely temperature models (i.e., realizations) are produced, from which temperature probability profiles are calculated at candidate well locations. To aid in choosing between the candidate locations, two quantities summarize the temperature probabilities: Vprior and entropy. Vprior quantifies the likelihood for economic temperatures at each candidate location, whereas entropy identifies where new information has the most potential to reduce uncertainty. In general, the cosimulation realizations have smoother spatial structure, and extrapolate high temperatures at candidate locations that are located along the direction of the longest spatial correlation, which are down dip from existing temperature logs. The smooth realizations result in tight temperature probability profiles that are easier to interpret, but they have unrealistic temperature reversals in some locations because the cosimulation technique does not enforce a conductive geothermal gradient as a baseline (i.e., linearly increasing temperature with depth). The local varying mean results produce realizations with more realistic geothermal gradients, with temperatures increasing downward since a depth-temperature relationship is included. However, because they have much noisier spatial nature compared to cosimulation, it is harder to interpret the temperature probability profiles. The different local varying mean results allow the geologist to determine which proxy (e.g., dilation versus distance to fault termination) should be used given the specific geothermal system. In general, Vprior from local varying mean results identify locations that are close to high values for the structural proxies: areas with highe r probabilities for higher temperatures. The entropy results identify where uncertainty is greatest and therefore new drilling information could be most useful. Though these techniques provide useful information, even when applied to areas of sparse data, our comp arison of these two techniques demonstrates the need for new geothermal geostatistics techniques that combine the advantages of these two methods and that are tailored to the spatial uncertainty issues inherent in geothermal exploration.

3D temperature modeling↗

DOE OSTI2023

Temperature uncertainty modelling with proxy structural data as geostatistical constraints for well siting: an example applied to Granite Springs Valley, NV, USA

Utilizing existing temperature and structural geology information around Granite Springs Valley, Nevada, we build 3D stochastic temperature models with the aims of evaluating the 3D uncertainty of temperature and choosing between candidate exploration well locations. The data used to support the modelling are measured temperatures and structural proxies from 3D geologic modelling (distance to fault, distance to fault intersections and terminations, Coulomb stress change and dilation tendency), the latter considered ‘secondary’ data. Two stochastic geostatistical techniques are explored for incorporating the structural proxies: cosimulation and local varying mean. With both the cosimulation and local varying mean methods, many equally-likely temperature models (i.e. realizations) are produced, from which temperature probability profiles are calculated at candidate well locations. To aid in choosing between the candidate locations, two quantities summarize the temperature probabilities: V prior and entropy. V prior quantifies the likelihood for economic temperatures at each candidate location, whereas entropy identifies where new information has the most potential to reduce uncertainty. In general, the cosimulation realizations have smoother spatial structure, and extrapolate high temperatures at candidate locations that are located along the direction of the longest spatial correlation, which are down dip from existing temperature logs. The smooth realizations result in tight temperature probability profiles that are easier to interpret, but they have unrealistic temperature reversals in some locations because of the dipping ellipsoid shape created and that the cosimulation technique does not enforce a conductive geothermal gradient as a baseline (i.e. linearly increasing temperature with depth). The local varying mean results produce realizations with more realistic geothermal gradients, with temperatures increasing downward since a depth-temperature relationship is included. However, because they have much noisier spatial nature compared to cosimulation, it is harder to interpret the temperature probability profiles. The different local varying mean results allow the geologist to determine which proxy (e.g. dilation v. distance to fault termination) should be used given the specific geothermal system. In general, V prior from local varying mean results identify locations that are close to high values for the structural proxies: areas with higher probabilities for higher temperatures. The entropy results identify where uncertainty is greatest and therefore new drilling information could be most useful. Though these techniques provide useful information, even when applied to areas of sparse data, our comparison of these two techniques demonstrates the need for new geothermal geostatistics techniques that combine the advantages of these two methods and that are tailored to the spatial uncertainty issues inherent in geothermal exploration.

15 GEOTHERMAL ENERGY↗

DOE OSTI2022

Entropy and its Relationship with Statistics

The purpose of our report is to discuss the notion of entropy and its relationship with statistics. Our goal is to provide a manner in which you can think about entropy, its central role within information theory and relationship with statistics. We review various relationships between information theory and statistics—nearly all are well-known but unfortunately are often not recognized. Entropy quantities the "average amount of surprise" in a random variable and lies at the heart of information theory, which studies the transmission, processing, extraction, and utilization of information. For us, data is information. What is the distinction between information theory and statistics? Information theorists work with probability distributions. Instead, statisticians work with samples. In so many words, information theory using samples is the practice of statistics.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Deforestation reshapes land-surface energy-flux partitioning

Land-use and land-cover change significantly modify local land-surface characteristics and water/energy exchanges, which can lead to atmospheric circulation and regional climate changes. In particular, deforestation accounts for a large portion of global land-use changes, which transforms forests into other land cover types, such as croplands and grazing lands. Many previous efforts have focused on observing and modeling land–atmosphere–water/energy fluxes to investigate land–atmosphere coupling induced by deforestation. However, interpreting land–atmosphere–water/energy-flux responses to deforestation is often complicated by the concurrent impacts from shifts in land-surface properties versus background atmospheric forcings. In this study, we used 29 paired FLUXNET sites, to improve understanding of how deforested land surfaces drive changes in surface-energy-flux partitioning. Each paired sites included an intact forested and non-forested site that had similar background climate. We employed transfer entropy, a method based on information theory, to diagnose directional controls between coupling variables, and identify nonlinear cause–effect relationships. Transfer entropy is a powerful tool to detective causal relationships in nonlinear and asynchronous systems. The paired eddy covariance flux measurements showed consistent and strong information flows from vegetation activity (gross primary productivity (GPP)) and physical climate (e.g. shortwave radiation, air temperature) to evaporative fraction (EF) over both non-forested and forested land surfaces. More importantly, the information transfers from radiation, precipitation, and GPP to EF were significantly reduced at non-forested sites, compared to forested sites. We then applied these observationally constrained metrics as benchmarks to evaluate the Energy Exascale Earth System Model (E3SM) land model (ELM). ELM predicted vegetation controls on EF relatively well, but underpredicted climate factors on EF, indicating model deficiencies in describing the relationships between atmospheric state and surface fluxes. Moreover, changes in controls on surface energy flux partitioning due to deforestation were not detected in the model. We highlight the need for benchmarking model simulated surface-energy fluxes and the corresponding causal relationships against those of observations, to improve our understanding of model predictability on how deforestation reshapes land surface energy fluxes.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

The Information Length Concept Applied to Plasma Turbulence

A methodology to study statistical properties of anomalous transport in fusion plasma is investigated. Three time traces generated by the full-f gyrokinetic code GKNET are analyzed for this purpose. The time traces consist of heat flux as a function of the radial position, which is studied in a novel manner using statistical methods. The simulation data exhibit transport processes with both medium and long correlation length along the radius. A typical example of a phenomenon with long correlation length is avalanches. In order to investigate the evolution of the turbulent state, two basic configurations are studied, one flux-driven and one gradient-driven with decaying turbulence. The information length concept in tandem with Boltzmann–Gibbs and Tsallis entropy is used in the investigation. It is found that the dynamical states in both flux-driven and gradient-driven cases are surprisingly similar, but the Tsallis entropy reveals differences between them. This indicates that the types of probability distribution function are nevertheless quite different since the higher moments are significantly different.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2021

Machine-learning-based inversion of nuclear responses

A microscopic description of the interaction of atomic nuclei with external electroweak probes is required for elucidating aspects of short-range nuclear dynamics and for the correct interpretation of neutrino oscillation experiments. Nuclear quantum Monte Carlo methods infer the nuclear electroweak response functions from their Laplace transforms. Inverting the Laplace transform is a notoriously ill-posed problem; and Bayesian techniques, such as maximum entropy, are typically used to reconstruct the original response functions in the quasielastic region. In this work, we present a physics-informed artificial neural network architecture suitable for approximating the inverse of the Laplace transform. Utilizing simulated, albeit realistic, electromagnetic response functions, we show that this physics-informed artificial neural network outperforms maximum entropy in both the low-energy transfer and the quasielastic regions, thereby allowing for robust calculations of electron scattering and neutrino scattering on nuclei and inclusive muon capture rates.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2020

Islands in asymptotically flat 2D gravity

The large-$\textit{N}$ limit of asymptotically flat two-dimensional dilaton gravity coupled to $\textit{N}$ free matter fields provides a useful toy model for semiclassical black holes and the information paradox. Analyses of the asymptotic information flux as given by the entanglement entropy show that it follows the Hawking curve, indicating that information is destroyed in these models. Recently, motivated by developments in AdS/CFT, a semiclassical island rule for entropy has been proposed. We define and compute the island rule entropy for black hole formation and evaporation in the large-$\textit{N}$ RST model of dilaton gravity and show that, in contrast, it follows the unitary Page curve. The relation of these two observations, and interesting properties of the dilaton gravity island rule, are discussed.

2D Gravity↗

DOE OSTI2026

Complexity analysis of a CT injection experiment on BRB

In this work, we use Jensen–Shannon complexity and permutation entropy to analyze the magnetic field fluctuations of an astrophysically scaled plasma experiment. The experiment was intended to emulate an interplanetary coronal mass ejection event in the lab, recreating the major sections seen in satellite data. We also use a technique called “delay,” in which we use select elements, skipping one or more data points at a time, in our time series data to obtain Jensen–Shannon complexity as a function of frequency and investigate the frequency of maximized complexity. We then compare the delay frequencies to other frequencies in the plasma. We found that the frequencies for maximum complexity do not correspond to the frequencies investigated, implying that other physical mechanisms lead to an increase in complexity at these frequencies.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2024

Holographic tensor networks with bulk gauge symmetries

Abstract Tensor networks are useful toy models for understanding the structure of entanglement in holographic states and reconstruction of bulk operators within the entanglement wedge. They are, however, constrained to only prepare so-called “fixed-area states” with flat entanglement spectra, limiting their utility in understanding general features of holographic entanglement. Here, we overcome this limitation by constructing a variant of random tensor networks that enjoys bulk gauge symmetries. Our model includes a gauge theory on a general graph, whose gauge-invariant states are fed into a random tensor network. We show that the model satisfies the quantum-corrected Ryu-Takayanagi formula with a nontrivial area operator living in the center of a gauge-invariant algebra. We also demonstrate nontrivial,n-dependent contributions to the Rényi entropy and Rényi mutual information from this area operator, a feature shared by general holographic states.

Physics↗

DOE OSTI2026

Structural and compositional complexities of hierarchical self-assembly: A hypergraph approach

Programmable self-assembly enables the construction of complex molecular, supramolecular, and crystalline architectures from well-designed building blocks. In this work, we introduce a hypergraph-based formalism, Blocks & Bonds (B&B), which generalizes classical chemical graph theory by incorporating directed and multicolored interactions, internal symmetries, and hierarchical organization. Within this framework, we develop the Structure Code (SC), a compact and versatile language for describing self-assembled architectures. We define a Kolmogorov-style structural complexity as the total information content of SC, obtained through its tokenization and Shannon information assignment. Complementing this encoding-based measure, we introduce a much simpler quantity, the compositional complexity, which depends only on the number and cumulative usage of block and bond types in the construction set. A central result of this work is a strong empirical correlation between the token-based structural complexity and the compositional complexity across all examined systems. Owing to this agreement, the compositional complexity emerges as the most practical and broadly applicable measure: it is easy to compute, requires no explicit encoding, and yet closely tracks the actual information content of structurally diverse architectures. Applications to molecular systems (ethylene glycol and glucose), DNA-origami lattices, and crystalline assemblies show that B&B hypergraphs provide a unified, scalable, and information-efficient representation of structural organization, naturally capturing symmetry, modularity, and stereochemistry. This framework establishes a quantitative foundation for complexity-aware classification and inverse design of programmable matter.

36 MATERIALS SCIENCE↗

DOE OSTI2026

Maximizing efficiency of dataset compression for machine learning potentials with information theory

Machine learning interatomic potentials (MLIPs) balance high accuracy and lower costs compared to density functional theory calculations, but their performance often depends on the size and diversity of training datasets. Large datasets improve model accuracy and generalization but are computationally expensive to produce and train on, while smaller datasets risk discarding rare but important atomic environments and compromising MLIP accuracy/reliability. Here, we develop an information-theoretical framework to quantify the efficiency of dataset compression methods and propose an algorithm that maximizes this efficiency. By framing atomistic dataset compression as an instance of the minimum set cover (MSC) problem over atom-centered environments, our method identifies the smallest subset of structures that contains as much information as possible from the original dataset while pruning redundant information. The approach is extensively demonstrated on the GAP-20 and TM23 datasets and validated on 64 varied datasets from the ColabFit repository. Across all cases, MSC consistently retains outliers, preserves dataset diversity, and reproduces the long-tail distributions of forces even at high compression rates, outperforming other subsampling methods. Furthermore, MLIPs trained on MSC-compressed datasets exhibit reduced error for out-of-distribution data even in low-data regimes. We explain these results using an outlier analysis and show that such quantitative conclusions could not be achieved with conventional dimensionality reduction methods. The algorithm is implemented in the open-source QUESTS package and can be used for several tasks in atomistic modeling, from data subsampling, outlier detection, and training improved MLIPs at a lower cost.

36 MATERIALS SCIENCE↗

DOE OSTI2023

Electronic and magnetic phase diagrams of the Kitaev quantum spin liquid candidate Na 2 Co 2 TeO 6

The 3⁢d 7 Co 2+ -based insulating magnet Na 2 ⁢Co 2 ⁢TeO 6 has recently been reported to have strong Kitaev interactions on a honeycomb lattice and is thus being considered as a Kitaev quandum spin liquid candidate. However, due to the existence of other types of interactions, a spontaneous long-range magnetic order occurs. This order is suppressed by applied magnetic fields leading to a succession of phases and ultimately saturation of the magnetic moments. The precise phase diagram, the nature of the phases, and the possibility that one of the field-induced phases is a Kitaev quantum spin liquid phase are still a matter of debate. Here, in this study, we measured an extensive set of physical properties to build the complete temperature-field phase diagrams to magnetic saturation at 10 T for magnetic fields along the a and a* axes, and a partial phase diagram up to 60 T along c. We probe the phases using magnetization, specific heat, magnetocaloric effect, magnetostriction, dielectric constant, and electric polarization, which is a symmetry-sensitive probe. With these measurements, we identify all the previously incomplete phase boundaries and find additional high-field phase boundaries. We find strong magnetoelectric coupling in the dielectric constant and moderate magnetostrictive coupling at several phase boundaries. Furthermore, we detect the symmetry of the magnetic order using electrical polarization measurements under magnetic fields. Based on our analysis, the absence of electric polarization under zero or finite magnetic field in any of the phases or after any combination of magnetic/electric field cooling suggests that a zigzag spin structure is more likely than a triple-Q spin structure at zero field. Finally, we investigate the hysteresis and first- or second-order nature of each phase transition and its entropy changes. With this information, we establish a map of the magnetic phases of this compound and its magnetic, thermodynamic, and magnetoelectric properties, and discuss where spin liquid or other phases may be sought in future studies.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

DOE OSTI2021

Revisiting Huffman Coding: Toward Extreme Performance on Modern GPU Architectures

Today's high-performance computing (HPC) applications are producing vast volumes of data, which are challenging to store and transfer efficiently during the execution, such that data compression is becoming a critical technique to mitigate the storage burden and data movement cost. Huffman coding is arguably the most efficient Entropy coding algorithm in information theory, such that it could be found as a fundamental step in many modern compression algorithms such as DEFLATE. On the other hand, today's HPC applications are more and more relying on the accelerators such as GPU on supercomputers, while Huffman encoding suffers from low throughput on GPUs, resulting in a significant bottleneck in the entire data processing. In this paper, we propose and implement an efficient Huffman encoding approach based on modern GPU architectures, which addresses two key challenges: (1) how to parallelize the entire Huffman encoding algorithm, including codebook construction, and (2) how to fully utilize the high memory-bandwidth feature of modern GPU architectures. The detailed contribution is fourfold. (1) We develop an efficient parallel codebook construction on GPUs that scales effectively with the number of input symbols. (2) We propose a novel reduction based encoding scheme that can efficiently merge the codewords on GPUs. (3) We optimize the overall GPU performance by leveraging the state-of-the-art CUDA APIs such as Cooperative Groups. (4) We evaluate our Huffman encoder thoroughly using six real-world application datasets on two advanced GPUs and compare with our implemented multi-threaded Huffman encoder. Experiments show that our solution can improve the encoding throughput by up to 5.0x and 6.8x on NVIDIA RTX 5000 and V100, respectively, over the state-of-the-art GPU Huffman encoder, and by up to 3.3x over the multi-thread encoder on two 28-core Xeon Platinum 8280 CPUs.

Tian, Jiannan↗

DOE OSTI2022

Tunable Geometries in Sparse Clifford Circuits

We investigate the emergence of different effective geometries in stochastic Clifford circuits with sparse coupling. By changing the probability distribution for choosing two-site gates as a function of distance, we generate sparse interactions that either decay or grow with distance as a function of a single tunable parameter. Tuning this parameter reveals three distinct regimes of geometry for the spreading of correlations and growth of entanglement in the system. We observe linear geometry for short-range interactions, treelike geometry on a sparse coupling graph for long-range interactions, and an intermediate fast scrambling regime at the crossover point between the linear and treelike geometries. This transition in geometry is revealed in calculations of the subsystem entanglement entropy and tripartite mutual information. We also study emergent lightcones that govern these effective geometries by teleporting a single qubit of information from an input qubit to an output qubit. These tools help to analyze distinct geometries arising in dynamics and correlation spreading in quantum many-body systems.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Revisiting Huffman Coding: Toward Extreme Performance on Modern GPU Architectures

Today's high-performance computing (HPC) applications are producing vast volumes of data, which are challenging to store and transfer efficiently during the execution, such that data compression is becoming a critical technique to mitigate the storage burden and data movement cost. Huffman coding is arguably the most efficient Entropy coding algorithm in information theory, such that it could be found as a fundamental step in many modern compression algorithms such as DEFLATE. On the other hand, today's HPC applications are more and more relying on the accelerators such as GPU on supercomputers, while Huffman encoding suffers from low throughput on GPUs, resulting in a significant bottleneck in the entire data processing. In this paper, we propose and implement an efficient Huffman encoding approach based on modern GPU architectures, which addresses two key challenges: (1) how to parallelize the entire Huffman encoding algorithm, including codebook construction, and (2) how to fully utilize the high memory-bandwidth feature of modern GPU architectures. The detailed contribution is fourfold. (1) We develop an efficient parallel codebook construction on GPUs that scales effectively with the number of input symbols. (2) We propose a novel reduction based encoding scheme that can efficiently merge the codewords on GPUs. (3) We optimize the overall GPU performance by leveraging the state-of-the-art CUDA APIs such as Cooperative Groups. (4) We evaluate our Huffman encoder thoroughly using six real-world application datasets on two advanced GPUs and compare with our implemented multithreaded Huffman encoder. Experiments show that our solution can improve the encoding throughput by up to 5.0× and 6.8× on NVIDIA RTX 5000 and V100, respectively, over the state-of-the-art GPU Huffman encoder, and by up to 3.3× over the multithread encoder on two 28-core Xeon Platinum 8280 CPUs.

Tian, Jiannan↗

NASA NTRS1985

Comparison of the information contents of Landsat TM and MSS data

A communications-theory approach is taken to analyze the dispersion and concentration of signal values in various data spaces, irrespective of specific class membership. Entropy is used to quantify information, and mutual information is used to measure the information represented by subsets of spectral variables. Several different comparisons of information content are made. These include comparisons of system design capacities, of data volumes occupied by agricultural data in the spaces defined by original bands and by transformed spectral (Tasseled Cap) variables, of the information contents of original bands and Tasseled Cap variables, and of the information contents of TM and MSS for the given agricultural data sets. Also, the effects of sample size, scene content, and quantization level are examined.

Malila, W. A.↗