Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Information Theory”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

QCD Theory Meets Information Theory

We present a novel technique to incorporate precision calculations from quantum chromodynamics into fully differential particle-level Monte Carlo simulations. By minimizing an information-theoretic quantity subject to constraints, our reweighted Monte Carlo incorporates systematic uncertainties absent in individual Monte Carlo predictions, achieving consistency with the theory input in precision and its estimated systematic uncertainties. Our method can be applied to arbitrary observables known from precision calculations, including multiple observables simultaneously. It generates strictly positive weights, thus offering a clear path to statistically powerful and theoretically precise computations for current and future collider experiments. As a proof of concept, we apply our technique to event-shape observables at electron-positron colliders, leveraging existing precision calculations of thrust. Our analysis highlights the importance of logarithmic moments of event shapes, which have not been previously studied in the collider physics literature.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

Environmental Controls on Water Vapor Deuterium Excess in the Coastal Boundary Layer: An Information Theory Perspective

We use information theory to quantify the environmental controls on water vapor deuterium excess (D-excess) in coastal Southern California from June 2023 through February 2024. Using Shannon entropy, mutual information (MI), and joint mutual information, metrics that capture both linear and nonlinear relationships, we identify the most informative variables and variable combinations governing D-excess across contrasting marine and continental regimes. Relative humidity with respect to sea surface temperature (RHS) is consistently the strongest individual predictor, explaining up to 27% of D-excess variability during marine conditions but only 10% in continental air masses. The Relative humidity(RHS) + sea surface temperature (SST) combination demonstrates synergistic effects, where their joint influence (explaining up to 36% of D-excess variability) exceeds what either variable achieves individually, confirming their coupled influence on deuterium excess. Wind direction complements RHS most effectively during continental conditions. The best three-variable combination (RHS + SST + Planetary Boundary Layer height) explains 38% of D-excess variability in marine air, while no combination exceeds 20% explanatory power during continental periods. Information theory shows that heteroscedasticity in D-excess relationships indicates regime shifts in controlling processes and quantifies fundamental constraints on predictor variables: some environmental factors like surface pressure or water vapor flux contain insufficient information content to explain D-excess variability regardless of their physical relevance. These results highlight the different predictability limits between marine and continental regimes, challenging the adequacy of linear models and providing a rigorous framework for quantifying the information content of isotope-climate relationships with implications for both modern and paleoclimate applications.

information theory↗
DOE OSTI2025

Information theory optimization of signals from small-angle scattering measurements

Small-angle X-ray scattering (SAXS) of particles in solution informs on the conformational states and assemblies of biological macromolecules (bioSAXS) outside of cryo- and solid-state conditions. In bioSAXS, the SAXS measurement under dilute conditions is resolution limited, and through an inverse Fourier transform, the measured SAXS intensities directly relate to the physical space occupied by the particles via the P (r)-distribution. Yet, this inverse transform of SAXS data has been historically cast as an ill-posed, ill-conditioned problem requiring an indirect approach. Here, we show that through the applications of matrix and information theories, the inverse transform of SAXS intensity data is a well-conditioned problem. The so-called ill-conditioning of the inverse problem is directly related to the Shannon number. By exploiting the oversampling enabled by modern detectors, a direct inverse Fourier transform of the SAXS data is possible, provided the recovered information does not exceed the Shannon number. The Shannon limit corresponds to the maximum number of significant singular values that can be recovered in a SAXS experiment, suggesting this relationship is a fundamental property of band-limited inverse integral transform problems. This correspondence reduces the complexity of the inverse problem to the Shannon limit and maximum dimension. We propose a hybrid scoring function using an information theory framework that assesses both the quality of the model-data fit as well as the quality of the recovered P (r)-distribution. The hybrid score utilizes the Akaike information criteria and Durbin-Watson statistic that considers parameter-model complexity, i.e., degrees of freedom, and the randomness of the model-data residuals. The described tests and findings extend the boundaries for bioSAXS by completing the information theory formalism initiated by Peter B. Moore to enable a quantitative measure of resolution in SAXS, robustly determine maximum dimension, and more precisely define the best parameter model appropriately representing the observed scattering data.

Rambo, Robert P. [Science and Technology Facilitie↗
DOE OSTI2020

Applying Information Theory to Design Optimal Filters for Photometric Redshifts

In this paper we apply ideas from information theory to create a method for the design of optimal filters for photometric redshift estimation. We show the method applied to a series of simple example filters in order to motivate an intuition for how photometric redshift estimators respond to the properties of photometric passbands. We then design a realistic set of six filters covering optical wavelengths that optimize photometric redshifts for z <= 2.3 and i < 25.3. We create a simulated catalog for these optimal filters and use our filters with a photometric redshift estimation code to show that we can improve the standard deviation of the photometric redshift error by 7.1% overall and improve outliers 9.9% over the standard filters proposed for the Large Synoptic Survey Telescope (LSST). We compare features of our optimal filters to the LSST and find that the LSST filters incorporate key features for optimal photometric redshift estimation. Finally, we describe how information theory can be applied to a range of optimization problems in astronomy.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

INSURE: An Information Theory iNspired diSentanglement and pURification modEl for Domain Generalization

Domain Generalization (DG) aims to learn a generalizable model on the unseen target domain by only training on the multiple observed source domains. Although a variety of DG methods have focused on extracting domain-invariant features, the domain-specific class-relevant features have attracted attention and been argued to benefit generalization to the unseen target domain. To take into account the class-relevant domain-specific information, in this paper we propose an Information theory iNspired diSentanglement and pURification modEl (INSURE) to explicitly disentangle the latent features to obtain sufficient and compact (necessary) class-relevant feature for generalization to the unseen domain. Specifically, we first propose an information theory inspired loss function to ensure the disentangled class-relevant features contain sufficient class label information and the other disentangled auxiliary feature has sufficient domain information. Additionally, we further propose a paired purification loss function to let the auxiliary feature discard all the class-relevant information and thus the class-relevant feature will contain sufficient and compact (necessary) class-relevant information. Moreover, instead of using multiple encoders, we propose to use a learnable binary mask as our disentangler to make the disentanglement more efficient and make the disentangled features complementary to each other. We conduct extensive experiments on five widely used DG benchmark datasets including PACS, VLCS, OfficeHome, TerraIncognita, and DomainNet. The proposed INSURE achieves state-of-the-art performance. We also empirically show that domain-specific class-relevant features are beneficial for domain generalization. The code is available at https://github.com/yuxi120407/INSURE .

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Intensity measure adequacy assessment for nonlinear site response using Information Theory

Nonlinear site response modeling is a crucial aspect of Probabilistic Seismic Hazard Analysis. Site amplification models routinely rely on a rock intensity measure to characterize the strength of the bedrock motion. However, the adequacy of such intensity measures towards predicting amplifications across the oscillator period range has not been investigated in the literature. This paper analyzes the adequacy of rock intensity measures using state of the art criterion established in Performance-Based Earthquake Engineering and techniques from Information Theory. The efficiency and the sufficiency of several rock intensity measure are assessed. It was found that spectral accelerations at low periods usually are adequate for predicting amplifications across the period range. This supports the current practice of using Peak Ground Acceleration in Ground Motion Models. However, for extremely soft sites, which demonstrate nonlinear effects well into the period range, it is better practice to ensure that amplification and spectral acceleration share the same oscillator period. Lastly, for predicting the peak shear strain (an important parameter that controls nonlinearity of site response), Peak Ground Velocity is generally adequate, and this conclusion is in line with the definition of proxy shear strain.

42 ENGINEERING↗
DOE OSTI2020

Exploring Hydrologic Model Process Connectivity at the Continental Scale Through an Information Theory Approach

Exploring water fluxes between hydrological model (HM) components is essential to assess and improve model realism. Many classical metrics for HM diagnosis rely solely on streamflow and hence provide limited insights into model performance across processes. This study applies an information theory measure known as “transfer entropy” (TE) to systematically quantify the transfer of information among major HM components. To test and demonstrate the benefits of TE, we use the Framework for Understanding Structural Errors (FUSE) model to mimic and compare four commonly used HM structures, VIC, PRMS, SACRAMENTO, and TOPMODEL, across 671 catchments spanning a variety of hydrologic regimes in the conterminous United States. We explore connections between HM components and catchment landscape characteristics (e.g., climate, topography, soil, and vegetation) and characterize their nonlinear associations using distance correlation and Spearman correlation coefficients. Our results indicate that while the information transferred from precipitation to runoff is similar across model structures (likely as a result of calibration), the information transferred among other components can vary significantly from a FUSE structure to another. We find that aridity, precipitation duration and frequency, snow fraction, mean elevation, forest area, and leaf area index are often significantly associated with TE between the main HM components. Here, we propose that the presence of meaningful nonlinear associations can be used to diagnose process representation in HMs. Our results highlight the necessity to enhance the conventional streamflow-only calibration approach for a more realistic representation of water dynamics in the models.

large-sample hydrology↗
DOE OSTI2026

Maximizing efficiency of dataset compression for machine learning potentials with information theory

Machine learning interatomic potentials (MLIPs) balance high accuracy and lower costs compared to density functional theory calculations, but their performance often depends on the size and diversity of training datasets. Large datasets improve model accuracy and generalization but are computationally expensive to produce and train on, while smaller datasets risk discarding rare but important atomic environments and compromising MLIP accuracy/reliability. Here, we develop an information-theoretical framework to quantify the efficiency of dataset compression methods and propose an algorithm that maximizes this efficiency. By framing atomistic dataset compression as an instance of the minimum set cover (MSC) problem over atom-centered environments, our method identifies the smallest subset of structures that contains as much information as possible from the original dataset while pruning redundant information. The approach is extensively demonstrated on the GAP-20 and TM23 datasets and validated on 64 varied datasets from the ColabFit repository. Across all cases, MSC consistently retains outliers, preserves dataset diversity, and reproduces the long-tail distributions of forces even at high compression rates, outperforming other subsampling methods. Furthermore, MLIPs trained on MSC-compressed datasets exhibit reduced error for out-of-distribution data even in low-data regimes. We explain these results using an outlier analysis and show that such quantitative conclusions could not be achieved with conventional dimensionality reduction methods. The algorithm is implemented in the open-source QUESTS package and can be used for several tasks in atomistic modeling, from data subsampling, outlier detection, and training improved MLIPs at a lower cost.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Model-free estimation of completeness, uncertainties, and outliers in atomistic machine learning using information theory

Abstract An accurate description of information is relevant for a range of problems in atomistic machine learning (ML), such as crafting training sets, performing uncertainty quantification (UQ), or extracting physical insights from large datasets. However, atomistic ML often relies on unsupervised learning or model predictions to analyze information contents from simulation or training data. Here, we introduce a theoretical framework that provides a rigorous, model-free tool to quantify information contents in atomistic simulations. We demonstrate that the information entropy of a distribution of atom-centered environments explains known heuristics in ML potential developments, from training set sizes to dataset optimality. Using this tool, we propose a model-free UQ method that reliably predicts epistemic uncertainty and detects out-of-distribution samples, including rare events in systems such as nucleation. This method provides a general tool for data-driven atomistic modeling and combines efforts in ML, simulations, and physical explainability.

36 MATERIALS SCIENCE↗
DOE OSTI2020

An Information Theory Approach on Deciding Spectroscopic Follow-ups

Classification and characterization of variable phenomena and transient phenomena are critical for astrophysics and cosmology. These objects are commonly studied using photometric time series or spectroscopic data. Given that many ongoing and future surveys are conducted in a time domain, and given that adding spectra provides further insights but requires more observational resources, it would be valuable to know which objects we should prioritize to have a spectrum in addition to a time series. We propose a methodology in a probabilistic setting that determines a priori which objects are worth taking a spectrum of to obtain better insights, where we focus on the insight of the type of the object (classification). Objects for which we query their spectrum are reclassified using their full spectral information. We first train two classifiers, one that uses photometric data and another that uses photometric and spectroscopic data together. Then for each photometric object we estimate the probability of each possible spectrum outcome. We combine these models in various probabilistic frameworks (strategies), which are used to guide the selection of follow-up observations. The best strategy depends on the intended use, whether it is obtaining more confidence or accuracy. For a given number of candidate objects (127, equal to 5% of the data set) for taking spectra, we improve the class prediction accuracy by 37% as opposed to 20% of a non-naive (non-random) best-baseline strategy. Our approach provides a general framework for follow-up strategies and can be extended beyond classification to include other forms of follow-ups beyond spectroscopy.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Combustion chemistry in the twenty-first century: Developing theory-informed chemical kinetics models

Over the past 20 to 25 years theoretical chemistry (particularly theoretical chemical kinetics) has played an increasingly important role in developing chemical kinetics models for combustion. Theoretical methods of obtaining rate parameters are now competitive in accuracy with experiment, particularly for small molecules. Moreover, theoretical methods can deal with conditions that experiments frequently cannot. In addition to increased accuracy, theory has rejuvenated methods and discovered phenomena that were completely unappreciated, or at least underappreciated, in the 20th century. Our primary interest here is in molecular-level issues, i.e. in calculating rate and transport parameters. However, dealing with kinetics models that involve thousands of reactions and hundreds of species is important for practical applications and is relatively new to the 21st century. Theory, in a general sense, and theoretical methods development have a role to play here too. We discuss in this review all these topics in some detail with an emphasis on issues and methods that have emerged in the last 20 years or so. Even so, our review is selective, rather than comprehensive, out of necessity.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Rigorous Oxidation State Assignments for Supported Ga-Containing Catalysts Using Theory-Informed X-ray Absorption Spectroscopy Signatures from Well-Defined Ga(I) and Ga(III) Compounds

Gallium-based heterogeneous catalysts originally developed for commercial use in propane dehydroaromatization have been explored extensively as potential replacements for Pt- and Cr-based catalysts used in propane dehydrogenation for on-demand propylene production. In a large number of experimental and theoretical studies, active sites with a variety of Ga nuclearities, coordination environments, and oxidation states have been proposed. Isolated Ga(I) ions are often invoked, despite the scarcity of well-defined molecular species and their well-documented instability. In this study, we investigate the appearance of a high-intensity, low-energy white line at the Ga K-edge upon reduction of Ga/HZSM-5 by H 2 at a temperature of ca. 500 °C, accompanied by a dramatic reduction in the extended X-ray absorption fine structure (EXAFS) intensity. In contrast, Ga/γ-Al 2 O 3 does not show such behavior. In order to lay a rigorous foundation for characterizing these types of systems and to establish experimental signatures for the elusive Ga(I) oxidation state, we recorded Ga K-edge X-ray absorption spectra [including high-energy-resolution fluorescence detection-X-ray absorption near-edge spectroscopy (HERFD-XANES)] for several well-defined molecular and crystalline Ga(I) compounds. XANES is essential to establishing the presence of Ga(I), despite the overlap in edge energies with organoGa(III) compounds, because Ga(I)-containing oxide materials show very weak EXAFS scattering. Compared to the XANES of trigonal Ga(III)-containing materials, Ga(I) spectra display a significantly more intense white line feature. Theoretical simulations agree well with this experimental observation and reveal that the strong XANES intensity originates from the superposition of transitions to several empty, nearly degenerate p-like states. Finally, these signatures provide compelling evidence for assigning the intense white line and dramatic loss of EXAFS intensity in Ga/HZSM-5 to the near-quantitative reduction of Ga(III) to Ga(I), while the weaker white line and conventional EXAFS signal of Ga/γ-Al 2 O 3 point to, at most, a minor fraction of Ga(I) sites.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Information theory and machine learning illuminate large‐scale metabolomic responses of Brachypodium distachyon to environmental change

SUMMARY Plant responses to environmental change are mediated via changes in cellular metabolomes. However, <5% of signals obtained from liquid chromatography tandem mass spectrometry (LC‐MS/MS) can be identified, limiting our understanding of how metabolomes change under biotic/abiotic stress. To address this challenge, we performed untargeted LC‐MS/MS of leaves, roots, and other organs of Brachypodium distachyon (Poaceae) under 17 organ–condition combinations, including copper deficiency, heat stress, low phosphate, and arbuscular mycorrhizal symbiosis. We found that both leaf and root metabolomes were significantly affected by the growth medium. Leaf metabolomes were more diverse than root metabolomes, but the latter were more specialized and more responsive to environmental change. We found that 1 week of copper deficiency shielded the root, but not the leaf metabolome, from perturbation due to heat stress. Machine learning (ML)‐based analysis annotated approximately 81% of the fragmented peaks versus approximately 6% using spectral matches alone. We performed one of the most extensive validations of ML‐based peak annotations in plants using thousands of authentic standards, and analyzed approximately 37% of the annotated peaks based on these assessments. Analyzing responsiveness of each predicted metabolite class to environmental change revealed significant perturbations of glycerophospholipids, sphingolipids, and flavonoids. Co‐accumulation analysis further identified condition‐specific biomarkers. To make these results accessible, we developed a visualization platform on the Bio‐Analytic Resource for Plant Biology website ( https://bar.utoronto.ca/efp_brachypodium_metabolites/cgi‐bin/efpWeb.cgi ), where perturbed metabolite classes can be readily visualized. Overall, our study illustrates how emerging chemoinformatic methods can be applied to reveal novel insights into the dynamic plant metabolome and stress adaptation.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Control-Theory-Informed Feature Selection for Detecting Malicious Tampering in Additive Layer Manufacturing Processes

Additive layer manufacturing (ALM) is rapidly becoming an appealing solution to the low-volume manufacturing of metal, polymer, or composite parts. However, ALM’s reliance on digital part specifications, microcontrollers, and modern networking makes these devices vulnerable to malicious tampering by cyber attackers, which can negatively affect part performance and even result in catastrophic failure. We present a hybrid analytic approach to feature discovery using control theoretic techniques and linear modelling on input-output data collected from a representative controller system. Employing this approach, we design, train, and test an anomaly detection system. The preliminary results show that the proposed approach effectively discovers useful input-output relationships for anomaly detection in a simulated ALM process. Application to larger and more complex systems are discussed.

Dawson, Joel↗