Utilization of information theory in digital data guidance systems Quarterly progress report no. 2, 1 Dec. 1965 - 28 Feb. 1966
Information theory in fabricating decoding schemes
INFORMATION THEORY↗
SEARCH · Engineering Papers
Results for “Information Theory”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Utilization of information theory in digital data guidance systems Quarterly progress report no. 1, 1 Sep. - 30 Nov. 1965
Information theory in improving digital data guidance system
GUIDANCE SYSTEM↗
Application of information theory to the design of line-scan imaging systems
Information theory is used to formulate a single figure of merit for assessing the performance of line scan imaging systems as a function of their spatial response (point spread function or modulation transfer function), sensitivity, sampling and quantization intervals, and the statistical properties of a random radiance field. Computational results for the information density and efficiency (i.e., the ratio of information density to data density) are intuitively satisfying and compare well with experimental and theoretical results obtained by earlier investigators concerned with the performance of TV systems.
Huck, F. O.↗
Information theory analysis of sensor-array imaging systems for computer vision
Information theory is used to assess the performance of sensor-array imaging systems, with emphasis on the performance obtained with image-plane signal processing. By electronically controlling the spatial response of the imaging system, as suggested by the mechanism of human vision, it is possible to trade-off edge enhancement for sensitivity, increase dynamic range, and reduce data transmission. Computational results show that: signal information density varies little with large variations in the statistical properties of random radiance fields; most information (generally about 85 to 95 percent) is contained in the signal intensity transitions rather than levels; and performance is optimized when the OTF of the imaging system is nearly limited to the sampling passband to minimize aliasing at the cost of blurring, and the SNR is very high to permit the retrieval of small spatial detail from the extensively blurred signal. Shading the lens aperture transmittance to increase depth of field and using a regular hexagonal sensor-array instead of square lattice to decrease sensitivity to edge orientation also improves the signal information density up to about 30 percent at high SNRs.
Huck, F. O.↗
Intensity measure adequacy assessment for nonlinear site response using Information Theory
Nonlinear site response modeling is a crucial aspect of Probabilistic Seismic Hazard Analysis. Site amplification models routinely rely on a rock intensity measure to characterize the strength of the bedrock motion. However, the adequacy of such intensity measures towards predicting amplifications across the oscillator period range has not been investigated in the literature. This paper analyzes the adequacy of rock intensity measures using state of the art criterion established in Performance-Based Earthquake Engineering and techniques from Information Theory. The efficiency and the sufficiency of several rock intensity measure are assessed. It was found that spectral accelerations at low periods usually are adequate for predicting amplifications across the period range. This supports the current practice of using Peak Ground Acceleration in Ground Motion Models. However, for extremely soft sites, which demonstrate nonlinear effects well into the period range, it is better practice to ensure that amplification and spectral acceleration share the same oscillator period. Lastly, for predicting the peak shear strain (an important parameter that controls nonlinearity of site response), Peak Ground Velocity is generally adequate, and this conclusion is in line with the definition of proxy shear strain.
42 ENGINEERING↗
An information theory approach for evaluating earth radiation budget (ERB) measurements - Nonuniform sampling of diurnal longwave flux variations
Satellite measurements are subject to a wide range of uncertainties due to their temporal, spatial, and directional sampling characteristics. An information-theory approach is suggested to examine the nonuniform temporal sampling of ERB measurements. The information (i.e., its entropy or uncertainty) before and after the measurements is determined, and information gain (IG) is defined as a reduction in the uncertainties involved. A stochastic model for the diurnal outgoing flux variations that affect the ERB is developed. Using Gaussian distributions for the a priori and measured radiant exitance fields, the IG is obtained by computing the a posteriori covariance. The IG for the monthly outgoing flux measurements is examined for different orbital parameters and orbital tracks, using the Earth Observing System orbital parameters as specific examples. Variations in IG due to changes in the orbit's inclination angle and the initial ascending node local time are investigated.
Halyo, Nesim↗
Exploring Hydrologic Model Process Connectivity at the Continental Scale Through an Information Theory Approach
Exploring water fluxes between hydrological model (HM) components is essential to assess and improve model realism. Many classical metrics for HM diagnosis rely solely on streamflow and hence provide limited insights into model performance across processes. This study applies an information theory measure known as “transfer entropy” (TE) to systematically quantify the transfer of information among major HM components. To test and demonstrate the benefits of TE, we use the Framework for Understanding Structural Errors (FUSE) model to mimic and compare four commonly used HM structures, VIC, PRMS, SACRAMENTO, and TOPMODEL, across 671 catchments spanning a variety of hydrologic regimes in the conterminous United States. We explore connections between HM components and catchment landscape characteristics (e.g., climate, topography, soil, and vegetation) and characterize their nonlinear associations using distance correlation and Spearman correlation coefficients. Our results indicate that while the information transferred from precipitation to runoff is similar across model structures (likely as a result of calibration), the information transferred among other components can vary significantly from a FUSE structure to another. We find that aridity, precipitation duration and frequency, snow fraction, mean elevation, forest area, and leaf area index are often significantly associated with TE between the main HM components. Here, we propose that the presence of meaningful nonlinear associations can be used to diagnose process representation in HMs. Our results highlight the necessity to enhance the conventional streamflow-only calibration approach for a more realistic representation of water dynamics in the models.
large-sample hydrology↗
What Information Theory Says About Best Response and About Binding Contracts
Product Distribution (PD) theory is the information-theoretic extension of conventional full- rationality game theory to bounded rational games. Here PD theory is used to investigate games in which the players use bounded rational best-response strategies. This investigation illuminates how to determine the optimal organization chart for a corporation, or more generally how to order the sequence of moves of the players / employees so as to optimize an overall objective function. It is then shown that in the continuum-time limit, bounded rational best response games result in a variant of the replicator dynamics of evolutionary game theory. This variant is then investigated for team games, in which the players share the same utility function, by showing that such continuum- limit bounded rational best response is identical to Newton-Raphson iterative optimization of the shared utility function. Next PD theory is used to investigate changing the coordinate system of the game, i.e., changing the mapping from the joint move of the players to the arguments in the utility functions. Such a change couples those arguments, essentially by making each players move be an offered binding contract.
Wolpert, David H.↗
Generalised squeezing and information theory approach to quantum entanglement
It is shown that the usual one- and two-mode squeezing are based on reducible representations of the SU(1,1) group. Generalized squeezing is introduced with the use of different SU(1,1) rotations on each irreducible sector. Two-mode squeezing entangles the modes and information theory methods are used to study this entanglement. The entanglement of three modes is also studied with the use of the strong subadditivity property of the entropy.
Vourdas, A.↗
The measurement of the earth's radiation budget as a problem in information theory - A tool for the rational design of earth observing systems
The measurement of the earth's radiation budget has been chosen to illustrate the technique of objective system design. The measurement process is an approximately linear transformation of the original field of radiant exitances, so that linear statistical techniques may be employed. The combination of variability, measurement strategy, and error propagation is presently made with the help of information theory, as suggested by Kondratyev et al. (1975) and Peckham (1974). Covariance matrices furnish the quantitative statement of field variability.
Barkstrom, B. R.↗
Maximizing efficiency of dataset compression for machine learning potentials with information theory
Machine learning interatomic potentials (MLIPs) balance high accuracy and lower costs compared to density functional theory calculations, but their performance often depends on the size and diversity of training datasets. Large datasets improve model accuracy and generalization but are computationally expensive to produce and train on, while smaller datasets risk discarding rare but important atomic environments and compromising MLIP accuracy/reliability. Here, we develop an information-theoretical framework to quantify the efficiency of dataset compression methods and propose an algorithm that maximizes this efficiency. By framing atomistic dataset compression as an instance of the minimum set cover (MSC) problem over atom-centered environments, our method identifies the smallest subset of structures that contains as much information as possible from the original dataset while pruning redundant information. The approach is extensively demonstrated on the GAP-20 and TM23 datasets and validated on 64 varied datasets from the ColabFit repository. Across all cases, MSC consistently retains outliers, preserves dataset diversity, and reproduces the long-tail distributions of forces even at high compression rates, outperforming other subsampling methods. Furthermore, MLIPs trained on MSC-compressed datasets exhibit reduced error for out-of-distribution data even in low-data regimes. We explain these results using an outlier analysis and show that such quantitative conclusions could not be achieved with conventional dimensionality reduction methods. The algorithm is implemented in the open-source QUESTS package and can be used for several tasks in atomistic modeling, from data subsampling, outlier detection, and training improved MLIPs at a lower cost.
36 MATERIALS SCIENCE↗
Model-free estimation of completeness, uncertainties, and outliers in atomistic machine learning using information theory
Abstract An accurate description of information is relevant for a range of problems in atomistic machine learning (ML), such as crafting training sets, performing uncertainty quantification (UQ), or extracting physical insights from large datasets. However, atomistic ML often relies on unsupervised learning or model predictions to analyze information contents from simulation or training data. Here, we introduce a theoretical framework that provides a rigorous, model-free tool to quantify information contents in atomistic simulations. We demonstrate that the information entropy of a distribution of atom-centered environments explains known heuristics in ML potential developments, from training set sizes to dataset optimality. Using this tool, we propose a model-free UQ method that reliably predicts epistemic uncertainty and detects out-of-distribution samples, including rare events in systems such as nucleation. This method provides a general tool for data-driven atomistic modeling and combines efforts in ML, simulations, and physical explainability.
36 MATERIALS SCIENCE↗
Relativistic quantum information theory
Explore the source record for details and available documents.
information theory quantum theory relativity↗
Debates: Does Information Theory Provide a New Paradigm for Earth Science? Emerging Concepts and Pathways of Information Physics
Explore the source record for details and available documents.
Rui A.P. Perdigão↗
What Information Theory Says about Bounded Rational Best Response
Probability Collectives (PC) provides the information-theoretic extension of conventional full-rationality game theory to bounded rational games. Here an explicit solution to the equations giving the bounded rationality equilibrium of a game is presented. Then PC is used to investigate games in which the players use bounded rational best-response strategies. Next it is shown that in the continuum-time limit, bounded rational best response games result in a variant of the replicator dynamics of evolutionary game theory. It is then shown that for team (shared-payoff) games, this variant of replicator dynamics is identical to Newton-Raphson iterative optimization of the shared utility function.
Wolpert, David H.↗
An Information Theory Approach on Deciding Spectroscopic Follow-ups
Classification and characterization of variable phenomena and transient phenomena are critical for astrophysics and cosmology. These objects are commonly studied using photometric time series or spectroscopic data. Given that many ongoing and future surveys are conducted in a time domain, and given that adding spectra provides further insights but requires more observational resources, it would be valuable to know which objects we should prioritize to have a spectrum in addition to a time series. We propose a methodology in a probabilistic setting that determines a priori which objects are worth taking a spectrum of to obtain better insights, where we focus on the insight of the type of the object (classification). Objects for which we query their spectrum are reclassified using their full spectral information. We first train two classifiers, one that uses photometric data and another that uses photometric and spectroscopic data together. Then for each photometric object we estimate the probability of each possible spectrum outcome. We combine these models in various probabilistic frameworks (strategies), which are used to guide the selection of follow-up observations. The best strategy depends on the intended use, whether it is obtaining more confidence or accuracy. For a given number of candidate objects (127, equal to 5% of the data set) for taking spectra, we improve the class prediction accuracy by 37% as opposed to 20% of a non-naive (non-random) best-baseline strategy. Our approach provides a general framework for follow-up strategies and can be extended beyond classification to include other forms of follow-ups beyond spectroscopy.
79 ASTRONOMY AND ASTROPHYSICS↗
The problem of applying information theory to efficient image transmission.
The main ideas of Shannon's (1948, 1960) theory of source encoding with a fidelity constraint, more commonly known as rate distortion theory, are summarized. The theory was specifically intended to provide a theoretical basis for efficient transmission of information such as images. What the theory has to contribute to the problem is demonstrated. Difficulties that impeded application of the theory to image transmission, and current efforts to solve these difficulties are discussed.
Sakrison, D. J.↗
An information theory of image gathering
Shannon's mathematical theory of communication is extended to image gathering. Expressions are obtained for the total information that is received with a single image-gathering channel and with parallel channels. It is concluded that the aliased signal components carry information even though these components interfere with the within-passband components in conventional image gathering and restoration, thereby degrading the fidelity and visual quality of the restored image. An examination of the expression for minimum mean-square-error, or Wiener-matrix, restoration from parallel image-gathering channels reveals a method for unscrambling the within-passband and aliased signal components to restore spatial frequencies beyond the sampling passband out to the spatial frequency response cutoff of the optical aperture.
Fales, Carl L.↗