Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Maximum Entropy Principle”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

A general maximum entropy framework for thermodynamic variational principles

Minimum free energy principles are familiar in equilibrium thermodynamics, as expressions of the second law. They also appear in statistical mechanics as variational approximation schemes, such as the mean-field and steepest-descent approximations. These well-known minimum free energy principles are here unified and extended to any system analyzable by MaxEnt, including non-equilibrium systems. The MaxEnt Lagrangian associated with a generic MaxEnt distribution p defines a generalized potential Ψ for an arbitrary probability distribution p-hat, such that Ψ is a minimum at (p-hat) = p. Minimization of Ψ with respect to p-hat thus constitutes a generic variational principle, and is equivalent to minimizing the Kullback-Leibler divergence between p-hat and p. Illustrative examples of min–Ψ are given for equilibrium and non-equilibrium systems. An interpretation of changes in Ψ is given in terms of the second law, although min–Ψ itself is an intrinsic variational property of MaxEnt that is distinct from the second law.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Probability of stress-corrosion fracture under random loading

Mathematical formulation is based on cumulative-damage hypothesis and experimentally-determined stress-corrosion characteristics. Under both stationary random loadings, mean value and variance of cumulative damage are obtained. Probability of stress-corrosion fracture is then evaluated, using principle of maximum entropy.

Yang, J. N.↗

Probability of stress-corrosion fracture under random loading.

A method is developed for predicting the probability of stress-corrosion fracture of structures under random loadings. The formulation is based on the cumulative damage hypothesis and the experimentally determined stress-corrosion characteristics. Under both stationary and nonstationary random loadings, the mean value and the variance of the cumulative damage are obtained. The probability of stress-corrosion fracture is then evaluated using the principle of maximum entropy. It is shown that, under stationary random loadings, the standard deviation of the cumulative damage increases in proportion to the square root of time, while the coefficient of variation (dispersion) decreases in inversed proportion to the square root of time. Numerical examples are worked out to illustrate the general results.

Yang, J.-N.↗

On Bayesian Inductive Inference & Predictive Estimation

We investigate Bayesian inference and the Principle of Maximum Entropy (PME) as methods for doing inference under uncertainty. This investigation is primarily through concrete examples that have been previously investigated in the literature. We find that it is possible to do Bayesian inference and PME inference using the same information, despite claims to the contrary, but that the results are not directly comparable. This is because Bayesian inference yields a probability density function (pdf) over the unknown model parameters, whereas PME yields point estimates. If mean estimates are extracted from the Bayesian pdfs, the resulting parameter estimates can differ radically from the PME values and also from the Maximum Likelihood values. We conclude that these differences are due to the Bayesian inference not assuming anything beyond the given prior probabilities and the data, whereas PME implicitly assumes that the given constraints are the only constraints that are operating. Since this assumption can be wrong, PME values may have to be revised when subsequent data shows evidence for more constraints. The entropy concentration previously "proved" by E. T. Jaynes is shown to be in error. Further, we show that PME is a generalized form of independence assumption, and so can be a very powerful method of inference when the variables being investigated are largely independent of each other.

Cheeseman, Peter↗

Finding a Minimally Informative Dirichlet Prior Distribution Using Least Squares

In a Bayesian framework, the Dirichlet distribution is the conjugate distribution to the multinomial likelihood function, and so the analyst is required to develop a Dirichlet prior that incorporates available information. However, as it is a multiparameter distribution, choosing the Dirichlet parameters is less straight-forward than choosing a prior distribution for a single parameter, such as p in the binomial distribution. In particular, one may wish to incorporate limited information into the prior, resulting in a minimally informative prior distribution that is responsive to updates with sparse data. In the case of binomial p or Poisson, the principle of maximum entropy can be employed to obtain a so-called constrained noninformative prior. However, even in the case of p, such a distribution cannot be written down in closed form, and so an approximate beta distribution is used in the case of p. In the case of the multinomial model with parametric constraints, the approach of maximum entropy does not appear tractable. This paper presents an alternative approach, based on constrained minimization of a least-squares objective function, which leads to a minimally informative Dirichlet prior distribution. The alpha-factor model for common-cause failure, which is widely used in the United States, is the motivation for this approach, and is used to illustrate the method. In this approach to modeling common-cause failure, the alpha-factors, which are the parameters in the underlying multinomial aleatory model for common-cause failure, must be estimated from data that is often quite sparse, because common-cause failures tend to be rare, especially failures of more than two or three components, and so a prior distribution that is responsive to updates with sparse data is needed.

99 GENERAL AND MISCELLANEOUS↗

Finding A Minimally Informative Dirichlet Prior Using Least Squares

In a Bayesian framework, the Dirichlet distribution is the conjugate distribution to the multinomial likelihood function, and so the analyst is required to develop a Dirichlet prior that incorporates available information. However, as it is a multiparameter distribution, choosing the Dirichlet parameters is less straightforward than choosing a prior distribution for a single parameter, such as p in the binomial distribution. In particular, one may wish to incorporate limited information into the prior, resulting in a minimally informative prior distribution that is responsive to updates with sparse data. In the case of binomial p or Poisson \lambda, the principle of maximum entropy can be employed to obtain a so-called constrained noninformative prior. However, even in the case of p, such a distribution cannot be written down in the form of a standard distribution (e.g., beta, gamma), and so a beta distribution is used as an approximation in the case of p. In the case of the multinomial model with parametric constraints, the approach of maximum entropy does not appear tractable. This paper presents an alternative approach, based on constrained minimization of a least-squares objective function, which leads to a minimally informative Dirichlet prior distribution. The alpha-factor model for common-cause failure, which is widely used in the United States, is the motivation for this approach, and is used to illustrate the method. In this approach to modeling common-cause failure, the alpha-factors, which are the parameters in the underlying multinomial model for common-cause failure, must be estimated from data that are often quite sparse, because common-cause failures tend to be rare, especially failures of more than two or three components, and so a prior distribution that is responsive to updates with sparse data is needed.

99 GENERAL AND MISCELLANEOUS↗

A new formulation of UMC and an Evaluation of the {sup 55}Mn(n,{gamma}){sup 56}Mn Dosimetry Reaction Cross Section (a.k.a. 'Toy Story 4')

Two relatively new approaches to neutron cross section data evaluation are described. They are known collectively as Unified Monte Carlo (versions UMC-G and UMC-B). Comparisons are made between these two methods, as well as with the well-known generalized least-squares (GLSQ) technique, through the use of simple, hypothetical (toy) examples. These new Monte Carlo methods are based on stochastic sampling of probability functions that are constructed with the use of theoretical and experimental data by applying the principle of maximum entropy. No further assumptions are involved in either UMC-G or UMC-B. However, the GLSQ procedure requires the linearization of non-linear terms, such as those that occur when cross section ratio data are included in an evaluation. It is shown that these two stochastic techniques yield results that agree well with each other, and with the GLSQ method, when linear data are involved, or when the perturbations due to data discrepancies and nonlinearity effects are small. Otherwise, there can be noticeable differences. The present investigation also demonstrates, as observed in earlier work, that the least-squares approach breaks down when these conditions are not satisfied. This paper also presents an actual evaluation of the {sup 55}Mn(n,{gamma}) {sup 56}Mn neutron dosimetry reaction cross section in the energy range from 100 keV to 20 MeV, which was performed using both GLSQ and UMC-G approaches. (authors)

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Prediction of Metabolite Concentrations, Rate Constants and Post-Translational Regulation Using Maximum Entropy-Based Simulations with Application to Central Metabolism of Neurospora crassa

We report the application of a recently proposed approach for modeling biological systems using a maximum entropy production rate principle in lieu of having in vivo rate constants. The method is applied in four steps: (1) a new ordinary differential equation (ODE) based optimization approach based on Marcelin’s 1910 mass action equation is used to obtain the maximum entropy distribution; (2) the predicted metabolite concentrations are compared to those generally expected from experiments using a loss function from which post-translational regulation of enzymes is inferred; (3) the system is re-optimized with the inferred regulation from which rate constants are determined from the metabolite concentrations and reaction fluxes; and finally (4) a full ODE-based, mass action simulation with rate parameters and allosteric regulation is obtained. From the last step, the power characteristics and resistance of each reaction can be determined. The method is applied to the central metabolism of Neurospora crassa and the flow of material through the three competing pathways of upper glycolysis, the non-oxidative pentose phosphate pathway, and the oxidative pentose phosphate pathway are evaluated as a function of the NADP/NADPH ratio. It is predicted that regulation of phosphofructokinase (PFK) and flow through the pentose phosphate pathway are essential for preventing an extreme level of fructose 1,6-bisphophate accumulation. Such an extreme level of fructose 1,6-bisphophate would otherwise result in a glassy cytoplasm with limited diffusion, dramatically decreasing the entropy and energy production rate and, consequently, biological competitiveness.

Cannon, William R.↗

Adaptive schemes for incomplete quantum process tomography

We propose an iterative algorithm for incomplete quantum process tomography with the help of quantum state estimation. The algorithm, which is based on the combined principles of maximum likelihood and maximum entropy, yields a unique estimator for an unknown quantum process when one has less than a complete set of linearly independent measurement data to specify the quantum process uniquely. We apply this iterative algorithm adaptively in various situations and so optimize the amount of resources required to estimate a quantum process with incomplete data.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

From Entropic Dynamics to Quantum Theory

Non-relativistic quantum theory is derived from information codified into an appropriate statistical model. The basic assumption is that there is an irreducible uncertainty in the location of particles so that the configuration space is a statistical manifold. The dynamics then follows from a principle of inference, the method of Maximum Entropy. The concept of time is introduced as a convenient way to keep track of change. The resulting theory resembles both Nelson's stochastic mechanics and general relativity. The statistical manifold is a dynamical entity: its geometry determines the evolution of the probability distribution which, in its turn, reacts back and determines the evolution of the geometry. There is a new quantum version of the equivalence principle: 'osmotic' mass equals inertial mass. Mass and the phase of the wave function are explained as features of purely statistical origin.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Physics-based stabilized finite element approximations of the Poisson–Nernst–Planck equations

We present and analyze two stabilized finite element methods for solving numerically the Poisson–Nernst–Planck equations. The stabilization we consider is carried out by using a shock detector and a discrete graph Laplacian operator for the ion equations, whereas the discrete equation for the electric potential need not be stabilized. Discrete solutions stemmed from the first algorithm preserve both maximum and minimum discrete principles. For the second algorithm, its discrete solutions are conceived so that they hold discrete principles and obey an entropy law provided that an acuteness condition is imposed for meshes. Remarkably the latter is found to be unconditionally stable. We validate our methodology through transient numerical experiments that show convergence toward steady-state solutions.

97 MATHEMATICS AND COMPUTING↗

High resolution schemes and the entropy condition

A systematic procedure for constructing semidiscrete, second order accurate, variation diminishing, five point band width, approximations to scalar conservation laws, is presented. These schemes are constructed to also satisfy a single discrete entropy inequality. Thus, in the convex flux case, convergence is proven to be the unique physically correct solution. For hyperbolic systems of conservation laws, this construction is used formally to extend the first author's first order accurate scheme, and show (under some minor technical hypotheses) that limit solutions satisfy an entropy inequality. Results concerning discrete shocks, a maximum principle, and maximal order of accuracy are obtained. Numerical applications are also presented.

Osher, S.↗

Probabilistic and maximum entropy modeling of chemical reaction systems: Characteristics and comparisons to mass action kinetic models

We demonstrate and characterize a first-principles approach to modeling the mass action dynamics of metabolism. Starting from a basic definition of entropy expressed as a multinomial probability density using Boltzmann probabilities with standard chemical potentials, we derive and compare the free energy dissipation and the entropy production rates. We express the relation between entropy production and the chemical master equation for modeling metabolism, which unifies chemical kinetics and chemical thermodynamics. Because prediction uncertainty with respect to parameter variability is frequently a concern with mass action models utilizing rate constants, we compare and contrast the maximum entropy model, which has its own set of rate parameters, to a population of standard mass action models in which the rate constants are randomly chosen. We show that a maximum entropy model is characterized by a high probability of free energy dissipation rate and likewise entropy production rate, relative to other models. We then characterize the variability of the maximum entropy model predictions with respect to uncertainties in parameters (standard free energies of formation) and with respect to ionic strengths typically found in a cell.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Efficient algorithms and implementations of entropy-based moment closures for rarefied gases

We present efficient algorithms and implementations of the 35-moment system equipped with the maximum-entropy closure in the context of rarefied gases. While closures based on the principle of entropy maximization have been shown to yield very promising results for moderately rarefied gas flows, the computational cost of these closures is in general much higher than for closure theories with explicit closed-form expressions of the closing fluxes, such as Grad's classical closure. Following a similar approach as Garrett et al. (2015) , we investigate efficient implementations of the computationally expensive numerical quadrature method used for the moment evaluations of the maximum-entropy distribution by exploiting its inherent fine-grained parallelism with the parallelism offered by multi-core processors and graphics cards. We show that using a single graphics card as an accelerator allows speed-ups of two orders of magnitude when compared to a serial CPU implementation. To accelerate the time-to-solution for steady-state problems, we propose a new semi-implicit time discretization scheme. The resulting nonlinear system of equations is solved with a Newton type method in the Lagrange multipliers of the dual optimization problem in order to reduce the computational cost. Additionally, fully explicit time-stepping schemes of first and second order accuracy are presented. We investigate the accuracy and efficiency of the numerical schemes for several numerical test cases, including a steady-state shock-structure problem.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Magnetic and magnetothermodynamic properties of Ho5Si4

The magnetic and magnetocaloric properties of Ho{sub 5}Si{sub 4} have been investigated. The compound undergoes a second order ferromagnetic transition at 76 K (T{sub c}) and a spin reorientation transition at about 15 K. The temperature dependencies of heat capacity data measured in various magnetic fields corroborate the second order ferromagnetic character of the transition at T{sub c}. For a field change ({Delta}H) of 50 kOe, the maximum values of isothermal magnetic entropy change ({Delta}S{sub M}) and adiabatic temperature change ({Delta}T{sub ad}) are -14.8 J kg{sup -1} K{sup -1} (114 mJ/cm{sup 3} K) and 6.1 K, respectively. From the first principles calculations the T{sub c} of Ho{sub 5}Si{sub 4} is estimated to be 78 K and the maximum value of {Delta}S{sub M} is estimated to be -12.6 J kg{sup -1} K{sup -1} for {Delta}H = 50 kOe.

36 MATERIALS SCIENCE↗

Use and validity of principles of extremum of entropy production in the study of complex systems

It is shown how both the principles of extremum of entropy production, which are often used in the study of complex systems, follow from the maximization of overall system conductivities, under appropriate constraints. In this way, the maximum rate of entropy production (MEP) occurs when all the forces in the system are kept constant. On the other hand, the minimum rate of entropy production (mEP) occurs when all the currents that cross the system are kept constant. A brief discussion on the validity of the application of the mEP and MEP principles in several cases, and in particular to the Earth’s climate is also presented. -- Highlights: •The principles of extremum of entropy production are not first principles. •They result from the maximization of conductivities under appropriate constraints. •The conditions of their validity are set explicitly. •Some long-standing controversies are discussed and clarified.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A crisis for the verification and validation of turbulence simulations

Three algorithms have been proposed for solution of the Rayleigh–Taylor turbulent mixing problem. They are based upon three different physical principles governing the Euler equations for fluid flow. The principles serve to select the physically relevant solution from among many nonunique solutions. The admissibility principle is in dispute. The three different algorithms, expressing the three physical admissibility principles can be formulated in terms of the three energy dissipation rates or the entropy production rates, as selected by the size of the sub grid scale coefficients. These have maximal values or less than maximal values. The resulting solutions are markedly different. We find strong validation evidence that supports the maximum rate principle, based on a review of prior results and on new results presented here. We review experimental data used for validation and sufficient to discriminate among the three. We present a new analysis of this data. We show that the hypothesized long wave length perturbations in the initial conditions are not significant, so that validation can be based on this data in a straight forward manner. One of the algorithms is labeled direct numerical simulation, but is not, and as a consequence, the two algorithms with less than maximal SGS coefficients are variants of one another. Finally recommendations for the numerical modeling of the deflagration to detonation transition in type Ia supernova are discussed.

79 ASTRONOMY AND ASTROPHYSICS↗