Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Markov process”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

A stochastic model and Monte Carlo algorithm for fluctuation-induced H{sub 2} formation on the surface of interstellar dust grains

A stochastic algorithm for simulation of fluctuation-induced kinetics of H{sub 2} formation on grain surfaces is suggested as a generalization of the technique developed in our recent studies [1] where this method was developed to describe the annihilation of spatially separate electrons and holes in a disordered semiconductor. The stochastic model is based on the spatially inhomogeneous, nonlinear integro-differential Smoluchowski equations with random source term. In this paper we derive the general system of Smoluchowski type equations for the formation of H{sub 2} from two hydrogen atoms on the surface of interstellar dust grains with physisorption and chemisorption sites. We focus in this study on the spatial distribution, and numerically investigate the segregation in the case of a source with a continuous generation in time and randomly distributed in space. The stochastic particle method presented is based on a probabilistic interpretation of the underlying process as a stochastic Markov process of interacting particle system in discrete but randomly progressed time instances. The segregation is analyzed through the correlation analysis of the vector random field of concentrations which appears to be isotropic in space and stationary in time.

79 ASTRONOMY AND ASTROPHYSICS↗

Reliability calculation using randomization for Markovian fault-tolerant computing systems

The randomization technique for computing transient probabilities of Markov processes is presented. The technique is applied to a Markov process model of a simplified fault tolerant computer system for illustrative purposes. It is applicable to much larger and more complex models. Transient state probabilities are computed, from which reliabilities are derived. An accelerated version of the randomization algorithm is developed which exploits ''stiffness' of the models to gain increased efficiency. A great advantage of the randomization approach is that it easily allows probabilities and reliabilities to be computed to any predetermined accuracy.

Miller, D. R.↗

Markov reward processes

Numerous applications in the area of computer system analysis can be effectively studied with Markov reward models. These models describe the behavior of the system with a continuous-time Markov chain, where a reward rate is associated with each state. In a reliability/availability model, upstates may have reward rate 1 and down states may have reward rate zero associated with them. In a queueing model, the number of jobs of certain type in a given state may be the reward rate attached to that state. In a combined model of performance and reliability, the reward rate of a state may be the computational capacity, or a related performance measure. Expected steady-state reward rate and expected instantaneous reward rate are clearly useful measures of the Markov reward model. More generally, the distribution of accumulated reward or time-averaged reward over a finite time interval may be determined from the solution of the Markov reward model. This information is of great practical significance in situations where the workload can be well characterized (deterministically, or by continuous functions e.g., distributions). The design process in the development of a computer system is an expensive and long term endeavor. For aerospace applications the reliability of the computer system is essential, as is the ability to complete critical workloads in a well defined real time interval. Consequently, effective modeling of such systems must take into account both performance and reliability. This fact motivates our use of Markov reward models to aid in the development and evaluation of fault tolerant computer systems.

Smith, R. M.↗

A stochastic model for tumor geometry evolution during radiation therapy in cervical cancer

Purpose: To develop mathematical models to predict the evolution of tumor geometry in cervical cancer undergoing radiation therapy. Methods: The authors develop two mathematical models to estimate tumor geometry change: a Markov model and an isomorphic shrinkage model. The Markov model describes tumor evolution by investigating the change in state (either tumor or nontumor) of voxels on the tumor surface. It assumes that the evolution follows a Markov process. Transition probabilities are obtained using maximum likelihood estimation and depend on the states of neighboring voxels. The isomorphic shrinkage model describes tumor shrinkage or growth in terms of layers of voxels on the tumor surface, instead of modeling individual voxels. The two proposed models were applied to data from 29 cervical cancer patients treated at Princess Margaret Cancer Centre and then compared to a constant volume approach. Model performance was measured using sensitivity and specificity. Results: The Markov model outperformed both the isomorphic shrinkage and constant volume models in terms of the trade-off between sensitivity (target coverage) and specificity (normal tissue sparing). Generally, the Markov model achieved a few percentage points in improvement in either sensitivity or specificity compared to the other models. The isomorphic shrinkage model was comparable to the Markov approach under certain parameter settings. Convex tumor shapes were easier to predict. Conclusions: By modeling tumor geometry change at the voxel level using a probabilistic model, improvements in target coverage and normal tissue sparing are possible. Our Markov model is flexible and has tunable parameters to adjust model performance to meet a range of criteria. Such a model may support the development of an adaptive paradigm for radiation therapy of cervical cancer.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Unifying Model-Based and Reactive Programming within a Model-Based Executive

Real-time, model-based, deduction has recently emerged as a vital component in AI's tool box for developing highly autonomous reactive systems. Yet one of the current hurdles towards developing model-based reactive systems is the number of methods simultaneously employed, and their corresponding melange of programming and modeling languages. This paper offers an important step towards unification. We introduce RMPL, a rich modeling language that combines probabilistic, constraint-based modeling with reactive programming constructs, while offering a simple semantics in terms of hidden state Markov processes. We introduce probabilistic, hierarchical constraint automata (PHCA), which allow Markov processes to be expressed in a compact representation that preserves the modularity of RMPL programs. Finally, a model-based executive, called Reactive Burton is described that exploits this compact encoding to perform efficIent simulation, belief state update and control sequence generation.

Williams, Brian C.↗

Numerical Simulation of Polynomial-Speed Convergence Phenomenon

We provide a hybrid method that captures the polynomial speed of convergence and polynomial speed of mixing for Markov processes. The hybrid method that we introduce is based on the coupling technique and renewal theory. We propose to replace some estimates in classical results about the ergodicity of Markov processes by numerical simulations when the corresponding analytical proof is difficult. After that, all remaining conclusions can be derived from rigorous analysis. Then we apply our results to seek numerical justification for the ergodicity of two 1D microscopic heat conduction models. The mixing rate of these two models are expected to be polynomial but very difficult to prove. In both examples, our numerical results match the expected polynomial mixing rate well.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Job-mix modeling and system analysis of an aerospace multiprocessor.

An aerospace guidance computer organization, consisting of multiple processors and memory units attached to a central time-multiplexed data bus, is described. A job mix for this type of computer is obtained by analysis of Apollo mission programs. Multiprocessor performance is then analyzed using: 1) queuing theory, under certain 'limiting case' assumptions; 2) Markov process methods; and 3) system simulation. Results of the analyses indicate: 1) Markov process analysis is a useful and efficient predictor of simulation results; 2) efficient job execution is not seriously impaired even when the system is so overloaded that new jobs are inordinately delayed in starting; 3) job scheduling is significant in determining system performance; and 4) a system having many slow processors may or may not perform better than a system of equal power having few fast processors, but will not perform significantly worse.

Mallach, E. G.↗

A statistical rain attenuation prediction model with application to the advanced communication technology satellite project. Part 2: Theoretical development of a dynamic model and application to rain fade durations and tolerable control delays for fade countermeasures

A dynamic rain attenuation prediction model is developed for use in obtaining the temporal characteristics, on time scales of minutes or hours, of satellite communication link availability. Analagous to the associated static rain attenuation model, which yields yearly attenuation predictions, this dynamic model is applicable at any location in the world that is characterized by the static rain attenuation statistics peculiar to the geometry of the satellite link and the rain statistics of the location. Such statistics are calculated by employing the formalism of Part I of this report. In fact, the dynamic model presented here is an extension of the static model and reduces to the static model in the appropriate limit. By assuming that rain attenuation is dynamically described by a first-order stochastic differential equation in time and that this random attenuation process is a Markov process, an expression for the associated transition probability is obtained by solving the related forward Kolmogorov equation. This transition probability is then used to obtain such temporal rain attenuation statistics as attenuation durations and allowable attenuation margins versus control system delay.

Manning, Robert M.↗

Expected Power-Utility Maximization Under Incomplete Information and with Cox-Process Observations

We consider the problem of maximization of expected terminal power utility (risk sensitive criterion). The underlying market model is a regime-switching diffusion model where the regime is determined by an unobservable factor process forming a finite state Markov process. The main novelty is due to the fact that prices are observed and the portfolio is rebalanced only at random times corresponding to a Cox process where the intensity is driven by the unobserved Markovian factor process as well. This leads to a more realistic modeling for many practical situations, like in markets with liquidity restrictions; on the other hand it considerably complicates the problem to the point that traditional methodologies cannot be directly applied. The approach presented here is specific to the power-utility. For log-utilities a different approach is presented in Fujimoto et al. (Preprint, 2012).

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Decoherence effects in Bose-Einstein condensate interferometry I. General theory

Research Highlights: > Theory of dephasing, decoherence effects for Bose-Einstein condensate interferometry. > Applies to single component, two mode condensate in double potential well. > Phase space theory using Wigner, positive P representations for condensate, non-condensate fields. > Stochastic condensate, non-condensate field equations and properties of noise fields derived. > Based on mean field theory with condensate modes given by generalised Gross-Pitaevskii equations. - Abstract: The present paper outlines a basic theoretical treatment of decoherence and dephasing effects in interferometry based on single component Bose-Einstein condensates in double potential wells, where two condensate modes may be involved. Results for both two mode condensates and the simpler single mode condensate case are presented. The approach involves a hybrid phase space distribution functional method where the condensate modes are described via a truncated Wigner representation, whilst the basically unoccupied non-condensate modes are described via a positive P representation. The Hamiltonian for the system is described in terms of quantum field operators for the condensate and non-condensate modes. The functional Fokker-Planck equation for the double phase space distribution functional is derived. Equivalent Ito stochastic equations for the condensate and non-condensate fields that replace the field operators are obtained, and stochastic averages of products of these fields give the quantum correlation functions that can be used to interpret interferometry experiments. The stochastic field equations are the sum of a deterministic term obtained from the drift vector in the functional Fokker-Planck equation, and a noise field whose stochastic properties are determined from the diffusion matrix in the functional Fokker-Planck equation. The stochastic properties of the noise field terms are similar to those for Gaussian-Markov processes in that the stochastic averages of odd numbers of noise fields are zero and those for even numbers of noise field terms are the sums of products of stochastic averages associated with pairs of noise fields. However each pair is represented by an element of the diffusion matrix rather than products of the noise fields themselves, as in the case of Gaussian-Markov processes. The treatment starts from a generalised mean field theory for two condensate modes, where generalised coupled Gross-Pitaevskii equations are obtained for the modes and matrix mechanics equations are derived for the amplitudes describing possible fragmentations of the condensate between the two modes. These self-consistent sets of equations are derived via the Dirac-Frenkel variational principle. Numerical studies for interferometry experiments would involve using the solutions from the generalised mean field theory in calculations for the stochastic fields from the Ito stochastic field equations.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A Bayesian model for visual space perception

A model for visual space perception is proposed that contains desirable features in the theories of Gibson and Brunswik. This model is a Bayesian processor of proximal stimuli which contains three important elements: an internal model of the Markov process describing the knowledge of the distal world, the a priori distribution of the state of the Markov process, and an internal model relating state to proximal stimuli. The universality of the model is discussed and it is compared with signal detection theory models. Experimental results of Kinchla are used as a special case.

Curry, R. E.↗

Error modeling for differential GPS

Differential Global Positioning System (DGPS) positioning is used to accurately locate a GPS receiver based upon the well-known position of a reference site. In utilizing this technique, several error sources contribute to position inaccuracy. This thesis investigates the error in DGPS operation and attempts to develop a statistical model for the behavior of this error. The model for DGPS error is developed using GPS data collected by Draper Laboratory. The Marquardt method for nonlinear curve-fitting is used to find the parameters of a first order Markov process that models the average errors from the collected data. The results show that a first order Markov process can be used to model the DGPS error as a function of baseline distance and time delay. The model's time correlation constant is 3847.1 seconds (1.07 hours) for the mean square error. The distance correlation constant is 122.8 kilometers. The total process variance for the DGPS model is 3.73 sq meters.

Blerman, Gregory S.↗

Control theory for random systems

A survey is presented of the current knowledge available for designing and predicting the effectiveness of controllers for dynamic systems which can be modeled by ordinary differential equations. A short discussion of feedback control is followed by a description of deterministic controller design and the concept of system state. The need for more realistic disturbance models led to the use of stochastic process concepts, in particular the Gauss-Markov process. A compensator controlled system, with random forcing functions, random errors in the measurements, and random initial conditions, is treated as constituting a Gauss-Markov random process; hence the mean-square behavior of the controlled system is readily predicted. As an example, a compensator is designed for a helicopter to maintain it in hover in a gusty wind over a point on the ground.

Bryson, A. E., Jr.↗

Need for Caution in Interpreting Extreme Weather Statistics

Given the reality of anthropogenic global warming, it is tempting to seek an anthropogenic component in any recent change in the statistics of extreme weather. This paper cautions that such efforts may, however, lead to wrong conclusions if the distinctively skewed and heavy-tailed aspects of the probability distributions of daily weather anomalies are ignored or misrepresented. Departures of several standard deviations from the mean, although rare, are far more common in such a distinctively non-Gaussian world than they are in a Gaussian world. This further complicates the problem of detecting changes in tail probabilities from historical records of limited length and accuracy. A possible solution is to exploit the fact that the salient non-Gaussian features of the observed distributions are captured by so-called stochastically generated skewed (SGS) distributions that include Gaussian distributions as special cases. SGS distributions are associated with damped linear Markov processes perturbed by asymmetric stochastic noise and as such represent the simplest physically based prototypes of the observed distributions. The tails of SGS distributions can also be directly linked to generalized extreme value (GEV) and generalized Pareto (GP) distributions. The Markov process model can be used to provide rigorous confidence intervals and to investigate temporal persistence statistics. The procedure is illustrated for assessing changes in the observed distributions of daily wintertime indices of large-scale atmospheric variability in the North Atlantic and North Pacific sectors over the period 1872–2011. No significant changes in these indices are found from the first to the second half of the period.

54 ENVIRONMENTAL SCIENCES↗

A stochastic-dynamical approach to the study of the natural variability of the climate

A method, suggested by Leith (1975), which employed stochastic-dynamic forecasts obtained from a general circulation model in such a way as to satisfy the definition of climatic noise, was used to validate assumptions accounting for the effects of external influences in estimating the climatic noise. Two assumptions were investigated: (1) that the weather fluctuations can be represented as a Markov process, and (2) that changing external conditions do not influence the atmosphere's statistical properties on short time scales. The general circulation model's simulation of the daily weather fluctuations was generated by performing integrations with prescribed climatological boundary conditions for random initial atmospheric states, with resulting dynamical forecasts providing an ensemble of simulated data for the autoregressive modeling of weather fluctuations. To estimate the climatic noise from the observational data (consisting of hourly values of sea level pressure and surface temperature at 54 U.S. stations for the month of January for the years 1949-1975) use of the short time-scale assumption is made. The simulated and observed data were found not to be consistent with either white noise or a Markov process of weather fluctuations. Good agreement was found between the results of the hypothetical testing of the simulated and the observed surface temperatures; and only partial support was found for the short time-scale assumption, i.e., for sea level pressure.

Straus, D. M.↗

Stationary Spatial Charging Demand Distribution for Commercial Electric Vehicles in Urban Area

The electric vehicle (EV) is an increasingly popu- lar mobility solution today. It is especially appealing for urban commercial usage including serving commuters and delivering goods. Commercial EVs, defined as electric taxi fleets in this paper, contribute a lot to the energy consumption and emission. One critical component to facilitate the adoption of EVs is to provide sufficient charging infrastructures for road users to alleviate the range anxiety. In this study, we develop an analytical model to profile the stationary spatial distribution of charging demand for any given fleet of commercial EVs, which serves as the necessary input for planning the location of charging infrastructures. Our model considers the movement of EVs as a random walk, with the underlying Markov process driven by the spatial demand that the commercial EV fleet needs to serve. We then define the electricity consumption and charging models and develop the probability distribution that an EV may need to recharge its battery at any given location in the city, based on the stationary distribution of the random walk process. We calibrate the performance parameters of EVs from the trip record and present numerical experiments based on New York City taxi trip data and compare the solution from simulation with our analytical solution. The result shows that analytical solution is quite close to the simulation result. This paper provides insights to the city agencies and private companies to make charging infrastructure location planning.

Qian, Xinwu↗

Diffusion approximation of stochastic master equations with jumps

In the presence of quantum measurements with direct photon detection, the evolution of open quantum systems is usually described by stochastic master equations with jumps. Heuristically, diffusion models can be obtained from these equations as an approximation. A condition for a general diffusion approximation for jump master equations is presented. This approximation is rigorously proved by using techniques for Markov processes, which are based on the convergence of Markov generators and martingale problems. This result is illustrated by rigorously obtaining the diffusion approximation for homodyne and heterodyne detection.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗