Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “MARKOV PROCESS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2013

Expected Power-Utility Maximization Under Incomplete Information and with Cox-Process Observations

We consider the problem of maximization of expected terminal power utility (risk sensitive criterion). The underlying market model is a regime-switching diffusion model where the regime is determined by an unobservable factor process forming a finite state Markov process. The main novelty is due to the fact that prices are observed and the portfolio is rebalanced only at random times corresponding to a Cox process where the intensity is driven by the unobserved Markovian factor process as well. This leads to a more realistic modeling for many practical situations, like in markets with liquidity restrictions; on the other hand it considerably complicates the problem to the point that traditional methodologies cannot be directly applied. The approach presented here is specific to the power-utility. For log-utilities a different approach is presented in Fujimoto et al. (Preprint, 2012).

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
NASA NTRS1972

A Bayesian model for visual space perception

A model for visual space perception is proposed that contains desirable features in the theories of Gibson and Brunswik. This model is a Bayesian processor of proximal stimuli which contains three important elements: an internal model of the Markov process describing the knowledge of the distal world, the a priori distribution of the state of the Markov process, and an internal model relating state to proximal stimuli. The universality of the model is discussed and it is compared with signal detection theory models. Experimental results of Kinchla are used as a special case.

Curry, R. E.↗
NASA NTRS1995

Error modeling for differential GPS

Differential Global Positioning System (DGPS) positioning is used to accurately locate a GPS receiver based upon the well-known position of a reference site. In utilizing this technique, several error sources contribute to position inaccuracy. This thesis investigates the error in DGPS operation and attempts to develop a statistical model for the behavior of this error. The model for DGPS error is developed using GPS data collected by Draper Laboratory. The Marquardt method for nonlinear curve-fitting is used to find the parameters of a first order Markov process that models the average errors from the collected data. The results show that a first order Markov process can be used to model the DGPS error as a function of baseline distance and time delay. The model's time correlation constant is 3847.1 seconds (1.07 hours) for the mean square error. The distance correlation constant is 122.8 kilometers. The total process variance for the DGPS model is 3.73 sq meters.

Blerman, Gregory S.↗
NASA NTRS1972

Control theory for random systems

A survey is presented of the current knowledge available for designing and predicting the effectiveness of controllers for dynamic systems which can be modeled by ordinary differential equations. A short discussion of feedback control is followed by a description of deterministic controller design and the concept of system state. The need for more realistic disturbance models led to the use of stochastic process concepts, in particular the Gauss-Markov process. A compensator controlled system, with random forcing functions, random errors in the measurements, and random initial conditions, is treated as constituting a Gauss-Markov random process; hence the mean-square behavior of the controlled system is readily predicted. As an example, a compensator is designed for a helicopter to maintain it in hover in a gusty wind over a point on the ground.

Bryson, A. E., Jr.↗
DOE OSTI2015

Need for Caution in Interpreting Extreme Weather Statistics

Given the reality of anthropogenic global warming, it is tempting to seek an anthropogenic component in any recent change in the statistics of extreme weather. This paper cautions that such efforts may, however, lead to wrong conclusions if the distinctively skewed and heavy-tailed aspects of the probability distributions of daily weather anomalies are ignored or misrepresented. Departures of several standard deviations from the mean, although rare, are far more common in such a distinctively non-Gaussian world than they are in a Gaussian world. This further complicates the problem of detecting changes in tail probabilities from historical records of limited length and accuracy. A possible solution is to exploit the fact that the salient non-Gaussian features of the observed distributions are captured by so-called stochastically generated skewed (SGS) distributions that include Gaussian distributions as special cases. SGS distributions are associated with damped linear Markov processes perturbed by asymmetric stochastic noise and as such represent the simplest physically based prototypes of the observed distributions. The tails of SGS distributions can also be directly linked to generalized extreme value (GEV) and generalized Pareto (GP) distributions. The Markov process model can be used to provide rigorous confidence intervals and to investigate temporal persistence statistics. The procedure is illustrated for assessing changes in the observed distributions of daily wintertime indices of large-scale atmospheric variability in the North Atlantic and North Pacific sectors over the period 1872–2011. No significant changes in these indices are found from the first to the second half of the period.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1981

A stochastic-dynamical approach to the study of the natural variability of the climate

A method, suggested by Leith (1975), which employed stochastic-dynamic forecasts obtained from a general circulation model in such a way as to satisfy the definition of climatic noise, was used to validate assumptions accounting for the effects of external influences in estimating the climatic noise. Two assumptions were investigated: (1) that the weather fluctuations can be represented as a Markov process, and (2) that changing external conditions do not influence the atmosphere's statistical properties on short time scales. The general circulation model's simulation of the daily weather fluctuations was generated by performing integrations with prescribed climatological boundary conditions for random initial atmospheric states, with resulting dynamical forecasts providing an ensemble of simulated data for the autoregressive modeling of weather fluctuations. To estimate the climatic noise from the observational data (consisting of hourly values of sea level pressure and surface temperature at 54 U.S. stations for the month of January for the years 1949-1975) use of the short time-scale assumption is made. The simulated and observed data were found not to be consistent with either white noise or a Markov process of weather fluctuations. Good agreement was found between the results of the hypothetical testing of the simulated and the observed surface temperatures; and only partial support was found for the short time-scale assumption, i.e., for sea level pressure.

Straus, D. M.↗
DOE OSTI2019

Stationary Spatial Charging Demand Distribution for Commercial Electric Vehicles in Urban Area

The electric vehicle (EV) is an increasingly popu- lar mobility solution today. It is especially appealing for urban commercial usage including serving commuters and delivering goods. Commercial EVs, defined as electric taxi fleets in this paper, contribute a lot to the energy consumption and emission. One critical component to facilitate the adoption of EVs is to provide sufficient charging infrastructures for road users to alleviate the range anxiety. In this study, we develop an analytical model to profile the stationary spatial distribution of charging demand for any given fleet of commercial EVs, which serves as the necessary input for planning the location of charging infrastructures. Our model considers the movement of EVs as a random walk, with the underlying Markov process driven by the spatial demand that the commercial EV fleet needs to serve. We then define the electricity consumption and charging models and develop the probability distribution that an EV may need to recharge its battery at any given location in the city, based on the stationary distribution of the random walk process. We calibrate the performance parameters of EVs from the trip record and present numerical experiments based on New York City taxi trip data and compare the solution from simulation with our analytical solution. The result shows that analytical solution is quite close to the simulation result. This paper provides insights to the city agencies and private companies to make charging infrastructure location planning.

Qian, Xinwu↗
NASA NTRS1999

Cooperation and Coordination Between Fuzzy Reinforcement Learning Agents in Continuous State Partially Observable Markov Decision Processes

Successful operations of future multi-agent intelligent systems require efficient cooperation schemes between agents sharing learning experiences. We consider a pseudo-realistic world in which one or more opportunities appear and disappear in random locations. Agents use fuzzy reinforcement learning to learn which opportunities are most worthy of pursuing based on their promise rewards, expected lifetimes, path lengths and expected path costs. We show that this world is partially observable because the history of an agent influences the distribution of its future states. We consider a cooperation mechanism in which agents share experience by using and-updating one joint behavior policy. We also implement a coordination mechanism for allocating opportunities to different agents in the same world. Our results demonstrate that K cooperative agents each learning in a separate world over N time steps outperform K independent agents each learning in a separate world over K*N time steps, with this result becoming more pronounced as the degree of partial observability in the environment increases. We also show that cooperation between agents learning in the same world decreases performance with respect to independent agents. Since cooperation reduces diversity between agents, we conclude that diversity is a key parameter in the trade off between maximizing utility from cooperation when diversity is low and maximizing utility from competitive coordination when diversity is high.

Berenji, Hamid R.↗
DOE OSTI2014

Large deviations in stochastic heat-conduction processes provide a gradient-flow structure for heat conduction

We consider three one-dimensional continuous-time Markov processes on a lattice, each of which models the conduction of heat: the family of Brownian Energy Processes with parameter m (BEP(m)), a Generalized Brownian Energy Process, and the Kipnis-Marchioro-Presutti (KMP) process. The hydrodynamic limit of each of these three processes is a parabolic equation, the linear heat equation in the case of the BEP(m) and the KMP, and a nonlinear heat equation for the Generalized Brownian Energy Process with parameter a (GBEP(a)). We prove the hydrodynamic limit rigorously for the BEP(m), and give a formal derivation for the GBEP(a). We then formally derive the pathwise large-deviation rate functional for the empirical measure of the three processes. These rate functionals imply gradient-flow structures for the limiting linear and nonlinear heat equations. We contrast these gradient-flow structures with those for processes describing the diffusion of mass, most importantly the class of Wasserstein gradient-flow systems. The linear and nonlinear heat-equation gradient-flow structures are each driven by entropy terms of the form -log ρ; they involve dissipation or mobility terms of order ρ² for the linear heat equation, and a nonlinear function of ρ for the nonlinear heat equation.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2016

Transitions in genetic toggle switches driven by dynamic disorder in rate coefficients

In biochemical systems, intrinsic noise may drive the system switch from one stable state to another. We investigate how kinetic switching between stable states in a bistable network is influenced by dynamic disorder, i.e., fluctuations in the rate coefficients. Using the geometric minimum action method, we first investigate the optimal transition paths and the corresponding minimum actions based on a genetic toggle switch model in which reaction coefficients draw from a discrete probability distribution. For the continuous probability distribution of the rate coefficient, we then consider two models of dynamic disorder in which reaction coefficients undergo different stochastic processes with the same stationary distribution. In one, the kinetic parameters follow a discrete Markov process and in the other they follow continuous Langevin dynamics. We find that regulation of the parameters modulating the dynamic disorder, as has been demonstrated to occur through allosteric control in bistable networks in the immune system, can be crucial in shaping the statistics of optimal transition paths, transition probabilities, and the stationary probability distribution of the network.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗