Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Markov process”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 919 records · Page 51

Markov properties of the magnetic field in the quiet solar photosphere

The observed magnetic field on the solar surface is characterized by a very complex spatial and temporal behavior. Although feature-tracking algorithms have allowed us to deepen our understanding of this behavior, subjectivity plays an important role in the identification and tracking of such features. In this paper, we study the temporal stochasticity of the magnetic field on the solar surface without relying on either the concept of magnetic feature or on the subjective assumptions about their identification and interaction. The analysis is applied to observations of the magnetic field of the quiet solar photosphere carried out with the Imaging Magnetograph eXperiment (IMaX) instrument on board the stratospheric balloon, Sunrise. We show that the joint probability distribution functions of the longitudinal (B{sub ∥}) and transverse (B{sub ⊥}) components of the magnetic field, as well as of the magnetic pressure (B{sup 2}=B{sub ⊥}{sup 2}+B{sub ∥}{sup 2}), verify the necessary and sufficient condition for the Markov chains. Therefore, we establish that the magnetic field as seen by IMaX with a resolution of 0.″15–0.″18 and 33 s cadence, which can be considered as a memoryless temporal fluctuating quantity.

79 ASTRONOMY AND ASTROPHYSICS↗

Extracting host galaxy dispersion measure and constraining cosmological parameters using fast radio burst data

The excessive dispersion measures (DMs) and high Galactic latitudes of fast radio bursts (FRBs) hint toward a cosmological origin of these mysterious transients. Methods of using measured DM and redshift z to study cosmology have been proposed, but one needs to assume a certain amount of DM contribution from the host galaxy (DM{sub HG}) in order to apply those methods. We introduce a slope parameter β(z)≡dln〈DM{sub E}〉/dlnz (where DM{sub E} is the observed DM subtracting the Galactic contribution), which can be directly measured when a sample of FRBs have z measured. We show that 〈DM{sub HG}〉 can be roughly inferred from β and the mean values, (〈DM{sub E}〉)-bar and z-bar , of the sample. Through Monte Carlo simulations, we show that the mean value of local host galaxy DM, 〈DM{sub HG,loc}〉, along with other cosmological parameters (mass density Ω{sub m} in the ΛCDM model, and the IGM portion of the baryon energy density Ω{sub b}f{sub IGM}), can be independently measured through Markov Chain Monte Carlo fitting to the data.

79 ASTRONOMY AND ASTROPHYSICS↗

Robotic Planning under Uncertainty in Spatiotemporal Environments in Expeditionary Science

In the expeditionary sciences, spatiotemporally varying environments -- hydrothermal plumes, algal blooms, lava flows, or animal migrations -- are ubiquitous. Mobile robots are uniquely well-suited to study these dynamic, mesoscale natural environments. We formalize expeditionary science as a sequential decision-making problem, modeled using the language of partially-observable Markov decision processes (POMDPs). Solving the expeditionary science POMDP under real-world constraints requires efficient probabilistic modeling and decision-making in problems with complex dynamics and observational models. Previous work in informative path planning, adaptive sampling, and experimental design have shown compelling results, largely in static environments, using data-driven models and information-based rewards. However, these methodologies do not trivially extend to expeditionary science in spatiotemporal environments: they generally do not make use of scientific knowledge such as equations of state dynamics, they focus on information gathering as opposed to scientific task execution, and they make use of decision-making approaches that scale poorly to large, continuous problems with long planning horizons and real-time operational constraints. In this work, we discuss these and other challenges related to probabilistic modeling and decision-making in expeditionary science, and present some of our preliminary work that addresses these gaps. We ground our results in a real expeditionary science deployment of an autonomous underwater vehicle (AUV) in the deep ocean for hydrothermal vent discovery and characterization. Our concluding thoughts highlight remaining work to be done, and the challenges that merit consideration by the reinforcement learning and decision-making community.

Preston, Victoria↗

Wave propagation in a random medium - A complete set of the moment equations with different wavenumbers

The propagation of waves in a random medium is studied in the 'quasi-optics' and the 'Markov random process' approximations. Under these assumptions, a Fokker-Planck equation satisfied by the characteristic functional of the random wave field is derived. A complete set of moment equations with different transverse coordinates and different wave numbers is then obtained from the Fokker-Planck equation of the characteristic functional. The application of those results to the pulse smearing of the pulsar signal and the frequency correlation function of the wave intensity in interstellar scintillation is briefly discussed.

Lee, L. C.↗

An application of extremal field guidance in statistical low thrust guidance analysis

The subject of this paper is the application of extremal field guidance to the covariance analysis of low thrust retargeting guidance. The general procedure for low thrust navigation analysis is briefly reviewed. The basic equations of trajectory optimization and extremal field guidance are presented with emphasis on the low thrust problem. The adopted formulation includes thruster on-off times as control variables and accounts for the effects of model parameter variations. A method of solution of the guidance equations is outlined; and the application of the solution in a covariance analysis, including a procedure for the treatment of model parameter variations as either biases or first order Gauss-Markov stochastic processes, is discussed.

Jacobson, R. A.↗

Optimum equipment maintenance/replacement policy. Part 2: Markov decision approach

Dynamic programming was utilized as an alternative optimization technique to determine an optimal policy over a given time period. According to a joint effect of the probabilistic transition of states and the sequence of decision making, the optimal policy is sought such that a set of decisions optimizes the long-run expected average cost (or profit) per unit time. Provision of an alternative measure for the expected long-run total discounted costs is also considered. A computer program based on the concept of the Markov Decision Process was developed and tested. The program code listing, the statement of a sample problem, and the computed results are presented.

Charng, T.↗

Space system operations and support cost analysis using Markov chains

This paper evaluates the use of Markov chain process in probabilistic life cycle cost analysis and suggests further uses of the process as a design aid tool. A methodology is developed for estimating operations and support cost and expected life for reusable space transportation systems. Application of the methodology is demonstrated for the case of a hypothetical space transportation vehicle. A sensitivity analysis is carried out to explore the effects of uncertainty in key model inputs.

Unal, Resit↗

Stochastic stability properties of jump linear systems

Jump linear systems are defined as a family of linear systems with randomly jumping parameters (usually governed by a Markov jump process) and are used to model systems subject to failures or changes in structure. The authors study stochastic stability properties in jump linear systems and the relationship among various moment and sample path stability properties. It is shown that all second moment stability properties are equivalent and are sufficient for almost sure sample path stability, and a testable necessary and sufficient condition for second moment stability is derived. The Lyapunov exponent method for the study of almost sure sample stability is discussed, and a theorem which characterizes the Lyapunov exponents of jump linear systems is presented.

Feng, Xiangbo↗

Interdecadal Variations in the Alaska Gyre

Climatic dynamic topography variations in the Alaska gyre during the period 1968-1990 are described with an objective analysis of more than 12000 STD and XBT stations, and COADS wind stress data Interannual the dynamic height and SST variations were correlated and were consistent with recently described large-scale climatic shifts in the North Pacific. The gyre was centered more to the east, circulation appeared stronger, and SST was lower during the early to mid-1970s than during the 1980s. The Aleutian low (NP and PNA indices) intensified during the interim, but the response did not appear as a gyre spinup. Instead, the associated wind stress anomalies forced a slowly varying dynamic height anomaly across the eastern and northern part of the gyre through Ekman convergence, which had the effect of displacing the gyre's low somewhat to the WSW in the 1980s. The wind curl spectrum was white, and the slow oceanic response was modeled as stochastic-forced climate variability with a simple first-order Markov autoregression process. Forcing was assumed to be Ekman pumping of the pycnocline, and the damping coefficient was estimated from the data to be approx. 1 yr. A hindcast with observed winds gave estimated dynamic height patterns similar to those observed, with a canonical correlation of 0.79 at 99% confidence. This response was weak in the western half of the gyre, where slow baroclinic variability may have been influenced by long Rossby wave propagation. A simple autoregression simulation using artificial white noise forcing shows the evolution of decadal variations similar in nature to those observed. This result, along with the low frequency correlation between dynamic height and SST, suggests that the upper-ocean climatic variability in this region is primarily wind forced.

Lagerloef, Gary S.E.↗

Exact Solutions to Time-dependent Mdps

We describe an extension of the Markov decision process model in which a continuous time dimension is included in the state space. This allows for the representation and exact solution of a wide range of problems in which transitions or rewards vary over time. We examine problems based on route planning with public transportation and telescope observation scheduling.

Boyan, Justin A.↗

Integrated Thermal Response Modeling System For Hypersonic Entry Vehicles

We describe all extension of the Markov decision process model in which a continuous time dimension is included ill the state space. This allows for the representation and exact solution of a wide range of problems in which transitions or rewards vary over time. We examine problems based on route planning with public transportation and telescope observation scheduling.

Chen, Y.-K.↗

Making the Impossible Possible: Strategies for Fast POMDP Monitoring

Systems modeled as partially observable Markov decision processes (POMDPs) can be tracked quickly with three restrictions: all actions are grouped together, the out-degree of each system state is bounded by a constant, and the number of non-zero elements in the belief state is bounded by a (different) constant. With these restrictions, the tracking algorithm operates in constant time and linear space. The first restriction assumes that the action itself is unobservable. The second restriction defines a subclass of POMDPs that covers however a wide range of problems. The third restriction is an approximation technique that can lead to a potentially vexing problem: an observation may be received that has probability according to the restricted belief state. This problem of impossibility will cause the belief state to collapse. In this paper we discuss the tradeoffs between the constant bound on the belief state and the quality of the solution. We concentrate on strategies for overcoming the impossibility problem and demonstrate initial experimental results that indicate promising directions.

Washington, Richard↗

Reinforcement Learning for Weakly-Coupled MDPs and an Application to Planetary Rover Control

Weakly-coupled Markov decision processes can be decomposed into subprocesses that interact only through a small set of bottleneck states. We study a hierarchical reinforcement learning algorithm designed to take advantage of this particular type of decomposability. To test our algorithm, we use a decision-making problem faced by autonomous planetary rovers. In this problem, a Mars rover must decide which activities to perform and when to traverse between science sites in order to make the best use of its limited resources. In our experiments, the hierarchical algorithm performs better than Q-learning in the early stages of learning, but unlike Q-learning it converges to a suboptimal policy. This suggests that it may be advantageous to use the hierarchical algorithm when training time is limited.

Bernstein, Daniel S.↗