Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Non-Markovianity”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

13 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2002

The Design of Collectives of Agents to Control Non-Markovian Systems

The 'Collective Intelligence' (COIN) framework concerns the design of collectives of reinforcement-learning agents such that their interaction causes a provided 'world' utility function concerning the entire collective to be maximized. Previously, we applied that framework to scenarios involving Markovian dynamics where no re-evolution of the system from counter-factual initial conditions (an often expensive calculation) is permitted. This approach sets the individual utility function of each agent to be both aligned with the world utility, and at the same time, easy for the associated agents to optimize. Here we extend that approach to systems involving non-Markovian dynamics. In computer simulations, we compare our techniques with each other and with conventional-'team games'. We show whereas in team games performance often degrades badly with time, it steadily improves when our techniques are used. We also investigate situations where the system's dimensionality is effectively reduced. We show that this leads to difficulties in the agents' ability to learn. The implication is that 'learning' is a property only of high-enough dimensional systems.

Lawson, John W.↗
NASA NTRS2004

The Design of Collectives of Agents to Control Non-Markovian Systems

The Collective Intelligence (COIN) framework concerns the design of collectives of reinforcement-learning agents such that their interaction causes a provided "world" utility function concerning the entire collective to be maximized. Previously, we applied that framework to scenarios involving Markovian dynamics where no re-evolution of the system from counter-factual initial conditions (an often expensive calculation) is permitted. This approach sets the individual utility function of each agent to be both aligned with the world utility, and at the same time, easy for the associated agents to optimize. Here we extend that approach to systems involving non-Markovian dynamics. In computer simulations, we compare our techniques with each other and with conventional "team games". We show whereas in team games performance often degrades badly with time, it steadily improves when our techniques are used. We also investigate situations where the system's dimensionality is effectively reduced. We show that this leads to difficulties in the agents ability to learn. The implication is that learning is a property only of high-enough dimensional systems.

Lawson, John W.↗
NASA NTRS2019

Simulation of Non-Markovian Dynamics on IBM QX

Currently available quantum computers allow us to run proof of principle algorithms that are unitary in their nature. Therefore, this architecture is unoptimized for simulation of an open quantum system. Here we present a method that helps us to overcome unitarity. We show how to run a non-Markovian evolution of a qubit system. We discuss all the discrepancies from theoretical predictions.

Wudarski, Filip A.↗
NASA NTRS1989

A non-Markovian model of avalanche gain statistics for a solid-state photomultiplier

A solid-state photomultiplier (SSPM) capable of continously detecting individual photons of wavelength between 0.4 and 28 microns has recently been disclosed (Petroff et al., 1987). The initial response of the SSPM to single photon is a fast, high-amplitude current pulse of between 10,000 and 100,000 electrons. A phenomenological model of the SSPM avalanche process is presented which successfully predicts the shape of the observed pulse-amplitude distribution by including small history-dependent effects on the carrier transport. The model clarifies the consequences of the electric field strength and the scattering of the electrons for the development of the avalanche in the SSPM.

Laviolette, Randall A.↗
NASA NTRS1976

A strictly Markovian expansion for plasma turbulence theory

The collision operator that appears in the equation of motion for a particle distribution function that was averaged over an ensemble of random Hamiltonians is non-Markovian. It is non-Markovian in that it involves a propagated integral over the past history of the ensemble averaged distribution function. All formal expansions of this nonlinear collision operator to date preserve this non-Markovian character term by term yielding an integro-differential equation that must be converted to a diffusion equation by an additional approximation. An expansion is derived for the collision operator that is strictly Markovian to any finite order and yields a diffusion equation as the lowest nontrivial order. The validity of this expansion is seen to be the same as that of the standard quasilinear expansion.

Jones, F. C.↗
NASA NTRS1978

A strictly Markovian expansion for plasma turbulence theory

The collision operator that appears in the equation of motion for a particle distribution function that has been averaged over an ensemble of random Hamiltonians is non-Markovian. It is non-Markovian in that it involves a propagated integral over the past history of the ensemble averaged distribution function. All formal expansions of this nonlinear collision operator to date preserve this non-Markovian character term by term yielding an integro-differential equation that must be converted to a diffusion equation by an additional approximation. In this note we derive an expansion of the collision operator that is strictly Markovian to any finite order and yields a diffusion equation as the lowest non-trivial order. The validity of this expansion is seen to be the same as that of the standard quasi-linear expansion.

Jones, F. C.↗
NASA NTRS2000

Analysis of Phase-Type Stochastic Petri Nets With Discrete and Continuous Timing

The Petri net formalism is useful in studying many discrete-state, discrete-event systems exhibiting concurrency, synchronization, and other complex behavior. As a bipartite graph, the net can conveniently capture salient aspects of the system. As a mathematical tool, the net can specify an analyzable state space. Indeed, one can reason about certain qualitative properties (from state occupancies) and how they arise (the sequence of events leading there). By introducing deterministic or random delays, the model is forced to sojourn in states some amount of time, giving rise to an underlying stochastic process, one that can be specified in a compact way and capable of providing quantitative, probabilistic measures. We formalize a new non-Markovian extension to the Petri net that captures both discrete and continuous timing in the same model. The approach affords efficient, stationary analysis in most cases and efficient transient analysis under certain restrictions. Moreover, this new formalism has the added benefit in modeling fidelity stemming from the simultaneous capture of discrete- and continuous-time events (as opposed to capturing only one and approximating the other). We show how the underlying stochastic process, which is non-Markovian, can be resolved into simpler Markovian problems that enjoy efficient solutions. Solution algorithms are provided that can be easily programmed.

Jones, Robert L.↗
NASA NTRS1973

Invalidity of standard perturbation techniques in cosmic ray transport theory

It is contended that the existence of particles with arbitrarily long correlation times invalidates the condition necessary for the applicability of standard perturbation techniques in cosmic ray transport theory. It is also argued that Klimas and Sandri's (1971) conclusion about a non-Markovian time development of the particle distribution is unwarranted.

Kaiser, T. B.↗
NASA NTRS1979

Towards fault-tolerant optimal control

The paper considers the design of fault-tolerant controllers that may endow systems with dynamic reliability. Results for jump linear quadratic Gaussian control problems are extended to include random jump costs, trajectory discontinuities, and a simple case of non-Markovian mode transitions.

Chizeck, H. J.↗
NASA NTRS1993

Simulation modeling for long duration spacecraft control systems

The use of simulation is described and it is contrasted to analytical solution techniques for evaluation of analytical reliability models. The role importance sampling plays in simulation of models of this type was also discussed. The simulator tool used for our analysis is described. Finally, the use of the simulator tool was demonstrated by applying it to evaluate the reliability of a fault tolerant hypercube multiprocessor intended for spacecraft designed for long duration missions. The reliability analysis was used to highlight the advantages and disadvantages offered by simulation over analytical solution of Markovian and non-Markovian reliability models.

Boyd, Mark A.↗
NASA NTRS1998

A Novel Solution-Technique Applied to a Novel WAAS Architecture

The Federal Aviation Administration has embarked on an historic task of modernizing and significantly improving the national air transportation system. One system that uses the Global Positioning System (GPS) to determine aircraft navigational information is called the Wide Area Augmentation System (WAAS). This paper describes a reliability assessment of one candidate system architecture for the WAAS. A unique aspect of this study regards the modeling and solution of a candidate system that allows a novel cold sparing scheme. The cold spare is a WAAS communications satellite that is fabricated and launched after a predetermined number of orbiting satellite failures have occurred and after some stochastic fabrication time transpires. Because these satellites are complex systems with redundant components, they exhibit an increasing failure rate with a Weibull time to failure distribution. Moreover, the cold spare satellite build-time is Weibull and upon launch is considered to be a good-as-new system with an increasing failure rate and a Weibull time to failure distribution as well. The reliability model for this system is non-Markovian because three distinct system clocks are required: the time to failure of the orbiting satellites, the build time for the cold spare, and the time to failure for the launched spare satellite. A powerful dynamic fault tree modeling notation and Monte Carlo simulation technique with importance sampling are shown to arrive at a reliability prediction for a 10 year mission.

Bavuso, J.↗
NASA NTRS2013

Modeling of Particle Acceleration at Multiple Shocks Via Diffusive Shock Acceleration: Preliminary Results

We present preliminary results from a model that diffusively accelerates particles at multiple shocks. Our basic approach is related to box models (Protheroe and Stanev, 1998; Moraal and Axford, 1983; Ball and Kirk, 1992; Drury et al., 1999) in which a distribution of particles is diffusively accelerated inside the box while simultaneously experiencing decompression through adiabatic expansion and losses from the convection and diffusion of particles outside the box (Melrose and Pope, 1993; Zank et al., 2000). We adiabatically decompress the accelerated particle distribution between each shock by either the method explored in Melrose and Pope (1993) and Pope and Melrose (1994) or by the approach set forth in Zank et al. (2000) where we solve the transport equation by a method analogous to operator splitting. The second method incorporates the additional loss terms of convection and diffusion and allows for the use of a variable time between shocks. We use a maximum injection energy (Emax) appropriate for quasi-parallel and quasi-perpendicular shocks (Zank et al., 2000, 2006; Dosch and Shalchi, 2010) and provide a preliminary application of the diffusive acceleration of particles by multiple shocks with frequencies appropriate for solar maximum (i.e., a non-Markovian process).

Parker, Linda Neergaard↗
NASA NTRS2013

Modeling of Particle Acceleration at Multiple Shocks via Diffusive Shock Acceleration: Preliminary Results

Successful forecasting of energetic particle events in space weather models require algorithms for correctly predicting the spectrum of ions accelerated from a background population of charged particles. We present preliminary results from a model that diffusively accelerates particles at multiple shocks. Our basic approach is related to box models in which a distribution of particles is diffusively accelerated inside the box while simultaneously experiencing decompression through adiabatic expansion and losses from the convection and diffusion of particles outside the box. We adiabatically decompress the accelerated particle distribution between each shock by either the method explored in Melrose and Pope (1993) and Pope and Melrose (1994) or by the approach set forth in Zank et al. (2000) where we solve the transport equation by a method analogous to operator splitting. The second method incorporates the additional loss terms of convection and diffusion and allows for the use of a variable time between shocks. We use a maximum injection energy (E(sub max)) appropriate for quasi-parallel and quasi-perpendicular shocks and provide a preliminary application of the diffusive acceleration of particles by multiple shocks with frequencies appropriate for solar maximum (i.e., a non-Markovian process).

Parker, L. Neergaard↗