Accelerated procedures for the solution of discret e Markov control problems
Accelerated procedures for Markov chain model optimal control problems solution with computation time advantage over usual dynamic programming
Kleinman, A. J.↗
SEARCH · Engineering Papers
Results for “Markov model”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
On the control, stability, and waiting time in a slotted ALOHA random-access system
This paper explores some of the boundaries in performance of slotted ALOHA systems by analyzing a simple and almost optimal centrally supervised control. The control results in a very simple Markov chain model and allows an examination of stability, conditional waiting time distribution of transmitting terminals, and many other system measures. The key to the simplicity is to have a probability of successful packet transmission that is independent of the number of transmitting terminals. In considering waiting time, we calculate the mean and other moments of the waiting time of a terminal when it enters the system to find (n - 1) other terminals already there competing for the channel. Under this control, the average time is proportional to n. The control requires exact knowledge of the number of terminals contending for the channel, and hence is not implementable, except as an approximation.
Ferguson, M. J.↗
Neural pulse frequency modulation of an exponentially correlated Gaussian process
The effect of NPFM (Neural Pulse Frequency Modulation) on a stationary Gaussian input, namely an exponentially correlated Gaussian input, is investigated with special emphasis on the determination of the average number of pulses in unit time, known also as the average frequency of pulse occurrence. For some classes of stationary input processes where the formulation of the appropriate multidimensional Markov diffusion model of the input-plus-NPFM system is possible, the average impulse frequency may be obtained by a generalization of the approach adopted. The results are approximate and numerical, but are in close agreement with Monte Carlo computer simulation results.
Hutchinson, C. E.↗
Analysis of a first order phase locked loop in the presence of Gaussian noise
A first-order digital phase locked loop is analyzed by application of a Markov chain model. Steady state loop error probabilities, phase standard deviation, and mean loop transient times are determined for various input signal to noise ratios. Results for direct loop simulation are presented for comparison.
Blasche, P. R.↗
Investigation of air transportation technology at Ohio University, 1980
Specific configurations of first and second order all digital phase locked loops were analyzed for both ideal and additive gaussian noise inputs. In addition, a design for a hardware digital phase locked loop capable of either first or second order operation was evaluated along with appropriate experimental data obtained from testing of the hardware loop. All parameters chosen for the analysis and the design of the digital phase locked loop were consistent with an application to an Omega navigation receiver although neither the analysis nor the design are limited to this application. For all cases tested, the experimental data showed close agreement with the analytical results indicating that the Markov chain model for first and second order digital phase locked loops are valid.
Mcfarland, R. H.↗
On the delay analysis of a TDMA channel with finite buffer capacity
The throughput performance of a TDMA channel with finite buffer capacity for transmitting data messages is considered. Each station has limited message buffer capacity and has Poisson message arrivals. Message arrivals will be blocked if the buffers are congested. Using the embedded Markov chain model, the solution procedure for the limiting system-size probabilities is presented in a recursive fashion. Numerical examples are given to demonstrate the tradeoffs between the blocking probabilities and the buffer sizing strategy.
Yan, T.-Y.↗
Reliability calculation using randomization for Markovian fault-tolerant computing systems
The randomization technique for computing transient probabilities of Markov processes is presented. The technique is applied to a Markov process model of a simplified fault tolerant computer system for illustrative purposes. It is applicable to much larger and more complex models. Transient state probabilities are computed, from which reliabilities are derived. An accelerated version of the randomization algorithm is developed which exploits ''stiffness' of the models to gain increased efficiency. A great advantage of the randomization approach is that it easily allows probabilities and reliabilities to be computed to any predetermined accuracy.
Miller, D. R.↗
Reliability analysis and fault-tolerant system development for a redundant strapdown inertial measurement unit
A methodology is developed and applied for quantitatively analyzing the reliability of a dual, fail-operational redundant strapdown inertial measurement unit (RSDIMU). A Markov evaluation model is defined in terms of the operational states of the RSDIMU to predict system reliability. A 27 state model is defined based upon a candidate redundancy management system which can detect and isolate a spectrum of failure magnitudes. The results of parametric studies are presented which show the effect on reliability of the gyro failure rate, both the gyro and accelerometer failure rates together, false alarms, probability of failure detection, probability of failure isolation, and probability of damage effects and mission time. A technique is developed and evaluated for generating dynamic thresholds for detecting and isolating failures of the dual, separated IMU. Special emphasis is given to the detection of multiple, nonconcurrent failures. Digital simulation time histories are presented which show the thresholds obtained and their effectiveness in detecting and isolating sensor failures.
Motyka, P.↗
The reliability analysis of a separated, dual fail operational redundant strapdown IMU
A methodology for quantitatively analyzing the reliability of redundant avionics systems, in general, and the dual, separated Redundant Strapdown Inertial Measurement Unit (RSDIMU), in particular, is presented. The RSDIMU is described and a candidate failure detection and isolation system presented. A Markov reliability model is employed. The operational states of the system are defined and the single-step state transition diagrams discussed. Graphical results, showing the impact of major system parameters on the reliability of the RSDIMU system, are presented and discussed.
Motyka, P.↗
Vector computer memory bank contention
A number of vector supercomputers feature very large memories. Unfortunately the large capacity memory chips that are used in these computers are much slower than the fast central processing unit (CPU) circuitry. As a result, memory bank reservation times (in CPU ticks) are much longer than on previous generations of computers. A consequence of these long reservation times is that memory bank contention is sharply increased, resulting in significantly lowered performance rates. The phenomenon of memory bank contention in vector computers is analyzed using both a Markov chain model and a Monte Carlo simulation program. The results of this analysis indicate that future generations of supercomputers must either employ much faster memory chips or else feature very large numbers of independent memory banks.
Bailey, D. H.↗
Vector computer memory bank contention
A number of vector supercomputers feature very large memories. Unfortunately the large capacity memory chips that are used in these computers are much slower than the fast central processing unit (CPU) circuitry. As a result, memory bank reservation times (in CPU ticks) are much longer than on previous generations of computers. A consequence of these long reservation times is that memory bank contention is sharply increased, resulting in significantly lowered performance rates. The phenomenon of memory bank contention in vector computers is analyzed using both a Markov chain model and a Monte Carlo simulation program. The results of this analysis indicate that future generations of supercomputers must either employ much faster memory chips or else feature very large numbers of independent memory banks.
Bailey, David H.↗
Meteorological and Environmental Inputs to Aviation Systems
Reports on aviation meteorology, most of them informal, are presented by representatives of the National Weather Service, the Bracknell (England) Meteorological Office, the NOAA Wave Propagation Lab., the Fleet Numerical Oceanography Center, and the Aircraft Owners and Pilots Association. Additional presentations are included on aircraft/lidar turbulence comparison, lightning detection and locating systems, objective detection and forecasting of clear air turbulence, comparative verification between the Generalized Exponential Markov (GEM) Model and official aviation terminal forecasts, the evaluation of the Prototype Regional Observation and Forecast System (PROFS) mesoscale weather products, and the FAA/MIT Lincoln Lab. Doppler Weather Radar Program.
Camp, Dennis W.↗
Reliability and performance evaluation of systems containing embedded rule-based expert systems
A method for evaluating the reliability of real-time systems containing embedded rule-based expert systems is proposed and investigated. It is a three stage technique that addresses the impact of knowledge-base uncertainties on the performance of expert systems. In the first stage, a Markov reliability model of the system is developed which identifies the key performance parameters of the expert system. In the second stage, the evaluation method is used to determine the values of the expert system's key performance parameters. The performance parameters can be evaluated directly by using a probabilistic model of uncertainties in the knowledge-base or by using sensitivity analyses. In the third and final state, the performance parameters of the expert system are combined with performance parameters for other system components and subsystems to evaluate the reliability and performance of the complete system. The evaluation method is demonstrated in the context of a simple expert system used to supervise the performances of an FDI algorithm associated with an aircraft longitudinal flight-control system.
Beaton, Robert M.↗
Efficient parametric analysis of performance measures for communication networks
Efficient techniques for estimating performance measures in communication networks in a steady-state or transient setting are developed. These techniques may be used in a simulation environment or in connection with real-time observations. For Markov chain models, the recently proposed standard clock approach is extended, and a class of real-time algorithms for simultaneously generating multiple sample paths under different parameter sets is presented. Attention is focused on the link crash time estimation problem, where a 'crash' is defined as the first time a buffer overflows, given some initial conditions. An algorithm for estimating crash times under various traffic shocks is derived, where all estimates are obtained in parallel to an actual network's normal operation. An algorithm for crash time estimation of a G/D/1 link model is also derived using a different (perturbation-analysis-based) approach. Finally, extensive simulation results are provided to validate the proposed algorithms and compare them to brute-force simulation.
Cassandras, Christos G.↗
Interface To The SURE Program
Abstract Semi-Markov Specification Interface to SURE Tool (ASSIST) computer program is interface program enabling reliability engineers to design large semi-Markov mathematical models accurately. Language enables efficient description of large, complicated systems. Also offers, as part of bundled package with SURE and PAWS/STEM, two other reliable analysis programs developed by Systems Validation Methods group at Langley Research Center.
Johnson, Sally C.↗
Experimental analysis of computer system dependability
This paper reviews an area which has evolved over the past 15 years: experimental analysis of computer system dependability. Methodologies and advances are discussed for three basic approaches used in the area: simulated fault injection, physical fault injection, and measurement-based analysis. The three approaches are suited, respectively, to dependability evaluation in the three phases of a system's life: design phase, prototype phase, and operational phase. Before the discussion of these phases, several statistical techniques used in the area are introduced. For each phase, a classification of research methods or study topics is outlined, followed by discussion of these methods or topics as well as representative studies. The statistical techniques introduced include the estimation of parameters and confidence intervals, probability distribution characterization, and several multivariate analysis methods. Importance sampling, a statistical technique used to accelerate Monte Carlo simulation, is also introduced. The discussion of simulated fault injection covers electrical-level, logic-level, and function-level fault injection methods as well as representative simulation environments such as FOCUS and DEPEND. The discussion of physical fault injection covers hardware, software, and radiation fault injection methods as well as several software and hybrid tools including FIAT, FERARI, HYBRID, and FINE. The discussion of measurement-based analysis covers measurement and data processing techniques, basic error characterization, dependency analysis, Markov reward modeling, software-dependability, and fault diagnosis. The discussion involves several important issues studies in the area, including fault models, fast simulation techniques, workload/failure dependency, correlated failures, and software fault tolerance.
Iyer, Ravishankar, K.↗
Eigenvalue error analysis of viscously damped structures using a Ritz reduction method
The efficient solution of the eigenvalue problem that results from inserting passive dampers with variable stiffness and damping coefficients into a structure is addressed. Eigenanalysis of reduced models obtained by retaining a number of normal modes augmented with Ritz vectors corresponding to the static solutions resulting from the load patterns introduced by the dampers has been empirically shown to yield excellent approximations to the full eigenvalue problem. An analysis of this technique in the case of a single damper is presented. A priori and a posteriori error estimates are generated and tested on numerical examples. Comparison theorems with modally truncated models and a Markov parameter matching reduced-order model are derived. These theorems corroborate the heuristic that residual flexibility methods improve low-frequency approximation of the system. The analysis leads to other techniques for eigenvalue approximation. Approximate closed-form solutions are derived that include a refinement to eigenvalue derivative methods for approximation. An efficient Newton scheme is also developed. A numerical example is presented demonstrating the effectiveness of each of these methods.
Chu, Cheng-Chih↗
Handover aspects for a Low Earth Orbit (LEO) CDMA Land Mobile Satellite (LMS) system
This paper addresses the problem of handoff in a land mobile satellite (LMS) system between adjacent satellites in a low earth orbit (LEO) constellation. In particular, emphasis is placed on the application of soft handoff in a direct sequence code division multiple access (DS-CDMA) LMS system. Soft handoff is explained in terms of terrestrial macroscopic diversity, in which signals transmitted via several independent fading paths are combined to enhance the link quality. This concept is then reconsidered in the context of a LEO LMS system. A two-state Markov channel model is used to simulate the effects of shadowing on the communications path from the mobile to each satellite during handoff. The results of the channel simulation form a platform for discussion regarding soft handoff, highlighting the potential merits of the scheme when applied in a LEO LMS environment.
Carter, P.↗