Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Markov”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
NASA NTRS1994

Hidden Markov models and neural networks for fault detection in dynamic systems

Neural networks plus hidden Markov models (HMM) can provide excellent detection and false alarm rate performance in fault detection applications, as shown in this viewgraph presentation. Modified models allow for novelty detection. Key contributions of neural network models are: (1) excellent nonparametric discrimination capability; (2) a good estimator of posterior state probabilities, even in high dimensions, and thus can be embedded within overall probabilistic model (HMM); and (3) simple to implement compared to other nonparametric models. Neural network/HMM monitoring model is currently being integrated with the new Deep Space Network (DSN) antenna controller software and will be on-line monitoring a new DSN 34-m antenna (DSS-24) by July, 1994.

Smyth, Padhraic↗
NASA NTRS1995

Reduction Of Sizes Of Semi-Markov Reliability Models

Trimming technique reduces computational effort by order of magnitude while introducing negligible error. Error bound depends on only three parameters from semi-Markov model: maximum sum of rates for failure transitions leaving any state, maximum average holding time for recovery-mode state, and operating time for system. Error bound computed before any model generated, enabling modeler to decide immediately whether or not model can be trimmed. Trimming procedure specified by precise and easy description, making it easy to include trimming procedure in program generating mathematical models for use in assessing reliability. Typical application of technique in design of digital control systems required to be extremely reliable. In addition to aerospace applications, fault-tolerant design has growing importance in wide range of industrial applications.

White, Allan L.↗
NASA NTRS1995

Hidden Markov models for fault detection in dynamic systems

The invention is a system failure monitoring method and apparatus which learns the symptom-fault mapping directly from training data. The invention first estimates the state of the system at discrete intervals in time. A feature vector x of dimension k is estimated from sets of successive windows of sensor data. A pattern recognition component then models the instantaneous estimate of the posterior class probability given the features, p(w(sub i) (vertical bar)/x), 1 less than or equal to i isless than or equal to m. Finally, a hidden Markov model is used to take advantage of temporal context and estimate class probabilities conditioned on recent past history. In this hierarchical pattern of information flow, the time series data is transformed and mapped into a categorical representation (the fault classes) and integrated over time to enable robust decision-making.

Smyth, Padhraic J.↗
NASA NTRS1997

On the Multilevel Solution Algorithm for Markov Chains

We discuss the recently introduced multilevel algorithm for the steady-state solution of Markov chains. The method is based on an aggregation principle which is well established in the literature and features a multiplicative coarse-level correction. Recursive application of the aggregation principle, which uses an operator-dependent coarsening, yields a multi-level method which has been shown experimentally to give results significantly faster than the typical methods currently in use. When cast as a multigrid-like method, the algorithm is seen to be a Galerkin-Full Approximation Scheme with a solution-dependent prolongation operator. Special properties of this prolongation lead to the cancellation of the computationally intensive terms of the coarse-level equations.

Horton, Graham↗
NASA NTRS1997

Complexity of Kronecker Operations on Sparse Matrices with Applications to the Solution of Markov Models

We present a systematic discussion of algorithms to multiply a vector by a matrix expressed as the Kronecker product of sparse matrices, extending previous work in a unified notational framework. Then, we use our results to define new algorithms for the solution of large structured Markov models. In addition to a comprehensive overview of existing approaches, we give new results with respect to: (1) managing certain types of state-dependent behavior without incurring extra cost; (2) supporting both Jacobi-style and Gauss-Seidel-style methods by appropriate multiplication algorithms; (3) speeding up algorithms that consider probability vectors of size equal to the "actual" state space instead of the "potential" state space.

Buchholz, Peter↗
NASA NTRS2003

Symbolic Heuristic Search for Factored Markov Decision Processes

We describe a planning algorithm that integrates two approaches to solving Markov decision processes with large state spaces. State abstraction is used to avoid evaluating states individually. Forward search from a start state, guided by an admissible heuristic, is used to avoid evaluating all states. We combine these two approaches in a novel way that exploits symbolic model-checking techniques and demonstrates their usefulness for decision-theoretic planning.

Morris, Robert↗
NASA NTRS2003

Transition-Independent Decentralized Markov Decision Processes

There has been substantial progress with formal models for sequential decision making by individual agents using the Markov decision process (MDP). However, similar treatment of multi-agent systems is lacking. A recent complexity result, showing that solving decentralized MDPs is NEXP-hard, provides a partial explanation. To overcome this complexity barrier, we identify a general class of transition-independent decentralized MDPs that is widely applicable. The class consists of independent collaborating agents that are tied up by a global reward function that depends on both of their histories. We present a novel algorithm for solving this class of problems and examine its properties. The result is the first effective technique to solve optimally a class of decentralized MDPs. This lays the foundation for further work in this area on both exact and approximate solutions.

Becker, Raphen↗
NASA NTRS2003

Hybrid Discrete-Continuous Markov Decision Processes

This paper proposes a Markov decision process (MDP) model that features both discrete and continuous state variables. We extend previous work by Boyan and Littman on the mono-dimensional time-dependent MDP to multiple dimensions. We present the principle of lazy discretization, and piecewise constant and linear approximations of the model. Having to deal with several continuous dimensions raises several new problems that require new solutions. In the (piecewise) linear case, we use techniques from partially- observable MDPs (POMDPS) to represent value functions as sets of linear functions attached to different partitions of the state space.

Feng, Zhengzhu↗
NASA NTRS2004

Dynamic Programming for Structured Continuous Markov Decision Problems

We describe an approach for exploiting structure in Markov Decision Processes with continuous state variables. At each step of the dynamic programming, the state space is dynamically partitioned into regions where the value function is the same throughout the region. We first describe the algorithm for piecewise constant representations. We then extend it to piecewise linear representations, using techniques from POMDPs to represent and reason about linear surfaces efficiently. We show that for complex, structured problems, our approach exploits the natural structure so that optimal solutions can be computed efficiently.

Dearden, Richard↗
NASA NTRS2004

Markov Chain Monte Carlo Joint Analysis of Chandra X-Ray Imaging Spectroscopy and Sunyaev-Zel'dovich Effect Data

X-ray and Sunyaev-Zel'dovich effect data can be combined to determine the distance to galaxy clusters. High-resolution X-ray data are now available from Chandra, which provides both spatial and spectral information, and Sunyaev-Zel'dovich effect data were obtained from the BIMA and Owens Valley Radio Observatory (OVRO) arrays. We introduce a Markov Chain Monte Carlo procedure for the joint analysis of X-ray and Sunyaev- Zel'dovich effect data. The advantages of this method are the high computational efficiency and the ability to measure simultaneously the probability distribution of all parameters of interest, such as the spatial and spectral properties of the cluster gas and also for derivative quantities such as the distance to the cluster. We demonstrate this technique by applying it to the Chandra X-ray data and the OVRO radio data for the galaxy cluster A611. Comparisons with traditional likelihood ratio methods reveal the robustness of the method. This method will be used in follow-up paper to determine the distances to a large sample of galaxy cluster.

Bonamente, Massimillano↗
NASA NTRS2004

Markov Jump-Linear Performance Models for Recoverable Flight Control Computers

Single event upsets in digital flight control hardware induced by atmospheric neutrons can reduce system performance and possibly introduce a safety hazard. One method currently under investigation to help mitigate the effects of these upsets is NASA Langley s Recoverable Computer System. In this paper, a Markov jump-linear model is developed for a recoverable flight control system, which will be validated using data from future experiments with simulated and real neutron environments. The method of tracking error analysis and the plan for the experiments are also described.

Zhang, Hong↗
NASA NTRS2006

Towards a Theory of Sampled-Data Piecewise-Deterministic Markov Processes

The analysis and design of practical control systems requires that stochastic models be employed. Analysis and design tools have been developed, for example, for Markovian jump linear continuous and discrete-time systems, piecewise-deterministic processes (PDP's), and general stochastic hybrid systems (GSHS's). These model classes have been used in many applications, including fault tolerant control and networked control systems. This paper presents initial results on the analysis of a sampled-data PDP representation of a nonlinear sampled-data system with a jump linear controller. In particular, it is shown that the state of the sampled-data PDP satisfies the strong Markov property. In addition, a relation between the invariant measures of a sampled-data system driven by a stochastic process and its associated discrete-time representation are presented. As an application, when the plant is linear with no external input, a sufficient testable condition for the convergence in distribution to the invariant delta Dirac measure is given.

Herencia-Zapana, Heber↗
NASA NTRS2008

A Stable Clock Error Model Using Coupled First and Second Order Gauss-Markov Processes

Long data outages may occur in applications of global navigation satellite system technology to orbit determination for missions that spend significant fractions of their orbits above the navigation satellite constellation(s). Current clock error models based on the random walk idealization may not be suitable in these circumstances, since the covariance of the clock errors may become large enough to overflow flight computer arithmetic. A model that is stable, but which approximates the existing models over short time horizons is desirable. A coupled first- and second-order Gauss-Markov process is such a model.

Carpenter, Russell↗
NASA NTRS2011

Markov Chain Monte Carlo Bayesian Learning for Neural Networks

Conventional training methods for neural networks involve starting al a random location in the solution space of the network weights, navigating an error hyper surface to reach a minimum, and sometime stochastic based techniques (e.g., genetic algorithms) to avoid entrapment in a local minimum. It is further typically necessary to preprocess the data (e.g., normalization) to keep the training algorithm on course. Conversely, Bayesian based learning is an epistemological approach concerned with formally updating the plausibility of competing candidate hypotheses thereby obtaining a posterior distribution for the network weights conditioned on the available data and a prior distribution. In this paper, we developed a powerful methodology for estimating the full residual uncertainty in network weights and therefore network predictions by using a modified Jeffery's prior combined with a Metropolis Markov Chain Monte Carlo method.

Goodrich, Michael S.↗
NASA NTRS2012

Using Markov Models of Fault Growth Physics and Environmental Stresses to Optimize Control Actions

A generalized Markov chain representation of fault dynamics is presented for the case that available modeling of fault growth physics and future environmental stresses can be represented by two independent stochastic process models. A contrived but representatively challenging example will be presented and analyzed, in which uncertainty in the modeling of fault growth physics is represented by a uniformly distributed dice throwing process, and a discrete random walk is used to represent uncertain modeling of future exogenous loading demands to be placed on the system. A finite horizon dynamic programming algorithm is used to solve for an optimal control policy over a finite time window for the case that stochastic models representing physics of failure and future environmental stresses are known, and the states of both stochastic processes are observable by implemented control routines. The fundamental limitations of optimization performed in the presence of uncertain modeling information are examined by comparing the outcomes obtained from simulations of an optimizing control policy with the outcomes that would be achievable if all modeling uncertainties were removed from the system.

Bole, Brian↗
NASA NTRS2018

A Markov Decision Process Framework for Optimal Airport Reconfiguration

The airport runway configuration is defined as a combination set of runways for arrivals and departures used at a point during operation of the airport. An optimal configuration of these runways depends on a number of factors, including traffic demand, wind magnitude and direction, other adverse weather conditions, and noise restrictions, among others. Based on the current state of these factors and predictions of traffic demand and weather conditions, runway configuration changes are made and coordinated between tower controller, other air traffic control facilities, pilots, and ground personnel. Reconfigurations can be quite disruptive to airport operations; minimizing their frequency and scheduling them well in advance is essential for mitigating some of the added workload for controllers and pilots. Unfortunately, deciding on an appropriate time to change is challenging for human decision makers. Not only do multiple factors need to be evaluated, but the uncertainty in their forecasts must also be considered. Previous optimization methods, such as mixed linear integer programming, have been proposed. Although these methods can reason over a large set of variables, they do not systematically handle the uncertainty associated with weather movement, traffic demands, and other variables. In this work, we introduce a Markov Decision Process (MDP)-based decision making framework which can reason effectively over the inherent uncertainties and make optimal decisions on if/when to change the airport configuration. In a prototype implementation, we present a single runway with three aircraft and utilize knowledge of the forecasted wind speed and direction to determine whether to keep or change the current runway configuration. Our aim through this work is to present a framework for airport reconfiguration which can be scalable to additional aircraft, multiple runways, and various input parameters. This technique will optimize the airport reconfiguration procedure by providing a proactive approach, optimizing not just at the next optimal opportunity for a reconfiguration based on varying atmospheric and traffic conditions in the terminal airspace, but also anticipating future necessary reconfigurations. This will eliminate the inefficiencies of frequent changes currently associated with runway reconfiguration procedures.

runway reconfiguration↗
NASA NTRS2019

Machine Learning, Markov Chain Monte Carlo, and Optimal Algorithms to Characterize the AdvACT Kilopixel Transition-Edge Sensor Arrays

Next-generation focal planes comprising dozens of kilopixel transition-edge sensor (TES) arrays require new methods to rapidly screen candidate arrays, evaluate array non-idealities in the field, identify outlier devices for removal, and optimize the array performance in the field. We demonstrate robust methods to estimate TES parameters (critical temperatures and thermal conductivity parameters) and their uncertainties using a custom Markov Chain Monte Carlo (MCMC) algorithm. We also constrain systematic effects in estimating the TES parameters from non-isothermal current-voltage curves (IVs) at approximately a ~3% level. Additionally, for the first time, we have applied Machine Learning (ML) algorithms to tune detector arrays and optimize their performance.

Maria Salatino↗
NASA NTRS2019

Search for Gravitational Waves From Scorpius X-1 in the Second Advanced LIGO Observing Run With an Improved Hidden Markov Model

We present results from a semi coherent search for continuous gravitational waves from the low-mass x-ray binary Scorpius X-1, using a hidden Markov model (HMM) to track spin wandering. This search improves on previous HMM-based searches of LIGO data by using an improved frequency domain matched filter, the J-statistic, and by analyzing data from Advanced LIGO’s second observing run. In the frequency range searched, from 60 to 650 Hz, we find no evidence of gravitational radiation. At 194.6 Hz, the most sensitive search frequency, we report an upper limit on gravitational wave strain (at 95% confidence) ofh95%0¼3.47×10−25when marginalizing over source inclination angle. This is the most sensitive search for Scorpius X-1, to date, that is specifically designed to be robust in the presence of spin wandering.

B P Abbott↗