SEARCH · Engineering Papers

Results for “Markov state models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

All sources NASA NTRS DOE OSTI

NASA NTRS1979

Local neighborhood transition probability estimation and its use in contextual classification

The problem of incorporating spatial or contextual information into classifications is considered. A simple model that describes the spatial dependencies between the neighboring pixels with a single parameter, Theta, is presented. Expressions are derived for updating the posteriori probabilities of the states of nature of the pattern under consideration using information from the neighboring patterns, both for spatially uniform context and for Markov dependencies in terms of Theta. Techniques for obtaining the optimal value of the parameter Theta as a maximum likelihood estimate from the local neighborhood of the pattern under consideration are developed.

Chittineni, C. B.↗

NASA NTRS1989

Load sharing in distributed real-time systems with state-change broadcasts

A decentralized dynamic load-sharing (LS) method based on state-change broadcasts is proposed for a distributed real-time system. Whenever the state of a node changes from underloaded to fully loaded and vice versa, the node broadcasts this change to a set of nodes, called a buddy set, in the system. The performance of the method is evaluated with both analytic modeling and simulation. It is modeled first by an embedded Markov chain for which numerical solutions are derived. The model solutions are then used to calculate the distribution of queue lengths at the nodes and the probability of meeting task deadlines. The analytical results show that buddy sets of 10 nodes outperform those of less than 10 nodes, and the incremental benefit gained from increasing the buddy set size beyond 15 nodes is insignificant. These and other analytical results are verified by simulation. The proposed LS method is shown to meet task deadlines with a very high probability.

Shin, Kang G.↗

NASA NTRS2017

A Space-Based Observational Strategy for Characterizing the First Stars and Galaxies Using the Redshifted 21cm Global Spectrum

The redshifted 21 cm monopole is expected to be a powerful probe of the epoch of the first stars and galaxies(10 less than z less than 35). The global 21 cm signal is sensitive to the thermal and ionization state of hydrogen gas and thusprovides a tracer of sources of energetic photonsprimarily hot stars and accreting black holeswhich ionize andheat the high redshift intergalactic medium (IGM). This paper presents a strategy for observations of the globalspectrum with a realizable instrument placed in a low-altitude lunar orbit, performing night-time 40120 MHzspectral observations, while on the farside to avoid terrestrial radio frequency interference, ionospheric corruption,and solar radio emissions. The frequency structure, uniformity over large scales, and unpolarized state of theredshifted 21 cm spectrum are distinct from the spectrally featureless, spatially varying, and polarized emissionfrom the bright foregrounds. This allows a clean separation between the primordial signal and foregrounds. Forsignal extraction, we model the foreground, instrument, and 21 cm spectrum with eigenmodes calculated viaSingular Value Decomposition analyses. Using a Markov Chain Monte Carlo algorithm to explore the parameterspace defined by the coefficients associated with these modes, we illustrate how the spectrum can be measured andhow astrophysical parameters (e.g., IGM properties, first star characteristics) can be constrained in the presence offoregrounds using the Dark Ages Radio Explorer (DARE).

Burns, Jack O.↗

NASA NTRS1989

Solving problems by interrogating sets of knowledge systems: Toward a theory of multiple knowledge systems

The main purpose is to develop a theory for multiple knowledge systems. A knowledge system could be a sensor or an expert system, but it must specialize in one feature. The problem is that we have an exhaustive list of possible answers to some query (such as what object is it). By collecting different feature values, in principle, it should be possible to give an answer to the query, or at least narrow down the list. Since a sensor, or for that matter an expert system, does not in most cases yield a precise value for the feature, uncertainty must be built into the model. Also, researchers must have a formal mechanism to be able to put the information together. Researchers chose to use the Dempster-Shafer approach to handle the problems mentioned above. Researchers introduce the concept of a state of recognition and point out that there is a relation between receiving updates and defining a set valued Markov Chain. Also, deciding what the value of the next set valued variable is can be phrased in terms of classical decision making theory such as minimizing the maximum regret. Other related problems are examined.

Dekorvin, Andre↗

NASA NTRS1989

Markov reliability models for digital flight control systems

The reliability of digital flight control systems can often be accurately predicted using Markov chain models. The cost of numerical solution depends on a model's size and stiffness. Acyclic Markov models, a useful special case, are particularly amenable to efficient numerical solution. Even in the general case, instantaneous coverage approximation allows the reduction of some cyclic models to more readily solvable acyclic models. After considering the solution of single-phase models, the discussion is extended to phased-mission models. Phased-mission reliability models are classified based on the state restoration behavior that occurs between mission phases. As an economical approach for the solution of such models, the mean failure rate solution method is introduced. A numerical example is used to show the influence of fault-model parameters and interphase behavior on system unreliability.

Mcgough, John↗

NASA NTRS2004

On the Stability of Jump-Linear Systems Driven by Finite-State Machines with Markovian Inputs

This paper presents two mean-square stability tests for a jump-linear system driven by a finite-state machine with a first-order Markovian input process. The first test is based on conventional Markov jump-linear theory and avoids the use of any higher-order statistics. The second test is developed directly using the higher-order statistics of the machine s output process. The two approaches are illustrated with a simple model for a recoverable computer control system.

Patilkulkarni, Sudarshan↗

NASA NTRS2012

Markov Modeling of Component Fault Growth over a Derived Domain of Feasible Output Control Effort Modifications

This paper introduces a novel Markov process formulation of stochastic fault growth modeling, in order to facilitate the development and analysis of prognostics-based control adaptation. A metric representing the relative deviation between the nominal output of a system and the net output that is actually enacted by an implemented prognostics-based control routine, will be used to define the action space of the formulated Markov process. The state space of the Markov process will be defined in terms of an abstracted metric representing the relative health remaining in each of the system s components. The proposed formulation of component fault dynamics will conveniently relate feasible system output performance modifications to predictions of future component health deterioration.

Bole, Brian↗

NASA NTRS1991

Hidden-Markov-Model Analysis Of Telemanipulator Data

Mathematical model and procedure based on hidden-Markov-model concept undergoing development for use in analysis and prediction of outputs of force and torque sensors of telerobotic manipulators. In model, overall task broken down into subgoals, and transition probabilities encode ease with which operator completes each subgoal. Process portion of model encodes task-sequence/subgoal structure, and probability-density functions for forces and torques associated with each state of manipulation encode sensor signals that one expects to observe at subgoal. Parameters of model constructed from engineering knowledge of task.

Hannaford, Blake↗

NASA NTRSDate not supplied

Energy-Optimized Path Planning for Uas in Varying Winds Via Reinforcement Learning

In this paper we propose a reinforcement learning (RL) algorithm for path planning of Unmanned Aviation Vehicles (UAVs) under varying wind conditions. Solutions to UAV path planning problems are becoming increasingly necessary as autonomous UAVs continue to enter commercial and government spaces. Path-planning is inherently challenging, as UAVs need to account for dynamically changing flying conditions such as weather, obstacle or no-fly zones, degraded vehicle health, and off-nominal battery power consumption. Machine learning methods such as reinforcement learning (RL) have the potential to revolutionize how vehicles navigate in such uncertain environments. In this study, we compute UAV trajectories from a pre-determined starting position to a target cell within a 7X7 grid environment by optimizing parameters for mission assurance and safety limits in addition to the energy consumption and operation time. The UAV navigates the grid by taking actions to move in any of the eight cardinal and inter-cardinal directions, under constant thrust profile. The resultant UAV state is sampled from a probability distribution which accounts for the UAV’s action, local wind velocity, and the presence of obstacles or boundaries. As the unmanned airspace gets more complex due to multiple vehicles and environmental uncertainties, trade-offs between energy consumption, operation time, risk tolerance, and mission assurance need to be made. Our Markov Decision Process (MDP) environment model can capture any combination of these in the optimization objective, making it novel compared to other work in the field.

trajectory planning↗

NASA NTRS1991

An error bound for instantaneous coverage

An error bound is derived for a reliability model approximation method. The approximation method is appropriate for the semi-Markov models of reconfigurable systems that are designed to achieve extremely high reliability. The semi-Markov models of these system are complex, and a significant amount of their complexity arises from the detailed descriptions of the reconfiguration processes. The reliability model approximation method consists of replacing a detailed description of a reconfiguration process with the probabilities of the possible outcomes of the reconfiguration process. These probabilities are included in the model as instantaneous jumps from the fault-occurrence state. Since little time is spent in the reconfiguration states, instantaneous jumps are a close approximation to the original model. This approximation procedure is shown to produce an overestimation for the probability of system failure, and an error bound is derived for this overestimation.

White, Allan L.↗

NASA NTRS2000

Back to the Future: Consistency-Based Trajectory Tracking

Given a model of a physical process and a sequence of commands and observations received over time, the task of an autonomous controller is to determine the likely states of the process and the actions required to move the process to a desired configuration. We introduce a representation and algorithms for incrementally generating approximate belief states for a restricted but relevant class of partially observable Markov decision processes with very large state spaces. The algorithm presented incrementally generates, rather than revises, an approximate belief state at any point by abstracting and summarizing segments of the likely trajectories of the process. This enables applications to efficiently maintain a partial belief state when it remains consistent with observations and revisit past assumptions about the process' evolution when the belief state is ruled out. The system presented has been implemented and results on examples from the domain of spacecraft control are presented.

Kurien, James↗

NASA NTRS1999

Dynamical Signatures of Living Systems

One of the main challenges in modeling living systems is to distinguish a random walk of physical origin (for instance, Brownian motions) from those of biological origin and that will constitute the starting point of the proposed approach. As conjectured, the biological random walk must be nonlinear. Indeed, any stochastic Markov process can be described by linear Fokker-Planck equation (or its discretized version), only that type of process has been observed in the inanimate world. However, all such processes always converge to a stable (ergodic or periodic) state, i.e., to the states of a lower complexity and high entropy. At the same time, the evolution of living systems directed toward a higher level of complexity if complexity is associated with a number of structural variations. The simplest way to mimic such a tendency is to incorporate a nonlinearity into the random walk; then the probability evolution will attain the features of diffusion equation: the formation and dissipation of shock waves initiated by small shallow wave disturbances. As a result, the evolution never "dies:" it produces new different configurations which are accompanied by an increase or decrease of entropy (the decrease takes place during formation of shock waves, the increase-during their dissipation). In other words, the evolution can be directed "against the second law of thermodynamics" by forming patterns outside of equilibrium in the probability space. Due to that, a specie is not locked up in a certain pattern of behavior: it still can perform a variety of motions, and only the statistics of these motions is constrained by this pattern. It should be emphasized that such a "twist" is based upon the concept of reflection, i.e., the existence of the self-image (adopted from psychology). The model consists of a generator of stochastic processes which represents the motor dynamics in the form of nonlinear random walks, and a simulator of the nonlinear version of the diffusion equation which represents the mental dynamics. It has been demonstrated that coupled mental-motor dynamics can simulate emerging self-organization, prey-predator games, collaboration and competition, "collective brain," etc.

Zak, M.↗

NASA NTRS1972

An Adaptive Technique for a Redundant-Sensor Navigation System

An on-line adaptive technique is developed to provide a self-contained redundant-sensor navigation system with a capability to utilize its full potentiality in reliability and performance. The gyro navigation system is modeled as a Gauss-Markov process, with degradation modes defined as changes in characteristics specified by parameters associated with the model. The adaptive system is formulated as a multistage stochastic process: (1) a detection system, (2) an identification system and (3) a compensation system. It is shown that the sufficient statistics for the partially observable process in the detection and identification system is the posterior measure of the state of degradation, conditioned on the measurement history.

Chien, T. T.↗

NASA NTRS1989

Reliability and performance evaluation of systems containing embedded rule-based expert systems

A method for evaluating the reliability of real-time systems containing embedded rule-based expert systems is proposed and investigated. It is a three stage technique that addresses the impact of knowledge-base uncertainties on the performance of expert systems. In the first stage, a Markov reliability model of the system is developed which identifies the key performance parameters of the expert system. In the second stage, the evaluation method is used to determine the values of the expert system's key performance parameters. The performance parameters can be evaluated directly by using a probabilistic model of uncertainties in the knowledge-base or by using sensitivity analyses. In the third and final state, the performance parameters of the expert system are combined with performance parameters for other system components and subsystems to evaluate the reliability and performance of the complete system. The evaluation method is demonstrated in the context of a simple expert system used to supervise the performances of an FDI algorithm associated with an aircraft longitudinal flight-control system.

Beaton, Robert M.↗

NASA NTRS1992

Dynamic rain fade compensation techniques for the advanced communications technology satellite

The dynamic and composite nature of propagation impairments that are incurred on earth-space communications links at frequencies in and above the 30/20 GHz Ka band necessitate the use of dynamic statistical identification and prediction processing of the fading signal in order to optimally estimate and predict the levels of each of the deleterious attenuation components. Such requirements are being met in NASA's Advanced Communications Technology Satellite (ACTS) project by the implementation of optimal processing schemes derived through the use of the ACTS Rain Attenuation Prediction Model and nonlinear Markov filtering theory. The ACTS Rain Attenuation Prediction Model discerns climatological variations on the order of 0.5 deg in latitude and longitude in the continental U.S. The time-dependent portion of the model gives precise availability predictions for the 'spot beam' links of ACTS. However, the structure of the dynamic portion of the model, which yields performance parameters such as fade duration probabilities, is isomorphic to the state-variable approach of stochastic control theory and is amenable to the design of such statistical fade processing schemes which can be made specific to the particular climatological location at which they are employed.

Manning, Robert M.↗

NASA NTRS1977

ALTKAL: An optimum linear filter for GEOS-3 altimeter data

ALTKAL is a computer program designed to smooth sea surface height data obtained from the GEOS 3 altimeter, and to produce minimum variance estimates of sea surface height and sea surface slopes, along with their standard derivations. The program operates by processing the data through a Kalman filter in both the forward and backward directions, and optimally combining the results. The sea surface height signal is considered to have a geoid signal, modeled by a third order Gauss-Markov process, corrupted by additive white noise. The governing parameters for the signal and noise processes are the signal correlation length and the signal-to-noise ratio. Mathematical derivations of the filtering and smoothing algorithms are presented. The smoother characteristics are illustrated by giving the frequency response, the data weighting sequence and the transfer function of a realistic steady-state smoother example. Based on nominal estimates for geoidal undulation amplitude and correlation length, standard deviations for the estimated sea surface height and slope are 12 cm and 3 arc seconds, respectively.

Fang, B. T.↗

NASA NTRS2008

Identifying Bearing Rotordynamic Coefficients using an Extended Kalman Filter

An Extended Kalman Filter is developed to estimate the linearized direct and indirect stiffness and damping force coefficients for bearings in rotor-dynamic applications from noisy measurements of the shaft displacement in response to imbalance and impact excitation. The bearing properties are modeled as stochastic random variables using a Gauss-Markov model. Noise terms are introduced into the system model to account for all of the estimation error, including modeling errors and uncertainties and the propagation of measurement errors into the parameter estimates. The system model contains two user-defined parameters that can be tuned to improve the filter s performance; these parameters correspond to the covariance of the system and measurement noise variables. The filter is also strongly influenced by the initial values of the states and the error covariance matrix. The filter is demonstrated using numerically simulated data for a rotor-bearing system with two identical bearings, which reduces the number of unknown linear dynamic coefficients to eight. The filter estimates for the direct damping coefficients and all four stiffness coefficients correlated well with actual values, whereas the estimates for the cross-coupled damping coefficients were the least accurate.

Miller, Bard A.↗

NASA NTRS2008

Identifying Bearing Rotordynamic Coefficients using an Extended Kalman Filter

Miller, Brad A.↗