Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Markov state models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Quantum Markov chain Monte Carlo with digital dissipative dynamics on quantum computers

Modeling the dynamics of a quantum system connected to the environment is critical for advancing our understanding of complex quantum processes, as most quantum processes in nature are affected by an environment. Modeling a macroscopic environment on a quantum simulator may be achieved by coupling independent ancilla qubits that facilitate energy exchange in an appropriate manner with the system and mimic an environment. This approach requires a large, and possibly exponential number of ancillary degrees of freedom which is impractical. In contrast, we develop a digital quantum algorithm that simulates interaction with an environment using a small number of ancilla qubits. By combining periodic modulation of the ancilla energies, or spectral combing, with periodic reset operations, we are able to mimic interaction with a large environment and generate thermal states of interacting many-body systems. We evaluate the algorithm by simulating preparation of thermal states of the transverse Ising model. Our algorithm can also be viewed as a quantum Markov chain Monte Carlo process that allows sampling of the Gibbs distribution of a multivariate model. To illustrate this we evaluate the accuracy of sampling Gibbs distributions of simple probabilistic graphical models using the algorithm.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Bayesian Exploration of Phenomenological EoS of Neutron/Hybrid Stars with Recent Observations

The description of the stellar interior of compact stars remains as a big challenge for the nuclear astrophysics community. The consolidated knowledge is restricted to density regions around the saturation of hadronic matter ρ 0 =2.8 × 10 14 g cm -3 , regimes where our nuclear models are successfully applied. As one moves towards higher densities and extreme conditions up to the quark/gluons deconfinement, little can be said about the microphysics of the equation of state (EoS). Here, we employ a Markov Chain Monte Carlo (MCMC) strategy to access the variability at high density regions of polytropic piecewise models for neutron star (NS) EoS or possible hybrid stars, i.e., a NS with a small quark-matter core. With a fixed description of the hadronic matter for low density, below the nuclear saturation density, we explore a variety of models for the high density regimes leading to stellar masses near to 2.5 M ⊙ , in accordance with the observations of massive pulsars. The models are constrained, including the observation of the merger of neutrons stars from VIRGO-LIGO and with the pulsar observed by NICER. In addition, we also discuss the possibility of the use of a Bayesian power regression model with heteroscedastic error. The set of EoS from the Laser Interferometer Gravitational-Wave Observatory (LIGO) was used as input and treated as the data set for the testing case.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

CHMMPY: A python package for constrained Hidden Markov Models

SAND2025-11909O chmmpy software analyzes multivariate timeseries data to detect patterns. It uses a Hidden Markov Model (HMM) and application-specific constraints that reflect known relationships among hidden states to accomplish this. The chmmpy software provides a generic framework for expressing application-specific constraints and supporting constrained HMM inference using optimization solvers. chmmpy is available on GitHub. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Hart, William↗
NASA NTRS1992

Identification of linear system models and state estimators for controls

The following paper is presented in viewgraph format and covers topics including: (1) linear state feedback control system; (2) Kalman filter state estimation; (3) relation between residual and stochastic part of output; (4) obtaining Kalman filter gain; (5) state estimation under unknown system model and unknown noises; and (6) relationship between filter Markov parameters and system Markov parameters.

Chen, Chung-Wen↗
NASA NTRS2000

Exact Solutions to Time-dependent Mdps

We describe an extension of the Markov decision process model in which a continuous time dimension is included in the state space. This allows for the representation and exact solution of a wide range of problems in which transitions or rewards vary over time. We examine problems based on route planning with public transportation and telescope observation scheduling.

Boyan, Justin A.↗
NASA NTRS2000

Integrated Thermal Response Modeling System For Hypersonic Entry Vehicles

We describe all extension of the Markov decision process model in which a continuous time dimension is included ill the state space. This allows for the representation and exact solution of a wide range of problems in which transitions or rewards vary over time. We examine problems based on route planning with public transportation and telescope observation scheduling.

Chen, Y.-K.↗
NASA NTRS2003

Closed-Loop Performance Measures for Flight Controllers Subject to Neutron-Induced Upsets

It has been observed that atmospheric neutrons can produce single event upsets in digital flight control hardware. The phenomenon has been studied extensively at the chip level, and now system level experiments are underway. In this paper analytical closed-loop performance measures for the tracking error are developed for a plant that is stabilized by a recoverable computer system subject to neutron induced upsets. The underlying model is a Markov jump-linear system with process noise. The steady-state tracking error is expressed in terms of a generalized observability Gramian.

Gray, W. Steven↗
NASA NTRS1992

Linear system identification via backward-time observer models

Presented here is an algorithm to compute the Markov parameters of a backward-time observer for a backward-time model from experimental input and output data. The backward-time observer Markov parameters are decomposed to obtain the backward-time system Markov parameters (backward-time pulse response samples) for the backward-time system identification. The identified backward-time system Markov parameters are used in the Eigensystem Realization Algorithm to identify a backward-time state-space model, which can be easily converted to the usual forward-time representation. If one reverses time in the model to be identified, what were damped true system modes become modes with negative damping, growing as the reversed time increases. On the other hand, the noise modes in the identification still maintain the property that they are stable. The shift from positive damping to negative damping of the true system modes allows one to distinguish these modes from noise modes. Experimental results are given to illustrate when and to what extent this concept works.

Juang, Jer-Nan↗
NASA NTRS2005

Monte Carlo Simulation of Markov, Semi-Markov, and Generalized Semi- Markov Processes in Probabilistic Risk Assessment

A standard tool of reliability analysis used at NASA-JSC is the event tree. An event tree is simply a probability tree, with the probabilities determining the next step through the tree specified at each node. The nodal probabilities are determined by a reliability study of the physical system at work for a particular node. The reliability study performed at a node is typically referred to as a fault tree analysis, with the potential of a fault tree existing.for each node on the event tree. When examining an event tree it is obvious why the event tree/fault tree approach has been adopted. Typical event trees are quite complex in nature, and the event tree/fault tree approach provides a systematic and organized approach to reliability analysis. The purpose of this study was two fold. Firstly, we wanted to explore the possibility that a semi-Markov process can create dependencies between sojourn times (the times it takes to transition from one state to the next) that can decrease the uncertainty when estimating time to failures. Using a generalized semi-Markov model, we studied a four element reliability model and were able to demonstrate such sojourn time dependencies. Secondly, we wanted to study the use of semi-Markov processes to introduce a time variable into the event tree diagrams that are commonly developed in PRA (Probabilistic Risk Assessment) analyses. Event tree end states which change with time are more representative of failure scenarios than are the usual static probability-derived end states.

English, Thomas↗
NASA NTRS2003

Control Law Design in a Computational Aeroelasticity Environment

A methodology for designing active control laws in a computational aeroelasticity environment is given. The methodology involves employing a systems identification technique to develop an explicit state-space model for control law design from the output of a computational aeroelasticity code. The particular computational aeroelasticity code employed in this paper solves the transonic small disturbance aerodynamic equation using a time-accurate, finite-difference scheme. Linear structural dynamics equations are integrated simultaneously with the computational fluid dynamics equations to determine the time responses of the structure. These structural responses are employed as the input to a modern systems identification technique that determines the Markov parameters of an "equivalent linear system". The Eigensystem Realization Algorithm is then employed to develop an explicit state-space model of the equivalent linear system. The Linear Quadratic Guassian control law design technique is employed to design a control law. The computational aeroelasticity code is modified to accept control laws and perform closed-loop simulations. Flutter control of a rectangular wing model is chosen to demonstrate the methodology. Various cases are used to illustrate the usefulness of the methodology as the nonlinearity of the aeroelastic system is increased through increased angle-of-attack changes.

Newsom, Jerry R.↗
NASA NTRS1979

Analysis and design of a second-order digital phase-locked loop

A specific second-order digital phase-locked loop (DPLL) was modeled as a first-order Markov chain with alternatives. From the matrix of transition probabilities of the Markov chain, the steady-state phase error of the DPLL was determined. In a similar manner the loop's response was calculated for a fading input. Additionally, a hardware DPLL was constructed and tested to provide a comparison to the results obtained from the Markov chain model. In all cases tested, good agreement was found between the theoretical predictions and the experimental data.

Blasche, P. R.↗
DOE OSTI2024

Decision-making based on Markov decision process in integrated artificial reasoning framework—Part I: Theory

This paper presents a decision-making framework based on an integrated artificial reasoning framework and Markov decision process (MDP). The integrated artificial reasoning framework provides a physics-based approach that converts system information into state transition models, and the analysis result will be represented by the transition probabilities that can be used with an MDP to find a traceable and explainable optimal pathway. A dynamic Bayesian network (DBN) is well suited for representing the structure of an MDP. The causality information among process variables (or among subsystems) is mathematically represented in a DBN by the conditional probabilities of the node’s states provided different probabilities of the parent node’s states. To define node states in a physically understandable manner, we used multilevel flow modeling (MFM). An MFM follows the fundamental energy and mass conservation laws and supports the selection of process variables that represent the system of interest so that causal relations among process variables are properly captured. An MFM-based DBN supports developing state transition models in an MDP to capture the effect of process variables of system having physical relations. The operators of the target system can capture stochastic system dynamics as multiple subsystem state transitions based on their physical relations and uncertainties coming from component degradation or random failures. We analyzed a simplified exemplary system to illustrate an optimal operational policy using the suggested approach.

Markov decision process↗
NASA NTRS2011

Orion Absolute Navigation System Progress and Challenges

The Orion spacecraft is being designed as NASA's next-generation exploration vehicle for crewed missions beyond Low-Earth Orbit. The navigation system for the Orion spacecraft is being designed in a Multi-Organizational Design Environment (MODE) team including contractor and NASA personnel. The system uses an Extended Kalman Filter to process measurements and determine the state. The design of the navigation system has undergone several iterations and modifications since its inception, and continues as a work-in-progress. This paper seeks to benchmark the current state of the design and some of the rationale and analysis behind it. There are specific challenges to address when preparing a timely and effective design for the Exploration Flight Test (EFT-1), while still looking ahead and providing software extensibility for future exploration missions. The primary measurements in a Near-Earth or Mid-Earth environment consist of GPS pseudorange and deltarange, but for future explorations missions the use of star-tracker and optical navigation sources need to be considered. Discussions are presented for state size and composition, processing techniques, and consider states. A presentation is given for the processing technique using the computationally stable and robust UDU formulation with an Agee-Turner Rank-One update. This allows for computational savings when dealing with many parameters which are modeled as slowly varying Gauss-Markov processes. Preliminary analysis shows up to a 50% reduction in computation versus a more traditional formulation. Several state elements are discussed and evaluated, including position, velocity, attitude, clock bias/drift, and GPS measurement biases in addition to bias, scale factor, misalignment, and non-orthogonalities of the accelerometers and gyroscopes. Another consideration is the initialization of the EKF in various scenarios. Scenarios such as single-event upset, ground command, pad alignment, cold start are discussed as are strategies for whole and partial state updates as well as covariance considerations. Strategies are given for dealing with latent measurements and high-rate propagation using multi-rate architecture. The details of the rate groups and the data ow between the elements is discussed and evaluated.

Holt, Greg N.↗
NASA NTRS2012

Orion Absolute Navigation System Progress and Challenge

The absolute navigation design of NASA's Orion vehicle is described. It has undergone several iterations and modifications since its inception, and continues as a work-in-progress. This paper seeks to benchmark the current state of the design and some of the rationale and analysis behind it. There are specific challenges to address when preparing a timely and effective design for the Exploration Flight Test (EFT-1), while still looking ahead and providing software extensibility for future exploration missions. The primary onboard measurements in a Near-Earth or Mid-Earth environment consist of GPS pseudo-range and delta-range, but for future explorations missions the use of star-tracker and optical navigation sources need to be considered. Discussions are presented for state size and composition, processing techniques, and consider states. A presentation is given for the processing technique using the computationally stable and robust UDU formulation with an Agee-Turner Rank-One update. This allows for computational savings when dealing with many parameters which are modeled as slowly varying Gauss-Markov processes. Preliminary analysis shows up to a 50% reduction in computation versus a more traditional formulation. Several state elements are discussed and evaluated, including position, velocity, attitude, clock bias/drift, and GPS measurement biases in addition to bias, scale factor, misalignment, and non-orthogonalities of the accelerometers and gyroscopes. Another consideration is the initialization of the EKF in various scenarios. Scenarios such as single-event upset, ground command, and cold start are discussed as are strategies for whole and partial state updates as well as covariance considerations. Strategies are given for dealing with latent measurements and high-rate propagation using multi-rate architecture. The details of the rate groups and the data ow between the elements is discussed and evaluated.

Holt, Greg N.↗
NASA NTRS1995

Hidden Markov models for fault detection in dynamic systems

The invention is a system failure monitoring method and apparatus which learns the symptom-fault mapping directly from training data. The invention first estimates the state of the system at discrete intervals in time. A feature vector x of dimension k is estimated from sets of successive windows of sensor data. A pattern recognition component then models the instantaneous estimate of the posterior class probability given the features, p(w(sub i) (vertical bar)/x), 1 less than or equal to i isless than or equal to m. Finally, a hidden Markov model is used to take advantage of temporal context and estimate class probabilities conditioned on recent past history. In this hierarchical pattern of information flow, the time series data is transformed and mapped into a categorical representation (the fault classes) and integrated over time to enable robust decision-making.

Smyth, Padhraic J.↗
NASA NTRS1993

Hidden Markov models for fault detection in dynamic systems

The invention is a system failure monitoring method and apparatus which learns the symptom-fault mapping directly from training data. The invention first estimates the state of the system at discrete intervals in time. A feature vector x of dimension k is estimated from sets of successive windows of sensor data. A pattern recognition component then models the instantaneous estimate of the posterior class probability given the features, p(w(sub i) perpendicular to x), 1 less than or equal to i is less than or equal to m. Finally, a hidden Markov model is used to take advantage of temporal context and estimate class probabilities conditioned on recent past history. In this hierarchical pattern of information flow, the time series data is transformed and mapped into a categorical representation (the fault classes) and integrated over time to enable robust decision-making.

Smyth, Padhraic J.↗
NASA NTRS2001

Development of Unsteady Aerodynamic State-Space Models from CFD-Based Pulse Responses

A method for computing discrete-time state-space models of linearized unsteady aerodynamic behavior directly from aeroelastic CFD codes is presented. The method involves the treatment of CFD-based pulse responses as Markov parameters for use in a system identification /realization algorithm. Results are presented for the AGARD 445.6 Aeroelastic Wing with four aeroelastic modes at a Mach number of 0.96 using the EZNSS Euler/Navier-Stokes flow solver with aeroelastic capability. The System/Observer/Controller Identification Toolbox (SOCIT) algorithm, based on the Ho-Kalman realization algorithm, is used to generate 15th- and 32nd-order discrete-time state-space models of the unsteady aerodynamic response of the wing over the entire frequency range of interest.

Silva, Walter A.↗
DOE OSTI2025

Reliability Analysis of Power Grids Considering Component Failures of Variable Energy Resources

This paper proposes an improved model for the reliability assessment of power systems considering component failures of variable energy resources (VER). The inherent intermittency of VER such as solar photovoltaic (PV) and wind farms, along with their susceptibility to component failures, present significant challenges to reliable system operation. These issues, combined with power grid operation and network constraints, complicate the reliable operation of VER-integrated power systems. Here, to address these concerns, this paper introduces a reliability assessment framework that considers VER input variability, its impact on component availability, and their resulting impact on overall system reliability. Stochastic models based on discrete Markov processes are developed to incorporate variable irradiance, wind speeds, and their effects on PV and wind component failure rates. A next-event and state transition-based approach is then developed to integrate the stochastic models into a mixed-timing sequential Monte Carlo simulation framework for composite reliability assessment. Case studies on the RTS-GMLC system demonstrate the effectiveness of the proposed model in evaluating the reliability of VER-integrated systems.

Pandit, Dilip [Sandia National Laboratories (SNL-N↗