Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Markov state models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Equation of State Optimization and Uncertainty Quantization: Implementation in the LANL EOS Production Code OpenSesame

We detail the approaches of particle swarm optimization and Bayesian inference through Markov chain Monte Carlo for equation of state development. This work includes formulation of the modeling for the equation of state, numeric optimization of the parametric models via particle swarm optimization, and generation of probability distributions of equations of state from Markov chain Monte Carlo.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Bayesian inference for plasmonic nanometrology

Here, we introduce a Bayesian method for the characterization of plasmonic nanoparticles, which is applicable to both near- and far-field problems. Designed to combine data generated from any photon-plasmon interaction experiment with physically motivated theoretical models, our approach leverages state-of-the-art Markov chain Monte Carlo sampling techniques and returns parameter estimates on nanometric scales. Simulated spectral data sets, describing resonant scattering of photons from ellipsoidal and toroidal nanoparticles, are explored as concrete examples of our approach, with the resulting Bayesian estimates showing excellent agreement with the ground truth, even under conditions of high statistical noise. By incorporating Bayes factors into the method as well, we reveal how model selection can determine which one of competing geometric shapes better explains the observed data. Our comprehensive nanometrology procedure can be tailored to a variety of light-particle interaction models, and its reliance on Bayesian inference furnishes automatic uncertainty quantification. In addition to applicability to a host of plasmonic configurations such as nanoparticle dimers, trimers, and array studies, it is proposed that the presented analysis can be extended to the quantum regime, where nonclassical photon statistics may provide additional insight for inference of scatterer properties.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

Decision system and method for separating faults from attacks

According to some embodiments, a plurality of monitoring nodes may each generate a series of current monitoring node values over time that represent a current operation of the industrial asset. A node classification computer may determine, for each monitoring node, a classification result indicating whether each monitoring node is in a normal or abnormal state. A disambiguation engine may receive the classification results from the node classification computer and associate a Hidden Markov Model (“HMM”) with each monitoring node. For each node in an abnormal state, the disambiguation engine may execute the HMM associated with that monitoring node to determine a disambiguation result indicating if the abnormal state is a result of an attack or a fault and output a current status of each monitoring node based on the associated classification result and the disambiguation result.

Giani, Annarita↗
DOE OSTI2025

Earth-Centered, Earth-Fixed Inertial Navigation System & Error-State Kalman Filter Reference Manual

This is a self-contained reference document that derives the equations necessary to build a combined inertial navigation system and error-state Kalman filter. Coordinate transform, linear time invariant system, inertial sensing, and error-state Kalman filtering theory is built up from first principles. This theory is then leveraged to derive the system equations for two combined inertial navigation system and error-state Kalman filters: (1) a 15-state system modeling white-noise-integrating accelerometer and gyroscope biases, and (2) a 39-state system modeling static and first-order Gauss-Markov accelerometer and gyroscope biases, scale factor errors, and cross-axis sensitivity errors.

42 ENGINEERING↗
DOE OSTI2022

Reaction–drift–diffusion models from master equations: application to material defects

We present a general method to produce well-conditioned continuum reaction–drift–diffusion equations directly from master equations on a discrete, periodic state space. We assume the underlying data to be kinetic Monte Carlo models (i.e. continuous-time Markov chains) produced from atomic sampling of point defects in locally periodic environments, such as perfect lattices, ordered surface structures or dislocation cores, possibly under the influence of a slowly varying external field. Our approach also applies to any discrete, periodic Markov chain. Here, the analysis identifies a previously omitted non-equilibrium drift term, present even in the absence of external forces, which can compete in magnitude with the reaction rates, thus being essential to correctly capture the kinetics. To remove fast modes which hinder time integration, we use a generalized Bloch relation to efficiently calculate the eigenspectrum of the master equation. A well conditioned continuum equation then emerges by searching for spectral gaps in the long wavelength limit, using an established kinetic clustering algorithm to define a proper reduced, Markovian state space.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Reinforcement learning for real-time process control in high-temperature superconductor manufacturing

With high efficiency and low energy loss, high-temperature superconductors (HTS) have demonstrated their profound applications in various fields, such as medical imaging, transportation, accelerators, microwave devices, and power systems. The high-field applications of HTS tapes have raised the demand for producing cost-effective tapes with long lengths in superconductor manufacturing. However, achieving the uniform and enhanced performance of a long HTS tape is challenging due to the unstable growth conditions in the manufacturing process. Although it is confirmed that the process parameters during the advanced metal organic chemical vapor deposition (A-MOCVD) process influence the uniformity of the produced HTS tapes, the high-dimensional process parameter signals and their complicated interactions make it difficult to develop an effective control policy. In this paper, we propose a local measure for the uniformity of HTS tapes to provide instant feedback for our control policy. Then, we model the manufacturing of HTS tapes as a Markov decision process (MDP) with continuous state and action spaces to assess the instant reward in real time in our feedback control model. As our MDP involves continuous and high-dimensional state and action spaces, a neural fitted Q-iteration (NFQ) algorithm is adopted to solve the MDP with artificial neural network (ANN) function approximation. The collinearity of process parameters can restrict our capability of adjusting the process parameters, which is addressed by the principal component analysis (PCA) in our method. The control policy adjusts the PCA of process parameters using the NFQ algorithm. In conclusion, based on our case studies on real A-MOCVD dataset, the obtained control policy increases the average uniformity of tapes by 5.6% and performs especially well on sample HTS tapes with a low uniformity.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2022

Reinforcement learning for adaptive maintenance policy optimization under imperfect knowledge of the system degradation model and partial observability of system states

Maintenance policy optimization usually is faced with challenges that arise from an imperfect knowledge of system degradation models and from the partial observability of system degradation states. Here, this paper proposes a reinforcement learning method to address these two challenges for a class of maintenance problems with Markov degradation processes. The reinforcement learning approach consists of a learning component and a planning component. Using sequentially collected observations, at each step of decision-making the learning component improves the knowledge of system degradation in terms of the probability distributions of the transition rates based on sequential Bayesian inference. Using the updated transition rates, at each step of decision-making the maintenance policy optimization problem is then formulated as a partially observable Markov decision problem, and the planning component computes the optimal maintenance policy that maximizes the expected cumulative reward. The proposed method is illustrated using a numerical example with repair and inspection maintenance actions. The result shows that as more observations are collected, the learning component progressively learns the true system degradation process, and the planning component adjusts the optimal maintenance policy accordingly as well, which leads to increased reward.

42 ENGINEERING↗
DOE OSTI2020

Improving qubit readout with hidden Markov models

We demonstrate the application of pattern recognition algorithms via hidden Markov models (HMM) for qubit readout. This scheme provides a state-path trajectory approach capable of detecting qubit-state transitions and makes for a robust classification scheme with higher starting-state assignment fidelity than when compared to a multivariate Gaussian or a support vector machine scheme. Therefore, the method also eliminates the qubit-dependent readout time optimization requirement in current schemes. Using a HMM state discriminator we estimate fidelities reaching the ideal limit. Unsupervised learning gives access to transition matrix, priors, and IQ distributions, providing a toolbox for studying qubit-state dynamics during strong projective readout.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Photovoltaic Inverter Failure Mechanism Estimation Using Unsupervised Machine Learning and Reliability Assessment

This article introduces a data-driven approach to assessing failure mechanisms and reliability degradation in outdoor photovoltaic (PV) string inverters. The manufacturer's stated PV inverter lifetime can vary due to the impact of operating site conditions. To address limitations in degradation estimation through accelerated testing, condition monitoring, or degradation modeling, we propose a machine learning (ML) oriented approach. Utilizing data from a 1.4 MW PV power plant operational since 2016, with 46 string PV inverters tied to the grid, we employ the unsupervised one-class support vector machine ML technique to analyze inverter and sensor data, capable of classifying humidity cycling and temperature fluctuations as dominant failure mechanisms. Utilizing the anomaly alert relationship and alert details specific to the inverter, the level of PV inverter output is considered as its availability or available reliability. Subsequently, a continuous Markov model is applied to six-month alert data, revealing an average stated reliability of 20% after 20 years of continuous operation. These results support recommendations for time-bound preventive measures to enhance PV inverter reliability under diverse outdoor conditions. Furthermore, the approach provides a nondestructive, top–down, and generalized method for analyzing any commercial PV inverter exposed to outdoor conditions, contingent on the availability of relevant data.

14 SOLAR ENERGY↗
DOE OSTI2020

On the Performance of Detecting Injection of Fabricated Messages into the CAN Bus

There have been several public demonstrations of attacks on connected vehicles showing the ability of an attacker to take control of a targeted vehicle by injecting messages into their Controller Area Network (CAN) bus. In this article, using injected speed reading and Revolutions Per Minute (RPM) reading messages in in-motion vehicle, we examine the ability of the Pearson correlation and the unsupervised learning methods k-means clustering and Hidden Markov Model (HMM) to differentiate ’no-attack’ and ’under-attack’ states of the given vehicle. We found that the Pearson correlation distinguishes the two states, the k-means clustering method has an acceptable accuracy but high false positive rate and HMM detects attacks with acceptable detection rate but has a high false positive in detecting attacks from speed readings when there is no attack. The accuracy of these unsupervised learning methods are comparable to the ones of the supervised learning methods used by CAN bus Intrusion Detection System (IDS) suppliers. In addition, the article shows that studying CAN anomaly detection techniques using off-vehicle test facilities may not properly evaluate the performance of the detection techniques. Finally, the results suggest using other features besides the data content of the CAN messages and integrate knowledge about how the Electronic Control Units (ECUs) collaborate in building effective techniques for the detection of injection of fabricated message attacks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Joint Estimation of Behind-the-Meter Solar Generation in a Community

Distribution grid planning, control, and optimization require accurate estimation of solar photovoltaic (PV) generation and electric load in the system. Most of the small residential solar PV systems are installed behind-the-meter making only the net load readings available to the utilities. This paper presents an unsupervised framework for joint disaggregation of the net load readings of a group of customers into the solar PV generation and electric load. Our algorithm synergistically combines a physical PV system performance model for individual solar PV generation estimation with a statistical model for joint load estimation. The electric loads for a group of customers are estimated jointly by a mixed hidden Markov model (MHMM) which enables modeling the general load consumption behavior present in all customers while acknowledging the individual differences. At the same time, the model can capture the change in load patterns over a time period by the hidden Markov states. The proposed algorithm is also capable of estimating the key technical parameters of the solar PV systems. Our proposed method is evaluated using the net load, electric load, and solar PV generation data gathered from residential customers located in Austin, Texas. Testing results show that our proposed method reduces the mean squared error of state-of-the-art net-load disaggregation algorithms by 67%.

behind-the-meter solar generation↗
DOE OSTI2022

Behavioral and Population Data-Driven Distribution System Load Modeling

Distribution system residential load modeling and analysis for different geographic areas within a utility or an independent system operator territory are critical for enabling small-scale, aggregated distributed energy resources to participate in grid services under Federal Energy Regulatory Commission Order No. 2222 [1]. In this study, we develop a methodology of modeling residential load profiles in different geographic areas with a focus on human behavior impact. First, we construct a behavior-based load profile model leveraging state-of-the-art appliance models. We simulate human activity and occupancy using Markov chain Monte Carlo methods calibrated with the American Time Use Survey data set. Second, we link our model with cleaned Current Population Survey data from the U.S. Census Bureau. Finally, we populate two sets of 500 households using California and Texas census data, respectively, to perform an initial analysis of the load in different geographic areas with various group features (e.g., different income levels). To distinguish the effect of population behavior differences on aggregated load, we simulate load profiles for both sets assuming fixed physical household parameters and weather data. Analysis shows that average daily load profiles vary significantly by income and income dependency varies by locality.

American Time Use Survey↗
DOE OSTI2022

A Logarithmic Bayesian Approach to Quantum Error Detection

We consider the problem of continuous quantum error correction from a Bayesian perspective, proposing a pair of digital filters using logarithmic probabilities that are able to achieve near-optimal performance on a three-qubit bit-flip code, while still being reasonable to implement on low-latency hardware. These practical filters are approximations of an optimal filter that we derive explicitly for finite time steps, in contrast with previous work that has relied on stochastic differential equations such as the Wonham filter. By utilizing logarithmic probabilities, we are able to eliminate the need for explicit normalization and can reduce the Gaussian noise distribution to a simple quadratic expression. The state transitions induced by the bit-flip errors are modeled using a Markov chain, which for log-probabilities must be evaluated using a LogSumExp function. We develop the two versions of our filter by constraining this LogSumExp to have either one or two inputs, which favors either simplicity or accuracy, respectively. Using simulated data, we demonstrate that the single-term and two-term filters are able to significantly outperform both a double threshold scheme and a linearized version of the Wonham filter in tests of error detection under a wide variety of error rates and time steps.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Quantum Markov chain Monte Carlo with digital dissipative dynamics on quantum computers

Modeling the dynamics of a quantum system connected to the environment is critical for advancing our understanding of complex quantum processes, as most quantum processes in nature are affected by an environment. Modeling a macroscopic environment on a quantum simulator may be achieved by coupling independent ancilla qubits that facilitate energy exchange in an appropriate manner with the system and mimic an environment. This approach requires a large, and possibly exponential number of ancillary degrees of freedom which is impractical. In contrast, we develop a digital quantum algorithm that simulates interaction with an environment using a small number of ancilla qubits. By combining periodic modulation of the ancilla energies, or spectral combing, with periodic reset operations, we are able to mimic interaction with a large environment and generate thermal states of interacting many-body systems. We evaluate the algorithm by simulating preparation of thermal states of the transverse Ising model. Our algorithm can also be viewed as a quantum Markov chain Monte Carlo process that allows sampling of the Gibbs distribution of a multivariate model. To illustrate this we evaluate the accuracy of sampling Gibbs distributions of simple probabilistic graphical models using the algorithm.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Bayesian Exploration of Phenomenological EoS of Neutron/Hybrid Stars with Recent Observations

The description of the stellar interior of compact stars remains as a big challenge for the nuclear astrophysics community. The consolidated knowledge is restricted to density regions around the saturation of hadronic matter ρ 0 =2.8 × 10 14 g cm -3 , regimes where our nuclear models are successfully applied. As one moves towards higher densities and extreme conditions up to the quark/gluons deconfinement, little can be said about the microphysics of the equation of state (EoS). Here, we employ a Markov Chain Monte Carlo (MCMC) strategy to access the variability at high density regions of polytropic piecewise models for neutron star (NS) EoS or possible hybrid stars, i.e., a NS with a small quark-matter core. With a fixed description of the hadronic matter for low density, below the nuclear saturation density, we explore a variety of models for the high density regimes leading to stellar masses near to 2.5 M ⊙ , in accordance with the observations of massive pulsars. The models are constrained, including the observation of the merger of neutrons stars from VIRGO-LIGO and with the pulsar observed by NICER. In addition, we also discuss the possibility of the use of a Bayesian power regression model with heteroscedastic error. The set of EoS from the Laser Interferometer Gravitational-Wave Observatory (LIGO) was used as input and treated as the data set for the testing case.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

CHMMPY: A python package for constrained Hidden Markov Models

SAND2025-11909O chmmpy software analyzes multivariate timeseries data to detect patterns. It uses a Hidden Markov Model (HMM) and application-specific constraints that reflect known relationships among hidden states to accomplish this. The chmmpy software provides a generic framework for expressing application-specific constraints and supporting constrained HMM inference using optimization solvers. chmmpy is available on GitHub. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Hart, William↗
DOE OSTI2024

Decision-making based on Markov decision process in integrated artificial reasoning framework—Part I: Theory

This paper presents a decision-making framework based on an integrated artificial reasoning framework and Markov decision process (MDP). The integrated artificial reasoning framework provides a physics-based approach that converts system information into state transition models, and the analysis result will be represented by the transition probabilities that can be used with an MDP to find a traceable and explainable optimal pathway. A dynamic Bayesian network (DBN) is well suited for representing the structure of an MDP. The causality information among process variables (or among subsystems) is mathematically represented in a DBN by the conditional probabilities of the node’s states provided different probabilities of the parent node’s states. To define node states in a physically understandable manner, we used multilevel flow modeling (MFM). An MFM follows the fundamental energy and mass conservation laws and supports the selection of process variables that represent the system of interest so that causal relations among process variables are properly captured. An MFM-based DBN supports developing state transition models in an MDP to capture the effect of process variables of system having physical relations. The operators of the target system can capture stochastic system dynamics as multiple subsystem state transitions based on their physical relations and uncertainties coming from component degradation or random failures. We analyzed a simplified exemplary system to illustrate an optimal operational policy using the suggested approach.

Markov decision process↗
DOE OSTI2022

Classification of Photovoltaic Failures with Hidden Markov Modeling, an Unsupervised Statistical Approach

Failure detection methods are of significant interest for photovoltaic (PV) site operators to help reduce gaps between expected and observed energy generation. Current approaches for field-based fault detection, however, rely on multiple data inputs and can suffer from interpretability issues. In contrast, this work offers an unsupervised statistical approach that leverages hidden Markov models (HMM) to identify failures occurring at PV sites. Using performance index data from 104 sites across the United States, individual PV-HMM models are trained and evaluated for failure detection and transition probabilities. This analysis indicates that the trained PV-HMM models have the highest probability of remaining in their current state (87.1% to 93.5%), whereas the transition probability from normal to failure (6.5%) is lower than the transition from failure to normal (12.9%) states. A comparison of these patterns using both threshold levels and operations and maintenance (O&M) tickets indicate high precision rates of PV-HMMs (median = 82.4%) across all of the sites. Although additional work is needed to assess sensitivities, the PV-HMM methodology demonstrates significant potential for real-time failure detection as well as extensions into predictive maintenance capabilities for PV.

classification↗