Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Markov process”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 865 records · Page 48

Review of mixing length estimates and effects of toroidicity in a fluid model for turbulent transport in tokamaks

Basic aspects of turbulent transport in toroidal magnetized plasmas are discussed. In particular the fluid closure has strong effects on zonal flows which are needed to create an absorbing boundary for long wave lengths and also to obtain the Dimits nonlinear upshift. The fluid resonance in the energy equation is found to be instrumental for generating the L–H transition, the spin-up of poloidal rotation in internal transport barriers, as well as the nonlinear Dimits upshift. The difference between the linearly fastest growing mode number and the corresponding longer nonlinear correlation length is also addressed. It is found that the Kadomtsev mixing length result is consistent with the non-Markovian diagonal limit of the transport at the nonlinearly obtained correlation length.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Quantum Dynamics of a Domain Wall in the Presence of Dephasing

We compare quantum dynamics in the presence of Markovian dephasing for a particle hopping on a chain and for an Ising domain wall whose motion leaves behind a string of flipped spins. Exact solutions show that on an infinite chain, the transport responses of the models are nearly identical. However, on finitelength chains, the broadening of discrete spectral lines is much more noticeable in the case of a domain wall.

97 MATHEMATICS AND COMPUTING↗

The Electron-Conformational Model of Ryanodine Receptors of the Heart Cell

The main ideas of an electron-conformational (EC) model of ligand-activated ryanodine channels (RyR) that reduces many degrees of freedom of this gigantic nanoscopic molecular complex to two so-called electron and conformational degrees of freedom are presented. In the toy model, the conformational potential or energy profile of the RyR channel represents two branches that describe the dependence of energy on the conformational coordinate (reaction coordinate) in the initial and ligand-activated states, respectively. A model extension that takes into account both the tetrameric structure of the RyR channel and additional (orthogonal) rotational conformational mode has been considered. The EC model has been demonstrated to give a biophysical basis to the traditional phenomenological model of Markov chains. To demonstrate the possibilities of the EC model, we refer to examples of model description of the dynamics of isolated RyR channels and clusters of RyR channels in release units of the heart cell.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Reciprocity Relations for Nonlinear Systems

A quantum proof of reciprocity relations for nonlinear nonstationary systems in the magnetic field is obtained in the approximation of the Markovian relaxation.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Learning Robust Marking Policies for Adaptive Mesh Refinement

Here in this work, we revisit the marking decisions made in the standard adaptive finite element method (AFEM). Experience shows that a naïve marking policy leads to inefficient use of computational resources for adaptive mesh refinement (AMR). Consequently, using AMR in practice often involves ad-hoc or time-consuming offline parameter tuning to set appropriate parameters for the marking subroutine. To address these practical concerns, we recast AMR as a Markov decision process in which refinement parameters can be selected on-the-fly at run time, without the need for pre-tuning by expert users. In this new paradigm, the refinement parameters are also chosen adaptively via a marking policy that can be optimized using methods from reinforcement learning. We use the Poisson equation to demonstrate our techniques on h- and hp-refinement benchmark problems, and our experiments suggest that superior marking policies remain undiscovered for many classical AFEM applications. Furthermore, an unexpected observation from this work is that marking policies trained on one family of PDEs are sometimes robust enough to perform well on problems far outside the training family. For illustration, we show that a simple hp-refinement policy trained on 2D domains with only a single re-entrant corner can be deployed on far more complicated 2D domains, and even 3D domains, without significant performance loss. For reproduction and broader adoption, we accompany this work with an open-source implementation of our methods.

97 MATHEMATICS AND COMPUTING↗

Automated Adversary-in-the-Loop Cyber-Physical Defense Planning

Security of cyber-physical systems (CPS) continues to pose new challenges due to the tight integration and operational complexity of the cyber and physical components. To address these challenges, this article presents a domain-aware, optimization-based approach to determine an effective defense strategy for CPS in an automated fashion—by emulating a strategic adversary in the loop that exploits system vulnerabilities, interconnection of the CPS, and the dynamics of the physical components. Our approach builds on an adversarial decision-making model based on a Markov Decision Process (MDP) that determines the optimal cyber (discrete) and physical (continuous) attack actions over a CPS attack graph. The defense planning problem is modeled as a non-zero-sum game between the adversary and defender. We use a model-free reinforcement learning method to solve the adversary’s problem as a function of the defense strategy. We then employ Bayesian optimization (BO) to find an approximate best-response for the defender to harden the network against the resulting adversary policy. This process is iterated multiple times to improve the strategy for both players. We demonstrate the effectiveness of our approach on a ransomware-inspired graph with a smart building system as the physical process. Numerical studies show that our method converges to a Nash equilibrium for various defender-specific costs of network hardening.

97 MATHEMATICS AND COMPUTING↗

Hybrid-RL-MPC4CLR (Hybird-Reinforcement-Learning-Model-Predictive-Control-for-Reserve-Policy-Assisted-Critical-Load-Restoration-in-Distribution-Grids)

Hybrid-RL-MPC4CLR was developed as a hybrid controller for active distribution grid critical load restoration, combining deep reinforcement learning (RL) and model predictive control (MPC) aiming at maximizing total restored load following an extreme event. The RL determines a policy for quantifying operating reserve requirements, thereby hedging against uncertainty, while the MPC models grid operations incorporating the RL policy actions (i.e., reserve requirements), renewable (wind and solar) power predictions, and load demand forecasts. The developers formulated the reserve requirement determination problem as a sequential decision-making problem based on the Markov Decision Process (MDP) and design an RL learning environment based on the OpenAI Gym framework and MPC simulation. The RL agent reward and MPC objective function aim to maximize and monotonically increase total restored load and minimize load shedding and renewable power curtailment. The software is developed using various software packages in Python. The MPC's optimal power flow (OPF) model is implemented using the Pyomo package, the RL simulation environment is implemented using the MPC simulation with various scenarios of renewable energy and load demand profiles and power outage beginning times, based on the OpenAI Gym framework. The RL agent training is performed using the RLlib Ray package. The RL algorithm is trained offline using historical forecasts of renewable generation and load demand profiles. Simulation analysis and performance tests are conducted using a modified IEEE 13-bus distribution test feeder containing wind turbine, photovoltaic, microturbine, and battery.

Eseye, Abinet Tesfaye↗

Parallel simulated annealing with embedded machine learning and multifidelity models for reactor core design

This paper presents extensions to a penalty-free, parallel simulated annealing (SA) algorithm for multi-constrained combinatorial optimization with the aim of embedding multi-fidelity physics models into the annealing procedure. The method uses a low-fidelity, quickly executing model for rapid design space exploration and a high-fidelity model for detailed constraint resolution and on-the-fly bias correction. Machine learning models updated within the annealing procedure were used to bridge the gap between the multi-fidelity models, which led to accurate rapid exploration and efficient detailed constraint resolution. A software implementation of the new multi-fidelity optimization methods, called ML-PSA, was demonstrated on a continuous multi-fidelity optimization problem and a constrained combinatorial PWR lattice design problem. These problems demonstrate some of the features, parallel performance characteristics, and extensible nature of the multi-fidelity SA methods. This paper shows that the developed software and procedure are a general optimization tool that can be applied to a wide variety of scientific and engineering design optimization applications. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Measuring small absorptions by exploiting photothermal self-phase modulation

We present a method for the measurement of small optical absorption coefficients. The method exploits the deformation of cavity Airy peaks that occur if the cavity contains an absorbing material with a nonzero thermorefractive coefficient dn/dT or a nonzero expansion coefficient ath. Light absorption leads to a local temperature change and to an intensity-dependent phase shift, i.e., to a photothermal self-phase modulation. The absorption coefficient is derived from a comparison of time-resolved measurements with a numerical time-domain simulation applying a Markov-chain Monte Carlo algorithm. We apply our method to the absorption coefficient of lithium niobate doped with 7mol.% magnesium oxide and derive a value of {alpha}LN=(5.9{+-}0.9)x10{sup -4}/cm. Our method should also apply to materials with much lower absorption coefficients. Based on our modeling, we estimate that, with cavity finesse values of the order of 10{sup 4}, absorption coefficients of as low as 10{sup -8}/cm can be measured.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Methodology for Path Planning with Dynamic Data-Driven Flight Capability Estimation

This paper presents methodology to enable path planning for an unmanned aerial vehicle that uses dynamic data-driven flight capability estimation. The main contribution of the work is a general mathematical approach that leverages offline high-fidelity physics-based modeling together with onboard sensor measurements to achieve dynamic path planning. The mathematical framework, expressed as a constrained partially observable Markov decision process, accounts for vehicle capability constraints and is robust to modeling error and disturbances in both the vehicle process and measurement models. Vehicle capability constraints are incorporated using probabilistic support vector machine surrogates of high-fidelity physics-based models that adequately capture the richness of the vehicle dynamics. Sensor measurements are treated in a general manner and can include combinations of multiple modalities such as that through global positioning system signals, inertial mass unit outputs, and structural strain data of the airframe. Results are presented for a simulated three-dimensional environment and point-mass airplane model. The vehicle can dynamically adjust its trajectory according to the observations it receives about its current state of health, thereby retaining a high probability of survival and mission success.

42 ENGINEERING↗

Pore Formation in the Laminar Structure of Wear-Resistant Modified Powder Coatings

The open porosity of wear-resistant modified coatings is investigated, and their macrostructure is analyzed. A statistical method is proposed for describing the formation of porosity in the laminar structure of a powder coating. The matrix of transition probabilities is determined for a uniform Markov chain. The porosity of a coating on a smooth substrate is calculated.

36 MATERIALS SCIENCE↗

Approximating Nash Equilibrium in Day-ahead Electricity Market Bidding with Multi-agent Deep Reinforcement Learning

In this paper, a day-ahead electricity market bidding problem with multiple strategic generation company (GEN-CO) bidders is studied. The problem is formulated as a Markov game model, where GENCO bidders interact with each other todevelop their optimal day-ahead bidding strategies. Considering unobservable information in the problem, a model-free and data-driven approach, known as multi-agent deep deterministic policy gradient (MADDPG), is applied for approximating the Nash equilibrium (NE) in the above Markov game. The MADDPG algorithm has the advantage of generalization due to the automatic feature extraction ability of the deep neural networks. The algorithm is tested on an IEEE 30-bus system with three competitive GENCO bidders in both an uncongested caseand a congested case. Comparisons with a truthful bidding strategy and state-of-the-art deep reinforcement learning methods including deep Q network and deep deterministic policy gradient (DDPG) demonstrate that the applied MADDPG algorithm can find a superior bidding strategy for all the market participants with increased profit gains. In addition, the comparison with a conventional model-based method shows that the MADDPG algorithm has higher computational efficiency, which is feasible for real-world applications.

24 POWER TRANSMISSION AND DISTRIBUTION↗

THE ORBIT AND MASS OF THE THIRD PLANET IN THE KEPLER-56 SYSTEM

While the vast majority of multiple-planet systems have orbital angular momentum axes that align with the spin axis of their host star, Kepler-56 is an exception: its two transiting planets are coplanar yet misaligned by at least 40° with respect to the rotation axis of their host star. Additional follow-up observations of Kepler-56 suggest the presence of a massive, non-transiting companion that may help explain this misalignment. We model the transit data along with Keck/HIRES and HARPS-N radial velocity data to update the masses of the two transiting planets and infer the physical properties of the third, non-transiting planet. We employ a Markov Chain Monte Carlo sampler to calculate the best-fitting orbital parameters and their uncertainties for each planet. We find the outer planet has a period of 1002 ± 5 days and minimum mass of 5.61 ± 0.38 M{sub Jup}. We also place a 95% upper limit of 0.80 m s{sup −1} yr{sup −1} on long-term trends caused by additional, more distant companions.

79 ASTRONOMY AND ASTROPHYSICS↗

DEVELOPING ATMOSPHERIC RETRIEVAL METHODS FOR DIRECT IMAGING SPECTROSCOPY OF GAS GIANTS IN REFLECTED LIGHT. I. METHANE ABUNDANCES AND BASIC CLOUD PROPERTIES

Upcoming space-based coronagraphic instruments in the next decade will perform reflected light spectroscopy and photometry of cool directly imaged extrasolar giant planets. We are developing a new atmospheric retrieval methodology to help assess the science return and inform the instrument design for such future missions, and ultimately interpret the resulting observations. Our retrieval technique employs a geometric albedo model coupled with both a Markov chain Monte Carlo Ensemble Sampler ( emcee ) and a multimodal nested sampling algorithm ( MultiNest ) to map the posterior distribution. This combination makes the global evidence calculation more robust for any given model and highlights possible discrepancies in the likelihood maps. As a proof of concept, our current atmospheric model contains one or two cloud layers, methane as a major absorber, and a H{sub 2}–He background gas. This 6-to-9 parameter model is appropriate for Jupiter-like planets and can be easily expanded in the future. In addition to deriving the marginal likelihood distribution and confidence intervals for the model parameters, we perform model selection to determine the significance of methane and cloud detection as a function of expected signal-to-noise ratio in the presence of spectral noise correlations. After internal validation, the method is applied to realistic spectra of Jupiter, Saturn, and HD 99492c, a model observing target. We find that the presence or absence of clouds and methane can be determined with high confidence, while parameter uncertainties are model dependent and correlated. Such general methods will also be applicable to the interpretation of direct imaging spectra of cloudy terrestrial planets.

79 ASTRONOMY AND ASTROPHYSICS↗

RESOLVED MILLIMETER-WAVELENGTH OBSERVATIONS OF DEBRIS DISKS AROUND SOLAR-TYPE STARS

The presence of debris disks around young main-sequence stars hints at the existence and structure of planetary systems. Millimeter-wavelength observations probe large grains that trace the location of planetesimal belts. The Formation and Evolution of Planetary Systems Spitzer Legacy survey of nearby young solar analogues yielded a sample of five debris disk-hosting stars with millimeter flux suitable for interferometric follow-up. We present observations with the Submillimeter Array (SMA) and the Combined Array for Research in Millimeter-wave Astronomy at ∼2″ resolution that spatially resolve the debris disks around these nearby (d ∼ 50 pc) stars. Two of the five disks (HD 377, HD 8907) are spatially resolved for the first time and one (HD 104860) is resolved at millimeter wavelengths for the first time. We combine our new observations with archival SMA and Atacama Large Millimeter/Submillimeter Array data to enable a uniform analysis of the full five-object sample. We simultaneously model the broadband photometric data and resolved millimeter visibilities to constrain the dust temperatures and disk morphologies, and perform a Markov Chain Monte Carlo analysis to fit for basic structural parameters. We find that the radii and widths of the cold outer belts exhibit properties consistent with scaled-up versions of the Solar System's Kuiper Belt. All the disks exhibit characteristic grain sizes comparable to the blowout size, and all the resolved observations of emission from large dust grains are consistent with an axisymmetric dust distribution to within the uncertainties. These results are consistent with comparable studies carried out at infrared wavelengths.

79 ASTRONOMY AND ASTROPHYSICS↗

OBSERVATIONAL SIGNATURES OF CORONAL LOOP HEATING AND COOLING DRIVEN BY FOOTPOINT SHUFFLING

The evolution of a coronal loop is studied by means of numerical simulations of the fully compressible three-dimensional magnetohydrodynamic equations using the HYPERION code. The footpoints of the loop magnetic field are advected by random motions. As a consequence, the magnetic field in the loop is energized and develops turbulent nonlinear dynamics characterized by the continuous formation and dissipation of field-aligned current sheets: energy is deposited at small scales where heating occurs. Dissipation is nonuniformly distributed so that only a fraction of the coronal mass and volume gets heated at any time. Temperature and density are highly structured at scales that, in the solar corona, remain observationally unresolved: the plasma of our simulated loop is multithermal, where highly dynamical hotter and cooler plasma strands are scattered throughout the loop at sub-observational scales. Numerical simulations of coronal loops of 50,000 km length and axial magnetic field intensities ranging from 0.01 to 0.04 T are presented. To connect these simulations to observations, we use the computed number densities and temperatures to synthesize the intensities expected in emission lines typically observed with the Extreme Ultraviolet Imaging Spectrometer on Hinode. These intensities are used to compute differential emission measure distributions using the Monte Carlo Markov Chain code, which are very similar to those derived from observations of solar active regions. We conclude that coronal heating is found to be strongly intermittent in space and time, with only small portions of the coronal loop being heated: in fact, at any given time, most of the corona is cooling down.

79 ASTRONOMY AND ASTROPHYSICS↗

Variations of the ISM compactness across the main sequence of star forming galaxies: Observations and simulations

The majority of star-forming galaxies follow a simple empirical correlation in the star formation rate (SFR) versus stellar mass (M{sub *}) plane, of the form SFR∝M{sub ∗}{sup α}, usually referred to as the star formation main sequence (MS). The physics that sets the properties of the MS is currently a subject of debate, and no consensus has been reached regarding the fundamental difference between members of the sequence and its outliers. Here we combine a set of hydro-dynamical simulations of interacting galactic disks with state-of-the-art radiative transfer codes to analyze how the evolution of mergers is reflected upon the properties of the MS. We present Chiburst, a Markov Chain Monte Carlo spectral energy distribution (SED) code that fits the multi-wavelength, broad-band photometry of galaxies and derives stellar masses, SFRs, and geometrical properties of the dust distribution. We apply this tool to the SEDs of simulated mergers and compare the derived results with the reference output from the simulations. Our results indicate that changes in the SEDs of mergers as they approach coalescence and depart from the MS are related to an evolution of dust geometry in scales larger than a few hundred parsecs. This is reflected in a correlation between the specific star formation rate, and the compactness parameter C, that parametrizes this geometry and hence the evolution of dust temperature (T{sub dust}) with time. As mergers approach coalescence, they depart from the MS and increase their compactness, which implies that moderate outliers of the MS are consistent with late-type mergers. By further applying our method to real observations of luminous infrared galaxies (LIRGs), we show that the merger scenario is unable to explain these extreme outliers of the MS. Only by significantly increasing the gas fraction in the simulations are we able to reproduce the SEDs of LIRGs.

79 ASTRONOMY AND ASTROPHYSICS↗