SEARCH · Engineering Papers
Results for “Parameterized state and action spaces”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Learning in continuous action space for developing high dimensional potential energy models
Reinforcement learning (RL) approaches that combine a tree search with deep learning have found remarkable success in searching exorbitantly large, albeit discrete action spaces, as in chess, Shogi and Go. Many real-world materials discovery and design applications, however, involve multi-dimensional search problems and learning domains that have continuous action spaces. Exploring high-dimensional potential energy models of materials is an example. Traditionally, these searches are time consuming (often several years for a single bulk system) and driven by human intuition and/or expertise and more recently by global/local optimization searches that have issues with convergence and/or do not scale well with the search dimensionality. Here, in a departure from discrete action and other gradient-based approaches, we introduce a RL strategy based on decision trees that incorporates modified rewards for improved exploration, efficient sampling during playouts and a “window scaling scheme" for enhanced exploitation, to enable efficient and scalable search for continuous action space problems. Using high-dimensional artificial landscapes and control RL problems, we successfully benchmark our approach against popular global optimization schemes and state of the art policy gradient methods, respectively. We demonstrate its efficacy to parameterize potential models (physics based and high-dimensional neural networks) for 54 different elemental systems across the periodic table as well as alloys. We analyze error trends across different elements in the latent space and trace their origin to elemental structural diversity and the smoothness of the element energy surface. Broadly, our RL strategy will be applicable to many other physical science problems involving search over continuous action spaces.
An infrared on-shell action and its implications for soft charge fluctuations in asymptotically flat spacetimes
We study the infrared on-shell action of Einstein gravity in asymptotically flat spacetimes (AFSs), obtaining an effective, gauge-invariant boundary action for memory and shockwave spacetimes. We show that the phase space is in both cases parameterized by the leading soft variables in AFSs, thereby extending the equivalence between shockwave and soft commutators to spacetimes with non-vanishing Bondi mass. We then demonstrate that our on-shell action is equal to three quantities studied separately in the literature: (i) the soft supertranslation charge; (ii) the shockwave effective action, or equivalently the modular Hamiltonian; and (iii) the soft effective action. Finally, we compute the quantum fluctuations in the soft supertranslation charge and, assuming the supertranslation parameter may be promoted to an operator, we obtain an area law, consistent with earlier results showing that the modular Hamiltonian has such fluctuations.
Dynamics of McMillan mappings II. axially symmetric map
Here, in this article, we investigate the transverse dynamics of a single particle in a model integrable accelerator lattice, based on a McMillan axially-symmetric electron lens. Although the McMillan e-lens has been considered as a device potentially capable of mitigating collective space charge forces, some of its fundamental properties have not been described yet. The main goal of our work is to close this gap and understand the limitations and potentials of this device. It is worth mentioning that the McMillan axially symmetric map provides the first-order approximations of dynamics for a general linear lattice plus an arbitrary thin lens with motion separable in polar coordinates. Therefore, advancements in its understanding should give us a better picture of more generic and not necessarily integrable round beams. In the first part of the article, we classify all possible regimes with stable trajectories and find the canonical action-angle variables. This provides an evaluation of the dynamical aperture, Poincaré rotation numbers as functions of amplitudes, and thus determines the spread in nonlinear tunes. Also, we provide a parameterization of invariant curves, allowing for the immediate determination of the map image forward and backward in time. The second part investigates the particle dynamics as a function of system parameters. We show that there are three fundamentally different configurations of the accelerator optics causing different regimes of nonlinear oscillations. Each regime is considered in great detail, including the limiting cases of large and small amplitudes. In addition, we analyze the dynamics in Cartesian coordinates and provide a description of observable variables and corresponding spectra.