Resilient Dynamic Channel Access via Robust Deep Reinforcement Learning
Not Available
97 MATHEMATICS AND COMPUTING↗
SEARCH · Engineering Papers
Results for “learning dynamics”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Neural controller for adaptive movements with unforeseen payloads
A theory and computer simulation of a neural controller that learns to move and position a link carrying an unforeseen payload accurately are presented. The neural controller learns adaptive dynamic control from its own experience. It does not use information about link mass, link length, or direction of gravity, and it uses only indirect uncalibrated information about payload and actuator limits. Its average positioning accuracy across a large range of payloads after learning is 3 percent of the positioning range. This neural controller can be used as a basis for coordinating any number of sensory inputs with limbs of any number of joints. The feedforward nature of control allows parallel implementation in real time across multiple joints.
Kuperstein, Michael↗
Machine learning nonequilibrium electron forces for spin dynamics of itinerant magnets
Abstract We present a generalized potential theory for conservative as well as nonconservative forces for the Landau-Lifshitz magnetization dynamics. Importantly, this formulation makes possible an elegant generalization of the Behler-Parrinello machine learning (ML) approach, which is a cornerstone of ML-based quantum molecular dynamics methods, to the modeling of force fields in adiabatic spin dynamics of out-of-equilibrium itinerant magnetic systems. We demonstrate our approach by developing a deep-learning neural network that successfully learns the electron-mediated exchange fields in a driven s-d model computed from the nonequilibrium Green’s function method. We show that dynamical simulations with forces predicted from the neural network accurately reproduce the voltage-driven domain-wall propagation. Our work also lays the foundation for ML modeling of spin transfer torques and opens a avenue for ML-based multi-scale modeling of nonequilibrium dynamical phenomena in itinerant magnets and spintronics.
36 MATERIALS SCIENCE↗
SMALE: Enhancing Scalability of Machine Learning Algorithms on Extreme-Scale Computing Platforms
Deployment and execution of machine learning tasks on extreme-scale computing platforms face several significant technical challenges: 1) High computing cost incurred by dense networks – The computing workload of deep networks with densely-connected topology increases rapidly with the network size, imposing a non-scalable computing model of extreme-scale computing platforms; 2) Non-optimized workload distribution – Many advanced deep learning algorithms, e.g., sparsification and irregular net-work topology, produce very unbalanced workload distribution on extreme-scale computing platforms. The computation efficiency is greatly hindered by the incurred data and computation redundancies as well as long tails of the node with extensive workload; 3) Constraints in data movement and I/O bottle-neck – Inter-node data movement in extreme-scale computing platforms are associated with high energy and latency costs, and subject to the constraints of I/O bandwidth; and 4) Generalization of algorithm realization and acceleration on computing platforms – The large varieties of machine learning algorithms and structures of extreme-scale computing platforms make the derivation of a generalized algorithm realization and acceleration method very challenging, which, however, is the requirement by domain scientists and interested users. We call the above challenges Smale’s Problems in Machine Learning and Understanding for High-Performance Computing Scientific Discovery. The objective of our three-year research project is to develop a holistic innovation set at structure, assembly, and acceleration layers of machine learning algorithms to address the above challenges in algorithm deployment and execution. Three tasks are particularly performed, including: At the algorithm structure level, we investigate the techniques that can structurally sparsify on the topology of deep networks for computing workload reduction. We also study clustering and pruning techniques that can optimize the workload distributions over the extreme-scale computing platforms; At the algorithm assembly level, we derive a unified learning framework for unsupervised transfer learning and dynamic growing capabilities. Novel training methods are also exploited to enhance the training efficiency of the proposed framework; At the algorithm acceleration level, we will develop a series of techniques that can accelerate the computation of sparse matrix operations, which are one of the core executions in deep learning and optimize memory access of the concerned platforms. Our proposed techniques attack the fundamental problems in machine learning algorithms running on extreme-scale computing platforms by vertically integrating the solutions at three closely entangled layers, paving the long-term scaling path of machine learning applications under DOE context. Three tasks corresponding to the above respective research orientations are performed during the three-year project period with our collaborators at ORNL. The outcome of the proposed project is anticipated to form a holistic solution set of novel algorithms and network topologies, efficient training techniques, and fast acceleration methods to promote the computing scalability of the machine learning applications of particular interest to DOE.
97 MATHEMATICS AND COMPUTING↗
Ferroelectric FET-based context-switching FPGA enabling dynamic reconfiguration for adaptive deep learning machines
Field programmable gate array (FPGA) is widely used in the acceleration of deep learning applications because of its reconfigurability, flexibility, and fast time-to-market. However, conventional FPGA suffers from the trade-off between chip area and reconfiguration latency, making efficient FPGA accelerations that require switching between multiple configurations still elusive. Here, we propose a ferroelectric field-effect transistor (FeFET)–based context-switching FPGA supporting dynamic reconfiguration to break this trade-off, enabling loading of arbitrary configuration without interrupting the active configuration execution. Leveraging the intrinsic structure and nonvolatility of FeFETs, compact FPGA primitives are proposed and experimentally verified. The evaluation results show our design shows a 63.0%/74.7% reduction in a look-up table (LUT)/connection block (CB) area and 82.7%/53.6% reduction in CB/switch box power consumption with a minimal penalty in the critical path delay (9.6%). Besides, our design yields significant time savings by 78.7 and 20.3% on average for context-switching and dynamic reconfiguration applications, respectively.
97 MATHEMATICS AND COMPUTING↗
DeepPhenoMem V1.0: deep learning modelling of canopy greenness dynamics accounting for multi-variate meteorological memory effects on vegetation phenology
Abstract. Vegetation phenology plays a key role in controlling the seasonality of ecosystem processes that modulate carbon, water and energy fluxes between the biosphere and atmosphere. Accurate modelling of vegetation phenology in the interplay of Earth's surface and the atmosphere is thus crucial to understand how the coupled system will respond to and shape climatic changes. Phenology is controlled by meteorological conditions at different timescales: on the one hand, changes in key meteorological variables (temperature, water, radiation) can have immediate effects on the vegetation development; on the other hand, phenological changes can be driven by past environmental conditions, known as memory effects. However, the processes governing meteorological memory effects on phenology are not completely understood, resulting in their limited performance of vegetation phenology represented in land surface models. A deep learning model, specifically a long short-term memory network (LSTM), has the potential to capture and model the meteorological memory effects on vegetation phenology. Here, we apply the LSTM to model the vegetation phenology using meteorological drivers and high-temporal-resolution canopy greenness observations through digital repeat photography by the PhenoCam network. We compare a multiple linear regression model, a no-memory-effect LSTM model and a full-memory-effect LSTM model to predict the whole seasonal greenness trajectory and the corresponding phenological transition dates across 50 sites and 317 site years during 2009–2018, covering deciduous broadleaf forests, evergreen needleleaf forests and grasslands. Results show that the deep learning model outperforms the multiple linear regression model, and the full-memory-effect LSTM model performs better than the no-memory-effect model for all three plant function types (median R2 of 0.878, 0.957 and 0.955 for broadleaf forests, evergreen needleleaf forests and grasslands). We also find that the full-memory-effect LSTM model is capable of predicting the seasonal dynamic variations of canopy greenness and reproducing trends in shifting phenological transition dates. We also performed a sensitivity analysis of the full-memory-effect LSTM model to assess its plausibility, revealing its coherence with established knowledge of vegetation phenology sensitivity to meteorological conditions, particularly changes in temperature. Our study highlights that (1) multi-variate meteorological memory effects play a crucial role in vegetation phenology, and (2) deep learning opens up new avenues for improving the representation of vegetation phenological processes in land surface models via a hybrid modelling approach.
Geology↗
Architecture for Adaptive Intelligent Systems
We identify a class of niches to be occupied by 'adaptive intelligent systems (AISs)'. In contrast with niches occupied by typical AI agents, AIS niches present situations that vary dynamically along several key dimensions: different combinations of required tasks, different configurations of available resources, contextual conditions ranging from benign to stressful, and different performance criteria. We present a small class hierarchy of AIS niches that exhibit these dimensions of variability and describe a particular AIS niche, ICU (intensive care unit) patient monitoring, which we use for illustration throughout the paper. We have designed and implemented an agent architecture that supports all of different kinds of adaptation by exploiting a single underlying theoretical concept: An agent dynamically constructs explicit control plans to guide its choices among situation-triggered behaviors. We illustrate the architecture and its support for adaptation with examples from Guardian, an experimental agent for ICU monitoring.
ACTIVE CONTROL↗
Investigating permafrost carbon dynamics in Alaska with artificial intelligence
Abstract Positive feedbacks between permafrost degradation and the release of soil carbon into the atmosphere impact land–atmosphere interactions, disrupt the global carbon cycle, and accelerate climate change. The widespread distribution of thawing permafrost is causing a cascade of geophysical and biochemical disturbances with global impacts. Currently, few earth system models account for permafrost carbon feedback (PCF) mechanisms. This research study integrates artificial intelligence (AI) tools and information derived from field-scale surveys across the tundra and boreal landscapes in Alaska. We identify and interpret the permafrost carbon cycling links and feedback sensitivities with GeoCryoAI, a hybridized multimodal deep learning (DL) architecture of stacked convolutionally layered, memory-encoded recurrent neural networks (NN). This framework integratesin-situmeasurements and flux tower observations for teacher forcing and model training. Preliminary experiments to quantify, validate, and forecast permafrost degradation and carbon efflux across Alaska demonstrate the fidelity of this data-driven architecture. More specifically, GeoCryoAI logs the ecological memory and effectively learns covariate dynamics while demonstrating an aptitude to simulate and forecast PCF dynamics—active layer thickness (ALT), carbon dioxide flux (CO 2 ), and methane flux (CH 4 )—with high precision and minimal loss (i.e. ALT RMSE : 1.327 cm [1969–2022]; CO 2 RMSE : 0.697µmolCO 2 m −2 s −1 [2003–2021]; CH 4 RMSE : 0.715 nmolCH 4 m −2 s −1 [2011–2022]). ALT variability is a sensitive harbinger of change, a unique signal characterizing the PCF, and our model is the first characterization of these dynamics across space and time.
Environmental Sciences & Ecology↗
DeepGraphONet: A Deep Graph Operator Network to Learn and Zero-Shot Transfer the Dynamic Response of Networked Systems
This article develops a deep graph operator network (DeepGraphONet) framework that learns to approximate the dynamics of a complex system (e.g., the power grid or traffic) with an underlying subgraph structure. Here, we build our DeepGraphONet by fusing the ability of graph neural networks to exploit spatially correlated graph information and deep operator networks to approximate the solution operator of dynamical systems. The resulting DeepGraphONet can then predict the dynamics within a given short/medium-term time horizon by observing a finite history of the graph state information. Furthermore, we design our DeepGraphONet to be resolution independent. That is, we do not require the finite history to be collected at the exact/same resolution. In addition, to disseminate the results from a trained DeepGraphONet, we design a zero-shot learning strategy that enables using it on a different subgraph. Finally, empirical results on the transient stability prediction problem of power grids and traffic flow forecasting problem of a vehicular system illustrate the effectiveness of the proposed DeepGraphONet.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Interaction of the spatial, dynamic, and temporal components of the working movement in learning to file metal
Interaction of spatial, dynamic, and temporal components of working movements in learning to file metal
HUMAN PERFORMANCE↗
On the specificity between mapping of initial and final states of the magneto Rayleigh-Taylor instability
In this LDRD we investigated the application of machine learning methods to understand dimensionality reduction and evolution of the Rayleigh-Taylor instability (RTI). As part of the project, we undertook a significant literature review to understand current analytical theory and machine learning based methods to treat evolution of this instability. We note that we chose to refocus on assessing the hydrodynamic RTI as opposed to the magneto-Rayleigh-Taylor instability originally proposed. This choice enabled utilizing a wealth of analytic test cases and working with relatively fast running open-source simulations of single-mode RTI. This greatly facilitated external collaboration with URA summer fellowship student, Theodore Broeren. In this project we studied the application of methods from dynamical systems learning and traditional regression methods to recover behavior of RTI ranging from the fully nonlinear to weakly nonlinear (wNL) regimes. Here we report on two of the tested methods SINDy and a more traditional regression-based approach inspired by analytic wNL theory with which we had the most success. We conclude with a discussion of potential future extensions to this work that may improve our understanding from both theoretical and phenomenological perspectives.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
A general Bayesian algorithm for the autonomous alignment of beamlines
Autonomous methods to align beamlines can decrease the amount of time spent on diagnostics, and also uncover better global optima leading to better beam quality. The alignment of these beamlines is a high-dimensional expensive-to-sample optimization problem involving the simultaneous treatment of many optical elements with correlated and nonlinear dynamics. Bayesian optimization is a strategy of efficient global optimization that has proved successful in similar regimes in a wide variety of beamline alignment applications, though it has typically been implemented for particular beamlines and optimization tasks. In this paper, we present a basic formulation of Bayesian inference and Gaussian process models as they relate to multi-objective Bayesian optimization, as well as the practical challenges presented by beamline alignment. We show that the same general implementation of Bayesian optimization with special consideration for beamline alignment can quickly learn the dynamics of particular beamlines in an online fashion through hyperparameter fitting with no prior information. We present the implementation of a concise software framework for beamline alignment and test it on four different optimization problems for experiments on X-ray beamlines at the National Synchrotron Light Source II and the Advanced Light Source, and an electron beam at the Accelerator Test Facility, along with benchmarking on a simulated digital twin. We discuss new applications of the framework, and the potential for a unified approach to beamline alignment at synchrotron facilities.
47 OTHER INSTRUMENTATION↗
Exacerbated drought impacts on global ecosystems due to structural overshoot
Vegetation dynamics are affected not only by the concurrent climate but also by memory-induced lagged responses. For example, favourable climate in the past could stimulate vegetation growth to surpass the ecosystem carrying capacity, leaving an ecosystem vulnerable to climate stresses. This phenomenon, known as structural overshoot, could potentially contribute to worldwide drought stress and forest mortality but the magnitude of the impact is poorly known due to the dynamic nature of overshoot and complex influencing timescales. Here, we use a dynamic statistical learning approach to identify and characterize ecosystem structural overshoot globally and quantify the associated drought impacts. We find that structural overshoot contributed to around 11% of drought events during 1981-2015 and is often associated with compound extreme drought and heat, causing faster vegetation declines and greater drought impacts compared to non-overshoot related droughts. The fraction of droughts related to overshoot is strongly related to mean annual temperature, with biodiversity, aridity and land cover as secondary factors. These results highlight the large role vegetation dynamics play in drought development and suggest that soil water depletion due to warming-induced future increases in vegetation could cause more frequent and stronger overshoot droughts.
54 ENVIRONMENTAL SCIENCES↗
Active learning with multifidelity modeling for efficient rare event simulation
Here, while multifidelity modeling provides a cost-effective way to conduct uncertainty quantification with computationally expensive models, much greater efficiency can be achieved by adaptively deciding the number of required high-fidelity (HF) simulations, depending on the type and complexity of the problem and the desired accuracy in the results. We propose a framework for active learning with multifidelity modeling emphasizing the efficient estimation of rare events. Our framework works by fusing a low-fidelity (LF) prediction with an HF-inferred correction, filtering the corrected LF prediction to decide whether to call the high-fidelity model, and for enhanced subsequent accuracy, adapting the correction for the LF prediction after every HF model call. The framework does not make any assumptions as to the LF model type or its correlations with the HF model. In addition, for improved robustness when estimating smaller failure probabilities, we propose using dynamic active learning functions that decide when to call the HF model. We demonstrate our framework using several academic case studies (including some high-dimensional problems) and two finite element model case studies: estimating Navier-Stokes velocities using the Stokes approximation and estimating stresses in a transversely isotropic model subjected to displacements via a coarsely meshed isotropic model. Across these case studies, not only did the proposed framework estimate the failure probabilities accurately, but compared with either Monte Carlo or a standard variance reduction method, it also required only a small fraction of the calls to the HF model.
42 ENGINEERING↗
Quantum advantage for differential equation analysis
Quantum algorithms for differential equation solving, data processing, and machine learning potentially offer an exponential speedup over all known classical algorithms. However, there also exist obstacles to obtaining this potential speedup in useful problem instances. The essential obstacle for quantum differential equation solving is that outputting useful information may require difficult postprocessing, and the essential obstacle for quantum data processing and machine learning is that inputting the data is a difficult task just by itself. In this study, we demonstrate that, when combined, these difficulties solve one another. We show how the output of quantum differential equation solving can serve as the input for quantum data processing and machine learning, allowing dynamical analysis in terms of principal components, power spectra, and wavelet decompositions. To illustrate this, we consider continuous-time Markov processes on epidemiological and social networks. These quantum algorithms provide an exponential advantage over existing classical Monte Carlo methods.
71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
Dynamical Study of Adsorbate-Induced Restructuring Kinetics in Bimetallic Catalysts Using the PdAu(111) Model System
The dynamic restructuring of bimetallic catalysts plays a crucial role in their catalytic activity and selectivity. In particular, catalyst pretreatment with species such as carbon monoxide and oxygen has been shown to be an effective strategy for tuning the surface composition and morphology. Mechanistic and kinetic understanding of such restructuring are fundamental to the chemistry and engineering of surface active sites but have remained challenging due to the large structural, chemical, and temporal degrees of freedom. Here, we combine time-resolved temperature-programmed infrared reflection absorption spectroscopy, ab initio thermodynamics, and machine-learning molecular dynamics to uncover previously unidentified timescale and kinetic parameters of in situ restructuring in Pd/Au(111), a highly relevant model system for dilute Pd-in-Au nanoparticle catalysts. The key innovation lies in utilizing CO not only as a chemically sensitive probe of surface Pd, but also as an agent that induces restructuring of the surface. Upon annealing in vacuum, as-deposited Pd islands became encapsulated by Au and partially dissolved into the subsurface, leaving behind isolated Pd monomers on the surface. Subsequent exposure to 0.1 mbar CO enabled Pd monomers to repopulate the surface up to 373 K, above which complete Pd dissolution occurred by 473 K, with apparent activation energies of 0.14 and 0.48 eV, respectively. Furthermore, these restructuring processes occurred over the span of ~1000 s at a given temperature. Such a minute-timescale dynamics not only elucidates the fluxional nature of alloy catalysts but also presents an opportunity to fine-tune the surface at moderate temperature and pressure conditions.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Surrogate Modelling of 3rd Integer Resonant Extraction at Fermilab Delivery Ring
We present an ongoing work in which a surrogate model is being developed to reproduce the response dynamics of the third-integer resonant extraction process in the Delivery Ring (DR) at Fermilab. This effort is in pursuit of smoothly extracting circulating beam to the Mu2e Experiment s production target, wherein the goal is to extract a uniform slice of the circulating $1e12$ protons in the DR over 25,000 turns (43~ms). The DR contains 3 harmonic sextupoles which excite a third-integer resonance as well as three fast, tune-ramping quadrupole magnets which drive the horizontal tune towards the $29/3$ resonance. In our initial work the surrogate model trains on a semi-analytical simulation provided in the same format as live data. Using Reinforcement Learning (and other potential ML methods), the trained surrogate acts as the environment in which a simple ML control agent could learn to dynamically adjust the quadrupole ramp at 430 break points within the 43 microsecond spill window. The control agent will be hosted on a dedicated Arria 10 FPGA, introducing its own requirements on control agent architecture. In this work we report the accuracy and fidelity of surrogate models in comparison to the response dynamics of the physics simulator.
Narayanan, Aakaash [Fermilab] (ORCID:0000000157944↗
Algorithm for Training a Recurrent Multilayer Perceptron
An improved algorithm has been devised for training a recurrent multilayer perceptron (RMLP) for optimal performance in predicting the behavior of a complex, dynamic, and noisy system multiple time steps into the future. [An RMLP is a computational neural network with self-feedback and cross-talk (both delayed by one time step) among neurons in hidden layers]. Like other neural-network-training algorithms, this algorithm adjusts network biases and synaptic-connection weights according to a gradient-descent rule. The distinguishing feature of this algorithm is a combination of global feedback (the use of predictions as well as the current output value in computing the gradient at each time step) and recursiveness. The recursive aspect of the algorithm lies in the inclusion of the gradient of predictions at each time step with respect to the predictions at the preceding time step; this recursion enables the RMLP to learn the dynamics. It has been conjectured that carrying the recursion to even earlier time steps would enable the RMLP to represent a noisier, more complex system.
Parlos, Alexander G.↗