Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “adaptive learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

Reinforcement Learning‐Based Adaptation of Grid Following Inverter's Internal Controller to Networked Microgrids' Strengths

The varying topological configurations, generator commitments and dispatches, and dynamic load demand lead to changing system's strengths during the operations of networked microgrids. When the system's strengths significantly change, the fixed control gains at large devices may result in unsatisfactory system performance; this necessitates the tuning of the control gains at large devices to adapt to the changing system's strengths. In this paper, observer-based reinforcement learning (RL) is utilised to automatically tune the proportional-integral (PI) gains of phase lock loop (PLL) controller of grid-following (GFL) inverters to adapt to the changing strengths of microgrids and networked microgrids. The RL agent in this framework augments an observer predicting system's strengths, from which the RL control policy will adjust accordingly to tune the PLL controller's gains towards the system's strengths. Also, to enhance the control performance, the recently introduced Barrier function-based RL framework is leveraged for the design of reward function to prevent the high frequency nadir. An operational 26 kV electric distribution system, which is modelled as networked microgrids, is used to illustrate the need and effectiveness of the proposed RL-tuned control.

frequency response↗

DeepDriveMD: Deep-Learning Driven Adaptive Molecular Simulations for Protein Folding

Simulations of biological macromolecules are important in understanding the physical basis of complex processes such as protein folding. However, even with increasing computational capacity and specialized architectures, the ability to simulate protein folding at atomistic scales still remains challenging. This stems from the dual aspects of high dimensionality of protein conformational landscapes, and the inability of atomistic molecular dynamics (MD) simulations to sufficiently sample these landscapes to observe folding events. Machine learning/deep learning (ML/DL) techniques, when combined with atomistic MD simulations offer the opportunity to potentially overcome these limitations by: (1) effectively reducing the dimensionality of MD simulations to automatically build latent representations that correspond to biophysically relevant reaction coordinates (RCs), and (2) driving MD simulations to automatically sample potentially novel conformational states based on these RCs. We examine how coupling DL approaches with MD simulations can lead to effective approaches to fold small proteins on supercomputers. In particular, we study the computational costs and effectiveness of scaling DL-coupled MD workflows implemented using RADICAL-Cybertools in folding two prototypical systems, namely Fs-peptide and the fast-folding variant of the villin head piece protein. We demonstrate that a DL-coupled MD workflow is able to effectively learn latent representations and drive adaptive simulations. Compared to traditional MD-based approaches, our approach achieves an effective performance gain in sampling the folded states by at least 2.3x. Together, our study provides quantitative basis to understand how coupling DL approaches to MD simulations, can lead to effective performance gains and reduced times to solution on supercomputing resources.

Lee, Hyungro↗

Deep Learning-Based Adaptive Remedial Action Scheme with Security Margin for Renewable-Dominated Power Grids

The Remedial Action Scheme (RAS) is designed to take corrective actions after detecting predetermined conditions to maintain system transient stability in large interconnected power grids. However, since RAS is usually designed based on a few selected typical operating conditions, it is not optimal in operating conditions that are not considered in the offline design, especially under frequently and dramatically varying operating conditions due to the increasing integration of intermittent renewables. The deep learning-based RAS is proposed to enhance the adaptivity of RAS to varying operating conditions. During the training, a customized loss function is developed to penalize the negative loss and suggest corrective actions with a security margin to avoid triggering under-frequency and over-frequency relays. Simulation results of the reduced United States Western Interconnection system model demonstrate that the proposed deep learning–based RAS can provide optimal corrective actions for unseen operating conditions while maintaining a sufficient security margin.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Target Detection via Cognitive Radars Using Change-Point Detection, Learning, and Adaptation

Many radar detection algorithms that assume a stationary environment (clutter) have been proposed and analyzed over the years. However, in practice, changes in the nonstationary environment can perturb the parameters of the clutter distribution, or even alter the clutter distribution family, which can greatly deteriorate the target detection capability. To avoid such potential performance degradation, cognitive radar systems are envisioned which are required to rapidly realize the nonstationarity, accurately learn the new characteristics of the environments, and adaptively update the detector. In this paper, aiming to develop a fully cognitive radar for target detection in nonstationary environments, we propose a unifying framework that integrates (i) change-point detection of clutter distributions by using a data-driven cumulative sum (CUSUM) algorithm and its extended version, (ii) learning/identification of clutter distribution by applying sparse theory and kernel density estimation methods, and (iii) adaptive target detection by automatically modifying the likelihood-ratio test and corresponding detection threshold. Further, with extensive numerical examples, we demonstrate the achieved improvements in detection performance due to the proposed framework in comparison to a nonadaptive case, an adaptive matched filter (AMF) method, and the clairvoyant case. Herein, we also use Wilcoxon rank-sum tests to evaluate the statistical significance of the performance improvements

42 ENGINEERING↗

System and method for cognitive processing for data fusion

A system and method for cognitive processing of sensor data. A processor array receiving analog sensor data and having programmable interconnects, multiplication weights, and filters provides for adaptive learning in real-time. A static random access memory contains the programmable data for the processor array and the stored data is modified to provide for adaptive learning.

Duong, Tuan A.↗

Skeleton-supported stochastic networks of organic memristive devices: Adaptations and learning

Stochastic networks of memristive devices were fabricated using a sponge as a skeleton material. Cyclic voltage-current characteristics, measured on the network, revealed properties, similar to the organic memristive device with deterministic architecture. Application of the external training resulted in the adaptation of the network electrical properties. The system revealed an improved stability with respect to the networks, composed from polymer fibers.

36 MATERIALS SCIENCE↗

Adapting Reinforcement Learning Treatment Policies Using Limited Data to Personalize Critical Care

We report reinforcement learning (RL) demonstrates promise for developing effective treatment policies in critical care settings. However, existing RL methods often require large and comprehensive patient data sets and do not readily lend themselves to settings in which certain patient subpopulations are severely underrepresented. In this study, we develop a new method, noisy Bayesian policy updates (NBPU), for selecting high-performing reinforcement learning–based treatment policies for underrepresented patient subpopulations using limited observations. Our method uses variational inference to learn a probability distribution over treatment policies based on a reference patient subpopulation for which sufficient data are available. It then exploits limited data from an underrepresented patient subpopulation to update this probability distribution and adapts its recommendations to this subpopulation. We demonstrate our method’s utility on a data set of ICU patients receiving intravenous blood anticoagulant medication. Our results show that NBPU outperforms state-of-the-art methods in terms of both selecting effective treatment policies for patients with nontypical clinical characteristics and predicting the corresponding policies’ performance for these patients.

60 APPLIED LIFE SCIENCES↗

Reinforcement learning for adaptive maintenance policy optimization under imperfect knowledge of the system degradation model and partial observability of system states

Maintenance policy optimization usually is faced with challenges that arise from an imperfect knowledge of system degradation models and from the partial observability of system degradation states. Here, this paper proposes a reinforcement learning method to address these two challenges for a class of maintenance problems with Markov degradation processes. The reinforcement learning approach consists of a learning component and a planning component. Using sequentially collected observations, at each step of decision-making the learning component improves the knowledge of system degradation in terms of the probability distributions of the transition rates based on sequential Bayesian inference. Using the updated transition rates, at each step of decision-making the maintenance policy optimization problem is then formulated as a partially observable Markov decision problem, and the planning component computes the optimal maintenance policy that maximizes the expected cumulative reward. The proposed method is illustrated using a numerical example with repair and inspection maintenance actions. The result shows that as more observations are collected, the learning component progressively learns the true system degradation process, and the planning component adjusts the optimal maintenance policy accordingly as well, which leads to increased reward.

42 ENGINEERING↗

Learning with Adaptive Conservativeness for Distributionally Robust Optimization: Incentive Design for Voltage Regulation

Information asymmetry between the Distribution System Operator (DSO) and Distributed Energy Resource Aggregators (DERAs) obstructs designing effective incentives for voltage regulation. To capture this effect, we employ a Stackelberg game-theoretic framework, where the DSO seeks to overcome the information asymmetry and refine its incentive strategies by learning from DERA behavior over multiple iterations. We introduce a model-based online learning algorithm for the DSO, aimed at inferring the relationship between incentives and DERA responses. Given the uncertain nature of these responses, we also propose a distributionally robust incentive design model to control the probability of voltage regulation failure and then reformulate it into a convex problem. This model allows the DSO to periodically revise distribution assumptions on uncertain parameters in the decision model of the DERA. Finally, we present a gradient-based method that permits the DSO to adaptively modify its conservativeness level, measured by the size of a Wasserstein metric-based ambiguity set, according to historical voltage regulation performance. The effectiveness of our proposed method is demonstrated through numerical experiments.

adaptation models↗

Learning with Adaptive Conservativeness for Distributionally Robust Optimization: Incentive Design for Voltage Regulation: Preprint

Information asymmetry between the Distribution System Operator (DSO) and Distributed Energy Resource Aggregators (DERAs) obstructs designing effective incentives for voltage regulation. To capture this effect, we employ a Stackelberg game-theoretic framework, where the DSO seeks to overcome the information asymmetry and refine its incentive strategies by learning from DERA behavior over multiple iterations. We introduce a model-based online learning algorithm for the DSO, aimed at inferring the relationship between incentives and DERA responses. Given the uncertain nature of these responses, we also propose a distributionally robust incentive design model to control the probability of voltage regulation failure and then reformulate it into a convex problem. This model allows the DSO to periodically revise distribution assumptions on uncertain parameters in the decision model of the DERA. Finally, we present a gradient-based method that permits the DSO to adaptively modify its conservativeness level, measured by the size of a Wasserstein metric-based ambiguity set, according to historical voltage regulation performance. The effectiveness of our proposed method is demonstrated through numerical experiments.

distribution system operator↗

Interactive prostate segmentation using atlas-guided semi-supervised learning and adaptive feature selection

Purpose: Accurate prostate segmentation is necessary for maximizing the effectiveness of radiation therapy of prostate cancer. However, manual segmentation from 3D CT images is very time-consuming and often causes large intra- and interobserver variations across clinicians. Many segmentation methods have been proposed to automate this labor-intensive process, but tedious manual editing is still required due to the limited performance. In this paper, the authors propose a new interactive segmentation method that can (1) flexibly generate the editing result with a few scribbles or dots provided by a clinician, (2) fast deliver intermediate results to the clinician, and (3) sequentially correct the segmentations from any type of automatic or interactive segmentation methods. Methods: The authors formulate the editing problem as a semisupervised learning problem which can utilize a priori knowledge of training data and also the valuable information from user interactions. Specifically, from a region of interest near the given user interactions, the appropriate training labels, which are well matched with the user interactions, can be locally searched from a training set. With voting from the selected training labels, both confident prostate and background voxels, as well as unconfident voxels can be estimated. To reflect informative relationship between voxels, location-adaptive features are selected from the confident voxels by using regression forest and Fisher separation criterion. Then, the manifold configuration computed in the derived feature space is enforced into the semisupervised learning algorithm. The labels of unconfident voxels are then predicted by regularizing semisupervised learning algorithm. Results: The proposed interactive segmentation method was applied to correct automatic segmentation results of 30 challenging CT images. The correction was conducted three times with different user interactions performed at different time periods, in order to evaluate both the efficiency and the robustness. The automatic segmentation results with the original average Dice similarity coefficient of 0.78 were improved to 0.865–0.872 after conducting 55–59 interactions by using the proposed method, where each editing procedure took less than 3 s. In addition, the proposed method obtained the most consistent editing results with respect to different user interactions, compared to other methods. Conclusions: The proposed method obtains robust editing results with few interactions for various wrong segmentation cases, by selecting the location-adaptive features and further imposing the manifold regularization. The authors expect the proposed method to largely reduce the laborious burdens of manual editing, as well as both the intra- and interobserver variability across clinicians.

07 ISOTOPE AND RADIATION SOURCES↗

Adaptive machine learning for time-varying systems: low dimensional latent space tuning

Machine learning (ML) tools such as encoder-decoder convolutional neural networks (CNN) can represent incredibly complex nonlinear functions which map between combinations of images and scalars. For example, CNNs can be used to map combinations of accelerator parameters and images which are 2D projections of the 6D phase space distributions of charged particle beams as they are transported between various particle accelerator locations. Despite their strengths, applying ML to time-varying systems, or systems with shifting distributions, is an open problem, especially for large systems for which collecting new data for re-training is impractical or interrupts operations. Particle accelerators are one example of large time-varying systems for which collecting detailed training data requires lengthy dedicated beam measurements which may no longer be available during regular operations. We present a novel method of adaptive ML for time-varying systems. Our approach is to map very high (N ≈ 100k) dimensional inputs (a combination of scalar parameters and images) into the low dimensional (N ≈ 2) latent space at the output of the encoder section of an encoder-decoder CNN. We then actively tune the low dimensional latent space-based representation of complex system dynamics by the addition of an adaptively tuned feedback vector directly before the decoder sections builds back up to our image-based high-dimensional phase space density representations. This method allows us to learn correlations within and to quickly tune the characteristics of incredibly large parameter space systems and to track their evolution in real time based on feedback without massive new data sets for re-training. We demonstrate that our method can accurately predict and track the phase space of charged particle beams at various locations in a particle accelerator by adaptively adjusting in real-time while the unknown input beam distribution of the accelerator is changing in shape, charge, and offset and while the RF system of the accelerator itself is also changing in an unpredictable way. For FACET-II we demonstrate that such an approach has the potential to use transverse deflecting cavity and energy spread spectrum beam measurements to accurately predict 2D projections of the 6D phase space of the electron beam at the plasma wakefield acceleration interaction point where such diagnostics are unavailable.

47 OTHER INSTRUMENTATION↗

Online Multi-Modal Learning and Adaptive Information Trajectory Planning for Autonomous Exploration

In robotic information gathering missions, scientists are typically interested in understanding variables which require proxy measurements from specialized sensor suites to estimate. However, energy and time constraints limit how often these sensors can be used in a mission. Robots are also equipped with cheaper to use navigation sensors such as cameras. In this paper, we explore a challenging planning problem in which a robot is required to learn about a scientific variable of interest in an initially unknown environment by planning informative paths and deciding when and where to use its sensors. To tackle this we present two innovations: a Bayesian generative model framework to automatically learn correlations between expensive science sensors and cheaper to use navigation sensors online, and a sampling based approach to plan for multiple sensors while handling long horizons and budget constraints. Our approach does not grow in complexity with data and is anytime making it highly applicable to field robotics. We tested our approach extensively in simulation and validated it with real data collected during the 2014 Mojave Volatiles Prospector Mission. Our planning algorithm performs statistically significantly better than myopic approaches and at least as well as a coverage-based algorithm in an initially unknown environment while having added advantages of being able to exploit prior knowledge and handle other intricacies of the real world without further algorithmic modifications.

learning↗

Fuzzy logic path planning system for collision avoidance by an autonomous rover vehicle

The Space Exploration Initiative of the United States will make great demands upon NASA and its limited resources. One aspect of great importance will be providing for autonomous (unmanned) operation of vehicles and/or subsystems in space flight and surface exploration. An additional, complicating factor is that much of the need for autonomy of operation will take place under conditions of great uncertainty or ambiguity. Issues in developing an autonomous collision avoidance subsystem within a path planning system for application in a remote, hostile environment that does not lend itself well to remote manipulation by Earth-based telecommunications is addressed. A good focus is unmanned surface exploration of Mars. The uncertainties involved indicate that robust approaches such as fuzzy logic control are particularly appropriate. Four major issues addressed are (1) avoidance of a fuzzy moving obstacle; (2) backoff from a deadend in a static obstacle environment; (3) fusion of sensor data to detect obstacles; and (4) options for adaptive learning in a path planning system. Examples of the need for collision avoidance by an autonomous rover vehicle on the surface of Mars with a moving obstacle would be wind-blown debris, surface flow or anomalies due to subsurface disturbances, another vehicle, etc. The other issues of backoff, sensor fusion, and adaptive learning are important in the overall path planning system.

Murphy, Michael G.↗

Adaptive Machine Learning for Bragg Coherent Diffraction Imaging (BCDI) of 3D Electron Density Maps with Application to La 2-x Ba x CuO 4 (LBCO) High Temperature Superconductor Studies [PowerPoint]

Understanding mesoscale heterogeneity is important for evaluating fatigue and failure of structural materials and for manufacturing processes. Data-driven tools can facilitate in guiding effort investment (measurements and computations) at various stages of the materials development. Our 3D reconstruction approach is to find a set of coefficients which define a bounding surface of the electron density. Data for training the 3D CNN was generated by sampling coefficients from uniform distributions. Test vs. prediction values are shown for the 28 even-valued coefficients for 1000 test structures, with the best and worst performers highlighted. The adaptive part of this work utilized a model independent extremum seeking (ES). The predictions of the CNN were used as the starting point for the ES algorithm. The convergence of the ES algorithm for 3 different structures is illustrated, and the robustness of the adaptive ML approach was demonstrated on experimentally measured 3D crystal from high energy diffraction microscopy. Reconstruction of a non-uniform density volume with different levels of Poisson noise is also shown. Preliminary attempts at reconstructing measured La 2-x Ba x CuO 4 diffraction patterns have so far failed.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

A junction-tree based learning algorithm to optimize network wide traffic control: A coordinated multi-agent framework

Our study develops a novel reinforcement learning algorithm for the challenging coordinated signal control problem. Traffic signals are modeled as intelligent agents interacting with the stochastic traffic environment. The model is built on the framework of coordinated reinforcement learning. The Junction Tree Algorithm (JTA) based reinforcement learning is proposed to obtain an exact inference of the best joint actions for all the coordinated intersections. Moreover, the algorithm is implemented and tested with a network containing 18 signalized intersections in VISSIM. Finally, our results show that the JTA based algorithm outperforms independent learning (Q-learning), real-time adaptive learning, and fixed timing plans in terms of average delay, number of stops, and vehicular emissions at the network level.

97 MATHEMATICS AND COMPUTING↗