Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Iterative Learning Control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Iterative Learning Control for Video-rate Atomic Force Microscopy

We present a control scheme for video-rate atomic force microscopy with rosette pattern. The controller structure involves a feedback internal-model-based controller and a feedforward iterative learning controller. The iterative learning controller is designed to improve tracking performance of the feedback-controlled scanner by rejecting the repetitive disturbances arising from the system nonlinearities. We investigate the performance of two inversion techniques for constructing the learning filter. We conduct tracking experiments using a two-degree-of-freedom microelectromechanical system (MEMS) nanopositioner at frame rates ranging from 5 to 20 frames per second. Furthermore, the results reveal that the algorithm converges rapidly and the iterative learning controller significantly reduces both the transient and steady-state tracking errors. We acquire and report a series of high-resolution time-lapsed video-rate AFM images with the rosette pattern.

42 ENGINEERING↗
DOE OSTI2024

Design, Modeling, and Control of a Hardware-in-the-Loop Testbed for Off-Road Vehicles

This paper presents the design, modeling, and control of a hardware-in-the-loop (HIL) testbed for off-road vehicles. The proposed HIL testbed employs a transient hydrostatic dynamometer to load a diesel engine to emulate any loading cycles of a wheel loader, which is a representative off-road vehicle. A fully validated wheel loader model is used to calculate the engine load, including both the drive and work functions. Besides, iterative learning control (ILC) has been designed for the loading torque tracking of the hydrostatic dynamometer to ensure accurate emulation of real-world operation scenarios. The developed HIL testbed is used to demonstrate more than 26% energy benefits of automated wheel loaders through systematic optimization compared with human-operated wheel loaders. As a result, this HIL testbed serves as a robust platform for advancing research and development across various off-road vehicles, including excavators, tractors, and harvesters.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2021

Neural Lyapunov Control for Power System Transient Stability: A Deep Learning-Based Approach

We report that power system control and transient stability analysis play essential roles in secure system operation. Control of power systems typically involves highly nonlinear and complex dynamics. Most of the existing works address such problems with additional assumptions in system dynamics, leading to a requirement for a complete and general solution. This paper, therefore, proposes a novel control framework for various power system control and stability problems leveraging a learning-based approach. The proposed framework includes a two-module structure that iteratively and jointly learns the candidate Lyapunov function and control law via deep neural networks in a learning module. Meanwhile, it guides the learning procedure towards valid results satisfying Lyapunov conditions in a falsification module. The introduced termination criteria ensure provable system stability. This control framework is verified through several studies handling different types of power system control problems. The results show that the proposed framework is generalizable and can simplify the control design for complex power systems with the stability guarantee and enlarged region of attraction.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Error field detection and correction studies towards ITER operation

In magnetic fusion devices, error field (EF) sources, spurious magnetic field perturbations, need to be identified and corrected for safe and stable (disruption-free) tokamak operation. Within Work Package Tokamak Exploitation RT04, a series of studies have been carried out to test the portability of the novel non-disruptive method, designed and tested in DIII-D (Paz-Soldan et al 2022 Nucl. Fusion62 126007), and to perform an assessment of model-based EF control strategies towards their applicability in ITER. In this paper, the lessons learned, the physical mechanism behind the magnetic island healing, which relies on enhanced viscous torque that acts against the static electro-magnetic torque, and the main control achievements are reported, together with the first design of the asynchronous EF correction current/density controller for ITER.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Stabilization of the 81-channel coherent beam combination using machine learning

We develop a rapidly converging algorithm for stabilizing a large channel-count diffractive optical coherent beam combination. An 81-beam combiner is controlled by a novel, machine-learning based, iterative method to correct the optical phases, operating on an experimentally calibrated numerical model. A neural-network is trained to detect phase errors based on interference pattern recognition of uncombined beams adjacent to the combined one. Due to the non-uniqueness of solutions in the full space of possible phases, the network is trained within a limited phase perturbation/error range. This also reduces the number of samples needed for training. Simulations have proven that the network can converge in one step for small phase perturbations. When the trained neural-network is applied to a realistic case of 360 degree full range, an iterative scheme exploits random walking at the beginning, with the accuracy of prediction on phase feedback direction, to allow the neural-network to step into the training range for fast convergence. This neural-network-based iterative method of phase detection works tens of times faster than the commonly used stochastic parallel gradient descent approach (SPGD) using a single-detector and random dither when both are tested with random phase perturbations.

Wang, Dan↗
DOE OSTI2023

Walking the ‘design–build–test–learn’ cycle: flux analysis and genetic engineering reveal the pliability of plant central metabolism

Oilseeds are of great economic importance for food and animal feed and their contribution to renewable energy production. Soybean seeds (Glycine max (L.) Merr.) contain c. 40% protein, 20% oil, and 30% carbohydrate (Song et al., 2023). Due to the massive scale of soybean production worldwide, even small improvements in seed protein and oil content make economic sense (Song et al., 2023). Successful manipulation of seed composition largely depends on a thorough understanding of the processes and pathways involved in the biosynthesis of fatty acids and amino acids, which are the building blocks of lipids and proteins. Rational engineering of the synthesis of storage reserves, that is, the rerouting of metabolic flux in central metabolism, is difficult to accomplish due to the complexity of the central metabolic network, the intricate regulation of its enzymes at multiple levels, and the often-unpredictable effects of genetic manipulation (Sweetlove et al., 2017). Therefore, the advancement of our understanding of central metabolism and its control of carbon partitioning requires following an iterative ‘design–build–test–learn’ (DBTL) cycle (Lin & Eudes, 2020) where metabolic flux analysis and hypothesis testing by transgenic approaches are important components. Previous metabolic studies on soybeans using isotopic tracers and metabolic flux analysis have provided insight into how lipid and protein biosynthesis occurs simultaneously during seed development (Allen et al., 2009; Allen & Young, 2013; Kambhampati et al., 2021). In an article published in this issue of New Phytologist, Morley et al. (2023; 1834–1851) put the insights they have gained into the delivery of metabolic precursors and energy cofactors to oil synthesis to the test and arrive at a successful metabolic engineering design. They show that an increase in seed oil content in soybeans can be achieved by overexpression of malic enzyme (ME) during seed development. Malic enzyme refers to a class of decarboxylating malate dehydrogenase enzymes that oxidize malate with NAD + or NADP + as redox cofactor while generating pyruvate and CO 2 . Like higher plants in general, soybean has distinct NADH- or NADPH-producing ME isoforms localized to the cytosol, plastid, or mitochondria (Gerrard Wheeler et al., 2016). As Morley et al. show, an increase in seed oil can be achieved in particular when a NADP+-dependent enzyme isoform (EC 1.1.1.40) is overexpressed in the plastid. Given the complex compartmentalization of pyruvate, malate, and redox metabolism (Fig. 1), increased oil production appears to depend on additional pyruvate and reducing equivalents being produced in the same compartment where de novo fatty acid biosynthesis occurs: the plastid.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Co-orchestration of multiple instruments to uncover structure–property relationships in combinatorial libraries

The rapid growth of automated and autonomous instrumentation brings forth opportunities for the co-orchestration of multimodal tools that are equipped with multiple sequential detection methods or several characterization techniques to explore identical samples. This is exemplified by combinatorial libraries that can be explored in multiple locations via multiple tools simultaneously or downstream characterization in automated synthesis systems. In co-orchestration approaches, information gained in one modality should accelerate the discovery of other modalities. Correspondingly, an orchestrating agent should select the measurement modality based on the anticipated knowledge gain and measurement cost. Herein, we propose and implement a co-orchestration approach for conducting measurements with complex observables, such as spectra or images. The method relies on combining dimensionality reduction by variational autoencoders with representation learning for control over the latent space structure and integration into an iterative workflow via multi-task Gaussian Processes (GPs). This approach further allows for the native incorporation of the system's physics via a probabilistic model as a mean function of the GPs. We illustrate this method for different modes of piezoresponse force microscopy and micro-Raman spectroscopy on a combinatorial Sm-BiFeO3 library. However, the proposed framework is general and can be extended to multiple measurement modalities and arbitrary dimensionality of the measured signals.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

Surface enrichment dictates block copolymer orientation

Orientation of block copolymer (BCP) morphology in thin films is critical to applications as nanostructured coatings. Despite being well-studied, the ability to control BCP orientation across all possible block constituents remains challenging. Here, in this study, we deploy coarse-grained molecular dynamics simulations to study diblock copolymer ordering in thin films, focusing on chain makeup, substrate surface energy, and surface tension disparity between the two constituent blocks. We explore the multi-dimensional parameter space of ordering using a machine-learning approach, where an autonomous loop using a Gaussian process (GP) control algorithm iteratively selects high-value simulations to compute. The GP kernel was engineered to capture known symmetries. The trained GP model serves as both a complete map of system response, and a robust means of extracting material knowledge. We demonstrate that the vertical orientation of BCP phases depends on several counter-balancing energetic contributions, including entropic and enthalpic material enrichment at interfaces, distortion of morphological objects through the film depth, and of course interfacial energies. BCP lamellae are found more resistant to these effects, and thus more robustly form vertical orientations across a broad range of conditions; while BCP cylinders are found to be highly sensitive to surface tension disparity.

77 NANOSCIENCE AND NANOTECHNOLOGY↗
DOE OSTI2021

On Distributed Model-Free Reinforcement Learning Control with Stability Guarantee

Distributed learning can enable scalable and effective decision making in numerous complex cyber-physical systems such as smart transportation, robotics swarm, power systems, etc. However, the stability of the system is usually not guaranteed in most existing learning paradigms; and this limitation can hinder the wide deployment of machine learning in decision making of safety-critical systems. This paper presents a stability guaranteed distributed reinforcement learning (SGDRL) framework for interconnected linear subsystems, without knowing the subsystem models. While the learning process requires data from a peer-to-peer (p2p) communication architecture, the control implementation of each subsystem is only based on its local state. The stability of the interconnected subsystems will be ensured by a diagonally dominant eigenvalue condition, which will then be used in a model-free RL algorithm to learn the feedback control gains. The RL algorithm structure follows an off-policy iterative framework, with interleaved policy evaluation and policy update steps. We numerically validate our theoretical results by performing simulations on four interconnected sub-systems.

Mukherjee, Sayak↗
DOE OSTI2023

Reinforcement Learning of Structured Stabilizing Control for Linear Systems With Unknown State Matrix

This paper delves into designing feedback control gains for a continuous-time linear quadratic regulator (LQR) problem that is constrained to certain predefined structure with unknown state matrix. We bring forth the ideas from reinforcement learning (RL) in conjunction with sufficient stability and performance guarantees in order to design these structured gains using the trajectory measurements of states and controls. Here we first formulate a model-based framework using dynamic programming (DP) to embed the structural constraint to the LQR gain computation in the continuous-time setting, and then subsequently, formulate a policy iteration RL algorithm that can alleviate the requirement of known state matrix in conjunction with maintaining the feedback gain structure. The design enables a distributed learning control design which is necessary for many large-scale cyber-physical systems. Theoretical guarantees are provided for stability and convergence of the structured reinforcement learning (SRL) algorithm. We validate our theoretical results with numerical simulations on a multi-agent networked linear time-invariant (LTI) dynamic system.

42 ENGINEERING↗
DOE OSTI2024

Modifications to the JET Shattered Pellet Injector to Optimize Disruption Mitigation Experiments for Supporting ITER’s DMS Design

Shattered pellet injection (SPI) experiments on Joint European Torus (JET) are an important element in determining the physics basis for mitigating disruptions in ITER. Here, the initial design of the JET SPI system included three barrels to produce pellets with diameters of 4.5, 8.1, and 12.5 mm. The variability of the pellet speed by operating with and without a mechanical punch was limited and led to poor pellet integrity, so the mechanical punch was removed. Fragment size distribution is a function of pellet speed and the desire to change the resulting fragment size distribution was not originally a requirement. After the first set of SPI experiments on JET, different pellet sizes were considered to enable dual injection experiments with identical pellet diameters. It was also determined that speed control is necessary to improve experimental repeatability and to determine how the fragment size distribution impacts mitigation performance. Two barrels were fabricated to form pellets of 10 mm diameter and a third to form an 8.1 mm diameter pellet. New propellant valves were also fabricated and characterized to improve repeatability and overall performance. To have fine control of pellet speed, inserts to reduce the breech volume were fabricated and installed. Laboratory testing was conducted to ensure pellet release and provide a comparison of propellant gas delivered versus pellet speeds for a range of pellet types and mixtures. This article will also discuss how lessons learned from the JET SPI modifications can be applied to other SPI systems, such as the ITER SPI system, as controlling pellet release with the least amount of propellant gas is essential for optimal SPI effectiveness.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2026

Autonomous fabrication of tailored defect structures in 2D materials using machine learning-enabled scanning transmission electron microscopy

Materials with tailored quantum properties can be engineered from atomic-scale assembly techniques, but existing methods often lack the agility and accuracy to precisely and intelligently control the manufacturing process. Here, we demonstrate a fully autonomous approach for fabricating atomic-level defects using electron beams in scanning transmission electron microscopy (STEM) that combines advanced machine learning and automated beam control. As a proof of concept, we achieved controlled fabrication of MoS-nanowire (MoS-NW) edge structures by iterative and targeted exposure of MoS 2 monolayer to a focused electron beam to selectively eject sulfur atoms, utilizing high-angle annular dark-field (HAADF) imaging for feedback-controlled monitoring of structural evolution of defects. A machine learning framework combining a random forest model and a convolutional neural network (CNN) was developed to decode the HAADF image and accurately identify atomic positions and species. This atomic-level information was then integrated into an autonomous decision-making platform, which applied predefined fabrication strategies to instruct beam control about atomic sites to be ejected. The selected sites were subsequently exposed to a localized electron beam using an FPGA-controlled scan routine with precise control over beam positioning and duration. While the MoS-NW edge structures produced exhibit promising mechanical and electronic properties, the proposed methods to build the autonomous fabrication framework is material-agnostic and can be extended to other 2D materials for the creation of diverse defect structures and heterostructures beyond Mo S2 .

Engineering↗
DOE OSTI2021

Distributed deep reinforcement learning for simulation control

Abstract Several applications in the scientific simulation of physical systems can be formulated as control/optimization problems. The computational models for such systems generally contain hyperparameters, which control solution fidelity and computational expense. The tuning of these parameters is non-trivial and the general approach is to manually ‘spot-check’ for good combinations. This is because optimal hyperparameter configuration search becomes intractable when the parameter space is large and when they may vary dynamically. To address this issue, we present a framework based on deep reinforcement learning (RL) to train a deep neural network agent that controls a model solve by varying parameters dynamically. First, we validate our RL framework for the problem of controlling chaos in chaotic systems by dynamically changing the parameters of the system. Subsequently, we illustrate the capabilities of our framework for accelerating the convergence of a steady-state computational fluid dynamics solver by automatically adjusting the relaxation factors of the discretized Navier–Stokes equations during run-time. The results indicate that the run-time control of the relaxation factors by the learned policy leads to a significant reduction in the number of iterations for convergence compared to the random selection of the relaxation factors. Our results point to potential benefits for learning adaptive hyperparameter learning strategies across different geometries and boundary conditions with implications for reduced computational campaign expenses 4 4 Data and codes available at https://github.com/Romit-Maulik/PAR-RL . .

42 ENGINEERING↗
DOE OSTI2023

Implementation of AI/DEEP learning disruption predictor into a plasma control system

Abstract This paper reports on advances in the state‐of‐the‐art deep learning disruption prediction models based on the Fusion Recurrent Neural Network (FRNN) originally introduced in a 2019 NATURE publication [ https://doi.org/10.1038/s41586‐019‐1116‐4 ]. In particular, the predictor now features not only the “disruption score,” as an indicator of the probability of an imminent disruption, but also a “sensitivity score” in real time to indicate the underlying reasons for the imminent disruption. This adds valuable physics interpretability for the deep learning model and can provide helpful guidance for control actuators now implemented into a modern plasma control system (PCS). The advance is a significant step forward in moving from modern deep learning disruption prediction to real‐time control and brings novel AI‐enabled capabilities relevant for application to the future burning plasma ITER system. Our analyses use large amounts of data from JET and DIII‐D vetted in the earlier NATURE publication. In addition to “when” a shot is predicted to disrupt, this paper addresses reasons “why” by carrying out sensitivity studies. FRNN is accordingly extended to use more channels of information, including measured DIII‐D signals such as (i) the “n1rms” signal that is correlated with the n = 1 modes with finite frequency, including neoclassical tearing mode and sawtooth dynamics; (ii) the bolometer data indicative of plasma impurity control; and (iii) “q‐min”—the minimum value of the safety factor relevant to the key physics of kink modes. The additional channels and interpretability features expand the ability of the deep learning FRNN software to provide information about disruption subcategories as well as more precise and direct guidance for the actuators in a PCS.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

Summary report of the 4th IAEA Technical Meeting on Fusion Data Processing, Validation and Analysis (FDPVA)

The objective of the Fourth Technical Meeting on Fusion Data Processing, Validation and Analysis was to provide a platform during which a set of topics relevant to fusion data processing, validation and analysis are discussed with the view of extrapolating needs to next step fusion devices such as ITER. The validation and analysis of experimental data obtained from diagnostics used to characterize fusion plasmas are crucial for a knowledge-based understanding of the physical processes governing the dynamics of these plasmas. This paper presents the recent progress and achievements in the domain of plasma diagnostics and synthetic diagnostics data analysis (including image processing, regression analysis, inverse problems, deep learning, machine learning, big data and physics-based models for control) reported at the meeting. The progress in these areas highlight trends observed in current major fusion confinement devices. A special focus is dedicated on data analysis requirements for ITER and DEMO with a particular attention paid to artificial intelligence for automatization and improving reliability of control processes.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

Domain Aware Deep-learning Algorithms Integrated with Scientific-computing Technologies (DADAIST)

This technical report summarized the contribution of the DADAIST project funded by the Data Model Convergence Initiative via the Laboratory Directed Research and Development (LDRD) investments at Pacific Northwest National Laboratory (PNNL). Specifically, we report the development of the NeuroMANCER (Neural Modules with Adaptive Nonlinear Constraints and Efficient Regularizations), a new open-source Scientific Machine Learning library for formulating and solving parametric constrained optimization problems, physics-informed system identification, and parametric optimal control problems. NeuroMANCER is using differentiable programming to combine modern data-driven models and optimization modeling language into a coherent algorithmic and software framework. NeuroMANCER is a Pytorch-based framework and adopts much of its philosophy focused on research and development, rapid prototyping, and streamlined deployment. Strong emphasis is given to extensibility, interoperability with the PyTorch ecosystem, and quick adaptability to custom domain problems. Neuromancer repository contains a comprehensive library of differentiable modules, including custom activation functions, matrix factorizations, deep learning architectures, neural differential equations, differential equation solvers, implicit layers such as iterative solvers, high-level API for symbolic expressions, API for modeling and control of dynamical systems, and extensive set of tutorial code examples in the form of python scripts and jupyter notebooks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Optimizing Perovskite Thin‐Film Parameter Spaces with Machine Learning‐Guided Robotic Platform for High‐Performance Perovskite Solar Cells

Abstract Simultaneously optimizing the processing parameters of functional thin films remains a challenge. The design and utilization of a fully automated platform called SPINBOT is presented for the engineering of solution‐processed functional thin films. The SPINBOT is capable of performing experiments with high sampling variability through the unsupervised processing of hundreds of substrates with exceptional experimental control. Through the iterative optimization process enabled by the Bayesian optimization (BO) algorithm, the SPINBOT explores an intricate parameter space, continuously improving the quality and reproducibility of the produced thin films. This machine learning (ML)‐guided reliable SPINBOT platform enables the acceleration of the optimization process of perovskite solar cells via a simple photoluminescence characterization of films. As a result, this study arrives at an optimal film that, when processed into a solar cell in an ambient atmosphere, immediately yields a champion power conversion efficiency (PCE) of 21.6% with satisfactory performance reproducibility. The unsealed devices retain 90% of their initial efficiency after 1100 h of continuous operation at 60–65 °C under metal‐halide lamps. It is anticipated that the integration of robotic platforms with the intelligent algorithm will facilitate the widespread adoption of effective autonomous experimentation to address the evolving needs and constraints within the materials science research community.

14 SOLAR ENERGY↗
DOE OSTI2024

Avoiding fusion plasma tearing instability with deep reinforcement learning

For stable and efficient fusion energy production using a tokamak reactor, it is essential to maintain a high-pressure hydrogenic plasma without plasma disruption. Therefore, it is necessary to actively control the tokamak based on the observed plasma state, to manoeuvre high-pressure plasma while avoiding tearing instability, the leading cause of disruptions. This presents an obstacle-avoidance problem for which artificial intelligence based on reinforcement learning has recently shown remarkable performance. However, the obstacle here, the tearing instability, is difficult to forecast and is highly prone to terminating plasma operations, especially in the ITER baseline scenario. Previously, we developed a multimodal dynamic model that estimates the likelihood of future tearing instability based on signals from multiple diagnostics and actuators. Here we harness this dynamic model as a training environment for reinforcement-learning artificial intelligence, facilitating automated instability prevention. We demonstrate artificial intelligence control to lower the possibility of disruptive tearing instabilities in DIII-D, the largest magnetic fusion facility in the United States. The controller maintained the tearing likelihood under a given threshold, even under relatively unfavourable conditions of low safety factor and low torque. In particular, it allowed the plasma to actively track the stable path within the time-varying operational space while maintaining H-mode performance, which was challenging with traditional preprogrammed control. This controller paves the path to developing stable high-performance operational scenarios for future use in ITER.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗