SEARCH · Engineering Papers
Results for “zero-order methods”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Two-Stage Reinforcement Learning Policy Search for Grid-Interactive Building Control
This paper develops an intelligent grid-interactive building controller, which optimizes building operation during both normal hours and demand response (DR) events. To avoid costly on-demand computation and to adapt to non-linear building models, the controller utilizes reinforcement learning (RL) and makes real-time decisions based on a near-optimal control policy. Learning such a policy typically amounts to solving a hard non-convex optimization problem. We propose to address this problem with a novel global-local policy search method. In the first stage, an RL algorithm based on zero-order gradient estimation is leveraged to search for the optimal policy globally, due to its scalability and the potential to escape some poor performing local optima. The obtained policy is then fine-tuned locally to bring the first-stage solution closer to that of the original unsmoothed problem. Experiments on a simulated five-zone commercial building demonstrate the advantages of the proposed method over existing learning approaches. They also show that the learned control policy outperforms a pragmatic linear model predictive controller (MPC) and approaches the performance of an oracle MPC in testing scenarios. Using a state-of-the-art advanced computing system, we demonstrate that the controller can be learned and deployed within hours of training.
Risk-Constrained Reinforcement Learning for Inverter-Dominated Power System Controls
Here, this paper develops a risk-aware controller for grid-forming inverters (GFMs) to minimize large frequency oscillations in GFM inverter-dominated power systems. To tackle the high variability from loads/renewables, we incorporate a mean-variance risk constraint into the classical linear quadratic regulator (LQR) formulation for this problem. The risk constraint aims to bound the time-averaged cost of state variability and thus can improve the worst-case performance for large disturbances. The resulting risk-constrained LQR problem is solved through the dual reformulation to a minimax problem, by using a reinforcement learning (RL) method termed as stochastic gradient-descent with max-oracle (SGDmax). In particular, the zero-order policy gradient (ZOPG) approach is used to simplify the gradient estimation using simulated system trajectories. Numerical tests conducted on the IEEE 68-bus system have validated the convergence of our proposed SGDmax for GFM model and corroborate the effectiveness of the risk constraint in improving the worst-case performance while reducing the variability of the overall control cost.
Reinforcement Learning-based Output Structured Feedback for Distributed Multi-Area Power System Frequency Control
Load frequency control (LFC) is a key factor to maintain the stable frequency in multi-area power systems. As the modern power systems evolve from centralized to decentralized paradigm, LFC needs to consider the decentralized scheme that considers limited information from the information-exchange graph for the generator control of each interconnected area. This paper aims to solve a data-driven constrained LQR problem with mean-variance risk constraints and output structured feedback, and applies this framework to solve the LFC problem in multi-area power systems. By reformulating the constrained optimization problem into a minimax problem, the stochastic gradient descent max-oracle (SGDmax) algorithm with zero-order policy gradient (ZOPG) is adopted to find the optimal feedback gain from the learning, while guaranteeing the convergence. In addition, to improve the adaptation of the proposed learning method to new or varying models, we construct an emulator grid that approximates the dynamics of a physical grid and performs training based on this model. Once the feedback gain is obtained from the emulator grid, it is applied to the physical grid with a robustness test to check whether the controller from the approximated emulator applies to the actual system. Numerical tests show that the obtained feedback controller can successfully control the frequency of each area, while mitigating the uncertainty from the loads, with reliable robustness that ensures the adaptability of the obtained feedback gain to the actual physical grid.
Fundamental mechanistic insights into the catalytic reactions of Li-S redox by Co single-atom electrocatalysts via operando methods
Lithium-sulfur batteries represent an attractive option for energy storage applications. A deeper understanding of the multistep lithium-sulfur reactions and the electrocatalytic mechanisms are required to develop advanced, high-performance batteries. We have systematically investigated the lithium-sulfur redox processes catalyzed by a cobalt single-atom electrocatalyst (Co-SAs/NC) via operando confocal Raman microscopy and x-ray absorption spectroscopy (XAS). The real-time observations, based on potentiostatic measurements, indicate that Co-SAs/NC efficiently accelerates the lithium-sulfur reduction/oxidation reactions, which display zero-order kinetics. Under galvanostatic discharge conditions, the typical stepwise mechanism of long-chain and intermediate-chain polysulfides is transformed to a concurrent pathway under electrocatalysis. In addition, operando cobalt K-edge XAS studies elucidate the potential-dependent evolution of cobalt’s oxidation state and the formation of cobalt-sulfur bonds. Our work provides fundamental insights into the mechanisms of catalyzed lithium-sulfur reactions via operando methods, enabling a deeper understanding of electrocatalysis and interfacial dynamics in electrical energy storage systems.
A finite micro-rotation material point method for micropolar solid and fluid dynamics with three-dimensional evolving contacts and free surfaces
This paper introduces an explicit material point method designed specifically for simulating the micropolar continuum dynamics in the finite deformation and finite microrotation regime. The material point method enables us to simulate large deformation problems while circumventing the potential mesh distortion without remeshing. To eliminate rotational motion damping and loss of angular momentum during the projection, we introduce the mapping for microinertia and angular momentum between particles and grids through the affine particle-in-cell approach. The microrotation and the curvature at each particle are updated through zero-order forward integration of the microgyration and its spatial gradient. We show that the microinertia and the angular momentum are conserved during the projections between particles and grids in our formulation. We verify the formulation and implementation by comparing with the analytical dispersion relation of micropolar waves under the small strain and small microrotation, as well as the analytical soliton solution for solids undergoing large deformation and large microrotation. Additionally we also demonstrate the capacity of the proposed computational framework to handle a wide spectrum of simulations that exhibit size effects in the geometrical nonlinear regime through three representative numerical examples, i.e., a cantilever beam torsion problem, a fragment-impact penetration problem, and a micropolar fluid discharging problem.
A method for crystallographic mapping of an alpha-beta titanium alloy with nanometre resolution using scanning precession electron diffraction and open-source software libraries
An approach for the crystallographic mapping of two-phase alloys on the nanoscale using a combination of scanned precession electron diffraction and open-source python libraries is introduced in this paper. This method is demonstrated using the example of a two-phase α/β titanium alloy. The data were recorded using a direct electron detector to collect the patterns, and recently developed algorithms to perform automated indexing and analyse the crystallography from the results. Very high-quality mapping is achieved at a 3 nm step size. The results show the expected Burgers orientation relationships between the α laths and β matrix, as well as the expected misorientations between α laths. A minor issue was found that one area was affected by 180° ambiguities in indexing occur due to this area being aligned too close to a zone axis of the α with twofold projection symmetry (not present in 3D) in the zero-order Laue Zone, and this should be avoided in data acquisition in the future. Nevertheless, this study demonstrates a good workflow for the analysis of nanocrystalline two- or multi-phase materials, which will be of widespread use in analysing two-phase titanium and other systems and how they evolve as a function of thermomechanical treatments.
Identification and quantification of distinct active sites in Hf-Beta zeolites for transfer hydrogenation catalysis
Despite the significant progress made in characterizing different framework heteroatom sites that exist in Lewis acidic zeolites with probe molecule adsorption and spectroscopy, methods to reliably quantify site counts remain indefinite and have been primarily limited to Sn and Ti Lewis acid sites. Here, methods to quantify framework Lewis acidic Hf 4+ sites in zeolite Beta (Hf-Beta) with two Lewis base titrants (pyridine, deuterated acetonitrile) were developed using infrared (IR) spectroscopy. Lewis acid site counts for Hf-Beta zeolites were validated by measuring integrated molar extinction coefficients (IMECs; ε, cm μmol –1 ) on Sn-Beta zeolites using identical Lewis base titrants to benchmark site counts with established literature procedures to quantify Lewis acid sites in Beta zeolites from IR spectra. Highlighting the importance of benchmarking active site counts against well-established experimental protocols, IMECs of CD 3 CN bound to open (ε(Sn; 2316 cm –1 ): 1.80 ± 0.25) and closed (ε(Sn; 2308 cm –1 ): 3.76 ± 0.33) Sn sites were ~1.8x larger than those previously reported while total Lewis acid site counts agreed with those measured by pyridine (ε(Sn; 1451 cm –1 ): 1.58 ± 0.16) on six different Sn-Beta zeolites. IMECs measured for IR peaks reflecting pyridine bound to Lewis acidic Hf sites (ε(Hf; 1448 cm –1 ): 1.54 ± 0.21) and CD 3 CN bound to open (ε(Hf; 2313 cm –1 ): 2.40 ± 0.22) and closed (ε(Hf; 2307 cm –1 ): 3.55 ± 0.41) Hf sites, gave similar counts for the total number of Lewis acidic sites across six Hf-Beta zeolites (Si/Hf = 100–413). Consistent with previous reports with Sn-Beta catalysts where open Sn sites are responsible for catalytic turnover, apparent first and zero-order MPVO rate constants (0.01–1 M cyclohexanone in 2-butanol; per total Hf, 373 K) correlated with the total number of open Hf sites, per total Hf, but not with the total number of closed Hf sites or total Lewis acid site counts. Measured initial MPVO rates (0.1 M cyclohexanone in 2-butanol, per open Hf, 373 K) were ~25x higher on hydrophobic Hf-Beta-F than on hydrophilic Hf-Beta-OH zeolites. Overall, the apparent first-order MPVO rate constants (2-butanol solvent, per open Hf, 373 K) were ~6x higher on Hf-Beta-F than on Hf-Beta-OH zeolites. The characterization methods reported here enable normalization of MPVO turnover rates on Sn- and Hf-Beta zeolites by their number of open sites. Finally, this enables performing quantitative rate comparisons across Lewis acid zeolites of varying active site identity, solvation, and pore topology used in liquid-phase catalysis.
Zero-Order Reaction Kinetics v3.0
Zero-RK is a software package that simulates chemically reacting systems using sparse, preconditioned, adaptive matrix methods to achieve orders-of-magnitude reduction in simulation time while maintaining accurate results.
Zero-Order Reaction Kinetics v. 3.4.hip
Zero-RK is a software package that simulates chemically reacting systems using sparse, preconditioned, adaptive matrix methods to achieve orders-of-magnitude reduction in simulation time while maintaining accurate results.
Zero-Order Reaction Kinetics v. 3.6
Zero-RK is a software package that simulates chemically reacting systems using sparse, preconditioned, adaptive matrix methods to achieve orders-of-magnitude reduction in simulation time while maintaining accurate results.