An optimal discrete control strategy for terminal guidance.
Optimal discrete control strategy for terminal guidance using computation procedure based on theory of dynamic programming
DYNAMIC PROGRAMMING↗
SEARCH · Engineering Papers
Results for “Dynamic Programming”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Satellite-tracking and Earth dynamics research programs
Tracking of LAGEOS for polar motion and Earth rotation studies and for other geophysical investigations, including crustal dynamics, Earth and ocean tides, and the general development of precision orbit determination continues. The BE-C and Starlette satellites were tracked for refined determinations of station coordinates and the Earth's gravity field and for studies of solid Earth dynamics.
Source record↗
Parallel and vector computation for stochastic optimal control applications
A general method for parallel and vector numerical solutions of stochastic dynamic programming problems is described for optimal control of general nonlinear, continuous time, multibody dynamical systems, perturbed by Poisson as well as Gaussian random white noise. Possible applications include lumped flight dynamics models for uncertain environments, such as large scale and background random atmospheric fluctuations. The numerical formulation is highly suitable for a vector multiprocessor or vectorizing supercomputer, and results exhibit high processor efficiency and numerical stability. Advanced computing techniques, data structures, and hardware help alleviate Bellman's curse of dimensionality in dynamic programming computations.
Hanson, F. B.↗
The use of CMAC neural architectures in obstacle avoidance
In this paper, CMAC neural architectures are used in conjunction with a hierarchical planning approach to find collision free paths over two dimensional analog valued obstacle fields. The method constructs a coarse resolution version of the original problem and then finds the corresponding coarse optimal path using multipass dynamic programming. CMAC artificial neural architectures are used to estimate the analog transition costs that dynamic programming requires. The coarse optimal path is then used as a baseline for the construction of a fine scale optimal path through the original obstacle array.
Peterson, James K.↗
How good are learning-based control v.s. model-based control for load shifting? Investigations on a single zone building energy system
Both model predictive control (MPC) and deep reinforcement learning control (DRL) have been presented as a way to approximate the true optimality of a dynamic programming problem, and these two have shown significant operational cost saving potentials for building energy systems. Furthermore, there is still a lack of in-depth quantitative studies on their approximation levels to the true optimality, especially in the building energy domain. To fill in the gap, this paper provides a numerical framework that enables the evaluation of the optimality levels of different controllers for building energy systems. This framework is then used to comprehensively compare the optimal control performance of both MPC and DRL controllers with given computation budgets for a single zone fan coil unit system. Note the optimality is estimated based on a user-specific selection of trade-off weights among energy costs, thermal comfort and control slew rates. Compared with the best optimality we can find through expensive optimization simulations, the best DRL agent can maximally approximate the optimality by 96.54%, which outperforms the best MPC whose optimality level is 90.11%. However, due to the stochasticity, the DRL agent is only expected to approximate the optimality by 90.42%, which is almost equivalent to the best MPC. Except for Proximal Policy Optimization (PPO), all DRL agents can have a better approximation to the optimality than the best MPC, and are expected to have better approximation than the MPC with a prediction horizon of 32 steps (15 min per step). In terms of reducing energy cost and thermal discomfort, MPC can outperform the rule-based control (RBC) by 18.47%–25.44%. DRL can be expected to outperform RBC by 18.95%–25.65% ,and the best DRL control policy can outperform RBC by 20.29%–29.72%. Although the comparison of the optimality level is performed in a perfect setting, e.g., MPC assumes perfect models, and DRL assumes a perfect offline training process and online deployment process, this can shed insight on their capabilities of approximating to the original dynamic programming problem.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Supercomputer optimizations for stochastic optimal control applications
Supercomputer optimizations for a computational method of solving stochastic, multibody, dynamic programming problems are presented. The computational method is valid for a general class of optimal control problems that are nonlinear, multibody dynamical systems, perturbed by general Markov noise in continuous time, i.e., nonsmooth Gaussian as well as jump Poisson random white noise. Optimization techniques for vector multiprocessors or vectorizing supercomputers include advanced data structures, loop restructuring, loop collapsing, blocking, and compiler directives. These advanced computing techniques and superconducting hardware help alleviate Bellman's curse of dimensionality in dynamic programming computations, by permitting the solution of large multibody problems. Possible applications include lumped flight dynamics models for uncertain environments, such as large scale and background random aerospace fluctuations.
Chung, Siu-Leung↗
Six degree of freedom FORTRAN program, ASTP docking dynamics, users guide
The digital program ASTP Docking Dynamics as outlined is intended to aid the engineer using the program to determine the docking system loads and attendant vehicular motion resulting from docking two vehicles that have an androgynous, six-hydraulic-attenuator, guide ring, docking interface similar to that designed for the Apollo/Soyuz Test Project (ASTP). This program is set up to analyze two different vehicle combinations: the Apollo CSM docking to Soyuz and the shuttle orbiter docking to another orbiter. The subroutine modifies the vehicle control systems to describe one or the other vehicle combinations; the rest of the vehicle characteristics are changed by input data. To date, the program has been used to predict and correlate ASTP docking loads and performance with docking test program results from dynamic testing. The program modified for use on IBM 360 computers. Parts of the original docking system equations in the areas of hydraulic damping and capture latches are modified to better describe the detail design of the ASTP docking system.
Mount, G. O., Jr.↗
Utilising demand response for distribution service restoration to achieve grid resiliency against natural disasters
The increased frequency of power outages due to natural disasters in recent years has highlighted the urgency of enhancing distribution grid resilience. The effective distribution service restoration (DSR) is an important measure for a resilient distribution grid. In this work, the authors demonstrate that DSR can be significantly improved by leveraging the flexibility provided by the inclusion of demand response (DR). The authors propose a framework for this by considering integrated control of household-level flexible appliances to vary the load demand at the distribution-grid level to improve DSR. The overall framework of the proposed system is modelled as a three-step method considering three optimization problems to (i) calculate feasible controllable aggregated load range for each bus, (ii) determine candidate buses to perform DR and their target load demand, and (iii) maintain the load level in each house through home energy management during the DSR, considering uncertainties in load and solar generation sequentially. The optimization problems are formulated as linear programming, mixed-integer linear programming, and multistage stochastic programming (solved using the stochastic dual dynamic programming) models. Finally, case studies performed in the IEEE 123-node test feeder show improvements in resilience in terms of energy restored compared to the restoration process without DR.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Calculation of flight vibration levels of the AH-1G helicopter and correlation with existing flight vibration measurements
NASA-Langley is sponsoring a rotorcraft structural dynamics program with the objective to establish in the U.S. a superior capability to utilize finite element analysis models for calculations to support industrial design of helicopter airframe structures. In the initial phase of the program, teams from the major U.S. manufacturers of helicopter airframes will apply extant finite element analysis methods to calculate loads and vibrations of helicopter airframes, and perform correlations between analysis and measurements. The aforementioned rotorcraft structural dynamics program was given the acronym DAMVIBS (Design Analysis Method for Vibrations). Sikorsky's RDYNE Rotorcraft Dynamics Analysis used for the correlation study, the specifics of the application of RDYNE to the AH-1G, and comparisons of the predictions of the method with flight data for loads and vibrations on the AH-1G are described. RDYNE was able to predict trends of variations of loads and vibrations with airspeed, but in some instances magnitudes differed from measured results by factors of two or three to one. Sensitivities were studied of predictions to rotor inflow modeling, effects of torsional modes, number of blade bending modes, fuselage structural damping, and hub modal content.
Sopher, R.↗
MultiSector Dynamics: 2023 Inaugural Workshop Report
Preface The MultiSector Dynamics (MSD) Community of Practice (CoP) hosted an inaugural workshop on October 3-5, 2023 at the University of California, Davis, to bring together members of the MSD community of practice to advance understanding of the co-evolution of human and natural systems, and to build the next generation of tools that bridge sectors, scales, and systems to realize a more resilient and equitable future. The theme of the workshop was "Advancing Complex Adaptive Human-Earth Systems Science in a World of Interconnected Risks". This document outlines the motivation for the workshop, its goals and objectives, the application process, the agenda, overviews of the training sessions offered to the workshop participants and a summary of each breakout session. The MSD workshop report further discusses the feedback from workshop participants and presents some reflections and next steps. The MSD Workshop organizers thank the DOE Office of Science, Earth and Environmental System Modeling, MultiSector Dynamics program area for financial support of its activities through the Integrated Multisector Multiscale Modeling (IM3) project. For more information related to the broader DOE MultiSector Dynamics Program please see https://climatemodeling.science.energy.gov/program-area/multisector-dynamics. D.L.M. and C.M.B. acknowledge support from the Laboratory Directed Research and Development Program of Oak Ridge National Laboratory (ORNL), managed by UT-Battelle, LLC, for the US Department of Energy (DOE). Disclaimer This report was prepared as an account of work sponsored by an agency of the United States Government. Neither theUnited States Government nor any agency thereof, nor Battelle Memorial Institute, nor any of their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, complete- ness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial products, process, or service by trade name,trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof, or Battelle Memorial Institute. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof. Pacific Northwest National Laboratory operated by Battelle for the United States Department of Energy Available from:Office of Scientific and Technical Information http://www.OSTI.gov multisectordynamics.org This work is made available under the terms of the Creative Commons Attribution- NonCommercial 4.0 International (CC BY-NC 4.0) https://creativecommons.org/licenses/by-nc/4 Suggested citation: Monier, E., Reed, P.M., Vernon, C.R., Hadjimichael, A., Brelsford, C.M., Burleyson, C.B., Dyreson, A.R., Fletcher, S.M., Giang, A., Gupta, R.S., Jackson, N.D., Jones, A.D., Lamontagne, J.R., McCollum, D.L., Morris, J.F., Moss, R.H., Peng, W., Saari, R.K., Srikrishnan, V., Szinai, J.K., Yoon, J. (2024) MultiSector Dynamics: 2023 Inaugural Workshop Report. MSD-LIVE Data Repository. doi:10.57931/2371710.
Monier, Erwan↗
Fluid-connector selection
Usefulness of dynamic programming for systems design is illustrated by study on selection of separable bolted connectors for aerospace applications. Report should be of specific interest to those considering other dynamic programming applications in fields such as construction of refineries, chemical plants, and electronics.
Prasthofer, W. P.↗
Satellite-tracking and Earth dynamics research programs
The major focus for operations during this period was the preliminary MERIT Campaign and its intensive tracking of LAGEOS for polar motion and Earth rotation studies. The data acquired from LAGEOS were used for other geophysical investigations, including studies of crustal dynamics, and Earth and ocean tides, and for the general development of precision orbit determination. The network performed regular tracking of several other retroreflector satellites including GEOS-1, GEOS-3, BE-C, and Starlette for refined determinations of station coordinates and Earth's gravity field and for studies of solid Earth dynamics.
Source record↗
A user's guide to the Flexible Spacecraft Dynamics and Control Program
A guide to the use of the Flexible Spacecraft Dynamics Program (FSD) is presented covering input requirements, control words, orbit generation, spacecraft description and simulation options, and output definition. The program can be used in dynamics and control analysis as well as in orbit support of deployment and control of spacecraft. The program is applicable to inertially oriented spinning, Earth oriented or gravity gradient stabilized spacecraft. Internal and external environmental effects can be simulated.
Fedor, J. V.↗
Perform - A performance optimizing computer program for dynamic systems subject to transient loadings
A description and applications of a computer capability for determining the ultimate optimal behavior of a dynamically loaded structural-mechanical system are presented. This capability provides characteristics of the theoretically best, or limiting, design concept according to response criteria dictated by design requirements. Equations of motion of the system in first or second order form include incompletely specified elements whose characteristics are determined in the optimization of one or more performance indices subject to the response criteria in the form of constraints. The system is subject to deterministic transient inputs, and the computer capability is designed to operate with a large linear programming on-the-shelf software package which performs the desired optimization. The report contains user-oriented program documentation in engineering, problem-oriented form. Applications cover a wide variety of dynamics problems including those associated with such diverse configurations as a missile-silo system, impacting freight cars, and an aircraft ride control system.
Pilkey, W. D.↗
A Unifying View of Estimation and Control Using Belief Propagation With Application to Path Planning
The use of estimation techniques on stochastic models to solve control problems is an emerging paradigm that falls under the rubric of Active Inference (AI) and Control as Inference (CAI). In this work, we use probability propagation on factor graphs to show that various algorithms proposed in the literature can be seen as specific composition rules in a factor graph. We show how this unified approach, presented both in probability space and in log of the probability space, provides a very general framework that includes the Sum-product, the Max-product, Dynamic programming and mixed Reward/Entropy criteria-based algorithms. The framework also expands algorithmic design options that lead to new smoother or sharper policy distributions. We propose original recursions such as: a generalized Sum/Max-product algorithm, a Smooth Dynamic programming algorithm and a modified versions of the Reward/Entropy algorithm. The discussion is carried over with reference to a path planning problem where the recursions that arise from various cost functions, although they may appear similar in scope, bear noticeable differences. We provide a comprehensive table of composition rules and a comparison through simulations, first on a synthetic small grid with a single goal with obstacles, and then on a grid extrapolated from a real-world scene with multiple goals and a semantic map.
Francesco A. N. Palmieri↗
A forward method for optimal stochastic nonlinear and adaptive control
A computational approach is taken to solve the optimal nonlinear stochastic control problem. The approach is to systematically solve the stochastic dynamic programming equations forward in time, using a nested stochastic approximation technique. Although computationally intensive, this provides a straightforward numerical solution for this class of problems and provides an alternative to the usual dimensionality problem associated with solving the dynamic programming equations backward in time. It is shown that the cost degrades monotonically as the complexity of the algorithm is reduced. This provides a strategy for suboptimal control with clear performance/computation tradeoffs. A numerical study focusing on a generic optimal stochastic adaptive control example is included to demonstrate the feasibility of the method.
Bayard, David S.↗
Neural Architectures for Control
The cerebellar model articulated controller (CMAC) neural architectures are shown to be viable for the purposes of real-time learning and control. Software tools for the exploration of CMAC performance are developed for three hardware platforms, the MacIntosh, the IBM PC, and the SUN workstation. All algorithm development was done using the C programming language. These software tools were then used to implement an adaptive critic neuro-control design that learns in real-time how to back up a trailer truck. The truck backer-upper experiment is a standard performance measure in the neural network literature, but previously the training of the controllers was done off-line. With the CMAC neural architectures, it was possible to train the neuro-controllers on-line in real-time on a MS-DOS PC 386. CMAC neural architectures are also used in conjunction with a hierarchical planning approach to find collision-free paths over 2-D analog valued obstacle fields. The method constructs a coarse resolution version of the original problem and then finds the corresponding coarse optimal path using multipass dynamic programming. CMAC artificial neural architectures are used to estimate the analog transition costs that dynamic programming requires. The CMAC architectures are trained in real-time for each obstacle field presented. The coarse optimal path is then used as a baseline for the construction of a fine scale optimal path through the original obstacle array. These results are a very good indication of the potential power of the neural architectures in control design. In order to reach as wide an audience as possible, we have run a seminar on neuro-control that has met once per week since 20 May 1991. This seminar has thoroughly discussed the CMAC architecture, relevant portions of classical control, back propagation through time, and adaptive critic designs.
Peterson, James K.↗
An assessment of separable fluid connector system parameters to perform a connector system design optimization study
The key to optimization of design where there are a large number of variables, all of which may not be known precisely, lies in the mathematical tool of dynamic programming developed by Bellman. This methodology can lead to optimized solutions to the design of critical systems in a minimum amount of time, even when there are a great number of acceptable configurations to be considered. To demonstrate the usefulness of dynamic programming, an analytical method is developed for evaluating the relationship among existing numerous connector designs to find the optimum configuration. The data utilized in the study were generated from 900 flanges designed for six subsystems of the S-1B stage of the Saturn 1B space carrier vehicle.
Prasthofer, W. P.↗