Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “stochastic optimal control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Convergence Analysis for an Online Data-Driven Feedback Control Algorithm

This paper presents convergence analysis of a novel data-driven feedback control algorithm designed for generating online controls based on partial noisy observational data. The algorithm comprises a particle filter-enabled state estimation component, estimating the controlled system’s state via indirect observations, alongside an efficient stochastic maximum principle-type optimal control solver. By integrating weak convergence techniques for the particle filter with convergence analysis for the stochastic maximum principle control solver, we derive a weak convergence result for the optimization procedure in search of optimal data-driven feedback control. Numerical experiments are performed to validate the theoretical findings.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1980

Optimal regulation in systems with stochastic time sampling

An optimal control theory that accounts for stochastic variable time sampling in a distributed microprocessor based flight control system is presented. The theory is developed by using a linear process model for the airplane dynamics and the information distribution process is modeled as a variable time increment process where, at the time that information is supplied to the control effectors, the control effectors know the time of the next information update only in a stochastic sense. An optimal control problem is formulated and solved for the control law that minimizes the expected value of a quadratic cost function. The optimal cost obtained with a variable time increment Markov information update process where the control effectors know only the past information update intervals and the Markov transition mechanism is almost identical to that obtained with a known and uniform information update interval.

Montgomery, R. C.↗
DOE OSTI2024

Binary Quantum Control Optimization with Uncertain Hamiltonians

Optimizing the controls of quantum systems plays a crucial role in advancing quantum technologies. The time-varying noises in quantum systems and the widespread use of inhomogeneous quantum ensembles raise the need for high-quality quantum controls under uncertainties. In this paper, we consider a stochastic discrete optimization formulation of a discretized binary optimal quantum control problem involving Hamiltonians with predictable uncertainties. We propose a sample-based reformulation that optimizes both risk-neutral and risk-averse measurements of control policies, and solve these with two gradient-based algorithms using sum-up-rounding approaches. Furthermore, we discuss the differentiability of the objective function and prove upper bounds of the gaps between the optimal solutions to binary control problems and their continuous relaxations. We conduct numerical simulations on various sized problem instances based on two applications of quantum pulse optimization; we evaluate different strategies to mitigate the impact of uncertainties in quantum systems. In conclusion, we demonstrate that the controls of our stochastic optimization model achieve significantly higher quality and robustness compared with the controls of a deterministic model.

conditional value-at-risk (CVaR)↗
NASA NTRS1975

Design of optimal partial state feedback controllers for linear systems in stochastic environments

The problem of obtaining an optimal control law, which is constrained to be a feedback of the available measurements, is considered for both continuous and discrete time linear systems subjected to additive white process noise and measurement noise. Necessary conditions are obtained for minimizing a quadratic performance function for both finite and infinite duration cases. The feedback gain matrices are constrained to be constant for the infinite duration cases. For all the cases considered, algorithms are derived for generating sequences of feedback gain matrices which successively improve the performance function. Computational aspects are discussed via application to two continuous time processes, including a helicopter/slung load system subjected to measurement noise and random wind gust input.

Joshi, S. M.↗
NASA NTRS2011

Optimal Control via Self-Generated Stochasticity

The problem of global maxima of functionals has been examined. Mathematical roots of local maxima are the same as those for a much simpler problem of finding global maximum of a multi-dimensional function. The second problem is instability even if an optimal trajectory is found, there is no guarantee that it is stable. As a result, a fundamentally new approach is introduced to optimal control based upon two new ideas. The first idea is to represent the functional to be maximized as a limit of a probability density governed by the appropriately selected Liouville equation. Then, the corresponding ordinary differential equations (ODEs) become stochastic, and that sample of the solution that has the largest value will have the highest probability to appear in ODE simulation. The main advantages of the stochastic approach are that it is not sensitive to local maxima, the function to be maximized must be only integrable but not necessarily differentiable, and global equality and inequality constraints do not cause any significant obstacles. The second idea is to remove possible instability of the optimal solution by equipping the control system with a self-stabilizing device. The applications of the proposed methodology will optimize the performance of NASA spacecraft, as well as robot performance.

Zak, Michail↗
NASA NTRS1996

Integrated Control Using the SOFFT Control Structure

The need for integrated/constrained control systems has become clearer as advanced aircraft introduced new coupled subsystems such as new propulsion subsystems with thrust vectoring and new aerodynamic designs. In this study, we develop an integrated control design methodology which accomodates constraints among subsystem variables while using the Stochastic Optimal Feedforward/Feedback Control Technique (SOFFT) thus maintaining all the advantages of the SOFFT approach. The Integrated SOFFT Control methodology uses a centralized feedforward control and a constrained feedback control law. The control thus takes advantage of the known coupling among the subsystems while maintaining the identity of subsystems for validation purposes and the simplicity of the feedback law to understand the system response in complicated nonlinear scenarios. The Variable-Gain Output Feedback Control methodology (including constant gain output feedback) is extended to accommodate equality constraints. A gain computation algorithm is developed. The designer can set the cross-gains between two variables or subsystems to zero or another value and optimize the remaining gains subject to the constraint. An integrated control law is designed for a modified F-15 SMTD aircraft model with coupled airframe and propulsion subsystems using the Integrated SOFFT Control methodology to produce a set of desired flying qualities.

Halyo, Nesim↗
DOE OSTI2021

Next-Cycle Optimal Fuel Control for Cycle-to-Cycle Variability Reduction in EGR-Diluted Combustion

In this simulation study, cycle-to-cycle fuel control was used to reduce CCV by injecting additional fuel in operating conditions with sporadic misfires and partial burns. An optimal control policy was proposed that utilizes 1) a physics-based model that tracks in-cylinder gas composition and 2) a one-step-ahead prediction of the combustion efficiency based on a kernel density estimator. The optimal solution, however, presents a tradeoff between the reduction in combustion CCV and the increase in fuel injection quantity required to stabilize the charge. Such a tradeoff can be ad- just by a single parameter embedded in the cost function.

Maldonado, BryanP. [Oak Ridge National Lab. (ORNL)↗
NASA NTRS1990

Stochastic ordering properties and optimal routing control for a class of finite capacity queueing systems

The problem of routing jobs to parallel queues with identical exponential servers and unequal finite buffer capacities is considered. Stochastic ordering and weak majorization properties on critical performance measures are established by means of event-driven inductions. In particular, it is shown that the intuitive 'join the shortest non-full queue' (SNQ) policy is optimal with respect to an overall function that accounts for holding and blocking costs. Moreover, the buffer allocation problem is solved by proving the intuitive result that, for a fixed total buffer capacity, the optimal allocation scheme is the one in which the difference between the maximum and minimum queue capacities is minimized, i.e., becomes either 0 or 1.

Towsley, Don↗
NASA NTRS1985

Evaluation of performance characteristics for a space antenna system subjected to stochastic disturbances

Control system synthesis for a large space antenna system is investigated based on stochastic linear optimal control techniques and the minimization of a quadratic Gaussian performance index. Parametric studies indicate that suitable combinations of plant and sensor noise characteristics, and state weighting matrices, can be found to meet the mission RMS pointing requirements. Using a combination of the Kalman filter and linear feedback, an optimal control law for the finite element model of the Hoop/Column structural system without damping is obtained. Removing the hoop-mounted actuator resulted in an increase in the RMS errors, an increased control effort, and an increase in the least damped modal time constant. Removing the hoop-mounted sensor resulted in an RMS error increase and estimator performance degradation, with less system RMS performance degradation than noted for removal of the hoop-mounted actuator.

Ananthakrishnan, S.↗
DOE OSTI2023

Stochastic scheduling for commercial building cooling systems: considering uncertainty in zone temperature prediction

Here, this paper presents the first attempt to address the uncertainty in zone temperature prediction with stochastic optimization. The uncertain zone temperature is a process uncertainty and has not been considered in the existing stochastic optimization for building control. To fill this gap, we proposed a novel formulation of stochastic optimization to handle process uncertainty in building control. Specifically, we first examined the accuracy of a typical linear model for predicting zone temperature. We then formulated the scheduling of the building cooling system as a stochastic optimization problem over a 24-hour look-ahead period to minimize the electricity cost of the studied building cooling system. After that, we applied the proposed stochastic load scheduling (SLS) to a direct expansion (DX) cooling system that serves a medium office building. Through simulation with a detailed building energy simulation software, EnergyPlus, we evaluated the operational cost and the thermal comfort compared with a deterministic load scheduling. The operation cost of scheduling was found to vary with the level of zone temperature prediction uncertainty. The proposed SLS can mitigate the impacts of uncertain zone temperature predictions on both operational cost and thermal comfort. The evaluation results indicate that the proposed SLS works better when the uncertainty level is more significant.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
NASA NTRS1984

Algorithms for output feedback, multiple-model, and decentralized control problems

The optimal stochastic output feedback, multiple-model, and decentralized control problems with dynamic compensation are formulated and discussed. Algorithms for each problem are presented, and their relationship to a basic output feedback algorithm is discussed. An aircraft control design problem is posed as a combined decentralized, multiple-model, output feedback problem. A control design is obtained using the combined algorithm. An analysis of the design is presented.

Halyo, N.↗
NASA NTRS1994

Design and evaluation of a Stochastic Optimal Feed-forward and Feedback Technology (SOFFT) flight control architecture

This paper describes the design and evaluation of a stochastic optimal feed-forward and feedback technology (SOFFT) control architecture with emphasis on the feed-forward controller design. The SOFFT approach allows the designer to independently design the feed-forward and feedback controllers to meet separate objectives and then integrate the two controllers. The feed-forward controller has been integrated with an existing high-angle-of-attack (high-alpha) feedback controller. The feed-forward controller includes a variable command model with parameters selected to satisfy level 1 flying qualities with a high-alpha adjustment to achieve desired agility guidelines, a nonlinear interpolation approach that scales entire matrices for approximation of the plant model, and equations for calculating feed-forward gains developed for perfect plant-model tracking. The SOFFT design was applied to a nonlinear batch simulation model of an F/A-18 aircraft modified for thrust vectoring. Simulation results show that agility guidelines are met and that the SOFFT controller filters undesired pilot-induced frequencies more effectively during a tracking task than a flight controller that has the same feedback control law but does not have the SOFFT feed-forward control.

Ostroff, Aaron J.↗
NASA NTRS1974

Control of Finite-State, Finite Memory Stochastic Systems

A generalized problem of stochastic control is discussed in which multiple controllers with different data bases are present. The vehicle for the investigation is the finite state, finite memory (FSFM) stochastic control problem. Optimality conditions are obtained by deriving an equivalent deterministic optimal control problem. A FSFM minimum principle is obtained via the equivalent deterministic problem. The minimum principle suggests the development of a numerical optimization algorithm, the min-H algorithm. The relationship between the sufficiency of the minimum principle and the informational properties of the problem are investigated. A problem of hypothesis testing with 1-bit memory is investigated to illustrate the application of control theoretic techniques to information processing problems.

Sandell, Nils R.↗
NASA NTRS1989

The insertion of human dynamics models in the flight control loops of V/STOL research aircraft. Appendix 2: The optimal control model of a pilot in V/STOL aircraft control loops

An overview is presented of research work focussed on the design and insertion of classical models of human pilot dynamics within the flight control loops of V/STOL aircraft. The pilots were designed and configured for use in integrated control system research and design. The models of human behavior that were considered are: McRuer-Krendel (a single variable transfer function model); and Optimal Control Model (a multi-variable approach based on optimal control and stochastic estimation theory). These models attempt to predict human control response characteristics when confronted with compensatory tracking and state regulation tasks. An overview, mathematical description, and discussion of predictive limitations of the pilot models is presented. Design strategies and closed loop insertion configurations are introduced and considered for various flight control scenarios. Models of aircraft dynamics (both transfer function and state space based) are developed and discussed for their use in pilot design and application. Pilot design and insertion are illustrated for various flight control objectives. Results of pilot insertion within the control loops of two V/STOL research aricraft (Sikorski Black Hawk UH-60A, McDonnell Douglas Harrier II AV-8B) are presented and compared against actual pilot flight data. Conclusions are reached on the ability of the pilot models to adequately predict human behavior when confronted with similar control objectives.

Zipf, Mark E.↗
NASA NTRS1972

Adaptive control of stochastic linear systems with unknown parameters

The problem of optimal control of linear discrete-time stochastic dynamical system with unknown and, possibly, stochastically varying parameters is considered on the basis of noisy measurements. It is desired to minimize the expected value of a quadratic cost functional. Since the simultaneous estimation of the state and plant parameters is a nonlinear filtering problem, the extended Kalman filter algorithm is used. Several qualitative and asymptotic properties of the open loop feedback optimal control and the enforced separation scheme are discussed. Simulation results via Monte Carlo method show that, in terms of the performance measure, for stable systems the open loop feedback optimal control system is slightly better than the enforced separation scheme, while for unstable systems the latter scheme is far better.

Ku, R. T.↗