Engineering Papers⌕ Search

Engineering topics

Zilberstein, Shlomo

Publications and source records attributed to Zilberstein, Shlomo.

A sampling-based optimization approach to handling environmental uncertainty for a planetary lander

Planning for unknown environments presents a number of technical challenges. The planner must ensure robustness to unknown phenomena and manage unpredictable variation in execution, all while operating in a capacity that maximizes its objective. Productivity in the face of these challenges re-quires an integrated approach to planning and execution that is capable of accomplishing goals, reacting to variation, and maximizing overall utility. We examine this problem in the context of a Europa Lander concept mission. We model the problem as a hierarchical task network, framing it as a utility maximization problem constrained on a depletable energy resource. We propose an uncertainty–sensitive deterministic planning framework that utilizes periodic replanning to better handle model uncertainty and variable execution. We demonstrate the efficacy of our framework through simulations of a Europa Lander concept mission in which our algorithm out-performs several baseline approaches in both utility maximization and robustness

Zilberstein, Shlomo↗

A Sampling-Based Optimization Approach to Handling Environmental Uncertainty for a Planetary Lander

Planning for unknown environments presents a number of technical challenges. The planner must ensure robustness to unknown phenomena and manage unpredictable variation in execution, all while operating in a capacity that maximizes its objective. Productivity in the face of these challenges re-quires an integrated approach to planning and execution that is capable of accomplishing goals, reacting to variation, and maximizing overall utility. We examine this problem in the context of a Europa Lander concept mission. We model the problem as a hierarchical task network, framing it as a utility maximization problem constrained on a depletable energy resource. We propose an uncertainty–sensitive deterministic planning framework that utilizes periodic replanning to better handle model uncertainty and variable execution. We demonstrate the efficacy of our framework through simulations of a Europa Lander concept mission in which our algorithm out-performs several baseline approaches in both utility maximization and robustness

Chien, Steve↗

A Decision-Theoretic Approach to Autonomous Planetary Rover Control

The report discusses the: Decentralized Control of Markov Decision Processes. Study the complexity of decentralized control of Markov decision processes, and develop algorithms for finding optimal control policies. Scheduling Contract Algorithms. Develop an optimal method for scheduling runs of a contract anytime algorithm (one that takes the deadline as input) in situations where the deadline is unknown, multiple problem instances must be solved, and a multi-processor machine is available. Planetary Rover Control as a Markov Decision Process.Use the Markov decision process framework to formalize and solve problems in planetary rover control. Adaptive Peer Selection. Use reinforcement learning to maximize the expected down-load speed for a client in a peer-to-peer file sharing system.

Zilberstein, Shlomo↗

Reinforcement Learning for Weakly-Coupled MDPs and an Application to Planetary Rover Control

Weakly-coupled Markov decision processes can be decomposed into subprocesses that interact only through a small set of bottleneck states. We study a hierarchical reinforcement learning algorithm designed to take advantage of this particular type of decomposability. To test our algorithm, we use a decision-making problem faced by autonomous planetary rovers. In this problem, a Mars rover must decide which activities to perform and when to traverse between science sites in order to make the best use of its limited resources. In our experiments, the hierarchical algorithm performs better than Q-learning in the early stages of learning, but unlike Q-learning it converges to a suboptimal policy. This suggests that it may be advantageous to use the hierarchical algorithm when training time is limited.

Bernstein, Daniel S.↗

Self-Directed Cooperative Planetary Rovers

The project is concerned with the development of decision-theoretic techniques to optimize the scientific return of planetary rovers. Planetary rovers are small unmanned vehicles equipped with cameras and a variety of sensors used for scientific experiments. They must operate under tight constraints over such resources as operation time, power, storage capacity, and communication bandwidth. Moreover, the limited computational resources of the rover limit the complexity of on-line planning and scheduling. We have developed a comprehensive solution to this problem that involves high-level tools to describe a mission; a compiler that maps a mission description and additional probabilistic models of the components of the rover into a Markov decision problem; and algorithms for solving the rover control problem that are sensitive to the limited computational resources and high-level of uncertainty in this domain.

Zilberstein, Shlomo↗

Decision-Theoretic Control of Planetary Rovers

Planetary rovers are small unmanned vehicles equipped with cameras and a variety of sensors used for scientific experiments. They must operate under tight constraints over such resources as operation time, power, storage capacity, and communication bandwidth. Moreover, the limited computational resources of the rover limit the complexity of on-line planning and scheduling. We describe two decision-theoretic approaches to maximize the productivity of planetary rovers: one based on adaptive planning and the other on hierarchical reinforcement learning. Both approaches map the problem into a Markov decision problem and attempt to solve a large part of the problem off-line, exploiting the structure of the plan and independence between plan components. We examine the advantages and limitations of these techniques and their scalability.

Zilberstein, Shlomo↗

Real-time robot deliberation by compilation and monitoring of anytime algorithms

Anytime algorithms are algorithms whose quality of results improves gradually as computation time increases. Certainty, accuracy, and specificity are metrics useful in anytime algorighm construction. It is widely accepted that a successful robotic system must trade off between decision quality and the computational resources used to produce it. Anytime algorithms were designed to offer such a trade off. A model of compilation and monitoring mechanisms needed to build robots that can efficiently control their deliberation time is presented. This approach simplifies the design and implementation of complex intelligent robots, mechanizes the composition and monitoring processes, and provides independent real time robotic systems that automatically adjust resource allocation to yield optimum performance.

Zilberstein, Shlomo↗