AI planning and learning for nonlinear control applications

Planning methods for optimal control use a model of the system and the reward function to derive an optimal control strategy. Here we will consider in particular optimistic planning, a recent predictive approach that optimistically explores  possible action sequences from the current state. Due to generality in the dynamics and objective functions that it can address, it has a wide range of potential applications to problems in nonlinear control. Reinforcement learning methods preserve this generality, and can additionally handle problems in which the model and even the reward function is unknown.

In this project the student will work either on fundamental developments in optimistic planning and reinforcement learning, on their real-time application to nonlinear control, or a combination of the two. Fundamental directions include e.g. novel algorithms for switching, hybrid, and continuous-input systems, analyzing the near-optimality of these methods, as well as the analysis of the stability properties of the optimal or near-optimal solutions. The application axis includes real-time results for the control of some nonlinear systems available in our lab, such as the Quanser rotational inverted pendulum or the Cyton Gamma robotic arm. We will start with existing real-time control implementations of optimistic planning methods for discrete and continuous inputs; the movie below shows some existing results.

This project is suitable for students who are motivated and able to invest themselves fully. Complementary skillsets are required, so we are looking both for people who are good at C/C++ and Matlab programming, as well as for people who enjoy more analyical, math challenges. The project involves a cooperation with the University of Lorraine, Nancy, France, so Erasmus stays there are possible.

Start date (duration): 
January, 2020
Project status and type: 

Inverted Rotary Pendulum