Learning Tasks in Robotics

Learing Tasks in Robotics

The idea of the project is to develop new algorithms to allow a user to teach a robot how to perform a new task. The user gives traces of how to perform the task to the robot, these traces are transformed into a relational representation and then a reinforcement learning algorithm is used to improve over the given traces. The user is allowed to provide feedback to the robot during the learning proces. The project was divided into three stages that correspond to different ways of giving the traces of the new task to the robot:

Learning by controling a robot how to perform a task with a joystick. The main contributions were in: (i) translating the low-level sensor information from the robot into a high-level representation, (ii) creating a novel mechanism to produce continuos action in the learned policy for the robot, and (iii) testing the approach with a real robot (master's thesis of Julio Hernandez)
Instructing a robot how to perform a new task with voice commands. The main contributions were in: (i) allowing the user to intervene during the learning process of the reinfocement learning algorithm with voice commands, creating a kind of dynamic reward shaping function, and (ii) having all the interctions for teaching the new task with voice commands. (master's thesis of Ana C.Tenorio)
Showing how to perform a task to a robot. This thesis, incorporated a Kinect sensor to show simple manipulation tasks to a robot arm as well as voice commands during the learning process, and was tested on a manipulator (master's thesis of Luis Adrian Leon)

Related Papers

A. León, E.F. Morales, L. Altamirano, J.R. Ruiz (2011). Teaching a Robot New Tasks through Imitation and Feedback. In Proc. Workshop: New Developments in Imitation Learning, as part of the Internacional Conference on Machine Learning (ICML-2011). PDF
A. León, E.F. Morales, L. Altamirano, J.R. Ruiz (2011). Teaching a robot to perform task through imitation and on-line feedback. In Proc. of the 16th. Iberoamerican Congress on Pattern Recognition} CIARP-2011 (accepted). PDF
J. Hernández, E.F. Morales (2010). Relational Reinforcement Learning with Continuous Actions by Combining Behavioral Cloning and Locally Weighted Regression. Journal of Intelligent Systems and Applications, 2:69-79. PDF
A.C. Tenorio-Gonzalez, E.F. Morales and L. Villaseñor-Pineda (2010). Dynamic Reward Shaping: Training a robot by voice. In Proceedings of the Ibero--American Conference on Artificial Intelligence (IBERAMIA-2010) (to be published). PDF
A.C. Tenorio-Gonzalez, E.F. Morales and L. Villaseñor-Pineda (2010). Teaching a Robot to Perform Tasks with Voice Commands. In Proceedings of the Mexican International Conference on Artificial Intelligence (MICAI-2010) (to be published). PDF
J. Hernández Zaragoza, E.F. Morales (2009). A two-stage relational reinforcement learning with continuous actions for real service robots. In Proc. of the International Conference on Artificial Intelligence (MICAI-09), pgs. 337-348. Lecture Notes in Artificial Intelligence (LNAI-5845), Springer-Verlag. A. Hernández, R. Monroy, C.A. Reyes (Eds.). Best Paper Award - Third Place PDF
B. Vargas-Govea, E.F. Morales (2009). Learning relational grammars from sequences of actions. In Proc. of the Conferencia Iberaamericana en Reconocimiento de Patrones (CIARP-09). Lecture Notes in Artificial Intelligence (LNAI-5856), E. Nayro-Corrochano and J.O. Eklundh (Eds.) Springer-Verlag, pgs. 892-900. PDF
B. Vargas, E.F. Morales (2009). Learning Navigation Teleo-Reactive Programs using Behavioural Cloning. In Proc. IEEE International Conference on Mechatronics (ICM-2009). PDF
B. Vargas-Govea, E.F. Morales (2008). Solving Navigation Tasks with Learned Teleo-Reactive Programs. IROS-2008 (poster) PDF

Some Videos

Demo of voice interaction from the user while learning
Demo of how to grab a cup. Examples and Final Policy
Demo from a single demonstration of how to stack blocks (Demonstration and Repetition)

This project is funded by CONACYT under grant No. 84162.