Learing from Internet
The idea of the project is to develop new algorithms to allow a robot
to learn how to recognize and how to find an unknown object in a known
environment. The user only gives the name of the object to a robot and
expects the robot to bring that object back to her/him. It is assumed
that the robot has a map of the environment and has access to
internet. The task requires to solve several challengues:
- How to search for the object. This in itself involves other
tasks:
- Find which is the most probable place to find the object: For
this task we applied a weighted combination of different of
different sources (Google, DBPedia, ConceptNet, Word2Vec)
- Decide how to search within a room: In this case, we assumed
that the object is on top of a flat surface (table), and the robot
knows where are the tables within each room. It uses an RGB-D
sensor to find objects on tables.
- Define a search strategy considering the most probable places
and the map of the environment: We used a heuristic search strategy
that considers the most probable, closer and smaller room in the
environment.
- How to learn a model of the object: We use an interface with the
user to define the particular object to search. E.g., if the user
says "mouse" this could a small rodent or a computer input
device. After showing images related with the object, the user
selects the "right" one and 100 images and downloaded from
internet. We use a pre-trained deep neural netowork (inception v3),
trim the last layers and used the 2,048 values as input to a SVM.
- How to localize and grab the object: To grab the object we used
the information from the RGB-D camera and send a robotic arm to
that point to grab the object.
The whole process of deciding where to search, learn a model of the
object and grabbing the object takes on average less that 3 minutes
(see videos).
Related Papers
- V. Lobato-Rios, A.C. Tenorio-Gonzalez, E.F. Morales (2018). Fast
Learning for Accurate Object Recognition Using a Pre-trained Deep
Neural Network. In Proc. of the Mexican International de Inteligencia
Artificial (MICAI). LNAI - 10632, Springer.
PDF
- R. Izquierdo-Cordova, E.F. Morales, L.E. Sucar,
R. Murrieta-Cid (2016). Searching Objects in Known Environments:
Empowering Simple Heuristic Strategies. RoboCup International
Symposium.
PDF
- R. Izquierdo-Cordova, E.F. Morales, L.E. Sucar (2016). Object
Location Estimation in Domestic Environments through Internet
Queries. Workshop on Autonomous Mobile Service Robots at
International Joint Conference on Artificial Intelligence
(IJCAI).
PDF
Some Videos
- Demo of how to find an object (video1 and video2 need to be played
at the same time)
- Demo showing the robot grabbing the object and returning to the
place where the user asked for it
(video1)
This project was funded by CONACYT under grant No. 250938.