The Segmented and Annotated IAPR TC-12 Benchmark

Sample image from the SAIAPR TC12 collection

An image collection composed of 20,000 images that have been manually segmented and annotated. This benchmark is intended for the evaluation of region labeling methods and for studying the impact of automatic image annotation in the task of multimedia image retrieval.

The resource comprises ~100,000 labeled regions, visual features extracted from the regions, spatial relationships between regions, segmentation masks and regions labeled with a conceptual hierarchy. The SAIAPR TC12 collection is publicly available for research purposes from this link.


The following archive contains the complete SAIAPR TC-12 Benchmark, which is now available free of charge and without any copyright restrictions:

There is a mirror here:

Important: Because of its size, the benchmark has been divided and compressed into 3 files (using winrar®), please follow the following instructions:
1.- download the following three files:

We are very grateful with Thomas Deselaers for his support on the storage and management of the data.

2.- copy these *.rar files into a directory
3.- unpack the first part-file saiaprtc12ok.part1.rar using winrar® or your preferred software.

Related resources

  • Conceptual hierarchy. This file contains the set of Figures that comprise the annotation hierarchy developed for the annotation of the IAPR TC12 collection.


Please cite the following paper for making reference to the SAIAPR TC12 benchmark:

  • The Segmented and Annotated IAPR-TC12 Benchmark. Hugo Jair Escalante, Carlos Hernandez, Jesus A. Gonzalez, Aurelio Lopez, Manuel Montes, Eduardo Morales, Enrique Sucar, Luis Villasenor, and Michael Grubinger. Computer Vision and Image Understanding Journal, 114(4):419–428, 2010.

More information about the collection can be found in the following documents:

  • Hugo Jair Escalante, Manuel Montes and Enrique Sucar. On the SAIAPR TC-12 Benchmark. In Peter Dunker, Judith Liebetrau, Stefanie Nowak, Henning Muller and Panagiotis Vlamos Editors, Proceedings of the 2009 Theseus/ImageCLEF Workshop, pp. 44–51, ISBN: 978-3-8396-0023-8, Corfu Greece, September 29, 2009.
  • Hugo Jair Escalante, Carlos Hernández-Gracidas, Jesús A. González, Aurelio López, Manuel Montes, Eduardo Morales, Enrique Sucar, Luis Villaseñor. Segmenting and annotating the IAPR-TC12 Benchmark. Technical Report INAOE CCC-08-005, November 1, 2008.
  • Michael Grubinger. Analysis and Evaluation of Visual Information Systems Performance. PhD Thesis. School of Computer Science and Mathematics Faculty of Health, Engineering and Science Victoria University, Melbourne, Australia, 2007.

This work was done while I was a PhD student at INAOE, all of the members of the TIA research group collaborated in the project. The work was done in collaboration with Michael Grubinger.

Particle Swarm Model Selection

Particle Swarm Model Selection (PSMS) is a tool for the automatic selection of full classification models; where a full model is that composed of methods for data preprocessing, feature selection and classification. PSMS searches the space of full models that can be generated with a machine learning toolbox with the goal of minimizing the (cross-validation) missclassification error rate. PSMS uses particle swarm optimization for exploring such a search space.

PSMS is currently implemented in the CLOP toolbox (an outdated version of PSMS is distributed with CLOP). It has been successfully applied in several domains, including the data sets considered in the ALvsPK machine learning challenge.


  • The implementation of PSMS described in the JMLR paper can be downloaded from this link.
  • The latest PSMS implementation can be downloaded from this link.
  • See the resource EPSMS above (under construction!) for the PSMS implementation that can be used for generating ensembles.


Please cite the following paper for making reference to PSMS:

More information about PSMS can be found in the following papers:

  • Hugo Jair Escalante, Manuel Montes and Enrique Sucar. Ensemble Particle Swarm Model Selection. Proceedings of the World Congress on Computational Intelligence, pp. 1814–1821, IEEE, Barcelona, Spain, July 18-23, 2010 Received the IJCNN2010 - Best Student Paper Award.
  • Hugo Jair Escalante, Enrique Sucar, Manuel Montes. Particle Swarm Optimization for Classifier Selection. Proccedings of the 9no. Encuentro de Investigacion, INAOE, pp. 143–146, Puebla, Mexico, November, 4-5, 2008.
  • Hugo Jair Escalante, Manuel Montes y Gomez, and Luis Enrique Sucar. PSMS for Neural Networks on the IJCNN 2007 Agnostic vs Prior Knowledge Challenge. In IJCNN’07: IEEE-INNS Proceedings of 20th International Joint Conference on Neural Networks, pp. 1191–1197, Orlando, Florida, USA, August 12-17, 2007.

This work was done while I was a PhD student at INAOE, in close collaboration with Isabelle Guyon.

GRASP with Path Relinking for the Commercial Territory Design Problem

We (Roger Ríos and myself) provide the Matlab code we developed for the commercial territory design (CTD) problem. We proposed a GRASP heuristic augmented with Path Relinking for the CTD problem. The data and code are from this link.

This work was done while I was a assistant professor at PISIS - UANL, in close collaboration with Roger Ríos Mercado.

Related papers: