Probabilistic Graphical Models: Principles and Applications

Web page: ccc.inaoep.mx/~esucar/Clases-mgp/mgp.html
Dr. L. Enrique Sucar, Ciencias Computacionales, INAOE

1. Objective

Probabilistic graphical models have become a powerful set of techniques used in several domains. This course provides a general introduction to probabilistic graphical models (PGMs) from an engineering perspective. It covers the fundamentals of the main classes of PGMs: Bayesian classifiers, hidden Markov models, Bayesian networks, dynamic and temporal Bayesian networks, Markov random fields, influence diagrams,  Markov decision processes, and relational and causal PGMs; including representation, inference, and learning principles for all the techniques. Realistic applications for each type of model are also covered.

2. Outline

Part I  Fundamentals

1. Introduction

2. Probability theory

3. Graph theory

Part II Probabilistic Models

4. Bayesian classifiers

5. Hidden Markov models

6. Markov random fields

7. Bayesian networks: representation and inference

8. Bayesian networks: learning

9. Dynamic and temporal Bayesian networks

Part III Decision models

10. Decision graphs

11. Markov decision processes

Part IV Relational and causal models

12. Relational probabilistic graphical models

13. Graphical causal models

3. Bibliography

Text book:

• L.E. Sucar, Probabilistic Graphical Models: Principles and Applications, Springer, 2015.
  

Additional references:

• PGMs:
• D. Koller, N. Friedman, Probabilistic Graphical Models, MIT Press, 2009.
• L.E. Sucar, E. Morales, J. Hoey, Decision Theory Models for Applications in Artificial Intelligence: Concepts and Solutions, IGI-Global, 2012.
• Probability:
  
• L. Wasserman, All of Statistics, Springer, 2004.
• E.T. Jaynes, Probability Theory: The Logic of Science, Cambridge, 2003.
  
• BNs:
  
• J. Pearl, Probabilistic Reasoning in Intelligent Systems, Morgan-Kaufmann, 1988.
• R. Neapolitan, Probabilistic Reasoning in Expert Systems, Addison-Wesley, 1990.
• R. Neapolitan, Learning Bayesian Networks, Pearson, 2004.
• A. Darwiche, Modeling and Reasoning with Bayesian Networks, Cambridge, 2009.
  
• Statistical prespective:
  
• C. Borgelt, R. Kruse, Graphical Models, Wiley, 2002.
• Whittaker, Graphical Models, Wiley, 1990.
• Differenet approaches on uncertainty:
  
• Shafer y Pearl (Eds.), Readings in Uncertain Reasoning, Morgan-Kaufmann, 1990.
• Decision models:
  
• F. Jensen, Bayesian Networks and Decision Graphs, Springer, 2001.
• R. Borrás, Análisis de incertidumbre y riesgo para la toma de decisiones, Orizaba, Mexico:
  Comunidad Morelos, 2001.
  
• M Putterman, Markov Decision Processes, Wiley, 1993.
• Classifiers, learning:
  
• D. Michie, D.J. Spiegelhalter, C.C. Taylor, Machine Learning, Neural and Statistical Classification, Elis Horwood, 1994.
• B. Sierra Araujo (Ed.), Aprendizaje automático: conceptos básicos y avanzados, Pearson, 2006.
• Causality:
  
• J. Pearl, D. Mackenzie, The Book of Why. Basic Books, 2018.
  
• J. Pearl, Causality: models, reasoning and inference. Cambridge, 2009.
  
• P. Sprites, C. Glaymour, R Scheines, Causation, Prediction and Search. MIT Press, 2001.
• Applications:
• O. Pourret, P. Naim, B. Marcot (Eds.), Bayesian Networks: A Practical Guide to Applications, Wiley, 2008
  

4. Main journals and conferences

• International Journal of Approximate Reasoning
• International Joint Conference on Artificial Intelligence (IJCAI)
  
• Uncertainty in Artificial Intelligence (UAI)
• Probabilistic Graphical Models (PGM)
• FLAIRS - Uncertain Reasoning Track
  
• Information Processing and Management of Uncertainty (IPMU)
• Symbolic and Quantitative Approaches to Uncertainty (ECSQAU)
• Neural Information Processing (NIPS)
  

5. Course policies

5.1 Prerrequisites

There are no particular requisites for the course. Basic knowledge of math and computing is assumed.

5.2 Course notes

All class notes will be available in the course web page. 

5.3 Homework, projects and exams

Homeworks: will be assigned each week, which in general are not required to be hand in. I will ask some of the students to solve the excercises in  class. In a few cases I will tell you if you have to deliver a report. Homeworks are individual.

Final Project: by the midde of the course a project will be assigned (could be individual or in two persons teams). The project should implement / apply a PGM technique to certain problem. It consists of 3 phases:

1. Proposal: short (1 page) description of the project objective, methodology and expected results.
   
2. Preliminary: a description of the work in the format of a scientific paper including introduction, related work, methodology, results and conclusions (max 10 pages in Springer LN format).
   
3. Final report: final paper taking into accound the feedback.
   

Exam: there will a mid term exam which will include the topics cover until the previous class. It consists of two parts: theory and problems.

5.4 Evaluation

• Homework - 20%
  
• Exam - 40 %
• Final project - 40 %

5.5 Individual advise

By appointment

5.6 Professor

Dr. L. Enrique Sucar S. (office / extension 8208)

e-mail: esucar (AT) inaoep.mx
web page: http://ccc.inaoep.mx/~esucar