Introduction to Graphical Models
In this post I would like to make a map of understanding graphical models especially for a beginner. There are a lot of publications on this topic, and they are quite advanced relative to a beginner. I realize that it is usually a pretty hard time for a beginner on how to get started on this topic. So the objective of this post is to provide some understanding maps to graphical models for a beginner by pointing out to some good tutorial papers, books and videos.
We can consider HMM a special case of Bayesian networks, however, you may find that HMM is a good point to start in order to see the overview of how to make an inference, how to learn the model parameters, how to learn the model structure, etc. Once we mastered HMM already, then we will find that Bayesian networks can be done the similar way too.
- “A Brief Introduction to Graphical Models and Bayesian Networks” by Kevin Murphy [link]: The website links to a lot of good tutorial papers and books, and this is one of the good intuitive paper to discuss about what graphical models can do?
- Z. Ghahramani, “An introduction to hidden markov models and bayesian networks,” pp. 9-42, 2002. ; This is a concise paper discussion HMMs as a special case of Bayesian networks. The paper covers all necessary topic related to BNs. Personally, this paper should be the first technical paper to read on HMMs and BNs.
- Kschischang, Frank R.; Brendan J. Frey and Hans-Andrea Loeliger (2001). “Factor Graphs and the Sum-Product Algorithm“. IEEE Transactions on Information Theory 47 (2): 498–519.: The paper discusses graphical models in the unified view. Very good to read.
- Chapter 8 of the book “Pattern Recognition and Machine Learning” by Christopher Bishop [pdf]: Graphical models (BNs and MRFs) are concisely explained in the chapter with inference methods (both exact and approximate). Also the chapter covers the unified view of graphical models using factor graphs. Even though the book does not explain every thing in details, it does a good job on giving you pointers to other good resources.
- “Bayesian Networks and Decision Graphs” by Finn Jensen [link]: This book explains BNs very clearly, and discusses BNs from the very basic probability theory, graph theory, representation using BNs, inference, parameter learning, and structure learning.
Check list for graphical models:
- What is graphical model?
- How to formulate the problem to a graphical model
- How to make an inference given a model
- Exact inference
- Approximate inference
- How to learn the model parameters
- Structure learning
- Model selection (BIC, MDL)