There are some fascinating seminars sponsored by UW, and most of them are recorded:
Every Tuesday 3:30 pm
Yahoo! Machine Learning Seminar
Every Tuesday from 12 – 1 pm
UWTV: Research/Technology/Discovery Channel
Broadcast all the new findings, research, technology for free!!
Gaussian Mixture Models (GMMs) are very popular in broad area of applications because its performance and its simplicity. However, it is still an open problem on how to determine the number of Gaussian components in a GMM. One simple solution to this problem is to use Bayesian Information Criteria (BIC) to penalize the complexity of the GMM. That is, the cost function of BIC-GMM is composed of 2 parts: 1) log-likelihood and 2) complexity penalty term. Consequently, the final GMM would be a model that can fit the data well, but not “overfitting” the model in BIC sense. There are tons of tutorials on the internet. Here I would like to share my MATLAB code for demo.
Note that Variational Bayes GMM (VBGMM) can also solve this problem in a different flavor and is worth to study and compare with GMM-BIC. I also provided some details of the derivations of VBGMM here.
This note is aimed for a beginner for variational EM algorithm. The note contains the following contents:
- motivation of variational EM
- Derivation in details
- Geometrical meaning of using KL divergence in variational EM
- Traditional EM vs. variational EM
Please download the note here. Your comments and suggestions are very appreciated.
This pdf file is the note used on the discussion. Also this is a good example of using Pearl’s message passing from the book “Expert Systems and Probabilistic Network Models” by E. Castillo, J.M. Gutiérrez, and A.S. Hadi.