Home > Academics, Research, Reviews, Tutorials > Minimum Description Length (MDL)

Minimum Description Length (MDL)

The basic idea of MDL is about the minimum length of code or symbol used to represent an entity. There are so many applications that MDL is applicable to. For example, model selection, data/signal compression, structure learning, curve fitting, etc. Good references for a beginner are:

  • MDL in Wiki [link]
  • MDL on Scholarpedia [link]
    • There are 2 recommended books in the page:
      • The Minimum Description Length Principle (Adaptive Computation and Machine Learning) [link]
      • Information and Complexity in Statistical Modeling (Information Science and Statistics) [link]
  • MDL on the web [link]
    • You can find good tutorials there. I would recommend the tutorial “P.Grünwald, A tutorial introduction to the minimum description length principle. In: Advances in Minimum Description Length: Theory and Applications (edited by P. Grünwald, I.J. Myung, M. Pitt), MIT Press, 2005 (80 pages).”
  • MDL tutorial by Prof. Rissanen “An Introduction to the MDL Principle” [pdf]

MDL has a strong connection with Kolmogorov Complexity. In terms of model selectin there might be some other topics that you may find interesting to make connection with MDL, for example, BIC, AIC and NML.

Advertisements
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: