Recently, I have been working on Deformable-Structure Conditional Random Fields (DSCRFs) for image classification which is about CRFs that can change the graph structure to fit the data (image) and make inference (of pixel label) simultaneously. One problem of this approach is when we estimate the parameters (I guess everyone in the field knows what I’m talking about ^_^), so I’m looking for some optimization algorithms to deal with this. Of course, first thing I’m thinking of is variational approximation. I have tried some already such as mean-field, structure variational, but there is one popular method, “Contrastive Divergence” (CD), that I have heard and wanted to try. I have read some papers on it, and here are what I really recommend to read.
- “Note on Contrastive Divergence” by Oliver Woodford [pdf]: For me, this paper is the best; precise, intuitive and make you hungry to know more!
- “Training Products of Experts by Minimizing Contrastive Divergence” by Geoffrey E. Hinton [pdf]: I guess this is the original paper of CD. This is the first paper I read on this topic, the paper did a good job to make me understand the math underlying CD, however, I did not have an intuitive idea of what CD really is after that first reading. Surprisingly, after I read , then come back to , I found that I can put pieces together and get a better intuition of this topic. So, I really recommend reading  before .
Using Fast Weights to Improve Persistent Contrastive Divergence
Last week I met with a long-lost friend in ICASSP 2010 held in Dallas, TX. More precisely, the very nice friend of mine, Duangmanee (Pew), was my senior student when we were in the same high-school. In the conference, we had a good time (almost an hour) discussing about our lives, updates on our others friends (heeheehee…a nice way to say “gossips”) and our research, and I’m very lucky that Pew is an expert on Topic Models that I’m interested in. Since I’m a beginner on this topic, so I think I will have to learn some more fundamental works on this topic first prior to understanding Pew’s paper.
This post is my effort to list all good papers, notes and tutorials on topic models in the hope that it might be useful for other beginners like me. Please feel free to suggest in order to make this post of the most useful to learners.
Independent Factor Topic Models
Duangmanee (Pew) Putthividhya
Useful links (I’m working on the list)
LDA on Wiki
Blei, David M.; Lafferty, John D. (2006). “Correlated topic models”. Advances in Neural Information Processing Systems
D. Blei and M. Jordan. Modeling annotated data. In Proceedings of the 26th annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 127–134. ACM Press, 2003. [PDF]
K. Barnard, P. Duygulu, N. de Freitas, D. Forsyth, D. Blei, and M. Jordan. Matching words and pictures. Journal of Machine Learning Research, 3:1107–1135, 2003. [PDF]
Blei, David M.; Jordan, Michael I.; Griffiths, Thomas L.; Tenenbaum; Joshua B (2004). “Hierarchical Topic Models and the Nested Chinese Restaurant Process”. Advances in Neural Information Processing Systems 16: [pdf]
Hanna M. Wallach (2008), “Structured Topic Model for Language” [PhD thesis]
Tomoharu Iwata, Takeshi Yamada, Naonori Ueda, “Modeling Social Annotation Data with Content Relevance using a Topic Model,” Advances in Neural Information Processing Systems (NIPS2009), 835-843, 2009 [pdf]
Wednesday, March 17
IVMSP-P4.1: MULTISCALE SEGMENTATION FOR MRC DOCUMENT COMPRESSION USING A MARKOV RANDOM FIELD MODEL
Eri Haneda, Charles Bouman, Purdue university, United States
IVMSP-P4.3: A LOW COMPLEXITY METHOD FOR DETECTION OF TEXT AREA IN NATURAL IMAGES
Katherine L. Bouman, University of Michigan, United States; Golnaz Abdollahian, Mireille Boutin, Edward Delp, Purdue University, United States
– Application on cell phone worth thinking about…we have to use simple and efficient.
– This work is simple and seems to work well in practice.
IVMSP-P4.4: FAST SEMI-SUPERVISED IMAGE SEGMENTATION BY NOVELTY SELECTION
Antonio Paiva, Tolga Tasdizen, University of Utah, United States
-Simple, but very important idea…very practical. Just change the perspective, we gain the speed!
IVMSP-P4.5: IMPROVING IMAGE SEGMENTATION VIA SHAPE PCA RECONSTRUCTION
Hui Wang, Hong Zhang, University of Alberta, Canada
– Simple idea, but very interesting, and should be applied to more complicated contour objects
IVMSP-P4.8: COMPLEXITY-BASED BORDER DETECTION FOR TEXTURED IMAGES
Tomas Crivelli, University of Buenos Aires, Argentina; Agustin Mailing, Bruno Cernuschi-Frias, University of Buenos Aires and Consejo Nacional de Investigaciones Científicas y Técnicas, Argentina
– Use Taylor series to approx log likelihood and get Gaussian. Check out paper by Kashyap and Chellappa “Estimation choice of neighbors..”
IVMSP-P4.9: ON SAND RIPPLE DETECTION IN SYNTHETIC APERTURE SONAR IMAGERY
David Williams, Enrique Coiras, NATO Undersea Research Centre, Italy
– Smart way to formulate using 2 orthogonal (Gabor) filters in absolute value
Branch and Bound for discrete optimization
Factor Graph for Optimization Problem
Optimization Problem –> Dynamic Programming
My work should change to Deformable Factor Graphs
Topic Model + Image Segmentation
Eclipse IDE for 64-bit Windows
I’m now trying Eclipse 3.5 (“Galileo”)
Important note from the download site:
This build requires a 64-bit JVM, and will not run with a 32-bit JVM.
You can, for example, use the Sun 64-bit 1.5 JVM for AMD64. Note
that the Sun 1.4.2 JVM for AMD64 is 32-bit and therefore cannot be
used to run this build.
Here is the instruction on how to download and install
Q: I’m using Windows7 64-bit and my LaTeX editor (build 0.536501) shows blank page for built-in DVI. What should I do?
A: You may want to check in the main menu Configuration>Options. Then expand the list in Application>DVI viewer. Make sure that you put the right path to GSDLL32.DLL directory and PostScript fonts. Make sure you have Ghostscript installed in your computer. In my case, I install GPL Ghostscript 8.64 for 64-bit Windows (link), so here is my configuration
Hope this helps!