Archive

Posts Tagged ‘computer vision’

How to install Greg Mori’s superpixel MATLAB code?

February 28, 2011 11 comments

This short note aims to show you how to use superpixel code from Greg Mori whose codes are observed to have very good results and used by a bunch of computer vision researchers. However, the installation process can be challenging sometimes ^_^, so I figured it’d be nice if I document the process so that it will be easier for absolute beginners to use the code, and more importantly…I can come back to read when I forget how to do it.

I have MATLAB R2010a installed on my Ubuntu 32-bit 10.04 LTS – the Lucid Lynx

I download Mori’s code, extract the zipped file to a folder called superpixels. The folder is located at
/home/student1/MATLABcodes/superpixels

Next I download the boundary detector code from the link
http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/code/segbench.tar.gz
I extract it to a folder called segbench, then I put it inside the superpixels
/home/student1/MATLABcodes/superpixels/segbench
—————————————————————————–
Here are the instructions of the folders

README (Mori’s)

– Run mex on *.c in yu_imncut directory
– Obtain mfm-pb boundary detector code from
http://www.cs.berkeley.edu/projects/vision/grouping/segbench/
– Change path names in sp_demo.m and pbWrapper.m
– Get a fast processor and lots of RAM
– Run sp_demo.m

README (segbench’s)
(1) For the image and segmentation reading routines in the Dataset
directory to work, make sure you edit Dataset/bsdsRoot.m to point to
your local copy of the BSDS dataset.

(2) Run ‘gmake install’ from this directory to build everything.  You
should then probably put the lib/matlab directory in your MATLAB path.

(3) Read the Benchmark/README file.
———————————————————————————————————————

According to the README instruction
– Run mex on *.c in yu_imncut directory
I run mex on all the .c file in the folder
/home/student1/MATLABcodes/superpixels/yu_imncut
I don’t know why the command mex *.c does not work, so I have to run mex on every file one by one. Each time I run mex, I will get message
Warning: You are using gcc version “4.4.3-4ubuntu5)”.  The version
currently supported with MEX is “4.2.3”.
For a list of currently supported compilers see:
http://www.mathworks.com/support/compilers/current_release/
However, it seems to work fine since I can see all the .mexglx files show up in the folder. So I assume I do it correctly and go on the next step.

In this step,
– Obtain mfm-pb boundary detector code from
http://www.cs.berkeley.edu/projects/vision/grouping/segbench/
I got the code already, so I follow the README (segbench’s). Firstly, I do
(1) For the image and segmentation reading routines in the Dataset
directory to work, make sure you edit Dataset/bsdsRoot.m to point to
your local copy of the BSDS dataset.
So, I go to the file /home/student1/MATLABcodes/superpixels/segbench/Dataset/bsdsRoot.m and change the root to
root = ‘/home/student1/MATLABcodes/superpixels’;
which contains the image I want to segment, “img_000070.jpg”

Next, I do (2) in README (segbench’s)
(2) Run ‘gmake install’ from this directory to build everything.  You
should then probably put the lib/matlab directory in your MATLAB path.
Now at the folder, student1@student1-desktop:~/MATLABcodes/superpixels/segbench$
we need to make MATLAB seen in this folder, so we export the MATLAB path
student1@student1-desktop:~/MATLABcodes/superpixels/segbench$ PATH=$PATH:/usr/share/matlabr2010a/bin
student1@student1-desktop:~/MATLABcodes/superpixels/segbench$ export PATH
then use make install, this time I got quite a long message in the terminal
student1@student1-desktop:~/MATLABcodes/superpixels/segbench$ make install
Then you will notice some files in the folder
/home/student1/MATLABcodes/superpixels/segbench/lib/matlab
What you have to do here is to addpath in MATLAB by typing in the command window
addpath(’/home/student1/MATLABcodes/superpixels/segbench/lib/matlab’);

Next, (3) Read the Benchmark/README file. I found that we don’t have to do anything in this step. So just skip this.

Now it’s the last step
Change path names in sp_demo.m and pbWrapper.m
so, go to the folder /home/student1/MATLABcodes/superpixels and change the path
in pbWrapper.m I make the path pointing to ‘/home/student1/MATLABcodes/superpixels/segbench/lib/matlab’
in sp_demo.m I make the path pointing to
‘/home/student1/MATLABcodes/superpixels/yu_imncut’

Now run the file sp_demo.m. Unfortunately you will get some error messages because of a function spmd. This happens because MATLAB 2010a has function spmd of its own which has the number of input argument different from that of spmd from the toolbox. One way to get around this is to change the name of spmd.c in the toolbox to spmd2.c, then compile spmd2.c using mex spmd2.c. Then replace spmd(…) with spmd2(…). If you encounter more errors from this point on, don’t panic, because it’s probably from this spmd issue, so just do the same thing and it will work fine.

That’s it! Enjoy Greg Mori’s code!

——————————————————————————

For windows users, please refer to Thanapong’s blog, whose URL is given below:
http://blog.thanapong.in.th/arah/?p=29

Image Segmentation using Gaussian Mixture Model (GMM) and BIC

February 27, 2011 3 comments

A while ago, I was so amazed about the image segmentation results using Gaussian Mixture Models (GMMs) because GMM gives pretty good results on normal/natural images. There are some results on my previous post. Of course, GMM is not the best for this job, but hey look at its speed and easiness to implement–it’s pretty good in that sense. However, one problem with GMM is that we need to pick the number of components. In general, the more component numbers we assume, the better log-likelihood it would be for GMM. In that case, we would simply send the number of components to infinity, right? Well…but there is nothing good come out of that because the segment would not be so meaningful–in fact, we overfit the data, which is bad.

Therefore, Bayesian Information Criteria (BIC) is introduced as a cost function composing of 2 terms; 1) minus of log-likelihood and 2) model complexity. Please see my old post. You will see that BIC prefers model that gives good result while the complexity remains small. In other words, the model whose BIC is smallest is the winner. Simple as that. Here is the MATLAB code. Below are some results from sweeping the number of components from 2 to 10. Unfortunately, the results are not what I (and maybe other audiences) desire or expect. As a human, my attention just focuses on skier, snow, sky/cloud and perhaps in the worst case, the shadows, so the suitable number of components should be 3-4. Instead, the BIC assigns 9-component model the winner which is far from I expected. So, Can I say that the straightforward BIC might not be a good model for image segmentation, in particular, for human perception? Well…give GMM-BIC a break– I think this is too early to blame BIC because I haven’t use other more sophisticated features like texture, shape, color histogram which might improve results from using GMM-BIC. The question is what are the suitable features and the number of components that makes the segmentation results using GMM-BIC similar to human perception? MATLAB code is made available here.

original image

original image

Plot of BIC of model using 7-10 components

Plot of BIC of model using 7-10 components

Irregular Tree Structure Bayesian Networks (ITSBNs)

February 24, 2011 Leave a comment

Irregular Tree Structure Bayesian Networks (ITSBNs)

This is my on-going work on structured image segmentation. I’m about to publish the algorithm soon, so the details will be posted after submission. ^_^ Please wait.

<details will be appeared soon>

Here are some results

original image

original image

segmented using GMM

segmented using GMM

segmented using ITSBN

segmented using ITSBN

How to use the ITSBN toolbox

  1. Install Bayesian networks MATLAB toolbox and VLFeat. Let’s say we put them in the folders Z:\research\FullBNT-1.0.4 and Z:\research\vlfeat-0.9.9 respectively.
  2. Download and unpack the ITSBN toolbox. Let’s say the folder location is “Z:\research\ITSBN”. The folder contains some MATLAB files and 2 subdirectories 1) Gaussian2Dplot and 2) QuickShift_ImageSegmentation
  3. Put any color image to be segmented in the same folder. In this case, we use the one from Berkeley image segmentation BSDS500 and the folder is ‘Z:\research\BSR\BSDS500\data\images\test’
  4. Open the file main_ITSBNImageSegm.m in MATLAB and make sure that all the paths pointing to their corresponding folders:
    1. vlfeat_dir = ‘Z:\research\vlfeat-0.9.9\toolbox/vl_setup’;
    2. BNT_dir = ‘Z:\research\FullBNT-1.0.4’;
    3. image_dataset_dir = ‘Z:\research\BSR\BSDS500\data\images\test’;
  5. Run main_ITSBNImageSegm.m. When finished you should see folders of segmented images in the folder ‘Z:\research\ITSBN’.

Image Segmentation using Gaussian Mixture Models

February 17, 2011 6 comments

Today I wanted to compare my image segmentation results against some traditional method, and of course, Gaussian Mixture Model (GMM) is my victim. I really hoped that GMM will be beaten badly and my algorithm would look super smart against GMM. However, the result from GMM is really good–much better than what I originally expected! The features are L*a*b pixel values and x-y pixel locations (all features are standardized). I tested my GMM segmentation code on some images, the results are pretty good relative to its short runtime. I can’t wait to share my MATLAB code and some results are shown below:

41004

Original image

Segmentation result using GMM with 3 components

Segmentation result using GMM with 3 components

Each color represents a class. The brightness represents the posterior probability--the dark pixels represent high uncertainty of the posterior distribution.

Each color represents a class. The brightness represents the posterior probability--the dark pixels represent high uncertainty of the posterior distribution.

108073108073_overlay_segm_gmm_class4_lab_xy_normalized_weighted108073_segm_gmm_class4_lab_xy_normalized_weighted

Balanced-Tree Structure Bayesian Networks (TSBNs) for Image Segmentation

February 15, 2011 6 comments

In this work, I made a more generalized version of Gaussian mixture model (GMM) by putting prior balanced-tree structure over the class variables (mixture components) in the hope that the induced correlation among the hidden variables would suppress the noise in the resulting segmentation. Unlike supervised image classification by [1], this work focuses on totally unsupervised segmentation using TSBN. In this work, it is interesting to see how the data will be “self-organized” according to the initial structure given by TSBN.

The MATLAB code is available here. The codes call inference routines in Bayesian network toolbox (BNT), so you may want to install the toolbox before using my TSBN code.

original image 136x136

original image 136x136

16x16 feature image. Each pixel represents 16-dimensional vector.

16x16 feature image. Each pixel represents 16-dimensional vector.

Segmentation result using TSBN by setting predefined number of classes to 3. It turns out that the classes are meaningful as they are sky, skier and snow.

Segmentation result using TSBN by setting predefined number of classes to 3. It turns out that the classes are meaningful as they are sky, skier and snow.

[1]  X. Feng, C.K.I. Williams, S.N. Felderhof, “Combining Belief Networks and Neural Networks for Scene Segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 467-483, April, 2002

Conditional Random Fields (CRFs) and Voxel on LIDAR

December 11, 2010 Leave a comment

I found some interesting papers using over-segmented data (voxel) concept and CRF with terrestrial LIDAR point cloud.

Volumetric Visualization of Multiple-return Lidar Data: Using Voxels by Jason Stoker [link]

Discriminative Learning of Markov Random Fields for Segmentation of 3D Scan Data by D. Anguelov, B. Taskar, V. Chatalbashev, D. Koller, D. Gupta, G. Heitz, and A. Ng. (CVPR2005)

Conditional Random Field for 3D point clouds with Adaptive Data Reduction by E. H. Lim and D. Suter (CW2007)

Multi-scale Conditional Random Fields for Over-segmented Irregular 3D Point Clouds Classification by E. H. Lim and D. Suter (CVPR2008)

3D terrestrial LIDAR classifications with super-voxels and multi-scale Conditional Random Fields by E. H. Lim and D. Suter (CVPR2008)

Shape-based Recognition of 3D Point Clouds in Urban Environments by Aleksey Golovinskiy et al (ICCV2009)

Local Binary Patterns (LBP)

October 5, 2010 Comments off

I came across with an interesting algorithm for extracting feature descriptors from an image or a video file. The LBP looks very simple and easy to program, but I haven’t had a chance to try it myself. A lot of people claim that it’s pretty good.

Scholarpedia gives very good and short overview of this method:
<a href="http://www.scholarpedia.org/ar