https://github.com/royshil/FoodcamClassifier

A Bag of Visual Words food classifier for Media Lab's FoodCam
https://github.com/royshil/FoodcamClassifier

Last synced: 17 days ago
JSON representation

A Bag of Visual Words food classifier for Media Lab's FoodCam

• Host: GitHub
• URL: https://github.com/royshil/FoodcamClassifier
• Owner: royshil
• Created: 2011-08-20T04:05:50.000Z (about 13 years ago)
• Default Branch: master
• Last Pushed: 2014-06-05T14:20:05.000Z (over 10 years ago)
• Last Synced: 2024-10-23T07:51:22.119Z (20 days ago)
• Language: C++
• Homepage:
• Size: 192 KB
• Stars: 115
• Watchers: 15
• Forks: 52
• Open Issues: 8
• Metadata Files:
  • Readme: README

Awesome Lists containing this project

README

        
Foodcam Classifier

==================

A Bag Of Visual Words classifier for MIT Media Lab's Foodcam.

Built using OpenCV 2.3

Implementing BOVW method: Visual Categorization with Bags of Keypoints by Gabriella Csurka, Christopher R. Dance, Lixin Fan, Jutta Willamowski, Cedric Bray, 2004.

Using Opponent-Color SURF features (192 floats): Van de Sande et al., CGIV 2008 Color Descriptors for Object Category Recognition. (they used SIFT)

Radial Basis Function SVMs for classification.

This is basically a primer to BOVW methods using OpenCV, which are dead simple.

I tweaked the basic method by:

	- using background subtraction (since these are pictures from a still camera).

	- using a sliding window. This enables better multi-class decisions, and actually paves the path to image segmentation. 

Compiling

---------

Basically, just run:

cmake -D CMAKE_CXX_FLAGS=-fopenmp . ; make -j4

CMake should take you all the way, but I also have an .xcodeproj/ included.

Working it

----------

Get the dataset:

http://www.media.mit.edu/~roys/shared/foodcamimages.zip

Assume the dataset is now in the same directory, under foodcamimages/.

Manually classify training and test:

./manual-classifier foodcamimages/TRAIN/ train.txt

./manual-classifier foodcamimages/TEST/ test.txt

(or you can use my manual classification included... lazy)

Create a background image:

./make-test-background

Build the vocabulary: (this will take a very very long time, KMeans on 1.3Million 192-long vectors to find 1000 cluster-centers...)

./build-vocabulary

(you may also like to use ./kmeans-trainer, if you break the operation before KMeans finished, which may take more than 10 hours)

Train the classifiers:

./train-bovw vocabulary_color_1000.yml with_color

(you may also make use of './train-SVM-alone', if you just wanna tweak the SVM parameters)

Test your work:

./test-classifiers 

(it will output a whole lotta things, and then the confusion matrix in neighbors list form)

Put it to work!:

./foodcam-predict some_640x480_image_of_food.png

Will output either one or two found classes of food, based on how close the prediction is.

Notes

-----

Some of the computation is sped up on multi-core machines by using OpenMP, so it's recommended to use it.

Results

-------

This is the confusion matrix:

classified ->	cookies	indian	italian	pizza	veggie+fruit	sandwiches

cookies		56.3%	0.0%	4.3%	0.0%	13.5%		15.4%

indian		0.0%	71.4%	6.4%	3.8%	3.4%		7.7%

italian		0.0%	0.0%	63.8%	0.0%	14.6%		0.0%

pizza		6.3%	0.0%	12.8%	92.3%	5.6%		15.4%

veggie+fruit	0.0%	0.0%	2.1%	3.8%	49.4%		0.0%

sandwiches	25.0%	14.3%	6.4%	0.0%	7.9%		61.5%

Quick scan shows that it can classify Pizzas pretty good!

The downfall are the veggie+fruit class, with less than 50% accuracy.

Rest are lukewarm, but this can probably be attribued to the very small training set.