{"id":13419002,"url":"https://github.com/leovandriel/caffe2_cpp_tutorial","last_synced_at":"2025-03-15T04:31:49.974Z","repository":{"id":46459940,"uuid":"95238039","full_name":"leovandriel/caffe2_cpp_tutorial","owner":"leovandriel","description":"C++ transcripts of the Caffe2 Python tutorials and other C++ example code","archived":false,"fork":false,"pushed_at":"2022-06-03T00:47:09.000Z","size":2226,"stargazers_count":432,"open_issues_count":37,"forks_count":94,"subscribers_count":28,"default_branch":"master","last_synced_at":"2024-08-05T02:01:19.057Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/leovandriel.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-06-23T16:38:01.000Z","updated_at":"2024-07-20T13:36:43.000Z","dependencies_parsed_at":"2022-09-23T01:12:09.650Z","dependency_job_id":null,"html_url":"https://github.com/leovandriel/caffe2_cpp_tutorial","commit_stats":null,"previous_names":["leonardvandriel/caffe2_cpp_tutorial"],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leovandriel%2Fcaffe2_cpp_tutorial","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leovandriel%2Fcaffe2_cpp_tutorial/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leovandriel%2Fcaffe2_cpp_tutorial/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leovandriel%2Fcaffe2_cpp_tutorial/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/leovandriel","download_url":"https://codeload.github.com/leovandriel/caffe2_cpp_tutorial/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243495503,"owners_count":20299922,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-30T22:01:09.996Z","updated_at":"2025-03-15T04:31:49.673Z","avatar_url":"https://github.com/leovandriel.png","language":"C++","funding_links":[],"categories":["TODO scan for Android support in followings"],"sub_categories":[],"readme":"\u003cimg src=\"icon.jpg\" alt=\"Caffe2 C++ Tutorials Icon\" width=\"72\"/\u003e\n\n# Caffe2 C++ Tutorials and Examples\n\n*C++ transcripts of the Caffe2 Python tutorials and other C++ example code.*\n\n\n## About\n\nCaffe2 has a strong C++ core but most tutorials only cover the outer Python layer of the framework. This project aims to provide example code written in C++, complementary to the Python documentation and tutorials. It covers verbatim transcriptions of most of the Python tutorials and other example applications.\n\nSome higher level tools, like brewing models and adding gradient operations are currently not available in Caffe2's C++. This repo therefore provides some model helpers and other utilities as replacements, which are probably just as helpful as the actual tutorials. You can find them in [include/caffe2/util](include/caffe2/util) and [src/caffe2/util](src/caffe2/util).\n\nCheck out the original Caffe2 Python tutorials at [https://caffe2.ai/docs/tutorials.html](https://caffe2.ai/docs/tutorials.html).\n\n## Tutorials\n\n1. [Intro Tutorial](#intro-tutorial)\n2. [Toy Regression](#toy-regression)\n3. [Pre-trained models](#loading-pre-trained-models)\n4. [MNIST from scratch](#mnist---create-a-cnn-from-scratch)\n5. [RNNs and LSTM](#rnns-and-lstm-networks)\n6. [ImageNet Classifiers](#imagenet-classifiers)\n7. [Fast Retrain](#fast-retrain)\n8. [Training from scratch](#training-from-scratch)\n9. [Deep Dream](#deep-dream)\n\n## Build\n\n1. Install dependencies\n\n    Install the dependencies CMake, leveldb and OpenCV. If you're on macOS, use Homebrew:\n\n        brew install cmake glog protobuf leveldb opencv eigen\n\n    On Ubuntu:\n\n        apt-get install cmake libgoogle-glog-dev libprotobuf-dev libleveldb-dev libopencv-dev libeigen3-dev curl\n\n    In case you're using CUDA an run into CMake issues with `NCCL`, try adding this to your `.bashrc` (assuming Caffe2 at `$HOME/caffe2`):\n\n        export CMAKE_LIBRARY_PATH=$CMAKE_LIBRARY_PATH:$HOME/caffe2/third_party/nccl/build/lib\n\n2. Install Caffe2\n\n    Follow the Caffe2 installation instructions: [https://caffe2.ai/docs/getting-started.html](https://caffe2.ai/docs/getting-started.html)\n\n3. Build using CMake\n\n    This project uses CMake. However easiest way to just build the whole thing is:\n\n        make\n\n    Internally it creates a `build` folder and runs CMake from there. This also downloads the resources that are required for running some of the tutorials.\n\n    Check out the [Build alternatives](#build-alternatives) section below if you wish to be more involved in the build process.\n\nNote: sources are developed and tested on macOS and Ubuntu.\n\n\n## Intro Tutorial\n\nThe [Intro Tutorial](https://caffe2.ai/docs/intro-tutorial.html) covers the basic building blocks of Caffe2. This tutorial is transcribed in [intro.cc](src/caffe2/binaries/intro.cc).\n\nMake sure to first run `make`. Then run the intro tutorial:\n\n    ./bin/intro\n\nThis should output some numbers, including a `loss` of about `2.2`.\n\n\u003cimg src=\"script/intro.jpg\" alt=\"Intro Tutorial\" width=\"284\"/\u003e\n\n## Toy Regression\n\nOne of the most basic machine learning tasks is linear regression (LR). The [Toy Regression](https://caffe2.ai/docs/tutorial-toy-regression.html) tutorial shows how to get accurate results with a two-parameter model. This tutorial is transcribed in [toy.cc](src/caffe2/binaries/toy.cc)\n\nRun the toy regression model:\n\n\t./bin/toy\n\nThis performs 100 steps of training, which should result in `W after` approximating `W ground truth`.\n\n\u003cimg src=\"script/toy.jpg\" alt=\"Toy Regression\" width=\"184\"/\u003e\n\n## Loading Pre-Trained Models\n\nOften training can be skipped by using a pre-trained model. The [Model Zoo](https://github.com/caffe2/caffe2/wiki/Model-Zoo) contains a few of the popular models, although many are only available for Caffe. Use [caffe_translator.py](https://github.com/caffe2/caffe2/blob/master/caffe2/python/caffe_translator.py) to convert models to Caffe2. See [Caffe2 Models](https://github.com/leonardvandriel/caffe2_models) for more info.\n\nThe [Loading Pre-Trained Models](https://caffe2.ai/docs/tutorial-loading-pre-trained-models.html) tutorial shows how to use these models to classify images. This tutorial and more is covered in [pretrained.cc](src/caffe2/binaries/pretrained.cc). The code takes an input image and classifies its content. By default it uses the image in `res/image_file.jpg`. Make sure the pre-trained [Squeezenet](https://github.com/caffe2/models/tree/master/squeezenet) model is present in `res/squeezenet_*_net.pb`. Note that\n\nTo run:\n\n    ./bin/pretrained\n\nThis should output something along the lines of `96% 'daisy'`.\n\n\u003cimg src=\"script/pretrained.jpg\" alt=\"Pre-Trained Models\" width=\"227\"/\u003e\n\nTo classify `giraffe.jpg`:\n\n    ./bin/pretrained --file giraffe.jpg\n\nThis tutorial is also a good test to see if OpenCV is working properly.\n\nTo export a model from Python:\n\n    model = model_helper.ModelHelper(..)\n    with open(\"init_net.pb\", 'wb') as f:\n      f.write(model.param_init_net._net.SerializeToString())\n    with open(\"predict_net.pb\", 'wb') as f:\n      f.write(model.net._net.SerializeToString())\n\nSee also:\n\n- [Image Pre-Processing](https://caffe2.ai/docs/tutorial-image-pre-processing.html)\n\n## MNIST - Create a CNN from Scratch\n\nA classical machine learning dataset is the [MNIST database of handwritten digits](http://yann.lecun.com/exdb/mnist/) by Yann LeCun. The Caffe2 tutorial [MNIST - Create a CNN from Scratch](https://caffe2.ai/docs/tutorial-MNIST.html) shows how to build a basic convolutional neural network (CNN) to recognize these handwritten digits. This tutorial is transcribed in [mnist.cc](src/caffe2/binaries/mnist.cc). Note that this and following tutorials rely on utility functions defined in [caffe2/util](include/caffe2/util).\n\nMake sure the databases folders `res/mnist-*-nchw-leveldb` are present. These should be generated by the  [download_resource.sh](script/download_resource.sh) script. Then run:\n\n    ./bin/mnist\n\nThis performs 100 training runs, which should provide about 90% accuracy.\n\nTo see the training in action, run with `--display`:\n\n    ./bin/mnist --display\n\n\u003cimg src=\"script/mnist.jpg\" alt=\"MNIST\" width=\"300\"/\u003e\n\nAfter testing the trained model is stored in `tmp/mnist_init_net.pb` and `tmp/mnist_predict_net.pb`. For an example implentation of how to use this trained model, take a look at `predict_example()` in [mnist.cc](https://github.com/leonardvandriel/caffe2_cpp_tutorial/blob/master/src/caffe2/binaries/mnist.cc#L321). This implementation is \"pure\" Caffe2 and does not rely on any helper methods. For an implementation that does use helper methods, take a look at `imagenet`.\n\n## RNNs and LSTM Networks\n\nIn [The Unreasonable Effectiveness of Recurrent Neural Networks](http://karpathy.github.io/2015/05/21/rnn-effectiveness/) Andrej Karpathy describes how to train a recurrent neural network (RNN) on a large volume of text and how to generate new text using such a network. The Caffe2 tutorial [RNNs and LSTM Networks](https://caffe2.ai/docs/RNNs-and-LSTM-networks.html) covers this technique using the [char_rnn.py](https://github.com/caffe2/caffe2/blob/master/caffe2/python/examples/char_rnn.py) script.\n\nThis tutorial is transcribed in [rnn.cc](src/caffe2/binaries/rnn.cc). It takes the same parameters as used in the tutorial. First make sure the file `res/shakespeare.txt` is present. Then run:\n\n    ./bin/rnn\n\nIn contrast to the tutorial, this script terminates after 10K iterations. To get more, use `--iters`:\n\n    ./bin/run --iters 100000\n\nTo get better results (loss \u003c 1), expand the hidden layer:\n\n    ./bin/rnn --iters 100000 --batch 32 --hidden_size 512 --seq_length 32\n\nThe file `res/dickens.txt` contains a larger volume of text. Because the writing is a bit more recent, it's more challenging to generate convincing results. Also, single newlines are stripped to allow for more creativity.\n\n    ./bin/rnn --iters 100000 --batch 32 --hidden_size 768 --seq_length 32 --train_data res/dickens.txt\n\nAfter 200K runs, the loss has not dropped below 36, in contrast to the shakespeare text. Perhaps this requires an additional hidden layer in the LSTM model.\n\n## ImageNet Classifiers\n\nMuch of the progress in image recognition is published after the yearly [ImageNet Large Scale Visual Recognition Challenge](http://www.image-net.org/challenges/LSVRC/) (ILSVRC). This competion is based on the [ImageNet](http://www.image-net.org/) dataset, which is a large volume of labeled images of nearly everything. The models for this challenge form the basis of much image recognition and processing research. One of the most basic challenges is classifying an image, which is covered in this example.\n\nTo classify the content of an image, run:\n\n    ./bin/imagenet --model resnet101 --file res/image_file.jpg\n\nWhere the model name is one of the following:\n\n* `alexnet`: [AlexNet](https://github.com/BVLC/caffe/tree/master/models/bvlc_alexnet)\n* `googlenet`: [GoogleNet](https://github.com/BVLC/caffe/tree/master/models/bvlc_googlenet)\n* `squeezenet`: [SqueezeNet](https://github.com/DeepScale/SqueezeNet)\n* `vgg16` and `vgg19`: [VGG Team](http://www.robots.ox.ac.uk/~vgg/research/very_deep/)\n* `resnet50`, `resnet101`, `resnet152`: [MSRA](https://github.com/KaimingHe/deep-residual-networks)\n* `mobilenet`, `mobilenet50`, `mobilenet25`: [MobileNet](https://research.googleblog.com/2017/06/mobilenets-open-source-models-for.html)\n\n\u003cimg src=\"script/imagenet.jpg\" alt=\"ImageNet Classifiers\" width=\"313\"/\u003e\n\nThe pre-trained weights for these models are automatically downloaded and stored in the `res/` folder. If you wish to download all models in one go, run:\n\n\t./script/download_extra.sh\n\nAdditional models can be made available on request!\n\nTo classify an image using a model that you trained yourself, specify the location of the init and predict `.pb` file including a `%` character. For example:\n\n    ./bin/imagenet --model res/mobilenet_%_net.pb --file res/image_file.jpg\n\n\nSee also:\n\n- [Model Zoo](https://github.com/caffe2/caffe2/wiki/Model-Zoo)\n- [Caffe2 Models](https://github.com/leonardvandriel/caffe2_models)\n- [Neural Network Architectures](https://medium.com/towards-data-science/neural-network-architectures-156e5bad51ba)\n\n## Fast Retrain\n\nThe article [DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition](https://arxiv.org/pdf/1310.1531v1.pdf) describes how to get good results on new datasets with minimal training efforts by reusing trained parameters of an existing model. For example, the above models are all trained on ImageNet data, which means they will only be able to classify ImageNet labels. However, by retraining just the top half of the model we can get high accuracy in a fraction of the time. If the image data has similar characteristics, it's possible to get good results by only retraining the top 'FC' layer.\n\nFirst divide all images in subfolders with the label a folder name. Then to retrain the final layer of GoogleNet:\n\n    ./bin/train --model googlenet --folder res/images --layer pool5/7x7_s1\n\nThe script starts out by collecting all images and running them through the pre-trained part of the model. This allows for very fast training on the pre-processed image data.\n\nIf you have more (GPU) power at your disposal retrain VGG16's final 2 layers:\n\n    ./bin/train --model vgg16 --folder res/images --layer fc6\n\nAdd `--display` for training visualization:\n\n    ./bin/train --model googlenet --folder res/images --layer pool5/7x7_s1 --display\n\n\u003cimg src=\"script/retrain.jpg\" alt=\"Fast Retrain\" width=\"250\"/\u003e\n\nSome models, like SqueezeNet require reshaping of their output to N x D tensor:\n\n    ./bin/train --model squeezenet --folder res/images --layer fire9/concat --reshape\n\nYou can also provide your own pre-trained model. Specify the location of the init and predict `.pb` file including a `%` character:\n\n    ./bin/train --model res/googlenet_%_net.pb --folder res/images --layer pool5/7x7_s1\n\nAfter the test runs, the model is saved in the `--folder` under the name `_\u003clayer\u003e_\u003cmodel\u003e_\u003cinit/predict\u003e_net.pb`. Please note that you'll need to specify the generated class .txt file by using the `--classes` flag. You can now use this model like any other, for example in the `imagenet` example:\n\n    ./bin/imagenet --model res/images/_pool5_7x7_s1_googlenet_%_net.pb --file res/images/dog/Tjoise.jpg --classes res/images/_pool5_7x7_s1_classes.txt\n\nAnother example implementation of a classifier can be found in [mnist.cc](https://github.com/leonardvandriel/caffe2_cpp_tutorial/blob/master/src/caffe2/binaries/mnist.cc#L321), see `predict_example()`.\n\nSee also:\n\n- [How to Retrain Inception's Final Layer for New Categories](https://www.tensorflow.org/tutorials/image_retraining)\n- [Train your own image classifier with Inception in TensorFlow](https://research.googleblog.com/2016/03/train-your-own-image-classifier-with.html)\n\n## Training from scratch\n\nTo fully train an existing image classification model from scratch, run without the `--layer` option:\n\n    ./bin/train --model resnet50 --folder res/images\n\nThe models currently available for training are the ones listed in the [ImageNet](#imagenet-classifiers) section. This will take a lot of time even when runnning on the GPU.\n\nSome models, like SqueezeNet require reshaping of their output to N x D tensor:\n\n    ./bin/train --model squeezenet --folder res/images --reshape\n\n\u003cimg src=\"script/train.jpg\" alt=\"Training from scratch\" width=\"400\"/\u003e\n\n## Deep Dream\n\nThe article [Inceptionism: Going Deeper into Neural Networks](https://research.googleblog.com/2015/06/inceptionism-going-deeper-into-neural.html) describes how hidden layers of a CNN can be visualized by training just the the input tensor based on the mean value of a particular channel. This technique is known for producing remarkable imagery and can be combined with existing images. Ths is referred to as Deep Dream.\n\nNB: this code is still a bit buggy and generated images are not representative of the original Deep Dream implementation.\n\nThe 139th channel in the `inception_4d/3x3` layer in GoogleNet:\n\n    ./bin/dream --model googlenet --layer inception_4d/3x3_reduce --channel 139\n\nThe resulting image will be written to `tmp/`. To visualize the process, add `--display`:\n\n    ./bin/dream --model googlenet --layer inception_4d/3x3_reduce --channel 139 --display\n\n\u003cimg src=\"script/dream.jpg\" alt=\"Deep Dream\" width=\"400\"/\u003e\n\nNB: these images differ from the ones presented in Google's article.\n\nMultiple channels can be rendered in parallel by increasing the batch size:\n\n    ./bin/dream --model googlenet --layer inception_4d/3x3_reduce --channel 133 --display --batch 11\n\n\u003cimg src=\"script/dream2.jpg\" alt=\"Deep Dream\" width=\"538\"/\u003e\n\nIf you have more (GPU) power at your disposal, the first channel in `conv3_1` layer in VGG16:\n\n    ./bin/dream --model vgg16 --layer conv3_1 --channel 0\n\nYou can also provide your own pre-trained model. Specify the location of the init and predict `.pb` file including a `%` character:\n\n    ./bin/dream --model res/squeezenet_%_net.pb --layer fire9/concat --channel 100 --display\n\nWe can also do some dreaming on MNIST. First train the [MNIST model](#mnist---create-a-cnn-from-scratch). Then run:\n\n    ./bin/dream --model tmp/mnist_%_net.pb --layer conv2 --size 28 --channel 0 --batch 50 --display\n\nSee also:\n\n- [Pre-rendered Inception](http://storage.googleapis.com/deepdream/visualz/tensorflow_inception/index.html)\n- [Pre-rendered VGG16](http://storage.googleapis.com/deepdream/visualz/vgg16/index.html)\n- [Deep Dreams (with Caffe)](https://github.com/google/deepdream/blob/master/dream.ipynb)\n\n## Plots\n\nSome of the examples have a `--display` option, which will show an OpenCV window with images and plots covering the training progress. These graphs are drawn using the [cvplot](https://github.com/leonardvandriel/cvplot) framework.\n\n\u003cimg src=\"script/plot.jpg\" alt=\"Plot Examples\" width=\"450\"/\u003e\n\n## Troubleshooting\n\nSee [http://rpg.ifi.uzh.ch/docs/glog.html](http://rpg.ifi.uzh.ch/docs/glog.html) for more info on logging. Try running the tools and examples with `--logtostderr=1`, `--caffe2_log_level=1`, and `--v=1`.\n\n## Build alternatives\n\nThe easiest way to build all sources is to run:\n\n    make\n\nTo run these steps manually:\n\n    mkdir -p build\n    cd build\n    cmake ..\n    make\n    cd ..\n    ./script/download_resource.sh\n\nCompiling the tutorials and examples individually can be a little more involved. One way to get more understanding of what CMake does internally is by running:\n\n    cd build\n    make VERBOSE=1\n    cd ..\n\nThe first three tutorials `intro`, `toy`, and `pretrained` can be compiled without CMake quite easily. For example `pretrained` on macOS:\n\n    c++ src/caffe2/binaries/pretrained.cc -o bin/pretrained -std=gnu++11 -Iinclude -I/usr/local/include/eigen3 -I/usr/local/include/opencv -lgflags -lglog -lprotobuf -lopencv_core -lopencv_imgproc -lopencv_highgui -lopencv_imgcodecs -lCaffe2_CPU\n\nAnd `pretrained` on Ubuntu:\n\n    c++ src/caffe2/binaries/pretrained.cc -o bin/pretrained -std=gnu++11 -Iinclude -I/usr/include/eigen3 -I/usr/include/opencv -lgflags -lglog -lprotobuf -lopencv_core -lopencv_imgproc -lopencv_highgui -lCaffe2_CPU\n\nOther examples require the compilation of additional `.cc` files. Take a look at the verbose output of `cd build \u0026\u0026 make VERBOSE=1`.\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fleovandriel%2Fcaffe2_cpp_tutorial","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fleovandriel%2Fcaffe2_cpp_tutorial","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fleovandriel%2Fcaffe2_cpp_tutorial/lists"}