{"id":13454784,"url":"https://github.com/XifengGuo/CapsNet-Keras","last_synced_at":"2025-03-24T06:31:24.042Z","repository":{"id":53745739,"uuid":"108827874","full_name":"XifengGuo/CapsNet-Keras","owner":"XifengGuo","description":"A Keras implementation of CapsNet in NIPS2017 paper \"Dynamic Routing Between Capsules\". Now test error ＝ 0.34%. ","archived":false,"fork":false,"pushed_at":"2020-05-19T02:40:59.000Z","size":1090,"stargazers_count":2464,"open_issues_count":29,"forks_count":654,"subscribers_count":98,"default_branch":"master","last_synced_at":"2025-03-17T03:12:29.376Z","etag":null,"topics":["capsnet","capsnet-keras","capsule","capsule-network","dynamic-routing","keras"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/XifengGuo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-10-30T09:18:29.000Z","updated_at":"2025-03-01T06:10:19.000Z","dependencies_parsed_at":"2022-09-03T06:30:20.471Z","dependency_job_id":null,"html_url":"https://github.com/XifengGuo/CapsNet-Keras","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/XifengGuo%2FCapsNet-Keras","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/XifengGuo%2FCapsNet-Keras/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/XifengGuo%2FCapsNet-Keras/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/XifengGuo%2FCapsNet-Keras/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/XifengGuo","download_url":"https://codeload.github.com/XifengGuo/CapsNet-Keras/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245222548,"owners_count":20580180,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["capsnet","capsnet-keras","capsule","capsule-network","dynamic-routing","keras"],"created_at":"2024-07-31T08:00:57.930Z","updated_at":"2025-03-24T06:31:23.509Z","avatar_url":"https://github.com/XifengGuo.png","language":"Python","funding_links":[],"categories":["Implementations","Python","Dynamic routing implementations"],"sub_categories":["Tensorflow","Keras"],"readme":"# CapsNet-Keras\n[![License](https://img.shields.io/github/license/mashape/apistatus.svg?maxAge=2592000)](https://github.com/XifengGuo/CapsNet-Keras/blob/master/LICENSE)\n\nA Keras (branch [tf2.2](https://github.com/XifengGuo/CapsNet-Keras/tree/tf2.2) supports TensorFlow 2) implementation of CapsNet in the paper:   \n[Sara Sabour, Nicholas Frosst, Geoffrey E Hinton. Dynamic Routing Between Capsules. NIPS 2017](https://arxiv.org/abs/1710.09829)   \nThe current `average test error = 0.34%` and `best test error = 0.30%`.   \n \n**Differences with the paper:**   \n- We use the learning rate decay with `decay factor = 0.9` and `step = 1 epoch`,    \nwhile the paper did not give the detailed parameters (or they didn't use it?).\n- We only report the test errors after `50 epochs` training.   \nIn the paper, I suppose they trained for `1250 epochs` according to Figure A.1?\nSounds crazy, maybe I misunderstood.\n- We use MSE (mean squared error) as the reconstruction loss and \nthe coefficient for the loss is `lam_recon=0.0005*784=0.392`.   \nThis should be **equivalent** with using SSE (sum squared error) and `lam_recon=0.0005` as in the paper.\n\n## Warnning\n\nPlease use Keras==2.0.7 with TensorFlow==1.2 backend, or the `K.batch_dot` function may not work correctly.\n\nHowever, if you use **Tensorflow\u003e=2.0**, then checkout branch [tf2.2](https://github.com/XifengGuo/CapsNet-Keras/tree/tf2.2)\n\n\n## Usage\n\n**Step 1. Clone this repository to local.**\n```\ngit clone https://github.com/XifengGuo/CapsNet-Keras.git capsnet-keras\ncd capsnet-keras\ngit checkout tf2.2 # Only if use Tensorflow\u003e=2.0\n```\n\n**Step 2.\nInstall Keras==2.0.7 with TensorFlow==1.2 backend.**\n```\npip install tensorflow-gpu==1.2\npip install keras==2.0.7\n```\n**or install Tensorflow\u003e=2.0**\n```\npip install tensorflow==2.2\n```\n\n\n\n**Step 3. Train a CapsNet on MNIST**  \n\nTraining with default settings:\n```\npython capsulenet.py\n```\n\nMore detailed usage run for help:\n```\npython capsulenet.py -h\n```\n\n**Step 4. Test a pre-trained CapsNet model**\n\nSuppose you have trained a model using the above command, then the trained model will be\nsaved to `result/trained_model.h5`. Now just launch the following command to get test results.\n```\n$ python capsulenet.py -t -w result/trained_model.h5\n```\nIt will output the testing accuracy and show the reconstructed images.\nThe testing data is same as the validation data. It will be easy to test on new data, \njust change the code as you want.\n\nYou can also just *download a model I trained* from \nhttps://pan.baidu.com/s/1sldqQo1\nor\nhttps://drive.google.com/open?id=1A7pRxH7iWzYZekzr-O0nrwqdUUpUpkik\n\n\n**Step 5. Train on multi gpus**   \n\nThis requires `Keras\u003e=2.0.9`. After updating Keras:   \n```\npython capsulenet-multi-gpu.py --gpus 2\n```\nIt will automatically train on multi gpus for 50 epochs and then output the performance on test dataset.\nBut during training, no validation accuracy is reported.\n\n## Results\n\n#### Test Errors   \n\nCapsNet classification test **error** on MNIST. Average and standard deviation results are\nreported by 3 trials. The results can be reproduced by launching the following commands.   \n ```\n python capsulenet.py --routings 1 --lam_recon 0.0    #CapsNet-v1   \n python capsulenet.py --routings 1 --lam_recon 0.392  #CapsNet-v2\n python capsulenet.py --routings 3 --lam_recon 0.0    #CapsNet-v3 \n python capsulenet.py --routings 3 --lam_recon 0.392  #CapsNet-v4\n```\n   Method     |   Routing   |   Reconstruction  |  MNIST (%)  |  *Paper*    \n   :---------|:------:|:---:|:----:|:----:\n   Baseline |  -- | -- | --             | *0.39* \n   CapsNet-v1 |  1 | no | 0.39 (0.024)  | *0.34 (0.032)* \n   CapsNet-v2  |  1 | yes | 0.36 (0.009)| *0.29 (0.011)*\n   CapsNet-v3 |  3 | no | 0.40 (0.016)  | *0.35 (0.036)*\n   CapsNet-v4  |  3 | yes| 0.34 (0.016) | *0.25 (0.005)*\n   \nLosses and accuracies:   \n![](result/log.png)\n\n\n#### Training Speed \n\nAbout `100s / epoch` on a single GTX 1070 GPU.   \nAbout `80s / epoch` on a single GTX 1080Ti GPU.   \nAbout `55s / epoch` on two GTX 1080Ti GPU by using `capsulenet-multi-gpu.py`.      \n\n#### Reconstruction result  \n\nThe result of CapsNet-v4 by launching   \n```\npython capsulenet.py -t -w result/trained_model.h5\n```\nDigits at top 5 rows are real images from MNIST and \ndigits at bottom are corresponding reconstructed images.\n\n![](result/real_and_recon.png)\n\n#### Manipulate latent code\n\n```\npython capsulenet.py -t --digit 5 -w result/trained_model.h5 \n```\nFor each digit, the *i*th row corresponds to the *i*th dimension of the capsule, and columns from left to \nright correspond to adding `[-0.25, -0.2, -0.15, -0.1, -0.05, 0, 0.05, 0.1, 0.15, 0.2, 0.25]` to \nthe value of one dimension of the capsule. \n\nAs we can see, each dimension has caught some characteristics of a digit. The same dimension of \ndifferent digit capsules may represent different characteristics. This is because that different \ndigits are reconstructed from different feature vectors (digit capsules). These vectors are mutually \nindependent during reconstruction.\n    \n![](result/manipulate-0.png)\n![](result/manipulate-1.png)\n![](result/manipulate-2.png)\n![](result/manipulate-3.png)\n![](result/manipulate-4.png)\n![](result/manipulate-5.png)\n![](result/manipulate-6.png)\n![](result/manipulate-7.png)\n![](result/manipulate-8.png)\n![](result/manipulate-9.png)\n\n\n## Other Implementations\n\n- PyTorch:\n  - [XifengGuo/CapsNet-Pytorch](https://github.com/XifengGuo/CapsNet-Pytorch)\n  - [timomernick/pytorch-capsule](https://github.com/timomernick/pytorch-capsule)\n  - [gram-ai/capsule-networks](https://github.com/gram-ai/capsule-networks)\n  - [nishnik/CapsNet-PyTorch](https://github.com/nishnik/CapsNet-PyTorch.git)\n  - [leftthomas/CapsNet](https://github.com/leftthomas/CapsNet)\n  \n- TensorFlow:\n  - [naturomics/CapsNet-Tensorflow](https://github.com/naturomics/CapsNet-Tensorflow.git)   \n  I referred to some functions in this repository.\n  - [InnerPeace-Wu/CapsNet-tensorflow](https://github.com/InnerPeace-Wu/CapsNet-tensorflow)   \n  - [chrislybaer/capsules-tensorflow](https://github.com/chrislybaer/capsules-tensorflow)\n\n- MXNet:\n  - [AaronLeong/CapsNet_Mxnet](https://github.com/AaronLeong/CapsNet_Mxnet)\n  \n- Chainer:\n  - [soskek/dynamic_routing_between_capsules](https://github.com/soskek/dynamic_routing_between_capsules)\n\n- Matlab:\n  - [yechengxi/LightCapsNet](https://github.com/yechengxi/LightCapsNet)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FXifengGuo%2FCapsNet-Keras","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FXifengGuo%2FCapsNet-Keras","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FXifengGuo%2FCapsNet-Keras/lists"}