{"id":17670808,"url":"https://github.com/sayannath/convmixer-tensorflow","last_synced_at":"2025-05-12T14:19:02.146Z","repository":{"id":105049385,"uuid":"417616867","full_name":"sayannath/ConvMixer-TensorFlow","owner":"sayannath","description":"Implementation of ConvMixer-Patches Are All You Need? in TensorFlow and Keras","archived":false,"fork":false,"pushed_at":"2021-10-31T18:04:10.000Z","size":1835,"stargazers_count":12,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-26T12:22:52.646Z","etag":null,"topics":["attention","cifar100","convmixer","convolution","deep-learning","iclr","iclr2022","machine-learning","mlp","patches","tensorflow","vision-transformer"],"latest_commit_sha":null,"homepage":"https://openreview.net/pdf?id=TVHS5Y4dNvM","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sayannath.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-10-15T19:45:26.000Z","updated_at":"2025-01-28T08:01:00.000Z","dependencies_parsed_at":null,"dependency_job_id":"f173c3e3-657b-4f8d-9ea4-4d97608a71e1","html_url":"https://github.com/sayannath/ConvMixer-TensorFlow","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sayannath%2FConvMixer-TensorFlow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sayannath%2FConvMixer-TensorFlow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sayannath%2FConvMixer-TensorFlow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sayannath%2FConvMixer-TensorFlow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sayannath","download_url":"https://codeload.github.com/sayannath/ConvMixer-TensorFlow/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243150963,"owners_count":20244489,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["attention","cifar100","convmixer","convolution","deep-learning","iclr","iclr2022","machine-learning","mlp","patches","tensorflow","vision-transformer"],"created_at":"2024-10-24T02:42:25.776Z","updated_at":"2025-03-12T03:31:19.240Z","avatar_url":"https://github.com/sayannath.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Patches Are All You Need? - ConvMixer\n\nConvMixer, an extremely simple model that is similar in spirit to the ViT and the even-more-basic MLP-Mixer in that it operates directly on patches as input, separates the mixing of spatial and channel dimensions, and maintains equal size and resolution throughout the network. In contrast, however, the ConvMixer uses only standard convolutions to achieve the mixing steps. Despite its simplicity, we show that the ConvMixer outperforms the ViT, MLP-Mixer, and some of their variants for similar parameter counts and data set sizes, in addition to outperforming classical vision models such as the ResNet.\n\n**Official GitHub Link:** https://github.com/tmp-iclr/convmixer\n\n**Paper Link:** https://openreview.net/pdf?id=TVHS5Y4dNvM \u003cbr\u003e\n\nNote: Paper is under review for ICLR 2022\n\n\n\u003ca href=\"https://colab.research.google.com/drive/1m-faU1DmBZlqkVY_tcYnOcepGlOyJ5K9?usp=sharing\" target=\"_parent\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/\u003e\u003c/a\u003e\n\n## Model Architechture\n\n![](https://i.imgur.com/Yd7gpMP.png) \n\n## Installation\n\n```\npip install -q tensorflow-addons\n```\n\nNote: We are using TensorFlow-Addons for using the `AdamW` optimizer and `GeLU` activation function.\n\n## Results\n\n![Unknown-2](https://user-images.githubusercontent.com/41967348/137559060-96c6c84a-7055-4f3d-ade1-415e5a756880.png) ![Unknown](https://user-images.githubusercontent.com/41967348/137559078-0f095bd4-e119-457c-ac79-7caa5e9a076e.png)\n\n\u003e TensorBoard Link: https://tensorboard.dev/experiment/bkhqOz0RQ1Cv5dwrDQySMQ/\n\nNote: Trained `25 Epochs` and got a top-5-accuracy of 64.41%\n\n## Future Work\n\n* To train on 150 epochs\n* To train model on ImageNet dataset\n\n## Citation\n```\n@inproceedings{\nanonymous2022patches,\ntitle={Patches Are All You Need?},\nauthor={Anonymous},\nbooktitle={Submitted to The Tenth International Conference on Learning Representations },\nyear={2022},\nurl={https://openreview.net/forum?id=TVHS5Y4dNvM},\nnote={under review}\n}\n```\n\n## License\n```\nMIT License\n\nCopyright (c) 2021 Sayan Nath\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsayannath%2Fconvmixer-tensorflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsayannath%2Fconvmixer-tensorflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsayannath%2Fconvmixer-tensorflow/lists"}