{"id":23399665,"url":"https://github.com/filipbasara0/simple-convnext","last_synced_at":"2026-05-14T20:33:18.350Z","repository":{"id":154306320,"uuid":"554225731","full_name":"filipbasara0/simple-convnext","owner":"filipbasara0","description":"Simple implementation of the ConvNext architecture in PyTorch","archived":false,"fork":false,"pushed_at":"2022-10-21T12:53:04.000Z","size":13,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-04T01:38:58.096Z","etag":null,"topics":["computer-vision","convnext","deep-learning","image-classification","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/filipbasara0.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-10-19T13:08:37.000Z","updated_at":"2024-01-23T21:52:36.000Z","dependencies_parsed_at":null,"dependency_job_id":"4750b13e-3f60-4ca3-936b-5e97e7df831d","html_url":"https://github.com/filipbasara0/simple-convnext","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/filipbasara0/simple-convnext","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/filipbasara0%2Fsimple-convnext","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/filipbasara0%2Fsimple-convnext/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/filipbasara0%2Fsimple-convnext/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/filipbasara0%2Fsimple-convnext/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/filipbasara0","download_url":"https://codeload.github.com/filipbasara0/simple-convnext/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/filipbasara0%2Fsimple-convnext/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33042210,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-13T13:14:54.681Z","status":"online","status_checked_at":"2026-05-14T02:00:06.663Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","convnext","deep-learning","image-classification","pytorch"],"created_at":"2024-12-22T10:15:33.703Z","updated_at":"2026-05-14T20:33:18.334Z","avatar_url":"https://github.com/filipbasara0.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# simple-convnext\n\nSimple implementation of the [ConvNext](https://arxiv.org/abs/2201.03545) architecture in PyTorch.\n\n\u003cp align=\"center\" width=\"100%\"\u003e\n    \u003cimg width=\"50%\" src=\"https://user-images.githubusercontent.com/29043871/197199637-9cd8b61b-632a-4cad-9361-b2db0af8c574.png\"\u003e \n\u003c/p\u003e\n\nSupports RandAug and Cutmix / Mixup augmentation and Label Smoothing, Random Erasing and Stochastic Depth regularization.\n\n### CIFAR-10\n\nAfter several iterations, I managed to get **0.92** F1 on CIFAR-10. The model should perform better with additional parameter optimization and longer training.\n\nAll models were trained from scratch, without fine-tuning.\n\nArchitecture is as follows:\n\n- Patch size: `1`\n  - Patch sizes of **2** and **4** resulted in roughly **6** and **12** lower F1 score, emphasizing the importance of a small patch size for low resolution images\n- Layer depths: `[2, 2, 2, 2]`\n- Block dims: `[64, 128, 256, 512]`\n- This model is almost 5 times smaller (`6.4M params`) compared to `ConvNeXt-T` - (`29M params`)\n- Image size: `64`\n\nAugmentation / Regularization params:\n\n- Mixup: `0.8`, Cutmix: `1.0`, Prob: `0.4`, Label smoothing: `0.1`\n- Stochastic Depth Rate: `0.0`\n- RandAug: `ops: 2, magnitude: 9`\n\nAdamW was used as an optimizer, with a learning rate of `1e-3` and weight decay `1e-1`.\nTraining was done for `100` epochs with a batch size of `256` on GTX 1070Ti.\n\n```\n              precision    recall  f1-score   support\n\n           0       0.92      0.94      0.93      1000\n           1       0.94      0.98      0.96      1000\n           2       0.90      0.89      0.90      1000\n           3       0.81      0.80      0.80      1000\n           4       0.93      0.91      0.92      1000\n           5       0.86      0.84      0.85      1000\n           6       0.93      0.95      0.94      1000\n           7       0.95      0.95      0.95      1000\n           8       0.96      0.96      0.96      1000\n           9       0.97      0.93      0.95      1000\n\n    accuracy                           0.92     10000\n   macro avg       0.92      0.92      0.92     10000\nweighted avg       0.92      0.92      0.92     10000\n```\n\n### Tiny ImageNet\n\nThe first run resulted with a F1 score of **0.64**.\n\nThere is deinitely more room to improve, since I didn't invest much time into tuning training and aug parameters.\nAlso, there is no doubt the model would continue improving, if trained for more than 100 epochs.\n\nAll models were trained from scratch, without fine-tuning or additional data.\n\nArchitecture for Tiny ImageNet is as follows:\n\n- Patch size: `2`\n- Layer depths: `[3, 3, 9, 3]`\n- Block dims: `[96, 192, 384, 768]`\n- This model is still smaller (`22M params`) compared to `ConvNeXt-T` - (`29M params`)\n- Image size: `64`\n\nAugmentation / Regularization params:\n\n- Mixup: `0.8`, Cutmix: `1.0`, Prob: `0.6`, Label smoothing: `0.1`\n- Stochastic Depth Rate: `0.1`\n- RandAug: `ops: 2, magnitude: 9`\n\nAdamW was used as an optimizer, with a learning rate of `2e-3` and weight decay `5e-2`.\nTraining was done for `100` epochs with a batch size of `128` on GTX 1070Ti and took ~21 hours.\n\n```\n            precision    recall  f1-score   support\n\n           0       0.81      0.86      0.83        50\n           1       0.83      0.80      0.82        50\n           2       0.71      0.60      0.65        50\n           3       0.62      0.58      0.60        50\n           4       0.71      0.68      0.69        50\n           ...\n           ...\n         195       0.78      0.78      0.78        50\n         196       0.43      0.30      0.35        50\n         197       0.48      0.42      0.45        50\n         198       0.46      0.46      0.46        50\n         199       0.55      0.58      0.56        50\n\n    accuracy                           0.64     10000\n   macro avg       0.65      0.64      0.64     10000\nweighted avg       0.65      0.64      0.64     10000\n```\n\n### Future steps\n\nHoping to get some better hardware to train on ImageNet-1k and fine tune the model for object detection. 🙏😅\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffilipbasara0%2Fsimple-convnext","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffilipbasara0%2Fsimple-convnext","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffilipbasara0%2Fsimple-convnext/lists"}