{"id":16271695,"url":"https://github.com/jianzhnie/self_supervised","last_synced_at":"2025-04-08T15:34:41.130Z","repository":{"id":103458108,"uuid":"437733787","full_name":"jianzhnie/self_supervised","owner":"jianzhnie","description":"self-supervised learning","archived":false,"fork":false,"pushed_at":"2021-12-14T10:36:46.000Z","size":44,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-02-14T11:53:04.503Z","etag":null,"topics":["beit","byol","mae","moco","self-supervised-learning","simclr"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jianzhnie.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-12-13T04:18:08.000Z","updated_at":"2021-12-14T10:36:49.000Z","dependencies_parsed_at":null,"dependency_job_id":"814693da-b279-4e3a-ab0a-09fb23e78fc2","html_url":"https://github.com/jianzhnie/self_supervised","commit_stats":{"total_commits":18,"total_committers":1,"mean_commits":18.0,"dds":0.0,"last_synced_commit":"d1e0f31ab032150ab0ad007c1e19773135a5fb79"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jianzhnie%2Fself_supervised","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jianzhnie%2Fself_supervised/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jianzhnie%2Fself_supervised/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jianzhnie%2Fself_supervised/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jianzhnie","download_url":"https://codeload.github.com/jianzhnie/self_supervised/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247871313,"owners_count":21010021,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["beit","byol","mae","moco","self-supervised-learning","simclr"],"created_at":"2024-10-10T18:14:28.186Z","updated_at":"2025-04-08T15:34:41.097Z","avatar_url":"https://github.com/jianzhnie.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003c!--\n * @Author: jianzhnie\n * @Date: 2021-12-13 12:19:08\n * @LastEditTime: 2021-12-14 14:47:11\n * @LastEditors: jianzhnie\n * @Description:\n *\n--\u003e\n# Self-Supervised Learning\n\n### Features\n\n[self_supervised](./self_supervised) offers features like\n\n- modular framework\n- support for multi-gpu training using PyTorch Lightning\n- easy to use and written in a PyTorch like style\n- supports custom backbone models for self-supervised pre-training\n\n#### Supported Models\n\n- [MoCo, 2019](https://arxiv.org/abs/1911.05722)\n- [SimCLR, 2020](https://arxiv.org/abs/2002.05709)\n- [SimSiam, 2021](https://arxiv.org/abs/2011.10566)\n- [Barlow Twins, 2021](https://arxiv.org/abs/2103.03230)\n- [BYOL, 2020](https://arxiv.org/abs/2006.07733)\n- [NNCLR, 2021](https://arxiv.org/abs/2104.14548)\n- [SwaV, 2020](https://arxiv.org/abs/2006.09882)\n- [MocoV2, 2020]()\n- [MocoV3, 2021]()\n\n\n#### Supported Loss Function\n\n- [NegativeCosineSimilarity]()\n- [SwaVLoss]()\n\n### Benchmarks\nCurrently implemented models and their accuracy on cifar10 and imagenette.\n#### ImageNette\n\n| Model       | Epochs | Batch Size | Test Accuracy |\n|-------------|--------|------------|---------------|\n| MoCo        |  800   | 256        | 0.827         |\n| SimCLR      |  800   | 256        | 0.847         |\n| SimSiam     |  800   | 256        | 0.827          |\n| BarlowTwins |  800   | 256        | 0.801         |\n| BYOL        |  800   | 256        | 0.851         |\n\n\n#### Cifar10\n\n| Model       | Epochs | Batch Size | Test Accuracy |\n|-------------|--------|------------|---------------|\n| MoCo        |  200   | 128        | 0.83          |\n| SimCLR      |  200   | 128        | 0.78          |\n| SimSiam     |  200   | 128        | 0.73          |\n| BarlowTwins |  200   | 128        | 0.84          |\n| BYOL        |  200   | 128        | 0.85          |\n| MoCo        |  200   | 512        | 0.85          |\n| SimCLR      |  200   | 512        | 0.83          |\n| SimSiam     |  200   | 512        | 0.81          |\n| BarlowTwins |  200   | 512        | 0.78          |\n| BYOL        |  200   | 512        | 0.84          |\n| MoCo        |  800   | 128        | 0.89          |\n| SimCLR      |  800   | 128        | 0.87          |\n| SimSiam     |  800   | 128        | 0.80          |\n| MoCo        |  800   | 512        | 0.90          |\n| SimCLR      |  800   | 512        | 0.89          |\n| SimSiam     |  800   | 512        | 0.91          |\n\n### Tutorials\n\nWant to jump to the tutorials and see lightly in action?\n\n- [Train MoCo on CIFAR-10]()\n- [Train SimCLR on clothing data]()\n- [Train SimSiam on satellite images]()\n- [Use lightly with custom augmentations]()\n\n## Further Reading\n\n**Self-supervised Learning:**\n- [A Simple Framework for Contrastive Learning of Visual Representations (2020)](https://arxiv.org/abs/2002.05709)\n- [Momentum Contrast for Unsupervised Visual Representation Learning (2020)](https://arxiv.org/abs/1911.05722)\n- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments (2020)](https://arxiv.org/abs/2006.09882)\n- [What Should Not Be Contrastive in Contrastive Learning (2020)](https://arxiv.org/abs/2008.05659)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjianzhnie%2Fself_supervised","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjianzhnie%2Fself_supervised","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjianzhnie%2Fself_supervised/lists"}