{"id":16402541,"url":"https://github.com/spijkervet/byol","last_synced_at":"2025-07-01T06:32:56.808Z","repository":{"id":56176726,"uuid":"278345036","full_name":"Spijkervet/BYOL","owner":"Spijkervet","description":"Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning","archived":false,"fork":false,"pushed_at":"2022-07-12T07:16:16.000Z","size":55,"stargazers_count":131,"open_issues_count":8,"forks_count":26,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-04-01T23:13:01.149Z","etag":null,"topics":["byol","deep-learning","pytorch","self-supervised-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Spijkervet.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-07-09T11:24:26.000Z","updated_at":"2025-03-30T00:47:03.000Z","dependencies_parsed_at":"2022-08-15T14:10:18.162Z","dependency_job_id":null,"html_url":"https://github.com/Spijkervet/BYOL","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/Spijkervet/BYOL","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Spijkervet%2FBYOL","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Spijkervet%2FBYOL/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Spijkervet%2FBYOL/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Spijkervet%2FBYOL/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Spijkervet","download_url":"https://codeload.github.com/Spijkervet/BYOL/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Spijkervet%2FBYOL/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262913514,"owners_count":23383723,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["byol","deep-learning","pytorch","self-supervised-learning"],"created_at":"2024-10-11T05:46:31.508Z","updated_at":"2025-07-01T06:32:56.703Z","avatar_url":"https://github.com/Spijkervet.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# BYOL - Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning\nPyTorch implementation of \"Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning\" by J.B. Grill et al.\n\n[Link to paper](https://arxiv.org/abs/2006.07733)\n\nThis repository includes a practical implementation of BYOL with:\n- **Distributed Data Parallel training**\n- Benchmarks on vision datasets (CIFAR-10 / STL-10)\n- Support for PyTorch **\u003c= 1.5.0**\n\nOpen BYOL in Google Colab Notebook\n\n[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1B68Ag_oRB0-rbb9AwC20onmknxyYho4B?usp=sharing)\n\n## Results\nThese are the top-1 accuracy of linear classifiers trained on the (frozen) representations learned by BYOL:\n\n| Method  | Batch size | Image size | ResNet | Projection output dim. | Pre-training epochs | Optimizer | STL-10 | CIFAR-10\n| ------------- | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- |\n| BYOL + linear eval.  | 192 | 224x224 | ResNet18 | 256 | 100 | Adam | _ | **0.832** | \n| Logistic Regression | - | - | - | - | - | - | 0.358 | 0.389 |\n\n\n## Installation\n```\ngit clone https://github.com/spijkervet/byol --recurse-submodules -j8\npip3 install -r requirements.txt\npython3 main.py\n```\n\n\n## Usage\n### Using a pre-trained model\nThe following commands will train a logistic regression model on a pre-trained ResNet18, yielding a top-1 accuracy of 83.2% on CIFAR-10.\n```\ncurl https://github.com/Spijkervet/BYOL/releases/download/1.0/resnet18-CIFAR10-final.pt -L -O\nrm features.p\npython3 logistic_regression.py --model_path resnet18-CIFAR10-final.pt\n```\n\n### Pre-training\nTo run pre-training using BYOL with the default arguments (1 node, 1 GPU), use:\n```\npython3 main.py\n```\n\nWhich is equivalent to:\n```\npython3 main.py --nodes 1 --gpus 1\n```\nThe pre-trained models are saved every *n* epochs in \\*.pt files, the final model being `model-final.pt`\n\n### Finetuning\nFinetuning a model ('linear evaluation') on top of the pre-trained, frozen ResNet model can be done using:\n```\npython3 logistic_regression.py --model_path=./model_final.pt\n```\n\nWith `model_final.pt` being file containing the pre-trained network from the pre-training stage.\n\n## Multi-GPU / Multi-node training\nUse `python3 main.py --gpus 2` to train e.g. on 2 GPU's, and `python3 main.py --gpus 2 --nodes 2` to train with 2 GPU's using 2 nodes.\nSee https://yangkky.github.io/2019/07/08/distributed-pytorch-tutorial.html for an excellent explanation.\n\n## Arguments\n```\n--image_size, default=224, \"Image size\"\n--learning_rate, default=3e-4, \"Initial learning rate.\"\n--batch_size, default=42, \"Batch size for training.\"\n--num_epochs, default=100, \"Number of epochs to train for.\"\n--checkpoint_epochs, default=10, \"Number of epochs between checkpoints/summaries.\"\n--dataset_dir, default=\"./datasets\", \"Directory where dataset is stored.\",\n--num_workers, default=8, \"Number of data loading workers (caution with nodes!)\"\n--nodes, default=1, \"Number of nodes\"\n--gpus, default=1, \"number of gpus per node\"\n--nr, default=0, \"ranking within the nodes\"\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspijkervet%2Fbyol","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fspijkervet%2Fbyol","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspijkervet%2Fbyol/lists"}