{"id":15784303,"url":"https://github.com/deepmancer/byol-pytorch","last_synced_at":"2025-04-01T16:30:49.066Z","repository":{"id":189022420,"uuid":"679892905","full_name":"deepmancer/byol-pytorch","owner":"deepmancer","description":"BYOL (Bootstrap Your Own Latent), implemented from scratch in Pytorch","archived":false,"fork":false,"pushed_at":"2024-08-16T11:23:33.000Z","size":258,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-10-11T20:06:19.592Z","etag":null,"topics":["bootstrap","byol","contrastive-learning","from-scratch","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/deepmancer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-17T21:28:05.000Z","updated_at":"2024-08-26T08:52:07.000Z","dependencies_parsed_at":"2024-08-16T12:10:43.093Z","dependency_job_id":null,"html_url":"https://github.com/deepmancer/byol-pytorch","commit_stats":null,"previous_names":["alirezahr79/byol","alirezaheidari-cs/byol","deepmancer/byol","deepmancer/byol-pytorch"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepmancer%2Fbyol-pytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepmancer%2Fbyol-pytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepmancer%2Fbyol-pytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepmancer%2Fbyol-pytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/deepmancer","download_url":"https://codeload.github.com/deepmancer/byol-pytorch/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246620337,"owners_count":20806754,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bootstrap","byol","contrastive-learning","from-scratch","pytorch"],"created_at":"2024-10-04T20:01:49.423Z","updated_at":"2025-04-01T16:30:49.053Z","avatar_url":"https://github.com/deepmancer.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 📈 BYOL (Bootstrap Your Own Latent) – From Scratch Implementation in PyTorch\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/PyTorch-%23EE4C2C.svg?style=for-the-badge\u0026logo=PyTorch\u0026logoColor=white\" alt=\"PyTorch\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/Python-3670A0?style=for-the-badge\u0026logo=Python\u0026logoColor=ffdd54\" alt=\"Python\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/Jupyter-F37626.svg?\u0026style=for-the-badge\u0026logo=Jupyter\u0026logoColor=white\" alt=\"Jupyter Notebook\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/license-MIT-blue.svg?style=for-the-badge\" alt=\"License\"\u003e\n  \u003cimg src=\"https://img.shields.io/github/stars/deepmancer/byol-pytorch?style=for-the-badge\" alt=\"GitHub Stars\"\u003e\n\u003c/p\u003e\n\nWelcome to the **BYOL (Bootstrap Your Own Latent)** repository! This project provides a comprehensive, from-scratch implementation of BYOL, a revolutionary self-supervised learning algorithm. Whether you're a researcher, developer, or enthusiast, this repository offers valuable insights into state-of-the-art unsupervised feature learning.\n\nIf you find this project useful, please star this repository! ⭐\n\n---\n\n## 🚀 Key Highlights\n\n- **From-Scratch Implementation**: Dive deep into the mechanics of BYOL with clean, understandable PyTorch code.\n- **Self-Supervised Learning**: Explore a negative-sample-free approach to feature learning.\n- **Documentation**: Clear explanations, visualizations, and results to aid your understanding.\n- **Interactive Notebooks**: Experiment hands-on with our Jupyter notebooks.\n\n---\n\n## 🧠 What is BYOL?\n\n**Bootstrap Your Own Latent (BYOL)** is a self-supervised learning algorithm that breaks new ground by learning from positive pairs (different augmentations of the same image) without relying on negative samples. This simplicity leads to faster training and competitive results across a range of benchmarks.\n\n### ✨ Key Features:\n- **No Negative Samples**: Reduces complexity while maintaining high performance.\n- **State-of-the-Art Results**: Competes effectively with contrastive methods on various image classification tasks.\n- **Minimal Computational Overhead**: Optimized for efficiency during training.\n\n\u003ch3 align=\"center\"\u003eBYOL Model Overview\u003c/h3\u003e\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/deepmancer/byol-pytorch/main/images/Byol.jpg\" width=\"600\" alt=\"BYOL Model Architecture\"\u003e\n\u003c/p\u003e\n\n---\n\n## 📊 Experimental Results\n\nWe evaluated our implementation using the **STL10 Dataset**, showcasing the impact of BYOL pretraining:\n\n| **Training Method**      | **Accuracy** |\n|---------------------------|--------------|\n| Without Pretraining       | 84.58%       |\n| With BYOL Pretraining     | **87.61%**   |\n\nThis demonstrates how BYOL can effectively enhance feature learning, even with relatively few epochs.\n\n---\n\n## 📁 Dataset: STL10\n\nThe **STL10 Dataset** is designed for unsupervised and self-supervised learning models, making it a perfect fit for BYOL.\n\n- **Classes**: 10 categories, including animals and vehicles.\n- **Train/Test Split**: 500 training images and 800 test images per class.\n- **Source**: [STL10 Dataset](https://cs.stanford.edu/~acoates/stl10/)\n\n\u003ch3 align=\"center\"\u003eSTL10 Dataset Sample\u003c/h3\u003e\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://cs.stanford.edu/~acoates/stl10/images.png\" width=\"600\" alt=\"STL10 Dataset Example\"\u003e\n\u003c/p\u003e\n\n---\n\n## 🔧 Getting Started\n\n**Prerequisites**\n- **Python 3.6+**\n- **PyTorch**\n- **Torchvision**\n- **Jupyter Notebook** (for interactive experiments)\n\nClone the repository and install the required dependencies\n\n---\n\n## 📚 Citations\n\n```bibtex\n@inproceedings{NEURIPS2020_f3ada80d,\n  author = {Grill, Jean-Bastien and Strub, Florian and Altch\\'{e}, Florent and Tallec, Corentin and Richemond, Pierre and Buchatskaya, Elena and Doersch, Carl and Avila Pires, Bernardo and Guo, Zhaohan and Gheshlaghi Azar, Mohammad and Piot, Bilal and kavukcuoglu, koray and Munos, Remi and Valko, Michal},\n  booktitle = {Advances in Neural Information Processing Systems},\n  editor = {H. Larochelle and M. Ranzato and R. Hadsell and M.F. Balcan and H. Lin},\n  pages = {21271--21284},\n  publisher = {Curran Associates, Inc.},\n  title = {Bootstrap Your Own Latent - A New Approach to Self-Supervised Learning},\n  url = {https://proceedings.neurips.cc/paper_files/paper/2020/file/f3ada80d5c4ee70142b17b8192b2958e-Paper.pdf},\n  volume = {33},\n  year = {2020}\n}\n```\n\n---\n\n## 🧾 License\n\nThis project is licensed under the **MIT License**. See the [LICENSE](LICENSE) file for details.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeepmancer%2Fbyol-pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdeepmancer%2Fbyol-pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeepmancer%2Fbyol-pytorch/lists"}