{"id":15650153,"url":"https://github.com/sayakpaul/barlow-twins-tf","last_synced_at":"2025-04-30T16:43:51.519Z","repository":{"id":106647660,"uuid":"366022413","full_name":"sayakpaul/Barlow-Twins-TF","owner":"sayakpaul","description":"TensorFlow implementation of Barlow Twins (https://arxiv.org/abs/2103.03230).","archived":false,"fork":false,"pushed_at":"2021-06-11T10:21:44.000Z","size":491,"stargazers_count":41,"open_issues_count":0,"forks_count":6,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-30T17:51:14.921Z","etag":null,"topics":["computer-vision","information-bottleneck-theory","representation-learning","self-supervised-learning","tensorflow2"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sayakpaul.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-05-10T11:41:00.000Z","updated_at":"2024-02-02T14:01:05.000Z","dependencies_parsed_at":null,"dependency_job_id":"69e471fc-0c1d-4e84-aa96-78e84ee7a28d","html_url":"https://github.com/sayakpaul/Barlow-Twins-TF","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sayakpaul%2FBarlow-Twins-TF","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sayakpaul%2FBarlow-Twins-TF/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sayakpaul%2FBarlow-Twins-TF/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sayakpaul%2FBarlow-Twins-TF/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sayakpaul","download_url":"https://codeload.github.com/sayakpaul/Barlow-Twins-TF/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251746966,"owners_count":21637341,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","information-bottleneck-theory","representation-learning","self-supervised-learning","tensorflow2"],"created_at":"2024-10-03T12:33:36.726Z","updated_at":"2025-04-30T16:43:51.416Z","avatar_url":"https://github.com/sayakpaul.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Barlow-Twins-TF\n\n\u003cdiv align=\"center\"\u003e\n\u003ca href=\"https://colab.research.google.com/github/sayakpaul/Barlow-Twins-TF/blob/main/Barlow_Twins.ipynb\" target=\"_parent\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/\u003e\u003c/a\u003e\n\u003c/div\u003e\u003cbr\u003e\n\nThis repository implements **Barlow Twins** ([Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://arxiv.org/abs/2103.03230)) in TensorFlow and demonstrates it on the CIFAR10 dataset.\n\n**Summary**:\n\nWith a ResNet20 as a trunk and a 3-layer MLP (each layer containing 2048 units) and 100 epochs of pre-training, [this training notebook](https://github.com/sayakpaul/Barlow-Twins-TF/blob/main/Barlow_Twins.ipynb) can give **62.61%** accuracy on the CIFAR10 test set. The pre-training total takes ~23 minutes on a single Tesla V100. There are minor differences from the [original implementation](https://github.com/facebookresearch/barlowtwins/). However, the original loss function and the other minor details like having a big enough projection dimension have been maintained.\n\nFor details on Barlow Twins, I suggest reading the original paper, it's really well-written. \n\n## Loss progress during pre-training\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://i.ibb.co/9Y0STVZ/image.png\"\u003e\u003c/img\u003e\n\u003c/div\u003e\n\n## Other notes\n* Pre-trained model is available [here](https://github.com/sayakpaul/Barlow-Twins-TF/releases/download/v1.0.0/barlow_twins.tar.gz). \n* To follow the original implementation details as closely as possible, a WarmUpCosine learning rate schedule has been used during pre-training:\n  \n  \u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"https://i.ibb.co/khbLyvZ/image.png\"\u003e\u003c/img\u003e\n  \u003c/div\u003e\n* During linear evaluation, Cosine Decay has been used. \n\n## Acknowledgements\n\nThanks to Stéphane Deny (one of the authors of the paper) for helping me catch a pesky bug.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsayakpaul%2Fbarlow-twins-tf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsayakpaul%2Fbarlow-twins-tf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsayakpaul%2Fbarlow-twins-tf/lists"}