{"id":66245,"url":"https://github.com/lightly-ai/awesome-self-supervised-learning","name":"awesome-self-supervised-learning","description":"Curated List of papers on Self-Supervised Representation Learning","projects_count":68,"last_synced_at":"2026-07-22T07:00:21.642Z","repository":{"id":256887980,"uuid":"845490894","full_name":"lightly-ai/awesome-self-supervised-learning","owner":"lightly-ai","description":"Curated List of papers on Self-Supervised Representation Learning","archived":false,"fork":false,"pushed_at":"2025-04-16T10:31:04.000Z","size":18,"stargazers_count":20,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2026-06-16T01:04:04.964Z","etag":null,"topics":["awesome-list","awesome-lists","self-supervised-learning"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lightly-ai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-08-21T11:00:30.000Z","updated_at":"2026-06-09T07:28:35.000Z","dependencies_parsed_at":"2025-05-07T22:01:07.798Z","dependency_job_id":"24c901c8-8b51-4feb-a99b-958077656ec9","html_url":"https://github.com/lightly-ai/awesome-self-supervised-learning","commit_stats":null,"previous_names":["lightly-ai/awesome-self-supervised-learning"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/lightly-ai/awesome-self-supervised-learning","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lightly-ai%2Fawesome-self-supervised-learning","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lightly-ai%2Fawesome-self-supervised-learning/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lightly-ai%2Fawesome-self-supervised-learning/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lightly-ai%2Fawesome-self-supervised-learning/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lightly-ai","download_url":"https://codeload.github.com/lightly-ai/awesome-self-supervised-learning/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lightly-ai%2Fawesome-self-supervised-learning/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35751644,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-07-20T02:08:10.276Z","status":"online","status_checked_at":"2026-07-22T02:00:06.236Z","response_time":124,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"created_at":"2024-09-25T01:46:28.959Z","updated_at":"2026-07-22T07:00:21.642Z","primary_language":null,"list_of_lists":false,"displayable":true,"categories":["2021","2022","2023","2020","2019","2018","2016","2024","Uncategorized"],"sub_categories":["Uncategorized"],"readme":"# Awesome Self Supervised Learning [![awesome](https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg)](https://github.com/sindresorhus/awesome) [![Discord](https://img.shields.io/discord/752876370337726585?logo=discord\u0026logoColor=white\u0026label=discord\u0026color=7289da)](https://discord.gg/xvNJW94)\n\nWant to leverage state of the art Self-Supervised Learning and Distillation to pretrain your models? Check out the following tools by the team from [Lightly AI](https://www.lightly.ai/):\n - ⚡️ [Lightly**Train**](https://github.com/lightly-ai/lightly-train): A framework to pretrain your computer vision backbones in 3 lines of code.\n - 💡 [Lightly**SSL**](https://github.com/lightly-ai/lightly): A research-focused collection of state-of-the-art self-supervised training methods.\n\n## 2024\n\n| Title | Relevant Links |\n|:-----|:--------------|\n| [Scalable Pre-training of Large Autoregressive Image Models](https://arxiv.org/abs/2401.08541) | [![arXiv](https://img.shields.io/badge/arXiv-2401.08541-b31b1b.svg)](https://arxiv.org/abs/2401.08541) [![Open In Colab](https://img.shields.io/badge/Colab-PyTorch-blue?logo=googlecolab)](https://colab.research.google.com/github/lightly-ai/lightly/blob/master/examples/notebooks/pytorch/aim.ipynb) |\n| [SAM 2: Segment Anything in Images and Videos](https://arxiv.org/abs/2408.00714) | [![arXiv](https://img.shields.io/badge/arXiv-2408.00714-b31b1b.svg)](https://arxiv.org/abs/2408.00714) [![Google Drive](https://img.shields.io/badge/Lightly_Reading_Group-4285F4?logo=googledrive\u0026logoColor=white)](https://drive.google.com/file/d/1kWvZclajy7z3ize2KNCLzCfvZN2pDien/view?usp=sharing) |\n| [Automatic Data Curation for Self-Supervised Learning: A Clustering-Based Approach](https://arxiv.org/abs/2405.15613) | [![arXiv](https://img.shields.io/badge/arXiv-2405.15613-b31b1b.svg)](https://arxiv.org/abs/2405.15613) |\n| [GLID: Pre-training a Generalist Encoder-Decoder Vision Model](https://arxiv.org/abs/2404.07603) | [![arXiv](https://img.shields.io/badge/arXiv-2404.07603-b31b1b.svg)](https://arxiv.org/abs/2404.07603) [![Google Drive](https://img.shields.io/badge/Lightly_Reading_Group-4285F4?logo=googledrive\u0026logoColor=white)](https://drive.google.com/file/d/1CEaZ00z-0hqGKp5cTN8fxP6tsHiHkFye/view?usp=sharing) |\n| [Rethinking Patch Dependence for Masked Autoencoders](https://arxiv.org/abs/2401.14391) | [![arXiv](https://img.shields.io/badge/arXiv-2401.14391-b31b1b.svg)](https://arxiv.org/abs/2401.14391) [![Google Drive](https://img.shields.io/badge/Lightly_Reading_Group-4285F4?logo=googledrive\u0026logoColor=white)](https://drive.google.com/file/d/1LtIPoes3y1ZOHD-UBeKgj9AYBoQ-nO5A/view?usp=sharing) |\n| [You Don't Need Data-Augmentation in Self-Supervised Learning](https://arxiv.org/abs/2406.09294) | [![arXiv](https://img.shields.io/badge/arXiv-2406.09294-b31b1b.svg)](https://arxiv.org/abs/2406.09294) |\n| [Occam's Razor for Self Supervised Learning: What is Sufficient to Learn Good Representations?](https://arxiv.org/abs/2406.10743) | [![arXiv](https://img.shields.io/badge/arXiv-2406.10743-b31b1b.svg)](https://arxiv.org/abs/2406.10743) |\n| [Asymmetric Masked Distillation for Pre-Training Small Foundation Models](https://openaccess.thecvf.com/content/CVPR2024/papers/Zhao_Asymmetric_Masked_Distillation_for_Pre-Training_Small_Foundation_Models_CVPR_2024_paper.pdf) | [![CVPR](https://img.shields.io/badge/CVPR-2024-b31b1b.svg)](https://openaccess.thecvf.com/content/CVPR2024/papers/Zhao_Asymmetric_Masked_Distillation_for_Pre-Training_Small_Foundation_Models_CVPR_2024_paper.pdf) [![GitHub](https://img.shields.io/badge/GitHub-100000?\u0026logo=github\u0026logoColor=white)](https://github.com/MCG-NJU/AMD) |\n| [Revisiting Feature Prediction for Learning Visual Representations from Video](https://arxiv.org/abs/2404.08471) | [![arXiv](https://img.shields.io/badge/arXiv-2404.08471-b31b1b.svg)](https://arxiv.org/abs/2404.08471) [![GitHub](https://img.shields.io/badge/GitHub-100000?\u0026logo=github\u0026logoColor=white)](https://github.com/facebookresearch/jepa) |\n| [Rethinking Patch Dependence for Masked Autoencoders](https://arxiv.org/abs/2401.14391) | [![arXiv](https://img.shields.io/badge/arXiv-2401.14391-b31b1b.svg)](https://arxiv.org/abs/2401.14391) [![GitHub](https://img.shields.io/badge/GitHub-100000?\u0026logo=github\u0026logoColor=white)](https://github.com/TonyLianLong/CrossMAE) |\n| [ARVideo: Autoregressive Pretraining for Self-Supervised Video Representation Learning](https://arxiv.org/abs/2405.15160) | [![arXiv](https://img.shields.io/badge/arXiv-2405.15160-b31b1b.svg)](https://arxiv.org/abs/2405.15160) |\n\n## 2023\n\n| Title | Relevant Links |\n|:-----|:--------------|\n| [A Cookbook of Self-Supervised Learning](https://arxiv.org/abs/2304.12210) | [![arXiv](https://img.shields.io/badge/arXiv-2304.12210-b31b1b.svg)](https://arxiv.org/abs/2304.12210) |\n| [Masked Autoencoders Enable Efficient Knowledge Distillers](https://arxiv.org/abs/2208.12256) | [![arXiv](https://img.shields.io/badge/arXiv-2208.12256-b31b1b.svg)](https://arxiv.org/abs/2208.12256) [![Google Drive](https://img.shields.io/badge/Lightly_Reading_Group-4285F4?logo=googledrive\u0026logoColor=white)](https://drive.google.com/file/d/1bzuOab5fvKK7jpxv5bMoGk1gW446SCUL/view?usp=sharing) |\n| [Understanding and Generalizing Contrastive Learning from the Inverse Optimal Transport Perspective](https://openreview.net/forum?id=DBlWCsOy94) | [![CVPR](https://img.shields.io/badge/CVPR-ICML_2023-b31b1b.svg)](https://openreview.net/forum?id=DBlWCsOy94) [![Google Drive](https://img.shields.io/badge/Lightly_Reading_Group-4285F4?logo=googledrive\u0026logoColor=white)](https://drive.google.com/file/d/1hBEy-yh_KtkqY3rjeato-Cuo6ITzhowr/view?usp=sharing) |\n| [CycleCL: Self-supervised Learning for Periodic Videos](https://arxiv.org/abs/2311.03402) | [![arXiv](https://img.shields.io/badge/arXiv-2311.03402-b31b1b.svg)](https://arxiv.org/abs/2311.03402) [![Google Drive](https://img.shields.io/badge/Lightly_Reading_Group-4285F4?logo=googledrive\u0026logoColor=white)](https://drive.google.com/file/d/1BDC891HX_JxF84UK_x8RKgHZockJqQFU/view?usp=sharing) |\n| [Temperature Schedules for Self-Supervised Contrastive Methods on Long-Tail Data](https://arxiv.org/abs/2303.13664) | [![arXiv](https://img.shields.io/badge/arXiv-2303.13664-b31b1b.svg)](https://arxiv.org/abs/2303.13664) [![Google Drive](https://img.shields.io/badge/Lightly_Reading_Group-4285F4?logo=googledrive\u0026logoColor=white)](https://drive.google.com/file/d/1RabJuwtOevH9hg9wuFTN4z8y4gjQxCT_/view?usp=sharing)  |\n| [Reverse Engineering Self-Supervised Learning](https://arxiv.org/abs/2305.15614) | [![arXiv](https://img.shields.io/badge/arXiv-2305.15614-b31b1b.svg)](https://arxiv.org/abs/2305.15614) [![Google Drive](https://img.shields.io/badge/Lightly_Reading_Group-4285F4?logo=googledrive\u0026logoColor=white)](https://drive.google.com/file/d/1KsqV9_HE0y0EwlNivUdZPKqqCkdM-4HB/view?usp=sharing) |\n| [Improved baselines for vision-language pre-training](https://arxiv.org/abs/2305.08675) | [![arXiv](https://img.shields.io/badge/arXiv-2305.08675-b31b1b.svg)](https://arxiv.org/abs/2305.08675) [![Google Drive](https://img.shields.io/badge/Lightly_Reading_Group-4285F4?logo=googledrive\u0026logoColor=white)](https://drive.google.com/file/d/1CNLvxt1jri7chCGy2ZqXBzDwPko0s6QP/view?usp=sharing) |\n| [DINOv2: Learning Robust Visual Features without Supervision](https://arxiv.org/abs/2304.07193) | [![arXiv](https://img.shields.io/badge/arXiv-2304.07193-b31b1b.svg)](https://arxiv.org/abs/2304.07193) [![Google Drive](https://img.shields.io/badge/Lightly_Reading_Group-4285F4?logo=googledrive\u0026logoColor=white)](https://drive.google.com/file/d/11szszgtsYESO3QF8jkFsLFTVtN797uH2/view?usp=sharing) |\n| [Segment Anything](https://arxiv.org/abs/2304.02643) | [![arXiv](https://img.shields.io/badge/arXiv-2304.02643-b31b1b.svg)](https://arxiv.org/abs/2304.02643) [![Google Drive](https://img.shields.io/badge/Lightly_Reading_Group-4285F4?logo=googledrive\u0026logoColor=white)](https://drive.google.com/file/d/18yPuL8J6boi5pB1NRO6VAUbYEwmI3tFo/view?usp=sharing) |\n| [Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture](https://arxiv.org/abs/2301.08243) | [![arXiv](https://img.shields.io/badge/arXiv-2301.08243-b31b1b.svg)](https://arxiv.org/abs/2301.08243) [![Google Drive](https://img.shields.io/badge/Lightly_Reading_Group-4285F4?logo=googledrive\u0026logoColor=white)](https://drive.google.com/file/d/1l5nHxqqbv7o3ESw3DLBqgJyXILJ0FgH6/view?usp=sharing) |\n| [Self-supervised Object-Centric Learning for Videos](https://arxiv.org/abs/2310.06907) | [![NeurIPS](https://img.shields.io/badge/NeurIPS_2023-2310.06907-b31b1b.svg)](https://arxiv.org/abs/2310.06907) |\n| [Patch n’ Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution](https://proceedings.neurips.cc/paper_files/paper/2023/file/06ea400b9b7cfce6428ec27a371632eb-Paper-Conference.pdf) | [![NeurIPS](https://img.shields.io/badge/NeurIPS_2023-2310.06907-b31b1b.svg)](https://proceedings.neurips.cc/paper_files/paper/2023/file/06ea400b9b7cfce6428ec27a371632eb-Paper-Conference.pdf) |\n| [An Information-Theoretic Perspective on Variance-Invariance-Covariance Regularization](https://proceedings.neurips.cc/paper_files/paper/2023/file/6b1d4c03391b0aa6ddde0b807a78c950-Paper-Conference.pdf) | [![NeurIPS](https://img.shields.io/badge/NeurIPS_2023-b31b1b.svg)](https://proceedings.neurips.cc/paper_files/paper/2023/file/6b1d4c03391b0aa6ddde0b807a78c950-Paper-Conference.pdf) |\n| [The Role of Entropy and Reconstruction in Multi-View Self-Supervised Learning](https://arxiv.org/abs/2307.10907) | [![arXiv](https://img.shields.io/badge/arXiv-2307.10907-b31b1b.svg)](https://arxiv.org/abs/2307.10907) [![GitHub](https://img.shields.io/badge/GitHub-100000?\u0026logo=github\u0026logoColor=white)](https://github.com/apple/ml-entropy-reconstruction) |\n| [Fast Segment Anything](https://arxiv.org/abs/2306.12156) | [![arXiv](https://img.shields.io/badge/arXiv-2306.12156-b31b1b.svg)](https://arxiv.org/abs/2306.12156) [![GitHub](https://img.shields.io/badge/GitHub-100000?\u0026logo=github\u0026logoColor=white)](https://github.com/CASIA-IVA-Lab/FastSAM) |\n| [Faster Segment Anything: Towards Lightweight SAM for Mobile Applications](https://arxiv.org/abs/2306.14289) | [![arXiv](https://img.shields.io/badge/arXiv-2306.14289-b31b1b.svg)](https://arxiv.org/abs/2306.14289) [![GitHub](https://img.shields.io/badge/GitHub-100000?\u0026logo=github\u0026logoColor=white)](https://github.com/ChaoningZhang/MobileSAM) |\n| [What Do Self-Supervised Vision Transformers Learn?](https://arxiv.org/abs/2305.00729) | [![arXiv](https://img.shields.io/badge/ICLR_2023-2305.00729-b31b1b.svg)](https://arxiv.org/abs/2305.00729) [![GitHub](https://img.shields.io/badge/GitHub-100000?\u0026logo=github\u0026logoColor=white)](https://github.com/naver-ai/cl-vs-mim) |\n| [Improved baselines for vision-language pre-training](https://arxiv.org/abs/2305.08675) | [![arXiv](https://img.shields.io/badge/arXiv-2305.08675-b31b1b.svg)](https://arxiv.org/abs/2305.08675) [![GitHub](https://img.shields.io/badge/GitHub-100000?\u0026logo=github\u0026logoColor=white)](https://github.com/facebookresearch/clip-rocket) |\n| [Active Self-Supervised Learning: A Few Low-Cost Relationships Are All You Need](https://arxiv.org/abs/2303.15256) | [![arXiv](https://img.shields.io/badge/arXiv-2303.15256-b31b1b.svg)](https://arxiv.org/abs/2303.15256) |\n| [EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything](https://arxiv.org/abs/2312.00863) | [![arXiv](https://img.shields.io/badge/arXiv-2312.00863-b31b1b.svg)](https://arxiv.org/abs/2312.00863) [![GitHub](https://img.shields.io/badge/GitHub-100000?\u0026logo=github\u0026logoColor=white)](https://github.com/yformer/EfficientSAM) |\n| [DropPos: Pre-Training Vision Transformers by Reconstructing Dropped Positions](https://arxiv.org/abs/2309.03576) | [![arXiv](https://img.shields.io/badge/arXiv-2309.03576-b31b1b.svg)](https://arxiv.org/abs/2309.03576) [![GitHub](https://img.shields.io/badge/GitHub-100000?\u0026logo=github\u0026logoColor=white)](https://github.com/Haochen-Wang409/DropPos) |\n| [VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking](https://openaccess.thecvf.com/content/CVPR2023/papers/Wang_VideoMAE_V2_Scaling_Video_Masked_Autoencoders_With_Dual_Masking_CVPR_2023_paper.pdf) | [![CVPR](https://img.shields.io/badge/CVPR-2023-b31b1b.svg)](https://openaccess.thecvf.com/content/CVPR2023/papers/Wang_VideoMAE_V2_Scaling_Video_Masked_Autoencoders_With_Dual_Masking_CVPR_2023_paper.pdf) |\n| [MGMAE: Motion Guided Masking for Video Masked Autoencoding](https://openaccess.thecvf.com/content/ICCV2023/papers/Huang_MGMAE_Motion_Guided_Masking_for_Video_Masked_Autoencoding_ICCV_2023_paper.pdf) | [![CVPR](https://img.shields.io/badge/CVPR-2023-b31b1b.svg)](https://openaccess.thecvf.com/content/ICCV2023/papers/Huang_MGMAE_Motion_Guided_Masking_for_Video_Masked_Autoencoding_ICCV_2023_paper.pdf) [![GitHub](https://img.shields.io/badge/GitHub-100000?\u0026logo=github\u0026logoColor=white)](https://github.com/MCG-NJU/MGMAE) |\n\n## 2022\n\n| Title | Relevant Links |\n|:-----|:--------------|\n| [Masked Siamese Networks for Label-Efficient Learning](https://arxiv.org/abs/2204.07141) | [![arXiv](https://img.shields.io/badge/arXiv-2204.07141-b31b1b.svg)](https://arxiv.org/abs/2204.07141) [![Google Drive](https://img.shields.io/badge/Lightly_Reading_Group-4285F4?logo=googledrive\u0026logoColor=white)](https://drive.google.com/file/d/15WGpYpxy4_1a927RWrmlkeJohZDznN8e/view?usp=sharing) [![Open In Colab](https://img.shields.io/badge/Colab-PyTorch-blue?logo=googlecolab)](https://colab.research.google.com/github/lightly-ai/lightly/blob/master/examples/notebooks/pytorch/msn.ipynb) |\n| [The Hidden Uniform Cluster Prior in Self-Supervised Learning](https://arxiv.org/abs/2210.07277) | [![arXiv](https://img.shields.io/badge/arXiv-2210.07277-b31b1b.svg)](https://arxiv.org/abs/2210.07277) [![Open In Colab](https://img.shields.io/badge/Colab-PyTorch-blue?logo=googlecolab)](https://colab.research.google.com/github/lightly-ai/lightly/blob/master/examples/notebooks/pytorch/pmsn.ipynb) |\n| [Unsupervised Visual Representation Learning by Synchronous Momentum Grouping](https://arxiv.org/abs/2207.06167) | [![arXiv](https://img.shields.io/badge/arXiv-2207.06167-b31b1b.svg)](https://arxiv.org/abs/2207.06167) [![Open In Colab](https://img.shields.io/badge/Colab-PyTorch-blue?logo=googlecolab)](https://colab.research.google.com/github/lightly-ai/lightly/blob/master/examples/notebooks/pytorch/smog.ipynb) |\n| [TiCo: Transformation Invariance and Covariance Contrast for Self-Supervised Visual Representation Learning](https://arxiv.org/abs/2206.10698) | [![arXiv](https://img.shields.io/badge/arXiv-2206.10698-b31b1b.svg)](https://arxiv.org/abs/2206.10698) [![Open In Colab](https://img.shields.io/badge/Colab-PyTorch-blue?logo=googlecolab)](https://colab.research.google.com/github/lightly-ai/lightly/blob/master/examples/notebooks/pytorch/tico.ipynb) |\n| [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://arxiv.org/abs/2105.04906) | [![arXiv](https://img.shields.io/badge/arXiv-2105.04906-b31b1b.svg)](https://arxiv.org/abs/2105.04906) [![Open In Colab](https://img.shields.io/badge/Colab-PyTorch-blue?logo=googlecolab)](https://colab.research.google.com/github/lightly-ai/lightly/blob/master/examples/notebooks/pytorch/vicreg.ipynb) |\n| [VICRegL: Self-Supervised Learning of Local Visual Features](https://arxiv.org/abs/2210.01571) | [![arXiv](https://img.shields.io/badge/arXiv-2210.01571-b31b1b.svg)](https://arxiv.org/abs/2210.01571) [![Open In Colab](https://img.shields.io/badge/Colab-PyTorch-blue?logo=googlecolab)](https://colab.research.google.com/github/lightly-ai/lightly/blob/master/examples/notebooks/pytorch/vicregl.ipynb) |\n| [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://arxiv.org/abs/2203.12602) | [![arXiv](https://img.shields.io/badge/arXiv-2203.12602-b31b1b.svg)](https://arxiv.org/abs/2203.12602) [![Google Drive](https://img.shields.io/badge/Lightly_Reading_Group-4285F4?logo=googledrive\u0026logoColor=white)](https://drive.google.com/file/d/1F0oyiyyxCKzWS9Gv8TssHxaCMFnAoxfb/view?usp=sharing) |\n| [Improving Visual Representation Learning through Perceptual Understanding](https://arxiv.org/abs/2212.14504) | [![arXiv](https://img.shields.io/badge/arXiv-2212.14504-b31b1b.svg)](https://arxiv.org/abs/2212.14504) [![Google Drive](https://img.shields.io/badge/Lightly_Reading_Group-4285F4?logo=googledrive\u0026logoColor=white)](https://drive.google.com/file/d/1n4Y0iiM368RaPxPg6qvsfACguaolFnhf/view?usp=sharing) |\n| [RankMe: Assessing the downstream performance of pretrained self-supervised representations by their rank](https://arxiv.org/abs/2210.02885) | [![arXiv](https://img.shields.io/badge/arXiv-2210.02885-b31b1b.svg)](https://arxiv.org/abs/2210.02885) [![Google Drive](https://img.shields.io/badge/Lightly_Reading_Group-4285F4?logo=googledrive\u0026logoColor=white)](https://drive.google.com/file/d/1cEP1_G2wMM3-AMMrdntGN6Fq1E5qwPi1/view?usp=sharing) |\n| [A Closer Look at Self-Supervised Lightweight Vision Transformers](https://arxiv.org/abs/2205.14443) | [![arXiv](https://img.shields.io/badge/arXiv-2205.14443-b31b1b.svg)](https://arxiv.org/abs/2205.14443) [![GitHub](https://img.shields.io/badge/GitHub-100000?\u0026logo=github\u0026logoColor=white)](https://github.com/wangsr126/mae-lite) |\n| [Beyond neural scaling laws: beating power law scaling via data pruning](https://arxiv.org/abs/2206.14486) | [![arXiv](https://img.shields.io/badge/NeurIPS_2022-2206.14486-b31b1b.svg)](https://arxiv.org/abs/2206.14486) [![GitHub](https://img.shields.io/badge/GitHub-100000?\u0026logo=github\u0026logoColor=white)](https://github.com/rgeirhos/dataset-pruning-metrics) |\n| [A simple, efficient and scalable contrastive masked autoencoder for learning visual representations](https://arxiv.org/abs/2210.16870) | [![arXiv](https://img.shields.io/badge/arXiv-2210.16870-b31b1b.svg)](https://arxiv.org/abs/2210.16870) |\n| [Masked Autoencoders are Robust Data Augmentors](https://arxiv.org/abs/2206.04846) | [![arXiv](https://img.shields.io/badge/arXiv-2206.04846-b31b1b.svg)](https://arxiv.org/abs/2206.04846) |\n| [Is Self-Supervised Learning More Robust Than Supervised Learning?](https://arxiv.org/abs/2206.05259) | [![arXiv](https://img.shields.io/badge/arXiv-2206.05259-b31b1b.svg)](https://arxiv.org/abs/2206.05259) |\n| [Can CNNs Be More Robust Than Transformers?](https://arxiv.org/abs/2206.03452) | [![arXiv](https://img.shields.io/badge/arXiv-2206.03452-b31b1b.svg)](https://arxiv.org/abs/2206.03452) [![GitHub](https://img.shields.io/badge/GitHub-100000?\u0026logo=github\u0026logoColor=white)](https://github.com/UCSC-VLAA/RobustCNN) |\n| [Patch-level Representation Learning for Self-supervised Vision Transformers](https://arxiv.org/abs/2206.07990) | [![arXiv](https://img.shields.io/badge/arXiv-2206.07990-b31b1b.svg)](https://arxiv.org/abs/2206.07990) [![GitHub](https://img.shields.io/badge/GitHub-100000?\u0026logo=github\u0026logoColor=white)](https://github.com/alinlab/selfpatch) |\n\n## 2021\n\n| Title | Relevant Links |\n|:-----|:--------------|\n| [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://arxiv.org/abs/2103.03230) | [![arXiv](https://img.shields.io/badge/arXiv-2103.03230-b31b1b.svg)](https://arxiv.org/abs/2103.03230) [![Open In Colab](https://img.shields.io/badge/Colab-PyTorch-blue?logo=googlecolab)](https://colab.research.google.com/github/lightly-ai/lightly/blob/master/examples/notebooks/pytorch/barlowtwins.ipynb) |\n| [Decoupled Contrastive Learning](https://arxiv.org/abs/2110.06848) | [![arXiv](https://img.shields.io/badge/arXiv-2110.06848-b31b1b.svg)](https://arxiv.org/abs/2110.06848) [![Open In Colab](https://img.shields.io/badge/Colab-PyTorch-blue?logo=googlecolab)](https://colab.research.google.com/github/lightly-ai/lightly/blob/master/examples/notebooks/pytorch/dcl.ipynb) |\n| [Dense Contrastive Learning for Self-Supervised Visual Pre-Training](https://arxiv.org/abs/2011.09157) | [![arXiv](https://img.shields.io/badge/arXiv-2011.09157-b31b1b.svg)](https://arxiv.org/abs/2011.09157) [![Open In Colab](https://img.shields.io/badge/Colab-PyTorch-blue?logo=googlecolab)](https://colab.research.google.com/github/lightly-ai/lightly/blob/master/examples/notebooks/pytorch/densecl.ipynb) |\n| [Emerging Properties in Self-Supervised Vision Transformers](https://arxiv.org/abs/2104.14294) | [![arXiv](https://img.shields.io/badge/arXiv-2104.14294-b31b1b.svg)](https://arxiv.org/abs/2104.14294) [![Open In Colab](https://img.shields.io/badge/Colab-PyTorch-blue?logo=googlecolab)](https://colab.research.google.com/github/lightly-ai/lightly/blob/master/examples/notebooks/pytorch/dino.ipynb) |\n| [Masked Autoencoders Are Scalable Vision Learners](https://arxiv.org/abs/2111.06377) | [![arXiv](https://img.shields.io/badge/arXiv-2111.06377-b31b1b.svg)](https://arxiv.org/abs/2111.06377) [![Open In Colab](https://img.shields.io/badge/Colab-PyTorch-blue?logo=googlecolab)](https://colab.research.google.com/github/lightly-ai/lightly/blob/master/examples/notebooks/pytorch/mae.ipynb) |\n| [With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representations](https://arxiv.org/abs/2104.14548) | [![arXiv](https://img.shields.io/badge/arXiv-2104.14548-b31b1b.svg)](https://arxiv.org/abs/2104.14548) [![Open In Colab](https://img.shields.io/badge/Colab-PyTorch-blue?logo=googlecolab)](https://colab.research.google.com/github/lightly-ai/lightly/blob/master/examples/notebooks/pytorch/nnclr.ipynb) |\n| [SimMIM: A Simple Framework for Masked Image Modeling](https://arxiv.org/abs/2111.09886) | [![arXiv](https://img.shields.io/badge/arXiv-2111.09886-b31b1b.svg)](https://arxiv.org/abs/2111.09886) [![Open In Colab](https://img.shields.io/badge/Colab-PyTorch-blue?logo=googlecolab)](https://colab.research.google.com/github/lightly-ai/lightly/blob/master/examples/notebooks/pytorch/simmim.ipynb) |\n| [Exploring Simple Siamese Representation Learning](https://arxiv.org/abs/2011.10566) | [![arXiv](https://img.shields.io/badge/arXiv-2011.10566-b31b1b.svg)](https://arxiv.org/abs/2011.10566) [![Open In Colab](https://img.shields.io/badge/Colab-PyTorch-blue?logo=googlecolab)](https://colab.research.google.com/github/lightly-ai/lightly/blob/master/examples/notebooks/pytorch/simsiam.ipynb) |\n| [When Does Contrastive Visual Representation Learning Work?](https://arxiv.org/abs/2105.05837) | [![arXiv](https://img.shields.io/badge/arXiv-2105.05837-b31b1b.svg)](https://arxiv.org/abs/2105.05837) |\n| [Efficient Visual Pretraining with Contrastive Detection](https://arxiv.org/abs/2103.10957) | [![arXiv](https://img.shields.io/badge/arXiv-2103.10957-b31b1b.svg)](https://arxiv.org/abs/2103.10957) |\n\n## 2020\n\n| Title | Relevant Links |\n|:-----|:--------------|\n| [Bootstrap your own latent: A new approach to self-supervised Learning](https://arxiv.org/abs/2006.07733) | [![arXiv](https://img.shields.io/badge/arXiv-2006.07733-b31b1b.svg)](https://arxiv.org/abs/2006.07733) [![Open In Colab](https://img.shields.io/badge/Colab-PyTorch-blue?logo=googlecolab)](https://colab.research.google.com/github/lightly-ai/lightly/blob/master/examples/notebooks/pytorch/byol.ipynb) |\n| [A Simple Framework for Contrastive Learning of Visual Representations](https://arxiv.org/abs/2002.05709) | [![arXiv](https://img.shields.io/badge/arXiv-2002.05709-b31b1b.svg)](https://arxiv.org/abs/2002.05709) [![Open In Colab](https://img.shields.io/badge/Colab-PyTorch-blue?logo=googlecolab)](https://colab.research.google.com/github/lightly-ai/lightly/blob/master/examples/notebooks/pytorch/simclr.ipynb) |\n| [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://arxiv.org/abs/2006.09882) | [![arXiv](https://img.shields.io/badge/arXiv-2006.09882-b31b1b.svg)](https://arxiv.org/abs/2006.09882) [![Open In Colab](https://img.shields.io/badge/Colab-PyTorch-blue?logo=googlecolab)](https://colab.research.google.com/github/lightly-ai/lightly/blob/master/examples/notebooks/pytorch/swav.ipynb) |\n\n## 2019\n\n| Title | Relevant Links |\n|:-----|:--------------|\n| [Momentum Contrast for Unsupervised Visual Representation Learning](https://arxiv.org/abs/1911.05722) | [![arXiv](https://img.shields.io/badge/arXiv-1911.05722-b31b1b.svg)](https://arxiv.org/abs/1911.05722) [![Open In Colab](https://img.shields.io/badge/Colab-PyTorch-blue?logo=googlecolab)](https://colab.research.google.com/github/lightly-ai/lightly/blob/master/examples/notebooks/pytorch/moco.ipynb) |\n\n## 2018\n\n| Title | Relevant Links |\n|:-----|:--------------|\n| [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://arxiv.org/abs/1805.01978) | [![arXiv](https://img.shields.io/badge/arXiv-1805.01978-b31b1b.svg)](https://arxiv.org/abs/1805.01978) |\n\n## 2016\n\n| Title | Relevant Links |\n|:-----|:--------------|\n| [Context Encoders: Feature Learning by Inpainting](https://arxiv.org/abs/1604.07379) | [![arXiv](https://img.shields.io/badge/arXiv-1604.07379-b31b1b.svg)](https://arxiv.org/abs/1604.07379) |\n","projects_url":"https://awesome.ecosyste.ms/api/v1/lists/lightly-ai%2Fawesome-self-supervised-learning/projects"}