https://github.com/VainF/Awesome-Contrastive-Learning

Awesome Contrastive Learning for CV & NLP
https://github.com/VainF/Awesome-Contrastive-Learning

List: Awesome-Contrastive-Learning

contrastive-learning

Last synced: 3 days ago
JSON representation

Awesome Contrastive Learning for CV & NLP

• Host: GitHub
• URL: https://github.com/VainF/Awesome-Contrastive-Learning
• Owner: VainF
• Created: 2020-05-06T13:10:17.000Z (over 4 years ago)
• Default Branch: master
• Last Pushed: 2021-07-31T17:00:53.000Z (over 3 years ago)
• Last Synced: 2024-05-22T06:02:10.611Z (6 months ago)
• Topics: contrastive-learning
• Homepage:
• Size: 41 KB
• Stars: 161
• Watchers: 7
• Forks: 20
• Open Issues: 1
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

• awesome-interesting-topics-in-nlp - CVNLP
• ultimate-awesome - Awesome-Contrastive-Learning - Awesome Contrastive Learning for CV & NLP. (Other Lists / PowerShell Lists)

README

        
# Awesome-Contrastive-Learning

Note: "*" refers to official code.

## Computer Vision

### 2021

* [Investigating the Role of Negatives in

Contrastive Representation Learning](https://arxiv.org/abs/2106.09943), Jordan T. Ash, 2021

* [Contrastive Semi-Supervised Learning for 2D Medical Image Segmentation](https://arxiv.org/abs/2106.06801), Prashant Pandey, MICCAI-2021

* [Contrastive Learning with Hard Negative Samples](https://arxiv.org/abs/2010.04592), Joshua Robinson, ICLR2021, [[pytorch](https://github.com/joshr17/HCL)*]

### 2020

* [Contrastive Representation Learning: A Framework and Review](https://arxiv.org/abs/2010.05113), Phuc H. Le-Khac

* [Supervised Contrastive Learning](https://arxiv.org/abs/2004.11362), Prannay Khosla, 2020, [[pytorch*](https://github.com/HobbitLong/SupContrast)]

* [A Simple Framework for Contrastive Learning of Visual Representations](https://arxiv.org/abs/2002.05709), Ting Chen, 2020, [[pytroch](https://github.com/sthalles/SimCLR), [tensorflow*](https://github.com/google-research/simclr)]

* [Improved Baselines with Momentum Contrastive Learning](https://arxiv.org/abs/2003.04297), Xinlei Chen, 2020, [[tensorflow](https://github.com/ppwwyyxx/moco.tensorflow)]

* [Contrastive Representation Distillation](https://arxiv.org/abs/1910.10699), Yonglong Tian, ICLR-2020 [[pytorch*](https://github.com/HobbitLong/RepDistiller)]

* [COBRA: Contrastive Bi-Modal Representation Algorithm](https://arxiv.org/ftp/arxiv/papers/2005/2005.03687.pdf), Vishaal Udandarao, 2020

* [What makes for good views for contrastive learning](https://arxiv.org/abs/2005.10243), Yonglong Tian, 2020

* [Prototypical Contrastive Learning of Unsupervised Representations](https://arxiv.org/pdf/2005.04966.pdf), Junnan Li, 2020

* [Contrastive Multi-View Representation Learning on Graphs](https://arxiv.org/abs/2006.05582), Kaveh Hassani, 2020

* [On Mutual Information in Contrastive Learning for Visual Representations](https://arxiv.org/abs/2005.13149), Mike Wu, 2020

* [Semi-Supervised Contrastive Learning with Generalized Contrastive Loss and Its Application to Speaker Recognition](https://arxiv.org/abs/2006.04326), Nakamasa Inoue, 2020

* [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://arxiv.org/abs/2005.10242),Tongzhou Wang, ICML2020, [[pytorch](https://github.com/SsnL/align_uniform)*]

### 2019

* [Momentum Contrast for Unsupervised Visual Representation Learning](https://arxiv.org/abs/1911.05722), Kaiming He, 2019, [[pytorch](https://github.com/peisuke/MomentumContrast.pytorch)]

* [Data-Efficient Image Recognition with Contrastive Predictive Coding](https://arxiv.org/abs/1905.09272), Olivier J. Hénaff, 2019

* [Contrastive Multiview Coding](https://arxiv.org/abs/1906.05849), Yonglong Tian, 2019, [[pytorch*](https://github.com/HobbitLong/CMC/)]

* [Learning deep representations by mutual information estimation and maximization](https://arxiv.org/abs/1808.06670), R Devon Hjelm, ICLR-2019, [[pytorch](https://github.com/rdevon/DIM*)]

* [Contrastive Adaptation Network for Unsupervised Domain Adaptation](http://openaccess.thecvf.com/content_CVPR_2019/papers/Kang_Contrastive_Adaptation_Network_for_Unsupervised_Domain_Adaptation_CVPR_2019_paper.pdf), Guoliang Kang, CVPR-2019

### 2018

* [Representation learning with contrastive predictive coding](https://arxiv.org/abs/1807.03748), Aaron van den Oord, 2018, [[pytorch](https://github.com/jefflai108/Contrastive-Predictive-Coding-PyTorch)]

* [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://arxiv.org/abs/1805.01978), Zhirong Wu, CVPR-2018, [[pytorch*](https://github.com/zhirongw/lemniscate.pytorch)]

* [Adversarial Contrastive Estimation](https://arxiv.org/abs/1805.03642), Avishek Joey Bose, ACL-2018,

### 2017

* [Time-Contrastive Networks: Self-Supervised Learning from Video](https://arxiv.org/abs/1704.06888), Pierre Sermanet, CVPR-2017

* [Contrastive Learning for Image Captioning](http://papers.nips.cc/paper/6691-contrastive-learning-for-image-captioning), Bo Dai, NeurIPS-2017, [[lua*](https://github.com/doubledaibo/clcaption_nips2017)]

### Before 2017

* [Noise-contrastive estimation for answer selection with deep neural networks](https://dl.acm.org/doi/abs/10.1145/2983323.2983872), Jinfeng Rao, 2016, [[torch](https://github.com/castorini/NCE-CNN-Torch)]

* [Improved Deep Metric Learning with Multi-class N-pair Loss Objective](https://papers.nips.cc/paper/6200-improved-deep-metric-learning-with-multi-class-n-pair-loss-objective), Kihyuk Sohn, NeurIPS-2016, [[pytorch](https://github.com/ChaofWang/Npair_loss_pytorch)]

* [Learning word embeddings efficiently with noise-contrastive estimation](http://papers.nips.cc/paper/5165-learning-word-embeddings), Andriy Mnih, NeurIPS-2013, 

* [Noise-contrastive estimation: A new estimation principle for unnormalized statistical models](http://proceedings.mlr.press/v9/gutmann10a/gutmann10a.pdf?source=post_page---------------------------), Michael Gutmann, AISTATS 2010, [[pytorch](https://github.com/demelin/Noise-Contrastive-Estimation-NCE-for-pyTorch)]

* [Dimensionality Reduction by Learning an Invariant Mapping](http://yann.lecun.com/exdb/publis/pdf/hadsell-chopra-lecun-06.pdf), Raia Hadsell, 2006

## Natural Language Processing

### 2021  

* [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://arxiv.org/abs/2104.08821), Tianyu Gao, 2021, [[pytorch](https://github.com/princeton-nlp/SimCSE)*]

* [ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer](https://arxiv.org/abs/2105.11741), Yuanmeng Yan, ACL2021, [[pytorch](https://github.com/yym6472/ConSERT)*]

* [DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations](https://arxiv.org/abs/2006.03659), John Giorgi, ACL2021, [[pytorch](https://github.com/JohnGiorgi/DeCLUTR)*]

* [Coco-lm: Correcting and contrasting text sequences

for language model pretraining](https://arxiv.org/abs/2102.08473), Yu Meng, 2021,  [[pytorch](https://github.com/lucidrains/coco-lm-pytorch)*]

* [Semantic Re-tuning with Contrastive Tension](https://openreview.net/forum?id=Ov_sMNau-PF), Fredrik Carlsson, ICLR2021, [[Tensorflow](https://github.com/FreddeFrallan/Contrastive-Tension)*]

### 2020

* [CLEAR: Contrastive Learning for Sentence Representation](https://arxiv.org/abs/2012.15466), Zhuofeng Wu, 2020

* [An Unsupervised Sentence Embedding Method by Mutual Information Maximization](https://www.aclweb.org/anthology/2020.emnlp-main.124.pdf), Yan Zhang, EMNLP2020, [[pytorch](https://github.com/yanzhangnlp/IS-BERT)*]

* [Cert: Contrastive self-supervised learning for language under- standing](), 

### Before 2017  

* [Efficient estimation of word representations in vector space](https://arxiv.org/abs/1301.3781), Tomas Mikolov, ICLR-Workshop 2013