Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.
Awesome Lists | Featured Topics | Projects
https://github.com/ContrastiveSR/Contrastive_Learning_Papers

A list of contrastive Learning papers
https://github.com/ContrastiveSR/Contrastive_Learning_Papers
computer-vision contrastive-learning deep-learning graph natural-language-processing natural-language-understanding research-paper self-supervised-learning
Last synced: 12 days ago
JSON representation
A list of contrastive Learning papers
Host: GitHub
URL: https://github.com/ContrastiveSR/Contrastive_Learning_Papers
Owner: ContrastiveSR
Created: 2021-04-22T02:10:50.000Z (over 3 years ago)
Default Branch: main
Last Pushed: 2022-04-13T20:39:26.000Z (over 2 years ago)
Last Synced: 2024-07-05T15:11:42.986Z (4 months ago)
Topics: computer-vision, contrastive-learning, deep-learning, graph, natural-language-processing, natural-language-understanding, research-paper, self-supervised-learning
Homepage:
Size: 299 KB
Stars: 299
Watchers: 6
Forks: 39
Open Issues: 0
Metadata Files:
- Readme: README.md
Awesome Lists containing this project

awesome-interesting-topics-in-nlp - ML
README

        # Contrastive_Learning_Papers

A list of papers in contrastive learning. 

## Computer Vision

| Year | Title                                                       | Venue | Code |

| -----|-------------------------------------------------------------| ----- | ---- |

| 2021 | [Detco: Unsupervised contrastive learning for object detection](https://arxiv.org/pdf/2102.04803) | arxiv | [Code](https://github.com/xieenze/DetCo) |

| 2021 | [SEED: Self-supervised Distillation For Visual Representation](https://openreview.net/forum?id=AHm3dbp7D1D) | ICLR | Code |

| 2021 | [PROTOTYPICAL CONTRASTIVE LEARNING OF UNSUPERVISED REPRESENTATIONS](https://arxiv.org/abs/2005.04966) | ICLR | [Code](https://github.com/salesforce/PCL) |

| 2021 | [Training GANs with Stronger Augmentations via Contrastive Discriminator](https://openreview.net/forum?id=eo6U4CAwVmg) | ICLR | [Code](https://github.com/jh-jeong/ContraD) |

| 2021 | [VIEWMAKER NETWORKS: LEARNING VIEWS FOR UNSUPERVISED REPRESENTATION LEARNING](https://arxiv.org/pdf/2010.07432.pdf) | ICLR | [Code](https://github.com/alextamkin/viewmaker) |

| 2021 | [Universal Weakly Supervised Segmentation by Pixel-to-Segment Contrastive Learning](https://arxiv.org/abs/2105.00957) | ICLR | [Code](https://github.com/twke18/SPML) |

| 2021 | [Active Contrastive Learning of Audio-Visual Video Representations](https://arxiv.org/abs/2009.09805) | ICLR | Code|

| 2021 | [Conditional Negative Sampling for Contrastive Learning of Visual Representations](https://openreview.net/forum?id=v8b3e5jN66j) | ICLR | Code |

| 2021 | [Learning a Few-shot Embedding Model with Contrastive Learning](https://static.aminer.cn/upload/pdf/956/389/173/6020e0109e795e62379b0e0d_0.pdf) | AAAI | [Code](https://github.com/corwinliu9669/Learning-a-Few-shot-Embedding-Model-with-Contrastive-Learning) |

| 2021 | [Contrastive Learning with Stronger Augmentations](https://arxiv.org/pdf/2104.07713v1.pdf) | IEEE | [Code](https://github.com/maple-research-lab/CLSA) |

| 2021 | [Dual Contrastive Learning for Unsupervised Image-to-Image Translation](https://arxiv.org/pdf/2104.07689v1.pdf) | NTIRE  | [Code](https://github.com/JunlinHan/DCLGAN) |

| 2021 | [With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representations](https://arxiv.org/pdf/2104.14548.pdf) | DeepMind | [code](https://github.com/lightly-ai/lightly) |

| 2021 | [How Well Do Self-Supervised Models Transfer?](https://arxiv.org/pdf/2011.13377.pdf) | CVPR | [Code](https://github.com/linusericsson/ssl-transfer) |

| 2021 | [Self-supervised Pretraining of Visual Features in the Wild](https://arxiv.org/pdf/2103.01988.pdf) | arxiv | [Code](https://github.com/facebookresearch/vissl) |

| 2021 | [VideoMoCo: Contrastive Video Representation Learning with Temporally Adversarial Examples](https://arxiv.org/pdf/2103.05905v2.pdf) | CVPR | [Code](https://github.com/tinapan-pt/VideoMoCo) |

| 2021 | [Temporal Contrastive Graph Learning for Video Action Recognition and Retrieval](https://arxiv.org/pdf/2101.00820.pdf) | arxiv | code |

| 2021 | [Social NCE: Contrastive Learning of Socially-aware Motion Representations](https://arxiv.org/pdf/2012.11717.pdf) | arxiv | [Code](https://github.com/vita-epfl/social-nce) |

| 2021 | [Spatiotemporal Contrastive Video Representation Learning](https://arxiv.org/pdf/2008.03800.pdf) | CVPR | [Code](https://paperswithcode.com/paper/spatiotemporal-contrastive-video#code) |

| 2021 | [What Should Not Be Contrastive in Contrastive Learning](https://arxiv.org/pdf/2008.05659.pdf) | ICLR | code |

| 2020 | [A Simple Framework for Contrastive Learning of Visual Representations](https://arxiv.org/pdf/2002.05709.pdf) | ICML | [Code](https://www.github.com/google-research/simclr) |

| 2020 | [Online Bag-of-Visual-Words Generation for Unsupervised Representation Learning](https://arxiv.org/pdf/2012.11552.pdf) | arxiv | [Code](https://github.com/valeoai/obow) |

| 2020 | [CASTing Your Model:Learning to Localize Improves Self-Supervised Representations](https://arxiv.org/pdf/2012.04630.pdf) | arxiv | code |

| 2020 | [Exploring Simple Siamese Representation Learning](https://arxiv.org/pdf/2011.10566.pdf) | arxiv | code |

| 2020 | [Hard Negative Mixing for Contrastive Learning](https://arxiv.org/pdf/2010.01028.pdf) | NeurIPS | code |

| 2020 | [Representation Learning via Invariant Causal Mechanisms](https://arxiv.org/pdf/2010.07922.pdf) | arxiv | code |

| 2020 | [Are all negatives created equal in contrastive instance discrimination?](https://arxiv.org/pdf/2010.06682.pdf) | arxiv | code |

| 2020 | [Bootstrap your own latent: A new approach to self-supervised Learning](https://arxiv.org/pdf/2006.07733.pdf) | arxiv | [Code](https://github.com/deepmind/deepmind-research/tree/master/byol) |

| 2020 | [Augmented Skeleton Based Contrastive Action Learning with Momentum LSTM for Unsupervised Action Recognition](https://arxiv.org/pdf/2008.00188.pdf) | Information Sciences | [Code](https://github.com/Mikexu007/AS_CAL) |

| 2020 | [Deep Robust Clustering by Contrastive Learning](https://arxiv.org/pdf/2008.03030.pdf) | arxiv | code |

| 2020 | [Contrastive Learning for Unpaired Image-to-Image Translation](https://arxiv.org/pdf/2007.15651.pdf) | ECCV | [Code](https://github.com/taesungp/contrastive-unpaired-translation) |

| 2020 | [Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biases](https://arxiv.org/pdf/2007.13916.pdf) | arxiv | code |

| 2020 | [Self-supervised Video Representation Learning Using Inter-intra Contrastive Framework](https://arxiv.org/pdf/2008.02531.pdf) | ACMMM | [Code](https://github.com/BestJuly/Inter-intra-video-contrastive-learning) |

| 2020 | [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://arxiv.org/pdf/2006.09882.pdf) | NeurIPS | [Code](https://github.com/facebookresearch/swav) |

| 2020 | [Contrastive Learning with Adversarial Examples](https://arxiv.org/pdf/2010.12050.pdf) | NeurIPS | code |

| 2020 | [ContraGAN: Contrastive Learning for Conditional Image Generation](https://arxiv.org/abs/2006.12681) | NeurIPS | [Code](https://github.com/POSTECH-CVLab/PyTorch-StudioGAN)|

| 2020 | [Prototypical Contrastive Learning of Unsupervised Representations](https://arxiv.org/pdf/2005.04966.pdf) | arxiv | [Code](https://github.com/salesforce/PCL) |

| 2020 | [CLOCS: Contrastive Learning of Cardiac Signals](https://arxiv.org/pdf/2005.13249v1.pdf) | arxiv | code |

| 2020 | [On Mutual Information in Contrastive Learning for Visual Representations](https://arxiv.org/pdf/2005.13149v2.pdf) | arxiv | code |

| 2020 | [What makes for good views for contrastive learning](https://arxiv.org/pdf/2005.10243v1.pdf) | ECCV | [Code](https://github.com/HobbitLong/PyContrast) |

| 2020 | [CURL: Contrastive Unsupervised Representations for Reinforcement Learning](https://arxiv.org/pdf/2004.04136v2.pdf) | arxiv | [Code](https://github.com/MishaLaskin/curl) |

| 2020 | [Supervised Contrastive Learning](https://arxiv.org/pdf/2004.11362v1.pdf) | arxiv | [Code](https://github.com/HobbitLong/SupContrast) |

| 2020 | [Clustering based Contrastive Learning for Improving Face Representations](https://arxiv.org/pdf/2004.02195v1.pdf) | IEEE | code |

| 2020 | [Improved Baselines with Momentum Contrastive Learning](https://arxiv.org/pdf/2003.04297v1.pdf) | arxiv | [Code](https://github.com/facebookresearch/moco) |

| 2020 | [Self-labelling via simultaneous clustering and representation learning](https://arxiv.org/pdf/1911.05371.pdf) | ICLR | [Code](https://github.com/yukimasano/self-label) |

| 2020 | [Momentum Contrast for Unsupervised Visual Representation Learning](https://arxiv.org/pdf/1911.05722.pdf) | CVPR | [Code](https://github.com/facebookresearch/moco) |

| 2020 | [Self-Supervised Learning of Pretext-Invariant Representations](https://arxiv.org/pdf/1912.01991.pdf) | CVPR/IEEE | code |

| 2020 | [Data-Efficient Image Recognition with Contrastive Predictive Coding](https://arxiv.org/pdf/1905.09272.pdf) | arxiv | [Code](https://paperswithcode.com/paper/data-efficient-image-recognition-with#code) |

| 2020 | [Contrastive Multiview Coding](https://arxiv.org/pdf/1906.05849.pdf) | arxiv | [Code](https://github.com/HobbitLong/CMC/) |

| 2019 | [Transferable Contrastive Network for Generalized Zero-Shot Learning](https://arxiv.org/pdf/1908.05832v1.pdf) | ICCV | [Code](http://vipl.ict.ac.cn/resources/codes.) |

| 2019 | [Selfie: Self-supervised Pretraining for Image Embedding](https://arxiv.org/pdf/1906.02940.pdf) | arxiv | [Code](https://paperswithcode.com/paper/selfie-self-supervised-pretraining-for-image#code) |

| 2019 | [Local Aggregation for Unsupervised Learning of Visual Embeddings](https://arxiv.org/pdf/1903.12355.pdf) |	arxiv | [Code](https://paperswithcode.com/paper/local-aggregation-for-unsupervised-learning#code) |

| 2019 | [Learning Representations by Maximizing Mutual Information Across Views](https://arxiv.org/pdf/1906.00910.pdf) |	arxiv | [Code](https://paperswithcode.com/paper/190600910#code) |

| 2019 | [Unsupervised Embedding Learning via Invariant and Spreading Instance Feature](https://arxiv.org/pdf/1904.03436.pdf) |	CVPR | [Code](https://github.com/mangye16/Unsupervised_Embedding_Learning) |

| 2019 | [Invariant Information Clustering for Unsupervised Image Classification and Segmentation](https://arxiv.org/pdf/1807.06653.pdf) | ICCV | [Code](https://github.com/xu-ji/IIC) |

| 2019 | [Learning deep representations by mutual information estimation and maximization](https://arxiv.org/pdf/1808.06670.pdf) | ICLR | [Code](https://github.com/rdevon/DIM) |

| 2019 | [Representation Learning with Contrastive Predictive Coding](https://arxiv.org/pdf/1807.03748.pdf) | arxiv | [Code](https://paperswithcode.com/paper/representation-learning-with-contrastive#code) |

| 2018 | [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://arxiv.org/pdf/1805.01978.pdf) | CVPR | [Code](https://github.com/zhirongw/lemniscate.pytorch) |

| 2018 | [Time-Contrastive Networks: Self-Supervised Learning from Video](https://arxiv.org/pdf/1704.06888.pdf) | arxiv | [Code](https://paperswithcode.com/paper/time-contrastive-networks-self-supervised#code) |

| 2017 | [Multi-task Self-Supervised Visual Learning](https://arxiv.org/pdf/1708.07860.pdf) | ICCV | code |

| 2016 | [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://arxiv.org/pdf/1603.09246.pdf) | ECCV | [Code](https://paperswithcode.com/paper/unsupervised-learning-of-visual-1#code) |

| 2015 | [Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networks](https://arxiv.org/pdf/1406.6909.pdf) | arxiv | [Code](https://paperswithcode.com/paper/discriminative-unsupervised-feature-learning#code) |

| 2010 | [Noise-contrastive estimation: A new estimation principle for unnormalized statistical models](http://proceedings.mlr.press/v9/gutmann10a/gutmann10a.pdf) | AISTATS | code |

## Natural Language Processing

| Year | Title                                                        | Venue | Code |

| -----|------------------------------------------------------------- | ----- | ---- |

|2021 | [Few-Shot Intent Detection via Contrastive Pre-Training and Fine-Tuning](https://arxiv.org/pdf/2109.06349v1.pdf) | EMNLP | Code|

|2021 | [CLINE: Contrastive Learning with Semantic Negative Examples for Natural Language Understanding](https://aclanthology.org/2021.acl-long.181.pdf) | ACL | [Code](https://github.com/kandorm/CLINE)|

|2021 | [Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning](https://openreview.net/forum?id=cu7IUiOhujH) | ICLR | Code| 

|2021 | [Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval](https://arxiv.org/abs/2007.00808) | ICLR | [Code](https://github.com/microsoft/ANCE)|

|2021 | [Contrastive Learning with Adversarial Perturbations for Conditional Text Generation](https://arxiv.org/abs/2012.07280) | ICLR | [Code](https://github.com/seanie12/CLAPS)|

|2021 | [Prototypical Contrastive Learning of Unsupervised Representations](https://arxiv.org/abs/2005.04966) | ICLR | [Code](https://github.com/salesforce/PCL)|

|2021 | [Contrastive Learning with Hard Negative Samples](https://openreview.net/forum?id=CR1XOQ0UTh-) | ICLR | Code|

|2021 | [FairFil: Contrastive Neural Debiasing Method for Pretrained Text Encoders](https://openreview.net/forum?id=N6JECD-PI5w) | ICLR | Code| 

|2021 | [Self-Supervised Contrastive Learning for Efficient User Satisfaction Prediction in Conversational Agents](https://arxiv.org/abs/2010.11230) | NAACL | Code| 

|2021 | [SOrT-ing VQA Models : Contrastive Gradient Learning for Improved Consistency](https://www.aclweb.org/anthology/2021.naacl-main.248/) | NAACL | Code| 

|2021 | [Supporting Clustering with Contrastive Learning](https://arxiv.org/abs/2103.12953) | NAACL | Code| 

|2021 | [Understanding Hard Negatives in Noise Contrastive Estimation](https://arxiv.org/abs/2104.06245) | NAACL | Code| 

|2021 | [Contextualized and Generalized Sentence Representations by Contrastive Self-Supervised Learning: A Case Study on Discourse Relation Analysis](https://www.aclweb.org/anthology/2021.naacl-main.442.pdf) | NAACL | Code| 

|2021 | [Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approach](https://arxiv.org/abs/2010.07835) | NAACL | [Code](https://github.com/yueyu1030/COSINE)| 

| 2021 | [COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining](https://arxiv.org/pdf/2102.08473.pdf) | arxiv | Code |

| 2021 | [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://arxiv.org/pdf/2104.08821.pdf) | EMNLP | [Code](https://github.com/princeton-nlp/SimCSE) |

| 2021 | [A Simple and Effective Self-Supervised Contrastive Learning Framework for Aspect Detection](https://arxiv.org/pdf/2009.09107.pdf) | AAAI | Code |

| 2020 | [ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators](https://arxiv.org/abs/2003.10555) | ICLR | [Code](https://github.com/google-research/electra) |

| 2020 | [DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations](https://arxiv.org/abs/2006.03659) | arxiv | [Code](https://github.com/JohnGiorgi/DeCLUTR) |

| 2020 | [Pretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language Models](https://arxiv.org/pdf/2005.10389.pdf) | ACL | [Code](https://github.com/google-research/language) |

| 2020 | [CLEAR: Contrastive Learning for Sentence Representation](https://arxiv.org/pdf/2012.15466.pdf) | arxiv | code |

| 2020 | [CERT: Contrastive Self-supervised Learning for Language Understanding](https://arxiv.org/pdf/2005.12766.pdf) | arxiv | code |

| 2019 | [A Theoretical Analysis of Contrastive Unsupervised Representation Learning](https://arxiv.org/pdf/1902.09229.pdf) | arxiv | code |

| 2019 | [Representation Learning with Contrastive Predictive Coding](https://arxiv.org/pdf/1807.03748.pdf) | arxiv | [Code](https://paperswithcode.com/paper/representation-learning-with-contrastive#code) |

## Graph

| Year | Title                                                        | Venue | Code |

| -----|------------------------------------------------------------- | ----- | ---- |

| 2021 | [An Empirical Study of Graph Contrastive Learning](https://arxiv.org/pdf/2109.01116.pdf) | arxiv | [code](https://github.com/GraphCL/PyGCL) |

| 2021 | [Self-supervised Heterogeneous Graph Neural Network with Co-contrastive Learning](https://arxiv.org/pdf/2105.09111.pdf) | KDD | [code](https://github.com/liun-online/HeCo) |

| 2021 | [Enhanced Graph Learning for Collaborative Filtering via Mutual Information Maximization](https://dl.acm.org/doi/abs/10.1145/3404835.3462928) | SIGIR | code |

| 2021 | [Self-supervised Graph Learning for Recommendation](https://arxiv.org/pdf/2010.10783.pdf) | SIGIR | [code](https://github.com/wujcan/SGL) |

| 2021 | [Graph Contrastive Learning with Adaptive Augmentation](https://arxiv.org/pdf/2010.14945.pdf) | TheWeb | [code](https://github.com/CRIPAC-DIG/GCA) |

| 2020 | [Contrastive Self-supervised Learning for Graph Classification](https://arxiv.org/pdf/2009.05923.pdf) | arxiv | code |

| 2020 | [Sub-graph Contrast for Scalable Self-Supervised Graph Representation Learning](https://arxiv.org/pdf/2009.10273.pdf) | ICDM | code |

| 2020 | [GraphCL: Contrastive Self-Supervised Learning of Graph Representations](https://arxiv.org/pdf/2007.08025.pdf) | Neurips | code |

| 2020 | [Deep Graph Contrastive Representation Learning](https://arxiv.org/pdf/2006.04131v1.pdf) | arxiv | [Code](https://github.com/CRIPAC-DIG/GRACE) |

| 2020 | [Graph Contrastive Learning with Augmentations](https://arxiv.org/pdf/2010.13902.pdf ) | NeurIPS | [Code](https://github.com/Shen-Lab/GraphCL) |

| 2020 | [Gcc: Graph contrastive coding for graph neural network pre-training](https://dl.acm.org/doi/pdf/10.1145/3394486.3403168) | KDD | [Code](https://github.com/THUDM/GCC) |

| 2019 | [Deep Graph Infomax](https://arxiv.org/pdf/1809.10341.pdf ) | ICLR | [Code](https://github.com/PetarV-/DGI) |

## Recommender System 

| Year | Title                                                        | Venue | Code |

| -----|------------------------------------------------------------- | ----- | ---- |

| 2022 | [Contrastive Learning for Representation Degeneration Problem in Sequential Recommendation](https://arxiv.org/pdf/2110.05730.pdf) | WSDM | code |

| 2021 | [Contrastive Self-supervised Sequential Recommendation with Robust Augmentation](https://arxiv.org/abs/2108.06479) | arxiv | [code](https://github.com/YChen1993/CoSeRec) |

| 2021 | [Contrastive Learning for Debiased Candidate Generation in Large-Scale Recommender Systems](https://arxiv.org/abs/2005.12964)| SIGIR |  [Code](NAN) |

| 2021 | [Self-supervised Graph Learning for Recommendation](https://arxiv.org/pdf/2010.10783.pdf)| SIGIR |  [Code](https://github.com/wujcan/SGL) |

| 2021 | [Contrastive Pre-training for Sequential Recommendation](https://arxiv.org/pdf/2010.14395.pdf)| arxiv |  Code |

| 2021 | [Adversarial and Contrastive Variational Autoencoder for Sequential Recommendation](https://arxiv.org/pdf/2103.10693.pdf) | WWW | [Code](https://github.com/ACVAE/ACVAE-PyTorch) |

| 2021 | [SELFCF: A Simple Framework for Self-supervised Collaborative Filtering](https://arxiv.org/pdf/2103.10693.pdf) | arxiv | Code | 

| 2020 | [Self-supervised Learning for Large-scale Item Recommendations](https://arxiv.org/pdf/2007.12865.pdf) | arxiv | code |

| 2020 | [S3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization](https://dl.acm.org/doi/pdf/10.1145/3340531.3411954) | CIKM | [code](https://github.com/RUCAIBox/CIKM2020-S3Rec) |

## Survey

| Year | Title                                                        | Venue | Code |

| -----|------------------------------------------------------------- | ----- | ---- |

| 2022 | [Self-Supervised Learning for Recommender Systems: A Survey](https://arxiv.org/abs/2203.15876) | arxiv | code | 

| 2021 | [Graph Self-Supervised Learning: A Survey](https://arxiv.org/abs/2103.00111) | arxiv | code |

| 2021 | [Self-supervised on Graphs: Contrastive, Generative,or Predictive](https://arxiv.org/abs/2105.07342) | arxiv | code |

| 2021 | [Self-supervised Learning: Generative or Contrastive](https://arxiv.org/pdf/2006.08218.pdf)| arxiv |  code |

| 2021 | [Self-Supervised Learning of Graph Neural Networks: A Unified Review](https://arxiv.org/pdf/2102.10757.pdf)| arxiv |  code |

| 2021 | [A survey on contrastive self-supervised learning](https://www.mdpi.com/2227-7080/9/1/2)| MDPI |  code |

## Others

| Year | Title                                                        | Venue | Code |

| -----|------------------------------------------------------------- | ----- | ---- |

| 2021 | [Towards Effective Context for Meta-Reinforcement Learning: an Approach based on Contrastive Learning](https://arxiv.org/pdf/2009.13891.pdf) | AAAI | code |

### Future Plan

Welcome to join us to expand this repo. In the future, we hope to make this list into finer categorizations. We know that in the computer vision and natural language processing area, there are already a lot of sub-areas are researching the contrastive learning. Therefore, it is important to create some sub-category to include those papers. Feel free to contact us if you are interested: [email protected]