Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/hiyouga/dual-contrastive-learning
Code for our paper "Dual Contrastive Learning: Text Classification via Label-Aware Data Augmentation"
https://github.com/hiyouga/dual-contrastive-learning
bert contrastive-learning deep-learning natural-language-processing neural-networks text-classification transformers
Last synced: about 1 month ago
JSON representation
Code for our paper "Dual Contrastive Learning: Text Classification via Label-Aware Data Augmentation"
- Host: GitHub
- URL: https://github.com/hiyouga/dual-contrastive-learning
- Owner: hiyouga
- License: mit
- Created: 2022-01-17T08:03:38.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2022-09-16T11:35:19.000Z (about 2 years ago)
- Last Synced: 2024-06-14T01:46:07.949Z (3 months ago)
- Topics: bert, contrastive-learning, deep-learning, natural-language-processing, neural-networks, text-classification, transformers
- Language: Python
- Homepage: https://arxiv.org/abs/2201.08702
- Size: 5.81 MB
- Stars: 146
- Watchers: 6
- Forks: 25
- Open Issues: 10
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- StarryDivineSky - hiyouga/Dual-Contrastive-Learning
README
# Dual-Contrastive-Learning
A PyTorch implementation for our paper "Dual Contrastive Learning: Text Classification via Label-Aware Data Augmentation".
You can download the paper via: [[ArXiv]](https://arxiv.org/abs/2201.08702) [[PapersWithCode]](https://paperswithcode.com/paper/dual-contrastive-learning-text-classification).
## One-Sentence Summary
This paper proposes a novel contrastive learning framework for supervised classification tasks by simultaneously learning the features of input samples and the parameters of classifiers in the same space.
## Abstract
Contrastive learning has achieved remarkable success in representation learning via self-supervision in unsupervised settings. However, effectively adapting contrastive learning to supervised learning tasks remains as a challenge in practice. In this work, we introduce a dual contrastive learning (DualCL) framework that simultaneously learns the features of input samples and the parameters of classifiers in the same space. Specifically, DualCL regards the parameters of the classifiers as augmented samples associating to different labels and then exploits the contrastive learning between the input samples and the augmented samples. Empirical studies on five benchmark text classification datasets and their low-resource version demonstrate the improvement in classification accuracy and confirm the capability of learning discriminative representations of DualCL.
## Requirement
- Python = 3.7
- torch = 1.11.0
- numpy = 1.17.2
- transformers = 4.19.2
## Preparation
### Clone
```bash
git clone https://github.com/hiyouga/Dual-Contrastive-Learning.git
```
### Create an anaconda environment:
```bash
conda create -n dualcl python=3.7
conda activate dualcl
pip install -r requirements.txt
```
## Usage
```sh
python main.py --method dualcl
```
## Citation
If this work is helpful, please cite as:
```bibtex
@article{chen2022dual,
title={Dual Contrastive Learning: Text Classification via Label-Aware Data Augmentation},
author={Qianben Chen and Richong Zhang and Yaowei Zheng and Yongyi Mao},
journal={arXiv preprint},
year={2022}
}
```
## Contact
hiyouga [AT] buaa [DOT] edu [DOT] cn
## License
MIT