https://github.com/chengtan9907/hyperspherical-consistency-regularization
The official implementation of the CVPR'2022 paper Hyperspherical Consistency Regularization.
https://github.com/chengtan9907/hyperspherical-consistency-regularization
deep-learning representation-learning
Last synced: about 1 year ago
JSON representation
The official implementation of the CVPR'2022 paper Hyperspherical Consistency Regularization.
- Host: GitHub
- URL: https://github.com/chengtan9907/hyperspherical-consistency-regularization
- Owner: chengtan9907
- Created: 2022-06-02T03:15:14.000Z (about 4 years ago)
- Default Branch: master
- Last Pushed: 2022-06-22T04:19:53.000Z (about 4 years ago)
- Last Synced: 2025-03-21T11:50:31.095Z (over 1 year ago)
- Topics: deep-learning, representation-learning
- Language: Python
- Homepage:
- Size: 366 KB
- Stars: 27
- Watchers: 2
- Forks: 1
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Hyperspherical Consistency Regularization
This repository contains the implementation code for paper:
**Hyperspherical Consistency Regularization**
[Cheng Tan](https://westlake-drug-discovery.github.io/cheng_tan.html), [Zhangyang Gao](https://westlake-drug-discovery.github.io/zhangyang_gao.html), [Lirong Wu](https://lirongwu.github.io/), [Siyuan Li](https://scholar.google.com/citations?user=SKTQTXwAAAAJ&hl), [Stan Z. Li](https://scholar.google.com/citations?user=Y-nyLGIAAAAJ&hl). In [CVPR](https://arxiv.org/abs/2206.00845), 2022.
## Brief introduction
In this work, we explore the relationship between self-supervised
learning and supervised learning, and study how self-supervised learning helps
robust data-efficient deep learning. We propose hyperspherical consistency
regularization (HCR), a simple yet effective plug-and-play method, to
regularize the classifier using feature-dependent information and thus avoid
bias from labels. Specifically, HCR first projects logits from the classifier
and feature projections from the projection head on the respective hypersphere,
then it enforces data points on hyperspheres to have similar structures by
minimizing binary cross entropy of pairwise distances' similarity metrics.
We consider the pairwise distance as the key geometry property, and force points on the classifier's hypersphere to have a similar structure as the projection head's, as follows:
## Dependencies
* torch
## Usage
The following Python code is all you need.
```python
from hcr import HCR
hcr_reg = HCR(classifier_network, lr)
for epoch in epochs:
for batch in dataloader:
# get logits and projections as the input of HCR
hcr_reg.update(logits, projections)
```
## Citation
If you are interested in our repository and our paper, please cite the following paper:
```
@InProceedings{Tan_2022_CVPR,
author = {Tan, Cheng and Gao, Zhangyang and Wu, Lirong and Li, Siyuan and Li, Stan Z.},
title = {Hyperspherical Consistency Regularization},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2022},
pages = {7244-7255}
}
```
Or,
```
@article{tan2022hyperspherical,
title={Hyperspherical Consistency Regularization},
author={Tan, Cheng and Gao, Zhangyang and Wu, Lirong and Li, Siyuan and Li, Stan Z},
journal={arXiv preprint arXiv:2206.00845},
year={2022}
}
```
## Contact
If you have any questions, feel free to contact me through email (tancheng@westlake.edu.cn). Enjoy!