{"id":27942533,"url":"https://github.com/coincheung/densecl","last_synced_at":"2025-10-07T08:24:10.917Z","repository":{"id":39925030,"uuid":"291669944","full_name":"CoinCheung/DenseCL","owner":"CoinCheung","description":"DenseCL + regionCL-D","archived":false,"fork":false,"pushed_at":"2022-12-27T08:47:57.000Z","size":119,"stargazers_count":15,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-05-07T11:57:01.527Z","etag":null,"topics":["deep-learning","metric-learning","pretrained","pytorch","regioncl-d","self-supervised-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CoinCheung.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":".github/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":".github/CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-08-31T09:18:42.000Z","updated_at":"2024-06-30T11:31:44.000Z","dependencies_parsed_at":"2023-01-31T03:00:51.594Z","dependency_job_id":null,"html_url":"https://github.com/CoinCheung/DenseCL","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/CoinCheung/DenseCL","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CoinCheung%2FDenseCL","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CoinCheung%2FDenseCL/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CoinCheung%2FDenseCL/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CoinCheung%2FDenseCL/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CoinCheung","download_url":"https://codeload.github.com/CoinCheung/DenseCL/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CoinCheung%2FDenseCL/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278741998,"owners_count":26037718,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-07T02:00:06.786Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","metric-learning","pretrained","pytorch","regioncl-d","self-supervised-learning"],"created_at":"2025-05-07T11:56:55.255Z","updated_at":"2025-10-07T08:24:10.867Z","avatar_url":"https://github.com/CoinCheung.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## DenseCL: Dense Contrastive Learning for Self-Supervised Visual Pre-Training\n\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./denseCL.png\" width=\"600\"\u003e\n\u003c/p\u003e\n\nThis is an unofficial PyTorch implementation of the [DenseCL paper](https://arxiv.org/abs/2011.09157), with the help and suggestions from @WXinlong and @DerrickWang005.\n\nCurrently, [regionCL-D](https://arxiv.org/abs/2111.12309) is added, and pretrained checkpoints are uploaded.\n\n\n### Preparation\n\nInstall PyTorch and ImageNet dataset following the [official PyTorch ImageNet training code](https://github.com/pytorch/examples/tree/master/imagenet).\n\nThis repo aims to be minimal modifications on that code. Check the modifications by:\n```\ndiff main_densecl.py \u003c(curl https://raw.githubusercontent.com/pytorch/examples/master/imagenet/main.py)\ndiff main_lincls.py \u003c(curl https://raw.githubusercontent.com/pytorch/examples/master/imagenet/main.py)\n```\n\n\n### Unsupervised Training \u0026 Linear Classification\n\nThis implementation only supports **multi-gpu**, **DistributedDataParallel** training, which is faster and simpler; single-gpu or DataParallel training is not supported.\n\nThis implementation only supports **ResNet50/ResNet101**, since we need to modify computing graph architecture and I only modified ResNet50/ResNet101.\n\nTo do unsupervised pre-training and linear-evaluation of a ResNet50/ResNet101 model on ImageNet in an 8-gpu machine, please refer to [dist_train.sh](./dist_train.sh) for relevant starting script.\n\nSince the paper says they use default mocov2 hyper-parameters, the above script uses same hyper-parameters as mocov2.\n\n***Note***: for 4-gpu training, we recommend following the [linear lr scaling recipe](https://arxiv.org/abs/1706.02677): `--lr 0.015 --batch-size 128` with 4 gpus. We got similar results using this setting.\n\n\n### Models\n\nOur pre-trained denseCL/RegionCL-D models can be downloaded as following:\n\u003ctable\u003e\u003ctbody\u003e\n\u003c!-- START TABLE --\u003e\n\u003c!-- TABLE HEADER --\u003e\n\u003cth valign=\"bottom\"\u003e\u003c/th\u003e\n\u003cth valign=\"bottom\"\u003eepochs\u003c/th\u003e\n\u003cth valign=\"bottom\"\u003emlp\u003c/th\u003e\n\u003cth valign=\"bottom\"\u003eaug+\u003c/th\u003e\n\u003cth valign=\"bottom\"\u003ecos\u003c/th\u003e\n\u003cth valign=\"bottom\"\u003eIM\u003cbr/\u003etop1\u003c/th\u003e\n\u003cth valign=\"bottom\"\u003eVOC\u003cbr/\u003eAP50\u003c/th\u003e\n\u003cth valign=\"bottom\"\u003emodel\u003c/th\u003e\n\u003cth valign=\"bottom\"\u003emd5\u003c/th\u003e\n\u003c!-- TABLE BODY --\u003e\n\u003ctr\u003e\u003ctd align=\"left\"\u003e\u003ca href=\"https://arxiv.org/abs/2003.04297\"\u003eMoCov2 R50\u003c/a\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e200\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u0026#x2713\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u0026#x2713\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u0026#x2713\u003c/td\u003e\n\u003ctd align=\"center\"\u003e67.7\u003c/td\u003e\n\u003ctd align=\"center\"\u003e82.4\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003ca href=\"https://dl.fbaipublicfiles.com/moco/moco_checkpoints/moco_v2_200ep/moco_v2_200ep_pretrain.pth.tar\"\u003edownload\u003c/a\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003ctt\u003e59fd9945\u003c/tt\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\u003ctd align=\"left\"\u003e\u003ca href=\"https://arxiv.org/abs/2011.09157\"\u003eDenseCL R50\u003c/a\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e200\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u0026#x2713\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u0026#x2713\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u0026#x2713\u003c/td\u003e\n\u003ctd align=\"center\"\u003e63.8\u003c/td\u003e\n\u003ctd align=\"center\"\u003e82.7\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003ca href=\"https://github.com/CoinCheung/DenseCL/releases/download/v0.0.1/densecl_r50_checkpoint_0199.pth.tar\"\u003edownload\u003c/a\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003ctt\u003e7cfc894c\u003c/tt\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\u003ctd align=\"left\"\u003e\u003ca href=\"https://arxiv.org/abs/2011.09157\"\u003eDenseCL R101\u003c/a\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e200\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u0026#x2713\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u0026#x2713\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u0026#x2713\u003c/td\u003e\n\u003ctd align=\"center\"\u003e65.4\u003c/td\u003e\n\u003ctd align=\"center\"\u003e83.5\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003ca href=\"https://github.com/CoinCheung/DenseCL/releases/download/v0.0.1/densecl_r101_checkpoint_0199.pth.tar\"\u003edownload\u003c/a\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003ctt\u003e006675e5\u003c/tt\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\u003ctd align=\"left\"\u003e\u003ca href=\"https://arxiv.org/abs/2111.12309\"\u003eRegionCL-D R50\u003c/a\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e200\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u0026#x2713\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u0026#x2713\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u0026#x2713\u003c/td\u003e\n\u003ctd align=\"center\"\u003e67.5\u003c/td\u003e\n\u003ctd align=\"center\"\u003e83.3\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003ca href=\"https://github.com/CoinCheung/DenseCL/releases/download/v0.0.1/regioncl_r50_checkpoint_0199.pth.tar\"\u003edownload\u003c/a\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003ctt\u003e8afad30e\u003c/tt\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\u003ctd align=\"left\"\u003e\u003ca href=\"https://arxiv.org/abs/2111.12309\"\u003eRegionCL-D R101\u003c/a\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e200\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u0026#x2713\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u0026#x2713\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u0026#x2713\u003c/td\u003e\n\u003ctd align=\"center\"\u003e67.5\u003c/td\u003e\n\u003ctd align=\"center\"\u003e84.3\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003ca href=\"https://github.com/CoinCheung/DenseCL/releases/download/v0.0.1/regioncl_r101_checkpoint_0199.pth.tar\"\u003edownload\u003c/a\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003ctt\u003ea1489ad4\u003c/tt\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\u003c/table\u003e\n\nHere **IM** is imagenet-1k dataset. We freeze pretrained weights and only fine tune the last classifier layer.\n\nPlease be aware that though DenseCL cannot match mocov2 in the filed of classification, it is superior to mocov2 in terms of object detection. More results of detection can be found [here](./detection).\n\n\n### Transferring to Object Detection\n\nFor details, see [./detection](./detection).\n\n\n### License\n\nThis project is under the CC-BY-NC 4.0 license. See [LICENSE](LICENSE) for details.\n\n\n\nregioncl, r50:  \n    Acc@1 67.518 Acc@5 88.256  \n    Acc@1 67.534 Acc@5 88.212  \nregioncl, r101:  \n    Acc@1 67.504 Acc@5 88.212  \n    Acc@1 67.470 Acc@5 88.104\n    \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcoincheung%2Fdensecl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcoincheung%2Fdensecl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcoincheung%2Fdensecl/lists"}