{"id":23773953,"url":"https://github.com/zdaxie/pixpro","last_synced_at":"2025-04-09T09:06:21.907Z","repository":{"id":43017425,"uuid":"331370251","full_name":"zdaxie/PixPro","owner":"zdaxie","description":"Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learning, CVPR 2021","archived":false,"fork":false,"pushed_at":"2021-10-24T22:56:47.000Z","size":184,"stargazers_count":346,"open_issues_count":22,"forks_count":42,"subscribers_count":10,"default_branch":"main","last_synced_at":"2025-04-02T02:38:07.494Z","etag":null,"topics":["contrastive-learning","cvpr2021","pixel-level-contrastive-learning","pixpro","unsupervised-learning"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2011.10043","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zdaxie.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-01-20T16:48:40.000Z","updated_at":"2025-03-11T19:20:02.000Z","dependencies_parsed_at":"2022-08-12T10:11:02.629Z","dependency_job_id":null,"html_url":"https://github.com/zdaxie/PixPro","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zdaxie%2FPixPro","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zdaxie%2FPixPro/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zdaxie%2FPixPro/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zdaxie%2FPixPro/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zdaxie","download_url":"https://codeload.github.com/zdaxie/PixPro/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248008629,"owners_count":21032556,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["contrastive-learning","cvpr2021","pixel-level-contrastive-learning","pixpro","unsupervised-learning"],"created_at":"2025-01-01T05:47:17.414Z","updated_at":"2025-04-09T09:06:21.878Z","avatar_url":"https://github.com/zdaxie.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learning\n\nBy [Zhenda Xie](https://scholar.google.com/citations?user=0C4cDloAAAAJ)\\*, [Yutong Lin](https://scholar.google.com/citations?user=mjUgH44AAAAJ)\\*, [Zheng Zhang](https://www.microsoft.com/en-us/research/people/zhez/), [Yue Cao](http://yue-cao.me), [Stephen Lin](https://www.microsoft.com/en-us/research/people/stevelin/) and [Han Hu](https://ancientmooner.github.io/).\n\nThis repo is an official implementation of [\"Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learning\"](https://arxiv.org/abs/2011.10043) on PyTorch.\n\n\n## Introduction\n\n`PixPro` (pixel-to-propagation) is an unsupervised visual feature learning approach by leveraging pixel-level pretext tasks. The learnt feature can be well transferred to downstream dense prediction tasks such as object detection and semantic segmentation. `PixPro` achieves the best transferring performance on Pascal VOC object detection (`60.2 AP` using C4) and COCO object detection (`41.4 / 40.5 mAP` using FPN / C4) with a ResNet-50 backbone.\n\n\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"demo/github_teaser.png\" height=\"300px\" /\u003e\n    \u003cp\u003eAn illustration of the proposed \u003cb\u003e\u003cem\u003ePixPro\u003c/em\u003e\u003c/b\u003e method.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"demo/github_pixpro_pipeline.png\" height=\"160px\" /\u003e\n    \u003cp\u003eArchitecture of the \u003cb\u003e\u003cem\u003ePixContrast\u003c/em\u003e\u003c/b\u003e and \u003cb\u003e\u003cem\u003ePixPro\u003c/em\u003e\u003c/b\u003e methods.\u003c/p\u003e\n\u003c/div\u003e\n\n\n## Citation\n\n```\n@article{xie2020propagate,\n  title={Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learning},\n  author={Xie, Zhenda and Lin, Yutong and Zhang, Zheng and Cao, Yue and Lin, Stephen and Hu, Han},\n  conference={IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},\n  year={2021}\n}\n```\n\n## Main Results\n\n### _PixPro pre-trained models_\n\n|Epochs|Arch|Instance Branch|Download|\n|:---:|:---:|:---:|:---:|\n|100|ResNet-50||[script](tools/pixpro_base_r50_100ep.sh) \\| [model](https://drive.google.com/file/d/1oZwYSLSYSOzTLtOFdi4jXW6T5cGkQTgD/view?usp=sharing) |\n|400|ResNet-50||[script](tools/pixpro_base_r50_400ep.sh) \\| [model](https://drive.google.com/file/d/1Ox2RoFbTrrllbwvITdZvwkNnKUQSUPmV/view?usp=sharing) |\n|100|ResNet-50|:heavy_check_mark:|-|\n|400|ResNet-50|:heavy_check_mark:|-|\n\n### _Pascal VOC object detection_\n\n#### Faster-RCNN with C4\n\n|Method|Epochs|Arch|AP|AP\u003csub\u003e50\u003c/sub\u003e|AP\u003csub\u003e75\u003c/sub\u003e|Download|\n|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n|Scratch|-|ResNet-50|33.8|60.2|33.1|-|\n|Supervised|100|ResNet-50|53.5|81.3|58.8|-|\n|MoCo|200|ResNet-50|55.9|81.5|62.6|-|\n|SimCLR|1000|ResNet-50|56.3|81.9|62.5|-|\n|MoCo v2|800|ResNet-50|57.6|82.7|64.4|-|\n|InfoMin|200|ResNet-50|57.6|82.7|64.6|-|\n|InfoMin|800|ResNet-50|57.5|82.5|64.0|-|\n|[PixPro (ours)](tools/pixpro_base_r50_100ep.sh)|100|ResNet-50|58.8|83.0|66.5|[config](transfer/detection/configs/Pascal_VOC_R_50_C4_24k_PixPro.yaml) \\| [model](https://drive.google.com/file/d/1yk-B5qo_jYqrMC_NcnlY5Z7OqWlJj2Nr/view?usp=sharing)|\n|[PixPro (ours)](tools/pixpro_base_r50_400ep.sh)|400|ResNet-50|60.2|83.8|67.7|[config](transfer/detection/configs/Pascal_VOC_R_50_C4_24k_PixPro.yaml) \\| [model](https://drive.google.com/file/d/1qoiKhAKI-KaWDj1MGHaPrgsQ4dr0RDjh/view?usp=sharing)|\n\n### _COCO object detection_\n\n#### Mask-RCNN with FPN\n\n|Method|Epochs|Arch|Schedule|bbox AP|mask AP|Download|\n|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n|Scratch|-|ResNet-50|1x|32.8|29.9|-|\n|Supervised|100|ResNet-50|1x|39.7|35.9|-|\n|MoCo|200|ResNet-50|1x|39.4|35.6|-|\n|SimCLR|1000|ResNet-50|1x|39.8|35.9|-|\n|MoCo v2|800|ResNet-50|1x|40.4|36.4|-|\n|InfoMin|200|ResNet-50|1x|40.6|36.7|-|\n|InfoMin|800|ResNet-50|1x|40.4|36.6|-|\n|[PixPro (ours)](tools/pixpro_base_r50_100ep.sh)|100|ResNet-50|1x|40.8|36.8|[config](transfer/detection/configs/COCO_R_50_FPN_1x.yaml) \\| [model](https://drive.google.com/file/d/1v5gYT-jjY9n-rkvbocQNDuv0UGxD3c7z/view?usp=sharing)|\n|PixPro (ours)|100*|ResNet-50|1x|41.3|37.1|-|\n|PixPro (ours)|400*|ResNet-50|1x|41.4|37.4|-|\n\n\\* Indicates methods with instance branch.\n\n#### Mask-RCNN with C4\n\n|Method|Epochs|Arch|Schedule|bbox AP|mask AP|Download|\n|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n|Scratch|-|ResNet-50|1x|26.4|29.3|-|\n|Supervised|100|ResNet-50|1x|38.2|33.3|-|\n|MoCo|200|ResNet-50|1x|38.5|33.6|-|\n|SimCLR|1000|ResNet-50|1x|38.4|33.6|-|\n|MoCo v2|800|ResNet-50|1x|39.5|34.5|-|\n|InfoMin|200|ResNet-50|1x|39.0|34.1|-|\n|InfoMin|800|ResNet-50|1x|38.8|33.8|-|\n|[PixPro (ours)](tools/pixpro_base_r50_100ep.sh)|100|ResNet-50|1x|40.0|34.8|[config](transfer/detection/configs/COCO_R_50_C4_1x.yaml) \\| [model](https://drive.google.com/file/d/1V_IUmaoxGYqq6Dty7AadoYHQgSruWixP/view?usp=sharing)|\n|[PixPro (ours)](tools/pixpro_base_r50_400ep.sh)|400|ResNet-50|1x|40.5|35.3|[config](transfer/detection/configs/COCO_R_50_C4_1x.yaml) \\| [model](https://drive.google.com/file/d/18zjhg7e_QZHI2JgNWjhrR90DFPJcpQzi/view?usp=sharing)|\n\n## Getting started\n\n### _Requirements_\n\nAt present, we have not checked the compatibility of the code with other versions of the packages, so we only recommend the following configuration.\n\n* Python 3.7\n* PyTorch == 1.4.0\n* Torchvision == 0.5.0\n* CUDA == 10.1\n* Other dependencies\n\n### _Installation_\n\nWe recommand using conda env to setup the experimental environments.\n```shell script\n# Create environment\nconda create -n PixPro python=3.7 -y\nconda activate PixPro\n\n# Install PyTorch \u0026 Torchvision\nconda install pytorch=1.4.0 cudatoolkit=10.1 torchvision -c pytorch -y\n\n# Install apex\ngit clone https://github.com/NVIDIA/apex\ncd apex\npip install -v --disable-pip-version-check --no-cache-dir --global-option=\"--cpp_ext\" --global-option=\"--cuda_ext\" ./\ncd ..\n\n# Clone repo\ngit clone https://github.com/zdaxie/PixPro ./PixPro\ncd ./PixPro\n\n# Create soft link for data\nmkdir data\nln -s ${ImageNet-Path} ./data/imagenet\n\n# Install other requirements\npip install -r requirements.txt\n```\n\n### _Pretrain with PixPro_\n\n```shell script\n# Train with PixPro base for 100 epochs.\n./tools/pixpro_base_r50_100ep.sh\n```\n\n### _Transfer to Pascal VOC or COCO object detection_\n\n```shell script\n# Convert a pre-trained PixPro model to detectron2's format\ncd transfer/detection\npython convert_pretrain_to_d2.py ${Input-Checkpoint(.pth)} ./output.pkl  \n\n# Install Detectron2\npython -m pip install detectron2==0.2.1 -f \\\n  https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/torch1.4/index.html\n\n# Create soft link for data\nmkdir datasets\nln -s ${Pascal-VOC-Path}/VOC2007 ./datasets/VOC2007\nln -s ${Pascal-VOC-Path}/VOC2012 ./datasets/VOC2012\nln -s ${COCO-Path} ./datasets/coco\n\n# Train detector with pre-trained PixPro model\n# 1. Train Faster-RCNN with Pascal-VOC\npython train_net.py --config-file configs/Pascal_VOC_R_50_C4_24k_PixPro.yaml --num-gpus 8 MODEL.WEIGHTS ./output.pkl\n# 2. Train Mask-RCNN-FPN with COCO\npython train_net.py --config-file configs/COCO_R_50_FPN_1x_PixPro.yaml --num-gpus 8 MODEL.WEIGHTS ./output.pkl\n# 3. Train Mask-RCNN-C4 with COCO\npython train_net.py --config-file configs/COCO_R_50_C4_1x_PixPro.yaml --num-gpus 8 MODEL.WEIGHTS ./output.pkl\n\n# Test detector with provided fine-tuned model\npython train_net.py --config-file configs/Pascal_VOC_R_50_C4_24k_PixPro.yaml --num-gpus 8 --eval-only \\\n  MODEL.WEIGHTS ./pixpro_base_r50_100ep_voc_md5_ec2dfa63.pth\n```\n\nMore models and logs will be released!\n\n## Acknowledgement\n\nOur testbed builds upon several existing publicly available codes. Specifically, we have modified and integrated the following code into this project:\n\n* https://github.com/facebookresearch/moco\n* https://github.com/HobbitLong/PyContrast\n\n## Contributing to the project\n\nAny pull requests or issues are welcomed.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzdaxie%2Fpixpro","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzdaxie%2Fpixpro","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzdaxie%2Fpixpro/lists"}