{"id":15601018,"url":"https://github.com/lucidrains/pixel-level-contrastive-learning","last_synced_at":"2025-04-06T10:11:35.374Z","repository":{"id":40407326,"uuid":"314619050","full_name":"lucidrains/pixel-level-contrastive-learning","owner":"lucidrains","description":"Implementation of Pixel-level Contrastive Learning, proposed in the paper \"Propagate Yourself\", in Pytorch","archived":false,"fork":false,"pushed_at":"2021-02-02T16:01:12.000Z","size":108,"stargazers_count":259,"open_issues_count":7,"forks_count":27,"subscribers_count":11,"default_branch":"main","last_synced_at":"2025-03-30T09:06:38.173Z","etag":null,"topics":["artificial-intelligence","contrastive-learning","deep-learning","self-supervised-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lucidrains.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-11-20T17:07:57.000Z","updated_at":"2025-03-20T16:55:51.000Z","dependencies_parsed_at":"2022-08-29T10:41:52.837Z","dependency_job_id":null,"html_url":"https://github.com/lucidrains/pixel-level-contrastive-learning","commit_stats":null,"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Fpixel-level-contrastive-learning","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Fpixel-level-contrastive-learning/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Fpixel-level-contrastive-learning/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Fpixel-level-contrastive-learning/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lucidrains","download_url":"https://codeload.github.com/lucidrains/pixel-level-contrastive-learning/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247464222,"owners_count":20942970,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","contrastive-learning","deep-learning","self-supervised-learning"],"created_at":"2024-10-03T02:11:55.468Z","updated_at":"2025-04-06T10:11:35.352Z","avatar_url":"https://github.com/lucidrains.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg src=\"./propagate.png\"\u003e\u003c/img\u003e\n\n## Pixel-level Contrastive Learning\n\nImplementation of Pixel-level Contrastive Learning, proposed in the paper \u003ca href=\"https://arxiv.org/abs/2011.10043\"\u003e\"Propagate Yourself\"\u003c/a\u003e, in Pytorch. In addition to doing contrastive learning on the pixel level, the online network further passes the pixel level representations to a Pixel Propagation Module and enforces a similarity loss to the target network. They beat all previous unsupervised and supervised methods in segmentation tasks.\n\n## Install\n\n```bash\n$ pip install pixel-level-contrastive-learning\n```\n\n## Usage\n\nBelow is an example of how you would use the framework to self-supervise training of a resnet, taking the output of layer 4 (8 x 8 'pixels').\n\n\n```python\nimport torch\nfrom pixel_level_contrastive_learning import PixelCL\nfrom torchvision import models\nfrom tqdm import tqdm\n\nresnet = models.resnet50(pretrained=True)\n\nlearner = PixelCL(\n    resnet,\n    image_size = 256,\n    hidden_layer_pixel = 'layer4',  # leads to output of 8x8 feature map for pixel-level learning\n    hidden_layer_instance = -2,     # leads to output for instance-level learning\n    projection_size = 256,          # size of projection output, 256 was used in the paper\n    projection_hidden_size = 2048,  # size of projection hidden dimension, paper used 2048\n    moving_average_decay = 0.99,    # exponential moving average decay of target encoder\n    ppm_num_layers = 1,             # number of layers for transform function in the pixel propagation module, 1 was optimal\n    ppm_gamma = 2,                  # sharpness of the similarity in the pixel propagation module, already at optimal value of 2\n    distance_thres = 0.7,           # ideal value is 0.7, as indicated in the paper, which makes the assumption of each feature map's pixel diagonal distance to be 1 (still unclear)\n    similarity_temperature = 0.3,   # temperature for the cosine similarity for the pixel contrastive loss\n    alpha = 1.,                      # weight of the pixel propagation loss (pixpro) vs pixel CL loss\n    use_pixpro = True,               # do pixel pro instead of pixel contrast loss, defaults to pixpro, since it is the best one\n    cutout_ratio_range = (0.6, 0.8)  # a random ratio is selected from this range for the random cutout\n).cuda()\n\nopt = torch.optim.Adam(learner.parameters(), lr=1e-4)\n\ndef sample_batch_images():\n    return torch.randn(10, 3, 256, 256).cuda()\n\nfor _ in tqdm(range(100000)):\n    images = sample_batch_images()\n    loss = learner(images) # if positive pixel pairs is equal to zero, the loss is equal to the instance level loss\n\n    opt.zero_grad()\n    loss.backward()\n    print(loss.item())\n    opt.step()\n    learner.update_moving_average() # update moving average of target encoder\n\n# after much training, save the improved model for testing on downstream task\ntorch.save(resnet, 'improved-resnet.pt')\n```\n\nYou can also return the number of positive pixel pairs on `forward`, for logging or other purposes\n\n```python\nloss, positive_pairs = learner(images, return_positive_pairs = True)\n```\n## Citations\n\n```bibtex\n@misc{xie2020propagate,\n    title={Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learning}, \n    author={Zhenda Xie and Yutong Lin and Zheng Zhang and Yue Cao and Stephen Lin and Han Hu},\n    year={2020},\n    eprint={2011.10043},\n    archivePrefix={arXiv},\n    primaryClass={cs.CV}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flucidrains%2Fpixel-level-contrastive-learning","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flucidrains%2Fpixel-level-contrastive-learning","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flucidrains%2Fpixel-level-contrastive-learning/lists"}