{"id":32976572,"url":"https://github.com/zhirongw/lemniscate.pytorch","last_synced_at":"2025-11-16T08:01:50.318Z","repository":{"id":38616143,"uuid":"132115652","full_name":"zhirongw/lemniscate.pytorch","owner":"zhirongw","description":"Unsupervised Feature Learning via Non-parametric Instance Discrimination","archived":false,"fork":false,"pushed_at":"2021-03-25T17:34:06.000Z","size":30,"stargazers_count":756,"open_issues_count":20,"forks_count":134,"subscribers_count":12,"default_branch":"master","last_synced_at":"2025-10-11T23:49:17.620Z","etag":null,"topics":["computer-vision","cvpr2018","deep-learning","imagenet","nce","pytorch","representation-learning","self-supervised-learning","unsupervised-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zhirongw.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-05-04T08:59:24.000Z","updated_at":"2025-09-16T02:13:49.000Z","dependencies_parsed_at":"2022-09-12T16:10:40.284Z","dependency_job_id":null,"html_url":"https://github.com/zhirongw/lemniscate.pytorch","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/zhirongw/lemniscate.pytorch","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhirongw%2Flemniscate.pytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhirongw%2Flemniscate.pytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhirongw%2Flemniscate.pytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhirongw%2Flemniscate.pytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zhirongw","download_url":"https://codeload.github.com/zhirongw/lemniscate.pytorch/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhirongw%2Flemniscate.pytorch/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":284678558,"owners_count":27045646,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-11-16T02:00:05.974Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","cvpr2018","deep-learning","imagenet","nce","pytorch","representation-learning","self-supervised-learning","unsupervised-learning"],"created_at":"2025-11-13T06:00:31.219Z","updated_at":"2025-11-16T08:01:50.313Z","avatar_url":"https://github.com/zhirongw.png","language":"Python","funding_links":[],"categories":["Computer Vision"],"sub_categories":["Image Representation Learning"],"readme":"## Unsupervised Feature Learning via Non-parameteric Instance Discrimination\n\nThis repo constains the pytorch implementation for the CVPR2018 unsupervised learning paper [(arxiv)](https://arxiv.org/pdf/1805.01978.pdf).\n\n## Updated Pretrained Model\n\nAn updated instance discrimination model with memory bank implementation and with nce-k=65536 negatives is provided.\nThe updated model is trained with Softmax-CE loss as in CPC/MoCo instead of the original NCE loss.\n\n- [ResNet 50](https://frontiers.blob.core.windows.net/pretraining/checkpoints/pil_pretrained_models/lemniscate/lemniscate_resnet50_update.pth) (Linear ImageNet Acc 58.5%)\n\n\n**Oldies**: original releases of ResNet18 and ResNet50 trained with 4096 negatives and the NCE loss.\nEach tar ball contains the feature representation of all ImageNet training images (600 mb) and model weights (100-200mb).\nYou can also get these representations by forwarding the network for the entire ImageNet images.\n\n- [ResNet 18](https://frontiers.blob.core.windows.net/pretraining/checkpoints/pil_pretrained_models/lemniscate/lemniscate_resnet18.pth) (top 1 nearest neighbor accuracy 41.0%)\n- [ResNet 50](https://frontiers.blob.core.windows.net/pretraining/checkpoints/pil_pretrained_models/lemniscate/lemniscate_resnet50.pth) (top 1 nearest neighbor accuracy 46.8%)\n\n\n## Highlight\n\n- We formulate unsupervised learning from a completely different non-parametric perspective.\n- Feature encodings can be as compact as 128 dimension for each image.\n- Enjoys the benefit of advanced architectures and techniques from supervised learning.\n- Runs seamlessly with nearest neighbor classifiers.\n\n## Nearest Neighbor\n\nPlease follow [this link](http://zhirongw.westus2.cloudapp.azure.com/nn.html) for a list of nearest neighbors on ImageNet.\nResults are visualized from our ResNet50 model, compared with raw image features and supervised features.\nFirst column is the query image, followed by 20 retrievals ranked by the similarity.\n\n## Usage\n\nOur code extends the pytorch implementation of imagenet classification in [official pytorch release](https://github.com/pytorch/examples/tree/master/imagenet). \nPlease refer to the official repo for details of data preparation and hardware configurations.\n\n- supports python27 and [pytorch=0.4](http://pytorch.org)\n\n- if you are looking for pytorch 0.3, please switch to tag v0.3\n\n- clone this repo: `git clone https://github.com/zhirongw/lemniscate.pytorch`\n\n- Training on ImageNet:\n\n  `python main.py DATAPATH --arch resnet18 -j 32 --nce-k 4096 --nce-t 0.07  --lr 0.03 --nce-m 0.5 --low-dim 128 -b 256 `\n\n  - parameter nce-k controls the number of negative samples. If nce-k sets to 0, the code also supports full softmax learning.\n  - nce-t controls temperature of the distribution. 0.07-0.1 works well in practice.\n  - nce-m stabilizes the learning process. A value of 0.5 works well in practice.\n  - learning rate is initialized to 0.03, a bit smaller than standard supervised learning.\n  - the embedding size is controlled by the parameter low-dim.\n\n- During training, we monitor the supervised validation accuracy by K nearest neighbor with K=1, as it's faster, and gives a good estimation of the feature quality.\n\n- Testing on ImageNet:\n\n  `python main.py DATAPATH --arch resnet18 --resume input_model.pth.tar -e` runs testing with default K=200 neighbors.\n\n- Training on CIFAR10:\n\n  `python cifar.py --nce-k 0 --nce-t 0.1 --lr 0.03`\n\n\n## Citation\n\n```\n@inproceedings{wu2018unsupervised,\n  title={Unsupervised Feature Learning via Non-Parametric Instance Discrimination},\n  author={Wu, Zhirong and Xiong, Yuanjun and Stella, X Yu and Lin, Dahua},\n  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},\n  year={2018}\n}\n```\n\n## Contact\n\nFor any questions, please feel free to reach \n```\nZhirong Wu: xavibrowu@gmail.com\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzhirongw%2Flemniscate.pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzhirongw%2Flemniscate.pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzhirongw%2Flemniscate.pytorch/lists"}