{"id":21630978,"url":"https://github.com/uber-research/permute-quantize-finetune","last_synced_at":"2025-04-11T14:08:39.663Z","repository":{"id":47760958,"uuid":"306659116","full_name":"uber-research/permute-quantize-finetune","owner":"uber-research","description":"Using ideas from product quantization for state-of-the-art neural network compression.","archived":false,"fork":false,"pushed_at":"2021-08-14T02:03:31.000Z","size":29633,"stargazers_count":141,"open_issues_count":2,"forks_count":15,"subscribers_count":10,"default_branch":"main","last_synced_at":"2023-11-07T19:00:23.493Z","etag":null,"topics":["deep-learning","network-compression","vector-quantization"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/uber-research.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-10-23T14:21:26.000Z","updated_at":"2023-08-16T02:34:37.000Z","dependencies_parsed_at":"2022-09-16T20:20:41.812Z","dependency_job_id":null,"html_url":"https://github.com/uber-research/permute-quantize-finetune","commit_stats":null,"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uber-research%2Fpermute-quantize-finetune","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uber-research%2Fpermute-quantize-finetune/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uber-research%2Fpermute-quantize-finetune/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uber-research%2Fpermute-quantize-finetune/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/uber-research","download_url":"https://codeload.github.com/uber-research/permute-quantize-finetune/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":226260483,"owners_count":17596495,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","network-compression","vector-quantization"],"created_at":"2024-11-25T02:12:52.895Z","updated_at":"2024-11-25T02:12:53.598Z","avatar_url":"https://github.com/uber-research.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Permute, Quantize, and Fine-tune\n\nThis repository contains the source code and compressed models for the paper\n_Permute, Quantize, and Fine-tune: Efficient Compression of Neural Networks_: https://arxiv.org/abs/2010.15703\n\n![Permutation optimization](./imgs/permutation.png?raw=true)\n\nOur method compresses the weight matrices of the network layers by\n\n1) Finding permutations of the weights that result in a functionally-equivalent, yet easier-to-compress network,\n2) Compressing the weights using product quantization [[1]](#1-product-quantization),\n3) Fine-tuning the codebooks via stochastic gradient descent.\n\nWe provide code for compressing and evaluating ResNet-18, ResNet-50 and Mask R-CNN.\n\n## Contents\n- [Requirements](#requirements)\n- [Data](#data)\n- [Training ResNet](#training-resnet)\n- [Training Mask R-CNN](#training-mask-r-cnn)\n- [Pretrained models](#pretrained-models)\n- [Evaluating ResNet](#evaluating-resnet)\n- [Evaluating Mask R-CNN](#evaluating-mask-r-cnn)\n- [Citation](#citation)\n- [References](#references)\n\n## Requirements\n\nOur code requires Python 3.6 or later. You also need these additional packages:\n\n* [PyTorch](https://pytorch.org/)\n* [Torchvision](https://github.com/pytorch/vision)\n* [Pycocotools](https://pypi.org/project/pycocotools/)\n* [Numpy](https://numpy.org/)\n* [PyYAML](https://pypi.org/project/PyYAML/)\n\nAdditionally, if you have installed [Horovod](https://github.com/horovod/horovod), you may train ResNet with multiple GPUs,\nbut the code will work with a single GPU even without Horovod.\n\n## Data\nOur experiments require either ImageNet (for classification) or COCO (for detection/segmentation).\nYou should set up a data directory with the datasets.\n\n```bash\n\u003cyour_data_path\u003e\n├── coco\n│   ├── annotations   (contains      6 json files)\n│   ├── train2017     (contains 118287 images)\n│   └── val2017       (contains   5000 images)\n└── imagenet\n    ├── train         (contains   1000 folders with images)\n    └── val           (contains   1000 folders with images)\n```\n\nThen, make sure to update the `imagenet_path` or `coco_path` field in the config files to point them to your data.\n\n## Training ResNet\n\nBesides making sure your ImageNet path is set up, make sure to also set up your `output_path` in the config file, or\npass them via the command line:\n\n```bash\npython -m src.train_resnet --config ../config/train_resnet50.yaml\n```\n\nThe `output_path` key inside the config file must specify a directory where all the training output should be saved.\nThis script will create 2 subdirectories, called `tensorboard` and `trained_models`, inside of the `output_path` directory.\n\nLaunching a tensorboard\nwith the `tensorboard` directory will allow you observe the training state and behavior over time.\n\n```bash\ntensorboard --logdir \u003cyour_tensorboard_path\u003e --bind_all --port 6006\n```\n\nThe `trained_models`\ndirectory will be populated with checkpoints of the saved model after initialization, and then after every epoch.\nIt will also separately store the \"best\" of these models (the one that attains the highest validation accuracy).\n\n## Training Mask R-CNN\n\nMask R-CNN (with a ResNet-50 backbone) can be trained by running the command:\n\n```bash\npython -m src.train_maskrcnn --config ../config/train_maskrcnn.yaml\n```\n\nOnce again, you need to specify the `output_path` and the dataset path in the config file before running this.\n\n## Pretrained models\n\nWe provide the compressed models we learned from running our code at\n\n```\n../compressed_models\n```\n\nAll models provided have been compressed with k = 256 centroids\n\n|Model (original top-1)              | Regime                       | Comp. ratio | Model size     | Top-1 accuracy (%)|\n|:---------------------              |:-----------                  |----------:  |-----------:    | -----------------:|\n| ResNet-18 (69.76%)                 | Small blocks\u003cbr\u003eLarge blocks | 29x\u003cbr\u003e43x  |1.54MB\u003cbr\u003e1.03MB| 66.74\u003cbr\u003e63.33    |\n| ResNet-50 (76.15%)                 | Small blocks\u003cbr\u003eLarge blocks | 19x\u003cbr\u003e31x  |5.09MB\u003cbr\u003e3.19MB| 75.04\u003cbr\u003e72.18    |\n| ResNet-50 Semi-Supervised (78.72%) | Small blocks                 | 19x         | 5.09MB         | 77.19             |\n\nWe also provide a compressed Mask R-CNN model that attains the following results compared to the uncompressed model:\n\n| Model                 | Size          | Comp. Ratio  | Box AP | Mask AP |\n| :------------         | ------------: | -----------: | -----: | ------: |\n| Original Mask R-CNN   | 169.4 MB      | -            | 37.9   | 34.6    |\n| Compressed Mask R-CNN | 6.65 MB       | 25.5x        | 36.3   | 33.5    |\n\nwhich you may use as given for evaluation.\n\n## Evaluating ResNet\n\nTo evaluate ResNet architectures run the following command from the project root:\n\n```bash\npython -m src.evaluate_resnet\n```\n\nThis will evaluate a ResNet-18 with small blocks by default. To evaluate a ResNet-18 with large blocks, use\n\n```bash\npython -m src.evaluate_resnet \\\n    --model.compression_parameters.large_subvectors True \\\n    --model.state_dict_compressed ../compressed_models/resnet18_large.pth\n```\n\nFor ResNet-50 with small blocks, use\n\n```bash\npython -m src.evaluate_resnet \\\n    --model.arch resnet50 \\\n    --model.compression_parameters.layer_specs.fc.k 1024 \\\n    --model.state_dict_compressed ../compressed_models/resnet50(_ssl).pth\n```\n\nYou may load the `resnet50_ssl.pth` model, which has been pretrained on an unsupervised dataset as well.\n\nAnd for ResNet-50 with large blocks, use\n\n```bash\npython -m src.evaluate_resnet \\\n    --model.arch resnet50 \\\n    --model.compression_parameters.pw_subvector_size 8 \\\n    --model.compression_parameters.large_subvectors True \\\n    --model.compression_parameters.layer_specs.fc.k 1024 \\\n    --model.state_dict_compressed ../compressed_models/resnet50_large.pth\n```\n\n## Evaluating Mask R-CNN\n\nSimply run the command:\n\n```bash\npython -m src.evaluate_maskrcnn\n```\n\nto load and evaluate the appropriate model.\n\n## Citation\n\nIf you use our code, please cite our work:\n\n```\n@article{martinez_2020_pqf,\n  title={Permute, Quantize, and Fine-tune: Efficient Compression of Neural Networks},\n  author={Martinez, Julieta and Shewakramani, Jashan and Liu, Ting Wei and B{\\^a}rsan, Ioan Andrei and Zeng, Wenyuan and Urtasun, Raquel},\n  journal={arXiv preprint arXiv:2010.15703},\n  year={2020}\n}\n```\n\n## References\n\n###### [1] [Product quantization for nearest neighbor search](https://hal.inria.fr/inria-00514462v2/document)\n###### [2] [And the bit goes down: Revisiting the quantization of neural networks](https://arxiv.org/abs/1907.05686)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fuber-research%2Fpermute-quantize-finetune","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fuber-research%2Fpermute-quantize-finetune","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fuber-research%2Fpermute-quantize-finetune/lists"}