{"id":17330601,"url":"https://github.com/yoshitomo-matsubara/sc2-benchmark","last_synced_at":"2025-04-13T09:36:54.093Z","repository":{"id":43109093,"uuid":"420762694","full_name":"yoshitomo-matsubara/sc2-benchmark","owner":"yoshitomo-matsubara","description":"[TMLR] \"SC2 Benchmark: Supervised Compression for Split Computing\"","archived":false,"fork":false,"pushed_at":"2025-03-13T00:13:24.000Z","size":6812,"stargazers_count":27,"open_issues_count":0,"forks_count":12,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-13T01:20:06.117Z","etag":null,"topics":["coco","image-classification","imagenet","knowledge-distillation","object-detection","pascal-voc","pytorch","semantic-segmentation","split-computing","supervised-compression"],"latest_commit_sha":null,"homepage":"https://yoshitomo-matsubara.net/sc2-benchmark/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yoshitomo-matsubara.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.bib","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-10-24T18:26:20.000Z","updated_at":"2025-03-13T00:10:12.000Z","dependencies_parsed_at":"2024-10-28T07:51:51.449Z","dependency_job_id":"68d3be04-0744-4531-9895-6c50d330e5cc","html_url":"https://github.com/yoshitomo-matsubara/sc2-benchmark","commit_stats":{"total_commits":246,"total_committers":4,"mean_commits":61.5,"dds":0.04065040650406504,"last_synced_commit":"212955cfb20bc33aa5c24a9482f7ca0513e852ee"},"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yoshitomo-matsubara%2Fsc2-benchmark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yoshitomo-matsubara%2Fsc2-benchmark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yoshitomo-matsubara%2Fsc2-benchmark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yoshitomo-matsubara%2Fsc2-benchmark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yoshitomo-matsubara","download_url":"https://codeload.github.com/yoshitomo-matsubara/sc2-benchmark/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245761295,"owners_count":20667895,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["coco","image-classification","imagenet","knowledge-distillation","object-detection","pascal-voc","pytorch","semantic-segmentation","split-computing","supervised-compression"],"created_at":"2024-10-15T14:51:44.036Z","updated_at":"2025-03-27T01:09:42.524Z","avatar_url":"https://github.com/yoshitomo-matsubara.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SC2 Benchmark: Supervised Compression for Split Computing\n[![PyPI version](https://badge.fury.io/py/sc2bench.svg)](https://badge.fury.io/py/sc2bench)\n[![GitHub Discussions](https://img.shields.io/github/discussions/yoshitomo-matsubara/sc2-benchmark)](https://github.com/yoshitomo-matsubara/sc2-benchmark/discussions)  \n\nThis is the official repository of `sc2bench` package and our TMLR paper, [\"SC2 Benchmark: Supervised Compression for Split Computing\"](#citation).\n\nAs an intermediate option between local computing and edge computing (full offloading), ***split computing*** has been \nattracting considerable attention from the research communities.\n\nIn split computing, we split a neural network model into two sequences so that some elementary feature transformations\nare applied by the first sequence of the model on a weak mobile (local) device. \nThen, intermediate, informative features are transmitted through a wireless communication channel to a powerful \nedge server that processes the bulk part of the computation (the second sequence of the model).  \n\n\n![R-D (rate-distortion), ExR-D, and Three-way tradeoffs](https://raw.githubusercontent.com/yoshitomo-matsubara/sc2-benchmark/main/imgs/ilsvrc2012-overview.png)\nImageNet (ILSVRC 2012): R-D (rate-distortion), ExR-D, and three-way tradeoffs for input compression and supervised compression with ResNet-50 as a reference model\n\n![Input compression vs. Supervised compression](https://raw.githubusercontent.com/yoshitomo-matsubara/sc2-benchmark/main/imgs/input_vs_supervised_compression.png)\n\nInput compression is an approach to save transmitted data, but it leads to transmitting information irrelevant to \nthe supervised task.\nTo achieve better supervised rate-distortion tradeoff, we define ***supervised compression*** as \nlearning compressed representations for supervised downstream tasks such as classification, detection, or segmentation.\nSpecifically for split computing, we term the problem setting **SC2** (*Supervised Compression for Split Computing*).  \n\nNote that the training process can be done offline (i.e., on a single device without splitting), \nand it is different from \"split learning\".\n\n## SC2 Metrics\n### 1. Encoder Size (to be minimized)\nLocal processing cost should be minimized as local (mobile) devices usually have battery constraints and limited computing power.\nAs a simple proxy for the computing costs, we measure the number of encoder parameters and \ndefine the encoder size as the total number of bits to represent the parameters of the encoder.\n\n### 2. Data Size (to be minimized)\nWe want to penalize large data being transferred from the mobile device to the edge server while the BPP does not \npenalize it when feeding higher resolution images to downstream models for achieving higher model accuracy.\n\n### 3. Model Accuracy (to be maximized)\nWhile minimizing the two metrics, we want to maximize model accuracy (minimize supervised distortion).\nExample supervised distortions are accuracy, mean average precision (mAP), and mean intersection over union (mIoU) for \nimage classification, object detection, and semantic segmentation, respectively.\n\n## Installation\n```shell\npip install sc2bench\n```\n\n### Virtual Environments\nFor pipenv users,\n```shell\npipenv install --python 3.9\n# or create your own pipenv environment\npipenv install sc2bench \n```\n\n## Datasets\nSee instructions [here](script#datasets)\n\n## Checkpoints\nYou can download our checkpoints including trained model weights [here](https://github.com/yoshitomo-matsubara/sc2-benchmark/releases/tag/v0.0.3).  \nUnzip the downloaded zip files under `./`, then there will be `./resource/ckpt/`.\n\n## Supervised Compression\n1. CR + BQ: [\"Neural Compression and Filtering for Edge-assisted Real-time Object Detection in Challenged Networks\"](https://arxiv.org/abs/2007.15818)\n2. End-to-End: [\"End-to-end Learning of Compressible Features\"](https://arxiv.org/abs/2007.11797) \n3. Entropic Student: [\"Supervised Compression for Resource-Constrained Edge Computing Systems\"](https://openaccess.thecvf.com/content/WACV2022/html/Matsubara_Supervised_Compression_for_Resource-Constrained_Edge_Computing_Systems_WACV_2022_paper.html)\n\n[README.md](https://github.com/yoshitomo-matsubara/sc2-benchmark/tree/main/script/task) explains how to train/test implemented supervised compression methods.\n\n## Baselines: Input Compression\n- [Codec-based input compression](https://github.com/yoshitomo-matsubara/sc2-benchmark/tree/main/script/codec_input_compression): JPEG, WebP, BPG\n- [Neural input compression](https://github.com/yoshitomo-matsubara/sc2-benchmark/tree/main/script/neural_input_compression): Factorized Prior, Scale Hyperprior, Mean-scale Hyperprior, and Joint Autoregressive Hierarchical Prior\n\nEach **README.md** gives instructions to run the baseline experiments.\n\n## Codec-based Feature Compression\n```shell\n# JPEG\npython script/task/image_classification.py -test_only --config configs/ilsvrc2012/feature_compression/jpeg-resnet50.yaml\n# WebP\npython script/task/image_classification.py -test_only --config configs/ilsvrc2012/feature_compression/webp-resnet50.yaml\n```\n\n## Citation\n[[Paper](https://openreview.net/forum?id=p28wv4G65d)] [[Preprint](https://arxiv.org/abs/2203.08875)]\n```bibtex\n@article{matsubara2023sc2,\n  title={{SC2 Benchmark: Supervised Compression for Split Computing}},\n  author={Matsubara, Yoshitomo and Yang, Ruihan and Levorato, Marco and Mandt, Stephan},\n  journal={Transactions on Machine Learning Research},\n  issn={2835-8856},\n  year={2023},\n  url={https://openreview.net/forum?id=p28wv4G65d}\n}\n```\n\n## Note\nFor measuring data size per sample precisely, it is important to keep test batch size of 1 when testing.  \nE.g., some baseline modules may expect larger batch size if you have multiple GPUs.  \nThen, add `CUDA_VISIBLE_DEVICES=0` before your execution command (e.g., `sh`, `bash`, `python`) \nso that you can force the script to use one GPU (use GPU: 0 in this case).\n\nFor instance, an input compression experiment using factorized prior (pretrained input compression model) \nand ResNet-50 (pretrained classifier)\n```shell\nCUDA_VISIBLE_DEVICES=0 sh script/neural_input_compression/ilsvrc2012-image_classification.sh factorized_prior-resnet50 8\n```\n\n## Issues / Questions / Requests\nThe documentation is work-in-progress. In the meantime, feel free to create an issue if you find a bug.  \nIf you have either a question or feature request, start a new discussion [here](https://github.com/yoshitomo-matsubara/sc2-benchmark/discussions).\n\n## References\n- [PyTorch (torchvision)](https://github.com/pytorch/vision)\n- [PyTorch Image Models (timm)](https://github.com/rwightman/pytorch-image-models)\n- [CompressAI](https://github.com/InterDigitalInc/CompressAI)\n- [torchdistill](https://github.com/yoshitomo-matsubara/torchdistill) \n- Johannes Ballé, David Minnen, Saurabh Singh, Sung Jin Hwang and Nick Johnston. [\"Variational image compression with a scale hyperprior\"](https://openreview.net/forum?id=rkcQFMZRb) (ICLR 2018)\n- David Minnen, Johannes Ballé and George D. Toderici. [\"Joint Autoregressive and Hierarchical Priors for Learned Image Compression\"](https://proceedings.neurips.cc/paper/2018/hash/53edebc543333dfbf7c5933af792c9c4-Abstract.html) (NeurIPS 2018)\n- Yoshitomo Matsubara and Marco Levorato. [\"Neural Compression and Filtering for Edge-assisted Real-time Object Detection in Challenged Networks\"](https://arxiv.org/abs/2007.15818) (ICPR 2020)\n- Saurabh Singh, Sami Abu-El-Haija, Nick Johnston, Johannes Ballé, Abhinav Shrivastava and George Toderici. [\"End-to-end Learning of Compressible Features\"](https://arxiv.org/abs/2007.11797) (ICIP 2020) \n- Yoshitomo Matsubara, Ruihan Yang, Marco Levorato and Stephan Mandt. [\"Supervised Compression for Resource-Constrained Edge Computing Systems\"](https://openaccess.thecvf.com/content/WACV2022/html/Matsubara_Supervised_Compression_for_Resource-Constrained_Edge_Computing_Systems_WACV_2022_paper.html) (WACV 2022)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyoshitomo-matsubara%2Fsc2-benchmark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyoshitomo-matsubara%2Fsc2-benchmark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyoshitomo-matsubara%2Fsc2-benchmark/lists"}