{"id":30874460,"url":"https://github.com/hvision-nku/offseg","last_synced_at":"2025-11-07T06:03:45.782Z","repository":{"id":313439960,"uuid":"1025630543","full_name":"HVision-NKU/OffSeg","owner":"HVision-NKU","description":"[ICCV 2025] Revisiting Efficient Semantic Segmentation: Learning Offsets for Better Spatial and Class Feature Alignment","archived":false,"fork":false,"pushed_at":"2025-09-06T02:39:35.000Z","size":5279,"stargazers_count":26,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-06T04:21:09.721Z","etag":null,"topics":["efficient-model","feature-alignment","offset-learning","semantic-segmentation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HVision-NKU.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-07-24T14:48:59.000Z","updated_at":"2025-09-06T02:39:38.000Z","dependencies_parsed_at":"2025-09-06T04:34:03.296Z","dependency_job_id":null,"html_url":"https://github.com/HVision-NKU/OffSeg","commit_stats":null,"previous_names":["hvision-nku/offseg"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/HVision-NKU/OffSeg","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HVision-NKU%2FOffSeg","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HVision-NKU%2FOffSeg/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HVision-NKU%2FOffSeg/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HVision-NKU%2FOffSeg/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HVision-NKU","download_url":"https://codeload.github.com/HVision-NKU/OffSeg/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HVision-NKU%2FOffSeg/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":283136761,"owners_count":26785489,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-11-07T02:00:06.343Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["efficient-model","feature-alignment","offset-learning","semantic-segmentation"],"created_at":"2025-09-08T00:47:51.632Z","updated_at":"2025-11-07T06:03:45.775Z","avatar_url":"https://github.com/HVision-NKU.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Revisiting Efficient Semantic Segmentation: Learning Offsets for Better Spatial and Class Feature Alignment (ICCV 2025)\n\n### [Project page](https://github.com/HVision-NKU/OffSeg) | [Paper](https://mftp.mmcheng.net/Papers/25ICCV-OffsetLearning.pdf) | [arXiv](https://arxiv.org/abs/2508.08811) | [中译版](resources/25ICCV-OffsetLearning-cn.pdf) | [Jittor]()\n\nThis repository contains the official Pytorch implementation of training \u0026 evaluation code and the trained models for [Offset Learning \u0026 OffSeg](https://arxiv.org/abs/2508.08811).\n\n### Offset Learning —— An efficient plug-and-play semantic segmentation paradigm that replaces existing per-pixel classification paradigm to boost performance with negligible parameters.\n\n![Framework Overview](resources/framework.png)\n*Overview of the Offset Learning framework for semantic segmentation.*\n\n\u003cdetails\u003e\n\u003csummary\u003eAbstract\u003c/summary\u003e\nOffset Learning is a new semantic segmentation paradigm that efficiently learns feature offsets and class offsets to dynamically refine both spatial features and class representations, addressing the inherent misalignment problem in per-pixel classification. Based on this paradigm, we design OffSeg, an efficient segmentation network that delivers consistent accuracy improvements on multiple benchmarks. Notably, the Offset Learning paradigm is plug-and-play, allowing it to directly replace other segmentation paradigms in existing models to achieve performance gains with only negligible parameter overhead.\n\u003c/details\u003e\n\n## Features\n* **Offset Learning**: Learns feature offsets and class offsets to dynamically refine spatial features and class representations.\n* **Plug-and-play**: Compatible with existing segmentation frameworks like SegFormer, SegNeXt, and Mask2Former.\n* **Lightweight \u0026 Efficient**: Achieves consistent accuracy gains on multiple benchmarks with negligible parameter overhead.\n* **Proven Effectiveness**: Validated across diverse models and datasets, showing strong improvements especially in lightweight settings.\n\n## News\n- **`2025.09.06`**: The [Chinese version](resources/25ICCV-OffsetLearning-cn.pdf) has been updated for Chinese readers.\n- **`2025.08.13`**: Add [tutorial](toturials/Toturial:Using_Offset_Learning_In_Your_Own_Model.md) on how to apply the Offset Learning paradigm to your own models.\n- **`2025.08.12`**: The full training \u0026 evaluation code \u0026 [Jittor version code](https://github.com/HVision-NKU/OffSeg/tree/jittor) and the trained models are released.\n- **`2025.06.26`**: Our paper is accepted to ICCV 2025!\n\n## TODO\n* [x] Release the full training \u0026 evaluation code and model weights. \n* [x] Tutorial on how to apply the Offset Learning paradigm to your own models.\n* [x] Release the jittor version for jittor users.\n* [ ] Release the Python library for easier installation via `pip install`.\n* [ ] Explore the generalization ability of Offset Learning on tasks beyond semantic segmentation.\n\n## Get Started\n\n### Installation\n\n```bash\nconda create -n offseg python=3.9 -y\nconda activate offseg\n\n# Install PyTorch (CUDA 11.8 example)\nconda install pytorch==2.0.0 torchvision==0.15.0 torchaudio==2.0.0 pytorch-cuda=11.8 -c pytorch -c nvidia\n\n# install mmcv using mim\npip install -U openmim\nmim install mmengine\nmim install mmcv==2.0.0\nmim install mmdet\n\npip install ftfy transformers==4.28.0\n\n# Install OffSeg\npip install -e .\n```\n\n### Data Preparation\n\nFor data preparation, please refer to the guidelines in [mmsegmentation](https://github.com/open-mmlab/mmsegmentation/blob/main/docs/en/user_guides/2_dataset_prepare.md#prepare-datasets).\nIt is recommended to symlink the dataset root to `OffSeg/data`.\n\n\u003cdetails\u003e\n\u003csummary\u003eFor convenience, the recommended folder structure is as follows: \u003c/summary\u003e\n\n```\nOffSeg\n├── data\n│   ├── ade\n│   │   ├── ADEChallengeData2016\n│   │   │   ├── annotations\n│   │   │   │   ├── training\n│   │   │   │   ├── validation\n│   │   │   ├── images\n│   │   │   │   ├── training\n│   │   │   │   ├── validation\n│   ├── cityscapes\n│   │   ├── leftImg8bit\n│   │   │   ├── train\n│   │   │   ├── val\n│   │   ├── gtFine\n│   │   │   ├── train\n│   │   │   ├── val\n│   ├── coco_stuff164k\n│   │   ├── images\n│   │   │   ├── train2017\n│   │   │   ├── val2017\n│   │   ├── annotations\n│   │   │   ├── train2017\n│   │   │   ├── val2017\n│   ├── VOCdevkit\n│   │   ├── VOC2010\n│   │   │   ├── JPEGImages\n│   │   │   ├── SegmentationClassContext\n│   │   │   ├── ImageSets\n│   │   │   │   ├── SegmentationContext\n│   │   │   │   │   ├── train.txt\n│   │   │   │   │   ├── val.txt\n│   │   │   ├── trainval_merged.json\n```\n\n\u003c/details\u003e\n\n### Checkpoints\n\nThe trained models can be downloaded at:\n| Model                          | GoogleDrive | OneDrive | BaiduNetdisk |\n|--------------------------------|-------------|----------|--------------|\n| OffSeg                         | [GoogleDrive](https://drive.google.com/drive/folders/1RpQ1ouZJpiIPQU0nRpl9wGeOVoQ-_ZEy?usp=sharing) | [OneDrive](https://mailnankaieducn-my.sharepoint.com/:f:/g/personal/zhangshichen_mail_nankai_edu_cn/Er6Lo--3WIlCnj03f6xOzJkBhuJuNDpdBJoTsTJa0vbQgQ) | [BaiduNetdisk](https://pan.baidu.com/s/1iMyz_u46antonjauVuaXcg?pwd=2025) |\n| SegFormer w/ Offset Learning   | [GoogleDrive](https://drive.google.com/drive/folders/1IADpcyR6Ld1_kLmKF2FV9y_hwZtNUPOB?usp=sharing) | [OneDrive](https://mailnankaieducn-my.sharepoint.com/:f:/g/personal/zhangshichen_mail_nankai_edu_cn/EnE8OIJwszFPrf0yJcugkQABLjthQQzwJagbJm6rbK6IZw) | [BaiduNetdisk](https://pan.baidu.com/s/1yqil_EQkXJHokza5jNuGPg?pwd=2025) |\n| SegNeXt w/ Offset Learning     | [GoogleDrive](https://drive.google.com/drive/folders/1OHgoN3wpWmqEHdtqxyBCwCy5k7m_vZlJ?usp=sharing) | [OneDrive](https://mailnankaieducn-my.sharepoint.com/:f:/g/personal/zhangshichen_mail_nankai_edu_cn/Eif6LjQW145Lg-gAOnpwqWABnDIQvMRoeH6j5HiZd6MQzg) | [BaiduNetdisk](https://pan.baidu.com/s/1S8VadoubEPJj-vB2-lUIEg?pwd=2025) |\n| Mask2Former w/ Offset Learning | [GoogleDrive](https://drive.google.com/drive/folders/13bsb0XTtw1nXfPpmA4FThZ3j1ZZN8_PL?usp=sharing) | [OneDrive](https://mailnankaieducn-my.sharepoint.com/:f:/g/personal/zhangshichen_mail_nankai_edu_cn/Esv293NPghdMmnB6G36RY0UBi2lb6AeXAxYR4W_xwoy4jA) | [BaiduNetdisk](https://pan.baidu.com/s/1M7i7ETrIEZ-nLnbt96yTUg?pwd=2025) |\n\n## Evaluation\n\n### Single GPU Evaluation\n\n```bash\n# OffSeg-T on ADE20K\npython tools/test.py local_configs/offseg/Tiny/offseg-t_ade20k_160k-512x512.py /path/to/checkpoint.pth\n\n# SegFormer-B0 with Offset Learning on COCO-Stuff\npython tools/test.py local_configs/segformer_offset_learning/B0/segformer_mit-b0_offset_learning_8xb2-80k_stuff164k-512x512.py /path/to/checkpoint.pth\n\n# SegNeXt-T with Offset Learning on Pascal Context\npython tools/test.py local_configs/segnext_offset_learning/Tiny/segnext_mscan-t_offset_learning_80k_pascal-context-59_480x480.py /path/to/checkpoint.pth\n```\n\n### Multi-GPU Evaluation\n\n```bash\n# Example with 8 GPUs\nbash tools/dist_test.sh local_configs/offseg/Tiny/offseg-t_ade20k_160k-512x512.py /path/to/checkpoint.pth 8\n```\n\n### Evaluation with Visualization\n\n```bash\n# Show results during evaluation\npython tools/test.py local_configs/offseg/Tiny/offseg-t_ade20k_160k-512x512.py /path/to/checkpoint.pth --show\n\n# Save visualization results\npython tools/test.py local_configs/offseg/Tiny/offseg-t_ade20k_160k-512x512.py /path/to/checkpoint.pth --show-dir ./vis_results\n```\n\n## Training\n\n### Single GPU Training\n\n```bash\n# Train OffSeg-B on ADE20K\npython tools/train.py local_configs/offseg/Base/offseg-b_ade20k_160k-512x512.py\n\n# Train with custom work directory\npython tools/train.py local_configs/offseg/Base/offseg-b_ade20k_160k-512x512.py --work-dir ./work_dirs/offseg-b_ade20k_160k-512x512\n\n# Resume training from checkpoint\npython tools/train.py local_configs/offseg/Base/offseg-b_ade20k_160k-512x512.py --resume\n```\n\n### Multi-GPU Training\n\n```bash\n# Example with 8 GPUs\nbash tools/dist_train.sh local_configs/offseg/Base/offseg-b_ade20k_160k-512x512.py 8\n```\n\n## Visualization\n\n### Image Demo\n\n```bash\n# Single image inference\npython demo/image_demo.py demo/demo.png local_configs/offseg/Tiny/offseg-t_ade20k_160k-512x512.py /path/to/checkpoint.pth --out-dir ./demo_results\n```\n\n## Citation\n\nIf you find this work useful for your research, please cite our paper:\n\n```bibtex\n@article{zhang2025revisiting,\n  title={Revisiting Efficient Semantic Segmentation: Learning Offsets for Better Spatial and Class Feature Alignment},\n  author={Zhang, Shi-Chen and Li, Yunheng and Wu Yu-Huan and Hou, Qibin and Cheng, Ming-Ming},\n  journal={arXiv preprint arXiv:2508.08811},\n  year={2025}\n}\n```\n\n## Acknowledgment\n\nThis project is built upon [MMSegmentation](https://github.com/open-mmlab/mmsegmentation). We thank the MMSegmentation team for their open-source contribution.\nWe also thank the following open-source projects for their inspiring work: [SegFormer](https://github.com/NVlabs/SegFormer), [SegNeXt](https://github.com/visual-attention-network/segnext), [Mask2Former](https://github.com/facebookresearch/Mask2Former), [FreqFusion](https://github.com/Linwei-Chen/FreqFusion), [EfficientFormerV2](https://github.com/snap-research/EfficientFormer).\n\n\n## License\nThe code is limited to non-commercial, academic, or research purposes only. For commercial use, please contact the authors for licensing.\n\n## Contact\n\nFor questions and issues, please contact:\n- Email: [zhangshichen@mail.nankai.edu.cn]\n- Issues: [GitHub Issues](https://github.com/HVision-NKU/OffSeg/issues)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhvision-nku%2Foffseg","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhvision-nku%2Foffseg","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhvision-nku%2Foffseg/lists"}