{"id":13497275,"url":"https://github.com/CoinCheung/BiSeNet","last_synced_at":"2025-03-28T21:32:19.840Z","repository":{"id":37732947,"uuid":"159607436","full_name":"CoinCheung/BiSeNet","owner":"CoinCheung","description":"Add bisenetv2.  My implementation of BiSeNet","archived":false,"fork":false,"pushed_at":"2024-12-19T12:25:46.000Z","size":3792,"stargazers_count":1490,"open_issues_count":24,"forks_count":320,"subscribers_count":16,"default_branch":"master","last_synced_at":"2025-03-22T22:01:45.106Z","etag":null,"topics":["ade20k","bisenet","cityscapes","cocostuff","ncnn","openvino","pytorch","tensorrt","triton-inference-server"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CoinCheung.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-11-29T04:27:51.000Z","updated_at":"2025-03-20T09:14:24.000Z","dependencies_parsed_at":"2024-12-05T14:01:13.270Z","dependency_job_id":"f4d40624-540d-415f-810a-381ec78635b0","html_url":"https://github.com/CoinCheung/BiSeNet","commit_stats":{"total_commits":60,"total_committers":1,"mean_commits":60.0,"dds":0.0,"last_synced_commit":"3d9b2cf592bcb1185cb52d710b476d8a1bde8120"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CoinCheung%2FBiSeNet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CoinCheung%2FBiSeNet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CoinCheung%2FBiSeNet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CoinCheung%2FBiSeNet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CoinCheung","download_url":"https://codeload.github.com/CoinCheung/BiSeNet/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246105503,"owners_count":20724320,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ade20k","bisenet","cityscapes","cocostuff","ncnn","openvino","pytorch","tensorrt","triton-inference-server"],"created_at":"2024-07-31T20:00:27.925Z","updated_at":"2025-03-28T21:32:19.816Z","avatar_url":"https://github.com/CoinCheung.png","language":"Python","readme":"# BiSeNetV1 \u0026 BiSeNetV2\n\nMy implementation of [BiSeNetV1](https://arxiv.org/abs/1808.00897) and [BiSeNetV2](https://arxiv.org/abs/2004.02147).\n\n\nmIOUs and fps on cityscapes val set:\n| none | ss | ssc | msf | mscf | fps(fp32/fp16/int8) | link |\n|------|:--:|:---:|:---:|:----:|:---:|:----:|\n| bisenetv1 | 75.44 | 76.94 | 77.45 | 78.86 | 112/239/435 | [download](https://github.com/CoinCheung/BiSeNet/releases/download/0.0.0/model_final_v1_city_new.pth) |\n| bisenetv2 | 74.95 | 75.58 | 76.53 | 77.08 | 103/161/198 | [download](https://github.com/CoinCheung/BiSeNet/releases/download/0.0.0/model_final_v2_city.pth) |\n\nmIOUs on cocostuff val2017 set:\n| none | ss | ssc | msf | mscf | link |\n|------|:--:|:---:|:---:|:----:|:----:|\n| bisenetv1 | 31.49 | 31.42 | 32.46 | 32.55 | [download](https://github.com/CoinCheung/BiSeNet/releases/download/0.0.0/model_final_v1_coco_new.pth) |\n| bisenetv2 | 30.49 | 30.55 | 31.81 | 31.73 | [download](https://github.com/CoinCheung/BiSeNet/releases/download/0.0.0/model_final_v2_coco.pth) |\n\nmIOUs on ade20k val set:\n| none | ss | ssc | msf | mscf | link |\n|------|:--:|:---:|:---:|:----:|:----:|\n| bisenetv1 | 36.15 | 36.04 | 37.27 | 36.58 | [download](https://github.com/CoinCheung/BiSeNet/releases/download/0.0.0/model_final_v1_ade20k.pth) |\n| bisenetv2 | 32.53 | 32.43 | 33.23 | 31.72 | [download](https://github.com/CoinCheung/BiSeNet/releases/download/0.0.0/model_final_v2_ade20k.pth) |\n\nTips: \n\n1. **ss** means single scale evaluation, **ssc** means single scale crop evaluation, **msf** means multi-scale evaluation with flip augment, and **mscf** means multi-scale crop evaluation with flip evaluation. The eval scales and crop size of multi-scales evaluation can be found in [configs](./configs/).\n\n2. The fps is tested in different way from the paper. For more information, please see [here](./tensorrt).\n\n3. The authors of bisenetv2 used cocostuff-10k, while I used cocostuff-123k(do not know how to say, just same 118k train and 5k val images as object detection). Thus the results maybe different from paper. \n\n4. The authors did not report results on ade20k, thus there is no official training settings, here I simply provide a \"make it work\" result. Maybe the results on ade20k can be boosted with better settings.\n\n5. The model has a big variance, which means that the results of training for many times would vary within a relatively big margin. For example, if you train bisenetv2 on cityscapes for many times, you will observe that the result of **ss** evaluation of bisenetv2 varies between 73.1-75.1. \n\n\n## deploy trained models\n\n1. tensorrt  \nYou can go to [tensorrt](./tensorrt) for details.  \n\n2. ncnn  \nYou can go to [ncnn](./ncnn) for details.  \n\n3. openvino  \nYou can go to [openvino](./openvino) for details.  \n\n4. tis  \nTriton Inference Server(TIS) provides a service solution of deployment. You can go to [tis](./tis) for details.\n\n\n## platform\n\nMy platform is like this: \n\n* ubuntu 18.04\n* nvidia Tesla T4 gpu, driver 450.80.02\n* cuda 10.2/11.3\n* cudnn 8\n* miniconda python 3.8.8\n* pytorch 1.11.0\n\n\n## get start\n\nWith a pretrained weight, you can run inference on an single image like this: \n\n```\n$ python tools/demo.py --config configs/bisenetv2_city.py --weight-path /path/to/your/weights.pth --img-path ./example.png\n```\n\nThis would run inference on the image and save the result image to `./res.jpg`.  \n\nOr you can run inference on a video like this:  \n```\n$ python tools/demo_video.py --config configs/bisenetv2_coco.py --weight-path res/model_final.pth --input ./video.mp4 --output res.mp4\n```\nThis would generate segmentation file as `res.mp4`. If you want to read from camera, you can set `--input camera_id` rather than `input ./video.mp4`.   \n\n\n## prepare dataset\n\n1.cityscapes  \n\nRegister and download the dataset from the official [website](https://www.cityscapes-dataset.com/). Then decompress them into the `datasets/cityscapes` directory:  \n```\n$ mv /path/to/leftImg8bit_trainvaltest.zip datasets/cityscapes\n$ mv /path/to/gtFine_trainvaltest.zip datasets/cityscapes\n$ cd datasets/cityscapes\n$ unzip leftImg8bit_trainvaltest.zip\n$ unzip gtFine_trainvaltest.zip\n```\n\n2.cocostuff   \n\nDownload `train2017.zip`, `val2017.zip` and `stuffthingmaps_trainval2017.zip` split from official [website](https://cocodataset.org/#download). Then do as following:  \n```\n$ unzip train2017.zip\n$ unzip val2017.zip\n$ mv train2017/ /path/to/BiSeNet/datasets/coco/images\n$ mv val2017/ /path/to/BiSeNet/datasets/coco/images\n\n$ unzip stuffthingmaps_trainval2017.zip\n$ mv train2017/ /path/to/BiSeNet/datasets/coco/labels\n$ mv val2017/ /path/to/BiSeNet/datasets/coco/labels\n\n$ cd /path/to/BiSeNet\n$ python tools/gen_dataset_annos.py --dataset coco\n```\n\n3.ade20k\n\nDownload `ADEChallengeData2016.zip` from this [website](http://sceneparsing.csail.mit.edu/) and unzip it. Then we can move the uncompressed folders to `datasets/ade20k`, and generate the txt files with the script I prepared for you:  \n```\n$ unzip ADEChallengeData2016.zip\n$ mv ADEChallengeData2016/images /path/to/BiSeNet/datasets/ade20k/\n$ mv ADEChallengeData2016/annotations /path/to/BiSeNet/datasets/ade20k/\n$ python tools/gen_dataset_annos.py --dataset ade20k\n```\n\n\n4.custom dataset  \n\nIf you want to train on your own dataset, you should generate annotation files first with the format like this: \n```\nmunster_000002_000019_leftImg8bit.png,munster_000002_000019_gtFine_labelIds.png\nfrankfurt_000001_079206_leftImg8bit.png,frankfurt_000001_079206_gtFine_labelIds.png\n...\n```\nEach line is a pair of training sample and ground truth image path, which are separated by a single comma `,`.   \n\nI recommand you to check the information of your dataset with the script:  \n```\n$ python tools/check_dataset_info.py --im_root /path/to/your/data_root --im_anns /path/to/your/anno_file\n```\nThis will print some of the information of your dataset.  \n\nThen you need to change the field of `im_root` and `train/val_im_anns` in the config file. I prepared a demo config file for you named [`bisenet_customer.py`](./configs/bisenet_customer.py). You can start from this conig file.\n\n\n## train\n\nTraining commands I used to train the models can be found in [here](./dist_train.sh).\n\nNote:  \n1. though `bisenetv2` has fewer flops, it requires much more training iterations. The the training time of `bisenetv1` is shorter.\n2. I used overall batch size of 16 to train all models. Since cocostuff has 171 categories, it requires more memory to train models on it. I split the 16 images into more gpus than 2, as I do with cityscapes.\n\n\n## finetune from trained model\n\nYou can also load the trained model weights and finetune from it, like this:\n```\n$ export CUDA_VISIBLE_DEVICES=0,1\n$ torchrun --nproc_per_node=2 tools/train_amp.py --finetune-from ./res/model_final.pth --config ./configs/bisenetv2_city.py # or bisenetv1\n```\n\n\n## eval pretrained models\nYou can also evaluate a trained model like this: \n```\n$ python tools/evaluate.py --config configs/bisenetv1_city.py --weight-path /path/to/your/weight.pth\n```\nor you can use multi gpus:  \n```\n$ torchrun --nproc_per_node=2 tools/evaluate.py --config configs/bisenetv1_city.py --weight-path /path/to/your/weight.pth\n```\n\n\n### Be aware that this is the refactored version of the original codebase. You can go to the `old` directory for original implementation if you need, though I believe you will not need it.\n\n\n","funding_links":[],"categories":["SemanticSeg","✂️ Image Segmentation"],"sub_categories":["🎭 Portrait \u0026 Semantic Segmentation"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FCoinCheung%2FBiSeNet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FCoinCheung%2FBiSeNet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FCoinCheung%2FBiSeNet/lists"}