{"id":16609337,"url":"https://github.com/willbrennan/semanticsegmentation","last_synced_at":"2025-08-21T18:23:29.625Z","repository":{"id":50284284,"uuid":"272821356","full_name":"WillBrennan/SemanticSegmentation","owner":"WillBrennan","description":"A framework for training segmentation models in pytorch on labelme annotations with pretrained examples of skin, cat, and pizza topping segmentation","archived":false,"fork":false,"pushed_at":"2023-10-21T06:59:57.000Z","size":2827,"stargazers_count":184,"open_issues_count":1,"forks_count":30,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-31T11:21:13.388Z","etag":null,"topics":["birds","bisenet","bisenetv2","cats","coco","computer-vision","labelme","labelme-annotations","pizza","pizza-toppings","pytorch","segmentation","semantic-segmentation","skin-detection","skin-segmentation","torchvision"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/WillBrennan.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null},"funding":{"patreon":"WillBrennan"}},"created_at":"2020-06-16T22:05:03.000Z","updated_at":"2025-02-23T11:19:31.000Z","dependencies_parsed_at":"2022-08-12T21:01:58.590Z","dependency_job_id":null,"html_url":"https://github.com/WillBrennan/SemanticSegmentation","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WillBrennan%2FSemanticSegmentation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WillBrennan%2FSemanticSegmentation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WillBrennan%2FSemanticSegmentation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WillBrennan%2FSemanticSegmentation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/WillBrennan","download_url":"https://codeload.github.com/WillBrennan/SemanticSegmentation/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252902731,"owners_count":21822292,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["birds","bisenet","bisenetv2","cats","coco","computer-vision","labelme","labelme-annotations","pizza","pizza-toppings","pytorch","segmentation","semantic-segmentation","skin-detection","skin-segmentation","torchvision"],"created_at":"2024-10-12T01:28:48.766Z","updated_at":"2025-05-07T15:21:57.007Z","avatar_url":"https://github.com/WillBrennan.png","language":"Python","funding_links":["https://patreon.com/WillBrennan"],"categories":[],"sub_categories":[],"readme":"# Semantic Segmentation\n## Overview\nThis project started as a replacement to the [Skin Detection](https://github.com/WillBrennan/SkinDetector) project that used traditional computer vision techniques. This project implements two models, \n\n- `FCNResNet101` from torchvision for accurate segmentation\n- `BiSeNetV2` for real-time segmentation\n\nThese models are trained with masks from labelme annotations. As labelme annotations allow for multiple categories per a pixel we use multi-label semantic segmentation. Both the accurate and real-time models are in the pretrained directory.\n\n## Getting Started\nThe pretrained models are stored in the repo with git-lfs, when you clone make sure you've pulled the files by calling, \n\n```bash\ngit lfs pull\n```\n or by downloading them from github directly. This project uses conda to manage its enviroment; once conda is installed we create the enviroment and activate it, \n```bash\nconda env create -f enviroment.yml\nconda activate semantic_segmentation\n```\n. On windows; powershell needs to be initialised and the execution policy needs to be modified. \n```bash\nconda init powershell\nSet-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser\n```\n\n## Pre-Trained Segmentation Projects\nThis project comes bundled with several pretrained models, which can be found in the `pretrained` directory. To infer segmentation masks on your images run `evaluate_images`.\n```bash\n# to display the output\npython evaluate_images.py --images ~/Pictures/ --model pretrained/model_segmentation_skin_30.pth --model-type FCNResNet101 --display\n# to save the output\npython evaluate_images.py --images ~/Pictures/ --model pretrained/model_segmentation_skin_30.pth --model-type FCNResNet101 --save\n```\n\nTo run the real-time models change the `--model-type`, \n```bash\n# to display the output\npython evaluate_images.py --images ~/Pictures/ --model pretrained/model_segmentation_realtime_skin_30.pth --model-type BiSeNetV2 --display\n# to save the output\npython evaluate_images.py --images ~/Pictures/ --model pretrained/model_segmentation_realtime_skin_30.pth --model-type BiSeNetV2 --save\n```\n\n\n### Skin Segmentation\nThis model was trained with a custom dataset of 150 images taken from COCO where skin segmentation annotations were added. This includes a wide variety of skin colours and lighting conditions making it more robust than the [Skin Detection](https://github.com/WillBrennan/SkinDetector) project. This model detects, \n\n- skin\n![Skin Segmentation](https://raw.githubusercontent.com/WillBrennan/SemanticSegmentation/master/pretrained/skin_examples.png)\n\n### Pizza Topping Segmentation\nThis was trained with a custom dataset of 89 images taken from COCO where pizza topping annotations were added. There's very few images for each type of topping so this model performs very badly and needs quite a few more images to behave well!\n\n- 'chilli', 'ham', 'jalapenos', 'mozzarella', 'mushrooms', 'olive', 'pepperoni', 'pineapple', 'salad', 'tomato'\n\n![Pizza Toppings](https://raw.githubusercontent.com/WillBrennan/SemanticSegmentation/master/pretrained/pizza_toppings_example.png)\n\n### Cat and Bird Segmentation\nAnnotated images of birds and cats were taken from COCO using the `extract_from_coco` script and then trained on. \n\n- cat, birds\n\n![Demo on Cat \u0026 Birds](https://raw.githubusercontent.com/WillBrennan/SemanticSegmentation/master/pretrained/cat_examples.png)\n\n\n## Training New Projects\nTo train a new project you can either create new labelme annotations on your images, to launch labelme run, \n\n```bash\nlabelme\n```\nand start annotating your images! You'll need a couple of hundred. Alternatively if your category is already in COCO you can run the conversion tool to create labelme annotations from them. \n\n```bash\npython extract_from_coco.py --images ~/datasets/coco/val2017 --annotations ~/datasets/coco/annotations/instances_val2017.json --output ~/datasets/my_cat_images_val --categories cat\n```\n\nOnce you've got a directory of labelme annotations you can check how the images will be shown to the model during training by running, \n\n```bash\npython check_dataset.py --dataset ~/datasets/my_cat_images_val\n# to show our dataset with training augmentation\npython check_dataset.py --dataset ~/datasets/my_cat_images_val --use-augmentation\n```\n. If your happy with the images and how they'll appear in training then train the model using, \n\n```bash\npython train.py --train ~/datasets/my_cat_images_train --val ~/datasets/my_cat_images_val --model-tag segmentation_cat --model-type FCNResNet101\n```\n. This may take some time depending on how many images you have. Tensorboard logs are available in the `logs` directory. To run your trained model on a directory of images run\n\n```bash\n# to display the output\npython evaluate_images.py --images ~/Pictures/my_cat_imgs --model models/model_segmentation_cat_30.pth --model-type FCNResNet101 --display \n# to save the output\npython evaluate_images.py --images ~/Pictures/my_cat_imgs --model models/model_segmentation_cat_30.pth --model-type FCNResNet101 --save\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwillbrennan%2Fsemanticsegmentation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwillbrennan%2Fsemanticsegmentation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwillbrennan%2Fsemanticsegmentation/lists"}