{"id":18792438,"url":"https://github.com/prbonn/pspa","last_synced_at":"2025-06-22T22:39:38.538Z","repository":{"id":237236942,"uuid":"733070820","full_name":"PRBonn/PSPA","owner":"PRBonn","description":"Panoptic Segmentation with Partial Annotations for Agricultural Robots","archived":false,"fork":false,"pushed_at":"2024-05-17T09:34:56.000Z","size":1104,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-05-17T10:38:49.758Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PRBonn.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-12-18T13:52:40.000Z","updated_at":"2024-05-17T09:35:00.000Z","dependencies_parsed_at":"2024-04-30T14:27:50.126Z","dependency_job_id":"3274ce72-000d-459c-bc46-90b467d827dd","html_url":"https://github.com/PRBonn/PSPA","commit_stats":null,"previous_names":["prbonn/pspa"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRBonn%2FPSPA","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRBonn%2FPSPA/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRBonn%2FPSPA/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRBonn%2FPSPA/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PRBonn","download_url":"https://codeload.github.com/PRBonn/PSPA/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223589732,"owners_count":17170031,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T21:19:52.254Z","updated_at":"2024-11-07T21:19:53.859Z","avatar_url":"https://github.com/PRBonn.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PSPA\n\n*[Panoptic Segmentation with Partial Annotations for Agricultural Robots](https://www.ipb.uni-bonn.de/wp-content/papercite-data/pdf/weyler2024ral.pdf)*\n\nWe present a novel approach to leverage partial annotations for panoptic segmentation.\nThese partial annotations contain\nground truth information only for a subset of pixels per image and are thus\nmuch faster to obtain than dense annotations. We propose a novel set of losses\nthat exploit measures from vector fields used in physics, i.e., divergence and\ncurl, to effectively supervise predictions without ground truth annotations.\n\n![Motivation](./static/network.jpg)\n\nExemplary, we show in the following figure a comparision between dense annotations (left)\nand partial annotations (right). The former contains ground truth annotations \nfor all instances, i.e., *things*, and each pixel in the background, i.e., *stuff*.\nIn contrast, the latter annotations contain annotations for a few instances and \nsome blob-like labels for the background.\n\n![Annotations](./static/annotations.jpg)\n\n# Requirements\n- We assume that ```wget``` is installed on your machine and that your CUDA Runtime version is \u003e= 11.0\n- We use ```conda``` to setup a virtual environment - in case ```conda``` is not installed on your machine follow the offical [instructions](https://docs.conda.io/projects/conda/en/latest/user-guide/install/linux.html) (we recommend miniconda)\n\n# Setup\nThe following script will setup a virtual environment denoted as *pspa*\n```bash\n./setup.sh\n```\n\n# Datasets\n- We use the [PhenoBench](https://www.phenobench.org/) dataset\n- However, as mentioned in the paper, we use a subset of images during training to ensure that each unique instance appears only once\n- Consequently, we provide in ```phenobench_auxiliary/split.yaml``` the filenames of images used during training\n- To download the dataset and organize it in the expected format run the following script (requires approx. 15GB)\n```bash \ncd ./phenobench_auxiliary\n./get_phenobench.sh \u003cfull/path/to/download/PhenoBench\u003e\ncd ..\n```\n\n\n# Pretrained Models\nWe provide pretrained models using ERFNet as network architecture that are trained with different amount of partial annotations:\n- [Model](https://www.ipb.uni-bonn.de/html/deeplearningmodels/weyler2024ral/model_100.ckpt) trained with all annotations\n- [Model](https://www.ipb.uni-bonn.de/html/deeplearningmodels/weyler2024ral/model_050.ckpt) trained with 50% of all annotations\n- [Model](https://www.ipb.uni-bonn.de/html/deeplearningmodels/weyler2024ral/model_025.ckpt) trained with 25% of all annotations\n- [Model](https://www.ipb.uni-bonn.de/html/deeplearningmodels/weyler2024ral/model_010.ckpt) trained with 10% of all annotations\n\n# Inference\n\nBefore you start the inference you need to specify the path to the dataset in the corresponding configuration file (```config/config-phenobench.yaml```), e.g.:\n```yaml\ndata:\n  path: \u003cfull/path/to/download/PhenoBench/test\u003e\n```\nPlease change only ```\u003cfull/path/to/download/PhenoBench\u003e``` according to your previously specified path to download PhenoBench but keep ```test``` as a directory at the very end.\n\nNext, you can run the model in inference mode\n```python\nconda activate pspa\npython predict.py --config ./config/config-phenobench.yaml  --export \u003cpath/to/export_dir\u003e --ckpt \u003cpath/to/checkpoint/model.ckpt\u003e\n```\nIn the specified ```\u003cpath/to/export_dir\u003e``` you will find the predicted semantics and plant instances.\n\nIn case you want to apply our proposed *fusing procedure*, i.e., assign an unique semantic class to each instance, you need run the following subsequently\n```python\nconda activate pspa\npython auxiliary/merge_plants_sem.py --semantics \u003cpath/to/previously/predicted/semantics\u003e --plants \u003cpath/to/previously/predicted/plant/instances\u003e --export \u003cpath/to/merged_export_dir\u003e\n```\n\nTo be more specific about the paths:\n- ```\u003cpath/to/previously/predicted/semantics\u003e``` should be ```\u003cpath/to/export_dir/lightning_logs/version_*/predict/export/semantics/000\u003e```\n- ```\u003cpath/to/previously/predicted/plant/instances\u003e``` shoulde be ```\u003cpath/to/export_dir/lightning_logs/version_*/predict/export/instances/000\u003e```\n- Both paths contain png files with the previous semantic and instance predictions\n\n\n# Evaluation\nSince PhenoBench provides a hidden test set you need to register for the corresponding [CodaLab challenge](https://codalab.lisn.upsaclay.fr/competitions/14153) and upload your results\nto run the evaluation. \n\n# Training\nSimiliarly you can train a new model\n```python\nconda activate pspa\npython train.py --config ./config/config-phenobench.yaml --export \u003cpath/to/export_dir\u003e\n```\n\n- Before you start the training you need to specify the path to the dataset in the corresponding configuration file, e.g.:\n```yaml\ndata:\n  path: \u003cfull/path/to/download/PhenoBench\u003e\n```\nwhere ```\u003cfull/path/to/download/PhenoBench\u003e``` matches the path you specified to download the PhenoBench dataset (i.e. without the \u003c.../test\u003e at the very end).\n\nIn case you face ```CUDA out of memory``` you may change the batch size during training via the configuration file (```config/config-phenobench.yaml```), e.g.:\n```yaml\ntrain:\n  batch_size: 2\n``````\n\n# License\nThis software is released under a creative commons license which allows for personal and research use only. For a commercial license please contact the authors. You can view a license summary [here](https://creativecommons.org/licenses/by-nc/4.0/).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprbonn%2Fpspa","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprbonn%2Fpspa","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprbonn%2Fpspa/lists"}