{"id":29360177,"url":"https://github.com/prbonn/weedsareweird","last_synced_at":"2026-03-05T10:03:04.310Z","repository":{"id":303224482,"uuid":"978819235","full_name":"PRBonn/WeedsAreWeird","owner":"PRBonn","description":null,"archived":false,"fork":false,"pushed_at":"2025-12-10T10:28:57.000Z","size":7860,"stargazers_count":22,"open_issues_count":1,"forks_count":1,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-12-10T13:52:19.602Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PRBonn.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-06T14:55:51.000Z","updated_at":"2025-12-10T10:29:01.000Z","dependencies_parsed_at":"2025-07-06T13:25:05.582Z","dependency_job_id":"05cc1de5-d5ed-4665-a69e-8553caec2858","html_url":"https://github.com/PRBonn/WeedsAreWeird","commit_stats":null,"previous_names":["prbonn/weedsareweird"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/PRBonn/WeedsAreWeird","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRBonn%2FWeedsAreWeird","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRBonn%2FWeedsAreWeird/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRBonn%2FWeedsAreWeird/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRBonn%2FWeedsAreWeird/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PRBonn","download_url":"https://codeload.github.com/PRBonn/WeedsAreWeird/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRBonn%2FWeedsAreWeird/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30118932,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-05T09:35:22.236Z","status":"ssl_error","status_checked_at":"2026-03-05T09:35:20.028Z","response_time":93,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-07-09T07:10:44.756Z","updated_at":"2026-03-05T10:03:04.295Z","avatar_url":"https://github.com/PRBonn.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Weeds are Weird (WaW)\n\nZero-Shot Semantic Segmentation for Robots in Agriculture (IROS 2025)   \n[Paper](https://www.ipb.uni-bonn.de/pdfs/chong2025iros.pdf)     \n[Demo (vegetation mask with SAM only)](https://huggingface.co/spaces/linnchong/veg_mask_SAM)   \n![Motivation](assets/motivation.png)\n\nOur approach can segment crop plants and weeds without labels. \nWe leverage foundation models [SAM](https://github.com/facebookresearch/segment-anything) and the [ViT from BioCLIP](https://imageomics.github.io/bioclip/) to build a bag of features representing crop plants. \nDuring inference, we extract plant features and compare them with the bag of features.\nPlant features with low similarity with the bag of features are inferred as weeds.\n\n![Results](assets/qual_results.png)\nQualitative results on different datasets. The top row shows input image, the second row shows ground truth, and the third row\nshows our performance.\n\n\n## Installation\n```bash\npip install -r requirements.txt\ncd src/ipb-loaders; pip install -U -e . \nwget -P scripts/sam/ https://dl.fbaipublicfiles.com/segment_anything/sam_vit_l_0b3195.pth\n```\nTODO: update to the correct list after all other checks\n/home/linn/venvs/waw-reprod\n\n## Usage\n### 0. Prepare data sets\nThe data set should follow the following directory structure:  \n```\n${DATASET_PARENT_DIR}  \n├── train  \n│   └── images  \n│   └── semantics  \n├── val  \n│   ├── images  \n│   └── semantics  \n└── test  \n    ├── images  \n```\nFor further details, see the data set directory structure of [PhenoBench](https://www.phenobench.org/).\n\n### 1. Curating the Bag of Features of Crop Plants\nYou can use the bag of features from the paper:\n+ [PhenoBench](https://www.ipb.uni-bonn.de/html/projects/chong2025iros/phenobench-final_crops.zip)\n+ [SB20](https://www.ipb.uni-bonn.de/html/projects/chong2025iros/sb20-final_crops.zip) \n+ [CropAndWeed-Sugar Beet](https://www.ipb.uni-bonn.de/html/projects/chong2025iros/cnw-sb-final_crops.zip)\n+ [CropAndWeed-Maize](https://www.ipb.uni-bonn.de/html/projects/chong2025iros/cnw-maize-final2_crops.zip)\n\nOR\n \nBuild your own (e.g., for a different dataset):\n1. Obtain vegetation segments for train split images with SAM\n    ```bash\n    python3 scripts/get_bb_clips.py \\\n      --input_dir \u003cpath to dataset parent dir\u003e \\\n      --output_dir \u003cpath of output dir\u003e \\\n      --split train \\\n      --vm_dir \u003coptional, path of output dir with vegetation masks\u003e \\\n      --vis_dir \u003coptional, path of output dir of visualisations\u003e \\\n      --aug_cfg \u003coptional, path to augmentation configuration file. defaults to ./cfgs/augs_clip.cfg\u003e \\\n      --points_per_side \u003coptional, number of point prompts for SAM\u003e \\\n      --is_remove_overlap\n    ```\n    This will give the patches saved as .png files. \n    Additionally, if specified, the point prompts used to prompt SAM in vis_dir \n    and the resultant vegetation masks of the input images in vm_dir.\n    \n2. Separate out the popular features using BioCLIP's ViT   \n    First, you need to create a .yaml file; see ./cfgs/vote_phenobench.yaml for an example.\n    ```bash\n    python scripts/bioclip_popularity.py \\ \n      --yaml_cfg ./cfgs/vote_phenobench.yaml \\\n      --output_dir \u003coutput dir for patches of crop plants\u003e\n    ```\n    (This might take some time, especially if the number of features is large)\n\n### 2. Inference \n\nFirst you need to get the vegetation masks on the test set:\n```bash\npython scripts/get_bb_clips.py \\\n  --input_dir \u003cpath to dataset parent dir\u003e \\\n  --output_dir \u003cpath of output dir\u003e \\\n  --split test \\\n  --vm_dir \u003cpath of output dir with vegetation masks\u003e \\\n  --vis_dir \u003coptional, path of output dir of visualisations\u003e \\\n  --points_per_side \u003coptional, number of point prompts for SAM\u003e \\\n  --aug_cfg ./cfgs/test_augs.cfg \n```\n\nThen, we can get the semantic segmentation:\n```bash\npython scripts/get_predictions_bioclip.py \\\n  --yaml_cfg ./cfgs/vote_phenobench.yaml \\\n  --crop_feats_dir \u003coutput dir for patches of crop plants\u003e \\\n  --vis_dir \u003coutput vis dir path\u003e;\npython scripts/get_cws.py \\\n  --input_dir \u003coutput vis dir path\u003e \\\n  --output_dir \u003cpredictions dir path\u003e \\\n  --vm_dir \u003cvegetation m1ask directory\u003e \\\n  --ds_dir \u003cdataset directory\u003e \\\n  --img_size \u003cimage size\u003e \\\n  --center_crop \\\n  --is_vis;\n```\n\n### 3. Evaluation\n```bash\npython scripts/evaluate.py \\\n  --semantics_dir \u003cpredictions dir path\u003e \\\n  --gt_dir \u003cground truth labels dir path\u003e \\\n  --img_size \u003cimage size in px\u003e \\\n  --center_crop;\n```\n\nNote: For PhenoBench test split evaluation, we used the CodaLab benchmark online.\n\n## Machine specs\nWe developed/tested this code on Python 3.12 and utilising a NVIDIA RTX A6000 GPU.\n\n## Cite us\n```\n@inproceedings{chong2025iros,\nauthor = {Y.L. Chong and L. Nunes and F. Magistri and X. Zhong and J. Behley and C. Stachniss},\ntitle = {{Zero-Shot Semantic Segmentation for Robots in Agriculture}},\nbooktitle = iros,\nyear = 2025,\ncodeurl = {https://github.com/PRBonn/WeedsAreWeird},\nvideurl = {https://youtu.be/1ORs07F0RsE}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprbonn%2Fweedsareweird","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprbonn%2Fweedsareweird","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprbonn%2Fweedsareweird/lists"}