{"id":20735141,"url":"https://github.com/belicknicko/autocvat","last_synced_at":"2025-04-23T23:41:52.455Z","repository":{"id":243835107,"uuid":"813520512","full_name":"BelickNicko/AutoCVAT","owner":"BelickNicko","description":"Use this project to automatically annotate your dataset for free in CVAT","archived":false,"fork":false,"pushed_at":"2024-09-16T09:21:25.000Z","size":12850,"stargazers_count":26,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-09-16T10:55:40.906Z","etag":null,"topics":["annotations","coco","cvat","cvat-annotations-pipeline","fastsam","ultralytics","yolo","yolo-world","yolov8"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BelickNicko.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-11T08:44:20.000Z","updated_at":"2024-09-16T09:21:25.000Z","dependencies_parsed_at":"2024-07-22T07:30:00.048Z","dependency_job_id":"cc019432-4e24-4334-a240-92f695ad492b","html_url":"https://github.com/BelickNicko/AutoCVAT","commit_stats":null,"previous_names":["belicknicko/autocvat"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BelickNicko%2FAutoCVAT","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BelickNicko%2FAutoCVAT/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BelickNicko%2FAutoCVAT/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BelickNicko%2FAutoCVAT/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BelickNicko","download_url":"https://codeload.github.com/BelickNicko/AutoCVAT/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225004730,"owners_count":17405659,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["annotations","coco","cvat","cvat-annotations-pipeline","fastsam","ultralytics","yolo","yolo-world","yolov8"],"created_at":"2024-11-17T05:34:33.396Z","updated_at":"2025-04-23T23:41:52.161Z","avatar_url":"https://github.com/BelickNicko.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# __Auto-labling CVAT__\n\nCreating __automatic annotation in CVAT__ using pre-trained [ultralytics models][1]. It supports both bounding box annotation for __object detection__ tasks and polygon annotation for __instance segmentation__ tasks.\n\nYou need to provide the path to the folder with images that will be uploaded to the task in CVAT. As a result of the program's operation, a zip archive with annotations obtained based on the neural network inference will be formed.\n\nAll that remains to be done: after creating the task, go to the Actions panel and select `Upload annotations`. Then, choose `COCO 1.0` in the `Import format` section and upload the archive.\n\n![CVAT Auto-labling example](documentation/upload_example.gif)\n\n## Installation:\nYou need to have Python 3.8 or newer installed.\nRun the following commands sequentially in your terminal:\n\n1. Clone this repository to your local machine.\n2. Navigate to the created folder using the command cd.\n\nInstall all necessary libraries:\n\n```pip install -r requirements.txt```\n\n__This repository supports all relevant models from Ultralytics:__ YOLOv8/v9/v10/11/12 (and lower) for detection and segmentation, RT-DETR, FastSAM, and YOLO-World (Real-Time Open-Vocabulary Object Detection).\n\n## How to run the code:\nTo interact with the repository, you need to set the following command in the cmd terminal:\n```\npython AutoCvat.py --img_folder=\"images\" --weights=yolo11m-seg.pt --yaml_pth=config.yaml\n```\nAn example of a more fine-tuning with all possible CLI parameters:\n```\npython AutoCvat.py --img_folder=images --weights=yolo11m-seg.pt --annotations_zip=cars_annotations --yaml_pth=config.yaml --all_conf=0.2 --cvat_json=True --save_images_to_annotations_zip=True --zero_shot_segmentation=False\n```\n\nTable 1. Explanation of CLI command values:\n\n| № | Command               | Description                                                                                   | Default value|\n|---|-----------------------|-----------------------------------------------------------------------------------------------|--------------|\n| 1 | img_folder  | Path to the folder containing images of CVAT task                                               |  -           |\n| 2 | weights     | Path to the ultralytics model weights file (ex: yolov8m-seg.pt, yolov9c.pt, FastSAM-x.pt)       |  -           |\n| 3 | yaml_pth       | The path to configuration yaml file                                                          | config.yaml |\n| 4 | annotations_zip       | Name of the COCO CVAT annotation zip archive produced by the program                  | annotations  |\n| 5 | save_images_to_annotations_zip      | Whether to add images files to .zip annotations to upload to CVAT                                     | False        |\n| 6 | cvat_json     | Should a json file with labels for CVAT be created                                            | False        |\n| 7 | all_conf    | The value of the confidence of all model classes, condidences from config file don`t use        | None         |\n| 8 | zero_shot_segmentation    | When set to True, it allows for zero-shot instance segmentation using SAM from any source detection network  | False |\n\nFor Russian users, there is a detailed video presentation of this project. YouTube video in Russian is available at this [link](https://www.youtube.com/watch?v=pyRvMj6JY_8).\n\n## Configuration file\n\nThe project includes a configuration file that allows you to customize various parameters for your YOLO model, whether it's a custom or pretrained version. Here's a clearer breakdown of the settings available in this configuration file:\n\n- CVAT Class Names Mapping (names): This parameter maps the class numbers from your model to the class names in your CVAT project, ensuring that the annotations are correctly labeled.\n\n- Confidence Scores (confs): This parameter sets the confidence thresholds for each class in the model. Higher thresholds result in fewer potential objects being detected for each class, enhancing precision.\n\n- Intersection over Union (iou): This parameter is used for Non-Maximum Suppression (NMS), helping to decide which bounding boxes to keep by setting a threshold for overlap between boxes.\n\n- Image Size (imgsz): This parameter determines the size to which input images are resized during model inference, affecting the resolution and processing speed.\n\n- Minimize Points (minimize_points): When set to True, this parameter reduces the number of points in the polygons of the final markup, which is recommended for efficiency and simplicity.\n\n- Segment Parameter (segment): This parameter specifies the type of task you want to perform:\n  - True: For instance segmentation, where the model identifies and outlines each instance of a class. Creates polygonal markup in CVAT.\n\n   - False: For object detection, where the model identifies the presence of objects without delineating their exact boundaries.Creates bounding box markup in CVAT.\n\nThis configuration file provides a flexible way to tailor the model's behavior to your specific needs, ensuring that the model's output aligns with your project requirements.\n\nBelow is an example of a YAML configuration file:\n\n```\nnames:\n  0: person\n  1: bicycle\n  2: car\n  3: motorcycle\n  14: bird\nconfs:\n  0: 0.7\n  1: 0.4\n  2: 0.4\n  3: 0.5\n  14: 0.6\niou: 0.7\nimgsz: 640\nminimize_points: False\nsegment: False\n```\nThe keys in the \"names\" are the numbering of the classes in your model, and the values are the names in the CVAT project (`predictions.boxes.cls.cpu().int().tolist()`)\nThe keys in the \"confs\" are also the numbering of the classes, and the values are the confidence parameter of each class of the model.\n\n**It is important to note that the number of confidentiality parameters must match the number of class names.**\n\nIf the \"segment\" parameter is set to True, but your model only supports detection, the output you will receive will be annotations from the detector. However, you can enable the --zero_shot_segmentation=True mode, which will allow the detection model to produce segmentation masks by using the detected bounding boxes as input prompts for a pre-trained neural network called SAM. The detailed explanation of this approach is provided in the \"How to do Zero-shot instance segmentation for automatic annotation\" section of the README.\n\n**If you solve the detection issue, you do not need to use \"minimize_points\" parameter. It only applies to the segmentation task**\n\n## Сlasses combining \nYou can also combine several classes into one by giving them the same name in the configuration class, as in the example:\n\n```\nnames:\n  0: person\n  2: car\n  5: car\n  7: car\n  14: bird\nconfs:\n  0: 0.7\n  2: 0.4\n  5: 0.4\n  7: 0.5\n  14: 0.3\niou: 0.7\nimgsz: 640\nminimize_points: True\nsegment: True\n```\nAt the exit you will get:\n![Cvat example union classes](documentation/example_class_union.jpg)\n\nIf you need to get the annotations of specific classes from YOLO, then you can see their numbers:\n[COCO classes supported by YOLO models][2] \n\n## Utilize Zero-shot object detectors for automatic annotation (YOLO-World)\n\nTo speed up your pipeline cycle of annotation, training and model improvement a good solution is to use the [YOLO-World model][3]. This model allows you to detect any object in the image based on descriptive texts.\n\nTo get started you need to install the clip library:\n\n```pip install git+https://github.com/openai/CLIP.git```\n\nAfter that, set the classes that you want to detect and save your custom model:\n```python\nfrom ultralytics import YOLO\n\n# Initialize a YOLO-World model\nmodel = YOLO(\"yolov8x-world.pt\")  # or select yolov8m/s-world.pt\n\n# Define custom classes\nmodel.set_classes([\"dinosaur\", \"piano\"])\n\n# Save the model with the defined offline vocabulary\nmodel.save(\"custom_yolo-world.pt\")\n```\nThen you can use AutoCvat to get the annotations using custom weights:\n\n![Cvat YOLO-World example](documentation/yolo_world_example.jpg)\n\n## How to do Zero-shot instance segmentation for automatic annotation\nIt is possible to transform any initial detection neural network into an instance segmentation network. The output detection boxes are fed as input prompts into the SAM (Segment Anything Model), which allows for the generation of hypothetical polygon contours of objects. As an example, this approach can be used in conjunction with the Zero-shot detector YOLO-World, enabling Zero-shot instance segmentation (similar to the Grounded SAM approach).\n\nTo activate this mode, you need to add `--zero_shot_segmentation=True` to the terminal command and specify `segment: True` in the YAML configuration file.\n\nExample of using the instance segmenter network obtained from YOLO-world:\n\n```\npython AutoCvat.py --img_folder=\"images\" --weights=custom_yolo-world.pt --yaml_pth=config.yaml --zero_shot_segmentation=True\n```\n\n![Cvat YOLO-World example](documentation/yolo_world_example_seg.jpg)\n\n## How to create lables for your CVAT project\nIf you will set cli command `--cvat_json=True`, you will get json file containing everything you need to create a project for your auto annotations.\nThe contents of the json file must be inserted in \"Raw\" in the header of the project you created in CVAT:\n\n![Cvat example](documentation/cvat_json.gif)\n\n\n[1]: https://docs.ultralytics.com/models/\n[2]: https://github.com/ultralytics/ultralytics/blob/main/ultralytics/cfg/datasets/coco.yaml\n[3]: https://docs.ultralytics.com/models/yolo-world/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbelicknicko%2Fautocvat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbelicknicko%2Fautocvat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbelicknicko%2Fautocvat/lists"}