{"id":15028051,"url":"https://github.com/spytensor/prepare_detection_dataset","last_synced_at":"2025-05-16T14:09:16.479Z","repository":{"id":37359283,"uuid":"176627295","full_name":"spytensor/prepare_detection_dataset","owner":"spytensor","description":"convert dataset to coco/voc format","archived":false,"fork":false,"pushed_at":"2021-09-01T07:55:38.000Z","size":28,"stargazers_count":1106,"open_issues_count":5,"forks_count":354,"subscribers_count":13,"default_branch":"master","last_synced_at":"2025-04-03T11:09:37.635Z","etag":null,"topics":["coco","csv","detection","labelme","voc"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/spytensor.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-03-20T01:24:54.000Z","updated_at":"2025-04-03T09:56:53.000Z","dependencies_parsed_at":"2022-07-12T12:31:20.675Z","dependency_job_id":null,"html_url":"https://github.com/spytensor/prepare_detection_dataset","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spytensor%2Fprepare_detection_dataset","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spytensor%2Fprepare_detection_dataset/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spytensor%2Fprepare_detection_dataset/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spytensor%2Fprepare_detection_dataset/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/spytensor","download_url":"https://codeload.github.com/spytensor/prepare_detection_dataset/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248564734,"owners_count":21125409,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["coco","csv","detection","labelme","voc"],"created_at":"2024-09-24T20:07:32.418Z","updated_at":"2025-04-12T11:48:49.547Z","avatar_url":"https://github.com/spytensor.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"**背景**\n\n万事开头难。之前写图像识别的博客教程，也是为了方便那些学了很多理论知识，却对实际项目无从下手的小伙伴，后来转到目标检测来了，师从烨兄、亚光兄，从他们那学了不少检测的知识和操作，今天也终于闲下了，准备写个检测系列的总结。一方面分享知识希望可以一起学习，另一方面让一部分人少走弯路，快速上路（入坑）。\n\n此部分代码：[Github](https://github.com/spytensor/prepare_detection_dataset)\n博客地址: [目标检测系列一：如何制作数据集?](http://www.spytensor.com/index.php/archives/48/)\n\n\n**更新**\n\n- (28/03/2019)\n    - 新增 `csv2labelme`\n\n\n\u003ch4 id=\"1\"\u003e1. 内容介绍\u003c/h4\u003e\n\n系列一主要介绍如何在常见的几种数据格式之间进行转换，以及万能中介`csv`格式的使用，这里列出以下几个：\n\n- csv to coco\n- csv to voc\n- labelme to coco\n- labelme to voc\n- csv to json\n\n\u003ch4 id=\"2\"\u003e2. 标准格式\u003c/h4\u003e\n\n在使用转换脚本之前，必须要明确的几种格式\n\n\u003ch5 id=\"2.1\"\u003e2.1 csv\u003c/h5\u003e\n\n不要一看是`csv`文件就直接拿来运行，如果不是，可以自行修改代码，或者修改标注文件。\n\n转换脚本支持的csv格式应为以下形式:\n\n- `csv/`\n    - `labels.csv`\n    - `images/`\n        - `image1.jpg`\n        - `image2.jpg`\n        - `...`\n\n`labels.csv` 的形式: \n\n`/path/to/image,xmin,ymin,xmax,ymax,label`\n\n例如:\n\n```\n/mfs/dataset/face/0d4c5e4f-fc3c-4d5a-906c-105.jpg,450,154,754,341,face\n/mfs/dataset/face/0ddfc5aea-fcdac-421-92dad-144.jpg,143,154,344,341,face\n...\n```\n注：图片路径请使用绝对路径\n\n\u003ch5 id=\"2.2\"\u003e2.2 voc\u003c/h5\u003e\n\n标准的voc数据格式如下：\n\n- `VOC2007/`\n    - `Annotations/`\n        - `0d4c5e4f-fc3c-4d5a-906c-105.xml`\n        - `0ddfc5aea-fcdac-421-92dad-144/xml`\n        - `...`\n    - `ImageSets/`\n        - `Main/`\n            - `train.txt`\n            - `test.txt`\n            - `val.txt`\n            - `trainval.txt`\n    - `JPEGImages/`\n        - `0d4c5e4f-fc3c-4d5a-906c-105.jpg`\n        - `0ddfc5aea-fcdac-421-92dad-144.jpg`\n        - `...`\n\n\u003ch5 id=\"2.3\"\u003e2.3 coco\u003c/h5\u003e\n\n此处未使用测试集\n\n- `coco/`\n    - `annotations/`\n        - `instances_train2017.json`\n        - `instances_val2017.json`\n    - `images/`\n        - `train2017/`\n            - `0d4c5e4f-fc3c-4d5a-906c-105.jpg`\n            - `...`\n        - `val2017`\n            - `0ddfc5aea-fcdac-421-92dad-144.jpg`\n            - `...`\n\n\u003ch5 id=\"2.4\"\u003e2.4 labelme\u003c/h5\u003e\n\n\n- `labelme/`\n    - `0d4c5e4f-fc3c-4d5a-906c-105.json`\n    - `0d4c5e4f-fc3c-4d5a-906c-105.jpg`\n    - `0ddfc5aea-fcdac-421-92dad-144.json`\n    - `0ddfc5aea-fcdac-421-92dad-144.jpg`\n\nJson file 格式:\n（imageData那一块太长了，不展示了）\n\n```json\n{\n  \"version\": \"3.6.16\",\n  \"flags\": {},\n  \"shapes\": [\n    {\n      \"label\": \"helmet\",\n      \"line_color\": null,\n      \"fill_color\": null,\n      \"points\": [\n        [\n          131,\n          269\n        ],\n        [\n          388,\n          457\n        ]\n      ],\n      \"shape_type\": \"rectangle\"\n    }\n  ],\n  \"lineColor\": [\n    0,\n    255,\n    0,\n    128\n  ],\n  \"fillColor\": [\n    255,\n    0,\n    0,\n    128\n  ],\n  \"imagePath\": \"004ffe6f-c3e2-3602-84a1-ecd5f437b113.jpg\",\n  \"imageData\": \"\"   # too long ,so not show here\n  \"imageHeight\": 1080,\n  \"imageWidth\": 1920\n}\n```\n\n\u003ch4 id=\"3\"\u003e3. 如何使用转换脚本\u003c/h4\u003e\n\n\u003ch5 id=\"3.1\"\u003e3.1 csv2coco\u003c/h5\u003e\n\n首先更改`csv2coco.py`中以下几个配置\n\n```\nclassname_to_id = {\"person\": 1}  # for your dataset classes\ncsv_file = \"labels.csv\"  # annatations file path\nimage_dir = \"images/\"    # original image path\nsaved_coco_path = \"./\"   # path to save converted coco dataset\n```\n\n然后运行 `python csv2coco.py`\n\n会自动创建文件夹并复制图片到相应位置，运行结束后得到如下：\n\n- `coco/`\n    - `annotations/`\n        - `instances_train2017.json`\n        - `instances_val2017.json`\n    - `images/`\n        - `train2017/`\n            - `0d4c5e4f-fc3c-4d5a-906c-105.jpg`\n            - `...`\n        - `val2017`\n            - `0ddfc5aea-fcdac-421-92dad-144.jpg`\n            - `...`\n\n\u003ch5 id=\"3.2\"\u003e3.2 csv2voc\u003c/h5\u003e\n\n首先更改`csv2voc.py`中以下几个配置\n\n```\ncsv_file = \"labels.csv\"\nsaved_path = \".VOC2007/\" # path to save converted voc dataset     \nimage_save_path = \"./JPEGImages/\"   # converted voc images path\nimage_raw_parh = \"images/\"          # original image path\n```\n\n然后运行 `python csv2voc.py`\n\n同样会自动创建文件夹并复制图片到相应位置，运行结束后得到如下：\n\n\n- `VOC2007/`\n    - `Annotations/`\n        - `0d4c5e4f-fc3c-4d5a-906c-105.xml`\n        - `0ddfc5aea-fcdac-421-92dad-144/xml`\n        - `...`\n    - `ImageSets/`\n        - `Main/`\n            - `train.txt`\n            - `test.txt`\n            - `val.txt`\n            - `trainval.txt`\n    - `JPEGImages/`\n        - `0d4c5e4f-fc3c-4d5a-906c-105.jpg`\n        - `0ddfc5aea-fcdac-421-92dad-144.jpg`\n        - `...`\n\n\u003ch5 id=\"3.3\"\u003e3.3 labelme2coco\u003c/h5\u003e\n\n首先更改`labelme2coco.py`中以下几个配置\n\n```\nclassname_to_id = {\"person\": 1}  # for your dataset classes\nlabelme_path = \"labelme/\"  # path for labelme dataset\nsaved_coco_path = \"./\"     # path for saved coco dataset\n```\n然后运行 `python labelme2coco.py`，生成文件形式同`csv2coco`\n\n\u003ch5 id=\"3.4\"\u003e3.4 labelme2voc\u003c/h5\u003e\n\n首先更改`labelme2voc.py`中以下几个配置\n\n```\nlabelme_path = \"labelme/\"  # path for labelme dataset\nsaved_coco_path = \"./\"     # path for saved coco dataset\n```\n然后运行 `python labelme2voc.py`，生成文件形式同`csv2voc`\n\n\u003ch5 id=\"3.5\"\u003e3.5 csv2labelme\u003c/h5\u003e\n\n首先更改`csv2labelme.py`中以下几个配置\n\n```\nimage_path = \"./images/\"  # path for images\ncsv_file = \"./\"     # path for csv annotations\n```\n然后运行 `python csv2labelme.py`，生成的`json`文件会保存在`image_path`下,切换路径过去,直接`labelme`便\n可以查看标签.\n\n\n\u003ch4 id=\"4\"\u003e4. 万能中介csv\u003c/h4\u003e\n\n从上面的转换格式中可以看出，并没有给出如何转到csv的，一是因为太过于简单，而是主流检测框架很少支持这种格式的数据输入。以下给出如何将标注信息写入`csv`\n\n```python\ninfo = [[filename0,\"xmin ymin xmax ymax label0\"],\n          filename1,\"xmin ymin xmax ymax label1\"]\ncsv_labels = open(\"csv_labels.csv\",\"w\")\nfor filename,bboxes in info:\n    bbox = bboxes.split(\" \")\n    label = bbox[-1]\n    csv_labels.write(filename+\",\"+bbox[0]+\",\"+bbox[1]+\",\"+bbox[2]+\",\"+bbox[3]+\",\"+label+\"\\n\")\ncsv_labels.close()\n```\n\n是不是非常简单。。。如果你不知道如何从原始的标签文件中读取得到标注信息，那没办法了，学学编程吧，23333\n\n### TODO\n- 1. [ ] Multiprocessing\n## 致谢\n感谢这么久以来对本项目支持的各位大佬！\n\n[![Stargazers repo roster for @spytensor/prepare_detection_dataset](https://reporoster.com/stars/spytensor/prepare_detection_dataset)](https://github.com/spytensor/prepare_detection_dataset/stargazers)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspytensor%2Fprepare_detection_dataset","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fspytensor%2Fprepare_detection_dataset","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspytensor%2Fprepare_detection_dataset/lists"}