{"id":18738945,"url":"https://github.com/waterbang/object-deteation-train","last_synced_at":"2025-04-12T19:53:34.258Z","repository":{"id":109932532,"uuid":"295134981","full_name":"waterbang/object-deteation-train","owner":"waterbang","description":"Object Detection Model Training","archived":false,"fork":false,"pushed_at":"2022-04-25T15:49:48.000Z","size":31635,"stargazers_count":5,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-12T19:53:17.570Z","etag":null,"topics":["object-detection","tensorflow"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/waterbang.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-09-13T11:15:23.000Z","updated_at":"2024-10-14T03:16:11.000Z","dependencies_parsed_at":"2023-03-15T00:47:24.445Z","dependency_job_id":null,"html_url":"https://github.com/waterbang/object-deteation-train","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/waterbang%2Fobject-deteation-train","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/waterbang%2Fobject-deteation-train/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/waterbang%2Fobject-deteation-train/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/waterbang%2Fobject-deteation-train/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/waterbang","download_url":"https://codeload.github.com/waterbang/object-deteation-train/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248625510,"owners_count":21135513,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["object-detection","tensorflow"],"created_at":"2024-11-07T15:31:44.613Z","updated_at":"2025-04-12T19:53:34.253Z","avatar_url":"https://github.com/waterbang.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# object-deteation-train\n对象检测模型训练,此仓库已经集成了cocoAPI，Object deteationAPI等环境，让您开箱即用。\n\n\u003e  建议在本地使用此镜像，因为模型训练将耗费大量内存，会出现资源耗尽。(It is recommended to use this mirror on the server, because the model training will consume a lot of time.)\n\n## dircetory\n-   .\n-   ├── Python ---\u003e 存放xml转csv，csv转TFRcords 的python脚本\n-   ├── annotations ---\u003e存放转换的TFRcords\n-   ├── cocoapi ---\u003e cocoapi\n-   ├── exported-models ---\u003e 导出的模型准备放这里\n-   ├── images ---\u003e 标记的图片train and test\n        |—— train\n        |—— test \n-   ├── model ---\u003e 训练模型的目录\n-   ├── models ---\u003e tensorflow Object deteation API\n-   └── pre-trained-models ---\u003e tensorflow Object deteation model\n\n## 开始 (start)\n\n前置条件： labelImg 和 docker。\n\n\n### 标记对象 (Tag object)\n使用[labelImg](https://github.com/tzutalin/labelImg), 标注对象，并保存xml.类似于：\n![labelImg](http://qiniu-waterbang.waterbang.top/object-deteation.png)\n\n### 准备映射训练集 (Prepare the mapping training set)\n收集完，将其放在任意目录下，训练集和测试集都放。比例自己决定。此项目提供了两种构建方法，推荐docker hub，因为它可以让您在服务器上训练，解放您的本地资源。\n\n\u003e K折交叉验证法，留出法， 留一法\n\n## pull images\n\n### docker hub pull\n拉镜像\n```\ndocker pull waterbang/object-deteation\n```\n\n\u003e ⚠️ 整个训练镜像为6G\n\n## 运行容器 (Run the container)\n\n使用本地目录映射替到容器内images目录，以便于对数据集进行操作。\n\u003e注意：以下目录对应您的训练集和测试集目录，请修改成您的数据存放地址。\n\u003e/Users/waterbang/Desktop/tensorflow/dog/data/images\n\n```\ndocker run -it --name object-deteation -v /root/tensorflow/images:/env/images waterbang/object-deteation:latest bash\n```\n显示如下：\n![tensorflow](http://qiniu-waterbang.waterbang.top/tensorflow-cmd.png)\n\n\u003e 如果您第二次进入运行：\n\u003edocker exec -it object-deteation bash\n\n## 构建 TFRcords (如果您已经有TFRcords跳过此步)\n\n\u003e 您可以运行以下命令来安装vim工具\n\u003e apt-get update\n\u003e apt-get install vim\n\n进入 /Python 目录\n### xml transform csv\n先打开，`xml_to_csv.py`，修改 xml文件夹地址 和 生成csv文件地址。\n记得训练集和测试集都需要转换。\n在 Python 目录下运行。\n``` shell\npython ./xml_to_csv.py \n```\n\n如果成功会输出如下内容：\n\n```shell\n..............\nvalue:  ('0288001.png', 60, 160, 'person', 1, 1, 60, 160)\nvalue:  ('0875004.png', 60, 160, 'person', 1, 1, 60, 160)\nvalue:  ('0388001.png', 60, 160, 'person', 1, 1, 60, 160)\nSuccessfully converted xml to csv.\n\n```\n\n### csv transform TFRcords\n\n#### 进入 `csv_to_TFRcords.py`文件，修改以下两点：\n1.  修改标签对应的种类数字\n2.  修改文件第110行，填入数据集地址\n\n```shell\nvim /env/Python/csv_to_TFRcords.py\n```\n\n#### 在 Python 目录下运行.(使用绝对路径)\n\n``` shell\npython csv_to_TFRcords.py --csv_input=/env/images/train/train.csv   --output_path=/env/annotations/train.record\n\npython csv_to_TFRcords.py --csv_input=/env/images/test/test.csv   --output_path=/env/annotations/test.record\n```\n\n显示如下内容为成功：\n```\n2020-11-06 03:40:26.553550: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1\nSuccessfully created the TFRecords: /env/annotations/train.record\n```\n\n## Training model\n\n### 先修改您的对象种类 (Edit the target first)\n```\nvim /env/annotations/label_map.pbtxt\n```\n\n### 修改`/env/pipeline.config`\n修改以下行：\n\n#### 1. 第3行,当前种类数量\n```\nnum_classes: 4\n```\n\n#### 2. 第131行，根据您第内存，增加货减少该值\n```\nbatch_size: 8\n```\n\n#### 3. 第161行，预训练模型地址\n```\n fine_tune_checkpoint: \"pre-trained-models/ssd_resnet50_v1_fpn/checkpoint/ckpt-0\"\n```\n\n#### 4. 第168行，如果您没有在TPU上进行培训，则将此设置为false。\n```\n use_bfloat16: false \n```\n\n#### 5. 测试集和训练集地址\n```\n.....\ntrain_input_reader {\n  label_map_path: \"annotations/label_map.pbtxt\" # Path to label map file\n  tf_record_input_reader {\n    input_path: \"annotations/train.record\" # Path to training TFRecord file\n  }\n}\neval_config {\n  metrics_set: \"coco_detection_metrics\"\n  use_moving_averages: false\n}\neval_input_reader {\n  label_map_path: \"annotations/label_map.pbtxt\" # Path to label map file\n  shuffle: false\n  num_epochs: 1\n  tf_record_input_reader {\n    input_path: \"annotations/test.record\" # Path to testing TFRecord\n  }\n}\n```\n\n### Training model\n\n在 `model_main_tf2.py`同级目录下运行（/env）：\n\n```\npython model_main_tf2.py --model_dir=./pre-trained-models/ssd_resnet50_v1_fpn --pipeline_config_path=./model/my_ssd_resnet50_v1_fpn/pipeline.config\n\n```\n\n## 辅助脚本 (Auxiliary script)\n帮您将相同大小的数据转换成xml：\n`/env/node/png_to_xml.js`\n\n批量移动文件，方便分割测试集和训练集：\n`/env/node/more_test.sh`\n\n删除一个文件夹下所有的xml文件\n`/env/Python/delete_xml.py`\n\n## 如果遇到了错误\n1.  请检查脚本文件路径。\n\n### 2.如果出现 Illegal instruction (core dumped)\n那么可能您的cpu较老，不支持AVX指令。您可以运行以下命令确认，是否有输出` -mavx -mavx2` ，如果缺少则可以确认缺少AVX支持。\n```\ngrep flags -m1 /proc/cpuinfo | cut -d \":\" -f 2 | tr '[:upper:]' '[:lower:]' | { read FLAGS; OPT=\"-march=native\"; for flag in $FLAGS; do case \"$flag\" in \"sse4_1\" | \"sse4_2\" | \"ssse3\" | \"fma\" | \"cx16\" | \"popcnt\" | \"avx\" | \"avx2\") OPT+=\" -m$flag\";; esac; done; MODOPT=${OPT//_/\\.}; echo \"$MODOPT\"; }\n``` \n\n3.  使用python3。\n\n\n### express heartfelt thanks\n\n1.  https://tensorflow-object-detection-api-tutorial.readthedocs.io/en/latest/install.html#try-out-the-examples\n\n2.  https://github.com/tzutalin/labelImg\n\n3.  https://www.tensorflow.org/\n\n4.  https://gist.github.com/olivoil/a2e0e4f3427db8b6ef4a6374f9c4cb32","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwaterbang%2Fobject-deteation-train","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwaterbang%2Fobject-deteation-train","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwaterbang%2Fobject-deteation-train/lists"}