{"id":13543285,"url":"https://github.com/yizt/keras-ctpn","last_synced_at":"2025-04-02T12:32:00.813Z","repository":{"id":65912365,"uuid":"175381217","full_name":"yizt/keras-ctpn","owner":"yizt","description":"keras复现场景文本检测网络CPTN: 《Detecting Text in Natural Image with Connectionist Text Proposal Network》；欢迎试用，关注，并反馈问题...","archived":false,"fork":false,"pushed_at":"2020-11-23T05:56:01.000Z","size":7153,"stargazers_count":107,"open_issues_count":19,"forks_count":38,"subscribers_count":8,"default_branch":"master","last_synced_at":"2024-11-03T10:32:18.581Z","etag":null,"topics":["ctpn","deep-learning","keras","ocr","text-detection"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yizt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-03-13T08:48:02.000Z","updated_at":"2024-04-30T03:23:42.000Z","dependencies_parsed_at":"2023-02-15T22:31:18.401Z","dependency_job_id":null,"html_url":"https://github.com/yizt/keras-ctpn","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yizt%2Fkeras-ctpn","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yizt%2Fkeras-ctpn/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yizt%2Fkeras-ctpn/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yizt%2Fkeras-ctpn/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yizt","download_url":"https://codeload.github.com/yizt/keras-ctpn/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246815777,"owners_count":20838508,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ctpn","deep-learning","keras","ocr","text-detection"],"created_at":"2024-08-01T11:00:28.668Z","updated_at":"2025-04-02T12:31:55.805Z","avatar_url":"https://github.com/yizt.png","language":"Python","funding_links":[],"categories":["Text detection and localization"],"sub_categories":["CTPN [paper:2016](https://arxiv.org/pdf/1609.03605.pdf)"],"readme":"# keras-ctpn\r\n\r\n[TOC]\r\n\r\n1. [说明](#说明)\r\n2. [预测](#预测)\r\n3. [训练](#训练)\r\n4. [例子](#例子)\u003cbr\u003e\r\n   4.1 [ICDAR2015](#ICDAR2015)\u003cbr\u003e\r\n   4.1.1 [带侧边细化](#带侧边细化)\u003cbr\u003e\r\n   4.1.2 [不带带侧边细化](#不带侧边细化)\u003cbr\u003e\r\n   4.1.3 [做数据增广-水平翻转](#做数据增广-水平翻转)\u003cbr\u003e\r\n   4.2 [ICDAR2017](#ICDAR2017)\u003cbr\u003e\r\n   4.3 [其它数据集](#其它数据集)\r\n5. [toDoList](#toDoList)\r\n6. [总结](#总结)\r\n\r\n## 说明\r\n\r\n​         本工程是keras实现的[CPTN: Detecting Text in Natural Image with Connectionist Text Proposal Network](https://arxiv.org/abs/1609.03605) . 本工程实现主要参考了[keras-faster-rcnn](https://github.com/yizt/keras-faster-rcnn) ; 并在ICDAR2015和ICDAR2017数据集上训练和测试。\r\n\r\n​         工程地址: [keras-ctpn](https://github.com/yizt/keras-ctpn)\r\n\r\n​         cptn论文翻译:[CTPN.md](https://github.com/yizt/cv-papers/blob/master/CTPN.md)\r\n\r\n**效果**：\r\n\r\n​        使用ICDAR2015的1000张图像训练在500张测试集上结果为：Recall: 37.07 % Precision: 42.94 % Hmean: 39.79 %;\r\n原文中的F值为61%；使用了额外的3000张图像训练。\r\n\r\n**关键点说明**:\r\n\r\na.骨干网络使用的是resnet50\r\n\r\nb.训练输入图像大小为720\\*720; 将图像的长边缩放到720,保持长宽比,短边padding;原文是短边600;预测时使用1024*1024\r\n\r\nc.batch_size为4, 每张图像训练128个anchor,正负样本比为1:1;\r\n\r\nd.分类、边框回归以及侧边细化的损失函数权重为1:1:1;原论文中是1:1:2\r\n\r\ne.侧边细化与边框回归选择一样的正样本anchor;原文中应该是分开选择的\r\n\r\nf.侧边细化还是有效果的(注:网上很多人说没有啥效果)\r\n\r\ng.由于有双向GRU，水平翻转会影响效果(见样例[做数据增广-水平翻转](#做数据增广-水平翻转))\r\n\r\nh.随机裁剪做数据增广，网络不收敛\r\n\r\n\r\n\r\n\r\n## 预测\r\n\r\na. 工程下载\r\n\r\n```bash\r\ngit clone https://github.com/yizt/keras-ctpn\r\n```\r\n\r\n\r\n\r\nb. 预训练模型下载\r\n\r\n​    ICDAR2015训练集上训练好的模型下载地址： [google drive](https://drive.google.com/open?id=12t-PFYvYwx4In2aRv7OgRFkHa9rCjjn7)，[百度云盘](https://pan.baidu.com/s/1GnDATacvBeFXpAwnBW6RaQ) 取码:wm47\r\n\r\nc.修改配置类config.py中如下属性\r\n\r\n```python\r\n\tWEIGHT_PATH = '/tmp/ctpn.h5'\r\n```\r\n\r\nd. 检测文本\r\n\r\n```shell\r\npython predict.py --image_path image_3.jpg\r\n```\r\n\r\n## 评估\r\n\r\na. 执行如下命令,并将输出的txt压缩为zip包\r\n```shell\r\npython evaluate.py --weight_path /tmp/ctpn.100.h5 --image_dir /opt/dataset/OCR/ICDAR_2015/test_images/ --output_dir /tmp/output_2015/\r\n```\r\n\r\nb. 提交在线评估\r\n   将压缩的zip包提交评估，评估地址:http://rrc.cvc.uab.es/?ch=4\u0026com=mymethods\u0026task=1\r\n\r\n## 训练\r\n\r\na. 训练数据下载\r\n```shell\r\n#icdar2013\r\nwget http://rrc.cvc.uab.es/downloads/Challenge2_Training_Task12_Images.zip\r\nwget http://rrc.cvc.uab.es/downloads/Challenge2_Training_Task1_GT.zip\r\nwget http://rrc.cvc.uab.es/downloads/Challenge2_Test_Task12_Images.zip\r\n```\r\n\r\n```shell\r\n#icdar2015\r\nwget http://rrc.cvc.uab.es/downloads/ch4_training_images.zip\r\nwget http://rrc.cvc.uab.es/downloads/ch4_training_localization_transcription_gt.zip\r\nwget http://rrc.cvc.uab.es/downloads/ch4_test_images.zip\r\n```\r\n\r\n```shell\r\n#icdar2017\r\nwget -c -t 0 http://datasets.cvc.uab.es/rrc/ch8_training_images_1~8.zip\r\nwget -c -t 0 http://datasets.cvc.uab.es/rrc/ch8_training_localization_transcription_gt_v2.zip\r\nwget -c -t 0 http://datasets.cvc.uab.es/rrc/ch8_test_images.zip\r\n```\r\n\r\n\r\n\r\nb. resnet50与训练模型下载\r\n\r\n```shell\r\nwget https://github.com/fchollet/deep-learning-models/releases/download/v0.2/resnet50_weights_tf_dim_ordering_tf_kernels_notop.h5\r\n```\r\n\r\n\r\n\r\nc. 修改配置类config.py中，如下属性\r\n\r\n```python\r\n\t# 预训练模型\r\n    PRE_TRAINED_WEIGHT = '/opt/pretrained_model/resnet50_weights_tf_dim_ordering_tf_kernels_notop.h5'\r\n\r\n    # 数据集路径\r\n    IMAGE_DIR = '/opt/dataset/OCR/ICDAR_2015/train_images'\r\n    IMAGE_GT_DIR = '/opt/dataset/OCR/ICDAR_2015/train_gt'\r\n```\r\n\r\nd.训练\r\n\r\n```shell\r\npython train.py --epochs 50\r\n```\r\n\r\n\r\n\r\n\r\n\r\n## 例子\r\n\r\n### ICDAR2015\r\n\r\n#### 带侧边细化\r\n\r\n![](image_examples/icdar2015/img_8.1.jpg)\r\n\r\n![](image_examples/icdar2015/img_200.1.jpg)\r\n\r\n#### 不带侧边细化\r\n![](image_examples/icdar2015/img_8.0.jpg)\r\n\r\n![](image_examples/icdar2015/img_200.0.jpg)\r\n\r\n#### 做数据增广-水平翻转\r\n![](image_examples/flip1.png)\r\n![](image_examples/flip2.png)\r\n\r\n### ICDAR2017\r\n\r\n\r\n![](image_examples/icdar2017/ts_img_01000.1.jpg)\r\n\r\n![](image_examples/icdar2017/ts_img_01001.1.jpg)\r\n\r\n### 其它数据集\r\n![](image_examples/bkgd_1_0_generated_0.1.jpg)\r\n![](image_examples/a2.png)\r\n![](image_examples/a1.png)\r\n![](image_examples/a3.png)\r\n![](image_examples/a0.png)\r\n\r\n## toDoList\r\n\r\n1. 侧边细化(已完成)\r\n2. ICDAR2017数据集训练(已完成)\r\n3. 检测文本行坐标映射到原图(已完成)\r\n4. 精度评估(已完成)\r\n5. 侧边回归,限制在边框内(已完成)\r\n6. 增加水平翻转(已完成)\r\n7. 增加随机裁剪(已完成)\r\n\r\n\r\n\r\n### 总结\r\n\r\n1. ctpn对水平文字检测效果不错\r\n2. 整个网络对于数据集很敏感;在2017上训练的模型到2015上测试效果很不好；同样2015训练的在2013上测试效果也很差\r\n3. 推测由于双向GRU，网络有存储记忆的缘故？在使用随机裁剪作数据增广时网络不收敛，使用水平翻转时预测结果也水平对称出现\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyizt%2Fkeras-ctpn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyizt%2Fkeras-ctpn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyizt%2Fkeras-ctpn/lists"}