{"id":13543014,"url":"https://github.com/Bartzi/kiss","last_synced_at":"2025-04-02T12:31:01.790Z","repository":{"id":36268476,"uuid":"222418267","full_name":"Bartzi/kiss","owner":"Bartzi","description":"Code for the paper \"KISS: Keeping it Simple for Scene Text Recognition\"","archived":false,"fork":false,"pushed_at":"2023-10-03T21:35:13.000Z","size":571,"stargazers_count":110,"open_issues_count":10,"forks_count":29,"subscribers_count":12,"default_branch":"master","last_synced_at":"2024-11-03T09:33:37.943Z","etag":null,"topics":["chainer","deep-learning","paper","scene-text-recognition","spatial-transformer-networks","transformer"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Bartzi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-11-18T10:06:46.000Z","updated_at":"2024-06-01T21:30:50.000Z","dependencies_parsed_at":"2024-01-15T23:27:00.521Z","dependency_job_id":"60f895b6-5dfb-4460-aafb-eae75b9cb9b7","html_url":"https://github.com/Bartzi/kiss","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Bartzi%2Fkiss","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Bartzi%2Fkiss/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Bartzi%2Fkiss/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Bartzi%2Fkiss/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Bartzi","download_url":"https://codeload.github.com/Bartzi/kiss/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246815372,"owners_count":20838435,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chainer","deep-learning","paper","scene-text-recognition","spatial-transformer-networks","transformer"],"created_at":"2024-08-01T11:00:21.412Z","updated_at":"2025-04-02T12:30:56.782Z","avatar_url":"https://github.com/Bartzi.png","language":"Python","funding_links":[],"categories":["Text detection and localization"],"sub_categories":["Form Segmentation"],"readme":"# KISS\nCode for the paper [KISS: Keeping it Simple for Scene Text Recognition](https://arxiv.org/abs/1911.08400).\n\nThis repository contains the code you can use in order to train a model based on our paper.\nYou will also find instructions on how to access our model and also how to evaluate the model.\n\n# Pretrained Model\n\nYou can find the pretrained model [here](https://bartzi.de/research/kiss).\nDownload the zip and put into any directory. We will refer to this\ndirectory as `\u003cmodel_dir\u003e`.\n\n# Prepare for using the Code\n\n- make sure you have at least Python **3.7** installed on your system\n- create a new virtual environment (or whatever you like to use)\n- install all requirements with `pip install -r requirements.txt`\n(if you do not have a CUDA capable device in your PC, you should remove\nthe package `cupy` from the file `requirements.txt`).\n\n# Datasets\n\nIf you want to train your model on the same datasets, as we did, you'll \nneed to get the train data first. Second, you can get the train annotation\nwe used from [here](https://bartzi.de/research/kiss).\n\n## Image Data\n\nYou can find the image data for each dataset, using the following links:\n- MJSynth: https://www.robots.ox.ac.uk/~vgg/data/text/ \n- SynthText: https://www.robots.ox.ac.uk/~vgg/data/scenetext/\n- SynthAdd: Follow instructions from [here](https://github.com/wangpengnorman/SAR-Strong-Baseline-for-Text-Recognition)\n\nOnce, you've downloaded all the images, you can get the gt-files we've prepared for\nthe MJSynth and SynthAdd datasets [here](https://bartzi.de/research/kiss).\n\nFor the SynthText dataset, you'll have to create them yourself.\nYou can do so by following these steps:\n1. Get the data and put it into a directory (lets assume we put the data into the\ndirectory `/data/oxford`)\n1. run the script `crop_words_from_oxford.py` (you can find it in `datasets/text_recognition`)\nwith the following command line parameters `python crop_words_from_oxford.py /data/oxford/gt.mat /data/oxford_words`.\n1. This will crop all words based on their axis aligned bounding box from the\noriginal oxford gt.\n1. Create train and validation split with the script `create_train_val_splits.py`.\n`python create_train_val_splits.py /data/oxford_words/gt.json`.\n1. Run the script `json_to_npz.py` with the following command line:\n`python json_to_npz /data/oxford_words/train.json ../../train_utils/char-map-bos.json`.\nThis will create a file called `train.npz` in the same directory as the file `gt.json` is currently located in.\n1. Repeat the last step with the files `validation.json`.\n\nOnce you are done with this, you'll need to combine all `npz` files into\none large `npz` file. You can use the `combine_npz_datasets.py` for this.\nAssume you saved the MJSynth dataset + npz file here `/data/mjsynth` and\nthe SynthAdd dataset + npz file here `/data/SynthAdd`, then you'll need\nto run the script in the following way: `python combine_npz_datasets.py \n/data/mjsynth/annotation_train.npz /data/oxford_words/train.npz /data/SynthAdd/gt.npz\n--destination /data/datasets_combined.npz`.\n\nSince the datasets may contain words that are longer than `N` characters (we always set `N` to 23),\nwe need to get rid of all words that are longer than `N` characters.\nYou can use the script `filter_word_length.py` for this.\nUse it like so: `python filter_word_length.py 23 /data/datasets_combined.npz --npz`.\nDo the same thing with the file `validation.npz` you obtained from splitting\nthe SynthText dataset.\n\nIf you want to follow our experiments with the balanced dataset, you can \ncreate a balanced dataset with the script `balance_dataset.py`.\nFor example: `python balance_dataset.py /data/datasets_combined_filtered_23.npz datasets_combined_balanced_23.npz -m 200000`.\nIf you do not use the `-m` switch the script will show you dataset statistics and you can choose your own value.\n\n## Evaluation Data\n\nIn this ssection we explain, hou you can get the evaluation data + annotation.\nFor getting the evaluation data you just need to do 2 steps per dataset:\n1. Clone the repository.\n1. Download the `npz` annotation file. And place it in the directory, where you cloned the git repository to.\n\n| Dataset  | Git Repo | NPZ-Link | Note |\n|---|---|---|---|\n| ICDAR2013 | https://github.com/ocr-algorithm-and-data/ICDAR2013 | [download](https://bartzi.de/documents/attachment/download?hash_value=4e1c652bf62fb2e454cb65c3d996f592_30) | Rename the directory `test` to `Challenge2_Test_Task3_Images` |\n| ICDAR2015 | https://github.com/ocr-algorithm-and-data/ICDAR2015 | [download](https://bartzi.de/documents/attachment/download?hash_value=8fdafd36ce2e1108913081fd247e54b4_31) | Rename the dir `TestSet` to `ch4_test_word_images_gt` |\n| CUTE80 | https://github.com/ocr-algorithm-and-data/CUTE80 | [download](https://bartzi.de/documents/attachment/download?hash_value=340f042991cf752d0ba6c700afa1bdb0_28) | - |\n| IIIT5K | https://github.com/ocr-algorithm-and-data/IIIT5K | [download](https://bartzi.de/documents/attachment/download?hash_value=af4799078f54a0138d43010563301d8b_32) | - |\n| SVT | https://github.com/ocr-algorithm-and-data/SVT | [download](https://bartzi.de/documents/attachment/download?hash_value=f7b99cc79d9b5bcecb1e755f4b5b1038_33) | Remove all subdirs, but the dir `test_crop`. Rename this dir to `img` |\n| SVTP | https://github.com/ocr-algorithm-and-data/SVT-Perspective | [download](https://bartzi.de/documents/attachment/download?hash_value=36238a96473f2a604f8bf4c6874de55c_34) | - |\n\n# Training\n \nNow you should be ready for training :tada:.\nYou can use the script `train_text_recognition.py`, which is in the \nroot-directory of this repo.\n\nBefore you can start your training, you'll need to adapt the config in \n`config.cfg`.\nSet the values following this list:\n- **train_file**: Set this to the file `/data/datasets_combined_filtered_23.npz`\n- **val_file**: Set this to `/data/oxford_words.validation.npz`\n- **keys in TEST_DATASETS** set those to the corresponding npz file you got [here](https://bartzi.de/research/kiss) and setup in the last step.\n\nYou can now run the training with, e.g.,\n`python train_text_recognition.py \u003cname for the log\u003e -g 0 -l tests --image-mode RGB --rdr 0.95`\nThis will start the training and create a new directlry with log entries in `logs/tests`.\nGet some coffee and sleep, because the training will take some time!\n\nYou can inspect the train progress with Tensorboard. Just start Tensorboard\nin the root directory like so: `tensorboard --logir logs`.\n\n# Evaluation\n\nOnce, you've trained a model or if you just downloaded the model we provided,\nyou can run the evaluation script on it.\n\nIf you want to know how the model performes on all datasets, you can use the \nscript `run_eval_on_all_datasets.py`. Lets assume you trained a model and \n`logs/tests/train` is the path to the log dir.\nNow, you can run the evaluation with this command: `python run_eval_on_all_datasets.py \nconfig.cfg 0 -b 16 --snapshot-dir logs/tests/train`.\nYou can also render the predictions of the model for each evaluation image\nby making the following changes to the command:\n`python run_eval_on_all_datasets.py config.cfg 0 -b 1\n--snapshot-dir logs/tests/train --render`.\nYou will then find the results for each image in the directory `logs/tests/train/eval_bboxes`.\n\n# Questions?\n\nFeel free to open an issue!\nYou want to contribute? Just open a PR :smile:!\n\n# License\n\nThis code is licensed under GPLv3, see the file `LICENSE` for more information.\n\n# Citation\n\nIf you find this code useful, please cite our paper:\n```bibtex\n@misc{bartz2019kiss,\n    title={KISS: Keeping It Simple for Scene Text Recognition},\n    author={Christian Bartz and Joseph Bethge and Haojin Yang and Christoph Meinel},\n    year={2019},\n    eprint={1911.08400},\n    archivePrefix={arXiv},\n    primaryClass={cs.CV}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FBartzi%2Fkiss","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FBartzi%2Fkiss","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FBartzi%2Fkiss/lists"}