{"id":14959004,"url":"https://github.com/stevefielding/tensorflow-anpr","last_synced_at":"2025-05-07T20:12:29.933Z","repository":{"id":47899197,"uuid":"130713491","full_name":"stevefielding/tensorflow-anpr","owner":"stevefielding","description":"Automatic Number (License) Plate Recognition using Tensorflow Object Detection API","archived":false,"fork":false,"pushed_at":"2019-10-29T21:13:46.000Z","size":3613,"stargazers_count":217,"open_issues_count":9,"forks_count":80,"subscribers_count":16,"default_branch":"master","last_synced_at":"2025-05-07T20:12:23.308Z","etag":null,"topics":["anpr","computer-vision","faster-rcnn","mturk-scripts","object-detection","object-in-object","python","ssd-inceptionv2","tensorflow","tensorflow-models","tensorflow-object-detection-api","tfod-api"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stevefielding.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-04-23T14:55:44.000Z","updated_at":"2025-04-05T13:30:13.000Z","dependencies_parsed_at":"2022-08-12T14:00:46.967Z","dependency_job_id":null,"html_url":"https://github.com/stevefielding/tensorflow-anpr","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stevefielding%2Ftensorflow-anpr","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stevefielding%2Ftensorflow-anpr/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stevefielding%2Ftensorflow-anpr/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stevefielding%2Ftensorflow-anpr/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stevefielding","download_url":"https://codeload.github.com/stevefielding/tensorflow-anpr/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252949285,"owners_count":21830153,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anpr","computer-vision","faster-rcnn","mturk-scripts","object-detection","object-in-object","python","ssd-inceptionv2","tensorflow","tensorflow-models","tensorflow-object-detection-api","tfod-api"],"created_at":"2024-09-24T13:18:40.808Z","updated_at":"2025-05-07T20:12:29.906Z","avatar_url":"https://github.com/stevefielding.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Automatic Number (License) Plate Recognition\n============================================\n![](https://github.com/stevefielding/tensorflow-anpr/raw/master/uploads/parkingLotShortClip.gif)  \nDetect vehicle license plates in videos and images using the tensorflow/object_detection API.  \nTrain object detection models for license plate detection using TFOD API, with either a single detection stage\nor a double detection stage.  \nThe single stage detector, detects plates and plate characters in a single inference stage.  \nThe double stage detector detects plates in the first inference stage,  \n![](https://github.com/stevefielding/tensorflow-anpr/raw/master/uploads/detect_plate.png)  \ncrops the detected plate from the image, passes the cropped plate image to the second inference stage, \nwhich detects plate characters.  \n![](https://github.com/stevefielding/tensorflow-anpr/raw/master/uploads/detect_chars.png)   \nThe double stage detector uses a single detection model that has been trained to detect plates in full images containing cars/plates, \nand trained to detect plate text in images containing tightly cropped plate images.  \n##### TF Record files, pre-trained models, and config files\nDownload [TF records, models, and config](https://drive.google.com/file/d/1fAafi6V6vtiqAirYNOQJmc6G7FLb4eC6/view?usp=sharing)  \nI have lumped everything together, which is not great for maintainability, but at least you won't have to figure out\na directory structure and copy the files to the correct locations.  \nYou will need a starting point for training. You can either use\nthe exported_models (which speeds up training) or you can download ssd_inception_v2_coco_2018_01_28, \nand faster_rcnn_resnet101_coco_2018_01_28 from the [zoo](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md)  \nThe models should be copied to experiment_ssd/YYYY_MM_DD/training, and experiment_faster_rcnn/YYYY_MM_DD/training  \nIn the config files, you will need to modify the paths:\n````\ntrain_config\n  fine_tune_checkpoint path/to/my/model.ckpt\ntrain_input_reader\n  tf_record_input_reader\n    input_path path/to/my/train.record\n  label_map_path path/to/my/classes.pbtxt\ntrain_eval_reader\n  tf_record_input_reader\n    input_path path/to/my/test.record\n  label_map_path path/to/my/classes.pbtxt\n````\n##### Performance\nSingle stage Faster RCNN:  \n[INFO] Processed 69 frames in 37.62 seconds. Frame rate: 1.83 Hz  \n[INFO] platesWithCharCorrect_recall: 97.1%, platesWithCharCorrect_precision: 97.1%,  \n       plateFrames_recall: 100.0%, plateFrames_precision: 100.0%,  \n       chars_recall: 99.6%, chars_precision: 99.6%  \nTwo stage SSD:  \n[INFO] Processed 69 frames in 6.17 seconds. Frame rate: 11.19 Hz  \n[INFO] platesWithCharCorrect_recall: 95.7%, platesWithCharCorrect_precision: 97.1%,  \n       plateFrames_recall: 98.6%, plateFrames_precision: 100.0%,  \n       chars_recall: 98.1%, chars_precision: 99.6%  \nIt seems that Faster_RCNN is slightly better than SSD, but the sample size is too small to be sure.  \n\n##### Object in object\nThis two stage technique of using a single model to detect characters within plates could also be used to detect any \nobject within another object. The two stage detector can perform inference faster than the single stage\ndetector, because it can use simpler models such as SSD, rather than Faster RCNN, and it can use a smaller\nimage size. For example, two stages of SSD based on Inception_V2, and 1280x960 image size can perform inference at \n1.8 fps on a Titan-X GPU, whereas, a single stage of Faster RCNN based on Resnet_101, and a resized 300x300 image \ncan perform inference at 15 fps on a Titan-X.  \nThe two stage SSD implementation opens the possibility of running on less powerful hardware, such as Intel \ni7-4790K CPU @ 4.00GHz with 16GB of RAM (2.9 fps), and Nvidia Jetson TX2 (2.8 fps).\n\nGenerating labelled images\n--------------------------\n##### mturk.html:\nDefines the web interface that will be used by the MTurk workers to label the images.\nModified from [original](https://github.com/kyamagu/bbox-annotator)\nUse this html/js code with Amazon mechanical Turk.\nInstructions [here](https://blog.mturk.com/tutorial-annotating-images-with-bounding-boxes-using-amazon-mechanical-turk-42ab71e5068a).  \nThis code needs to be improved to allow image zoom. Without zoom capability it is too difficult for the workers \nto create the boxes for the small characters of the license plate text. \nConsequently you can spend a lot of time re-arranging the boxes in the labelimg utility.\n\n##### genImageListForAWS.py\nUse this module to generate a csv file that can be uploaded to MTurk. You will need the csv file when you\npublish a batch of images for processing\n\n##### inspectHITs.py:\nOnce the batch has been completed by the workers you will need to download the results in csv file format,\nand approve or reject each HIT. This application will read the HIT results and overlay the bounding boxes\nand labels onto the images. A text box is provided for accepting or rejecting each HIT. Once complete, your\naccept/reject response will be added to the downloaded csv file, and the new csv file can be uploaded to MTurk\n \n##### csvToPascalXml.py:\nReads the csv file generated by inspectHITs.py, and generates PASCAL VOC style xml annotation files. \nOne xml file for each image.\n\n##### labelImg\nOnce the annotations are in Pascal VOC style xml, you can use [labelImg](https://tzutalin.github.io/labelImg/) \nto fix any mistakes in the labelled images.  \nMake sure that you take at least one pass through your dataset, and for each image, click on verify image.\n\n##### gen_plates.py\nCreate an articial set of images containing random plates, along with corresponding PASCAL VOC xml annotation files.\nYou will need the [SUN database](vision.princeton.edu/projects/2010/SUN/SUN397.tar.gz) in order to generate the background images.\nSee [Mat Earl ANPR](https://github.com/matthewearl/deep-anpr).  \nAfter generating the images, I manually edited the images, using Gimp, to remove text in the background. \nNot sure that this is a good idea, but the rationale was that I did not\nwant to train the network to learn that the text in the background was not license plate text.\nI feel that this makes the classification process harder to train, as it has to be more discriminative.\nAnd if background text is detected during inference, this is usually OK, because it will be discarded \nif it is not within a plate bounding box.\n````\npython gen_plates.py --numImages 1000 --imagePath artificial_images/CA --xmlPath artificial_images/CA_ann\n````\n\n##### build_tf_records.py:\nReads a group of PASCAL VOC style xml annotation files, and combines with associated images \nto build a TFrecord dataset. Requires a predefined label map file that maps labels to integers.  \nWill only use images where the corresponding annotation file has the verified field set to 'yes'.  \nlabel_split defines the top level label.  \nFor example if label_split == 'plate', the original image is split into\na full image and a cropped image of the license plate, and the original annotation is split into a plate annotation,\nand a character annotation. The images are padded to make them square, and the full image is scaled down to avoid\nproblems with excessive memory usage during training.   \nIf label_split == 'none', then all the images will be full size originals, and the annotations will contain labels\nfor every object.\n````\nExample usage, with plate char split:\n    python build_tf_records.py \\\n    --record_dir=datasets/records \\\n    --annotations_dir=images \\\n    --label_map_file=datasets/records/classes.pbtxt \\\n    --view_mode=False \\\n    --image_scale_factor=0.4 \\\n    --test_record_file=testing_plate_char_split.record \\\n    --train_record_file=training_plate_char_split.record\n    --split_label='plate'\n    \nExample usage, with no split:\n    python build_tf_records.py \\\n    --record_dir=datasets/records \\\n    --annotations_dir=images \\\n    --label_map_file=datasets/records/classes.pbtxt \\\n    --view_mode=False \\\n    --test_record_file=testing_plate_char_combined.record \\\n    --train_record_file=training_plate_char_combined.record \\\n    --split_label=none\n````\n##### Directory layout\nIt is important to spend some time figuring out the best directory layout.   \nImages and annotations are grouped by camera and date of image capture\nAnnotations are stored in a directory with the same name as the image directory, but suffixed with '_ann'\nIf you are extracting images from video clips, then these can be stored in the video directory.\n````\nimages  \n└── SJ7STAR  \n    ├── images  \n    │   ├── 2018_02_24_11-00  \n    │   ├── 2018_02_24_11-00_ann  \n    │   ├── 2018_02_24_15-00  \n    │   ├── 2018_02_24_9-00  \n    │   ├── 2018_02_24_9-00_ann  \n    │   ├── 2018_02_26  \n    │   └── 2018_02_27  \n    └── video\n        └── 2018_02_27\n````\nDatasets are grouped by experiment name and date of run.\nThere is a separate directory, records, which contains the tfrecord files, and the classes.pbtxt   \nYou will need to create the evaluation, exported_model and training directories. \nfaster_rcnn_resnet101_coco_2018_01_28 is created when you unzip the pre-trained base model.\nThe other sub-directories are created when you run object_detection/train.py  \n'tensorflow_object_detection_datasets' is linked to 'tensorflow/models/research/object_detection/anpr'\n````\ntensorflow_object_detection_datasets  \n├── experiment_faster_rcnn  \n│   ├── 2018_05_28  \n│   │   ├── evaluation  \n│   │   ├── exported_model  \n│   │   │   └── saved_model  \n│   │   │       └── variables  \n│   │   └── training  \n│   │       └── faster_rcnn_resnet101_coco_2018_01_28  \n│   │           └── saved_model  \n│   │               └── variables  \n│   └── 2018_06_01  \n├── experiment_ssd  \n└── records\n````\n\nTrain the object_detection model\n--------------------------------\nNow you can use the TFOD API, at tensorflow/models/research/object_detection, to train the model.\nIt goes something like this. Assuming python virtualenv called tensorflow, \na single GPU for training and CPU for eval:\n\n##### Training\n````\nworkon tensoflow  \ncd tensorflow/models/research/object_detection\npython train.py --logtostderr \\  \n--pipeline_config_path ../anpr/experiment_faster_rcnn/2018_06_12/training/faster_rcnn_anpr.config \\  \n--train_dir ../anpr/experiment_faster_rcnn/2018_06_12/training\n````\n##### Eval\nIf you are running the eval on a CPU, then limit the number of images to evaluate by modifying your config file:  \n````\n130 eval_config: {  \n131 num_examples: 5  \n````\nNew terminal \n```` \ncd tensorflow/models/research/object_detection\nworkon tensoflow  \nexport CUDA_VISIBLE_DEVICES=\"\"  \npython eval.py --logtostderr \\  \n--checkpoint_dir ../anpr/experiment_faster_rcnn/2018_06_12/training \\  \n--pipeline_config_path ../anpr/experiment_faster_rcnn/2018_06_12/training/faster_rcnn_anpr.config \\  \n--eval_dir ../anpr/experiment_faster_rcnn/2018_06_12/evaluation\n````\nNew terminal\n````\ncd tensorflow/models/research  \nworkon tensoflow  \ntensorboard --logdir anpr/experiment_faster_rcnn  \n````\n##### Export model\n````\ncd tensorflow/models/research/object_detection\nworkon tensorflow  \npython export_inference_graph.py --input_type image_tensor \\  \n--pipeline_config_path ../anpr/experiment_faster_rcnn/2018_06_12/training/faster_rcnn_anpr.config \\  \n--trained_checkpoint_prefix ../anpr/experiment_faster_rcnn/2018_06_12/training/model.ckpt-60296 \\  \n--output_directory ../anpr/experiment_faster_rcnn/2018_06_12/exported_model\n````\nTesting the trained model\n-------------------------\n##### predict_images.py\nBack to this project directory to run predict_images.py\nTest your exported model against an image dataset. Works with single and double stage prediction.\nPrints the detected plate text, and displays the annotated image.  \npred_stages = 1\n````\nworkon tensorflow\npython predict_images.py --model datasets/experiment_faster_rcnn/2018_07_25_14-00/exported_model/frozen_inference_graph.pb \\\n --pred_stages 1 \\\n --labels datasets/records/classes.pbtxt \\\n --imagePath images/SJ7STAR_images/2018_02_24_9-00 \\\n --num-classes 37 \\\n --image_display True \n````\npred_stages = 2\n````\npython predict_images.py --model datasets/experiment_ssd/2018_07_25_14-00/exported_model/frozen_inference_graph.pb \\\n --pred_stages 2 \\\n --labels datasets/records/classes.pbtxt \\\n --imagePath images/SJ7STAR_images/2018_02_24_9-00 \\\n --num-classes 37 \\\n --image_display True \n````\n##### predict_video.py\nTest your exported model against a video dataset. Uses one or two two stage prediction.\nOutputs an annotated video and a series of still images along with image annotations. The still images are grouped to reduce \nthe output of images with duplicate plates.\nThe image annotations can be viewed and edited using labelImg, and they can be used to further enlarge the training dataset.\n````\npython predict_video.py --conf conf/lplates_smallset_ssd.json\n````\n##### predict_images_and_score.py\nTest a trained model against an annotated dataset. Annotations must be in PASCAL VOC style xml files.\nRun with image_display true if you wish to see each annotated image displayed.\nCan be executed with single or double prediction stages.  \npred_stages = 1:  \nplates and characters are predicted in a single pass\n````\npython predict_images_and_score.py --model datasets/experiment_faster_rcnn/2018_07_15/exported_model/frozen_inference_graph.pb \\\n--labels datasets/records/classes.pbtxt \\\n--annotations_dir images_verification \\\n--num-classes 37 \\\n--min-confidence 0.5 \\\n--pred_stages 1\n````\npred_stages = 2:  \nThe first prediction stage predicts plates, crops the predicted plate from the image, and then \nthe cropped plate image is used as input to the second prediction stage, which predicts characters. \n````\npython predict_and_score.py --model datasets/experiment_ssd/2018_07_25_14-00/exported_model/frozen_inference_graph.pb \\\n--labels datasets/records/classes.pbtxt \\\n--annotations_dir images_verification \\\n--num-classes 37 \\\n--min-confidence 0.1 \\\n--pred_stages 2\nSingle stage faster R-CNN usage\n````\nYour results should look something like this:\n````\n[INFO] Processed 925 frames in 70.13 seconds. Frame rate: 13.19 Hz\n[INFO] platesWithCharCorrect_recall: 93.2%, platesWithCharCorrect_precision: 93.9%, \n       plateFrames_recall: 99.2%, plateFrames_precision: 100.0%, \n       chars_recall: 98.3%, chars_precision: 99.2%\n[INFO] Definitions. Precision: Percentage of all the objects detected that are correct. Recall: Percentage of ground truth objects that are detected\n[INFO] Processed 925 xml annotation files\n````\nAnd an explanation of the performance metrics:\n\n````\nplatesWithCharCorrect_recall - Plates detected in correct location and containing correct characters in the correct locations\n                               divided by the number of ground truth plates\nplatesWithCharCorrect_precision - Plates detected in correct location and containing correct characters in the correct locations\n                                  divided by the total number of plates predicted (ie true pos plus false pos)\nplateFrames_recall - Plate frames in correct location (no checking of characters) divided by the\n                     number of ground truth plates\nplateFrames_precision - Plate frames in correct location (no checking of characters) divided by the\n                        the total number of plates predicted (ie true pos plus false pos)\nchars_recall - Characters detected in the correct place with the correct contents\n               divided by the number of ground truth characters\nchars_precision - Chars detected outside of the correct location, or the location is correct,\n                  but the contents are wrong. Divided by the total number of plates predicted (ie true pos plus false pos)\n ````\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstevefielding%2Ftensorflow-anpr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstevefielding%2Ftensorflow-anpr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstevefielding%2Ftensorflow-anpr/lists"}