{"id":17069237,"url":"https://github.com/mogeko/opencv-ocr","last_synced_at":"2025-10-23T17:32:15.034Z","repository":{"id":157081890,"uuid":"616540355","full_name":"mogeko/opencv-ocr","owner":"mogeko","description":"Implement OCR based on OpenCV (opencv-python).","archived":false,"fork":false,"pushed_at":"2023-10-04T01:33:12.000Z","size":2793,"stargazers_count":3,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-28T17:16:07.485Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mogeko.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-20T15:30:42.000Z","updated_at":"2024-02-25T16:59:04.000Z","dependencies_parsed_at":null,"dependency_job_id":"f17e2384-a13c-4535-bd43-4cae49e9b953","html_url":"https://github.com/mogeko/opencv-ocr","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mogeko%2Fopencv-ocr","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mogeko%2Fopencv-ocr/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mogeko%2Fopencv-ocr/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mogeko%2Fopencv-ocr/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mogeko","download_url":"https://codeload.github.com/mogeko/opencv-ocr/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245090895,"owners_count":20559296,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-14T11:16:38.374Z","updated_at":"2025-10-23T17:32:14.918Z","avatar_url":"https://github.com/mogeko.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# opencv-ocr\n\n![result](https://user-images.githubusercontent.com/26341224/235641560-74889377-f5e1-41ae-841f-5b44072e0f71.jpg)\n\nImplement OCR based on OpenCV ([opencv-python](https://pypi.org/project/opencv-python)).\n\n## General idea\n\nTo implement OCR (Optical character recognition) with OpenCV, we will follow these general steps:\n\n1. **Preprocess the image**: OCR requires a clear, bright, and noise-free image, so the first step is to preprocess the image, such as removing noise, smoothing, enhancing contrast, binarizing, and so on.\n2. **Text localization**: OCR needs to recognize the text, which must be localized first. We can use [edge detection algorithms](https://docs.opencv.org/3.4/da/d22/tutorial_py_canny.html) and [morphological operations](https://docs.opencv.org/4.x/d9/d61/tutorial_py_morphological_ops.html) provided by OpenCV, such as erosion and dilation, to detect and segment text regions.\n3. **Character segmentation**: Our OCR task is to recognize individual characters rather than whole words, so that we need to use character segmentation algorithms to further segment the text regions into individual characters.\n4. **Feature extraction**: Once our text regions or characters are segmented, we need to extract their features for recognition. we can use [feature extraction algorithms](https://docs.opencv.org/3.4/db/d27/tutorial_py_table_of_contents_feature2d.html) provided by OpenCV, such as [SIFT](https://docs.opencv.org/3.4/da/df5/tutorial_py_sift_intro.html), [SURF](https://docs.opencv.org/3.4/df/dd2/tutorial_py_surf_intro.html), or [ORB](https://docs.opencv.org/3.4/d1/d89/tutorial_py_orb.html), etc.\n5. **Train the model**: Once we have prepared the feature data, we can start training the model. We can use various machine learning algorithms, such as [Support Vector Machine (SVM)](https://docs.opencv.org/3.4/d1/d73/tutorial_introduction_to_svm.html), [Neural Networks](https://docs.opencv.org/4.x/d2/d58/tutorial_table_of_content_dnn.html), [Random Forest](https://docs.opencv.org/3.4/d0/d65/classcv_1_1ml_1_1RTrees.html), etc.\n6. **Recognize text**: When our model is trained, we can use it to recognize text. We can use the model to predict the text in the image and return the results to the user.\n\nFlowchart:\n\n```mermaid\nflowchart LR\n    g[Deep Learning] -.-\u003e e\n    a[Original] --\u003e b[Pre-processed] --\u003e c[\"Chars (Image)\"] --\u003e e[\"Chars (String)\"] --\u003e f[Result]\n    b --\u003e d[Location] ---\u003e f\n    a -----\u003e f\n```\n\n## Implementation principle\n\nNext, I will explain the specific steps for us to implement this OCR program with examples.\n\nBefore we start, this is the example picture we used:\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://user-images.githubusercontent.com/26341224/235652932-d20549d2-7cbf-4028-bcf0-32e5b93f2447.jpg\"\n       alt=\"original\" width=\"80%\"\u003e\n\u003c/p\u003e\n\n### Preprocess the image\n\nImage preprocessing is a key step to achieve an efficient and accurate OCR program. This step is mainly to enhance image **quality** and **optimize the next steps**.\n\nWe use operations such as **denoising**, **smoothing**, **enhanced** **contrast** and binaryization to better identify text. Clear, bright and noise-free images help to improve the accuracy of the OCR system. In addition, image preprocessing can also adapt the OCR program to different types of images. In real life, the image may be affected by lighting, camera quality, angle and other factors, resulting in poor image quality. Preprocessing helps to solve these problems, so that the OCR program can work normally under different conditions. **The pre-processed image has better quality and clarity, which helps to improve the effect of next steps, such as text positioning, character segmentation, feature extraction, etc.**\n\n```python\ndef preprocess_image(img, ksize=3):\n    # Grayscale\n    img = cv.cvtColor(img, cv.COLOR_BGR2GRAY)\n\n    # Denoising\n    img = cv.medianBlur(img, ksize=ksize)\n\n    # Binaryization\n    _, img_bin = cv.threshold(\n        img, 0, 255, cv.THRESH_BINARY + cv.THRESH_OTSU)\n\n    # Inverse the binary image\n    img_bin = 255 - img_bin\n\n    return img_bin\n```\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://user-images.githubusercontent.com/26341224/235645117-2febb4e3-e4e7-4b32-b10e-5befe9f37866.jpg\"\n       alt=\"processed\" width=\"80%\"\u003e\n\u003c/p\u003e\n\n### Character segmentation\n\nThen we need to divide the text area in the image into single characters. Compared with complete sentences or words, recognizing a single letter is less prone to ambiguity (because there are only 26 possibilities in total), thus improving the overall recognition accuracy. And after dividing the sentence into a single letter, the problem can be simplified to a classification problem and a classifier can be established for each letter. In this way, the difficulty of the training model is reduced, and the calculation cost is also reduced.\n\nFinally, after dividing into single letters, the OCR system can support multiple languages more easily, because most languages are composed of basic characters (our OCR software does not support multiple languages for the time being).\n\nWe use the projection method to determine the position of each character and then cut it according to its position. At the same time, we record the position of each character for subsequent use.\n\nWe first calculate the horizontal projection and cut the picture horizontally through it:\n\n```python\ndef get_h_progection(img):\n    r, c = img.shape\n    h_progection = np.zeros(img.shape, np.uint8)\n    hrowsum = [0]*r\n    for i, j in product(range(r), range(c)):\n        if img[i, j] == 255:\n            hrowsum[i] += 1\n    for i in range(r):\n        for j in range(hrowsum[i]):\n            h_progection[i, j] = 255\n    cv.show('h_progection', h_progection)\n    return hrowsum\n```\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://user-images.githubusercontent.com/26341224/235650099-2137a6df-f04c-49fd-ab93-41ee94fd42db.png\"\n       alt=\"h_progection\" width=\"80%\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://user-images.githubusercontent.com/26341224/235650541-7182b552-dbe6-42ff-ad71-223bddb74be6.jpg\"\n       alt=\"h_cut\" width=\"80%\"\u003e\n\u003c/p\u003e\n\nThen we calculate the vertical projection and cut the picture vertically through it (don't forget to border each character for deep learning model recognition):\n\n```python\ndef get_v_progection(img):\n    r, c = img.shape\n    v_progection = np.zeros(img.shape, np.uint8)\n    vcolsum = [0]*c\n    for i, j in product(range(r), range(c)):\n        if img[i, j] == 255:\n            vcolsum[j] += 1\n    for j in range(c):\n        for i in range(r-vcolsum[j], r):\n            v_progection[i, j] = 255\n    cv.imshow('v_progection', v_progection)\n    return vcolsum\n```\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://user-images.githubusercontent.com/26341224/235652290-e29d4985-ac9e-4ae3-bb6b-634e5ba9a8cf.png\"\n       alt=\"v_progection\" width=\"80%\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://user-images.githubusercontent.com/26341224/236053978-134c3324-b1bf-4a5c-a324-a585576943f3.jpg\"\n       alt=\"v_cut\"\u003e\n\u003c/p\u003e\n\n### Recognize text\n\nIn order to save time, we skipped the steps of training the deep learning model and directly used the open source model of Google's Tesseract team.\n\nDirectly call Tesseract's API to identify character images:\n\n```python\ndef recognize_text(char):\n    return pytesseract.image_to_string(char, lang='eng', config='--psm 10').strip()\n```\n\n### Synthesis results\n\nFinally, draw the position information of the character (green box) and the recognition result (red character) to the original picture:\n\n```python\nstr = []\nfont_size, font_weight = 1, 2\nfor i in range(len(rows)):\n    for j in range(len(rows[i])):\n        text = recognize_text(rows[i][j])\n        cv.rectangle(img, (p[i][j][1]-p[i][j][3], p[i][j]\n                           [0]-p[i][j][2]), (p[i][j][1], p[i][j][0]), (0, 255, 0), 2)\n        cv.putText(img, text, (p[i][j][1]-p[i][j][3]-4, p[i][j][0]-p[i]\n                   [j][2]-1), cv.FONT_HERSHEY_COMPLEX, font_size, (50, 50, 255), font_weight)\n        str.append(text)\n\nprint(''.join(str), end=None)\ncv.imshow('Result', img)\ncv.waitKey(0)\n```\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://user-images.githubusercontent.com/26341224/235641560-74889377-f5e1-41ae-841f-5b44072e0f71.jpg\"\n       alt=\"result\" width=\"80%\"\u003e\n\u003c/p\u003e\n\n## License\n\nThe code in this project is released under the [MIT License](./LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmogeko%2Fopencv-ocr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmogeko%2Fopencv-ocr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmogeko%2Fopencv-ocr/lists"}