{"id":49193145,"url":"https://github.com/dbetm/handwritten-flowchart-with-cnn","last_synced_at":"2026-04-23T08:03:11.338Z","repository":{"id":37660845,"uuid":"235933972","full_name":"dbetm/handwritten-flowchart-with-cnn","owner":"dbetm","description":"Recognition of handwritten flowcharts using convolutional neural networks to generate C source code and reconstructed digital flowchart.","archived":false,"fork":false,"pushed_at":"2023-07-08T22:31:00.000Z","size":33725,"stargazers_count":45,"open_issues_count":9,"forks_count":8,"subscribers_count":5,"default_branch":"master","last_synced_at":"2023-07-08T23:22:39.535Z","etag":null,"topics":["artificial-vision","convolutional-neural-networks","deep-learning","flowchart","flowchart2code","graphs","handwritten","handwritten-flowcharts","image2code","object-detection","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dbetm.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-01-24T03:33:17.000Z","updated_at":"2023-05-30T14:52:30.000Z","dependencies_parsed_at":"2023-01-25T06:15:46.872Z","dependency_job_id":null,"html_url":"https://github.com/dbetm/handwritten-flowchart-with-cnn","commit_stats":null,"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"purl":"pkg:github/dbetm/handwritten-flowchart-with-cnn","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dbetm%2Fhandwritten-flowchart-with-cnn","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dbetm%2Fhandwritten-flowchart-with-cnn/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dbetm%2Fhandwritten-flowchart-with-cnn/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dbetm%2Fhandwritten-flowchart-with-cnn/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dbetm","download_url":"https://codeload.github.com/dbetm/handwritten-flowchart-with-cnn/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dbetm%2Fhandwritten-flowchart-with-cnn/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32171223,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-23T02:19:40.750Z","status":"ssl_error","status_checked_at":"2026-04-23T02:17:55.737Z","response_time":53,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-vision","convolutional-neural-networks","deep-learning","flowchart","flowchart2code","graphs","handwritten","handwritten-flowcharts","image2code","object-detection","python"],"created_at":"2026-04-23T08:03:10.281Z","updated_at":"2026-04-23T08:03:11.317Z","avatar_url":"https://github.com/dbetm.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Recognition of handwritten flowcharts with CNNs\nRecognition of handwritten flowcharts using convolutional neural networks to generate C source code and reconstructed digital flowcharts.\n\n## Overview\nThe pipeline implemented in order to solve the problem of handwritten flowchart recognition uses image preprocessing, the input image is sent to two detectors, the shape-connector detector and the text detector. For text flow, the image is binarize and it uses [Keras OCR](https://pypi.org/project/keras-ocr/) to locate text and an implemented model with CNN + LSTM for character classifing; on the flow of shapes and connectors it uses unsharp masking and a model that is called [Faster R-CNN](https://arxiv.org/abs/1506.01497) with backbone VGG-16, which is an object detection model.\n\nIn order to augment the precision of the text detector, the technique called continual learning is used. After, some training with the text style of a specific user, the model will improve the text recognition.\n\nFinally, the outputs are the generated source code in C, its compilation output and the digital reconstructed diagram as an image.\n\n**Note**: Flowcharts used for testing are constructed with a defined shape-connector set. You can check it [here](https://github.com/dbetm/handwritten-flowchart-with-cnn/tree/master/model/set_shapes.png).\n\n## Set up for testing detections\n1. Create a virtual environment (venv) with Conda with name `handwritten-flowchart-recog`. The project was tested on Python 3.6 and Python 3.7. So, consider to use the same version.\n2. Download / clone this repo.\n3. Acivate the new venv, move to the project directory and install the requirements: `$ pip install -r requirements.txt`\n4. Shapes-connectors model:\n    - Download the folder from [here](https://drive.google.com/drive/folders/1Pax_lIypAP5qYj-oDi1fFL0COUnjLe0l?usp=sharing) - I will give you access on Google Drive as soon as possible.\n    - Paste it (unzipped) into `model/training_results/` (path inside the repo), so must be `model/training_results/9`\n5. Text model:\n    - Download IAM dataset from [here](https://drive.google.com/file/d/1XNb3sJa5v_5EDll5BGk4tVbdlMAwts0q/view?usp=sharing).\n        - Inside `text_model`, please create a folder with name `data_model`.\n        - `iam.hdf5` (94.1 MB) paste into `text_model/data_model/`\n    - Download pre-trained model from [here](https://drive.google.com/file/d/1JikohW11j74PhV-FhtvTY7XorLCFUWhN/view?usp=sharing).\n        - `checkpoint_weights.hdf5` (38.5 MB) paste into `text_model/output/iam/puigcerver/`\n\n## Usage\n1. Please, activate your Conda enviroment. `$ conda activate handwritten-flowchart-recog`\n2. Move to inside repository folder, example: `$ cd handwritten-flowchart-with-cnn`\n3. Type: ```$ python3 handler.py ```. Alternatively you can pass the Conda env. name: `$ python3 handler.py --env another-conda-env`\n4. Use \"Recognize flowchart\" option to process a handwritten flowchart.\n5. Click at \"Predict\" button.\n6. A window with the **shape** detection will appear. You can close it pressing 'q' key or using the X button.\n7. A window with the **text** detection will appear. You can close it pressing 'q' key or using the X button.\n8. The final results will be saved on `results/results_x`.\n\n### Some examples of the results\n![example 1](https://github.com/dbetm/handwritten-flowchart-with-cnn/blob/master/Images/some_results/fibo.png \"Fibonacci sequence\")\n\nCalculate the nth term of the Fibonacci sequence.\n\n![example 2](https://github.com/dbetm/handwritten-flowchart-with-cnn/blob/master/Images/some_results/hello_world.png \"Hello world\")\n\nHello world.\n\n------\n\n## Paper\nA paper was written in 2022 and published on International Journal of Computer Applications, you can find it here: [Recognition of Handwritten Flowcharts using Convolutional Neural Networks](https://www.ijcaonline.org/archives/volume184/number1/32301-2022921969)\n\n------\n\n## Dataset\nWould you like to download the training dataset? [Link to Kaggle](https://www.kaggle.com/davbetm/flowchart-3b). On Kaggle you will find details about it.\n\nPlease cite the dataset with:\n\n- Author: ISC UPIIZ students\n- Title: Flowchart 3b\n- Version: 3.0\n- Date: May 2020.\n- Editors: Onder F. Campos and David Betancourt.\n- Publisher Location: Zacatecas, Mexico.\n- Electronic Retrieval Location: https://www.kaggle.com/davbetm/flowchart-3b\n\n## Notes\n\n- This project was finished on July 2020 as a final school project, since then, only minor fixes and improvements have happened.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdbetm%2Fhandwritten-flowchart-with-cnn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdbetm%2Fhandwritten-flowchart-with-cnn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdbetm%2Fhandwritten-flowchart-with-cnn/lists"}