{"id":13801654,"url":"https://github.com/BigIskander/Handwriting-keyboard-for-Linux-tesseract","last_synced_at":"2025-05-13T11:31:29.781Z","repository":{"id":219420324,"uuid":"748679721","full_name":"BigIskander/Handwriting-keyboard-for-Linux-tesseract","owner":"BigIskander","description":"Handwriting keyboard for Linux using local OCR engine.","archived":false,"fork":false,"pushed_at":"2025-04-19T15:36:53.000Z","size":598,"stargazers_count":7,"open_issues_count":3,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-19T19:15:48.954Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://bigiskander.github.io/2024/02/01/handwritingkeyboard.html","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BigIskander.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-26T14:29:01.000Z","updated_at":"2025-04-19T15:36:50.000Z","dependencies_parsed_at":"2024-02-01T02:47:20.574Z","dependency_job_id":"b76808fe-84b4-4077-970a-1c2fa15f79aa","html_url":"https://github.com/BigIskander/Handwriting-keyboard-for-Linux-tesseract","commit_stats":null,"previous_names":["bigiskander/handwriting-keyboard-for-linux-tesseract"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BigIskander%2FHandwriting-keyboard-for-Linux-tesseract","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BigIskander%2FHandwriting-keyboard-for-Linux-tesseract/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BigIskander%2FHandwriting-keyboard-for-Linux-tesseract/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BigIskander%2FHandwriting-keyboard-for-Linux-tesseract/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BigIskander","download_url":"https://codeload.github.com/BigIskander/Handwriting-keyboard-for-Linux-tesseract/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253932898,"owners_count":21986473,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-04T00:01:25.440Z","updated_at":"2025-05-13T11:31:29.771Z","avatar_url":"https://github.com/BigIskander.png","language":"Rust","funding_links":[],"categories":["Applications"],"sub_categories":["Office \u0026 Writing"],"readme":"# Handwriting-keyboard-for-Linux.\n\nThis is program written for Linux desktop environment.\n\nTo recognize handwritten pattern program uses OCR engine.\n\nAt the moment program supports 2 OCR engines: Tesseract OCR and PaddleOCR.\n\nTo send the keyboard input program uses xdotool or ydotool.\n\nYou can find compiled .deb, .rpm and .AppImage packages in releases page.\n\nThis is the instruction for version 2, instruction for version 1 is located at v1 branch of this repository.\n\n## How to use the program\n\n0) Launch the program with or without command line options \n1) write text in the canvas by using mouse or stylus (on graphical tablet) \n2) press 'recognize' button \n3) press to recognized text, program will type this text or copy to clipboard and paste by triggering ***ctrl+V*** (or ***shift+ctrl+V***) keypress\n\nNote: before using the program you need to install [dependencies](#installing-dependencies).\n\nIf program's window is in focus, before sending keyboard input program will trigger ***alt+Tab*** keypress to return focus to previous active window and only then send the input (this does not applied when '--skip-taskbar' option is set).\n\n## Command line options \n\n```\nUsage: handwriting-keyboard-t [OPTIONS]\n\nOptions:\n      --use-paddle-ocr...\n          Use PaddleOCR to recognize handwriting pattern. \n          By default program uses Tesseract OCR.\n\n  -l, --lang \u003clang\u003e\n          Language used to recognize handwriting pattern. Value depends on OCR engine. \n          Default value is 'chi_all' for Tesseract OCR and 'ch' for PaddleOCR.\n\n      --tessdata-dir \u003ctessdata-dir\u003e\n          A directory with *.traineddata files for Tesseract OCR engine. \n          Tesseract OCR specefic.\n\n  -a, --automode...\n          Automatically send recognize text request to OCR engine after every stroke.\n\n      --use-tmp-file...\n          Save canvas as temporary image file and send path of this file to OCR engine. \n          By default program sends image data to OCR engine via stdin.\n\n      --use-ydotool...\n          Use ydotool to send keyboard input. By default program uses xdotool.\n\n      --use-clipboard...\n          Copy text to clipboard and paste it via triggering ctrl+V (or shift+ctrl+V) \n          kyepress to paste the text. \n          By default program will try to type text \n          (ydotool only supports typing latin characters).\n\n      --use-shift...\n          Trigger shift+ctrl+V kyepress to paste text from clipboard. \n          By default program uses ctrl+V. \n          Only applyed when '--use-clipboard' option is set.\n\n      --return-focus...\n          Program will return focus to previous window by triggering alt+Tab kyepress \n          every time when program gains focus (after mouseup event inside the window). \n          Will not work if option '--skip-taskbar' is set.\n\n      --return-keyboard...\n          After sending keyboard input program will trigger alt+Tab keypress \n          to return focus back to keyboad's window. \n          Will not work if option '--skip-taskbar' is set.\n\n      --fly-to-bottom...\n          At launch program window will fly to the bottom of the screen \n          and resize to screen width.\n\n      --skip-taskbar...\n          Program window will skip taskbar.\n\n      --dark-theme...\n          Use dark theme. Change colors of the application to dark theme.\n\n      --debug...\n          Output some debug info in console.\n\n  -h, --help\n          Print help (see a summary with '-h')\n\n  -V, --version\n          Print version\n```\n\nExample of using command line options:\n\n```\nhandwriting-keyboard-t --tessdata-dir=/home/user/ --lang=chi_sim -a\n```\n\nIn this case (above), to recognize hand written pattern program will use Tesseract OCR (as default OCR engine) with training data from folder \"**/home/user/**\" and language \"**chi_sim**\" (Chinese simplified), particularly the file \"**/home/user/chi_sim.traineddata**\". Also in this case the program will automatically send request to tesseract-ocr after every stroke, because it was launched with \"**-a**\" option.\n\n## Installing dependencies\n\n1) Install OCR engine: Tesseract OCR or/and PaddleOCR.\n    - in debian based linux system you can install Tesseract OCR from repository:\n        ```\n        sudo apt install tesseract-ocr\n        ```\n        for other linux distributions you can find instructions in their github repository: https://github.com/tesseract-ocr/tesseract?tab=readme-ov-file#installing-tesseract \n    - to install PaddleOCR, I would recommend to:\n        1) set up conda envirenment by [this instruction](https://paddlepaddle.github.io/PaddleOCR/main/en/ppocr/environment.html), because PaddleOCR's python dependencies might conflict with existing python environment\n        2) then activate this conda environment and install PaddleOCR:\n            ```\n            conda activate paddle_env\n            \n            python -m pip install paddlepaddle==3.0.0rc1 -i https://www.paddlepaddle.org.cn/packages/stable/cpu/\n            \n            pip install \"paddleocr\u003e=2.0.1\"\n            ```\n            For different type of settings and instructions visit PaddleOCR's official website https://paddlepaddle.github.io/PaddleOCR/main/en/index.html\n\n2) Install xdotool or/and ydotool.\n    - in debian based linux system you can install xdotool from repository:\n        ```\n        sudo apt install xdotool\n        ```\n        for other linux distributions you can find instructions in their github repository: https://github.com/jordansissel/xdotool?tab=readme-ov-file#installation\n    - to install ydotool, I would recommend to:\n        1) compile ydotool from source code in their repository: https://github.com/ReimuNotMoe/ydotool?tab=readme-ov-file#build\n        2) then copy **ydotool** and **ydotoold** binaries into **~/bin** folder \n\n## Notes about dependencies\n\n1) If you use the program with Tesseract OCR, I would recommend to install tesseract 4 (instead of tesseract 5). Because the results is more accurate when using with tesseract 4 (at least for recognition of text (writing) in Chinese language).\n\n2) If you use the program with Tesseract OCR, you also need to download model data for tesseract-ocr and copy [.traineddata](https://github.com/tesseract-ocr/tessdata) files to data folder of tesseract-ocr (for example for tesseract-ocr 4.0 it would be this folder **/usr/share/tesseract-ocr/4.00/tessdata/**). Or alternatively you can put these files in whatever folder you like and run program with `--tessdata-dir` command line option and point to the folder where model data files are located.\n\n3) By default program uses Tesseract OCR with language set as **chi_all**, *.traineddata files for which you can download by [this link](https://github.com/gumblex/tessdata_chi).\n\n4) PaddleOCR is significantly more accurate than Tesseract OCR, at least in regognizing Chinese characters, however it is also slower at least on my hardware.\n\n5) If you use the program with PaddleOCR. PaddleOCR downloads model data at first use, then it can be used offline. List of available languages can be found by this [link](https://paddlepaddle.github.io/PaddleOCR/main/en/ppocr/blog/multi_languages.html#5-support-languages-and-abbreviations).\n\n6) If you use the program with PaddleOCR and it is installed in conda environment, you need to activete conda environment first and then launch this program.\n\n7) As for sending keyboard input:\n    - **xdotool** - only supports X11 desktop environment \n    - **ydotool** - works in X11 and Wayland desktop environment, ydotool can type only latin characters and **ydotoold** process should be running (in background or in separate terminal) in order to **ydotool** to work\n    - instead of typing program can copy the text to clipboard and paste by trigerring ***ctrl+V*** (or ***shift+ctrl+V***) keypress\n\n## Notes about Wayland\n\nThis program (application) can work in Wayland desktop environment, however it is not fully supported.\n\nSome specifics of using this program in Wayland:\n\n1) **Always on top** property of the window can not be set programmatically. As workaround you can set it manually by right clicking on title bar of the window and checking 'Always on top' option (it should work, at least in gnome based desktop environment).\n\n2) **set_accept_focus(false)** property of the window is not working in Wayland. Program's window will gain focus anyway (when interacting with it).\n\n3) **--skip-taskbar** command line option will not work in Wayland.\n\n4) **--fly-to-bottom** command line option will not position program's window correctly.\n\n5) **Xdotool** does not support Wayland. For that reasoun I would recommend to use this program with **ydotool** instead.\n\n6) If you use this program in Wayland desktop environment. I would recommed to launch it with **--use-ydotool** and **--use-clipboard** command line options.\n\n## Some technical details\n\nProgram written by using tauri framework https://tauri.app/\n\nThe script from https://github.com/ChenYuHo/handwriting.js is used to make a writing canvas.\n\nTo recognize handwritten pattern program uses [Tesseract OCR](https://github.com/tesseract-ocr/tesseract) or [PaddleOCR](https://paddlepaddle.github.io/PaddleOCR/main/en/index.html).\n\nTo send keyboard input program uses [xdotool](https://github.com/jordansissel/xdotool) or [ydotool](https://github.com/ReimuNotMoe/ydotool).\n\nIn order to run from code or compile the programm: You need to install [Node.js 20](https://nodejs.org/en) or newer version and [Rust](https://www.rust-lang.org/) as well.\n\nInstall Node.js dependencies: \n```\nnpm install\n```\n\nRun program in development environment: \n\n```\nnpm run tauri dev\n```\n\nRun program in development environment with cli (command line) options: \n\n```\nnpm run tauri dev -- -- -- cli_options\n```\n\nCompile the program: \n\n```\nnpm run tauri build\n```\n\nOlder version of this program using Google API instead is available by this link: [https://github.com/BigIskander/Handwriting-keyboard-for-Linux.](https://github.com/BigIskander/Handwriting-keyboard-for-Linux.)\n\n## Recommended IDE Setup\n\n- [VS Code](https://code.visualstudio.com/) + [Tauri](https://marketplace.visualstudio.com/items?itemName=tauri-apps.tauri-vscode) + [rust-analyzer](https://marketplace.visualstudio.com/items?itemName=rust-lang.rust-analyzer)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FBigIskander%2FHandwriting-keyboard-for-Linux-tesseract","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FBigIskander%2FHandwriting-keyboard-for-Linux-tesseract","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FBigIskander%2FHandwriting-keyboard-for-Linux-tesseract/lists"}