{"id":22316451,"url":"https://github.com/colintr/livedesktoptranslator","last_synced_at":"2026-04-30T00:33:54.814Z","repository":{"id":259049659,"uuid":"876150382","full_name":"ColinTr/LiveDesktopTranslator","owner":"ColinTr","description":"Live capture your screen and replace textual elements with their translations","archived":false,"fork":false,"pushed_at":"2024-11-20T14:07:43.000Z","size":192642,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-01-31T01:51:52.164Z","etag":null,"topics":["electron","layout-analysis","ocr","python","translation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ColinTr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-21T13:36:47.000Z","updated_at":"2024-11-20T14:11:23.000Z","dependencies_parsed_at":"2024-10-28T03:31:28.556Z","dependency_job_id":null,"html_url":"https://github.com/ColinTr/LiveDesktopTranslator","commit_stats":null,"previous_names":["colintr/livedesktoptranslator"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ColinTr%2FLiveDesktopTranslator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ColinTr%2FLiveDesktopTranslator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ColinTr%2FLiveDesktopTranslator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ColinTr%2FLiveDesktopTranslator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ColinTr","download_url":"https://codeload.github.com/ColinTr/LiveDesktopTranslator/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245581605,"owners_count":20639004,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["electron","layout-analysis","ocr","python","translation"],"created_at":"2024-12-03T23:06:37.229Z","updated_at":"2026-04-30T00:33:54.750Z","avatar_url":"https://github.com/ColinTr.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e\n  Live Desktop Translator\n\u003c/h1\u003e\n\n\n\u003cp align=\"center\"\u003e\n  Proof of concept for a desktop application that translates on-screen elements based on optical character recognition (OCR).\n\u003c/p\u003e\n\n\u003cdiv align=\"center\"\u003e\n\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n\u003c/div\u003e\n\n\n## 🔍 Overview\n\nWhile web pages can be easily translated using browser extensions, the textual elements of some applications cannot be easily extracted (e.g. video games, images, scanned documents, ...).\nThe goal of this application is to 1) capture a portion of the screen, 2) recognize the text it contains, 3) translate it and finally 4) paint it back on the screen, all in real time.\nCurrently, this application comes in the form of an overlay and implements [SuryaOCR](https://github.com/VikParuchuri/surya) for layout understanding, [EasyOCR](https://github.com/JaidedAI/EasyOCR) for optical character recognition and either [Argos Translate](https://github.com/argosopentech/argos-translate) for local translation or [MyMemory](https://mymemory.translated.net/doc/spec.php) for online translation. \n\nBelow is an example of translation from English to French by overlaying the application on top of a PDF:\n![Screenshot](readme_images/pdf_translation_example.png)\n\nThis main limitation of this application is its ability to correctly understand the layout (i.e. should the recognized words be concatenated into a sentence or not).\nSuryaOCR, the layout understanding model currently in use, was trained on structured documents (mainly PDFs and newspapers) and does not work well in various scenarios such as algorithms, tables, game footage, captions, ... \n\n## 🛠️ Development\n\n- [ ] Modal for adding/removing other input/output languages\n- [ ] Application packaging\n- [ ] Parameters save button\n- [ ] set updateIgnoreMouseEvents using coordinates instead of alpha value\n- [ ] Identify sub-tasks for specialized layout detection models (i.e. video game, outdoor, PDF, etc)\n\n\n## ⚙️ Starting the app (for development)\n\n### 1. Electron\n\n```bash\ncd electron_gui\n\n# Install the required packages\nnpm install\n\n# Launch the Electron app\nnpm start --enable-logging\n```\n\n\n### 2. Python\n\n#### Option 1 (recommended) - With Anaconda\n\nDownload [Anaconda](https://www.anaconda.com/download/success).\n\nFor Windows users, if conda is not recognized as a command by the terminal, add `C:\\ProgramData\\anaconda3\\Scripts` to the user's Path environment variables.\n\n```bash\ncd python_server\n\n# Create the virtual environment and install the packages with conda\nconda env create --file environment.yml --prefix ./ldtvenv\n\n# Activate the virtual environment\nconda activate .\\ldtvenv\n```\n\n#### Option 2 (untested) - With pip\n\nDownload [Python 3.12.7](https://www.python.org/downloads/release/python-3127/) (don't forget to add it to the PATH during install).\n\n```bash\ncd python_server\n\n# Create the empty virtual environment\npy -3.12 -m venv ldtvenv\n\n# Activate the virtual environment\n# On windows:\n  .\\ldtvenv\\Scripts\\activate\n# On linux:\n  source ldtvenv/bin/activate\n\n# Install pytorch\npip install torch torchvision --index-url https://download.pytorch.org/whl/cu118\n\n# Install PaddleOCR\npip install paddlepaddle-gpu==3.0.0b1 -i https://www.paddlepaddle.org.cn/packages/stable/cu118/\n\n# Install the rest of the packages\npip install -r requirements.txt\n```\n\n\n## 💾 Packaging the application\n\nStart by bundling the Python application and its dependencies into a single executable that can be run by the user without installing Python.\nWe'll use [PyInstaller](https://pyinstaller.org/en/stable/):\n```bash\ncd python_server\npyinstaller --onefile server.py\n```\n\n**Currently, you need to copy the generated file `python_server\\dist\\server.exe` into `electron_gui\\assets\\`.**\n\nThen, we'll create the Electron executable using [electron-forge](https://www.electronforge.io/):\n```bash\ncd electron_gui\nnpm run make\n```\n\nYou'll find the resulting application in a path similar to `electron_gui\\out\\live_desktop_translator-win32-x64` (the last folder depends on your system's architecture).\n\n## ⚖️ License\n\nThis code is released under the MIT license. See the LICENSE file for more information.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcolintr%2Flivedesktoptranslator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcolintr%2Flivedesktoptranslator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcolintr%2Flivedesktoptranslator/lists"}