{"id":13579524,"url":"https://github.com/iuliaturc/detextify","last_synced_at":"2025-04-05T21:31:54.789Z","repository":{"id":64920632,"uuid":"576039946","full_name":"iuliaturc/detextify","owner":"iuliaturc","description":"Remove text from AI-generated images","archived":false,"fork":false,"pushed_at":"2023-03-24T01:34:13.000Z","size":2531,"stargazers_count":246,"open_issues_count":10,"forks_count":24,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-08-01T15:33:11.755Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/iuliaturc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-12-08T21:48:34.000Z","updated_at":"2024-07-30T16:32:02.000Z","dependencies_parsed_at":"2024-01-07T16:30:19.189Z","dependency_job_id":null,"html_url":"https://github.com/iuliaturc/detextify","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iuliaturc%2Fdetextify","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iuliaturc%2Fdetextify/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iuliaturc%2Fdetextify/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iuliaturc%2Fdetextify/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/iuliaturc","download_url":"https://codeload.github.com/iuliaturc/detextify/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223214892,"owners_count":17107604,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T15:01:40.190Z","updated_at":"2024-11-05T17:31:58.608Z","avatar_url":"https://github.com/iuliaturc.png","language":"Python","readme":"# Detextify\n\n## What is this?\n\n**TL;DR**: A Python library to remove unwanted pseudo-text from images generated by your favorite generative AI models (Stable Diffusion, Midjourney, DALL·E).\n\n| Before                      | After                                  |\n|-----------------------------|----------------------------------------|\n| ![before](data/octopus.png) | ![after](data/octopus_detextified.png) |\n\n## So, why should I care?\n\nWe all know generative AI is the coolest thing since sliced bread 🍞.\n\nBut try using any off-the-shelf generative vision model and you'll quickly see that these systems can get... creative with interpreting\nyour prompts.\n\nSpecifically, you'll observe all kinds of weird artifacts on your images from extra fingers on hands, to arms coming out of chests,\nto alien text written in random places.\n\nFor generative systems to actually be usable in downstream applications, we need to better control these outputs\nand mitigate unwanted effects.\n\nWe believe the next frontier for generative AI is about **robustness** and **trust**. In other words, how can we architect\nthese systems to be controllable, relevant, and predictably consistent with our needs?\n\n`Detextify` is the first phase in our vision of robustifying generative AI.\n\nIf we get this right, we will unlock slews of new applications for generative systems that will change the landscape of human-AI collaboration. 🌎\n\n## Cute, but what are you actually doing?\n\n`Detextify` runs text detection on your image, masks the text boxes, and in-paints the masked regions\nuntil your image is text-free. `Detextify` can be run entirely on your local machine (using\n[Tesseract](https://github.com/tesseract-ocr/tesseract) for text detection and\n[Stable Diffusion](https://huggingface.co/stabilityai/stable-diffusion-2-inpainting) for in-painting), or can call existing APIs\n([Azure](https://azure.microsoft.com/en-us/products/cognitive-services/computer-vision/) for text detection and\n[OpenAI](https://openai.com/dall-e-2/) or [Replicate](https://replicate.com/) for in-painting).\n\n## Installation\n```commandline\npip install detextify\n```\n\nAdditionally:\n- To run text detection locally (as opposed to using the Azure API), you need to [install Tesseract](https://tesseract-ocr.github.io/tessdoc/Installation.html).\n- To run in-painting locally (as opposed to using the OpenAI or Replicate APIs), you need a GPU with CUDA and cuDNN installed.\n\n## Usage\nSee [this Colab notebook](https://colab.research.google.com/drive/1a8BJ55yT88IKDyzFuk3jqdATZ_jZ2cwy?usp=sharing) for how to use the library, or follow the instructions below.\n\nYou can remove unwanted text from your image in just a few lines 💪:\n```python\nfrom detextify.text_detector import TesseractTextDetector\nfrom detextify.inpainter import LocalSDInpainter\nfrom detextify.detextifier import Detextifier\n\ntext_detector = TesseractTextDetector(\"/path/to/tesseract/installation\")\ndetextifier = Detextifier(text_detector, LocalSDInpainter())\ndetextifier.detextify(\"/my/input/image/path.png\", \"/my/output/image/path.png\")\n```\n\nand 💣💥, just like that, your image is cleared of any bizarre text artifacts.\n\nOr if you want to clean up a directory of PNG images, just wrap it in a for-loop:\n```python\nimport glob\nfrom detextify.text_detector import TesseractTextDetector\nfrom detextify.inpainter import LocalSDInpainter\nfrom detextify.detextifier import Detextifier\n\ntext_detector = TesseractTextDetector(\"/path/to/tesseract/installation\")\ndetextifier = Detextifier(text_detector, LocalSDInpainter())\nfor img_file in glob.glob(\"/path/to/dir/*.png\"):\n    detextifier.detextify(img_file, img_file.replace(\".png\", \"_detextified.png\"))\n```\n\nWe provide multiple implementations for text detection and in-painting (both local and API-based), and you are also free to add your own.\n\n### Text Detectors\n1. `TesseractTextDetector` (based on [Tesseract](https://github.com/tesseract-ocr/tesseract)) runs locally.\nFollow [this guide](https://tesseract-ocr.github.io/tessdoc/Installation.html) to install the `tesseract` library locally. On Ubuntu:\n```\nsudo apt install tesseract-ocr\nsudo apt install libtesseract-dev\n```\nTo find the path where it was installed (and pass it to the `TesseractTextDetector` constructor):\n```\nwhereis tesseract\n```\n\n2. `AzureTextDetector` calls a computer vision API from Microsoft Azure. You will first need to create a\n[Computer Vision resource](https://portal.azure.com/#create/Microsoft.CognitiveServicesComputerVision) via the Azure\nportal. Once created, take note of the endpoint and the key.\n```python\nAZURE_CV_ENDPOINT = \"https://your-endpoint.cognitiveservices.azure.com\"\nAZURE_CV_KEY = \"your-azure-key\"\ntext_detector = AzureTextDetector(AZURE_CV_ENDPOINT, AZURE_CV_KEY)\n```\nOur evaluation shows that the two text detectors produce comparable results.\n\n### In-painters\n1. `LocalSDInpainter` (implemented via Huggingface's `diffusers` library) runs locally and requires a GPU. Defaults to\n[Stable Diffusion v2 for in-painting](https://huggingface.co/stabilityai/stable-diffusion-2-inpainting).\n2. `ReplicateSDInpainter` calls the [Replicate](https://replicate.com) API. Defaults to Stable Diffusion v2 for\nin-painting (and requires an API key).\n3. `DalleInpainter` calls the [DALL·E 2](https://labs.openai.com) API from OpenAI (and requires an API key).\n```python\n# You only need to instantiate one of the following:\nlocal_inpainter = LocalSDInpainter()\nreplicate_inpainter = ReplicateSDInpainter(\"your-replicate-key\")\ndalle_inpainter = DalleInpainter(\"your-openai-key\")\n```\n\n## Contributing\nTo contribute, clone the repository, make your changes, commit and push to your clone, and submit a pull request.\n\nTo build the library, you need to install [poetry](https://python-poetry.org/):\n```commandline\ncurl -sSL https://install.python-poetry.org | python3 -\n# Add poetry to your PATH. Note the specific path will differ depending on your system.\nexport PATH=\"/home/ubuntu/.local/bin:$PATH\"\n# Check the installation was successful:\npoetry --version\n```\nInstall dependencies for `detextify`:\n```commandline\npoetry install\n```\nTo execute a script, run:\n```commandline\npoetry run python your_script.py\n```\nPlease run the unit tests to make sure that your changes are not breaking the codebase:\n```commandline\npoetry run pytest\n```\n\n## Authors\nThis project was authored by [Mihail Eric](https://twitter.com/mihail_eric) and [Julia Turc](https://twitter.com/juliarturc). If you are building in the generative AI space, we want to hear from you!\n","funding_links":[],"categories":["Image Synthesis","Python"],"sub_categories":["Inbox: Stable Diffusion"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiuliaturc%2Fdetextify","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fiuliaturc%2Fdetextify","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiuliaturc%2Fdetextify/lists"}