{"id":15657532,"url":"https://github.com/lromul/gramtion","last_synced_at":"2025-05-05T15:51:33.991Z","repository":{"id":95995902,"uuid":"302050809","full_name":"lRomul/gramtion","owner":"lRomul","description":"Twitter bot for generating photo descriptions (alt text)","archived":false,"fork":false,"pushed_at":"2021-07-01T12:18:29.000Z","size":989,"stargazers_count":23,"open_issues_count":1,"forks_count":2,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-26T14:07:05.452Z","etag":null,"topics":["accessibility","alt-text","deep-learning","image-captioning","twitter","twitter-bot"],"latest_commit_sha":null,"homepage":"https://twitter.com/GramtionBot","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lRomul.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-10-07T13:48:47.000Z","updated_at":"2023-05-23T00:29:15.000Z","dependencies_parsed_at":"2023-03-27T14:35:48.831Z","dependency_job_id":null,"html_url":"https://github.com/lRomul/gramtion","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lRomul%2Fgramtion","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lRomul%2Fgramtion/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lRomul%2Fgramtion/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lRomul%2Fgramtion/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lRomul","download_url":"https://codeload.github.com/lRomul/gramtion/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252526645,"owners_count":21762542,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["accessibility","alt-text","deep-learning","image-captioning","twitter","twitter-bot"],"created_at":"2024-10-03T13:07:52.935Z","updated_at":"2025-05-05T15:51:33.969Z","avatar_url":"https://github.com/lRomul.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/lRomul/gramtion\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/lRomul/gramtion/master/pics/gramtion.jpeg\" alt=\"Title GRAMTION on the background of a black and white photo of clouds in the sky\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n    \u003cem\u003eTwitter bot for generating photo descriptions\u003c/em\u003e\n\u003c/p\u003e\n\n\u003e According to Twitter support bot account has been suspended due to violations of rules:\n\u003e * Creating serial and/or multiple accounts with overlapping uses\n\u003e * Evading a permanent suspension by creating or using another account\n\u003e * Cross-posting Tweets or links across multiple accounts\n\u003e * Aggressive following, particularly through automated means\n\n---\n\n**Twitter**: \u003ca href=\"https://twitter.com/GramtionBot\" target=\"_blank\"\u003ehttps://twitter.com/GramtionBot\u003c/a\u003e\n\n**Source Code**: \u003ca href=\"https://github.com/lRomul/gramtion\" target=\"_blank\"\u003ehttps://github.com/lRomul/gramtion\u003c/a\u003e\n\n---\n\nThis repo contains the source code of the Twitter [@GramtionBot](https://twitter.com/GramtionBot) for generating photo descriptions.  \nUse cases and intends: \n* Help visually impaired Twitter users. \nGood image descriptions (alt text) will help them understand what is happening in an image. \nInstagram and Facebook use deep learning for image captioning. \nTwitter users can only add custom alt text descriptions themselves. \nAutomation of alt text generation will help Twitter be more accessible. \n* Collect dataset for image captioning (legal issues for this use case are yet to be discussed). \nAnnotations can be done by creating polls about prediction quality and getting corrected descriptions from users. \nTwitter API v1.1 has not the ability to create polls, but it will be added in API v2. \n\n## How to use\n\nTweet photo with mention [@GramtionBot](https://twitter.com/GramtionBot) or reply with mention to a tweet with a photo and the bot will send you an auto-generated image description.  \n\n## Dependencies\n\nGramtion is mainly built from ready-to-use third party libraries:\n* Image captioning model taken from [self-critical.pytorch](https://github.com/ruotianluo/self-critical.pytorch).\n* Evaluate text and image similarity with [CLIP](https://openai.com/blog/clip/) by OpenAI.\n* OCR and image labels by [Google Vision Ai](https://cloud.google.com/vision).\n* Bot written with [Tweepy](https://github.com/tweepy/tweepy).\n* Configuration settings implemented with [pydantic](https://github.com/samuelcolvin/pydantic/).\n* Docker image based on [Dokai](https://github.com/osai-ai/dokai).\n\n## Current issues \n\n* Some descriptions may be confusing. Annotations may be created by using polls about prediction quality and getting corrected descriptions from users. Twitter API v1.1 has not the ability to create polls, but it will be added in API v2 endpoint `POST /2/tweets`.\n* For drawings and some other types of images, the predictions are pretty random. \n* Some results may reflect inherent gender and racial biases of open datasets.\n\n## Run own bot\n\nTo run an instance of the bot you need to install [Docker](https://www.docker.com/) and create [Twitter API auth credentials](https://realpython.com/twitter-bot-python-tweepy/#creating-twitter-api-authentication-credentials).  \nIf you have a Twitter developer account, but don't want to use it as a bot account, you can authenticate a new user that’s not has a developer account with [twurl](https://github.com/twitter/twurl).\n\n* Create `.env` file with credentials. \n\n    ```\n    CONSUMER_KEY={{ consumer_key }}\n    CONSUMER_SECRET={{ consumer_secret }}\n    ACCESS_TOKEN={{ access_token }}\n    ACCESS_TOKEN_SECRET={{ access_token_secret }}\n    ```\n\n* Setup Google Vision AI and create account key ([link](https://cloud.google.com/vision/docs/setup)). Copy key as `google_key.json `.\n\n* Run Docker container with running the bot  \n\n    ```bash\n    docker run -d --restart=always \\\n        --env-file .env \\\n        -v google_key.json:/workdir/google_key.json \\\n        --name=gramtion \\\n        ghcr.io/lromul/gramtion:0.0.5\n    ```\n\n* Open logs \n\n    ```bash\n    docker logs -f gramtion\n    ```\n\n* Stop container\n\n    ```bash\n    docker stop gramtion\n    docker rm gramtion\n    ```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flromul%2Fgramtion","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flromul%2Fgramtion","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flromul%2Fgramtion/lists"}