{"id":15821286,"url":"https://github.com/daaain/image-alt-text-generator-extension","last_synced_at":"2025-03-15T00:31:02.384Z","repository":{"id":205314354,"uuid":"708542512","full_name":"daaain/image-alt-text-generator-extension","owner":"daaain","description":null,"archived":false,"fork":false,"pushed_at":"2023-11-04T12:18:36.000Z","size":9254,"stargazers_count":4,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-10-12T07:46:20.106Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/daaain.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-22T21:19:08.000Z","updated_at":"2023-11-07T12:40:07.000Z","dependencies_parsed_at":null,"dependency_job_id":"28fc4fdb-45a6-4c14-834d-1b7a169ffff5","html_url":"https://github.com/daaain/image-alt-text-generator-extension","commit_stats":{"total_commits":10,"total_committers":2,"mean_commits":5.0,"dds":0.4,"last_synced_commit":"78b1dd2d699e60206952a0bdaa4113069b267006"},"previous_names":["daaain/image-alt-text-generator-extension"],"tags_count":0,"template":false,"template_full_name":"fregante/browser-extension-template","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daaain%2Fimage-alt-text-generator-extension","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daaain%2Fimage-alt-text-generator-extension/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daaain%2Fimage-alt-text-generator-extension/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daaain%2Fimage-alt-text-generator-extension/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/daaain","download_url":"https://codeload.github.com/daaain/image-alt-text-generator-extension/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243667668,"owners_count":20328032,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-05T07:21:27.201Z","updated_at":"2025-03-15T00:31:01.796Z","avatar_url":"https://github.com/daaain.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Image alt text generator\n\n![Screenshot demonstrating the idea](./docs/screenshot.png)\n\nThis is a proof of concept browser extension to generate descriptive alt text for images that don't\nhave one, on any web page.\n\nTo make this possible, it does:\n\n* Recursively traverse the DOM to find images and check their alt property\n* Collect images that have missing or bad (\u003c20 characters currently) alt text\n* Convert these images to Base64 JPEGs\n* Send them to a multi-modal (text + vision) model and ask it to describe the image\n* Inject the detailed (usually 2 paragraphs) description back into the image alt property\n\nMost of this is currently implemented in [source/content.js](./source/content.js).\n\nIn its current form it is not ready to be published as an extension, but there are many possible\ndirections to take it to:\n\n* Move the UI into the browser bar rather than rendering the button on the content page\n* Do the work in background worker threads (although the extension itself isn't doing that much)\n* Possibly run the model in the browser using WebAssembly / WebGPU\n* Use GPT-4V API once it's available to simplify usage\n* Fully test with screen readers and implement keyboard shortcuts and other UX improvements\n* Save the generated description into a database or file and upload somewhere to prevent the same image having to be redone by every visitor on a page\n\n## 📹 Demo screencast\n\nhttps://github.com/daaain/image-alt-text-generator-extension/assets/69962/b0fdf1a8-320f-480a-8f2a-1f603d738030\n\n## 🌋 LLaVa model\n\nFor this extension to work, you need to have LLaVa 1.5 running in the Llama.cpp web server.\n\nTo set these up you can follow these steps on an Apple silicon (M1/M2) Mac:\n\n```sh\ngit clone https://github.com/ggerganov/llama.cpp.git\ncd llama.cpp\nmake\ncurl -L https://huggingface.co/mys/ggml_llava-v1.5-7b/resolve/main/ggml-model-q5_k.gguf -o ggml-model-q5_k.gguf\ncurl -L https://huggingface.co/mys/ggml_llava-v1.5-7b/resolve/main/mmproj-model-f16.gguf -o mmproj-model-f16.gguf\n./server -m ggml-model-q5_k.gguf --mmproj mmproj-model-f16.gguf -ngl 1\n```\n\n## 🛠 Development\n\n1. Run `npm install` to install all required dependencies\n1. Run `npm run all` to build with Parcel and watch for changes; run the Web-ext library to pack the\ndevelopment extension and open it in Chromium; and to serve up the contents of `test_page`\n\n## Credits\n\nBrowser extension template from [fregante/browser-extension-template](https://github.com/fregante/browser-extension-template)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdaaain%2Fimage-alt-text-generator-extension","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdaaain%2Fimage-alt-text-generator-extension","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdaaain%2Fimage-alt-text-generator-extension/lists"}