{"id":28412588,"url":"https://github.com/sharukat/vista-meta-hackathon","last_synced_at":"2026-05-01T18:33:11.155Z","repository":{"id":292405076,"uuid":"980808506","full_name":"sharukat/vista-meta-hackathon","owner":"sharukat","description":"AI-powered browser extension enabling visually impaired users to interact with any website via voice, combining web scraping, LLMs, and speech technology.","archived":false,"fork":false,"pushed_at":"2025-05-23T15:02:52.000Z","size":27,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-18T14:22:35.239Z","etag":null,"topics":["chrome-extension","flask","llama3","python","retrival-augmented-generation","speech-to-text","text-to-speech","visually-impaired","voice-assistant"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sharukat.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-09T18:49:34.000Z","updated_at":"2025-05-23T15:02:55.000Z","dependencies_parsed_at":null,"dependency_job_id":"86217d09-4db1-40b2-ad60-a7cc68d5d51c","html_url":"https://github.com/sharukat/vista-meta-hackathon","commit_stats":null,"previous_names":["sharukat/luma-meta-hackathon-2024","sharukat/vista-meta-hackathon"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sharukat/vista-meta-hackathon","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sharukat%2Fvista-meta-hackathon","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sharukat%2Fvista-meta-hackathon/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sharukat%2Fvista-meta-hackathon/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sharukat%2Fvista-meta-hackathon/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sharukat","download_url":"https://codeload.github.com/sharukat/vista-meta-hackathon/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sharukat%2Fvista-meta-hackathon/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32508901,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-30T13:12:12.517Z","status":"online","status_checked_at":"2026-05-01T02:00:05.856Z","response_time":64,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chrome-extension","flask","llama3","python","retrival-augmented-generation","speech-to-text","text-to-speech","visually-impaired","voice-assistant"],"created_at":"2025-06-02T23:14:34.877Z","updated_at":"2026-05-01T18:33:11.115Z","avatar_url":"https://github.com/sharukat.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# VISTA: Visually Impaired Speech Technology Assistant\n\n**VISTA** is an AI-powered browser extension designed to enable visually impaired users to navigate and interact with websites using natural voice commands. Developed during the **Meta AI Llama Hackathon 2024** in Toronto, VISTA combines advanced web scraping, large language models, and speech technology to provide seamless, conversational access to web content.\n\n## 🚀 Motivation\n\nThe web remains largely inaccessible to visually impaired individuals, especially when websites are poorly structured or lack assistive tagging. VISTA was built to address this gap—empowering users to obtain relevant information through voice in a natural, intuitive manner.\n\nBy integrating a transformer-based web scraper, the **Llama 3.1-8B** model, and **Retrieval-Augmented Generation (RAG)**, VISTA delivers context-aware answers extracted directly from any webpage. Paired with **speech-to-text** and **text-to-speech** capabilities, it enables fluid conversation with the web, enhancing independence and accessibility.\n\n## 🚀 Getting Started\n\n### Prerequisites\n- Python 3.9+\n- [Ollama](https://ollama.com) (for running Llama 3.1 models locally)\n- Chrome or any Chromium-based browser (for extension deployment)\n\n### Installation\n\n1. **Clone the repository**\n   ```bash\n   git clone https://github.com/your-username/vista-ai-assistant.git\n   cd vista-meta-hackathon\n   ```\n2. **Install Python dependencies**\n   ```bash\n   pip install -r requirements.txt -r rag/requirements.txt\n   ```\n3. **Run Flask Server**\n   ```bash\n   cd rag\n   export FLASK_APP=main.py       # Use 'set FLASK_APP=main.py' on Windows\n   flask run\n   ```\n4. **Running the Speech Assistant**\n   ```bash\n   python speech_tasks/speech.py\n   ```\n5. **Start Voice Assistant Server**\n   ```bash\n   python server.py\n   ```\n6. **Load the Chrome Extension**\n\t- Open Chrome and go to: ```chrome://extensions```\n\t- Enable Developer Mode (top right)\n\t- Click **“Load unpacked”**\n\t- Select the url_server directory (contains ```manifest.json```, ```popup.js```, etc.)\n7. **Using the Extension**\n    1. Click the extension icon in your Chrome toolbar.\n    2. The extension:\n        - Displays the current URL.\n        - Extracts all links on the current page.\n        - Sends this data to your running Flask server (`/process-links`).\n    3. If the voice assistant is active, say **“Hey Vista”** to start interaction.\n   \n\n\n## 💻 Technology Stack\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://go-skill-icons.vercel.app/\"\u003e\n    \u003cimg\n      src=\"https://go-skill-icons.vercel.app/api/icons?i=python,javascript,flask,langchain,ollama,git\"\n    /\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n\n## 👏 Contributing\nI would love your help! Contribute by forking the repo and opening pull requests.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsharukat%2Fvista-meta-hackathon","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsharukat%2Fvista-meta-hackathon","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsharukat%2Fvista-meta-hackathon/lists"}