{"id":50876681,"url":"https://github.com/idcnys/jarvis","last_synced_at":"2026-06-15T10:31:12.667Z","repository":{"id":358099390,"uuid":"1239923991","full_name":"idcnys/jarvis","owner":"idcnys","description":"A powerful, local-first AI orchestration layer that unifies advanced LLM reasoning, real-time voice synthesis, and local system automation. Featuring a Flask-based web interface, this system uses a smart Gemini API rotation strategy with a Groq fallback.","archived":false,"fork":false,"pushed_at":"2026-05-15T18:04:36.000Z","size":102,"stargazers_count":0,"open_issues_count":2,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2026-05-15T20:07:42.289Z","etag":null,"topics":["ai-assistant","flask","gemini-api","groq-api","jarvis-ai","kokoro-82m","orchestration"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/idcnys.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-15T15:28:27.000Z","updated_at":"2026-05-15T18:04:43.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/idcnys/jarvis","commit_stats":null,"previous_names":["idcnys/jarvis"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/idcnys/jarvis","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/idcnys%2Fjarvis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/idcnys%2Fjarvis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/idcnys%2Fjarvis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/idcnys%2Fjarvis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/idcnys","download_url":"https://codeload.github.com/idcnys/jarvis/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/idcnys%2Fjarvis/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34357285,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-15T02:00:07.085Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-assistant","flask","gemini-api","groq-api","jarvis-ai","kokoro-82m","orchestration"],"created_at":"2026-06-15T10:31:11.903Z","updated_at":"2026-06-15T10:31:12.663Z","avatar_url":"https://github.com/idcnys.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Project Overview\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"#-features\"\u003e🚀 Features\u003c/a\u003e • \n  \u003ca href=\"#️-tech-stack\"\u003e🛠️ Tech Stack\u003c/a\u003e • \n  \u003ca href=\"#-prerequisites\"\u003e📋 Prerequisites\u003c/a\u003e  • \n  \u003ca href=\"#-setup\"\u003e📋 Setup\u003c/a\u003e   • \n    \u003ca href=\"#-architecture\"\u003e📋 Architecture\u003c/a\u003e\n\u003c/p\u003e\n\n### AI Orchestration Layer with Flask Web UI\n\nA powerful, local-first AI orchestration layer that unifies advanced LLM reasoning, real-time voice synthesis, and local system automation. Featuring a Flask-based web interface, this system uses a smart Gemini API rotation strategy with a Groq fallback, allowing it to seamlessly perform CRUD operations on files, automate OS tasks via PyAutoGUI, tell jokes, and act as a fully capable local AI assistant.\n\n\n\u003cimg width=\"1920\" height=\"913\" alt=\"image\" src=\"https://github.com/user-attachments/assets/929eef8b-583d-4fe6-b982-df7f7bdad421\" /\u003e\n\n\n## 🚀 Features\n\n### 🧠 Intelligent Brain \u0026 Routing\n* **Gemini API Rotator:** Automatically rotates through multiple Gemini API keys to maximize rate limits and prevent service interruptions.\n* **Groq API Fallback:** Instantly falls back to Groq's ultra-low latency API if all Gemini keys are exhausted or rate-limited.\n\n### 🎙️ Audio \u0026 Voice\n* **Kokoro-82M Voice Synthesis:** Integrated with the lightweight, highly efficient Kokoro-82M model for natural, low-latency text-to-speech (TTS) responses.\n\n### 💻 System Automation (OS Agent)\n* **PyAutoGUI Simulation:** Grants the AI the ability to interact with the host OS—opening/closing applications, typing, and taking screenshots.\n* **File \u0026 Folder CRUD:** Full workspace management allowing the agent to create, read, update, and delete files or directories safely via natural language.\n\n### 🌐 Web Interface\n* **Flask Web UI:** A clean, responsive front-end dashboard to interact with the assistant, view logs, and monitor system tasks in real-time.\n\n---\n\n## 🛠️ Tech Stack\n\n* **Backend Framework:** Flask (Python)\n* **LLM Providers:** Google Gemini API, Groq API\n* **TTS Engine:** Kokoro-82M Model\n* **OS Automation:** PyAutoGUI,keyboard, Python `os` \u0026 `shutil` libraries\n* **Frontend:** HTML5, CSS3, JavaScript (Fetch API / WebSockets)\n\n---\n\n## 📋 Prerequisites\n\nBefore setting up, ensure you have the following installed on your host machine:\n* Python 3.10 or higher\n* Pip (Python package manager)\n* *Note for Linux users:* PyAutoGUI may require `scrot`, `python3-tk`, and `python3-dev` installed via your system package manager. Besides `PyAutoGUI` does not work properly in wayland [use x11 instead]\n\n## 📋 Setup\n\n## Automatic (Recommended)\n* Clone this repository locally using `git clone https://github.com/idcnys/jarvis.git .` command\n* Create a virtual env using `python -m venv env` , activate it `env\\Scripts\\activate.bat` then install the requirements using `pip install -r requirements.txt`\n* Run `python setup.py` it will download the necessary files and create missing files and directories.\n* Run `python setup_api.py` ang give the gemini and groq api keys. ([get Gemini API key here](https://aistudio.google.com/app/api-keys) ) and a groq API key for fallback ([get groq API KEY here](https://console.groq.com/keys))\n* Run `python server.py` \n\n\n## Manual\n\n* Clone this repository locally using `git clone https://github.com/idcnys/jarvis.git .` command\n* Create a virtual env using `python -m venv env` , activate it `env\\Scripts\\activate.bat` then install the requirements using `pip install -r requirements.txt`\n* Download kokoro-82M model files from their github repo. file names ` kokoro-v1.0.onnx` and ` voices-v1.0.bin` [download from here](https://github.com/thewh1teagle/kokoro-onnx/releases/tag/model-files-v1.0) and put them in the `voice_files/` folder\n* Add your own gemini API_KEY(S) ([get Gemini API key here](https://aistudio.google.com/app/api-keys) ) and a groq API key for fallback ([get groq API KEY here](https://console.groq.com/keys)), (You can also connect local LLM), make sure that you have added the gemini API keys in the `user_data/APIs.txt` file `one per line` and groq API key in the `user_data/groq_config.txt` file. In this format `{ \"api_key\": \"YOUR KEY\", \"model_name\": \"model that you prefer\" }` .For groq i prefer `llama-3.3-70b-versatile` it is better for tool calling.\n* Once you have done everything right and your folder structure looks like the given structure. you can run the app using `env\\Scripts\\python.exe server.py` or directly inside the virtual env `python server.py` then open the link in the browser. You can also access the link `192.168.0.102:5000 ( something looks like this)` from your other devices connected to your wifi as well.\n* Make sure to add your own api keys as well as keydict. run `key_test.py` to get the hidden keycodes and add them in the `user_data/keydict.txt`.\n* if you face any kind of setup issue you can ask me in the [discussion](https://github.com/idcnys/jarvis/discussions) or you can submit [an issure here](https://github.com/idcnys/jarvis/issues) as well.\n\n##  After you have done everything your folder should look like this\n\n```\nrequirements.txt\nserver.py\nconstants/\ncurrent/\n    API.txt\n    exhausted.txt\nenv/\nhelpers/\nmemory/\n    skills.json\nstatic/\n    css/\n    js/\ntemplates/\nuser_data/\n    APIs.txt\n    groq_config.txt\n    history.json\n    instructions.txt\n    keydict.txt\nvariables/\nvoice_files/\n    kokoro-v1.0.onnx\n    voices-v1.0.bin\nworkspace/\n```\n\nThis README provides an overview of the folder structure for the project. Each folder and file serves a specific purpose in the application. For further details, refer to the respective files and directories.\n\n## 📋 Architecture\n\n\u003cimg width=\"1333\" height=\"711\" alt=\"image\" src=\"https://github.com/user-attachments/assets/dcc83476-48cb-4e23-8ede-61d9915a71d6\" /\u003e\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fidcnys%2Fjarvis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fidcnys%2Fjarvis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fidcnys%2Fjarvis/lists"}