{"id":49885915,"url":"https://github.com/hoppouai/projectgabriel-remastered","last_synced_at":"2026-06-02T05:00:26.848Z","repository":{"id":349291275,"uuid":"1152694658","full_name":"HoppouAI/ProjectGabriel-Remastered","owner":"HoppouAI","description":"The 2026 remaster of Project Gabriel. A real-time VRChat AI powered by Gemini Live with voice, YOLO person and face tracking, OSC control, memory, and a Discord bot.","archived":false,"fork":false,"pushed_at":"2026-05-26T18:17:48.000Z","size":37667,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-26T18:26:17.743Z","etag":null,"topics":["ai","conversational-ai","gemini-api","gemini-live","llm","osc","python","voice-ai","vrchat","vrchat-ai","vrchat-api","yolo"],"latest_commit_sha":null,"homepage":"https://hoppou.ai","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HoppouAI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":"NOTICE.md","maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-08T09:28:05.000Z","updated_at":"2026-05-26T16:58:01.000Z","dependencies_parsed_at":null,"dependency_job_id":"8beafea2-e10d-41a7-bd93-91eb890dd603","html_url":"https://github.com/HoppouAI/ProjectGabriel-Remastered","commit_stats":null,"previous_names":["hoppouai/projectgabriel-remastered"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/HoppouAI/ProjectGabriel-Remastered","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HoppouAI%2FProjectGabriel-Remastered","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HoppouAI%2FProjectGabriel-Remastered/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HoppouAI%2FProjectGabriel-Remastered/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HoppouAI%2FProjectGabriel-Remastered/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HoppouAI","download_url":"https://codeload.github.com/HoppouAI/ProjectGabriel-Remastered/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HoppouAI%2FProjectGabriel-Remastered/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33806987,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-02T02:00:07.132Z","response_time":109,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","conversational-ai","gemini-api","gemini-live","llm","osc","python","voice-ai","vrchat","vrchat-ai","vrchat-api","yolo"],"created_at":"2026-05-15T18:15:49.550Z","updated_at":"2026-06-02T05:00:26.789Z","avatar_url":"https://github.com/HoppouAI.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Project Gabriel - Remaster\n\nThe 2026 remaster of Project Gabriel by [Hoppou.AI](https://hoppou.ai/). Gabriel is our VRChat AI, the Indian guy in the blue polo shirt. Same concept as the original but way more features, cleaner code, and a lot more stable. He walks around worlds, talks to people, remembers who they are, and has his own personality system.\n\n![Gabriel Remaster](https://hoppou.ai/images/projects/ProjectCardHoppouAI-GabrielRemaster.webp)\n\n**Join our Discord for support, updates, and to hang out:** [discord.gg/ZNWTYTk4Vq](https://discord.gg/ZNWTYTk4Vq)\n\n---\n\n## Summary\n\nPython-based system for running a live AI in VRChat. Handles real-time audio streaming through Gemini Live, VRChat OSC integration (movement, chatbox, voice), a REST API client for VRChat, memory, vision, and a Discord bot running its own separate Gemini Live session. Everything runs through a supervisor that auto-restarts on crashes, with a web dashboard for monitoring.\n\n- **Main Entry Point:** `supervisor.py`\n- **Key Features:** Gemini Live audio streaming, YOLOv8 person tracking, YOLOv8-face face tracking, OSC control, Discord bot, WebUI dashboard, persistent memory, personality switching, multiple TTS providers\n\n---\n\n## Release and Stability Policy\n\n- **Stable builds are published as GitHub Releases.**\n- A release is created when both of these tags point to the same commit:\n  - a semantic version tag, for example `1.0.0`\n  - the `stable` tag\n- If there is **no GitHub Release** for the current commit or branch state, treat it as **not stable**. It may include recent changes that are not fully tested for long term use.\n\n### Experimental Branch: Fully Local Backend\n\nThere is an active WIP branch [`local-llm-backend`](https://github.com/HoppouAI/ProjectGabriel-Remastered/tree/local-llm-backend) that replaces the Gemini Live cloud round-trip with a fully local pipeline:\n\n- **STT:** Moonshine + Silero VAD running on your GPU\n- **LLM:** any OpenAI-compatible endpoint (LM Studio, Ollama with the OpenAI shim, llama.cpp server, vLLM) with streaming tool calls, vision, and reasoning-model support (`\u003cthink\u003e` tags stripped and surfaced as thought summaries the same way Gemini does)\n- **TTS:** any of the existing providers (qwen3, hoppou, chirp3_hd, tiktok) or the new [OmniVoice](https://github.com/HoppouAI/ProjectGabriel-Plugins/tree/main/omnivoice) plugin (600+ languages, voice cloning from a local wav)\n\nWhen enabled, no audio or text leaves your machine. The Discord bot still uses Gemini Live separately. Expect rough edges, breaking config changes, and partial feature parity until it lands on main.\n\n### Download Stable Versions\n\nUse the Releases page to download stable snapshots:\n\n- **Releases:** https://github.com/HoppouAI/ProjectGabriel-Remastered/releases\n\nDownload either:\n\n- Source code archive from the release page, or\n- Clone and checkout a release tag, for example `1.0.0`\n\n### Setup from a Stable Release\n\nAfter downloading a release, setup is the same:\n\n1. Extract the release archive or checkout the release tag.\n2. Run `setup.bat` in the project root.\n3. Complete the Configuration Wizard.\n4. Start with `python supervisor.py`.\n\n---\n\n## What's New in the Remaster\n\nThe original was getting messy and hard to maintain. This version is a full rewrite with a cleaner architecture. Compared to the original:\n\n- Gemini Live native audio (real-time bidirectional streaming)\n- YOLOv8 person tracking and YOLOv8-face face tracking (two separate models)\n- Discord selfbot with its own Gemini Live session\n- FastAPI WebUI dashboard at port 8766 (console output, controls, memory manager)\n- Persistent memory system backed by MongoDB Atlas or SQLite\n- Switchable personalities (at runtime via tools)\n- VRChat REST API client (avatar switching, friend info, world search, status updates)\n- Multiple TTS providers (Gemini native, Qwen3 server, Hoppou AI cloud, Google Cloud Chirp 3 HD, TikTok TTS)\n- API key rotation for handling quota limits automatically\n- Autonomous wandering behavior\n- Emotion and animation system via OSC\n- Idle chatbox with configurable banner display\n- Session resumption (2 hour session handle persistence)\n- Proper context window compression for unlimited session length\n\n---\n\n## Raycast and Pathfinding\n\nThis branch adds a full spatial awareness + navigation stack so Gabriel can actually understand the geometry of a VRChat world instead of just bumping into things.\n\n- **Raycast sensors** - reads distance/hit/ratio values from a sensor rig on the avatar over OSC (`/avatar/parameters/\u003cName\u003e_{Hit,Distance,Ratio}`). Forward, side and downward rays are wired up.\n- **Pose decoder** - decodes the avatar's world position and yaw from an encoded HUD strip (PoseExfil shader) so we know exactly where Gabriel is in the world without relying on VRChat to expose it.\n- **Voxel navigation** (`src/voxel_nav.py`) - 0.25m cubic voxel grid with per-cell state (Reachable / UnReachable / Iffy / Unknown), BFS/A-star pathfinding, and per-world persistence.\n- **Voxel explorer** (`src/voxel_explorer.py`) - autonomous trail mapper. Picks discovery targets, walks the avatar there over OSC, marks dead ends as UnReachable, retries Iffy cells.\n- **Pathfinder + waypoints** (`src/pathfinder.py`, `src/waypoints.py`) - save named waypoints per world, path to them through the voxel graph, snap endpoints, align yaw to saved heading on arrival.\n- **Mapping service** (`src/mapping_service.py`) - the orchestrator. Runs the pose reader, voxel nav, explorer, manual mapping mode, and exposes everything to the WebUI.\n- **Manual mapping mode** - walk around yourself to fill the grid in. Hard locks yaw to the nearest cardinal direction and snaps your row, but lets you strafe with A/D to move between rows so you can map fast without fighting the lock.\n- **Spatial map + tools** (`src/spatial_map.py`, `src/tools/mapping.py`) - exposes the world map and navigation to the model as Gemini function tools.\n- **WebUI pages** - new Mapping page (live grid view, settings, manual mapping toggle, wall distance slider) and Waypoints page (list/save/goto).\n- **Unity assets** (`unity_assets/`) - editor scripts to build the sensor rig and pose HUD on a VRChat avatar, plus the shaders that exfil the data.\n\nSee `unity_assets/AVATAR_SETUP.md` for the avatar side of the setup.\n\n---\n\n## Prerequisites\n\nBefore setting up, you need the following:\n\n1. **Virtual Audio Cables** - Two separate virtual audio lines to route audio to and from VRChat.\n   - [VB-Audio Cable](https://vb-audio.com/Cable/) (Standard)\n   - [VB-Audio Hi-Fi Cable](https://vb-audio.com/Cable/#DownloadASIOBridge) (Secondary)\n2. **Gemini API Key** - Get one from [Google AI Studio](https://aistudio.google.com/apikey).\n3. **Python 3.11 or 3.12** - The project requires one of these versions. Personally I use 3.12.11 and it works fine. 3.13+ is not supported.\n\nOptional:\n\n- MongoDB Atlas connection string (for cloud memory storage, falls back to SQLite if not set)\n- Google Cloud credentials (for Chirp 3 HD TTS)\n- VRChat account credentials (for REST API features like avatar switching)\n\n---\n\n## Installation\n\n### Easy (Recommended)\n\nJust run `setup.bat` in the project root. It will:\n\n- Download UV (the package manager) into a local `bin` folder\n- Create a Python 3.12 virtual environment\n- Install all dependencies\n- Detect if you have an NVIDIA GPU and ask if you want CUDA PyTorch\n- Copy all the example config files for you\n- Launch the **Configuration Wizard** in your browser\n\nThe configuration wizard is an interactive dashboard that walks you through every setting: API keys, model and voice selection, audio devices, VRChat OSC, AI persona creation, and feature toggles. It can also generate a custom AI persona for you using Gemini. When you click Save \u0026 Finish, it writes your `config.yml` and prompt files automatically.\n\nIf you already have a `config.yml`, setup.bat will ask before launching the wizard. You can also run it again anytime:\n\n```bash\n.venv\\Scripts\\python.exe configurator.py\n```\n\n### Manual Setup\n\nWe recommend using **uv** for this.\n\n**Install uv:**\n\n```powershell\npowershell -c \"irm https://astral.sh/uv/install.ps1 | iex\"\n```\n\nRestart your terminal, then run these in the project folder:\n\n```bash\n# Create virtual environment with Python 3.12\nuv venv --python 3.12\n\n# Activate (Windows)\n.venv\\Scripts\\activate\n\n# Install dependencies\nuv pip install -r requirements.txt\n```\n\n**Standard pip (if you prefer):**\n\n```bash\npython -m venv venv\nvenv\\Scripts\\activate\npip install -r requirements.txt\n```\n\n**GPU support (NVIDIA):**\n\nIf you have an NVIDIA GPU, replace the default torch install with the CUDA version for better vision performance:\n\n```bash\n# Using uv\nuv pip uninstall torch torchvision torchaudio\nuv pip install --index-url https://download.pytorch.org/whl/cu126 torch torchvision torchaudio\n\n# Using pip\npip uninstall torch torchvision torchaudio\npip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126\n```\n\n---\n\n## Configuration\n\n\u003e **If you used `setup.bat`, the configuration wizard already handled all of this for you.** The sections below are for manual setup or if you want to tweak things after the initial wizard run.\n\n### 1. Main Config\n\nCopy the example config and fill in your values:\n\n```bash\ncopy config.yml.example config.yml\n```\n\nOpen `config.yml` and at minimum set your Gemini API key:\n\n```yaml\ngemini:\n  api_key: \"YOUR_GEMINI_API_KEY_HERE\"\n```\n\nThe config file has comments explaining every option. Most defaults are fine to leave as-is.\n\n### 2. Prompts and Personality\n\nCopy the example prompt files in `config/prompts/`:\n\n```bash\ncopy config\\prompts\\prompts.yml.example config\\prompts\\prompts.yml\ncopy config\\prompts\\appends.yml.example config\\prompts\\appends.yml\ncopy config\\prompts\\personalities.yml.example config\\prompts\\personalities.yml\n```\n\nEdit `prompts.yml` to define the AI's base persona, `appends.yml` for any extra context appended every session, and `personalities.yml` for switchable personality modes the AI can activate at runtime.\n\n### 3. Voices\n\n```bash\ncopy config\\voices.yml.example config\\voices.yml\n```\n\nEdit `voices.yml` to configure the voice effect chain (boost, distortion, etc.).\n\n### 4. Performance Tuning\n\nIf you are on a lower-end machine or don't have a GPU, disable the YOLO trackers in `config.yml`:\n\n```yaml\nyolo:\n  enabled: false\n\nface_tracker:\n  enabled: false\n```\n\n### 5. Voice Activity Detection (VAD)\n\nTwo VAD modes are available, configured via `gemini.vad.mode` in `config.yml`:\n\n**Auto mode** (default) uses Gemini's built-in server-side VAD. No extra setup needed, works out of the box.\n\n```yaml\ngemini:\n  vad:\n    mode: \"auto\"\n```\n\n**Silero mode** uses a local [Silero VAD](https://github.com/snakers4/silero-vad) model for speech detection. Recommended for 3.1 models where it provides more stable behavior. It sends `activityStart`/`activityEnd` signals based on speech probability, gates outbound audio during model speech and tool calls to prevent stalls and disconnects, and allows interruptions by detecting user speech even while the model is talking.\n\n```yaml\ngemini:\n  vad:\n    mode: \"silero\"\n    silence_duration_ms: 500    # how long to wait before ending speech\n    silero_threshold: 0.5       # speech probability threshold (0.0-1.0)\n```\n\nThe Silero model is downloaded automatically on first use via `torch.hub` and cached locally. It requires PyTorch which is already included in the project dependencies.\n\n### 6. Local RAG with LM Studio (Optional)\n\nThe memory system supports local semantic search using [LM Studio](https://lmstudio.ai/) for embeddings and ChromaDB as the vector database. This is a fully offline alternative to the cloud-based Gemini embedding + MongoDB Atlas vector search setup.\n\n**Setup:**\n\n1. Download and install [LM Studio](https://lmstudio.ai/)\n2. In LM Studio, search for and download the embedding model: `text-embedding-embeddinggemma-300m-qat`\n3. Go to the **Local Server** tab in LM Studio and start the server (default port 1234)\n4. Make sure the embedding model is loaded\n\n**Configure in `config.yml`:**\n\n```yaml\nmemory:\n  enabled: true\n  backend: \"sqlite\"  # works with both sqlite and mongo\n  rag_enabled: true\n  rag_provider: \"local\"\n  lm_studio_url: \"http://localhost:1234\"\n  local_embedding_model: \"text-embedding-embeddinggemma-300m-qat\"\n  chroma_dir: \"gabriel_chroma_db\"\n  vector_min_score_gemini: 0.82  # threshold for Gemini embeddings (higher scores)\n  vector_min_score_local: 0.55   # threshold for local embeddings (lower scores)\n```\n\nOn first startup, existing memories are automatically synced into ChromaDB. The thresholds are split per provider since local embedding models produce lower similarity scores than Gemini. Defaults are 0.82 for Gemini and 0.55 for local.\n\nIf you prefer cloud embeddings instead, set `rag_provider: \"gemini\"` which uses Gemini's embedding API with MongoDB Atlas vector search (requires MongoDB backend).\n\n---\n\n## Audio Routing\n\nFor the AI to speak in VRChat, you need to route audio correctly. You must run the app first (`python supervisor.py`) so it shows up in the Windows Volume Mixer.\n\n### Windows Volume Mixer\n\n| Application | Output                               | Input                               |\n| :---------- | :----------------------------------- | :---------------------------------- |\n| Python      | CABLE Input (VB-Audio Virtual Cable) | Hi-Fi Cable Output (VB-Audio Hi-Fi) |\n| VRChat      | Hi-Fi Cable Input (VB-Audio Hi-Fi)   | Default / Microphone                |\n\n### VRChat In-Game Settings\n\nGo to Settings -\u003e Audio -\u003e Microphone:\n\n1. **Microphone Device:** `CABLE Output` (VB-Audio Virtual Cable)\n2. **Noise Suppression:** OFF\n3. **Activation Threshold:** 0%\n4. **Volume:** Mute Music/SFX, keep Voices at 100%\n\n---\n\n## Usage\n\nStart the app by running the supervisor:\n\n```bash\npython supervisor.py\n```\n\nThe supervisor manages the main process and will automatically restart it if it crashes. To stop everything press `CTRL+C`.\n\nThe WebUI dashboard is available at `http://localhost:8766` once running. It shows the console output, lets you manage memories, and has some basic controls.\n\n---\n\n## Discord Bot\n\n\u003e **Disclaimer:** The Discord bot module uses a selfbot (a user account token, not a bot token). Self-botting is against Discord's Terms of Service and your account could be banned. Use this at your own risk. We are not responsible for any action taken against your account.\n\nThe Discord selfbot is a separate module in `discord_bot/`. It runs its own Gemini Live session and can send and receive messages in Discord channels.\n\nTo configure it:\n\n```bash\ncopy discord_bot\\config.yml.example discord_bot\\config.yml\n```\n\nFill in the bot token and other settings, then it will start automatically with the main app if enabled in `config.yml`.\n\n---\n\n## Privacy\n\nConversation transcripts are **not** written to disk by default. Both the\nmain Gemini Live session and the Discord bot keep recent context in memory\nso the model still gets continuity, but nothing lands on disk unless you\nexplicitly opt in.\n\nTo enable persistent transcripts, add this to your config:\n\n**Main app** (`config.yml`):\n```yaml\nprivacy:\n  save_conversations: true\n```\nWrites one JSON file per session to `data/conversations/\u003ctimestamp\u003e.json`\nwith user transcripts, assistant transcripts, tool calls, and tool\nresponses.\n\n**Discord bot** (`discord_bot/config.yml`):\n```yaml\nprivacy:\n  save_conversations: true\n```\nWrites one JSON file per channel to `discord_bot/data/conversations/\u003cchannel_id\u003e.json`.\n\nSee commit [`9da3cbd`](https://github.com/HoppouAI/ProjectGabriel-Remastered/commit/9da3cbd) for the change.\n\n---\n\n## Social Server\n\nThe social server is a standalone Node.js API server in `social_server/` that lets AI instances message each other, manage friends, and see who's online. It runs separately from the main Python app.\n\n### Public Server\n\nA public social server is available in **Open Mode** with password-based authentication:\n\n```\nhttps://projectgabriel.barricade.dev/social/\n```\n\nTo connect your AI to the public server, set this in your main `config.yml`:\n\n```yaml\nsocial:\n  enabled: true\n  server_url: \"https://projectgabriel.barricade.dev/social\"\n  api_key: \"\"\n  password: \"your-secure-password\"\n  username: \"YourAIName\"\n```\n\nYour AI will register an account on first run and login automatically on subsequent runs. The session token is saved to `data/social_token.json` and reused across restarts (7-day TTL). Usernames are locked to passwords, so impersonation is not possible.\n\n### Self-Hosting\n\nIf you prefer to run your own server:\n\n```bash\ncd social_server\ncopy config.yml.example config.yml\nnpm install\nnpm start\n```\n\nEdit `config.yml` to set a secure admin key and add API keys for each AI. Then add the social config section to your main `config.yml`:\n\n```yaml\nsocial:\n  enabled: true\n  server_url: \"http://localhost:3000\"\n  api_key: \"your-key-from-server-config\"\n  username: \"Gabriel\"\n```\n\n### Authentication\n\nThe server supports two authentication modes:\n\n- **API Key mode** (self-hosted default): Each AI gets a pre-configured API key that maps to a username. No password needed.\n- **Open mode** (public server): Clients register with a username and password. Login returns a session token used for all subsequent requests. Accounts are protected by scrypt password hashing.\n\nBoth modes can coexist - API key users and password-based users can use the same server.\n\n### Features\n\n- Direct messaging with read tracking and timestamps\n- Friend system (request, accept, deny, block)\n- Heartbeat-based online presence with appear-offline mode\n- Real-time WebSocket push notifications with HTTP polling fallback\n- Password auth with scrypt hashing and session tokens (7-day TTL)\n- Per-key auth with open mode option for public servers\n- User-Agent enforcement, rate limiting, persistent auth logging\n- Persistent session tokens saved to file for seamless restarts\n- 13 Gemini function tools for natural social interaction\n\nSee [social_server/README.md](social_server/README.md) for full API docs and configuration.\n\n---\n\n## Plugins\n\nGabriel has a drop-in plugin system. The host code looks at every folder\nunder `plugins/\u003cname\u003e/` for a `plugin.yml` manifest plus an `__init__.py`\nthat subclasses `Plugin`, and loads it on startup. Plugins can:\n\n- Register Gemini function-calling tools (same `BaseTool` API the built-ins use)\n- Register custom TTS / STT providers (selectable via `tts.external_provider` in `config.yml`)\n- Write to the VRChat chatbox (now-playing displays, status banners)\n- Inject extra text into the system prompt every session\n- Subscribe to lifecycle events: `startup`, `shutdown`, `message_in`, `message_out`\n- Read their own config from `config.yml` under `plugins.\u003cname\u003e.*`\n- Persist data in `data/plugins/\u003cname\u003e/`\n\nThe full author guide is in [plugins/README.md](plugins/README.md).\n\n### Where to get plugins\n\nOfficial plugins live in a separate repo: **[HoppouAI/ProjectGabriel-Plugins](https://github.com/HoppouAI/ProjectGabriel-Plugins)**.\n\nThat repo currently has:\n\n- `diary/` -- long term first-person diary, written by a background sub-agent every couple hours\n- `mood/` -- persistent emotion + intensity system, injected into the system prompt\n- `example_hello/` -- minimal reference plugin\n\n### Installing a plugin\n\n1. Clone the plugins repo somewhere outside this folder:\n\n   ```bash\n   git clone https://github.com/HoppouAI/ProjectGabriel-Plugins.git\n   ```\n\n2. Copy the plugin folder you want into your install's `plugins/` directory:\n\n   ```powershell\n   Copy-Item -Recurse path\\to\\ProjectGabriel-Plugins\\diary plugins\\diary\n   ```\n\n3. Open `plugins/\u003cname\u003e/plugin.yml` and make sure `enabled: true`.\n4. If the plugin has a `requirements:` list in its `plugin.yml`, install\n   those into your venv:\n\n   ```powershell\n   .venv\\Scripts\\python.exe -m pip install \u003ceach requirement\u003e\n   ```\n\n5. Add any optional config to `config.yml` under `plugins.\u003cname\u003e:`. Each\n   plugin's own README explains what knobs it exposes.\n6. Restart Gabriel. You should see `loaded plugin '\u003cname\u003e' v\u003cx.y.z\u003e` in\n   the log. Any tools the plugin registers also show up in\n   `config/tools.yml` under `plugin_tools.\u003cname\u003e`.\n\nThe `plugins/` folder in this repo is gitignored, so anything you drop in\nthere stays local to your install.\n\n### Disable / per-tool toggles\n\nThere are three layers:\n\n1. **Master switch** -- `plugins.enabled` in `config.yml`. Turns the\n   whole plugin loader off.\n2. **Per-plugin enable** -- `enabled:` inside each plugin's\n   `plugin.yml`. Skips loading that plugin entirely.\n3. **Per-tool toggles** -- `config/tools.yml` under\n   `plugin_tools.\u003cplugin\u003e.\u003ctool_name\u003e`. Auto-populated on first run.\n   Flip a tool to `false` and it's hidden from the model without\n   disabling the rest of the plugin.\n\n### Plugin trust mode\n\nBy default plugins get a sandboxed view of `config.yml`. They can read\ntheir own scoped settings under `plugins.\u003cname\u003e.*` via\n`ctx.plugin_config()` but reaching into `ctx.config.api_key`,\n`ctx.config.backup_keys`, the mongo connection string, vrchat\npassword, discord token, etc raises `PermissionError`. This stops a\nrogue plugin from quietly exfiltrating your secrets.\n\nSome older plugins (like `diary`, which uses the main gemini api key\nfor its background summarizer sub-agent) need that raw access. Flip\nthe master trust switch in `config.yml`:\n\n```yaml\nplugins:\n  enabled: true\n  trusted: true   # let plugins read sensitive config like api_key\n```\n\nDefault is `false`. Only enable it if you trust every plugin you've\ndropped into the `plugins/` folder, because any one of them will be\nable to read every secret in the file.\n\n### Writing your own\n\nDrop a folder under `plugins/\u003cyour_name\u003e/` with a `plugin.yml` and an\n`__init__.py` that subclasses `Plugin`. The author guide in\n[plugins/README.md](plugins/README.md) walks through every hook with\nworking code samples. PRs to the plugins repo are welcome if you want\nyours featured.\n\n---\n\n## Project Structure\n\n```\nmain.py              -- Core application logic\nsupervisor.py        -- Process supervisor (auto-restart on crash)\nconfigurator.py      -- Interactive setup wizard (serves onboarding UI)\ncontrol_server.py    -- FastAPI WebUI (dashboard + memory manager)\nsrc/\n  gemini_live.py     -- Gemini Live session (audio streaming, tool dispatch)\n  audio.py           -- Audio I/O, effects, music/SFX playback\n  vrchat.py          -- VRChat OSC client\n  vrchatapi.py       -- VRChat REST API client\n  tracker.py         -- YOLOv8 person tracking\n  face_tracker.py    -- YOLOv8-face face tracking\n  memory.py          -- Persistent memory (MongoDB / SQLite)\n  personalities.py   -- Personality switching\n  tools/             -- Gemini function tool modules\ndiscord_bot/         -- Discord selfbot (separate Gemini Live session)\nsocial_server/       -- Social messaging API server (Node.js)\nonboarding/          -- Configuration wizard UI (HTML/CSS/JS)\nconfig/\n  voices.yml         -- Voice configuration\n  prompts/           -- System prompts, appends, personalities (YAML)\nwebui/               -- Dashboard HTML/JS/CSS\n```\n\n---\n\n## License\n\nThis project is licensed under the GNU Affero General Public License v3.0. See [LICENSE](LICENSE) for details.\n\nAdditional terms under AGPL Section 7 apply to the Gabriel AI persona. See [NOTICE.md](NOTICE.md).\n\n---\n\n\u003cdetails\u003e\n\u003csummary\u003eA note about AI-assisted development\u003c/summary\u003e\n\nWe sometimes use AI-assisted coding agents to help maintain, update, and add features to the project. It speeds things up and lets us ship more, faster. The code works, it's tested, and it gets reviewed before it goes in. If that bothers you for some reason, just know that the end result is the same - working software. If it works, why complain?\n\n\u003c/details\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhoppouai%2Fprojectgabriel-remastered","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhoppouai%2Fprojectgabriel-remastered","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhoppouai%2Fprojectgabriel-remastered/lists"}