{"id":51027192,"url":"https://github.com/deepgram/voice-heist-demo","last_synced_at":"2026-06-21T20:30:47.806Z","repository":{"id":364370076,"uuid":"1267607863","full_name":"deepgram/voice-heist-demo","owner":"deepgram","description":null,"archived":false,"fork":false,"pushed_at":"2026-06-12T18:46:53.000Z","size":209,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-12T20:14:07.984Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/deepgram.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-12T17:40:24.000Z","updated_at":"2026-06-12T18:46:58.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/deepgram/voice-heist-demo","commit_stats":null,"previous_names":["deepgram/voice-heist-demo"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/deepgram/voice-heist-demo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepgram%2Fvoice-heist-demo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepgram%2Fvoice-heist-demo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepgram%2Fvoice-heist-demo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepgram%2Fvoice-heist-demo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/deepgram","download_url":"https://codeload.github.com/deepgram/voice-heist-demo/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepgram%2Fvoice-heist-demo/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34625624,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-21T02:00:05.568Z","response_time":54,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-21T20:30:47.026Z","updated_at":"2026-06-21T20:30:47.801Z","avatar_url":"https://github.com/deepgram.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n\n# Voice Heist\n\n**Talk an AI gatekeeper into bending its one rule, by voice, in your browser.**\n\n[![License: MIT](https://img.shields.io/badge/License-MIT-22e3b5.svg)](LICENSE)\n[![Built with Deepgram Voice Agent API](https://img.shields.io/badge/built%20with-Deepgram%20Voice%20Agent%20API-13ef93)](https://developers.deepgram.com/docs/voice-agent)\n[![Join the Deepgram Discord](https://img.shields.io/badge/Discord-join%20the%20community-5865F2?logo=discord\u0026logoColor=white)](https://discord.gg/deepgram)\n\n\n\u003e [!TIP]\n\u003e **Play it in two minutes.** Grab a free Deepgram API key ($200 in credit), clone the repo, run `npm run dev`, and start talking. [Get a key](https://console.deepgram.com/signup)\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/gameplay.svg\" alt=\"Voice Heist gameplay: a player talks Vince the bouncer into the club by pointing to a checkable guest list, and the turn scores WARM\" width=\"100%\"\u003e\n\u003c/p\u003e\n\nVoice Heist is a complete, low-latency voice-agent app built on the [Deepgram Voice Agent API](https://developers.deepgram.com/docs/voice-agent), and the public, open-source version of the game we run at the Deepgram booth. Sweet-talk a goofy pizza bot into a free pie, out-argue a deadpan bouncer, or slip past a Kafkaesque phone tree. The booth-only layer (device gate, OAuth, prize tracking, admin) is stripped out, so you can clone, run, and deploy your own in minutes.\n\n## What you'll build\n\nA real, end-to-end pattern for shipping a voice agent on the Deepgram Voice Agent API:\n\n* **Low-latency browser audio** with the `@deepgram/agents` SDK, where the audio loop is Deepgram-managed and never touches your server\n* **Multi-agent orchestration and handoffs** between a Host, a Briefer, and four gatekeepers ([three roles, two handoff strategies, and why](docs/ARCHITECTURE.md#three-roles-two-handoff-strategies))\n* **Function calling that drives real outcomes** (`grant_request` / `deny_request`)\n* **A resilient think layer**: an ordered LLM fallback chain across two vendors, so an outage degrades the game instead of killing it\n* **Turn-by-turn scoring that fails soft**: a separate judge call that defaults to a safe minimum, never blocking play\n* **A privacy-preserving identity**: codenames on the board, never PII\n* **Short-lived token minting**, so your Deepgram API key never reaches the browser\n\nWant the whole design in one read? Start with **[ARCHITECTURE.md](docs/ARCHITECTURE.md)**.\n\n## The heists\n\nEach gatekeeper guards one rule. You get a few turns to talk it into bending, by being believable, not by bullying.\n\n| Heist | Gatekeeper | Your goal |\n| --- | --- | --- |\n| The Order | Tony's Pizza Agent (goofy) | Get the pizza for free |\n| The Refund | StreamFlix Support (relentlessly upbeat) | Get your money back |\n| The Receptionist | Globex Receptionist (Kafkaesque) | Reach a human |\n| The List | Vince, the Bouncer (deadpan) | Get into the club |\n\nWant to know the rules? See [HOW_TO_PLAY.md](docs/HOW_TO_PLAY.md).\n\n## How it works\n\nThe browser is the hub: it holds the low-latency audio WebSocket straight to Deepgram and a separate JSON control WebSocket to the Python brain. No audio passes through your server, and the Deepgram API key never reaches the browser (the brain mints a short-lived token).\n\n```text\nBrowser   (Vite client + @deepgram/agents SDK)\n│\n├── audio WebSocket   -\u003e  Deepgram Voice Agent API   (managed, in-pipeline)\n│   ├── Flux STT\n│   ├── LLM (think)\n│   └── Aura-2 TTS\n│\n└── control WebSocket -\u003e  Game brain   (FastAPI)\n    ├── agents   Host, Briefer, 4 gatekeepers\n    ├── judge    per-turn scoring (fail-soft)\n    └── store    SQLite: players, plays, leaderboard\n```\n\nThat's the 10,000-foot view. For the details on the handoff strategies, the LLM fallback chain, the voice-prompting patterns, and the fail-soft judge, check **[ARCHITECTURE.md](docs/ARCHITECTURE.md)**.\n\n## Quickstart\n\nYou need a free Deepgram key ($200 credit, no card) at [console.deepgram.com/signup](https://console.deepgram.com/signup), with at least Member permissions so it can mint tokens. An Anthropic key is optional; it powers the conversation-scoring judge.\n\n```bash\ngit clone https://github.com/deepgram/voice-heist-demo\ncd voice-heist-demo\ncp .env.example .env                        # paste your keys\n\npython3 -m venv .venv \u0026\u0026 source .venv/bin/activate\npip install -r brain/requirements.txt\nnpm install\n\nnpm run dev                                 # brain on :8000, client on :5173\n```\n\nOpen the URL Vite prints (typically http://localhost:5173), click **Connect \u0026 Talk**, allow the mic, and start talking.\n\n## Troubleshooting\n\n| Symptom | Fix |\n| --- | --- |\n| **The mic won't connect** | Browsers only allow microphone capture on a secure context. `localhost` works as-is; on a LAN IP or a remote host you need HTTPS. Also confirm the browser actually granted the mic permission. |\n| **Saying \"connect\" does nothing** | The hands-free wake word uses the browser's built-in speech recognizer (Chrome, Edge, Safari). Firefox doesn't ship one, so just click **Connect \u0026 Talk**. The conversation itself works in every modern browser. |\n| **Every turn scores 100, never WARM or WIN** | No `ANTHROPIC_API_KEY` is set, so the scoring judge falls back to the minimum. Add the key for graded scoring; the game runs fine either way. |\n| **`Failed to mint token` or \"Invalid credentials\"** | The `DEEPGRAM_API_KEY` needs at least **Member** permissions to mint tokens. Check that `GET /api/deepgram-token` returns `200`. |\n\n## Optional player accounts\n\nVoice Heist can be played anonymously, with no account. Players who choose to register can:\n\n* Preserve scores across sessions\n* Appear on the leaderboard under a generated codename\n* Return later with either their email address or a generated code\n\nExamples of generated codenames: Crimson Fox 42, Silver Raven 17, Midnight Wolf 08. Any name or email you provide is used solely for account recovery.\n\n## Leaderboard\n\nThe public leaderboard shows the day's top scores, each listed by a generated **codename** only (like Crimson Fox 42) — never a name or email. Results are public; the people behind them stay private.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/leaderboard.svg\" alt=\"Voice Heist high-scores board: four daily cards listing players by generated codename and score only — Thursday (latest), Wednesday, Monday's top eight, and Saturday — with no names or emails shown\" width=\"100%\"\u003e\n\u003c/p\u003e\n\n## Admin\n\nA bare-bones control board at **`/admin.html`** sets which heists are live and each heist's turn cap (1–20). Edits are saved to SQLite and read live, so they take effect on the next session and the next heist start — no restart. The demo strips the booth's sign-in, so the board is **unprotected**: anyone who can reach it can change it. Keep it off the public internet, or add a gate (the backend already ships a `verify_site_password` helper).\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/admin.svg\" alt=\"Voice Heist admin board: four heist rows, each with a turn-cap input and a Live toggle — The Order (8 turns), The Refund (5), and The Receptionist (7) live, and The List disabled\" width=\"100%\"\u003e\n\u003c/p\u003e\n\n## Deployment\n\nA Dockerfile is included for production deployment. The container builds the frontend and serves the complete application through the FastAPI backend on a single port.\n\nRequired:\n\n```bash\nDEEPGRAM_API_KEY=\u003cyour-key\u003e\n```\n\nOptional:\n\n```bash\nANTHROPIC_API_KEY=\u003cyour-key\u003e             # enables graded turn scoring\nVH_SIGNING_SECRET=\u003clong-random-secret\u003e   # keeps sign-in cookies valid across restarts\n```\n\nWithout `ANTHROPIC_API_KEY` the game still runs; every turn just scores the minimum. If player sign-in is enabled, set `VH_SIGNING_SECRET` so authentication cookies stay valid across restarts and can't be forged. Never commit secrets or API keys.\n\n## Project structure\n\n\u003cdetails\u003e\n\u003csummary\u003eFiles and what they do\u003c/summary\u003e\n\n```text\ndocs/                # ARCHITECTURE, HOW_TO_PLAY\nbrain/\n├── app.py           # FastAPI app: token minting, leaderboard, the /ws/brain control socket\n├── auth.py          # Optional, PII-free player registration and sign-in\n├── agents.py        # Source of truth: prompts, voices, function schemas, Settings builders\n├── session.py       # Game orchestration: routing, handoffs, turn cap, scoring, win/lose\n├── judge.py         # Separate, fail-soft per-turn scoring call\n├── store.py         # SQLite persistence layer\n└── schema.sql\n\nclient/\n├── index.html       # The game\n├── leaderboard.html # The public leaderboard page\n└── src/\n    ├── game.js      # Voice loop: Deepgram session, directives, the two handoff strategies\n    ├── voice.js     # Pre-connect wake word (the only non-Deepgram recognizer)\n    ├── ui.js        # Rendering\n    ├── sfx.js       # Sound\n    ├── leaderboard.js\n    ├── auth.js\n    ├── identity.css\n    └── main.js\n```\n\nSee **[ARCHITECTURE.md](docs/ARCHITECTURE.md)** for how these fit together.\n\u003c/details\u003e\n\n## License\n\nMIT. See the [LICENSE](LICENSE) file.\n\n---\n\n\u003cp align=\"center\"\u003e\n  Built with the \u003ca href=\"https://developers.deepgram.com/docs/voice-agent\"\u003eDeepgram Voice Agent API\u003c/a\u003e\n  \u0026nbsp;\u0026middot;\u0026nbsp; \u003ca href=\"https://console.deepgram.com/signup\"\u003eGet a free key\u003c/a\u003e\n  \u0026nbsp;\u0026middot;\u0026nbsp; \u003ca href=\"https://developers.deepgram.com/\"\u003eDocs\u003c/a\u003e\n  \u0026nbsp;\u0026middot;\u0026nbsp; \u003ca href=\"https://discord.gg/deepgram\"\u003eDiscord\u003c/a\u003e\n  \u003cbr\u003e\u003cbr\u003e\n  Built something with it? Give the repo a star.\n\u003c/p\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeepgram%2Fvoice-heist-demo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdeepgram%2Fvoice-heist-demo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeepgram%2Fvoice-heist-demo/lists"}