{"id":24562509,"url":"https://github.com/openai/openai-realtime-agents","last_synced_at":"2025-05-13T20:18:59.406Z","repository":{"id":273834786,"uuid":"917437058","full_name":"openai/openai-realtime-agents","owner":"openai","description":"This is a simple demonstration of more advanced, agentic patterns built on top of the Realtime API.","archived":false,"fork":false,"pushed_at":"2025-05-06T18:45:42.000Z","size":985,"stargazers_count":5500,"open_issues_count":25,"forks_count":651,"subscribers_count":115,"default_branch":"main","last_synced_at":"2025-05-06T19:48:03.655Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/openai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-16T01:29:28.000Z","updated_at":"2025-05-06T18:45:40.000Z","dependencies_parsed_at":null,"dependency_job_id":"8c76d33c-a39b-4bae-b6c9-5360f6651009","html_url":"https://github.com/openai/openai-realtime-agents","commit_stats":null,"previous_names":["openai/openai-realtime-agents"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openai%2Fopenai-realtime-agents","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openai%2Fopenai-realtime-agents/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openai%2Fopenai-realtime-agents/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openai%2Fopenai-realtime-agents/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/openai","download_url":"https://codeload.github.com/openai/openai-realtime-agents/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254020659,"owners_count":22000757,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-23T09:02:28.367Z","updated_at":"2025-05-13T20:18:59.400Z","avatar_url":"https://github.com/openai.png","language":"TypeScript","funding_links":[],"categories":["others","TypeScript","Openai","HarmonyOS","Frameworks \u0026 Platforms | 框架与平台","排行榜 [2025-03-18]","AI \u0026 LLM","📦 Official Examples","🌐 Web Development - Frontend"],"sub_categories":["Windows Manager","Comprehensive Frameworks | 综合性框架","Agents \u0026 Orchestration"],"readme":"# Realtime API Agents Demo\n\nThis is a simple demonstration of more advanced, agentic patterns built on top of the Realtime API. In particular, this demonstrates:\n- Sequential agent handoffs according to a defined agent graph (taking inspiration from [OpenAI Swarm](https://github.com/openai/swarm))\n- Background escalation to more intelligent models like o1-mini for high-stakes decisions\n- Prompting models to follow a state machine, for example to accurately collect things like names and phone numbers with confirmation character by character to authenticate a user.\n\nHere's a quick [demo video](https://x.com/OpenAIDevs/status/1880306081517432936) if you'd like a walkthrough. You should be able to use this repo to prototype your own multi-agent realtime voice app in less than 20 minutes!\n\n![Screenshot of the Realtime API Agents Demo](/public/screenshot.png)\n\n## Setup\n\n- This is a Next.js typescript app\n- Install dependencies with `npm i`\n- Add your `OPENAI_API_KEY` to your env. Either add it to your `.bash_profile` or equivalent file, or copy `.env.sample` to `.env` and add it there.\n- Start the server with `npm run dev`\n- Open your browser to [http://localhost:3000](http://localhost:3000) to see the app. It should automatically connect to the `simpleExample` Agent Set.\n\n## Configuring Agents\nConfiguration in `src/app/agentConfigs/simpleExample.ts`\n```javascript\nimport { AgentConfig } from \"@/app/types\";\nimport { injectTransferTools } from \"./utils\";\n\n// Define agents\nconst haikuWriter: AgentConfig = {\n  name: \"haikuWriter\",\n  publicDescription: \"Agent that writes haikus.\", // Context for the agent_transfer tool\n  instructions:\n    \"Ask the user for a topic, then reply with a haiku about that topic.\",\n  tools: [],\n};\n\nconst greeter: AgentConfig = {\n  name: \"greeter\",\n  publicDescription: \"Agent that greets the user.\",\n  instructions:\n    \"Please greet the user and ask them if they'd like a Haiku. If yes, transfer them to the 'haiku' agent.\",\n  tools: [],\n  downstreamAgents: [haikuWriter],\n};\n\n// add the transfer tool to point to downstreamAgents\nconst agents = injectTransferTools([greeter, haikuWriter]);\n\nexport default agents;\n```\n\nThis fully specifies the agent set that was used in the interaction shown in the screenshot above.\n\n### Next steps\n- Check out the configs in `src/app/agentConfigs`. The example above is a minimal demo that illustrates the core concepts.\n- [frontDeskAuthentication](src/app/agentConfigs/frontDeskAuthentication) Guides the user through a step-by-step authentication flow, confirming each value character-by-character, authenticates the user with a tool call, and then transfers to another agent. Note that the second agent is intentionally \"bored\" to show how to prompt for personality and tone.\n- [customerServiceRetail](src/app/agentConfigs/customerServiceRetail) Also guides through an authentication flow, reads a long offer from a canned script verbatim, and then walks through a complex return flow which requires looking up orders and policies, gathering user context, and checking with `o1-mini` to ensure the return is eligible. To test this flow, say that you'd like to return your snowboard and go through the necessary prompts!\n\n### Defining your own agents\n- You can copy these to make your own multi-agent voice app! Once you make a new agent set config, add it to `src/app/agentConfigs/index.ts` and you should be able to select it in the UI in the \"Scenario\" dropdown menu.\n- To see how to define tools and toolLogic, including a background LLM call, see [src/app/agentConfigs/customerServiceRetail/returns.ts](src/app/agentConfigs/customerServiceRetail/returns.ts)\n- To see how to define a detailed personality and tone, and use a prompt state machine to collect user information step by step, see [src/app/agentConfigs/frontDeskAuthentication/authentication.ts](src/app/agentConfigs/frontDeskAuthentication/authentication.ts)\n- To see how to wire up Agents into a single Agent Set, see [src/app/agentConfigs/frontDeskAuthentication/index.ts](src/app/agentConfigs/frontDeskAuthentication/index.ts)\n- If you want help creating your own prompt using these conventions, we've included a metaprompt [here](src/app/agentConfigs/voiceAgentMetaprompt.txt), or you can use our [Voice Agent Metaprompter GPT](https://chatgpt.com/g/g-678865c9fb5c81918fa28699735dd08e-voice-agent-metaprompt-gpt)\n\n### Customizing Output Guardrails\nAssistant messages are checked for safety and compliance using a guardrail function before being finalized in the transcript. This is implemented in [`src/app/hooks/useHandleServerEvent.ts`](src/app/hooks/useHandleServerEvent.ts) as the `processGuardrail` function, which is invoked on each assistant message to run a moderation/classification check. You can review or customize this logic by editing the `processGuardrail` function definition and its invocation inside `useHandleServerEvent`.\n\n## UI\n- You can select agent scenarios in the Scenario dropdown, and automatically switch to a specific agent with the Agent dropdown.\n- The conversation transcript is on the left, including tool calls, tool call responses, and agent changes. Click to expand non-message elements.\n- The event log is on the right, showing both client and server events. Click to see the full payload.\n- On the bottom, you can disconnect, toggle between automated voice-activity detection or PTT, turn off audio playback, and toggle logs.\n\n## Core Contributors\n- Noah MacCallum - [noahmacca](https://x.com/noahmacca)\n- Ilan Bigio - [ibigio](https://github.com/ibigio)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenai%2Fopenai-realtime-agents","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopenai%2Fopenai-realtime-agents","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenai%2Fopenai-realtime-agents/lists"}