{"id":43988706,"url":"https://github.com/vaheed/sip-ai-agent","last_synced_at":"2026-02-07T10:12:01.731Z","repository":{"id":286490334,"uuid":"961557458","full_name":"vaheed/sip-ai-agent","owner":"vaheed","description":"This project registers a Python SIP client as an extension in Asterisk/FreePBX and connects calls to OpenAI Voice Agent in real-time using WebSocket.","archived":false,"fork":false,"pushed_at":"2025-09-18T20:15:46.000Z","size":47818,"stargazers_count":10,"open_issues_count":2,"forks_count":12,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-18T21:22:38.625Z","etag":null,"topics":["agent","ai","asterisk","voip"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vaheed.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-04-06T18:55:42.000Z","updated_at":"2025-09-18T20:15:50.000Z","dependencies_parsed_at":"2025-04-06T20:19:44.897Z","dependency_job_id":"7c3c8f3a-fce8-4acd-ae44-df12c40e8dda","html_url":"https://github.com/vaheed/sip-ai-agent","commit_stats":null,"previous_names":["vaheed/sip-ai-agent"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/vaheed/sip-ai-agent","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vaheed%2Fsip-ai-agent","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vaheed%2Fsip-ai-agent/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vaheed%2Fsip-ai-agent/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vaheed%2Fsip-ai-agent/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vaheed","download_url":"https://codeload.github.com/vaheed/sip-ai-agent/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vaheed%2Fsip-ai-agent/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29192210,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-07T07:37:03.739Z","status":"ssl_error","status_checked_at":"2026-02-07T07:37:03.029Z","response_time":63,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agent","ai","asterisk","voip"],"created_at":"2026-02-07T10:12:01.255Z","updated_at":"2026-02-07T10:12:01.715Z","avatar_url":"https://github.com/vaheed.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)\n\n# OpenAI SIP Voice Agent\n\n## Table of contents\n- [Overview](#overview)\n- [Architecture at a glance](#architecture-at-a-glance)\n- [Call flow](#call-flow)\n- [Project layout](#project-layout)\n- [Quick start](#quick-start)\n  - [Requirements](#requirements)\n  - [Configure the environment](#configure-the-environment)\n  - [Validate and regenerate configs](#validate-and-regenerate-configs)\n  - [Run with Docker Compose](#run-with-docker-compose)\n  - [Run locally (optional)](#run-locally-optional)\n  - [Access the dashboard](#access-the-dashboard)\n- [Monitoring \u0026 observability](#monitoring--observability)\n- [Configuration reference](#configuration-reference)\n  - [Core credentials \u0026 AI session](#core-credentials--ai-session)\n  - [Audio pipeline](#audio-pipeline)\n  - [NAT traversal \u0026 media security](#nat-traversal--media-security)\n  - [Retry \u0026 resilience](#retry--resilience)\n  - [Monitor authentication](#monitor-authentication)\n- [Integration guides](#integration-guides)\n- [Operations \u0026 troubleshooting](#operations--troubleshooting)\n- [Development workflow](#development-workflow)\n- [CLI utilities \u0026 helper scripts](#cli-utilities--helper-scripts)\n- [Ports \u0026 deployment topology](#ports--deployment-topology)\n- [License](#license)\n\n## Overview\nOpenAI SIP Voice Agent registers as a SIP endpoint via PJSIP, bridges audio between your PBX and OpenAI’s realtime or legacy voice APIs, and streams responses back to callers without leaving your telephony domain.\n\nA built-in FastAPI monitor and React/Tailwind dashboard authenticate administrators, stream live call status, logs, and metrics over WebSockets, and surface runtime health while configuration stays in the `.env` file.\n\nStructured JSON logging with correlation IDs, safe reload tooling, and a strongly typed environment schema make the agent production-friendly from day one.\n\n## Architecture at a glance\n- **sip-agent** – Asyncio PJSIP client that answers calls, bridges RTP frames into OpenAI’s realtime or legacy WebSocket APIs, and tracks token usage for observability.\n- **Monitor service** – FastAPI app that validates `.env` updates, streams status, exposes metrics, and coordinates safe reloads once active calls drain.\n- **Dashboard** – React + Tailwind UI served by Nginx, connected to the monitor via an authenticated WebSocket for status and logs while configuration lives in `.env`.\n- **Observability layer** – Structured logging with correlation IDs and an in-memory metrics collector that publishes JSON snapshots on `/metrics`.\n- **Container packaging** – Multi-stage Docker build compiles the dashboard, installs PJSIP bindings, and is orchestrated via `docker-compose` for both development and production deployments.\n\n```\nPBX / SIP phones ──(SIP/RTP)──► sip-agent (PJSIP + asyncio)\n                               │\n                               ├── WebSocket ─► OpenAI realtime / legacy APIs\n                               │\n                               └── FastAPI monitor ──(REST + WebSocket)──► React dash (Nginx)\n```\n\n## Call flow\n1. **Startup \u0026 configuration** – The monitor boots first; if environment validation fails the agent exits with detailed errors. Setting `ENABLE_SIP=false` leaves the dashboard running without touching the PBX.\n2. **Registration** – PJSIP is initialised with jitter-buffer preferences, STUN/TURN/SRTP knobs, codec priorities, and authenticates to your registrar with digest credentials.\n3. **Call handling** – Incoming and outbound calls create an `AudioCallback` bridge that feeds PCM frames into asyncio queues while retaining correlation IDs for monitoring.\n4. **Realtime streaming** – When `OPENAI_MODE=realtime`, the agent opens a WebSocket to OpenAI, issues a `session.update` with model/voice/audio parameters, base64-encodes audio deltas, and commits the buffer once a turn completes.\n5. **Legacy streaming** – In legacy mode the agent forwards raw PCM frames to `/v1/audio/speech` and queues returned audio directly to the SIP leg.\n6. **Metrics \u0026 logs** – The monitor records call lifecycle events, retry counters, and aggregated token usage while structured logs carry correlation IDs end-to-end.\n7. **Dashboard updates** – Authenticated WebSocket subscribers receive status snapshots, call history, metrics, and log lines with automatic reconnect/backoff logic in the browser hook.\n8. **Edge serving** – Nginx serves the built dashboard assets and proxies `/api`, `/ws`, `/metrics`, `/healthz`, `/login`, and `/logout` back to the monitor for a cohesive admin surface.\n\n## Project layout\n- `app/` – Backend agent, configuration loader, monitor API, and static dashboard assets.\n- `web/` – React/Vite/Tailwind dashboard with hooks, components, tests, and build tooling.\n- `scripts/` – Utilities such as `install_pjsua2.py` that compile pjproject and Python bindings.\n- `deploy/` – Deployment assets including the Nginx reverse-proxy configuration used in container builds.\n- `tests/` – Pytest suite covering audio pipelines, monitor routes, configuration validation, and timers.\n- `Makefile` – One-liner targets for installing dependencies, linting, typing, testing, formatting, and environment validation.\n\n## Quick start\n\n### Requirements\n- Asterisk-based PBX (e.g., FreePBX or VICIdial) with an extension dedicated to the agent.\n- Docker and Docker Compose.\n- OpenAI API key with access to the `gpt-realtime` model for realtime mode.\n\n### Configure the environment\nClone the repo and copy the sample configuration:\n\n```bash\ngit clone https://github.com/vaheed/sip-ai-agent.git\ncd sip-ai-agent\ncp env.example .env\n```\n\nPopulate SIP credentials, OpenAI keys, and session preferences:\n\n```env\n# PBX\nSIP_DOMAIN=pbx.example.com\nSIP_USER=1001\nSIP_PASS=secret\n\n# OpenAI\nOPENAI_API_KEY=sk-...\nAGENT_ID=pmpt_...\n\n# Feature toggles \u0026 realtime session\nENABLE_SIP=true\nENABLE_AUDIO=true\nOPENAI_MODE=realtime\nOPENAI_MODEL=gpt-realtime\nOPENAI_VOICE=alloy\nOPENAI_TEMPERATURE=0.3\nSYSTEM_PROMPT=You are a helpful voice assistant.\n\n\n# Monitor authentication\nMONITOR_ADMIN_USERNAME=admin\nMONITOR_ADMIN_PASSWORD=admin\nMONITOR_SESSION_SECRET=change-me\n\n```\n\nCore settings are validated on startup through a Pydantic schema, so missing or malformed values surface immediately in logs and the dashboard.\n\n### Validate and regenerate configs\nCheck any `.env` file and regenerate `env.example` without touching your secrets:\n\n```bash\npython -m app.config validate --path .env\npython -m app.config sample --write --path env.example\nmake env-validate    # wraps the same validator\nmake env-sample\n```\n\nThe CLI accepts `--include-os` to merge process environment values during validation and prints detailed field-level errors on failure.\n\n### Run with Docker Compose\nBuild and run both containers locally:\n\n```bash\ndocker compose up --build\n```\n\nOr pull the published images:\n\n```bash\ndocker compose -f docker-compose.production.yml up -d\n```\n\n`sip-agent` exposes UDP 5060 for SIP and 16000–16100 for RTP, while `web` serves the dashboard on TCP 8080 and proxies control-plane endpoints back to the agent.\n\n### Run locally (optional)\nFor direct execution or CI, install dependencies and compile PJSIP bindings:\n\n```bash\npip install -r requirements.txt\npython scripts/install_pjsua2.py\n# or\nmake install\n```\n\nThe helper downloads pjproject 2.12, builds shared libraries, and installs the `pjsua2` module—mirroring the container build sequence.\n\n### Access the dashboard\nBrowse to `http://\u003cdocker-host\u003e:8080` and log in with `MONITOR_ADMIN_USERNAME` / `MONITOR_ADMIN_PASSWORD` (defaults to `admin` / `admin`). Update SIP, OpenAI, and feature settings directly in the `.env` file and restart the containers to apply changes.\n\n## Monitoring \u0026 observability\nThe dashboard summarises SIP registration state, active calls, token usage, realtime channel health, and live logs, with dark/light theme support and auto-refreshing WebSocket data feeds.\n\nThe browser hook handles authentication failures, reconnect backoff, and incremental log streaming so operators can keep a single page open during deployments.\n\nConfiguration validation runs on startup and via the CLI helpers so invalid `.env` values surface immediately in logs and the dashboard health summaries.\n\nStructured JSON logs (with correlation IDs) and the metrics endpoint expose retry counters, call durations, token usage, and realtime WebSocket health. Pull `/metrics` in JSON for dashboards or quick `curl | jq` inspection during incidents.\n\n## Configuration reference\n\n### Core credentials \u0026 AI session\n| Key(s) | Default | Notes |\n| --- | --- | --- |\n| `SIP_DOMAIN`, `SIP_USER`, `SIP_PASS` | *(required)* | PBX account details; empty values fail validation. |\n| `OPENAI_API_KEY`, `AGENT_ID` | *(required)* | API key and assistant ID used for all conversations. |\n| `ENABLE_SIP` | `true` | Disable to run only the dashboard/monitor. |\n| `ENABLE_AUDIO` | `true` | Disable to keep SIP signalling without bridging RTP. |\n| `OPENAI_MODE` | `legacy` | Set `realtime` to enable ultra-low-latency streaming. |\n| `OPENAI_MODEL` | `gpt-realtime` | Model advertised in the realtime session update. |\n| `OPENAI_VOICE` | `alloy` | Voice name for realtime synthesis. |\n| `OPENAI_TEMPERATURE` | `0.3` | Sampling temperature (0–2). |\n| `SYSTEM_PROMPT` | `You are a helpful voice assistant.` | Sent in each realtime session update. |\n\nDefaults originate from `env.example` and are enforced via the `Settings` dataclass.\n\n### Audio pipeline\n| Key | Default | Description |\n| --- | --- | --- |\n| `SIP_TRANSPORT_PORT` | `5060` | Local UDP port for SIP signalling. |\n| `SIP_PREFERRED_CODECS` | `PCMU,PCMA,opus` | Priority-ordered codec list; friendly names (e.g. `PCMU`, `opus`) are normalised to the codec IDs reported by PJSIP and unavailable codecs are ignored. |\n| `SIP_JB_MIN` / `SIP_JB_MAX` / `SIP_JB_MAX_PRE` | `0` | Jitter buffer bounds to trade latency for resilience. |\n\nThese values adjust PJSIP media configuration before registration.\n\n### NAT traversal \u0026 media security\n| Key | Default | Description |\n| --- | --- | --- |\n| `SIP_ENABLE_ICE` | `false` | Enable ICE negotiation for symmetric NAT environments. |\n| `SIP_ENABLE_TURN` | `false` | Relay RTP via TURN when direct paths fail. |\n| `SIP_STUN_SERVER` | *(blank)* | Optional STUN URI (e.g., `stun:stun.l.google.com:19302`). |\n| `SIP_TURN_SERVER`, `SIP_TURN_USER`, `SIP_TURN_PASS` | *(blank)* | TURN relay credentials. |\n| `SIP_ENABLE_SRTP` | `false` | Negotiate SRTP; combine with `SIP_SRTP_OPTIONAL=false` to enforce encrypted media only. |\n\nThe agent applies these during account creation, mapping to PJSIP NAT and SRTP configuration structures.\n\n### Retry \u0026 resilience\n| Key | Default | Description |\n| --- | --- | --- |\n| `SIP_REG_RETRY_BASE` / `SIP_REG_RETRY_MAX` | `2.0` / `60.0` | Exponential backoff window for registration retries. |\n| `SIP_INVITE_RETRY_BASE` / `SIP_INVITE_RETRY_MAX` | `1.0` / `30.0` | Retry cadence for outbound INVITEs on 4xx/5xx. |\n| `SIP_INVITE_MAX_ATTEMPTS` | `5` | Maximum INVITE attempts before marking the call failed. |\n\nRetry scheduling emits log events and increments metrics counters for visibility.\n\n### Monitor authentication\nMonitor defaults can be overridden via environment variables:\n\n- `MONITOR_ADMIN_USERNAME` / `MONITOR_ADMIN_PASSWORD` – Admin credentials (default `admin` / `admin`).\n- `MONITOR_SESSION_COOKIE` – Cookie name for sessions (default `monitor_session`).\n- `MONITOR_SESSION_TTL` – Session lifetime in seconds (default 86400).\n- `MONITOR_SESSION_SECRET` – Optional secret used to sign dashboard sessions (defaults to a value derived from the credentials).\n\n## Integration guides\n\n### FreePBX\n1. Create a PJSIP extension matching `SIP_USER` / `SIP_PASS`.\n2. Open UDP 16000–16100 on your firewall for RTP media.\n3. Add a dialplan entry (e.g., extension `5000`) that dials `PJSIP/${SIP_USER}@${SIP_DOMAIN}` and reload the dialplan. Dial `5000` from any internal phone to reach the assistant.\n\n### VICIdial\n1. Create a campaign or in-group that dials an internal extension (e.g., `5000`).\n2. Add a matching dialplan stanza that calls `PJSIP/${SIP_USER}@${SIP_DOMAIN}`.\n3. Route transfers or manual dials to that extension; the agent behaves like any SIP endpoint registered to the PBX.\n\n## Operations \u0026 troubleshooting\n- **SIP failure codes** – Map common responses (401, 403, 404/484, 415/488, 480/503, 500/502) to corrective actions using the dashboard log stream and metrics counters such as `register_retries` and `invite_retries`.\n- **Firewall planning** – Open UDP 5060 (or your configured port) plus 16000–16100 for RTP; disable SIP ALG where possible to avoid SDP rewriting.\n- **NAT \u0026 SRTP** – Toggle ICE, TURN, STUN, and SRTP entirely through environment variables—no code changes required.\n- **Metrics \u0026 logs** – Use `/metrics` for machine-readable health snapshots and rely on correlation IDs to trace a call across SIP events, WebSocket activity, and audio bridge state changes.\n\n## Development workflow\n- Run `make dev`, `make lint`, `make type`, `make test`, and `make format` to install toolchains, run Ruff, MyPy, Pytest, and formatting respectively; CI mirrors these commands.\n- The Python test suite covers realtime/legacy audio loops, monitor routes, configuration validation, and timer utilities—extend it when introducing new behaviours.\n- Frontend development lives under `web/`: `npm install`, `npm run dev`, and `npm run build` leverage Vite, Tailwind, ESLint, Vitest, and Playwright tooling defined in `package.json`.\n- Contribution guidelines, coding standards, and review expectations are documented in `CONTRIBUTING.md`.\n\n## CLI utilities \u0026 helper scripts\n- `python -m app.config validate --path .env [--include-os]` – Validate environment files against the schema with detailed field errors.\n- `python -m app.config sample --write --path env.example` – Regenerate the canonical sample file (printable to stdout).\n- `make env-validate` / `make env-sample` – Convenience wrappers around the same CLI.\n- `python scripts/install_pjsua2.py [--prefix PATH] [--force]` – Download and build pjproject with shared libs and the `pjsua2` Python module (skips rebuilding if already installed unless `--force` is provided).\n\n## Ports \u0026 deployment topology\n- `sip-agent` container: UDP 5060 for SIP, UDP 16000–16100 for RTP, TCP 8080 for the monitor API/WebSocket.\n- `web` container: TCP 8080 (host) serving the compiled dashboard via Nginx, proxying control-plane traffic to `sip-agent`. Production compose files pull the published images by default but respect `SIP_AGENT_IMAGE` / `SIP_DASHBOARD_IMAGE` overrides.\n- Nginx forwards `/api/`, `/ws/`, `/metrics`, `/healthz`, `/login`, `/logout`, and `/dashboard` routes to the monitor while serving static assets from `/usr/share/nginx/html`.\n\n## License\nReleased under the GNU General Public License v3.0.\n\n```\n\n_Source notes:_ Key behaviours—such as SIP registration, realtime bridging, environment validation, monitor routes, dashboard streaming, and deployment topology—are documented directly from `app/agent.py`, `app/config.py`, `app/monitor.py`, the React dashboard (`web/src`), the helper scripts, and deployment manifests cited above.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvaheed%2Fsip-ai-agent","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvaheed%2Fsip-ai-agent","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvaheed%2Fsip-ai-agent/lists"}