{"id":51468491,"url":"https://github.com/bulletinmybeard/agent-forge","last_synced_at":"2026-07-06T14:01:26.565Z","repository":{"id":364160043,"uuid":"1260758590","full_name":"bulletinmybeard/agent-forge","owner":"bulletinmybeard","description":"Self-hosted AI agent and RAG platform. Index your code, docs, and data into Qdrant and answer over them, run a tool-calling agent, and drive multi-step pipelines across pluggable LLM backends. Runs as a Docker Compose stack.","archived":false,"fork":false,"pushed_at":"2026-06-28T11:17:37.000Z","size":2054,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2026-06-28T12:07:12.489Z","etag":null,"topics":["ai-agent","bedrock","docker","fastapi","llm","ollama","python","qdrant","rag","redis","self-hosted"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bulletinmybeard.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":"docs/SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-05T20:56:50.000Z","updated_at":"2026-06-28T10:08:17.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/bulletinmybeard/agent-forge","commit_stats":null,"previous_names":["bulletinmybeard/agent-forge"],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/bulletinmybeard/agent-forge","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bulletinmybeard%2Fagent-forge","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bulletinmybeard%2Fagent-forge/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bulletinmybeard%2Fagent-forge/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bulletinmybeard%2Fagent-forge/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bulletinmybeard","download_url":"https://codeload.github.com/bulletinmybeard/agent-forge/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bulletinmybeard%2Fagent-forge/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35193679,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-06T02:00:07.184Z","response_time":106,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-agent","bedrock","docker","fastapi","llm","ollama","python","qdrant","rag","redis","self-hosted"],"created_at":"2026-07-06T14:01:24.504Z","updated_at":"2026-07-06T14:01:26.556Z","avatar_url":"https://github.com/bulletinmybeard.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AgentForge\n\n[![CI](https://github.com/bulletinmybeard/agent-forge/actions/workflows/ci.yml/badge.svg)](https://github.com/bulletinmybeard/agent-forge/actions/workflows/ci.yml)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](./LICENSE)\n[![Python 3.12+](https://img.shields.io/badge/python-3.12%2B-blue.svg)](https://www.python.org/)\n[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)\n[![Docker](https://img.shields.io/badge/stack-Docker%20Compose-2496ED?logo=docker\u0026logoColor=white)](./docker-compose.yml)\n\n\u003c!-- prettier-ignore --\u003e\n\u003e [!NOTE]\n\u003e **Experimental: a personal learning project.**\n\u003e AgentForge is active and rough in places, and not fully covered by tests.\n\u003e I built it to learn how AI fits into my daily work, and how coding agents like Gemini CLI, Codex, and Claude Code work under the hood: how they decide which tools to call, how they manage chat history and context, and where the sharp edges are.\n\u003e Expect breaking changes. Treat it as a sandbox, not a product.\n\nAgentForge is a self-hosted AI agent and RAG platform.\nIt indexes your code, docs, and data into a vector store and answers questions over them, runs a tool-calling agent (filesystem, shell, Docker, Git, SSH, web, ...), and drives multi-step pipelines, all over pluggable LLM backends selected per task.\n\nIt runs as a **Docker Compose stack**.\nThe full stack is the chat backend, RAG API, queue workers, Qdrant, Redis, and scraper sidecar. A [light preset](#light-mode) trims it to the chat backend + one worker for MacBook use.\n\n## Built on AgentForge\n\nThese projects use AgentForge as their backend and don't run without it:\n\n- [agent-forge-webui](https://github.com/bulletinmybeard/agent-forge-webui): a React SPA chat front-end supporting all agent modes, tool calls, and connectors in one UI, with the agent's think \u003e act \u003e observe loop streamed.\n- [agent-forge-ask-page](https://github.com/bulletinmybeard/agent-forge-ask-page): a Chrome side-panel extension — pick any element or scan the whole page, then ask an LLM to extract, summarise, list, translate, or audit it, with real tool calls (web fetch, file download).\n- [agent-forge-felix](https://github.com/bulletinmybeard/agent-forge-felix): an autonomous diagnose -\u003e fix -\u003e verify CLI agent for operational problems (Docker, disk, HTTP).\n\n## Features\n\n- **Backends**: Ollama (local + cloud relay), AWS Bedrock, and any OpenAI-compatible API (DeepInfra, OpenRouter, ...). Selected per role via named profiles. Switch the whole stack with one `provider_override`.\n- **Agent loop**: think -\u003e act -\u003e observe with tool calling, error recovery, and optional web-search escalation.\n- **Tools**: filesystem, shell, system info, Docker, Git, SSH, archives, network diagnostics, web search/fetch/render, media, code editing, macOS notifications, **Apple Reminders** (list/create/edit/complete/delete via `remindctl`), and more.\n- **RAG**: index OpenAPI/SQL schemas, source code, docs, and transcripts into Qdrant. Query with refinement, reranking, and dedup.\n- **Knowledge Database**: personal store for notes, references, documentation, attached documents, cheatsheets, and snippets. One-call ingest, semantic search, tag faceting, and smart updates (re-embeds only when content changes). Multi-collection routing (`X-Knowledge-Collection` header) separates the KB SPA and AgentForge Notes into independent Qdrant collections. Original attachment files (PDFs, etc.) are stored alongside extracted text and downloadable via the API.\n- **Connectors**: link external accounts as agent tools. Gmail, Drive, BigQuery, and YouTube through one Google OAuth client, plus GitLab and GitHub via personal access tokens. Multi-account, in-process, read-only by default.\n- **Pluggable**: add your own tools via a `register(registry)` entry point. No fork needed.\n- **Pipelines**: typed multi-step runner, parallel fan-out, and discovery.\n\n## Documentation\n\nOperator guides live in [`docs/`](docs/README.md):\n\n- [Stack architecture](docs/architecture.md): how the containers fit together: services, ports, worker localities, data stores, and request flow. **Start here.**\n- [HTTP API](docs/api.md): REST + the `/ws/chat` agent WebSocket, memory endpoints, the Knowledge Database (`/knowledge/*`), and the live OpenAPI spec.\n- [Modes](docs/modes.md): the `@mode` prefixes (built-in modes, custom agents, connectors) and when to use each.\n- [Tools](docs/tools.md): every built-in agent tool by category, plus locality and confirmation gates.\n- [Chunking and indexing into Qdrant](chunking/README.md): the mappers (OpenAPI, SQL/tbls, live DB, code, CLI docs, Markdown), the index pipeline, the `/indexer/*` + `/search/*` endpoints, and dedup/drift QA.\n- [Deploying with custom local domains](docs/local-domains.md): running the stack behind Traefik, the `deploy.env` knobs, and the split-host worker.\n- [Connectors](docs/connectors.md): linking external accounts. The unified Google OAuth connector (Gmail, Drive, BigQuery, YouTube) and the GitLab and GitHub token connectors.\n- [Authoring tools and private overlays](docs/plugin-authoring.md): adding private tools, the `AGENTFORGE_TOOL_PLUGINS` seam, and the local overlay files.\n- [Instruction markdown](markdown/README.md): the `skills/` and `custom-agents/` markdown you edit to tune agents without touching Python.\n- [Security](docs/SECURITY.md): the auth model, sidecar/internal tokens, interactive sudo, SSRF and read-only guards.\n\n## Run it locally\n\nYou need Docker (with Compose) and an LLM backend.\nThe default backend is [Ollama](https://ollama.com). **When the Ollama provider is selected (the default), Ollama must be running on the same host as the stack** and your Mac for a local deploy, the remote box for a remote one — and reachable by the containers at `host.docker.internal:11434`, which means it has to listen on `0.0.0.0`, not just `127.0.0.1`:\n\n```bash\nOLLAMA_HOST=0.0.0.0:11434 ollama serve     # or set OLLAMA_HOST=0.0.0.0 for a brew/launchd service\n```\n\nPull the models your profiles reference. To run on a cloud backend instead (Bedrock, DeepInfra, OpenRouter), point the providers at it in `framework-config.yaml` and then you don't need Ollama for prompts at all. Note embeddings default to local Ollama, so move those to a cloud embedder too if you want to skip Ollama entirely.\n\n```bash\ngit clone https://github.com/bulletinmybeard/agent-forge.git\ncd agent-forge\nscripts/deploy-local.sh          # builds + starts the whole stack\n```\n\nThe agent backend (WebSocket + REST) is then on **http://localhost:8200** and the RAG API on **http://localhost:8100** (health at `/api/health`).\nThis repo ships no frontend. Connect a WebSocket client to `/ws/chat`, or build a SPA into the web image to have it served from `:8200`.\nTear down with `scripts/teardown-local.sh` (data preserved unless you pass `--volumes`).\n\nThe script seeds `config.yaml` + `framework-config.yaml` from the committed examples on first run. Edit them to point at your backends.\n\n### Light mode\n\nThe full stack is eight containers. On a MacBook you can run just the agent, web app + one SAQ worker (+ Redis, + Qdrant) with in-process tool dispatch:\n\n```bash\nscripts/deploy-local.sh --preset light\n```\n\nPut repeatable local settings in `deploy.local.env` (copy from `deploy.local.example.env`) — preset, plus `AGENTFORGE_QDRANT=host` / `AGENTFORGE_REDIS=host` to reuse services you already run (`brew services`), or `AGENTFORGE_QDRANT=off` to skip the vector DB. It's local-only and kept separate from the remote `deploy.env`. Without Qdrant, RAG/`@qdrant`/semantic-memory are off (the agent still works); without a web-search key, `@search` is simply unavailable. See [Stack architecture -\u003e Deployment presets](docs/architecture.md#deployment-presets-light-vs-full).\n\n## Service stack\n\nThe full stack (default `full` preset). The [light preset](#light-mode) runs only `agentforge-web` + one SAQ worker (+ Redis/Qdrant):\n\n| Service                 | Port     | Role                                                     |\n| ----------------------- | -------- | -------------------------------------------------------- |\n| `agentforge-web`        | `8200`   | Chat WebSocket + REST + agent runners (the entrypoint).  |\n| `agentforge-api`        | `8100`   | RAG indexing + vector search (LAN-only).                 |\n| `agentforge-sidecar`    | `8300`   | Hardened browser extraction for stealthy web fetches.    |\n| `qdrant`                | `6333`   | Vector database.                                         |\n| `redis`                 | internal | Tool cache, SAQ queues, audit streams, pub/sub.          |\n| SAQ workers + dashboard | `8086`   | Run the agent/tool jobs. The dashboard shows the queues. |\n\nSee [docs/architecture.md](docs/architecture.md) for how it all connects, and [docs/local-domains.md](docs/local-domains.md) for deploying behind a proxy.\n\n## Configuration\n\n`config.yaml` holds app settings (Qdrant, search, memory, per-mode toggles, the API-key list).\n`framework-config.yaml` holds backends, credentials, and named profiles. Provider-specific model profiles live under `profiles/providers/*.yaml`.\nAll of these are gitignored; copy them from the committed `*.example.yaml` templates.\nEnvironment variables override matching keys (`OLLAMA_HOST`, `DEEPINFRA_API_KEY`, `AGENTFORGE_PROVIDER`, ...).\nOptional prompt refinement (rewrite the opening prompt for clarity before it runs) is off by default — see `prompt_refinement` in `config.yaml` and [docs/modes.md](docs/modes.md#prompt-refinement-optional).\n\n### Provider profiles\n\nCopy the providers you actually use:\n\n```bash\ncp profiles/providers/ollama.example.yaml profiles/providers/ollama.yaml\n```\n\n**`ollama.yaml` is the minimum** — Ollama is the default provider and the base layer every capability tier falls back to. The cloud providers (`bedrock`, `deepinfra`, `openrouter`) are optional; only create those if you set `AGENTFORGE_PROVIDER` (or `ai.provider_override`) to one of them. With only `ollama.yaml` present, `/api/providers` lists Ollama as the sole provider — fully functional. The loader skips `*.example.yaml` templates, so they sit harmlessly next to your real configs.\n\n## Authentication\n\nAPI-key auth is **off by default** (open).\nSet one or more keys to require them on every HTTP + WebSocket request. Do this before exposing AgentForge on a public host, since the agent can run shell, SSH, and Docker.\n\nGenerate a key (the `agf_` prefix is just a recognisable convention):\n\n```bash\necho \"agf_$(openssl rand -hex 32)\"                                   # openssl\npython3 -c \"import secrets; print('agf_' + secrets.token_hex(32))\"   # Python fallback\n```\n\nAdd it to `config.yaml` (or set `AGENTFORGE_API_KEYS`, comma-separated, which wins):\n\n```yaml\nsecurity:\n  api_keys:\n    - \"agf_\u003cyour_generated_key\u003e\"\n```\n\nClients send the key as `Authorization: Bearer agf_...` or `X-API-Key: agf_...`.\nBrowsers (which can't set WebSocket headers) pass it as a `Sec-WebSocket-Protocol` subprotocol or `?api_key=agf_...`.\n`/health` and internal worker callbacks are exempt.\n\nFor public deploys, `AGENTFORGE_REQUIRE_AUTH=1` refuses to boot without keys (and the app already fails closed when the Docker socket is mounted with none)! `AGENTFORGE_ALLOW_INSECURE=1` is the trusted-network escape hatch. See [docs/SECURITY.md](docs/SECURITY.md) for the full checklist and [deploy.example.env](deploy.example.env) for every knob.\n\n## Custom tools\n\nAdd your own tools without forking: expose a `register(registry)` function and advertise it under the `agentforge.tools` entry-point group (or point `AGENTFORGE_TOOL_PLUGINS` at it).\nFull guide: [docs/plugin-authoring.md](docs/plugin-authoring.md).\n\n## Development\n\n```bash\npip install -e .            # production (framework + full service stack)\npip install -e \".[dev]\"     # production + ruff, ty, pytest\n\nruff check .                # lint\nruff format --check .       # formatting\nty check                    # type check (phased scope; see pyproject.toml)\npytest                      # tests\n```\n\nCI runs lint, format, type check, and build + smoke on pull requests (`.github/workflows/ci.yml`).\n\nTo exercise the framework directly without the Docker/web stack, see the [sandbox harness](sandbox/README.md): a short Python script driving `AIClient` / `ToolRegistry` / the agent loop against your Ollama.\n\n## License\n\nMIT, see [LICENSE](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbulletinmybeard%2Fagent-forge","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbulletinmybeard%2Fagent-forge","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbulletinmybeard%2Fagent-forge/lists"}