{"id":50472494,"url":"https://github.com/semcod/codot","last_synced_at":"2026-06-01T11:04:10.755Z","repository":{"id":353204491,"uuid":"1218320412","full_name":"semcod/codot","owner":"semcod","description":null,"archived":false,"fork":false,"pushed_at":"2026-04-22T21:59:41.000Z","size":18331,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-22T23:25:35.805Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/semcod.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-22T18:55:44.000Z","updated_at":"2026-04-22T21:59:43.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/semcod/codot","commit_stats":null,"previous_names":["semcod/codot"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/semcod/codot","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semcod%2Fcodot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semcod%2Fcodot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semcod%2Fcodot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semcod%2Fcodot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/semcod","download_url":"https://codeload.github.com/semcod/codot/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semcod%2Fcodot/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33771634,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-01T02:00:06.963Z","response_time":115,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-01T11:04:09.619Z","updated_at":"2026-06-01T11:04:10.750Z","avatar_url":"https://github.com/semcod.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"# codot is CQRS-URL Platform\n\n\n## AI Cost Tracking\n\n![PyPI](https://img.shields.io/badge/pypi-costs-blue) ![Version](https://img.shields.io/badge/version-0.1.13-blue) ![Python](https://img.shields.io/badge/python-3.9+-blue) ![License](https://img.shields.io/badge/license-Apache--2.0-green)\n![AI Cost](https://img.shields.io/badge/AI%20Cost-$2.10-orange) ![Human Time](https://img.shields.io/badge/Human%20Time-6.3h-blue) ![Model](https://img.shields.io/badge/Model-openrouter%2Fqwen%2Fqwen3--coder--next-lightgrey)\n\n- 🤖 **LLM usage:** $2.1000 (14 commits)\n- 👤 **Human dev:** ~$626 (6.3h @ $100/h, 30min dedup)\n\nGenerated on 2026-04-23 using [openrouter/qwen/qwen3-coder-next](https://openrouter.ai/qwen/qwen3-coder-next)\n\n---\n\nCommands and Queries as **URL-addressable resources**. Operate on arbitrary data over pluggable protocols (`http://`, `https://`, `file://`, `data:`, …), with runtime JSON-Schema validation and policy-based access control — no DTOs, no codegen, no per-command type churn.\n\nThis is the reference implementation of the design discussed in the accompanying articles:\n\n- CQRS decoupled from data models (bytes + Struct envelope)\n- Command/Query as a URL resource (`PUT /commands/converttojson`)\n- Required schemas at runtime (JSON Schema at `schema_uri`)\n- Controlling who can do what on which URI (policy engine with URL globs)\n\n## Quick start\n\n### Full Docker stack\n\n```bash\n# 1. Build and run everything\nmake build\nmake up\n\n# 2. Issue a token (admin)\nmake token\n\n# 3. Open the playground\n#    → http://localhost:8000\n#\n#    Sign in with admin/admin, alice/alice (analyst), or bob/bob (user).\n#    Hit one of the preset buttons: CSV → JSON, render posts, pipeline.\n\n# 4. Smoke-test the API (15 tests: commands, queries, policy, agents, compile_service)\nmake test\n```\n\n### Local development (agents / MCP)\n\nFor agent execution with local MCP servers, run the API directly (outside Docker) so the spawned MCP subprocesses can access your filesystem:\n\n```bash\n# Start the API locally\ncd api \u0026\u0026 python3 -m uvicorn main:app --host 0.0.0.0 --port 28080\n\n# Run a standalone MCP agent\npython3 codot_run.py examples/agent_mcp.json --url http://localhost:28080 --agent\n\n# Run a workflow with an agent step\npython3 codot_run.py examples/workflow_agent_mcp.json --url http://localhost:28080\n# or simply:\nmake workflow\n\n# Run the agent integration test suite\nmake test-agent\n```\n\n## Endpoints\n\n| Method | Path                          | Purpose                                   |\n|--------|-------------------------------|-------------------------------------------|\n| GET    | `/health`                     | liveness probe                            |\n| GET    | `/catalog`                    | public catalog of commands/queries/protocols/backends |\n| POST   | `/auth/token`                 | issue a dev JWT                           |\n| GET    | `/auth/me`                    | current principal                         |\n| GET    | `/commands`                   | list commands (auth)                      |\n| PUT    | `/commands/{name}`            | run a command                             |\n| GET    | `/queries`                    | list queries (auth)                       |\n| POST   | `/queries/{name}`             | run a query                               |\n| POST   | `/agents/{agent_id}/run`      | execute an agent (MCP, LiteLLM, Bash, etc.) |\n| GET    | `/agents/backends`            | list registered agent backends            |\n| PUT    | `/commands/compile_service`   | compile a bundle into deployable artifacts |\n| GET    | `/docs`                       | OpenAPI / Swagger UI                      |\n\n## Bundled commands\n\n| Name            | Purpose                                                                 |\n|-----------------|-------------------------------------------------------------------------|\n| `fetch`         | read a resource from any protocol and return its raw bytes (base64)    |\n| `converttojson` | fetch + transform CSV/text/XML to JSON (+ optional schema validation)   |\n| `converttoxml`  | fetch JSON/CSV and emit XML                                             |\n| `converttocsv`  | fetch JSON list-of-objects and emit CSV                                 |\n| `converttobase64` | base64-encode any resource (useful for PDFs, images)                  |\n| `render`        | Jinja2 template → HTML page (data from URI or inline)                   |\n| `pipeline`      | chain other commands; use `\"$previous.output\"` as a URI reference       |\n| `compile_service`| compile a SERVICE_BUNDLE or VIEW_BUNDLE into deployable artifacts (Python/Docker/PHP/k8s) |\n\nAdding your own command is three steps: subclass `Command`, register it, optionally add a policy rule.\n\n## Bundled queries\n\n| Name         | Purpose                                                 |\n|--------------|---------------------------------------------------------|\n| `from-url`   | fetch one or more URIs and return them in a list        |\n| `introspect` | list commands, queries, protocols                       |\n\n## Agent backends\n\nThe platform now supports autonomous agents via multiple communication backends. An agent is defined by a `role`, `goal`, `tools`, and a `backend_config`.\n\n| Backend     | Driver | Typical use |\n|-------------|--------|-------------|\n| `mcp`       | `MCPStdioClient` / `MCPSseClient` | Any MCP-compatible server (JSON-RPC 2.0 over stdio or SSE) |\n| `litellm`   | `httpx` | LLM inference via LiteLLM / OpenAI-compatible APIs |\n| `bash_cli`  | `asyncio.create_subprocess_shell` | Shell scripts, local tools |\n| `http_api`  | `httpx` | Generic REST / GraphQL endpoints |\n| `websocket` | `websockets` | Real-time streaming agents |\n\nAgents can be invoked standalone (`POST /agents/{id}/run`) or embedded inside a `pipeline` step via the optional `agent_node` field. The pipeline automatically decodes `data:` URIs from `$previous.output` and injects them into the agent context.\n\n## CLI runner\n\nRun workflows and agents from shell without writing curl:\n\n```bash\n# Standalone MCP agent (reads API_BASE_URL from .env by default)\npython3 codot_run.py examples/agent_mcp.json --agent\n\n# Workflow with an agent step\npython3 codot_run.py examples/workflow_agent_mcp.json\n```\n\n## Protocols\n\n| Scheme          | Notes                                                   |\n|-----------------|---------------------------------------------------------|\n| `http`, `https` | standard fetch via httpx, size-limited                  |\n| `file://`       | local reads limited to `ALLOWED_LOCAL_ROOTS` (default: `/data`, `/schemas`) |\n| `data:`         | RFC 2397 inline payloads, base64 or percent-encoded     |\n\nAdding a new protocol (e.g. `s3://`, `ftp://`, `sqlite://`) is a matter of writing a class with a `scheme` attribute and an async `fetch(uri)` method, then registering it in `protocols/__init__.py`.\n\n## Policy\n\nPolicies are loaded from `api/policy/rules.yaml` at startup. Each rule matches by role and lists glob patterns of allowed command/query names, URIs and schema URIs. Reload without rebuild: edit the file and restart the container (`make restart`).\n\nThree built-in roles:\n\n- **admin** — everything\n- **analyst** — all commands/queries, all `http(s)://` and `file:///data`, `file:///schemas`\n- **user** — only `fetch`, `converttojson`, `converttobase64`, `render` and public queries, only against `http://cqrs-data/*`, `https://public-*`, `file:///data/public/*`, `data:*`\n\nSee also: `api/policy/__init__.py` (the engine), `api/auth/__init__.py` (JWT issuance), `api/main.py` (enforcement point).\n\n## Layout\n\n```\n.\n├── api/                 FastAPI service\n│   ├── commands/        one file per command\n│   ├── queries/         one file per query\n│   ├── protocols/       pluggable URI fetchers\n│   ├── policy/          RBAC engine + rules.yaml\n│   ├── validators/      JSON Schema over arbitrary URIs\n│   ├── auth/            JWT issuance + FastAPI dependencies\n│   ├── agent.py         multi-backend agent execution (MCP, LiteLLM, Bash, HTTP, WS)\n│   ├── mcp_client.py    JSON-RPC 2.0 MCP client (stdio + SSE)\n│   ├── models.py        envelope (CommandRequest/Response, AgentNode, AgentRequest)\n│   ├── config.py        env-based settings\n│   ├── test_all_agents.py  integration tests for all agent backends\n│   └── main.py          HTTP layer\n├── codot_run.py         CLI runner for workflows and agents\n├── mcp_servers/         example MCP servers for local testing\n│   └── summary_server.py\n├── examples/            example workflow and agent JSONs\n│   ├── workflow_agent_mcp.json\n│   └── agent_mcp.json\n├── frontend/            nginx + static playground (HTML/CSS/JS)\n├── schemas/             JSON Schemas served at http://schemas/\n├── sample-data/         demo data served at http://cqrs-data/\n├── tests/\n│   ├── smoke.sh         curl-based end-to-end tests\n│   ├── test_policy.py   pytest unit tests\n│   └── test_protocols.py\n├── cqrs-workflow-editor/  React + Vite visual workflow editor (@xyflow/react)\n├── articles/            status articles (Markdown, for WordPress)\n├── docker-compose.yml\n└── Makefile\n```\n\n## Adding a command\n\n1. Create `api/commands/my_thing.py`:\n\n```python\nfrom . import Command\nfrom models import CommandRequest, CommandResponse\n\nclass MyThingCommand(Command):\n    name = \"mything\"\n    description = \"Short sentence.\"\n    input_hint = {\"input_uri\": \"...\", \"meta.foo\": \"...\"}\n\n    async def execute(self, request: CommandRequest) -\u003e CommandResponse:\n        # ... do work, return CommandResponse(payload_b64=..., mime=..., meta=...)\n```\n\n2. Register it in `api/commands/__init__.py::register_default_commands`.\n3. Optionally add an entry in `rules.yaml` if you want non-admins to call it.\n\nThat's the whole loop.\n\n## Environment\n\nCopy `.env.example` → `.env` and adjust. Key variables:\n\n- `JWT_SECRET` — must be ≥ 32 chars in production\n- `ACCESS_TOKEN_EXPIRE_MINUTES` — default 60\n- `ALLOWED_LOCAL_ROOTS` — comma list, default `/data,/schemas`\n- `FETCH_MAX_BYTES` — size cap for fetched resources (default 50 MiB)\n\n## License\n\nLicensed under Apache-2.0.\n\n\u003c!-- taskill:status:start --\u003e\n\n## Status\n\n_Last updated by [taskill](https://github.com/oqlos/taskill) at 2026-04-25 13:36 UTC_\n\n| Metric | Value |\n|---|---|\n| HEAD | `1cb5ec9` |\n| Coverage | — |\n| Failing tests | — |\n| Commits in last cycle | 25 |\n\n\u003e Primarily documentation updates and widespread refactoring. Added a docs-oriented code analysis engine and CLI improvements, and fixed Markdown output in the docs pipeline.\n\n\u003c!-- taskill:status:end --\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsemcod%2Fcodot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsemcod%2Fcodot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsemcod%2Fcodot/lists"}