{"id":49812433,"url":"https://github.com/joemunene-by/joe","last_synced_at":"2026-05-13T02:01:58.131Z","repository":{"id":357472524,"uuid":"1236969969","full_name":"joemunene-by/joe","owner":"joemunene-by","description":"Local-first agent shell with self-training, AI-blame, knowledge graph, and live pair-programming. Runs on your own GPU via ollama.","archived":false,"fork":false,"pushed_at":"2026-05-13T00:17:17.000Z","size":424,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-13T00:27:21.495Z","etag":null,"topics":["agent","apple-silicon","cli","developer-tools","fine-tuning","gemma","knowledge-graph","llm","local-first","lora","macos","mcp","ollama","rag","self-hosted","terminal","tui","voice-assistant"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/joemunene-by.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-12T18:47:03.000Z","updated_at":"2026-05-13T00:17:19.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/joemunene-by/joe","commit_stats":null,"previous_names":["joemunene-by/joe"],"tags_count":12,"template":false,"template_full_name":null,"purl":"pkg:github/joemunene-by/joe","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joemunene-by%2Fjoe","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joemunene-by%2Fjoe/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joemunene-by%2Fjoe/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joemunene-by%2Fjoe/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/joemunene-by","download_url":"https://codeload.github.com/joemunene-by/joe/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joemunene-by%2Fjoe/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32964444,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-12T23:30:32.555Z","status":"online","status_checked_at":"2026-05-13T02:00:07.132Z","response_time":115,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agent","apple-silicon","cli","developer-tools","fine-tuning","gemma","knowledge-graph","llm","local-first","lora","macos","mcp","ollama","rag","self-hosted","terminal","tui","voice-assistant"],"created_at":"2026-05-13T02:01:33.630Z","updated_at":"2026-05-13T02:01:58.123Z","avatar_url":"https://github.com/joemunene-by.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# joe\n\nA local-first, self-personalising agent shell for your machine. Think\nClaude Code, but every model runs on your own GPU, every byte of state\nsits in `~/.joe-agent/`, and the agent gets better at understanding you\nthe longer you use it.\n\n```\njoe (your terminal)\n  ├── orchestrator    joe-gemma             (gemma3:4b, Modelfile-personalised)\n  ├── coder delegate  qwen2.5-coder:7b/14b\n  ├── planner         deepseek-r1:14b       (optional; used by /plan)\n  ├── fast model      qwen2.5:3b            (joe-pair save-watcher, trivials)\n  └── tools           read/write/edit/bash/grep/glob/web_search/web_fetch\n                      test/lint/build/plan/image/delegate\n\nstate under ~/.joe-agent/\n  ├── sessions        full chat history per named session\n  ├── lessons         /undo corrections, distilled into prompt rules\n  ├── provenance      every write/edit stamped with prompt + model + agent\n  ├── knowledge       (subject, relation, object) triples from each turn\n  ├── agents          named subagent personas, summoned with @@name\n  ├── macros          recorded slash-command sequences\n  └── (training data, profiles, journals, tasks, snapshots, ...)\n```\n\n## why this exists\n\nCloud agents see your code through a straw. Local agents are starting to\ncatch up, but most are just chat REPLs with one or two tools bolted on.\njoe sits between the two: a real tool-using agent on hardware you own,\nwith the receipts (provenance, lessons, knowledge graph) to actually\nimprove over time without retraining.\n\nThe seven features that make joe interesting:\n\n1. **AI-blame / provenance** — `joe blame src/foo.py:42` tells you which\n   prompt, under which agent, in which session, on which date produced\n   that line. Git blame for *intent*.\n2. **Lessons-learned loop** — every `/undo` is a signal. `/lessons distill`\n   turns recent corrections into 1-line preference rules that get injected\n   into the system prompt of every future turn. Auto-evolving personality.\n3. **Knowledge graph** — every conversation is parsed into\n   `(subject, relation, object)` triples in the background. `/ask` queries\n   the graph for grounded answers with session citations.\n4. **Smart inbox** — `joe inbox` aggregates PRs awaiting your review,\n   issues mentioning you, failing CI runs across active repos, running\n   background tasks. One panel.\n5. **Self-eval / drift detection** — snapshot your past accepted answers\n   as a private benchmark. Rerun weekly against the current model and\n   measure how much it has drifted.\n6. **Live pair-programming** — `joe-pair` watches your files via fswatch.\n   On each save it runs your tests; if they pass, it briefly asks the\n   model whether anything in the change is worth flagging. Silent 90% of\n   the time.\n7. **Self-training** — `joe train collect` builds a JSONL training set\n   from your accepted answers + lessons. `joe train modelfile` bakes\n   them into a fresh ollama Modelfile (`joe-gemma:vYYYYMMDD`).\n   `joe train lora` hands the same data to mlx-lm for a real LoRA on\n   Apple Silicon.\n\nPlus the everyday surface: macros, scheduled jobs, cross-session memory,\nvoice input/output, vision, iOS bridge over ntfy, cross-machine sync\nthrough a private git repo, per-repo profiles with auto-detected style\nfingerprints, a web dashboard at `joe-http` for the visual view (with\nstreaming-SSE chat right in the browser), model auto-promotion when a\nfreshly-trained candidate beats the current default by a threshold,\ntime-aware semantic recall (\"what was I working on yesterday morning?\"),\n`joe replay` to step through any past session and rerun prompts against\nalternate models, `joe-wake` for always-on \"hey joe\" voice activation,\nand a Tauri menu-bar app for click-to-open access.\n\n## quickstart\n\nRequires Python 3.10+, `ollama` running locally, and macOS for the voice /\nnotification / launchd bits. Most things work on Linux too if you ignore\nthose.\n\n```sh\ngit clone https://github.com/joemunene-by/joe ~/code/joe\ncd ~/code/joe\n./install.sh                          # symlinks bin/* into ~/.local/bin\nollama pull gemma3:4b\nollama pull qwen2.5-coder:7b\nollama pull qwen2.5-coder:14b\nollama pull nomic-embed-text          # for cross-session RAG + eval\n\npython3 -m pip install --user rich prompt_toolkit fastapi uvicorn mcp\njoe doctor                            # confirm everything is wired\njoe                                   # start the REPL\n```\n\nFirst time you run `joe`, it auto-seeds seven default subagents at\n`~/.joe-agent/agents/{reviewer,doc-writer,security,explainer,oncall,release-manager,refactor-specialist}.toml`.\nTry one:\n\n```\n@@reviewer review the current diff\n```\n\nIf you'd rather have `joe doctor` install missing pieces for you instead\nof installing manually, run `joe doctor --fix` after `./install.sh`. It\npulls missing ollama models and pip-installs missing Python deps.\n\nFor a guided tour of the killer features on a sample repo, run\n`joe demo`: it creates a fresh project, makes a deliberately-buggy file,\nruns `@@reviewer` and `@@security` on it, then walks you through\n`/blame`, `/ask`, and `/tasks`. The thing to record a launch GIF of.\n\n## the input prefixes\n\njoe parses your input by its first character before treating it as chat:\n\n| Prefix | Meaning                                                    |\n| ------ | ---------------------------------------------------------- |\n| `/`    | slash command (no model call)                              |\n| `\u0026`    | background task; gets an id, runs detached, pings on exit  |\n| `@`    | switch cwd to that active repo for this turn               |\n| `@@`   | run the turn under a named subagent persona                |\n| `///`  | enter multi-line input (terminate with another `///`)      |\n\n## the slash commands\n\n`/help` inside the REPL shows the live menu. Highlights:\n\n```\n/plan \u003ctask\u003e         multi-step plan with per-step approval\n/review              joe-gemma audits the current git diff\n/commit | /push      AI-drafted commit message, then push\n/test | /lint | /build  auto-detected for python/rust/node/go\n/blame \u003cp\u003e:\u003cline\u003e    AI-blame: prompt/model/agent that produced this code\n/lessons distill     turn recent /undo events into prompt rules\n/macro record \u003cn\u003e    capture a sequence of inputs; replay with /macro run\n/recall \u003cq\u003e          search past sessions by semantic similarity\n/ask \u003cq\u003e             query the knowledge graph for a grounded answer\n/inbox               PRs / mentions / failing CI / pending tasks\n/reflect             write today's developer-journal entry\n/snap save|restore   tar repo state to ~/.joe-agent/snaps/\n/listen / /speak     voice in/out\n/paste / /attach     image attachment for vision\n/sync push|pull      cross-machine state sync over git\n```\n\n## the tool protocol (what the model can do)\n\nThe model emits XML tags; joe parses, executes, and feeds the result\nback as `\u003ctool_result\u003e`:\n\n```xml\n\u003cread path=\"...\" /\u003e\n\u003cwrite path=\"...\"\u003econtent\u003c/write\u003e\n\u003cedit path=\"...\"\u003e\u003cold\u003eX\u003c/old\u003e\u003cnew\u003eY\u003c/new\u003e\u003c/edit\u003e\n\u003cbash\u003ecommand\u003c/bash\u003e\n\u003cgrep pattern=\"...\" path=\"...\" /\u003e\n\u003cglob pattern=\"...\" /\u003e\n\u003cdelegate model=\"qwen2.5-coder:14b\"\u003espec\u003c/delegate\u003e\n\u003ctest path=\"...\" /\u003e\n\u003clint path=\"...\" /\u003e\n\u003cbuild path=\"...\" /\u003e\n\u003cweb_search query=\"...\" /\u003e\n\u003cweb_fetch url=\"...\" /\u003e\n\u003cplan title=\"...\"\u003enumbered steps\u003c/plan\u003e\n\u003cimage path=\"...\" /\u003e\n```\n\nBy default, bash runs against an allow-list. Pass `--unsafe-bash` or\nset `JOE_AUTO_YES=1` to broaden / auto-approve. Reading the threat model\nbelow before flipping either is recommended.\n\n## customising\n\nThree places to leave your fingerprints:\n\n### per-repo profiles (`.joe-profile.toml` at a repo root)\n\n```toml\ndefault_coder = \"qwen2.5-coder:14b\"\ndefault_base = \"main\"\nlint_cmd = \"ruff check .\"\ntest_cmd = \"pytest\"\nbuild_cmd = \"python3 -m build\"\ncontext_extra = \"\"\"\nThis is the GhostLM training repo. Always run pytest after edits in ghostlm/.\nNever touch RESULTS.md by hand; it's auto-generated.\n\"\"\"\n```\n\nGenerate a starter with `joe init` — it also scans the repo and bakes a\nstyle fingerprint (indent / quote / naming / line-length) into the\n`context_extra` so coder delegates match existing convention.\n\n### subagent personas (`~/.joe-agent/agents/\u003cname\u003e.toml`)\n\n```toml\ndescription = \"release manager\"\nmodel = \"joe-gemma\"\ncoder = \"qwen2.5-coder:14b\"\nauto_yes = false\nsystem_prompt = \"\"\"\nYou are my release manager. Run /test and /lint first. If both pass,\ndraft a CHANGELOG entry, then a release commit, then a tag.\n\"\"\"\n```\n\nSummon with `@@release-manager cut a v0.9.34 from main`.\n\n### lessons supplement (`~/.joe-agent/prompt-supplement.md`)\n\nFree-form markdown that joins every system prompt. Use it for facts\nabout you and your stack that don't change: \"I use ruff format with line\nlength 88\", \"in ghostloop, the embedded runtime is the ESP32 build\",\netc. `/lessons distill` writes here automatically; `/lessons review`\nopens it in `$EDITOR` for cleanup.\n\n## privacy + security\n\njoe is local-first, but the surface area is broad. A few flags to\nunderstand before turning them on:\n\n- **`--unsafe-bash` / `JOE_AUTO_YES=1`** — disables the bash allow-list\n  and auto-approves every write/edit/bash. Suitable for a sandboxed VM\n  you can throw away; risky on your daily-driver Mac. Off by default.\n- **`joe-proactive`** — watches your zsh history file and pops a banner\n  when it detects retry patterns. It reads your shell history. Opt-in\n  (`JOE_PROACTIVE=1` for the `--autostart` form).\n- **`joe sync push`** — syncs `~/.joe-agent/` to a git remote you\n  configure. The default `.gitignore` excludes credentials and large\n  blobs, but conversation logs (`joe-sessions.sqlite`) DO sync. Use a\n  private repo.\n- **`joe-http`** — exposes a FastAPI server on a port of your choice.\n  Bearer-token auth, no TLS. Bind to `127.0.0.1` for local-only or front\n  it with Tailscale for off-LAN access. Never open port 8765 to the\n  public internet.\n- **iOS bridge** — `joe ios-setup` prints a Shortcut recipe. Pushes go\n  through ntfy.sh which is third-party. Use a guessable-but-not-obvious\n  topic name (no public listing exists, but topics aren't private by\n  design).\n- **`/web` and `\u003cweb_fetch\u003e`** — scrape DuckDuckGo HTML and arbitrary\n  URLs. Your IP is visible to those endpoints.\n\nNo network calls happen unless you explicitly invoke a web tool, sync,\nntfy, or the iOS bridge. Conversations stay between you and ollama on\nlocalhost.\n\n## architecture in one paragraph\n\nThe orchestrator (joe-gemma) emits XML tool tags; joe's REPL parses each\ntag, executes it (write/edit/bash/test/etc.), captures stdout, feeds the\nresult back as `\u003ctool_result\u003e`, and loops until the model stops emitting\ntags or hits an 8-turn cap. Code-writing is delegated to qwen-coder\nbecause the smaller orchestrator handles meta-work better than raw\ngeneration. Cross-session memory uses nomic-embed-text against a\nSQLite-backed vector store. The lessons supplement and knowledge-graph\ntriples are stored separately and merged into every prompt's preamble.\n\n## the web dashboard\n\n`joe-http` exposes a single-page dashboard at `GET /` (or `/dashboard`)\nthat renders, all in one screen:\n\n- eval-trend sparklines per model (with delta vs previous run)\n- the last 10 provenance entries (which prompt / agent produced each write)\n- knowledge-graph stats and the 12 newest triples\n- active distilled lessons\n- recent sessions and background tasks\n\nAuth is the same bearer token as the JSON endpoints, passed via\n`?token=\u003cTOKEN\u003e` so browsers can hit it without setting headers:\n\n```\nhttp://localhost:8765/dashboard?token=\u003cfrom ~/.joe-agent/http-token\u003e\n```\n\nFront it with Tailscale to view from your phone on the couch.\n\n## tests\n\n```sh\npython3 -m pip install --user pytest\npython3 -m pytest tests/ -v\n```\n\n34 tests, all offline, no ollama dependency. Covers the tool-call\nparser, time-phrase extractor, blob store, subagent loader, lessons +\nprovenance round-trip, active-repo discovery, and the small math\nhelpers underpinning embeddings + eval scoring.\n\n## menu-bar app (Tauri)\n\n```sh\ncd joe-tauri\ncargo tauri dev          # dev build, hot-reload\ncargo tauri build        # signed .app in src-tauri/target/release/bundle/macos/\n```\n\nClick the menu-bar icon to toggle the dashboard window. Left-click\ntoggles; right-click opens the menu (Open / Refresh / Quit). The app\nreuses your existing `joe-http` server and auth token.\n\n## bidirectional MCP\n\njoe is now both an MCP server **and** a client. As of v0.7.0:\n\n- **server side** (`joe-mcp`): exposes joe's tools (ask, search_repos,\n  pr_draft, standup) to any MCP client (Claude Code, Cursor, Continue).\n- **client side**: connects to external MCP servers configured at\n  `~/.joe-agent/mcp-clients.json` and lets the orchestrator call their\n  tools via a `\u003cmcp\u003e` tag.\n\nSetup:\n\n```sh\njoe mcp init                  # scaffolds ~/.joe-agent/mcp-clients.json\n                              # with filesystem + github sample servers\n                              # (both disabled-by-default)\n# edit the file, flip `enabled: true` on the servers you want\npython3 -m pip install --user mcp   # if you don't already have it\njoe                           # at startup, joe connects each enabled server\n```\n\nInside the REPL:\n\n```\n/mcp list      # show connected servers + their tools\n/mcp reload    # tear down + reconnect after editing the config\n```\n\nThe orchestrator sees available tools in an `\u003cmcp_tools\u003e` block in its\nsystem prompt and can call them with:\n\n```xml\n\u003cmcp server=\"github\" tool=\"list_repos\"\u003e{\"owner\": \"joemunene-by\"}\u003c/mcp\u003e\n```\n\nResults come back through the standard `\u003ctool_result\u003e` loop.\n\n### example: joe ↔ ghostloop (bidirectional)\n\njoe pairs naturally with [ghostloop](https://github.com/joemunene-by/ghostloop),\nthe embodied-agent runtime. Both directions of the loop are now wired:\n\n**direction 1 — joe drives ghostloop** (joe as MCP client, ghostloop as MCP server).\nghostloop ships `examples/mcp_robot.py` as an MCP server. Drop one entry into\n`~/.joe-agent/mcp-clients.json`:\n\n```json\n{\n  \"servers\": [{\n    \"name\": \"ghostloop\",\n    \"transport\": \"stdio\",\n    \"enabled\": true,\n    \"command\": \"python3\",\n    \"args\": [\"/absolute/path/to/ghostloop/examples/mcp_robot.py\"],\n    \"env\": {\"GHOSTLOOP_PROFILE\": \"franka_arm\", \"GHOSTLOOP_BACKEND\": \"mock\"}\n  }]\n}\n```\n\njoe-gemma now sees `move_to`, `pick`, `place`, `scan` etc. in `\u003cmcp_tools\u003e` and\ncan drive the robot from any prompt. ghostloop's policy pipeline (geofence,\nforce-cap, human-in-the-loop) still gates every intent so joe can use the\nrobot but can't bypass safety. Full sample at\n`examples/mcp-clients.ghostloop.json`.\n\n**direction 2 — ghostloop uses joe-gemma as its policy brain** via the\nOpenAI-shaped shim on `joe-http`:\n\n```sh\n# on the box running joe:\njoe-http --host 0.0.0.0          # exposes /v1/chat/completions + /v1/models\n\n# on the box running ghostloop:\nOPENAI_BASE_URL=http://\u003cjoe-host\u003e:8765/v1 \\\nOPENAI_API_KEY=$(ssh \u003cjoe-host\u003e cat ~/.joe-agent/http-token) \\\nOPENAI_MODEL=joe-gemma \\\n    python3 examples/direct_llm_arm.py\n```\n\njoe-http proxies ollama's already-OpenAI-compatible endpoint, with two\nadditions worth the indirection:\n  1. **personalisation injection** — the active lessons supplement gets\n     prepended as a system message when the caller doesn't supply one,\n     so ghostloop's `LLMPolicy` inherits your distilled preferences.\n  2. **bearer auth + cross-machine access** — ollama only listens on\n     localhost; joe-http can be exposed over LAN or Tailscale, so\n     ghostloop on another machine can reach joe-gemma without exposing\n     ollama directly.\n\nSample ghostloop runner at `examples/ghostloop-uses-joe.py`.\n\n## what's not in here yet\n\nThe headline open items are shipped. Future polish:\n\n- HTTP transport for MCP clients (currently stdio-only).\n- A `joe stats --export csv` flag.\n- More piper-tts voices via a per-language `--setup-piper \u003clocale\u003e` flow.\n\nPRs welcome.\n\n## license\n\nMIT.\n\n## acknowledgements\n\nBuilt by Joe Munene with deliberate use of Claude as a pair-programmer.\nThe agent's name is incidental; the design takes inspiration from\nClaude Code, Aider, and the years of work on local LLMs that made\ngemma3:4b + qwen-coder good enough to do this seriously.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjoemunene-by%2Fjoe","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjoemunene-by%2Fjoe","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjoemunene-by%2Fjoe/lists"}