{"id":47721142,"url":"https://github.com/mycellm/mycellm","last_synced_at":"2026-04-08T08:01:06.359Z","repository":{"id":346715522,"uuid":"1186020395","full_name":"mycellm/mycellm","owner":"mycellm","description":"Distributed LLM inference across heterogeneous hardware. Pool GPUs into a P2P network with QUIC transport, Ed25519 identity, and an OpenAI-compatible API.","archived":false,"fork":false,"pushed_at":"2026-04-01T19:17:26.000Z","size":4661,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-03T05:37:48.705Z","etag":null,"topics":["decentralized","distributed-computing","fleet-management","gpu","inference","llama-cpp","llm","machine-learning","openai-api","peer-to-peer","python","quic","self-hosted"],"latest_commit_sha":null,"homepage":"https://mycellm.ai","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mycellm.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-19T07:19:45.000Z","updated_at":"2026-04-02T10:07:07.000Z","dependencies_parsed_at":null,"dependency_job_id":"a2597034-a06b-49b2-8614-561eea4cf911","html_url":"https://github.com/mycellm/mycellm","commit_stats":null,"previous_names":["mycellm/mycellm"],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/mycellm/mycellm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mycellm%2Fmycellm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mycellm%2Fmycellm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mycellm%2Fmycellm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mycellm%2Fmycellm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mycellm","download_url":"https://codeload.github.com/mycellm/mycellm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mycellm%2Fmycellm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31545906,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-07T16:28:08.000Z","status":"online","status_checked_at":"2026-04-08T02:00:06.127Z","response_time":54,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["decentralized","distributed-computing","fleet-management","gpu","inference","llama-cpp","llm","machine-learning","openai-api","peer-to-peer","python","quic","self-hosted"],"created_at":"2026-04-02T19:35:42.604Z","updated_at":"2026-04-08T08:01:06.345Z","avatar_url":"https://github.com/mycellm.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/mycellm/mycellm/main/docs/site/docs/assets/mycellm-logo.svg\" width=\"80\" alt=\"mycellm\"\u003e\n\u003c/p\u003e\n\n\u003ch1 align=\"center\"\u003emycellm_\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cstrong\u003ePool GPUs worldwide. Earn credits. No cloud required.\u003c/strong\u003e\u003cbr\u003e\n  \u003cem\u003eA peer-to-peer inference network with credits, privacy, and federation.\u003c/em\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/badge/license-Apache%202.0-blue.svg\" alt=\"License\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://pypi.org/project/mycellm/\"\u003e\u003cimg src=\"https://img.shields.io/pypi/v/mycellm.svg?color=green\" alt=\"PyPI\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://www.python.org/\"\u003e\u003cimg src=\"https://img.shields.io/badge/python-3.11+-blue.svg\" alt=\"Python\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://mycellm.ai\"\u003e\u003cimg src=\"https://img.shields.io/badge/website-mycellm.ai-spore\" alt=\"Website\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://mycellm.ai\"\u003eWebsite\u003c/a\u003e ·\n  \u003ca href=\"https://docs.mycellm.dev/quickstart/install\"\u003eDocs\u003c/a\u003e ·\n  \u003ca href=\"https://github.com/mycellm/mycellm-ios\"\u003eiOS App\u003c/a\u003e ·\n  \u003ca href=\"https://mycellm.ai/join\"\u003eJoin the network\u003c/a\u003e\n\u003c/p\u003e\n\n---\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/mycellm/mycellm/main/docs/screenshots/dashboard-overview.png\" alt=\"mycellm dashboard — fleet overview with network health, hardware cards, and QUIC peer topology\" width=\"100%\"\u003e\n\u003c/p\u003e\n\n## What is mycellm?\n\nmycellm pools GPUs across the internet into a single inference network. Contribute compute and earn credits. Chat with open models for free. No blockchain, no tokens, no cloud vendor — just peers serving peers.\n\n- **Credit economy** — earn credits by seeding, spend them consuming. Ed25519-signed receipts for every request. No cryptocurrency.\n- **Sensitive Data Guard** — outgoing prompts are scanned on-device for API keys, passwords, and PII. Sensitive queries route to your local model automatically.\n- **Private networks** — create invite-only inference networks for your team, lab, or org. Federation bridges multiple networks. Fleet management for enterprise.\n- **OpenAI-compatible API** — drop-in replacement at `/v1/chat/completions`. Works with Claude Code, aider, Continue.dev, or any tool that accepts an OpenAI base URL.\n- **iOS app** — native app for iPad and iPhone. Your iPad serves inference at 30+ tokens/sec on Metal and earns credits as a full network peer.\n- **No cloud, no lock-in** — QUIC transport with NAT traversal. Works across the internet, not just your LAN. Your hardware, your models.\n\n## Quick Start\n\n```bash\n# Install\npip install mycellm\n\n# Create identity and join the public network\nmycellm init\n\n# Start serving (auto-detects GPU)\nmycellm serve\n```\n\nYour node is now live. Load a model and start earning credits:\n\n```bash\n# Interactive chat\nmycellm chat\n\n# Or use the OpenAI-compatible API\ncurl http://localhost:8420/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"model\": \"auto\", \"messages\": [{\"role\": \"user\", \"content\": \"Hello!\"}]}'\n```\n\n## One-liner Install\n\n```bash\ncurl -fsSL https://mycellm.ai/install.sh | sh\n```\n\nOr with Docker:\n\n```bash\ndocker run -p 8420:8420 -p 8421:8421/udp ghcr.io/mycellm/mycellm serve\n```\n\n## How It Works\n\n```\nYou (consumer) ──QUIC──▶ Bootstrap (relay) ──QUIC──▶ Seeder (GPU)\n                                                       │\n                                                  llama.cpp / vLLM\n                                                       │\n                                              Tokens stream back ◀──\n```\n\n1. **Consumers** send prompts via the API or chat interface\n2. **Bootstrap** relays requests to available seeders via QUIC\n3. **Seeders** run inference on their local GPU and stream tokens back\n4. **Credits** flow to seeders — signed Ed25519 receipts for every request\n5. **NAT traversal** enables direct P2P connections when possible\n\n## Why mycellm?\n\n| | mycellm | Cloud APIs | Local-only tools | Blockchain projects |\n|---|---|---|---|---|\n| **Works over internet** | QUIC + NAT traversal | N/A | LAN only | Varies |\n| **No vendor lock-in** | Your hardware | Their hardware | Your hardware | Token buy-in |\n| **Credit accounting** | Signed receipts | Pay per token | None | Token economics |\n| **Private networks** | Invite-only federation | N/A | N/A | Public by default |\n| **Privacy** | PII scanning + local redirect | Trust the provider | Full control | Trust the miner |\n| **Mobile nodes** | Native iOS app | N/A | N/A | N/A |\n| **Cost** | Free (contribute compute) | $$$ | Free | Buy tokens |\n\n## Architecture\n\n| Layer | Purpose | Tech |\n|-------|---------|------|\n| **Canopy** | Client access | iOS app, CLI chat, web UI, OpenAI API |\n| **Mycelium** | Routing \u0026 discovery | QUIC transport, Kademlia DHT, STUN/ICE |\n| **Roots** | Inference compute | llama.cpp (Metal/CUDA/ROCm/CPU), vLLM |\n| **Ledger** | Accounting | Ed25519 signed receipts, per-network credit tracking |\n\n## Features\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/mycellm/mycellm/main/docs/screenshots/dashboard-models.png\" alt=\"mycellm models — fleet device management with HuggingFace model browser\" width=\"100%\"\u003e\n\u003c/p\u003e\n\n### Inference\n- **llama.cpp** backend with Metal, CUDA, ROCm, and CPU support\n- **Streaming** token generation via SSE\n- **Model management** — download from HuggingFace, load/unload, scope control\n- **Thermal throttling** — auto-adjusts on mobile devices\n\n### Networking\n- **QUIC** transport with bidirectional streams (NWConnectionGroup on iOS, aioquic on Python)\n- **NAT traversal** — STUN discovery + UDP hole punching for direct P2P\n- **HTTP fallback** — works when QUIC is blocked\n- **Bootstrap relay** — always works, even behind symmetric NAT\n\n### Security \u0026 Privacy\n- **Sensitive Data Guard** — scans every outgoing prompt for API keys, passwords, credit cards, and PII. High-severity matches are automatically redirected to your local model — sensitive data never leaves your device.\n  - Gateway: returns 422 with explanation for flagged requests\n  - Override: `X-Privacy-Override: acknowledged` header\n- **Ed25519 identity** — account key → device cert → peer ID. Every node has a cryptographic identity.\n- **Signed receipts** — cryptographic proof of inference served. Verifiable accounting without a blockchain.\n- **Fleet management** — remote node control with admin key auth\n\n### Multi-Network\n- **Public network** — open to all, auto-approved\n- **Private networks** — invite-only with Ed25519-signed tokens\n- **Federation** — gateway nodes bridge multiple networks\n- **Fleet** — enterprise management with remote commands\n- **Trust levels** — strict (verify all), relaxed (verify, don't enforce), honor (trusted LAN)\n\n## Use Cases\n\n### AI Coding Assistants\nmycellm works as a drop-in backend for OpenAI-compatible coding tools:\n\n- **[OpenClaw](https://openclaw.ai)** — autonomous AI agent framework. Point it at `http://localhost:8420/v1` and your fleet serves the inference.\n- **[OpenCode](https://github.com/opencode-ai/opencode)** — open-source coding assistant. Set `OPENAI_BASE_URL` to your mycellm node.\n- **Claude Code / aider / Continue.dev** — any tool that accepts an OpenAI base URL.\n\nNo API keys to manage, no usage limits, no vendor lock-in. Your hardware, your models.\n\n### Homelab GPU Fleet\nPool every GPU in your house into one inference endpoint. An M1 Max Mac Studio, an old gaming PC with an RTX 3090, an iPad Pro — they all join the same network and share the load. The dashboard lets you manage models across all devices from a single browser tab.\n\n### Research Labs \u0026 Universities\nCreate a private mycellm network for your lab. Students and researchers get free inference from shared departmental GPUs. Ed25519 identity ensures accountability. Credit-based access prevents one user from monopolizing the cluster.\n\n### At Scale\nWhen dozens of nodes contribute compute, mycellm's quality-aware routing shines:\n- **Tier routing** — route to the best model that fits the request (1B for quick tasks, 70B for complex reasoning)\n- **Automatic failover** — if a node goes offline, requests route to the next best\n- **Credit economics** — contributors earn credits, consumers spend them, freeloaders get throttled\n\n## iOS App\n\nNative app for iPad and iPhone. Your iPad is a full peer on the network — serve inference at 30+ tokens/sec on Metal, earn credits, and chat with privacy protection.\n\n- **On-device inference** — llama.cpp on Metal, optimized for M-series iPads\n- **Network + local routing** — toggle between network and on-device per message, with automatic fallback\n- **Chat persistence** — threaded conversations with full metadata (model, node, tokens/sec, route). Export and share threads. Private ephemeral sessions.\n- **Sensitive Data Guard** — prompts are scanned on-device; sensitive queries route to your local model\n- **Serves an OpenAI API** — your iPad exposes `/v1/chat/completions` on your LAN for other tools to use\n\nRequires iOS 17.0+. Also works on iPhone. [Source on GitHub.](https://github.com/mycellm/mycellm-ios)\n\n## Configuration\n\n```bash\n# Environment variables\nMYCELLM_API_HOST=0.0.0.0        # API listen address\nMYCELLM_API_PORT=8420            # API port\nMYCELLM_QUIC_PORT=8421           # QUIC transport port\nMYCELLM_LOG_LEVEL=INFO           # Logging level\nMYCELLM_FLEET_ADMIN_KEY=...      # Fleet management key (optional)\nMYCELLM_NO_DHT=true              # Disable Kademlia DHT\n```\n\nSee [docs/config](https://docs.mycellm.dev/config/settings/) for full reference.\n\n## API\n\nOpenAI-compatible. Works with any client that supports the OpenAI API format.\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(base_url=\"http://localhost:8420/v1\", api_key=\"unused\")\nresponse = client.chat.completions.create(\n    model=\"auto\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello!\"}],\n)\nprint(response.choices[0].message.content)\n```\n\n### Endpoints\n\n| Method | Path | Description |\n|--------|------|-------------|\n| GET | `/health` | Health check |\n| GET | `/v1/models` | List available models |\n| POST | `/v1/chat/completions` | Chat (streaming + non-streaming) |\n| GET | `/v1/node/status` | Node status |\n| GET | `/v1/node/peers` | Connected peers |\n| POST | `/v1/node/models/load` | Load a model |\n| POST | `/v1/node/federation/invite` | Create network invite |\n| POST | `/v1/node/federation/join` | Join a network |\n\nSee [API docs](https://docs.mycellm.dev/api/overview/) for the full reference.\n\n## Private Networks\n\nCreate a private network for your team, lab, or organization:\n\n```bash\n# On the bootstrap node\nmycellm init --bootstrap --name \"my-org\"\n\n# Generate an invite\nmycellm network invite --max-uses 10\n\n# On member nodes\nmycellm network join mcl_invite_eyJ...\n```\n\n## Contributing\n\nmycellm is open source under the Apache 2.0 license.\n\n```bash\ngit clone https://github.com/mycellm/mycellm\ncd mycellm\npip install -e \".[dev]\"\npytest\n```\n\n## Built with AI\n\nThis project was developed in collaboration with [Claude Code](https://claude.ai/code) by Anthropic. Claude served as a pair-programming partner throughout architecture design, implementation, and testing. All technical decisions, project direction, and code review are my own.\n\n## Credits\n\nBuilt by [Michael Gifford-Santos](https://github.com/mijkal).\n\n- **AI pair programming**: [Claude Code](https://claude.ai/code) by Anthropic\n- **Protocol**: QUIC + CBOR + Ed25519\n- **Inference**: [llama.cpp](https://github.com/ggerganov/llama.cpp) by Georgi Gerganov\n- **DHT**: [kademlia](https://github.com/bmuller/kademlia) by Brian Muller\n- **iOS inference**: [llama.swift](https://github.com/mattt/llama.swift) by Mattt\n\n## License\n\nApache 2.0 — see [LICENSE](LICENSE).\n\n\"mycellm\" and the mycellm logo are trademarks of Michael Gifford-Santos.\nSee [TRADEMARK.md](TRADEMARK.md) for usage guidelines.\n\n---\n\n\u003cp align=\"center\"\u003e\n  \u003csub\u003emycellm_ — /my·SELL·em/ — mycelium + LLM\u003c/sub\u003e\n\u003c/p\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmycellm%2Fmycellm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmycellm%2Fmycellm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmycellm%2Fmycellm/lists"}