An open API service indexing awesome lists of open source software.

https://github.com/insider77circle/darkswarm

P2P decentralized swarm for collective open-source LLM inference. Uncensored, self-hosted, distributed AI. No central authority, no API fees. llama.cpp-based peer-to-peer compute mesh for local LLMs.
https://github.com/insider77circle/darkswarm

decentralized distributed llm open-source p2p peer-to-peer privacy self-hosted swarm uncensored

Last synced: 1 day ago
JSON representation

P2P decentralized swarm for collective open-source LLM inference. Uncensored, self-hosted, distributed AI. No central authority, no API fees. llama.cpp-based peer-to-peer compute mesh for local LLMs.

Awesome Lists containing this project

README

          


DarkSwarm P2P LLM Mesh Demo

# DarkSwarm

### The open-source P2P mesh for decentralized LLM inference. Run local AI across self-hosted nodes with zero API fees.

**Privacy. Control. Freedom. No central authority.**


[![GitHub stars](https://img.shields.io/github/stars/Insider77Circle/DarkSwarm?style=for-the-badge&logo=github)](https://github.com/Insider77Circle/DarkSwarm/stargazers)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue?style=for-the-badge)](LICENSE)
[![Python 3.9+](https://img.shields.io/badge/Python-3.9+-green?style=for-the-badge&logo=python&logoColor=white)](https://www.python.org/)
[![Latest Release](https://img.shields.io/badge/Release-v0.1.0-orange?style=for-the-badge)](https://github.com/Insider77Circle/DarkSwarm/releases)

Discord
Follow on X
Ask Questions

---

## 🎯 Why DarkSwarm?

DarkSwarm solves a critical problem: **centralized AI is expensive, censored, and extracts your data.**

| | OpenAI/Claude/Gemini | Self-Hosted Alone | **DarkSwarm** |
|---|---|---|---|
| **Cost per 1M tokens** | $15–$60 | Hardware + power | **$0** (peer-shared) |
| **Privacy** | ❌ Sent to OpenAI | ✅ Local only | ✅ Local only |
| **Latency** | 100–500ms+ | 10–100ms | 10–100ms (optimized) |
| **Uptime** | API dependent | Single point of failure | **99.9%+ (mesh redundancy)** |
| **Censorship** | ❌ OpenAI content policy | ✅ Your policy | ✅ Your policy |
| **Model Freedom** | Limited models | ✅ Any GGUF | ✅ Any GGUF |

DarkSwarm is a **peer-to-peer compute mesh** that runs local LLMs (via llama.cpp) across self-hosted nodes. Think of it as a **decentralized, censorship-resistant alternative to OpenAI's API**, but owned and operated by you and your community.

---

## 🚀 Quick Start

**Prerequisites:**
- Python 3.9+ or Docker
- [A local LLM model](https://huggingface.co/models?other=llm&sort=trending&search=GGUF)

### One-Command Start

```bash
# Clone the repo
git clone https://github.com/Insider77Circle/DarkSwarm.git && cd DarkSwarm

# Install dependencies
pip install -r requirements.txt

# Run a node (adjust model path)
python3 run_node.py --model /path/to/model.gguf --name my-node-01
```

**Expected output:**
```
[INFO] Node 'my-node-01' initialized
[INFO] Listening on P2P port 5000
[INFO] Web UI available at http://localhost:8080
[INFO] Ready to accept peer connections...
```

Visit **`http://localhost:8080`** to see your node's dashboard.

👉 **[Full Quickstart Guide](docs/QUICKSTART.md)** — Python, Docker, and multi-node examples

---

## ✨ Key Features

### 🔗 **Peer-to-Peer Mesh Architecture**
- **Decentralized node discovery** — nodes find each other without a central server
- **Self-healing network** — nodes join/leave gracefully, mesh keeps working
- **No single point of failure** — if one node goes down, inference continues

### ⚡ **Distributed Inference Engine**
- **Intelligent load balancing** — routes requests to the fastest available node
- **Model distribution** — different nodes run different models; mesh optimizes routing
- **Batch inference** — combines requests for higher throughput
- **Real-time latency metrics** — see which nodes are fastest for your workload

### 🛡️ **Privacy-First Design**
- **Zero data leaves your infrastructure** — all inference stays local, P2P
- **End-to-end encryption** — all inter-node communication is encrypted
- **No telemetry or tracking** — complete transparency
- **Open-source & auditable** — inspect every line of code

### 🎨 **Simple Web UI**
- **Real-time node monitoring** — see active nodes, model status, throughput
- **Interactive inference testing** — test models directly from browser
- **Network visualizer** — watch your mesh topology live
- **Performance dashboard** — latency, throughput, resource utilization

### 🔧 **Developer-Friendly API**
```python
from darkswarm import DarkSwarmClient

client = DarkSwarmClient(bootstrap_nodes=["localhost:5000"])
response = client.infer(
prompt="What is decentralized AI?",
model="auto",
temperature=0.7
)
print(response.text)
```

### 📦 **Compatible with llama.cpp Ecosystem**
- Run **any GGUF-format model** (Llama, Mistral, Qwen, etc.)
- Leverage **llama.cpp's optimizations** (ARM support, GPU acceleration)
- Drop-in replacement for oobabooga, text-generation-webui, ollama

---

## 🏗️ Architecture Overview

DarkSwarm uses **mesh topology** where nodes communicate directly:

```
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Node 1 │ │ Node 2 │ │ Node 3 │
│ Llama 2-7B │◄────►│ Mistral-7B │◄────►│ Qwen-14B │
└─────────────┘ └─────────────┘ └─────────────┘
▲ ▲ ▲
│ │ │
└────────┬───────────┴─────────────────────┘
│ P2P Mesh Network (libp2p)
┌────────▼──────────┐
│ Your Application │
│ (Client) │
└───────────────────┘
```

**How It Works:**
1. Client submits inference request to any node
2. Local node queries peers for model availability
3. Request routes to best-fit peer (lowest latency + available model)
4. Inference runs on selected peer, results stream back
5. Network automatically load-balances and recovers from failures

📖 **[Full Architecture Guide](docs/ARCHITECTURE.md)** — Protocol, security model, message flow

---

## 🎯 Use Cases

### 💻 **Local AI for Development**
Run LLMs locally during development without API costs or rate limits.

### 🏥 **Privacy-Sensitive Applications**
Healthcare, legal, finance — keep data completely local and encrypted.

### 🌍 **Edge Deployment**
Run inference on edge devices, IoT networks, or geographically distributed infrastructure.

### 👥 **Community AI Sharing**
Join a trusted P2P mesh with colleagues/friends to pool compute resources and costs.

### 🔐 **Censorship-Resistant AI**
Ensure your AI service isn't subject to any third-party API provider's policies.

### 📡 **Offline-First Systems**
Mesh works without internet — just connect nodes over LAN or intranet.

---

## 💰 Cost Comparison

| Scenario | Traditional API | Self-Hosted Alone | **DarkSwarm** |
|----------|---|---|---|
| **Single app, 1M inferences/month** | $30–$60 | GPU: $300–$800/mo | **$0** (your hardware) |
| **Scale to 10M inferences** | $300–$600 | Multiple servers | **Add nodes, same cost** |
| **Data privacy** | ❌ Shared with provider | ✅ Local only | ✅ Local only |
| **Downtime risk** | Provider outage | Single server fails | **Mesh continues** |
| **Vendor lock-in** | Heavy (API format) | Medium (model format) | **None (pure P2P)** |

**Bottom line:** For organizations doing serious inference work, DarkSwarm eliminates recurring API fees and gives you complete control.

---

## 📚 Documentation

- **[🚀 Quickstart Guide](docs/QUICKSTART.md)** — Get running in 5 min (Python/Docker/Multi-node)
- **[🏗️ Architecture & Protocol](docs/ARCHITECTURE.md)** — Deep dive into mesh, routing, security
- **[📖 API Reference](docs/API.md)** — Full client library and REST API docs
- **[🔐 Security Model](docs/SECURITY.md)** — Threat model, encryption, trust assumptions
- **[🤝 Contributing Guide](CONTRIBUTING.md)** — How to contribute code, docs, features
- **[📜 Code of Conduct](CODE_OF_CONDUCT.md)** — Community guidelines

---

## 🎮 Real-World Examples

### Example 1: Multi-Node Inference

```bash
# Terminal 1: Start node with Mistral
python3 run_node.py --model ./models/mistral-7b.gguf --name node-1 --port 5000

# Terminal 2: Start node with Llama and connect
python3 run_node.py --model ./models/llama2-13b.gguf --name node-2 --port 5001 --peer localhost:5000

# Terminal 3: Query the mesh
curl -X POST http://localhost:8080/infer \
-H "Content-Type: application/json" \
-d '{"prompt": "Explain quantum computing", "model": "auto"}'
```

The mesh **automatically routes to the fastest node**:
```json
{
"text": "Quantum computing leverages quantum mechanics...",
"model_used": "llama2-13b",
"node_used": "node-2",
"latency_ms": 387,
"peer_latencies": {"node-1": 523, "node-2": 387}
}
```

### Example 2: Python Client

```python
from darkswarm import DarkSwarmClient

# Create client
client = DarkSwarmClient(bootstrap_nodes=["localhost:5000"])

# Simple inference
response = client.infer(
prompt="What is P2P networking?",
model="auto",
max_tokens=256
)
print(f"Response: {response.text}")
print(f"Latency: {response.latency_ms}ms")

# Streaming for long responses
for chunk in client.infer_stream(
prompt="Write a story about AI",
max_tokens=500
):
print(chunk.text, end="", flush=True)

# Batch processing
responses = client.infer_batch(
prompts=["What is AI?", "What is ML?", "What is DL?"],
model="mistral-7b"
)
```

---

## 🗺️ Roadmap

| Version | Features | ETA |
|---------|----------|-----|
| **v0.2** | Model caching & optimization layer | Q3 2025 |
| **v0.3** | Payment/incentive system for mesh participants | Q4 2025 |
| **v0.4** | GPU acceleration & hardware-specific optimizations | Q1 2026 |
| **v0.5** | Web-based mesh explorer & analytics dashboard | Q2 2026 |
| **v1.0** | Production-ready with comprehensive benchmarks | Q3 2026 |

👉 **[Vote on features](https://github.com/Insider77Circle/DarkSwarm/discussions)** — Tell us what you want!

---

## 🤝 Contributing

We welcome contributions! Whether code, docs, or ideas:

- **Code:** Bug fixes, features, performance improvements
- **Documentation:** Guides, tutorials, examples
- **Community:** Testing, feedback, spreading the word

👉 **[Contributing Guide](CONTRIBUTING.md)** — Get started in 5 minutes

**First-time contributor?** Look for [`good-first-issue`](https://github.com/Insider77Circle/DarkSwarm/labels/good-first-issue) labels.

---

## 🐛 Support & Community

- **Questions?** [Open a Discussion](https://github.com/Insider77Circle/DarkSwarm/discussions)
- **Found a bug?** [File an Issue](https://github.com/Insider77Circle/DarkSwarm/issues)
- **Chat with us:** [Join Discord](https://discord.gg/darkswarm)
- **Follow updates:** [X / Twitter](https://x.com/darkswarm_ai)

---

## 📜 License & Acknowledgments

DarkSwarm is released under the **MIT License** — see [**LICENSE**](LICENSE).

**Built on:**
- [**llama.cpp**](https://github.com/ggerganov/llama.cpp) — efficient local LLM inference
- [**libp2p**](https://libp2p.io/) — P2P networking protocol
- [**Hugging Face**](https://huggingface.co/) — LLM model distribution
- Open-source community — privacy advocates & distributed systems researchers

---

## ⭐ Support DarkSwarm

**Love DarkSwarm?**
- ⭐ **Star the repo** ([GitHub](https://github.com/Insider77Circle/DarkSwarm))
- 🐦 **Share on X/Twitter** (mention [@darkswarm_ai](https://x.com/darkswarm_ai))
- 👥 **Invite others** to join the community
- 📝 **Write about it** — blog posts, tutorials, case studies

---

> [!WARNING]
> **Legal Disclaimer:** You are responsible for complying with all applicable laws and regulations regarding data privacy, AI use, and open-source licensing. Ensure you have permission to run LLMs on your infrastructure and that your use case complies with model licenses and local regulations.

---

**Ready to take back control of your AI?**

[**Get Started Now →**](docs/QUICKSTART.md)