{"id":26270993,"url":"https://github.com/techmedaddy/torrentedge","last_synced_at":"2026-04-09T01:31:24.215Z","repository":{"id":257731756,"uuid":"859125394","full_name":"techmedaddy/TorrentEdge","owner":"techmedaddy","description":"TorrentEdge is a distributed BitTorrent client using Node.js, Nginx, and Kafka. It enables peer-to-peer file sharing, real-time updates, and scalable communication. Docker is used for deployment, and the frontend is built with React.","archived":false,"fork":false,"pushed_at":"2026-03-14T17:56:22.000Z","size":923,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-03-15T04:05:39.637Z","etag":null,"topics":["docker","docker-compose","expressjs","javascript","kafka","mongodb","nginx","nodejs"],"latest_commit_sha":null,"homepage":"https://medium.com/@techmedaddy/torrentedge-a-distributed-bittorrent-client-acb4c06b9c27","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/techmedaddy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-09-18T06:09:16.000Z","updated_at":"2026-03-14T17:56:25.000Z","dependencies_parsed_at":"2024-09-18T10:16:53.587Z","dependency_job_id":"b00dc5ea-18b3-4396-b6a9-3d1c2ebc3cc2","html_url":"https://github.com/techmedaddy/TorrentEdge","commit_stats":null,"previous_names":["umarei/torrentedge","techmedaddy/torrentedge"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/techmedaddy/TorrentEdge","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/techmedaddy%2FTorrentEdge","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/techmedaddy%2FTorrentEdge/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/techmedaddy%2FTorrentEdge/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/techmedaddy%2FTorrentEdge/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/techmedaddy","download_url":"https://codeload.github.com/techmedaddy/TorrentEdge/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/techmedaddy%2FTorrentEdge/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31581864,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-08T14:31:17.711Z","status":"ssl_error","status_checked_at":"2026-04-08T14:31:17.202Z","response_time":54,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","docker-compose","expressjs","javascript","kafka","mongodb","nginx","nodejs"],"created_at":"2025-03-14T06:18:10.668Z","updated_at":"2026-04-09T01:31:24.204Z","avatar_url":"https://github.com/techmedaddy.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TorrentEdge 🚀\r\n\r\nA modern, production-ready BitTorrent client built from scratch in Node.js with real-time monitoring and event streaming capabilities.\r\n\r\n[![Node.js](https://img.shields.io/badge/Node.js-18+-green.svg)](https://nodejs.org/)\r\n[![License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)\r\n[![Docker](https://img.shields.io/badge/docker-ready-brightgreen.svg)](docker-compose.yml)\r\n\r\n\r\n\r\n\u003cimg width=\"1600\" height=\"772\" alt=\"image\" src=\"https://github.com/user-attachments/assets/dca162c8-8e19-4ab7-a1ff-673ff049d749\" /\u003e\r\n\r\n\r\n## ✨ Features\r\n\r\n### Core BitTorrent Protocol\r\n- ✅ **Full BitTorrent Protocol** implementation (BEP 3)\r\n- ✅ **Magnet Link Support** with metadata exchange (BEP 9)\r\n- ✅ **Extension Protocol** for advanced features (BEP 10)\r\n- ✅ **Multi-file Torrents** with proper piece alignment\r\n- ✅ **Piece Verification** using SHA1 hashing\r\n- ✅ **Resume Downloads** with state persistence\r\n- ✅ **Tracker Protocol** (HTTP/HTTPS/UDP)\r\n\r\n### Advanced Features\r\n- 🔥 **Multi-Torrent Queue Management** with priority-based scheduling\r\n- 🔥 **Bandwidth Throttling** using token bucket algorithm\r\n- 🔥 **Upload Management** with tit-for-tat choking algorithm\r\n- 🔥 **Super-Seeding Mode** for efficient distribution\r\n- 🔥 **Piece Selection Strategies**: Rarest-first, endgame mode\r\n- 🔥 **Comprehensive Error Handling** with retry logic and peer banning\r\n- 🔥 **Per-File Progress Tracking** for multi-file torrents\r\n\r\n  \u003cimg width=\"1600\" height=\"772\" alt=\"image\" src=\"https://github.com/user-attachments/assets/ca19036e-ac5d-4c23-bcc6-44bca2604780\" /\u003e\r\n\r\n\r\n### Real-time \u0026 Scalable Architecture\r\n- 📡 **Socket.IO** for live progress updates to web clients\r\n- 📊 **Kafka Event Streaming** for analytics and monitoring\r\n- 💾 **MongoDB** for torrent metadata and user management\r\n- 🚀 **Redis Caching** for improved performance (optional)\r\n- 🐳 **Docker \u0026 Docker Compose** for easy deployment\r\n- 📈 **Health Monitoring** with tracker and peer health tracking\r\n\u003cimg width=\"1600\" height=\"772\" alt=\"image\" src=\"https://github.com/user-attachments/assets/d35a62ce-426c-4eed-b52c-bc72857bc3a8\" /\u003e\r\n\r\n### Production Ready\r\n- ⚡ **Concurrent Download Management** (default: 3 active, unlimited queued)\r\n- 🔒 **Peer Ban System** with strike tracking\r\n- 🔄 **Automatic Reconnection** with exponential backoff\r\n- 📦 **State Backup \u0026 Recovery** with rotation\r\n- 🎯 **Smart Piece Selection** with availability tracking\r\n- 📝 **Comprehensive Logging** with categorized errors\r\n\r\n## 🏗️ Architecture\r\n\r\n```\r\n┌─────────────────────────────────────────────────────────────┐\r\n│                       TorrentEdge API                        │\r\n│                    (Express + Socket.IO)                     │\r\n└────────────┬───────────────────────────────────┬────────────┘\r\n             │                                   │\r\n    ┌────────▼─────────┐              ┌─────────▼──────────┐\r\n    │  Torrent Engine  │              │   Kafka Producer   │\r\n    │   (Core Logic)   │──────────────▶│   (Analytics)      │\r\n    └────────┬─────────┘              └────────────────────┘\r\n             │\r\n    ┌────────▼──────────────────────────────────────────────┐\r\n    │              Component Layer                          │\r\n    ├───────────────┬───────────────┬──────────────────────┤\r\n    │ Queue Manager │ State Manager │  Peer Manager        │\r\n    │ Download Mgr  │ Upload Mgr    │  Throttler           │\r\n    │ File Writer   │ Piece Manager │  Retry Manager       │\r\n    └───────────────┴───────────────┴──────────────────────┘\r\n                     │\r\n            ┌────────▼─────────┐\r\n            │  Peer Network    │\r\n            │  (TCP + Tracker) │\r\n            └──────────────────┘\r\n```\r\n\r\n### Key Components\r\n- **Torrent Engine**: Core orchestration for all torrent operations\r\n- **Queue Manager**: Priority-based multi-torrent scheduling\r\n- **Download Manager**: Piece selection, endgame mode, retry logic\r\n- **Upload Manager**: Tit-for-tat choking, super-seeding\r\n- **Peer Manager**: Connection pooling, health tracking, reconnection\r\n- **State Manager**: Persistence with backup rotation\r\n- **Tracker Manager**: Multi-tracker failover with health states\r\n\r\n  \u003cimg width=\"1600\" height=\"772\" alt=\"image\" src=\"https://github.com/user-attachments/assets/cb8418b7-bba8-445f-b273-b2a549b4d3ec\" /\u003e\r\n\r\n\r\n## 🚀 Quick Start\r\n\r\n### Prerequisites\r\n- **Node.js** 18+ and npm\r\n- **MongoDB** 4.4+ (for persistence)\r\n- **Redis** 6+ (optional, for caching)\r\n- **Apache Kafka** 2.8+ (optional, for analytics)\r\n\r\n### Installation\r\n\r\n```bash\r\n# Clone the repository\r\ngit clone https://github.com/yourusername/torrentedge.git\r\ncd torrentedge\r\n\r\n# Install dependencies\r\nnpm install\r\n\r\n# Configure environment\r\ncp .env.example .env\r\n# Edit .env with your configuration\r\n\r\n# Start the server\r\nnpm run dev\r\n```\r\n\r\n### Docker Setup\r\n\r\n```bash\r\n# Start all services (MongoDB, Redis, Kafka, TorrentEdge)\r\ndocker-compose up -d\r\n\r\n# View logs\r\ndocker-compose logs -f torrentedge\r\n\r\n# Stop services\r\ndocker-compose down\r\n```\r\n\r\nThe application will be available at:\r\n- **API**: http://localhost:3000\r\n- **WebSocket**: ws://localhost:3000\r\n\r\n## 📡 API Reference\r\n\r\n### Add Torrent from File\r\n\r\n```bash\r\ncurl -X POST http://localhost:3000/api/torrents \\\r\n  -H \"Authorization: Bearer YOUR_TOKEN\" \\\r\n  -F \"file=@ubuntu.torrent\"\r\n```\r\n\r\n### Add Magnet Link\r\n\r\n```bash\r\ncurl -X POST http://localhost:3000/api/torrents \\\r\n  -H \"Authorization: Bearer YOUR_TOKEN\" \\\r\n  -H \"Content-Type: application/json\" \\\r\n  -d '{\r\n    \"magnetURI\": \"magnet:?xt=urn:btih:HASH\u0026dn=Name\u0026tr=tracker_url\"\r\n  }'\r\n```\r\n\r\n### Get All Torrents\r\n\r\n```bash\r\ncurl http://localhost:3000/api/torrents \\\r\n  -H \"Authorization: Bearer YOUR_TOKEN\"\r\n```\r\n\r\n### Get Torrent Status\r\n\r\n```bash\r\ncurl http://localhost:3000/api/torrents/:infoHash \\\r\n  -H \"Authorization: Bearer YOUR_TOKEN\"\r\n```\r\n\r\n### Pause/Resume Torrent\r\n\r\n```bash\r\n# Pause\r\ncurl -X POST http://localhost:3000/api/torrents/:infoHash/pause \\\r\n  -H \"Authorization: Bearer YOUR_TOKEN\"\r\n\r\n# Resume\r\ncurl -X POST http://localhost:3000/api/torrents/:infoHash/resume \\\r\n  -H \"Authorization: Bearer YOUR_TOKEN\"\r\n```\r\n\r\n### Remove Torrent\r\n\r\n```bash\r\ncurl -X DELETE http://localhost:3000/api/torrents/:infoHash \\\r\n  -H \"Authorization: Bearer YOUR_TOKEN\"\r\n```\r\n\r\n### Set Priority\r\n\r\n```bash\r\ncurl -X PUT http://localhost:3000/api/torrents/:infoHash/priority \\\r\n  -H \"Authorization: Bearer YOUR_TOKEN\" \\\r\n  -H \"Content-Type: application/json\" \\\r\n  -d '{\"priority\": \"high\"}'\r\n```\r\n\r\n### Get Statistics\r\n\r\n```bash\r\ncurl http://localhost:3000/api/statistics \\\r\n  -H \"Authorization: Bearer YOUR_TOKEN\"\r\n```\r\n\r\n## 🔌 WebSocket Events\r\n\r\n### Client → Server\r\n\r\n```javascript\r\nconst socket = io('http://localhost:3000');\r\n\r\n// Authenticate\r\nsocket.emit('authenticate', { token: 'YOUR_TOKEN' });\r\n\r\n// Subscribe to torrent updates\r\nsocket.emit('subscribe:torrent', { infoHash: 'HASH' });\r\n```\r\n\r\n### Server → Client\r\n\r\n```javascript\r\n// Torrent added\r\nsocket.on('torrent:added', (data) =\u003e {\r\n  console.log('New torrent:', data.infoHash);\r\n});\r\n\r\n// Progress update\r\nsocket.on('torrent:progress', (data) =\u003e {\r\n  console.log(`${data.name}: ${data.percentage.toFixed(2)}%`);\r\n  console.log(`Speed: ${formatBytes(data.downloadSpeed)}/s`);\r\n  console.log(`Peers: ${data.peers.connected}/${data.peers.total}`);\r\n});\r\n\r\n// Piece completed\r\nsocket.on('torrent:piece', (data) =\u003e {\r\n  console.log(`Piece ${data.index} completed`);\r\n});\r\n\r\n// Download complete\r\nsocket.on('torrent:complete', (data) =\u003e {\r\n  console.log(`Download complete: ${data.name}`);\r\n});\r\n\r\n// Seeding started\r\nsocket.on('torrent:seeding', (data) =\u003e {\r\n  console.log(`Seeding: ${data.name}`);\r\n  console.log(`Ratio: ${data.ratio.toFixed(2)}`);\r\n});\r\n\r\n// Error occurred\r\nsocket.on('torrent:error', (data) =\u003e {\r\n  console.error(`Error: ${data.message}`);\r\n});\r\n\r\n// Queue updated\r\nsocket.on('queue:updated', (data) =\u003e {\r\n  console.log(`Active: ${data.stats.activeCount}`);\r\n  console.log(`Queued: ${data.stats.queuedCount}`);\r\n});\r\n```\r\n\u003cimg width=\"1600\" height=\"772\" alt=\"image\" src=\"https://github.com/user-attachments/assets/27be6a13-04b4-4585-a7c6-738744f04760\" /\u003e\r\n\r\n## ⚙️ Configuration\r\n\r\n### Environment Variables\r\n\r\n| Variable | Description | Default |\r\n|----------|-------------|---------|\r\n| `PORT` | Server port | `3000` |\r\n| `MONGODB_URI` | MongoDB connection string | `mongodb://localhost:27017/torrentedge` |\r\n| `REDIS_HOST` | Redis host | `localhost` |\r\n| `REDIS_PORT` | Redis port | `6379` |\r\n| `KAFKA_BROKERS` | Kafka broker list | `localhost:9092` |\r\n| `JWT_SECRET` | JWT signing secret | `your-secret-key` |\r\n| `DOWNLOAD_PATH` | Default download directory | `./downloads` |\r\n| `MAX_CONCURRENT_TORRENTS` | Max active downloads | `3` |\r\n| `MAX_PEER_CONNECTIONS` | Max peers per torrent | `50` |\r\n| `UPLOAD_LIMIT` | Global upload limit (bytes/s) | `0` (unlimited) |\r\n| `DOWNLOAD_LIMIT` | Global download limit (bytes/s) | `0` (unlimited) |\r\n\r\n### Torrent Engine Options\r\n\r\n```javascript\r\nconst engine = new TorrentEngine({\r\n  downloadPath: './downloads',\r\n  port: 6881,\r\n  maxConcurrent: 3,\r\n  maxConnections: 50,\r\n  uploadLimit: 1024 * 1024,      // 1 MB/s\r\n  downloadLimit: 5 * 1024 * 1024, // 5 MB/s\r\n  seedRatioLimit: 2.0,            // Stop seeding at 2.0 ratio\r\n  seedTimeLimit: 0,               // Seed forever (minutes)\r\n  autoResume: true,               // Resume on restart\r\n  verifyOnResume: false           // Skip verification on resume\r\n});\r\n```\r\n\r\n## 🧪 Testing\r\n\r\n```bash\r\n# Run all tests\r\nnpm test\r\n\r\n# Run unit tests\r\nnpm run test:unit\r\n\r\n# Run integration tests\r\nnpm run test:integration\r\n\r\n# Run with coverage\r\nnpm run test:coverage\r\n```\r\n\r\n## 📊 Performance\r\n\r\n- ✅ Handles **100+ concurrent torrents**\r\n- ✅ Supports **1000+ peer connections**\r\n- ✅ Tested with torrents up to **50GB+**\r\n- ✅ Memory efficient: **~50MB base + ~10MB per active torrent**\r\n- ✅ CPU efficient: **~5% on average, ~20% during verification**\r\n\r\n### Benchmarks\r\n\r\n| Operation | Performance |\r\n|-----------|-------------|\r\n| Piece verification | ~150 MB/s |\r\n| File writes | ~200 MB/s |\r\n| Peer handshakes | ~100/s |\r\n| Socket.IO events | ~10k/s |\r\n| Kafka messages | ~50k/s |\r\n\r\n## 🛠️ Tech Stack\r\n\r\n### Core\r\n- **Runtime**: Node.js 18+\r\n- **Protocol**: BitTorrent (BEP 3, 9, 10)\r\n- **Network**: TCP/IP, UDP\r\n- **Cryptography**: SHA1, crypto-js\r\n\r\n### Backend\r\n- **Framework**: Express.js\r\n- **Real-time**: Socket.IO\r\n- **Database**: MongoDB + Mongoose\r\n- **Cache**: Redis\r\n- **Message Queue**: Apache Kafka + KafkaJS\r\n\r\n### DevOps\r\n- **Containerization**: Docker\r\n- **Orchestration**: Docker Compose\r\n- **Reverse Proxy**: Nginx\r\n\r\n## 📁 Project Structure\r\n\r\n```\r\nTorrentEdge/\r\n├── src/\r\n│   ├── server/\r\n│   │   ├── torrentEngine/          # Core BitTorrent implementation\r\n│   │   │   ├── torrent.js          # Torrent state machine\r\n│   │   │   ├── engine.js           # Main engine orchestrator\r\n│   │   │   ├── queueManager.js     # Multi-torrent queue\r\n│   │   │   ├── stateManager.js     # Persistence layer\r\n│   │   │   ├── peerManager.js      # Peer connection pool\r\n│   │   │   ├── peerConnection.js   # Individual peer protocol\r\n│   │   │   ├── downloadManager.js  # Download coordination\r\n│   │   │   ├── uploadManager.js    # Seeding and uploads\r\n│   │   │   ├── fileWriter.js       # Disk I/O\r\n│   │   │   ├── pieceManager.js     # Piece verification\r\n│   │   │   ├── throttler.js        # Bandwidth limiting\r\n│   │   │   ├── retryManager.js     # Error handling\r\n│   │   │   └── tracker.js          # Tracker protocol\r\n│   │   ├── controllers/            # API controllers\r\n│   │   ├── models/                 # MongoDB models\r\n│   │   ├── routes/                 # Express routes\r\n│   │   ├── middleware/             # Auth, validation\r\n│   │   ├── kafka/                  # Kafka producers/consumers\r\n│   │   └── server.js               # Main entry point\r\n│   └── api/                        # API controllers\r\n├── tests/                          # Test suites\r\n├── nginx/                          # Nginx configuration\r\n├── docker-compose.yml              # Docker setup\r\n├── package.json\r\n└── README.md\r\n```\r\n\r\n## 🔍 Key Algorithms\r\n\r\n### Piece Selection\r\n1. **Rarest First**: Download rarest pieces first to improve swarm health\r\n2. **Random First**: First 4 pieces selected randomly for quick startup\r\n3. **Endgame Mode**: Request last pieces from multiple peers (at 95% completion)\r\n\r\n### Choking Algorithm (Tit-for-Tat)\r\n- Unchoke top 4 uploaders every 10 seconds\r\n- Optimistic unchoke rotates every 30 seconds\r\n- Rewards peers who upload to us\r\n- Prevents freeloading\r\n\r\n### Token Bucket Throttling\r\n- Bucket fills at configured rate (bytes/second)\r\n- Each transfer consumes tokens\r\n- Burst support up to 1 second worth of tokens\r\n- Separate buckets for upload/download\r\n\r\n### Error Handling\r\n- **Retry Manager**: Exponential backoff with jitter\r\n- **Peer Banning**: Strike system (3 strikes → temporary ban)\r\n- **Tracker Failover**: Multi-tracker with health states\r\n- **Automatic Recovery**: Reconnection with backoff\r\n\r\n## 🤝 Contributing\r\n\r\nContributions are welcome! Please follow these steps:\r\n\r\n1. Fork the repository\r\n2. Create a feature branch (`git checkout -b feature/amazing-feature`)\r\n3. Commit your changes (`git commit -m 'Add amazing feature'`)\r\n4. Push to the branch (`git push origin feature/amazing-feature`)\r\n5. Open a Pull Request\r\n\r\n\r\n\r\n\r\n\r\n---\r\n\r\n\r\n\r\nFor detailed architecture documentation, see [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md).\r\n\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftechmedaddy%2Ftorrentedge","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftechmedaddy%2Ftorrentedge","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftechmedaddy%2Ftorrentedge/lists"}