{"id":29406316,"url":"https://github.com/7irelo/aptiverse-worker","last_synced_at":"2025-07-10T23:19:58.953Z","repository":{"id":303221169,"uuid":"1014522214","full_name":"7irelo/aptiverse-worker","owner":"7irelo","description":"AI Task Processor for RabbitMQ Queue","archived":false,"fork":false,"pushed_at":"2025-07-06T11:56:42.000Z","size":8,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-06T12:44:07.072Z","etag":null,"topics":["csharp","polly","rabbitmq","restsharp"],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/7irelo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-05T22:23:50.000Z","updated_at":"2025-07-06T12:03:40.000Z","dependencies_parsed_at":"2025-07-06T12:44:08.944Z","dependency_job_id":"32752608-fa73-439a-a38c-a0c6d2f3694b","html_url":"https://github.com/7irelo/aptiverse-worker","commit_stats":null,"previous_names":["7irelo/aptiverse-worker"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/7irelo/aptiverse-worker","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/7irelo%2Faptiverse-worker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/7irelo%2Faptiverse-worker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/7irelo%2Faptiverse-worker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/7irelo%2Faptiverse-worker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/7irelo","download_url":"https://codeload.github.com/7irelo/aptiverse-worker/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/7irelo%2Faptiverse-worker/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264686771,"owners_count":23649564,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["csharp","polly","rabbitmq","restsharp"],"created_at":"2025-07-10T23:19:58.291Z","updated_at":"2025-07-10T23:19:58.939Z","avatar_url":"https://github.com/7irelo.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🧠 Aptiverse Worker – AI Task Processor for RabbitMQ Queue\n\nThe **Aptiverse Worker** is a background service responsible for consuming messages from **RabbitMQ**, processing requests related to AI/ML models (hosted in a Python FastAPI service), and returning results or updating relevant data stores.\n\nThis service acts as a decoupling layer between the **Aptiverse API (.NET 8)** and the **FastAPI-based AI services**, enabling **asynchronous**, **resilient**, and **scalable** model execution.\n\n---\n\n## 🎯 Purpose\n\n* ✅ Decouple long-running AI inference from the main API\n* ✅ Asynchronously process tasks such as summarization, emotion detection, or learning analysis\n* ✅ Communicate with FastAPI microservices over HTTP or internal APIs\n* ✅ Enable retry, failure logging, and dead-letter queueing\n* ✅ Support concurrent message consumption with worker scaling\n\n---\n\n## 🔧 Tech Stack\n\n| Component        | Technology                         |\n| ---------------- | ---------------------------------- |\n| Language         | .NET 8 / C#                        |\n| Message Broker   | RabbitMQ                           |\n| API Bridge       | RestSharp (to call FastAPI)        |\n| Hosting          | Docker, AWS ECS / EC2              |\n| Config \u0026 Secrets | `appsettings.json`, ENV vars       |\n| Monitoring       | Serilog / OpenTelemetry (optional) |\n\n---\n\n## 📦 Project Structure\n\n```\nAptiverse.Worker\n├── Program.cs               → Worker entry point (HostBuilder)\n├── Services\n│   └── MessageProcessor.cs  → Handles incoming messages\n├── Consumers\n│   └── RabbitMqConsumer.cs  → Connects to queue and consumes tasks\n├── Clients\n│   └── FastApiClient.cs     → Uses RestSharp to call Python models\n├── Models\n│   └── TaskPayload.cs       → Task definitions and data contracts\n├── Config\n│   └── RabbitMqSettings.cs  → Queue \u0026 connection configuration\n├── Utilities\n│   └── RetryPolicy.cs       → Polly-based retry handling\n└── appsettings.json         → Config for queue, API, and logging\n```\n\n---\n\n## 🚀 Getting Started\n\n### 1. Clone the Repository\n\n```bash\ngit clone https://github.com/your-org/aptiverse-worker.git\ncd aptiverse-worker\n```\n\n### 2. Configure RabbitMQ and FastAPI Endpoint\n\nUpdate `appsettings.json`:\n\n```json\n\"RabbitMQ\": {\n  \"Host\": \"rabbitmq://localhost\",\n  \"Queue\": \"ai-tasks\"\n},\n\"FastApi\": {\n  \"BaseUrl\": \"http://localhost:8000/api\"\n}\n```\n\nOr use environment variables in deployment.\n\n### 3. Run the Worker\n\n```bash\ndotnet build\ndotnet run\n```\n\nOr run with Docker:\n\n```bash\ndocker build -t aptiverse-worker .\ndocker run --env-file .env aptiverse-worker\n```\n\n---\n\n## 📬 Message Format (Example)\n\n```json\n{\n  \"taskId\": \"abc123\",\n  \"userId\": \"student001\",\n  \"taskType\": \"summarization\",\n  \"inputText\": \"Photosynthesis is the process by which...\"\n}\n```\n\n---\n\n## 🔁 Message Flow\n\n1. `.NET API` sends JSON payload to RabbitMQ queue (`ai-tasks`)\n2. `RabbitMqConsumer` listens for messages\n3. `MessageProcessor` routes the task type (e.g., summarization → `/summarize`)\n4. `FastApiClient` calls the correct endpoint\n5. Result is either:\n\n   * Sent back via HTTP webhook or callback\n   * Stored in database\n   * Logged and acknowledged\n\n---\n\n## 🧪 Testing the Worker\n\nYou can simulate a job by manually publishing to the queue using tools like:\n\n* **RabbitMQ Management UI**\n* `rabbitmqadmin` CLI\n* `Postman + API endpoint` that enqueues test jobs\n\n---\n\n## 🛡️ Error Handling\n\n* ✅ Retry policy using [Polly](https://github.com/App-vNext/Polly)\n* ❌ Failed messages routed to a **dead-letter queue**\n* 📦 Logs are emitted to console, file, or centralized log service\n\n---\n\n## 📈 Monitoring \u0026 Scaling\n\n* Add health check endpoints with [AspNetCore.Diagnostics.HealthChecks](https://github.com/Xabaril/AspNetCore.Diagnostics.HealthChecks)\n* Use **Prometheus + Grafana** or **AWS CloudWatch** for queue depth \u0026 worker status\n* Run multiple worker instances to scale horizontally\n\n---\n\n## 📌 Example Use Cases\n\n| Task Type         | FastAPI Endpoint    | Output Example              |\n| ----------------- | ------------------- | --------------------------- |\n| `summarization`   | `/summarize`        | Short paragraph summary     |\n| `emotion_check`   | `/emotion-analysis` | `[\"stressed\", \"motivated\"]` |\n| `topic_extractor` | `/topics`           | `[\"algebra\", \"cells\"]`      |\n\n---\n\n## 🧠 Future Improvements\n\n* Add OpenTelemetry tracing to visualize full API → MQ → AI call chain\n* Add job result callback to `.NET API` or store results in PostgreSQL\n* Add in-memory cache for duplicate task detection\n* Rate limit queue submissions per student or IP\n\n---\n\n## 🤝 Contribution\n\nWe welcome feedback and contributions! Open an issue, create a pull request, or suggest improvements to task routing and error handling.\n\n---\n\n## 🪪 License\n\nThis project is proprietary under Aptiverse Labs. Licensing options for collaborators will be available in the public launch phase.\n\n---\n\n## 💡 Part of the Aptiverse Ecosystem\n\n\u003e *“Bridging the gap between intelligence, emotion, and access.”*\n\u003e The Aptiverse Worker is the engine that powers real-time educational insights, smart recommendations, and scalable AI processing across South Africa's next-generation learning platform.\n\n---\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F7irelo%2Faptiverse-worker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F7irelo%2Faptiverse-worker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F7irelo%2Faptiverse-worker/lists"}