{"id":50973551,"url":"https://github.com/7irelo/dispatch-core","last_synced_at":"2026-06-19T05:02:19.051Z","repository":{"id":341389789,"uuid":"1169990714","full_name":"7irelo/dispatch-core","owner":"7irelo","description":"A distributed, horizontally scalable job processing engine built with .NET 8, PostgreSQL, and Redis. Provides delayed execution, exponential retries with jitter, dead-letter handling, per-tenant rate limiting, and safe multi-worker coordination using transactional job claiming and distributed locking.","archived":false,"fork":false,"pushed_at":"2026-03-18T10:21:33.000Z","size":59,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-03-19T01:34:13.552Z","etag":null,"topics":["concurrency","dapper","distributed-locking","distributed-systems","docker","dotnet","multi-tenant","opentelemetry","rate-limiting","redis","serilog"],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/7irelo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-01T14:41:30.000Z","updated_at":"2026-03-18T10:21:39.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/7irelo/dispatch-core","commit_stats":null,"previous_names":["7irelo/dispatch-core"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/7irelo/dispatch-core","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/7irelo%2Fdispatch-core","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/7irelo%2Fdispatch-core/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/7irelo%2Fdispatch-core/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/7irelo%2Fdispatch-core/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/7irelo","download_url":"https://codeload.github.com/7irelo/dispatch-core/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/7irelo%2Fdispatch-core/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34517752,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-19T02:00:06.005Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["concurrency","dapper","distributed-locking","distributed-systems","docker","dotnet","multi-tenant","opentelemetry","rate-limiting","redis","serilog"],"created_at":"2026-06-19T05:02:18.154Z","updated_at":"2026-06-19T05:02:19.041Z","avatar_url":"https://github.com/7irelo.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Dispatch Core\n\nA production-grade distributed job processing system built with .NET 8. Designed as an alternative to Hangfire with first-class support for multi-tenancy, rate limiting, and horizontal scaling.\n\n[![Build \u0026 Test](https://github.com/7irelo/dispatch-core/actions/workflows/build.yml/badge.svg)](https://github.com/7irelo/dispatch-core/actions/workflows/build.yml)\n\n\u003cp\u003e\n  \u003cimg width=\"49%\" alt=\"Dashboard\" src=\"https://github.com/user-attachments/assets/6939e6d4-0776-451f-9fb2-2d8599475b70\" /\u003e\n  \u003cimg width=\"49%\" alt=\"Job Detail\" src=\"https://github.com/user-attachments/assets/00d41d0d-aed8-44cb-ae33-cf986d1dfbb0\" /\u003e\n\u003c/p\u003e\n\n## Features\n\n- **Atomic job claiming** — `SELECT ... FOR UPDATE SKIP LOCKED` prevents double-processing\n- **Multi-tenant** — every job is scoped to a `TenantId` with per-tenant rate limiting\n- **Idempotent submissions** — duplicate `IdempotencyKey` per tenant returns the original job\n- **Exponential backoff** — retries with jitter, automatic dead-lettering after max attempts\n- **Redis token bucket** — configurable rate limits per tenant (default 10 jobs/min)\n- **Distributed locking** — Redis locks with Lua-based compare-and-delete release\n- **Partition sharding** — workers can target specific `PartitionKey` values for workload isolation\n- **Channel-based scheduler** — bounded `Channel\u003cT\u003e` with backpressure and configurable concurrency\n- **Crash recovery** — lock reaper resets expired locks so stalled jobs get re-processed\n- **Blazor dashboard** — real-time metrics, job search, cancel/requeue actions\n- **Observability** — Serilog structured logging, OpenTelemetry tracing + metrics, health checks\n\n## Architecture\n\n```\n┌──────────────┐     ┌──────────────────┐     ┌───────────────────┐\n│   API        │     │   Worker         │     │   Dashboard       │\n│   (REST)     │     │   (BackgroundSvc)│     │   (Blazor Server) │\n└──────┬───────┘     └────────┬─────────┘     └─────────┬─────────┘\n       │                      │                         │\n       ├──────────────────────┼─────────────────────────┤\n       │                      │                         │\n  ┌────▼────┐  ┌──────────────▼──────────────┐    ┌─────▼─────┐\n  │Contracts│  │  Core (models, interfaces,  │    │  Storage   │\n  │ (DTOs)  │  │  retry policy, scheduling)  │    │  (Dapper)  │\n  └─────────┘  └──────────────┬──────────────┘    └─────┬─────┘\n                              │                         │\n               ┌──────────────┼──────────────┐          │\n               │              │              │          │\n          ┌────▼───┐   ┌─────▼─────┐  ┌─────▼─────┐   │\n          │Locking │   │ RateLimit │  │ Executor  │   │\n          │(Redis) │   │ (Redis)   │  │ (Channel) │   │\n          └────────┘   └───────────┘  └───────────┘   │\n                                                       │\n                              ┌─────────────────────────┘\n                              │\n                    ┌─────────▼─────────┐\n                    │  PostgreSQL + Redis │\n                    └───────────────────┘\n```\n\n| Project | Description |\n|---------|-------------|\n| `DispatchCore.Api` | .NET 8 Minimal API — job submission, queries, admin |\n| `DispatchCore.Worker` | Worker Service — polls, executes, reaps stale locks |\n| `DispatchCore.Dashboard` | Blazor Server — admin UI with metrics and job management |\n| `DispatchCore.Contracts` | Shared DTOs — `CreateJobRequest`, `JobResponse`, `MetricsResponse` |\n| `DispatchCore.Core` | Domain — `Job` model, interfaces, `RetryPolicy` |\n| `DispatchCore.Storage` | Postgres persistence — Dapper repos, migration runner |\n| `DispatchCore.Locking` | Redis distributed locking with Lua release scripts |\n| `DispatchCore.RateLimit` | Redis token bucket rate limiter |\n| `DispatchCore.Executor` | Bounded channel scheduler, handler registry, execution pipeline |\n\n## Getting Started\n\n### Prerequisites\n\n- [.NET 8 SDK](https://dotnet.microsoft.com/download/dotnet/8.0)\n- [Docker](https://www.docker.com/) (for Postgres and Redis)\n\n### 1. Start Infrastructure\n\n```bash\ndocker-compose up -d\n```\n\nThis starts:\n- **PostgreSQL 16** on port `5432` (db: `dispatch_core`, user: `dispatch`, pass: `dispatch_secret`)\n- **Redis 7** on port `6379`\n\n### 2. Run the API\n\n```bash\ndotnet run --project src/DispatchCore.Api\n```\n\nMigrations run automatically on startup. The API is available at `http://localhost:5000`.\n\n### 3. Run the Worker\n\n```bash\ndotnet run --project src/DispatchCore.Worker\n```\n\nThe worker starts polling for due jobs immediately.\n\n### 4. Run the Dashboard (optional)\n\n```bash\ndotnet run --project src/DispatchCore.Dashboard\n```\n\nNavigate to `http://localhost:5002` and click \"Login as Admin\".\n\n## API Endpoints\n\n| Method | Path | Description |\n|--------|------|-------------|\n| `POST` | `/jobs` | Submit a new job |\n| `GET` | `/jobs/{id}` | Get job by ID |\n| `GET` | `/tenants/{tenantId}/jobs` | List jobs for a tenant (`?limit=50\u0026offset=0`) |\n| `POST` | `/jobs/{id}/cancel` | Cancel a pending/scheduled job |\n| `POST` | `/admin/requeue-deadletter/{id}` | Requeue a dead-lettered job |\n| `GET` | `/admin/metrics` | Aggregate job status counts |\n| `GET` | `/health` | Health check (Postgres + Redis) |\n\n### Submit a Job\n\n```bash\ncurl -X POST http://localhost:5000/jobs \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"tenantId\": \"acme-corp\",\n    \"type\": \"email.send\",\n    \"payload\": { \"to\": \"user@example.com\", \"subject\": \"Hello\" },\n    \"maxAttempts\": 5,\n    \"idempotencyKey\": \"welcome-email-user-42\"\n  }'\n```\n\nSubmitting the same `idempotencyKey` for the same `tenantId` returns the original job instead of creating a duplicate.\n\n### Schedule a Job\n\n```bash\ncurl -X POST http://localhost:5000/jobs \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"tenantId\": \"acme-corp\",\n    \"type\": \"report.generate\",\n    \"payload\": { \"reportId\": \"monthly-sales\" },\n    \"runAt\": \"2024-12-01T09:00:00Z\"\n  }'\n```\n\n## Job Lifecycle\n\n```\n  ┌─────────┐    RunAt \u003c= now    ┌─────────┐\n  │ Pending ├───────────────────►│ Running │\n  └────┬────┘                    └────┬────┘\n       │                              │\n  RunAt \u003e now                    ┌────┴────┐\n       │                         │         │\n  ┌────▼─────┐            success│         │failure\n  │Scheduled ├──RunAt\u003c=now──►    │         │\n  └──────────┘                   │         │\n                           ┌─────▼──┐  ┌───▼────────┐\n                           │Succeeded│  │ attempts \u003c │\n                           └────────┘  │ max?       │\n                                       └──┬────┬────┘\n                                     yes  │    │ no\n                                ┌─────────▼┐ ┌─▼──────────┐\n                                │  Pending  │ │ DeadLetter │\n                                │(retry w/  │ └────────────┘\n                                │ backoff)  │\n                                └──────────┘\n```\n\nJobs can also be **cancelled** (moves to `Failed` with \"Cancelled by user\") or **requeued** from dead letter (resets to `Pending` with attempts zeroed).\n\n## Job Model\n\n| Column | Type | Description |\n|--------|------|-------------|\n| `job_id` | `UUID` | Primary key |\n| `tenant_id` | `TEXT` | Tenant identifier |\n| `type` | `TEXT` | Handler type (e.g. `email.send`) |\n| `payload` | `JSONB` | Arbitrary JSON payload |\n| `status` | `ENUM` | Pending, Scheduled, Running, Succeeded, Failed, DeadLetter |\n| `run_at` | `TIMESTAMPTZ` | When the job becomes eligible for processing |\n| `attempts` | `INT` | Current attempt count |\n| `max_attempts` | `INT` | Max retries before dead-lettering (default 3) |\n| `last_error` | `TEXT` | Error message from last failure |\n| `locked_by` | `TEXT` | Worker ID holding the lock |\n| `lock_until` | `TIMESTAMPTZ` | Lock expiry (reaper resets if past) |\n| `partition_key` | `TEXT` | Optional partition for worker sharding |\n| `idempotency_key` | `TEXT` | Unique per tenant for deduplication |\n\n## Writing Job Handlers\n\nImplement `IJobHandler` and register it in the worker's handler registry:\n\n```csharp\npublic sealed class InvoiceHandler : IJobHandler\n{\n    public string JobType =\u003e \"invoice.generate\";\n\n    public async Task HandleAsync(Job job, CancellationToken ct)\n    {\n        var payload = JsonSerializer.Deserialize\u003cInvoicePayload\u003e(job.Payload);\n        // your logic here\n    }\n}\n```\n\nRegister in `Program.cs`:\n\n```csharp\nregistry.Register(new InvoiceHandler(logger));\n```\n\nBuilt-in sample handlers: `email.send`, `report.generate`.\n\n## Worker Configuration\n\nConfigure via `appsettings.json` or environment variables:\n\n```json\n{\n  \"Worker\": {\n    \"PollIntervalMs\": 1000,\n    \"BatchSize\": 10,\n    \"Concurrency\": 5,\n    \"ReaperIntervalMs\": 30000\n  }\n}\n```\n\n| Setting | Default | Description |\n|---------|---------|-------------|\n| `PollIntervalMs` | `1000` | Milliseconds between polling cycles |\n| `BatchSize` | `10` | Max jobs claimed per poll |\n| `Concurrency` | `5` | Max parallel job executions |\n| `ReaperIntervalMs` | `30000` | Milliseconds between lock reaper sweeps |\n\n### Partition Sharding\n\nRun workers targeting specific partitions to isolate workloads:\n\n```bash\nWORKER_PARTITION_KEY=us-east dotnet run --project src/DispatchCore.Worker\nWORKER_PARTITION_KEY=eu-west dotnet run --project src/DispatchCore.Worker\n```\n\nWorkers without a partition key process all jobs regardless of partition.\n\n## Rate Limiting\n\nRate limiting uses a Redis token bucket algorithm. The default is **10 jobs per minute per tenant**.\n\nOverride per-tenant limits in the `tenant_rate_limits` table:\n\n```sql\nINSERT INTO tenant_rate_limits (tenant_id, max_per_minute)\nVALUES ('high-volume-tenant', 100)\nON CONFLICT (tenant_id) DO UPDATE SET max_per_minute = 100, updated_at = now();\n```\n\nWhen a tenant exceeds their limit, the job is **rescheduled without incrementing the attempt counter** — it retries transparently after a short delay.\n\n## Migrations\n\nSQL migration scripts live in `/migrations` and are applied idempotently on startup by both the API and Worker:\n\n| Script | Purpose |\n|--------|---------|\n| `001_create_jobs_table.sql` | Jobs table, `job_status` enum, indexes |\n| `002_create_rate_limit_config.sql` | Per-tenant rate limit overrides |\n| `003_create_migration_history.sql` | Migration tracking table |\n\nTo add a new migration, create `004_your_migration.sql` in the `/migrations` directory. It will be applied automatically on next startup.\n\n## Testing\n\n```bash\n# Unit tests (no infrastructure needed)\ndotnet test tests/DispatchCore.Tests.Unit\n\n# Integration tests (requires Docker for Testcontainers)\ndotnet test tests/DispatchCore.Tests.Integration\n```\n\n**Unit tests** (19 tests) cover:\n- Retry policy — exponential backoff calculation, dead letter threshold\n- Idempotency — deduplication logic, tenant isolation\n- Job executor — rate limit reschedule, handler dispatch, retry/dead letter flow\n\n**Integration tests** (9 tests) use Testcontainers to spin up real Postgres and Redis:\n- Job CRUD and round-trip persistence\n- Idempotency key enforcement and tenant isolation\n- Atomic claiming with `FOR UPDATE SKIP LOCKED`\n- Partition-based polling\n- Lock reaper behavior\n- Token bucket rate limiter enforcement\n\n## CI/CD\n\nGitHub Actions runs on every push and pull request:\n\n- **Build \u0026 Unit Tests** — restore, build, run unit tests, upload `.trx` artifacts\n- **Integration Tests** — spins up Postgres 16 + Redis 7 service containers, runs integration tests\n\n## Tech Stack\n\n| Component | Technology |\n|-----------|------------|\n| Runtime | .NET 8 |\n| API | ASP.NET Core Minimal APIs |\n| Worker | `BackgroundService` |\n| Dashboard | Blazor Server |\n| Database | PostgreSQL 16 |\n| Cache/Locking | Redis 7 |\n| ORM | Dapper + Npgsql |\n| Logging | Serilog |\n| Tracing | OpenTelemetry |\n| Testing | xUnit, FluentAssertions, NSubstitute, Testcontainers |\n\n## License\n\n[MIT](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F7irelo%2Fdispatch-core","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F7irelo%2Fdispatch-core","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F7irelo%2Fdispatch-core/lists"}