{"id":50838203,"url":"https://github.com/bside89/dispatch-api","last_synced_at":"2026-06-14T05:30:44.165Z","repository":{"id":346871725,"uuid":"1179312674","full_name":"bside89/dispatch-api","owner":"bside89","description":"E-commerce API built with NestJS, Redis \u0026 PostgreSQL. Async processing with BullMQ, idempotency, and observability with Grafana + Loki.","archived":false,"fork":false,"pushed_at":"2026-06-07T08:16:31.000Z","size":1299,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-07T10:12:33.503Z","etag":null,"topics":["backend","bullmq","docker","grafana","i18n","loki","nestjs","nodejs","postgresql","redis","stripe-api","typescript"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bside89.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-11T22:51:05.000Z","updated_at":"2026-05-06T21:06:57.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/bside89/dispatch-api","commit_stats":null,"previous_names":["bside89/order-flow-api","bside89/dispatch-api"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/bside89/dispatch-api","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bside89%2Fdispatch-api","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bside89%2Fdispatch-api/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bside89%2Fdispatch-api/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bside89%2Fdispatch-api/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bside89","download_url":"https://codeload.github.com/bside89/dispatch-api/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bside89%2Fdispatch-api/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34310801,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-14T02:00:07.365Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backend","bullmq","docker","grafana","i18n","loki","nestjs","nodejs","postgresql","redis","stripe-api","typescript"],"created_at":"2026-06-14T05:30:43.241Z","updated_at":"2026-06-14T05:30:44.160Z","avatar_url":"https://github.com/bside89.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Dispatch\n\n![Node](https://img.shields.io/badge/node-20+-green)\n![NestJS](https://img.shields.io/badge/nestjs-backend-red)\n![Docker](https://img.shields.io/badge/docker-ready-blue)\n![License](https://img.shields.io/badge/license-MIT-lightgrey)\n\n---\n\n## Overview\n\nDispatch is an order management API built with NestJS. It is a portfolio project built to work through the architectural problems that come up in real e-commerce backends: async processing, distributed locking, authentication/authorization and transactional guarantees.\n\n---\n\n## Why I built this\n\nMost backend systems eventually hit the same problems:\n\n- Processing things asynchronously without losing data\n- Decoupling logic so parts of the app can scale\n- Tracking down a bug across multiple distributed flows\n- Retrying failed jobs safely without duplicating records\n\nThe instinct is to reach for microservices to solve these, but that brings a lot of operational baggage. I wanted to see how far I could push these patterns while keeping the monolith.\n\n---\n\n## Quick start\n\n**Prerequisites**\n\nBefore starting, make sure you have the following installed:\n\n- **Docker \u0026 Docker Compose**: To orchestrate the containers.\n- **Git**: To clone the repository.\n\n**Getting Started**\n\n**1.** Clone the repository:\n\n```bash\ngit clone https://github.com/bside89/dispatch-api\ncd dispatch-api\n```\n\n**2.** Run the installation script:\n\n```bash\nchmod +x install.sh \u0026\u0026 ./install.sh\n```\n\nThis script will automatically create your `.env.local` and `.env.production` files from the example files and start the services using `docker compose --env-file .env.production up --build`.\n\n\u003e **Note:** If you prefer to run manually, ensure you copy `.env.example.production` to `.env.production` and `.env.example.local` to `.env.local` before running `docker compose --env-file .env.production up --build`.\n\u003e **Note:** The Compose file reads `${...}` values during interpolation, so when using `.env.production` you must pass it explicitly with `docker compose --env-file .env.production up --build`.\n\n\u003e **Stripe tip:** Set `STRIPE_EXEC_MODE` to `local`, `docker`, or `live` depending on how you want to test payments. The details are in the Stripe testing section below.\n\n**3.** Access:\n\nAPI: http://localhost:3000\n\nBull Board: http://localhost:3000/bull-board\n\nGrafana: http://localhost:3001\n\nWhen `SEED_TEST_DATA` is `true`, the app creates a mock admin user on startup if it does not already exist:\n\n- Name: João Silva Admin\n- Email: joao.silva@email.com\n- Password: password123\n- Role: admin\n\nThis user is meant for local and development testing only. In `production`, it is not created.\n\n---\n\n## Architecture highlights\n\n- **Queue-based processing (BullMQ)**  \n  Orders, payments and side effects (like notifications) are processed through the respective queues in the background with exponential backoff. If the process job fails, it retries up to 3 times before triggering the compensation flow.\n\n- **Strategy + Factory patterns**  \n  Each order job type (PROCESS, CANCEL, REFUND) has a dedicated strategy class. Adding a new job type means adding one class — the processor stay untouched.\n\n- **Idempotent job execution**  \n  Jobs carry the order ID and target status in their payload. Before executing, the strategy re-reads the database and validates the precondition. A PAID → PROCESSED job running twice gets blocked on the second run.\n\n- **Cache-aside pattern**  \n  Heavy-requested endpoints in the frontend (like listing Items/Products) are cached inside Redis with a default TTL. After any Product modification the cache is invalidated.\n\n- **Payments gateway (Stripe)**  \n  PaymentIntent objects are created along with Order. Customer objects are created along with User. When the payment is confirmed, the application receives the appropriate webhook through a endpoint and start processing the Order.\n\n- **Centralized logging with correlationId**  \n  Every request gets a correlation ID injected at the middleware level. Async jobs carry it forward so you can trace a single order across all log lines, even across queue hops.\n\n- **High-throughput outbox processor**  \n  Uses recursive polling with `setImmediate` between batches to yield back to the event loop. A spike in queued events doesn't starve other requests.\n\n- **Race conditions control**  \n  Methods and jobs are executed with lock protection. Before running the method/job acquires a Redlock lock with specific operation key, preventing same method/job running at the same time.\n\n- **Transactional operations**  \n  Database-write methods/jobs run inside a TransactionalContext. If some error occur before the operation completes, a rollback occur and nothing is persisted, guaranteeing atomicity.\n\n---\n\n## Order processing flow\n\n1. Client creates an order — Stripe PaymentIntent is created, order sits at PENDING\n2. Stripe fires a webhook when the payment settles\n3. On success: order moves to PAID, ORDER_PROCESS is added to the outbox\n4. Outbox processor dispatches ORDER_PROCESS to BullMQ\n5. ORDER_PROCESS worker runs automatically: PAID → PROCESSED\n6. Admin ships the order: `PATCH /orders/:id/ship` → PROCESSED → SHIPPED (accepts optional `trackingNumber` and `carrier`)\n7. Admin confirms delivery: `PATCH /orders/:id/deliver` → SHIPPED → DELIVERED\n8. Admin can cancel pre-shipment: `PATCH /orders/:id/cancel` — ORDER_CANCEL is enqueued, stock is restored, order ends at CANCELLED\n9. Admin can trigger a refund: `PATCH /orders/:id/refund` — ORDER_REFUND is enqueued, Stripe processes the refund\n10. On payment failure: ORDER_CANCEL is enqueued automatically — same cancel and restore logic applies\n\nThe sequence diagram:\n\n```mermaid\nsequenceDiagram\n    autonumber\n    participant Client\n    participant API as Orders Controller/Service\n    participant Stripe as Stripe API\n    participant DB as PostgreSQL (Transaction)\n    participant Worker as Outbox Processor\n    participant Queue as BullMQ (Order \u0026 Notify)\n\n    Note over Client, Stripe: [PHASE 1: ORDER CREATION]\n    Client-\u003e\u003eAPI: POST /orders\n    activate API\n    Note over API, DB: Start Transaction\n    API-\u003e\u003eDB: Save Order (status: PENDING)\n    API-\u003e\u003eStripe: Create PaymentIntent\n    Stripe--\u003e\u003eAPI: PaymentIntent (id, clientSecret)\n    API-\u003e\u003eDB: Save paymentIntentId + paymentIntentStatus\n    DB--\u003e\u003eAPI: Success\n    API--\u003e\u003eClient: 201 Created (clientSecret, Correlation-ID)\n    deactivate API\n\n    Note over Stripe, DB: [PHASE 2: PAYMENT WEBHOOK]\n    Stripe-\u003e\u003eAPI: POST /payments/webhook (payment_intent.succeeded)\n    activate API\n    Note over API, DB: Start Transaction\n    API-\u003e\u003eDB: Update Order (status: PAID)\n    API-\u003e\u003eDB: Save Outbox (ORDER_PROCESS)\n    DB--\u003e\u003eAPI: Success\n    API--\u003e\u003eStripe: 200 OK\n    deactivate API\n\n    Note over DB, Queue: [PHASE 3: OUTBOX DISPATCH]\n    loop Continuous Processing\n        Worker-\u003e\u003eDB: Fetch pending Outbox events\n        DB--\u003e\u003eWorker: Events list\n        Worker-\u003e\u003eQueue: Dispatch jobs\n        Queue--\u003e\u003eWorker: Ack (Job IDs)\n        Worker-\u003e\u003eDB: Delete processed Outbox entries\n    end\n\n    Note over Queue, DB: [PHASE 4: AUTOMATIC PROCESSING]\n    rect rgba(128, 128, 128, 0.1)\n        Note right of Queue: ORDER_PROCESS\n        Queue-\u003e\u003eDB: Update Order to PROCESSED\n        Queue-\u003e\u003eDB: Save Outbox (EVENTS_NOTIFY_USER)\n    end\n\n    Note over Client, DB: [PHASE 5: MANUAL FULFILLMENT]\n    rect rgba(100, 149, 237, 0.1)\n        Client-\u003e\u003eAPI: PATCH /orders/:id/ship (Admin only)\n        activate API\n        API-\u003e\u003eDB: Update Order (SHIPPED, shippedAt, trackingNumber, carrier)\n        API-\u003e\u003eDB: Save Outbox (EVENTS_NOTIFY_USER)\n        DB--\u003e\u003eAPI: Success\n        API--\u003e\u003eClient: 200 OK\n        deactivate API\n\n        Client-\u003e\u003eAPI: PATCH /orders/:id/deliver (Admin only)\n        activate API\n        API-\u003e\u003eDB: Update Order (DELIVERED, deliveredAt)\n        API-\u003e\u003eDB: Save Outbox (EVENTS_NOTIFY_USER)\n        DB--\u003e\u003eAPI: Success\n        API--\u003e\u003eClient: 200 OK\n        deactivate API\n    end\n```\n\n---\n\n## Observability and monitoring\n\n- Structured logging with Pino (JSON)\n- Correlation ID for end-to-end tracing\n- Log aggregation via Promtail + Loki\n- Visualization with Grafana\n\n---\n\n## Testing strategy\n\nIntegration and E2E tests spin up real PostgreSQL and Redis containers via Testcontainers. No mocked databases, no \"works on my machine\" surprises.\n\nThere's also a k6 load test that hammers the queue under concurrent load to confirm jobs don't get processed twice when retries kick in.\n\n---\n\n## Stripe testing\n\nStripe behavior is controlled by `STRIPE_EXEC_MODE`.\n\n- `local`: starts `stripe-mock` in Docker and points the app to `localhost:12111`. Use this when you run the API on your machine.\n- `docker`: starts `stripe-mock` in Docker and points the app to `stripe-mock:12111`. Use this when the whole stack runs inside Docker.\n- `live`: talks to Stripe's test environment. You need to put your own Stripe test secret key in `.env`.\n\nThe integration and E2E tests mock `PaymentGatewaysService`, so they do not depend on Stripe at all. If you want to test real Stripe behavior, switch to `live`. If you just want the app to run without external calls, keep `local` or `docker`.\n\n---\n\n## Features\n\n- Async user notification system\n- Cache endpoints with intensive read\n- Idempotency for requests and jobs\n- Atomic and secure (from race conditions) operations\n- Authentication with role-based support\n- Secure logout (session invalidation)\n- Efficient error tracking with structured logs\n\n---\n\n## Engineering trade-offs\n\n| Decision                    | Reason                                                                                                                                                                                              |\n| --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Monolith over Microservices | No service discovery, no cross-service network calls, no distributed tracing setup. The constraints are worth it for a project at this scale.                                                       |\n| BullMQ over Kafka           | Kafka's strength is ordered, partitioned streams across consumer groups. BullMQ with Redis covers the actual requirements: reliable retries, per-queue concurrency caps, and rate limiting.         |\n| Partial event-driven        | Only order processing and notifications go through the queues. Auth and user management are plain request/response — adding async complexity there would solve a problem this project doesn't have. |\n\n---\n\n## Production considerations\n\nA few things that would matter in a real deployment:\n\nThe outbox pattern gives at-least-once delivery guarantees. Events are written to the database in the same transaction as the state change, so a crash between \"state updated\" and \"event dispatched\" can't lose the event. Duplicate dispatch is prevented by idempotency checks at the job level.\n\nDistributed locking via Redlock ensures concurrent webhook deliveries for the same order don't cause split-brain state. The lock covers the full transaction.\n\nBullMQ retries with exponential backoff handle transient failures. Jobs that exhaust all retries get logged with full context so failures are traceable.\n\n---\n\n## What's worth looking at\n\nA few things in this codebase that aren't obvious from the feature list:\n\nThe outbox processor (`shared/modules/outbox/`) uses a recursive `setImmediate` loop to drain event batches without blocking the event loop. Under load, it batches aggressively while still yielding between iterations.\n\nEach Order job has its compensation logic. If the job failed after all retries and payment is not processed, the Order has its status changed to CANCELED. If the payment is already processed, a job calls the refund endpoint from Stripe and change the Order status to REFUNDED.\n\n---\n\n## Final thoughts\n\nI built this to work through patterns I reach for in production — the outbox, distributed locking, hybrid sync/async flows. It's a portfolio project, but the problems it's solving are real.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbside89%2Fdispatch-api","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbside89%2Fdispatch-api","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbside89%2Fdispatch-api/lists"}