{"id":50664750,"url":"https://github.com/marcuwynu23/nodejs-rabbitmq-sample","last_synced_at":"2026-06-08T05:03:58.419Z","repository":{"id":357033902,"uuid":"1234316481","full_name":"marcuwynu23/nodejs-rabbitmq-sample","owner":"marcuwynu23","description":"This project demonstrates a simple RabbitMQ implementation using Node.js and TypeScript. It follows the Producer-Consumer pattern to handle file upload metadata asynchronously.","archived":false,"fork":false,"pushed_at":"2026-05-11T02:00:31.000Z","size":21,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-11T03:34:22.902Z","etag":null,"topics":["demonstration","example","message-broker","message-queue","rabbitmq","rabbitmq-consumer","rabbitmq-producer"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/marcuwynu23.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-10T02:54:53.000Z","updated_at":"2026-05-11T02:00:36.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/marcuwynu23/nodejs-rabbitmq-sample","commit_stats":null,"previous_names":["marcuwynu23/nodejs-rabbitmq-sample"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/marcuwynu23/nodejs-rabbitmq-sample","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcuwynu23%2Fnodejs-rabbitmq-sample","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcuwynu23%2Fnodejs-rabbitmq-sample/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcuwynu23%2Fnodejs-rabbitmq-sample/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcuwynu23%2Fnodejs-rabbitmq-sample/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/marcuwynu23","download_url":"https://codeload.github.com/marcuwynu23/nodejs-rabbitmq-sample/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcuwynu23%2Fnodejs-rabbitmq-sample/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34048707,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-08T02:00:07.615Z","response_time":111,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["demonstration","example","message-broker","message-queue","rabbitmq","rabbitmq-consumer","rabbitmq-producer"],"created_at":"2026-06-08T05:03:56.496Z","updated_at":"2026-06-08T05:03:58.405Z","avatar_url":"https://github.com/marcuwynu23.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Node.js RabbitMQ File Upload Metadata Sample\n\nThis project demonstrates a simple RabbitMQ implementation using Node.js and TypeScript. It follows the **Producer-Consumer** pattern to handle file upload metadata asynchronously.\n\n## Workflow Diagram\n\n```mermaid\nsequenceDiagram\n    participant User\n    participant Producer as Express API (Producer)\n    participant Queue as RabbitMQ Queue\n    participant Consumer as Worker (Consumer)\n\n    User-\u003e\u003eProducer: POST /upload-metadata (Metadata)\n    Note over Producer: Validates Metadata\n    Producer-\u003e\u003eQueue: Publish Message (Persistent)\n    Producer--\u003e\u003eUser: 202 Accepted (Message Queued)\n\n    Queue-\u003e\u003eConsumer: Deliver Message\n    Note over Consumer: Parses JSON \u0026 Processes\n    Consumer--\u003e\u003eQueue: Acknowledge (Ack)\n    Note over Consumer: Processing Complete\n```\n\n## How It Works\n\n1.  **Producer (Express Server)**:\n    - Listens on port `3000`.\n    - Exposes an endpoint `POST /upload-metadata`.\n    - When it receives metadata, it validates the fields and pushes the data onto a RabbitMQ queue named `file_metadata_queue`.\n    - The producer returns an immediate `202 Accepted` response, ensuring the user doesn't wait for processing.\n\n2.  **RabbitMQ (Message Broker)**:\n    - Acts as a buffer between the API and the worker.\n    - Ensures messages are persistent (stored on disk) so they aren't lost if the server restarts.\n\n3.  **Consumer (Background Worker)**:\n    - Connects to the same `file_metadata_queue`.\n    - Consumes messages one by one.\n    - Simulates processing (e.g., database entry, thumbnail generation) and acknowledges the message once done.\n\n## Prerequisites\n\n- [Node.js](https://nodejs.org/) (v16+)\n- [Docker Desktop](https://www.docker.com/products/docker-desktop/)\n\n## Getting Started\n\n### 1. Setup Environment\n\nClone the repository and install dependencies:\n\n```bash\npnpm install\n```\n\n### 2. Start RabbitMQ\n\nRun the RabbitMQ server using Docker Compose from the root of the project directory:\n\n```bash\ndocker-compose up -d\n```\n\n\u003e You can access the RabbitMQ Management UI at [http://localhost:15672](http://localhost:15672) (User/Pass: `guest`/`guest`).\n\n### 3. Run the Application\n\nOpen two separate terminals:\n\n**Terminal 1 (Consumer):**\n\n```bash\npnpm run start:consumer\n```\n\n**Terminal 2 (Producer):**\n\n```bash\npnpm run start:producer\n```\n\n### 4. Test the API\n\nSend a sample metadata request using PowerShell:\n\n```powershell\nInvoke-RestMethod -Uri http://localhost:3000/upload-metadata -Method Post -Body (@{ filename=\"vacation_photo.jpg\"; size=2048576; type=\"image/jpeg\"; uploader=\"mark_wayne\" } | ConvertTo-Json) -ContentType \"application/json\"\n```\n\n## Project Structure\n\n- `src/producer.ts`: Express server that publishes messages to RabbitMQ.\n- `src/consumer.ts`: Background worker that consumes and processes messages.\n- `docker-compose.yml`: RabbitMQ service configuration.\n- `test_request.ps1`: Helper script for testing the API.\n\n---\n\n## Why use RabbitMQ for File Uploads?\n\n### RabbitMQ vs. Direct Processing\n\nIn a **Direct Processing** model, the server handles everything (upload, validation, database entry, thumbnail generation) before sending a response to the user.\n\n- **Problem**: Large files or heavy processing (like video transcoding) cause high latency. If the server crashes during processing, the job is lost.\n- **RabbitMQ Solution**: The server only handles the \"receipt\" of the upload and immediately hands off the task to a queue. This **decouples** the API from the heavy lifting, providing a faster user experience and better reliability.\n\n### RabbitMQ vs. Kafka\n\nBoth are powerful, but they serve different purposes:\n\n| Feature          | RabbitMQ (Message Broker)                                                  | Apache Kafka (Event Streaming)                               |\n| :--------------- | :------------------------------------------------------------------------- | :----------------------------------------------------------- |\n| **Primary Goal** | Delivering messages to consumers as fast as possible.                      | Storing streams of events for long-term replay.              |\n| **Complexity**   | Lightweight and easy to set up (Smart Broker/Dumb Consumer).               | More complex setup (Dumb Broker/Smart Consumer).             |\n| **Ordering**     | Guaranteed within a queue.                                                 | Guaranteed within a partition.                               |\n| **Usecase**      | **Best for task queues**, background jobs, and microservice communication. | Best for log aggregation, real-time analytics, and big data. |\n\n**Summary**: For a file upload scenario where you just need to ensure a background task (like processing metadata or generating a thumbnail) gets done, **RabbitMQ** is usually the simpler and more appropriate choice. Kafka is overkill unless you need to replay those upload events multiple times for different analytics engines.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarcuwynu23%2Fnodejs-rabbitmq-sample","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarcuwynu23%2Fnodejs-rabbitmq-sample","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarcuwynu23%2Fnodejs-rabbitmq-sample/lists"}