{"id":19419043,"url":"https://github.com/anchan828/nest-cloud-run-queue","last_synced_at":"2025-04-30T22:15:40.699Z","repository":{"id":37050818,"uuid":"233642584","full_name":"anchan828/nest-cloud-run-queue","owner":"anchan828","description":"Create a Queue/Worker for NestJS application in Cloud Run.","archived":false,"fork":false,"pushed_at":"2025-04-28T23:26:50.000Z","size":64364,"stargazers_count":41,"open_issues_count":4,"forks_count":4,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-30T22:15:24.858Z","etag":null,"topics":["nestjs","pubsub"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/anchan828.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-01-13T16:37:09.000Z","updated_at":"2025-04-28T23:26:52.000Z","dependencies_parsed_at":"2023-11-16T10:26:28.830Z","dependency_job_id":"d04db98e-efec-4fe0-85d4-2c3d3acfed03","html_url":"https://github.com/anchan828/nest-cloud-run-queue","commit_stats":{"total_commits":1601,"total_committers":6,"mean_commits":266.8333333333333,"dds":0.434103685196752,"last_synced_commit":"3d8990e79577a6e24d04bd6f2af7379952126b96"},"previous_names":["anchan828/nest-cloud-run-pubsub"],"tags_count":323,"template":false,"template_full_name":"anchan828/nest-lerna-template","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anchan828%2Fnest-cloud-run-queue","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anchan828%2Fnest-cloud-run-queue/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anchan828%2Fnest-cloud-run-queue/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anchan828%2Fnest-cloud-run-queue/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/anchan828","download_url":"https://codeload.github.com/anchan828/nest-cloud-run-queue/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251789622,"owners_count":21644087,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["nestjs","pubsub"],"created_at":"2024-11-10T13:15:59.635Z","updated_at":"2025-04-30T22:15:40.676Z","avatar_url":"https://github.com/anchan828.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# @anchan828/nest-cloud-run-queue\n\nCreate a Queue/Worker for [NestJS](https://nestjs.com/) application in Cloud Run.\n\n![nest-cloud-run-queue](https://user-images.githubusercontent.com/694454/164212037-6afd1e3a-ab0f-4f61-b607-469826d04ffb.png)\n\n## Overview\n\nI am wondering how to implement Queue when running an application with Cloud Run. While @nestjs/bull is a very good library, Cloud Run, which is serverless, cannot be used because the server is not always running. You can use \"CPU always allocated\" for resolve this issue, but it doesn't make sense to use Cloud Run.\n\nTherefore, I used Cloud Pub/Sub or Cloud Tasks so that I could implement Queue via HTTP requests. This package supports both, so you can choose whichever you prefer.\n\n[Choose Cloud Tasks or Pub/Sub](https://cloud.google.com/tasks/docs/comp-pub-sub)\n\nOf course, these packages can work without using Cloud Run on workers since they handle tasks via HTTP requests.\n\n## Demo\n\nSee [https://github.com/anchan828/nest-cloud-run-queue/tree/master/packages/demo](https://github.com/anchan828/nest-cloud-run-queue/tree/master/packages/demo#readme)\n\nThis demo uses an emulator, which runs PubSub and Tasks locally. Please see [docker compose.yml](https://github.com/anchan828/nest-cloud-run-queue/blob/master/docker compose.yml) if you are interested.\n\n## Packages\n\nThere are two types of packages.\n\n### Publisher\n\n| Package                                                                                                                            | Description                                       |\n| :--------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------ |\n| [@anchan828/nest-cloud-run-queue-pubsub-publisher](https://www.npmjs.com/package/@anchan828/nest-cloud-run-queue-pubsub-publisher) | Library for sending messages using Cloud Pub/Sub. |\n| [@anchan828/nest-cloud-run-queue-tasks-publisher](https://www.npmjs.com/package/@anchan828/nest-cloud-run-queue-tasks-publisher)   | Library for sending messages using Cloud Tasks.   |\n\n### Worker\n\n| Package                                                                                                        | Description                                                          |\n| :------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------- |\n| [@anchan828/nest-cloud-run-queue-worker](https://www.npmjs.com/package/@anchan828/nest-cloud-run-queue-worker) | Library for creating applications that receive and process messages. |\n\n## Getting started\n\n## 1. Create publisher application\n\n### Using Cloud Pub/Sub\n\nSee: [@anchan828/nest-cloud-run-queue-pubsub-publisher - README.md](https://github.com/anchan828/nest-cloud-run-queue/tree/master/packages/pubsub-publisher#readme)\n\n### Using Cloud Tasks\n\nSee: [@anchan828/nest-cloud-run-queue-tasks-publisher - README.md](https://github.com/anchan828/nest-cloud-run-queue/tree/master/packages/tasks-publisher#readme)\n\n## 2. Create worker application\n\n#### Import worker module\n\n```ts\n@Module({\n  imports: [QueueWorkerModule.register()],\n})\nexport class WorkerAppModule {}\n```\n\n#### Create worker provider\n\n```ts\n@QueueWorker(\"Worker name\")\n// @QueueWorker({ name: \"Worker name\" })\nclass Worker {\n  @QueueWorkerProcess()\n  public async process(message: string | object, raw: QueueWorkerRawMessage): Promise\u003cvoid\u003e {\n    console.log(\"Message:\", message);\n    console.log(\"Raw message:\", raw);\n  }\n}\n```\n\n### Add as provider\n\n```ts\n@Module({\n  imports: [QueueWorkerModule.register()],\n  providers: [Worker],\n})\nexport class WorkerAppModule {}\n```\n\n### Customize worker controller\n\nThe Controller who receives the message is automatically defined. You can customize it.\n\n```ts\n@Module({\n  imports: [\n    QueueWorkerModule.register({\n      workerController: {\n        method: RequestMethod.GET,\n        path: \"/worker\",\n      },\n\n      // Default\n      // workerController: {\n      //   method: RequestMethod.POST,\n      //   path: \"/\",\n      // },\n    }),\n  ],\n  providers: [Worker],\n})\nexport class WorkerAppModule {}\n```\n\nYou can also define your own Controller. In that case, set workerController to null.\n\n```ts\n@Controller(\"/worker\")\nclass WorkerController {\n  constructor(private readonly service: QueueWorkerService) {}\n\n  @Post()\n  public async execute(@Body() body: QueueWorkerReceivedMessage): Promise\u003cvoid\u003e {\n    const results = await this.service.execute(body.message);\n\n    for (const result of results) {\n      console.log(result.success);\n    }\n\n    // or\n\n    // const decodedMessage = decodeMessage(body.message);\n    // const results = await this.service.execute(decodedMessage);\n  }\n}\n\n@Module({\n  controllers: [WorkerController],\n  imports: [\n    QueueWorkerModule.register({\n      workerController: null,\n    }),\n  ],\n  providers: [Worker],\n})\nexport class WorkerAppModule {}\n```\n\n### Create bootstrap function\n\n```ts\nasync function bootstrap(): Promise\u003cvoid\u003e {\n  const app = await NestFactory.create(WorkerAppModule);\n  await app.listen(process.env.PORT || 8080);\n}\n\nbootstrap();\n```\n\n## Disable worker/provider\n\nYou can disable worker/provider by setting `enabled` to false.\nFor example, if you are reusing the same application, you can disable process on Cloud Run Service and enable it to run on Cloud Run Job.\n\n```ts\n@QueueWorker({ name: \"Worker name\", enabled: config.isEnabledWorker })\nclass Worker {\n  @QueueWorkerProcess({ enabled: config.isEnabledProcess })\n  public async process(message: string | object, raw: QueueWorkerRawMessage): Promise\u003cvoid\u003e {}\n\n  @QueueWorkerProcess()\n  public async process2(message: string | object, raw: QueueWorkerRawMessage): Promise\u003cvoid\u003e {\n    console.log(\"Message:\", message);\n    console.log(\"Raw message:\", raw);\n  }\n}\n```\n\n## Execute worker/processor manually\n\nYou can execute worker/processor manually.\n\n```ts\n@Controller(\"/worker\")\nclass WorkerController {\n  constructor(private readonly service: QueueWorkerService) {}\n\n  @Post()\n  public async execute(@Body() body: QueueWorkerReceivedMessage): Promise\u003cvoid\u003e {\n    const workers = await this.service.getWorkers(body.message);\n\n    for (const worker of workers) {\n      const processors = worker.getProcessors();\n\n      for (const processor of processors) {\n        const result = await processor.execute();\n\n        if (result.success) {\n          console.log(\"Success\");\n        } else {\n          console.log(\"Failed:\" + result.error.message);\n        }\n      }\n    }\n  }\n}\n```\n\n## Get all workers\n\nIf you want to set something up using worker metadata, you can retrieve all workers and process them.\n\n```ts\nasync function bootstrap(): Promise\u003cvoid\u003e {\n  const app = await NestFactory.create(WorkerAppModule);\n  const service = app.get(QueueWorkerService);\n  const allWorkers = service.getAllWorkers();\n  const allProcessors = allWorkers.flatMap((w) =\u003e w.processors);\n\n  // Do somethings...\n}\n```\n\n## Using Cloud Scheduler\n\nYou can use Cloud Scheduler as trigger.\n\nPayload is JSON string `{\"name\": \"worker name\", \"data\": \"str\"}`\n\n![](https://i.gyazo.com/a778c6a67eed6e525c38dd42378aa8bf.png)\n\n## Using as standalone\n\nThere may be times when you want to use it for a one-time call, such as Cloud Run jobs.\n\n```ts\nimport { QueueWorkerService } from \"@anchan828/nest-cloud-run-queue-worker\";\n\nasync function bootstrap(): Promise\u003cvoid\u003e {\n  const app = await NestFactory.createApplicationContext(WorkerAppModule);\n  await app.get(QueueWorkerService).execute({\n    name: \"worker name\",\n    data: \"str\",\n  });\n}\n\nbootstrap();\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanchan828%2Fnest-cloud-run-queue","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fanchan828%2Fnest-cloud-run-queue","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanchan828%2Fnest-cloud-run-queue/lists"}