{"id":26487212,"url":"https://github.com/gurbaj5124871/workflow-schedular-runtime","last_synced_at":"2026-04-11T18:08:36.522Z","repository":{"id":281795788,"uuid":"607190448","full_name":"gurbaj5124871/workflow-schedular-runtime","owner":"gurbaj5124871","description":"Service to schedule tasks/jobs and process them in background at scale","archived":false,"fork":false,"pushed_at":"2025-03-11T07:20:35.000Z","size":2897,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-11T07:36:19.019Z","etag":null,"topics":["background-jobs","monorepo","nestjs","nodejs","postgresql","scheduler","temporal","typescript","workers"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gurbaj5124871.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-27T13:58:01.000Z","updated_at":"2025-03-11T07:20:35.000Z","dependencies_parsed_at":"2025-03-11T07:36:32.234Z","dependency_job_id":"1a62fccf-2173-44bd-bc0f-200ec2704d52","html_url":"https://github.com/gurbaj5124871/workflow-schedular-runtime","commit_stats":null,"previous_names":["gurbaj5124871/workflow-schedular-runtime"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gurbaj5124871%2Fworkflow-schedular-runtime","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gurbaj5124871%2Fworkflow-schedular-runtime/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gurbaj5124871%2Fworkflow-schedular-runtime/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gurbaj5124871%2Fworkflow-schedular-runtime/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gurbaj5124871","download_url":"https://codeload.github.com/gurbaj5124871/workflow-schedular-runtime/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244566975,"owners_count":20473451,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["background-jobs","monorepo","nestjs","nodejs","postgresql","scheduler","temporal","typescript","workers"],"created_at":"2025-03-20T06:35:13.752Z","updated_at":"2026-04-11T18:08:36.493Z","avatar_url":"https://github.com/gurbaj5124871.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Workflow Schedular Runtime\n\nA highly scalable service to schedule workflow jobs/tasks with timers with persistance. The system is build to be distributed to scale with resilience.\n\n## Technologies Used\n\n1. [Temporal](https://temporal.io/) - It is a scalable and reliable runtime for Reentrant Processes. It's battle-tested at scale and used in production by companies like uber, netflix, snap (snapchat) etc. More about temporal:\n\n   - [Docs](https://docs.temporal.io/temporal)\n   - [TL;DR Intro Video](https://www.youtube.com/watch?v=2HjnQlnA5eY)\n\n2. [PostgreSQL](https://www.postgresql.org/) - Used for persistance of workflows alongside temporal.\n3. [Nest](https://github.com/nestjs/nest) - NodeJS TypeScript Framework.\n\n## Description\n\nThe service is a monorepo divided into two systems:\n\n1. server: The service is responsible for serving requests to schedule webhook triggers into temporal\n2. worker: The service responsible for polling workflows/jobs/tasks from temporal and execute the webhook triggers\n\nSystem Design Architecture Diagram:\n\n![diagram](/system-design-architecture.png)\n\n## Prerequisites\n\n1. Make sure [Docker](https://www.docker.com/products/docker-desktop/) is installed on the system\n2. Copy the `.env` file from `.env.example`\n\n```bash\n$ cp .env.example .env\n```\n\n## Running the app\n\nIn the main directory, run the following command:\n\n```bash\n$ docker compose up -d\n```\n\n## Helper tools\n\n1. Swagger Docs: For testing out REST endpoints, Hosted at `/api-docs` path, for [localhost](http://localhost:3000/api-docs)\n2. Temporal UI: For deep visibility into scheduled tasks/jobs, for [localhost](http://localhost:8080)\n\n## How it works\n\n### 1. POST /timers\n\n```mermaid\n  sequenceDiagram\n     client -\u003e\u003e server: HTTP request with webhook url and time to delay\n     server -\u003e\u003e temporal client SDK: Request to create workflow to trigger workflow with deadline\n     temporal client SDK -\u003e\u003e temporal: gRPC call to register and persist the workflow task\n     temporal -\u003e\u003e postgreSQL: persist the new workflow task\n     postgreSQL -\u003e\u003e temporal: success response\n     temporal worker SDK -\u003e\u003e temporal: poll temporal for new workflows to run\n     temporal worker SDK -\u003e\u003e worker: run the workflow and start the timer\n     worker -\u003e\u003e webhook trigger: trigger the post request on the provided url after deadline\n```\n\n### 2. GET /timers/:id\n\n```mermaid\n   sequenceDiagram\n   client -\u003e\u003e server / temporal client: HTTP request to fetch the time left for task by id\n   server / temporal client -\u003e\u003e temporal: \"getDeadline\" Query to fetch the timer expiration\n   temporal -\u003e\u003e worker / temporal worker: query forwarded from client to workflow\n   worker / temporal worker -\u003e\u003e temporal: returns the deadline/timeout\n   temporal -\u003e\u003e server / temporal client: forwards the deadline/timeout from worker\n   server / temporal client -\u003e\u003e client: Returns the difference between current time and deadline, if positive then returns time diff in seconds otherwise returns 0\n\n\n```\n\n## Test\n\n```bash\n# install packages\n$ npm i\n\n# e2e tests\n$ npm run test:e2e\n```\n\n## Tasks\n\n- [x] POST /timers api to schedule the webhook trigger with timer and return id of the scheduled task. Shoots post http call with empty body after timeout\n- [x] GET /timers/:id api to fetch time left in seconds until timer expires. Return 0 if already expired\n- [x] Persist timer and timers shouldn't reset if system is down(They are persisted inside postgreSQL via temporal)\n- [x] If all the workers are down, and when they are back, they pick up the tasks to be processed and if timers are expired already they shoot post request\n- [x] Docker compose file to run the whole system by single command\n- [x] API test cases (not all cases covered yet)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgurbaj5124871%2Fworkflow-schedular-runtime","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgurbaj5124871%2Fworkflow-schedular-runtime","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgurbaj5124871%2Fworkflow-schedular-runtime/lists"}