{"id":17740480,"url":"https://github.com/ryhkml/tasks-server","last_synced_at":"2025-07-06T19:04:04.692Z","repository":{"id":256921345,"uuid":"848063086","full_name":"ryhkml/tasks-server","owner":"ryhkml","description":"Advanced asynchronous tasks execution","archived":false,"fork":false,"pushed_at":"2025-03-20T12:04:18.000Z","size":283,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-15T00:33:07.618Z","etag":null,"topics":["asynchronous","bun","http","linux","queue","rxjs","scheduling","tasks"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ryhkml.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-08-27T04:05:53.000Z","updated_at":"2025-03-20T12:04:21.000Z","dependencies_parsed_at":"2024-11-20T06:26:44.833Z","dependency_job_id":"95a4b435-23db-4b9b-822e-39b7f6cefae7","html_url":"https://github.com/ryhkml/tasks-server","commit_stats":{"total_commits":119,"total_committers":1,"mean_commits":119.0,"dds":0.0,"last_synced_commit":"abcb944b395924a8f3630712788f56baec31e612"},"previous_names":["ryhkml/tasks-server"],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/ryhkml/tasks-server","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryhkml%2Ftasks-server","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryhkml%2Ftasks-server/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryhkml%2Ftasks-server/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryhkml%2Ftasks-server/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ryhkml","download_url":"https://codeload.github.com/ryhkml/tasks-server/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryhkml%2Ftasks-server/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263957686,"owners_count":23535603,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["asynchronous","bun","http","linux","queue","rxjs","scheduling","tasks"],"created_at":"2024-10-26T03:06:07.214Z","updated_at":"2025-07-06T19:04:04.645Z","avatar_url":"https://github.com/ryhkml.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n    \u003ch1\u003e\n        \u003cb\u003eTasks Server\u003c/b\u003e\n    \u003c/h1\u003e\n\u003c/div\u003e\n\nTasks Server is a self-hosted task management service designed to execute and distribute tasks efficiently. A task is essentially an object representing a resource intended for one-time use. You can request tasks, which will be executed at a later time.\n\n\u003cbr\u003e\n\n\u003cdiv align=\"center\"\u003e\n\t\u003cimg src=\"./diagram.png\" alt=\"Diagram Tasks Server\"\u003e\n\u003c/div\u003e\n\n## Features\n\n1. HTTP requests with selected http curl options\n2. Configurable retry mechanism\n3. Custom scheduling options\n4. Automatically reschedules tasks if the server shuts down unexpectedly.\n\nThere are two comparison tables that compare Pub/Sub and the Cron Job Scheduler.\n\n### Tasks vs Pub/Sub\n\n| **Feature**               | **Tasks**                                                                                                                            | **Pub/Sub**                                                                                  |\n| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------- |\n| **Purpose**               | Execute scheduled and delayed tasks                                                                                                  | Distribute messages in real-time                                                             |\n| **Communication Model**   | Point-to-point (queue based)                                                                                                         | Publish/subscribe (broadcast)                                                                |\n| **Scheduling**            | Yes (with flexibility)                                                                                                               | No                                                                                           |\n| **Delivery Warranty**     | At-least-once delivery (with retries)                                                                                                | At-least-once delivery (duplicates are possible)                                             |\n| **Delivery Rate Control** | Limited 1000 task in queue                                                                                                           | Unlimited                                                                                    |\n| **Failure Handling**      | Best (with retries)                                                                                                                  | Message acknowledgment (subscribers must acknowledge messages after processing)              |\n| **Use Cases**             | Processing asynchronous tasks such as sending emails or updating databases. Running scheduled tasks such as generating daily reports | Real-time data streaming or Pub/sub systems such as sending notifications or updating caches |\n\n### Tasks vs Cron Job Scheduler\n\n| **Feature**               | **Tasks**                                                                                                                            | **Cron Job Scheduler**                                                                     |\n| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------ |\n| **Purpose**               | Execute scheduled and delayed tasks                                                                                                  | Schedule fully managed cron jobs                                                           |\n| **Communication Model**   | Point-to-point (queue based)                                                                                                         | Point-to-point (triggers actions)                                                          |\n| **Scheduling**            | Yes (with flexibility)                                                                                                               | Yes (cron-schedule fixed interval)                                                         |\n| **Delivery Warranty**     | At-least-once delivery (with retries)                                                                                                | Depends entirely on the target service being triggered                                     |\n| **Delivery Rate Control** | Limited 1000 task in queue                                                                                                           | Unlimited                                                                                  |\n| **Failure Handling**      | Best (with retries)                                                                                                                  | No, or Depends entirely on the target service                                              |\n| **Use Cases**             | Processing asynchronous tasks such as sending emails or updating databases. Running scheduled tasks such as generating daily reports | Running recurring tasks on a schedule (nightly backups, daily reports, cleanup processing) |\n\n## Getting Started\n\nEnsure that you have [bun](https://bun.sh/docs/installation), with [curl](https://curl.se/download.html) being optional. Then execute the `sh` or `bash` commands shown below.\n\nTo install dependencies, run:\n\n```sh\nsh install.sh\n```\n\nTo update dependencies after a `git pull`, run:\n\n```sh\nsh update.sh\n```\n\n## Development\n\nTo start the development server, run:\n\n```sh\nbun run serve:dev\n```\n\n## Test\n\nTo start the test server, run:\n\n```sh\nbun run test\n```\n\nOr, test specifically by file name\n\n```sh\nbun --env-file=.env.test test \u003cFILENAME\u003e\n```\n\n## Single-file executable\n\nTo compile single-file executable, run:\n\n```sh\nbun run bin\n```\n\n## Docker build\n\nTo build docker image, run:\n\n```sh\ndocker compose -p tasks --env-file \u003cENV_FILE\u003e up -d --build\n```\n\nTask server uses Nix store when building docker image. The Nix store is an abstraction that stores immutable file system data (such as software packages) which can have dependencies on other such data. In this case, Tasks server copies the Nix store directory to a final stage that only requires the curl binary and its dependencies.\n\n## Path configuration\n\nYou can use absolute path or current working path, for example:\n\n```sh\n# Absolute path\n/tmp/tasks/tasks.db\n# Current working path\n.database/tasks.db\n```\n\n## Endpoints\n\n### Task\n\nEach task name can have a maximum of 1000 tasks in queue.\n\n- ✅ `GET /v1/tasks/:name` - Get a task name\n- ✅ `DELETE /v1/tasks/:name` - Delete a task name\n- ✅ `POST /v1/tasks/register` - Register a task name to create a queue\n\n### Queue\n\nA queue is a collection of tasks scheduled for later execution. Queues can be paused, resumed, and forced to execute.\n\n- ✅ `GET /v1/queues` - Get a list of all tasks in queue\n- ✅ `GET /v1/queues/:id` - Get a task in queue\n- ❌ `PATCH /v1/queues/:id` - Edit a task in queue\n- ✅ `DELETE /v1/queues/:id` - Delete a task history in queue\n- ❌ `GET /v1/queues/:id/config` - Get a task config in queue\n- ✅ `PATCH /v1/queues/:id/pause` - Pause a task in queue\n- ✅ `PATCH /v1/queues/:id/resume` - Resume a task in queue\n- ✅ `PATCH /v1/queues/:id/revoke` - Revoke a task in queue\n- ✅ `POST /v1/queues/register` - Register task execution\n- ✅ `POST /v1/queues/:id/execute` - Force execute a task in queue\n\nAn example of requesting a task:\n\n```sh\ncurl -X POST http://localhost:9420/v1/queues/register \\\n    -d \"...\" \\\n    -H \"Authorization: Bearer \u003cSECRET_KEY\u003e\" \\\n    -H \"Content-Type: application/json\" \\\n    -H \"X-Task-Id: \u003cTASK_ID\u003e\"\n```\n\nPayload:\n\n```json\n{\n    \"httpRequest\": {\n        \"url\": \"https://target-service\",\n        \"method\": \"POST\"\n    },\n    \"config\": {\n        \"executionDelay\": 86400000,\n        \"retry\": 5,\n        \"retryInterval\": 3600000,\n        \"retryExponential\": false\n    }\n}\n```\n\nThe response:\n\n```json\n{\n    \"id\": \"...\",\n    \"state\": \"RUNNING\",\n    \"createdAt\": \"...\",\n    \"statusCode\": 0,\n    \"estimateEndAt\": 0,\n    \"estimateExecutionAt\": \"...\",\n    \"response\": null,\n    \"metadata\": \"...\"\n}\n```\n\nThe example above shows a task scheduled to execute after a 1-day delay. If the task encounters a 4xx or 5xx error response, it will be retried 5 times with a 1-hour interval between each attempt. If `retryExponential` is set to `true`, the interval between retries will increase\n\n```txt\nretryInterval = 3600000ms\n\nRetry-1: 3600000 * 1 = 3600000ms\nRetry-2: 3600000 * 2 = 7200000ms\nRetry-3: 3600000 * 3 = 10800000ms\n\nAnd so on...\n```\n\nAlternatively, you can schedule a task to execute at a specific time using `executeAt`. By default, the task server uses **Coordinated Universal Time** ([UTC](https://currentmillis.com/tutorials/system-currentTimeMillis.html#utc)). Visit [currentmillis.com](https://currentmillis.com) to get the time in milliseconds since the UNIX epoch (January 1, 1970 00:00:00 UTC)\n\n```json\n{\n    \"httpRequest\": {\n        \"url\": \"https://target-service\",\n        \"method\": \"POST\"\n    },\n    \"config\": {\n        \"executeAt\": 1355245932000\n    }\n}\n```\n\nTo specify a particular time zone, you can use the specific time zone offset you want, like the example below\n\n```json\n{\n    \"httpRequest\": {\n        \"url\": \"https://target-service\",\n        \"method\": \"POST\"\n    },\n    \"config\": {\n        \"executeAt\": \"Dec 12 2012 12:12:12 AM +07:00\"\n    }\n}\n```\n\n`+07:00` indicates a time zone offset of +7 hours from **Coordinated Universal Time** (UTC). This means the time is 7 hours ahead of UTC.\n\n\u003e [!NOTE]\n\u003e\n\u003e Properties ending with `At` are in UNIX time format, such as `executeAt`, `retryAt`, and `timeoutAt`. Using `retryAt` or `timeoutAt` will execute only once\n\nTo ensure consistent timekeeping, configure the task server to use the UTC time zone. This can be achieved by setting the `TZ` environment variable to `UTC`.\n\n## Input File\n\nTasks Server allows configuring custom certificates to modify the curl options [--cacert](https://curl.se/docs/manpage.html#--cacert), [--cert](https://curl.se/docs/manpage.html#-E), and [--key](https://curl.se/docs/manpage.html#--key). To add a certificate file when the Task Server is running in a host environment, use the direct path as a curl option. For example\n\n```json\n{\n    \"httpRequest\": {\n        \"url\": \"https://target-service\",\n        \"method\": \"POST\",\n        \"transport\": \"curl\"\n    },\n    \"config\": {\n        \"executeAt\": \"Dec 12 2012 12:12:12 AM +07:00\",\n        \"ca\": \"/tmp/ca.crt\",\n        \"cert\": {\n            \"value\": \"/tmp/fullchain.pem\"\n        },\n        \"key\": \"/tmp/key.pem\"\n    }\n}\n```\n\nWhen running the Tasks Server in a container environment, you can\n\n- Create a volume. Just like the example above, adjust the path inside the container **or**\n- Use base64-encoded. To make it base64-encoded, you can use command\n\n    ```sh\n    cat /tmp/ca.crt | base64 -w 0\n    ```\n\n    ```json\n    {\n        \"httpRequest\": {\n            \"url\": \"https://target-service\",\n            \"method\": \"POST\",\n            \"transport\": \"curl\"\n        },\n        \"config\": {\n            \"executeAt\": \"Dec 12 2012 12:12:12 AM +07:00\",\n            \"ca\": \"base64...\",\n            \"cert\": {\n                \"value\": \"base64...\"\n            },\n            \"key\": \"base64...\"\n        }\n    }\n    ```\n\nIf there are multiple certificates, you can include them all in a single file:\n\n```txt\n-----BEGIN CERTIFICATE-----\n...\n...\n-----END CERTIFICATE-----\n\n-----BEGIN CERTIFICATE-----\n...\n...\n-----END CERTIFICATE-----\n```\n\n## SQLite Backup\n\nThere are two backup methods:\n\n- **Local**. The local method copies the database file, then moves it to another directory. This method is active by default **or**\n- **Object Storage**. The [object storage](https://en.wikipedia.org/wiki/Object_storage) method uploads the database file to an object storage. To authenticate to Object Storage, a compatible authentication method is required.\n\n```ts\ntype SqliteBackupMethod = \"LOCAL\" | \"OBJECT_STORAGE\";\n```\n\n```txt\n# Backup method\nBACKUP_METHOD_SQLITE=\n# Endpoint. The S3-compatible service endpoint URL\nBACKUP_OBJECT_STORAGE_ENDPOINT=\n# Access key\nBACKUP_OBJECT_STORAGE_ACCESS_KEY=\n# Secret key\nBACKUP_OBJECT_STORAGE_SECRET_KEY=\n# Bucket name\nBACKUP_OBJECT_STORAGE_BUCKET_NAME=\n# Path\nBACKUP_OBJECT_STORAGE_PATH=\n```\n\nThere are several lists of object storage compatibility:\n\n- Amazon S3\n- Google Cloud Storage\n- Cloudflare R2\n- DigitalOcean Spaces\n- MinIO **and** many more vendors\n\nVisit [Bun S3](https://bun.sh/docs/api/s3#support-for-s3-compatible-services) documentation for information on Object Storage compatibility.\n\n## TODO\n\n- [ ] Create documentation\n- [ ] Create an API for editing task\n- [ ] Create an API to get task configuration\n- [ ] Create priority queue\n- [x] Backup SQLite database\n- [x] Create a mechanism to reschedule tasks if the server unexpected shutdown\n- [x] Create a [cluster](https://bun.sh/guides/http/cluster) of HTTP server (Linux only)\n- [ ] Enable test coverage configuration\n- [ ] Ensure test coverage reaches 90% or more\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fryhkml%2Ftasks-server","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fryhkml%2Ftasks-server","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fryhkml%2Ftasks-server/lists"}