{"id":46567392,"url":"https://github.com/nativebpm/camunda","last_synced_at":"2026-05-31T11:00:29.237Z","repository":{"id":318521888,"uuid":"1071644938","full_name":"nativebpm/camunda","owner":"nativebpm","description":null,"archived":false,"fork":false,"pushed_at":"2026-05-31T07:42:12.000Z","size":64,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-31T08:08:05.910Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nativebpm.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-07T16:10:30.000Z","updated_at":"2026-05-31T07:42:14.000Z","dependencies_parsed_at":null,"dependency_job_id":"8d94e8fa-98c9-4e89-951d-6d0060cad368","html_url":"https://github.com/nativebpm/camunda","commit_stats":null,"previous_names":["nativebpm/camunda"],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/nativebpm/camunda","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nativebpm%2Fcamunda","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nativebpm%2Fcamunda/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nativebpm%2Fcamunda/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nativebpm%2Fcamunda/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nativebpm","download_url":"https://codeload.github.com/nativebpm/camunda/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nativebpm%2Fcamunda/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33728391,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-05-31T02:00:06.040Z","response_time":95,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-03-07T07:36:49.053Z","updated_at":"2026-05-31T11:00:29.231Z","avatar_url":"https://github.com/nativebpm.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Camunda External Task Client\n\nA high-performance Go client for Camunda 7 external tasks, supporting both traditional REST polling and ultra-high-throughput **WAL-based Change Data Capture (CDC)** processing using Sequin.\n\n---\n\n## 1. Architectural Patterns\n\nThis client supports two distinct execution patterns depending on your system load requirements and deployment complexity:\n\n### A. Standard REST Polling Architecture (Classic)\nIn the standard pattern, the client periodically sends `/fetchAndLock` REST requests to the Camunda Engine.\n- **Pros**: Simple, zero database integration or CDC setups required. Works with any Camunda 7 database.\n- **Cons**: High database \u0026 network polling overhead at rest. High concurrent completions on parallel multi-instances can trigger `OptimisticLockingExceptions` inside Camunda's API, causing 60-second lock timeouts under peak loads.\n\n### B. High-Performance WAL CDC Architecture (Sequin-based) - Database-Free\nInstead of polling the REST API, the CDC architecture captures task insertion events directly from the PostgreSQL **Write-Ahead Log (WAL)** using **Sequin** stream consumer, and processes tasks with optimized REST-based locking. The worker has **zero direct connection** to the Camunda database.\n\n```mermaid\ngraph TD\n    subgraph \"Go Client (Worker App)\"\n        Worker[\"SequinWorker (camunda.SequinWorker)\"]\n        TC[\"TaskCompletion (internal/tasks)\"]\n        TF[\"TaskFailure (internal/tasks)\"]\n        Handlers[\"Registered TaskHandlers\"]\n    end\n    \n    subgraph \"CDC Pipeline Layer\"\n        Sequin[\"Sequin Service (:7376)\"]\n        WAL[\"Logical Replication Slot (sequin_slot)\"]\n    end\n    \n    subgraph \"Camunda Process Engine\"\n        REST[\"Camunda REST API (:8080)\"]\n        DB[\"PostgreSQL / Database (:7477)\"]\n    end\n\n    %% WAL CDC path\n    DB --\u003e|WAL Logs| WAL\n    WAL --\u003e|Logical CDC Stream| Sequin\n    Sequin --\u003e|1. HTTP Pull Events| Worker\n    \n    %% Task execution path\n    Worker --\u003e|2. POST /lock| REST\n    Worker --\u003e|3. GET /variables| REST\n    Worker --\u003e|4. Run Handler| Handlers\n    Handlers --\u003e|5. Complete Task| TC\n    TC --\u003e|6. POST /complete| REST\n    REST --\u003e|7. Commit Transaction| DB\n```\n\n#### Detailed CDC Workflow:\n1. **Event Capture**: When a new external task is created, a row is inserted into Camunda's `act_ru_ext_task` table. PostgreSQL writes this transaction to the Write-Ahead Log (WAL).\n2. **Streaming via Sequin**: Sequin captures this transaction through a logical replication slot (`sequin_slot`) and publication (`sequin_pub`) and exposes it as an HTTP Pull queue.\n3. **HTTP Pull Delivery**: The `SequinWorker` pulls messages from Sequin via `/receive`. Sequin's **Visibility Timeout** guarantees that this message is only delivered to one worker, eliminating the need for database-level concurrency locks.\n4. **REST Lock Activation**: The worker locks the task via Camunda REST API (`POST /external-task/{id}/lock`) to satisfy the engine completion requirements.\n5. **Variable Querying**: Once locked, the worker queries process variables via Camunda REST API (`GET /process-instance/{id}/variables`).\n6. **Execution**: The registered handler is executed.\n7. **Task Completion**: The handler finishes execution and uses the REST API (`/external-task/{id}/complete`) to commit the task completion back to the Camunda Engine.\n8. **Acknowledgement**: The worker sends an HTTP ACK request to Sequin to remove the processed event from the queue. If a transient error occurs (e.g. `OptimisticLockingException`), the worker sends a NACK to Sequin, triggering an instant retry.\n\n---\n\n## 2. Tuning \u0026 Database Migrations\n\n### Database Setup with Atlas Go\nTo enable CDC replication slots and publication schemas safely without editing default docker configs, we use [Atlas Go](https://atlasgo.io/). The configuration is version-controlled in [atlas.hcl](docker/camunda/atlas.hcl).\n\nMigrations are automatically applied during deployment using the `arigaio/atlas:latest-alpine` runner container, which waits for Camunda schema tables to initialize before configuring replication:\n- **`20260531100000_init_sequin.sql`**: Creates the Sequin user, replication slot, and publication.\n- **`20260531100001_enable_replica_identity.sql`**: Configures `REPLICA IDENTITY FULL` on `act_ru_ext_task` to ensure the CDC payloads capture complete row updates.\n\n### High-Performance Configuration\n1. **Target Filtering**: Restrict Sequin sinks to `\"public.act_ru_ext_task\"` to avoid performance bottlenecks caused by variable or history table modifications.\n2. **HTTP Keep-Alives**: Both REST and CDC workers share a tuned `http.Transport` pool (`MaxIdleConns = 100`, `MaxIdleConnsPerHost = 100`, `IdleConnTimeout = 90s`) to avoid macOS/Linux ephemeral port exhaustion (`TIME_WAIT`).\n\n---\n\n## 3. Usage Examples\n\n### A. Running standard REST Polling Worker\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"log/slog\"\n\t\"time\"\n\t\"github.com/nativebpm/camunda\"\n)\n\nfunc main() {\n\tlogger := slog.Default()\n\tclient, err := camunda.NewClient(\"http://localhost:8080\", \"classic-worker\")\n\tif err != nil {\n\t\tlogger.Error(\"Failed to init client\", \"error\", err)\n\t\treturn\n\t}\n\n\tworker := camunda.NewWorker(client, logger)\n\tworker.SetMaxTasks(20)\n\tworker.SetPollInterval(100 * time.Millisecond)\n\n\tworker.RegisterHandler(\"creditScoreChecker\", func(ctx context.Context, client *camunda.Client, task camunda.ExternalTask, complete camunda.CompleteFunc, fail camunda.FailFunc) error {\n\t\t// Business logic here\n\t\treturn complete().Variable(\"score\", 750).Execute()\n\t}, 60000, []string{\"score\"})\n\n\tworker.Start(context.Background())\n}\n```\n\n### B. Running WAL-based CDC Worker (Sequin) - Database-Free\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"log/slog\"\n\t\"github.com/nativebpm/camunda\"\n)\n\nfunc main() {\n\tlogger := slog.Default()\n\t\n\t// Initialize API client for completions, locks and variables fetching\n\tclient, err := camunda.NewClient(\"http://localhost:8080\", \"sequin-worker\")\n\tif err != nil {\n\t\tlogger.Error(\"Failed to init client\", \"error\", err)\n\t\treturn\n\t}\n\n\t// Initialize Sequin worker with Sequin endpoint and consumer\n\tsequinURL := \"http://localhost:7376\"\n\tconsumer := \"camunda_tasks\"\n\n\tsequinWorker, err := camunda.NewSequinWorker(client, sequinURL, consumer, logger)\n\tif err != nil {\n\t\tlogger.Error(\"Failed to create Sequin worker\", \"error\", err)\n\t\treturn\n\t}\n\n\tsequinWorker.RegisterHandler(\"creditScoreChecker\", func(ctx context.Context, client *camunda.Client, task camunda.ExternalTask, complete camunda.CompleteFunc, fail camunda.FailFunc) error {\n\t\t// Business logic here\n\t\treturn complete().Variable(\"score\", 750).Execute()\n\t})\n\n\tsequinWorker.Start(context.Background())\n}\n```\n\n---\n\n## 4. Performance Metrics\n\nUnder benchmark testing deploying the `loan-granting.bpmn` workflow, we evaluated traditional REST polling and WAL CDC (using Sequin Pull consumer) under heavy scaling pressures:\n- **REST Polling**: Aggressive polling at scale experiences lock wait states and transactional rollbacks, leading to 60-second lock timeouts under concurrent completion of parallel tasks.\n- **Sequin WAL CDC (Database-Free)**:\n  - **500 instances** (2,047 tasks): Completed in **20.51s** at **24.37 RPS / 99.79 TPS**.\n  - **1000 instances** (4,014 tasks): Completed in **71.52s** at **13.98 RPS / 56.13 TPS** (Latency metrics: p50=57s, p90=67.5s, p99=70.4s).\n  - **2000 instances** (8,036 tasks): Completed in **22.41s** at **89.26 RPS / 358.66 TPS** (Latency metrics: p50=13.8s, p90=20.4s, p99=21.1s).\n  - **3000 instances** (12,023 tasks): Completed in **46.34s** at **64.74 RPS / 259.45 TPS** (Optimized concurrency control, G1GC tuning, structured decoding, and adaptive connection pooling. Under peak scaling, the database lock pressure remains stable with zero failed tasks).\n\nFor full benchmark reports, refer to the detailed [report](examples/loadtest/camunda-load-test-results.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnativebpm%2Fcamunda","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnativebpm%2Fcamunda","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnativebpm%2Fcamunda/lists"}