{"id":51051306,"url":"https://github.com/softwaremill/okapi","last_synced_at":"2026-06-22T17:02:35.643Z","repository":{"id":346749957,"uuid":"1189475095","full_name":"softwaremill/okapi","owner":"softwaremill","description":"Reliable message delivery using the transactional outbox pattern for Kotlin","archived":false,"fork":false,"pushed_at":"2026-06-08T08:29:20.000Z","size":577,"stargazers_count":5,"open_issues_count":2,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-08T09:17:08.592Z","etag":null,"topics":["kafka","kotlin","mysql","outbox-pattern","postgresql","spring-boot","transactional-outbox"],"latest_commit_sha":null,"homepage":"https://softwaremill.com/open-source/","language":"Kotlin","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/softwaremill.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-23T11:06:16.000Z","updated_at":"2026-06-08T08:29:39.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/softwaremill/okapi","commit_stats":null,"previous_names":["softwaremill/okapi"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/softwaremill/okapi","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/softwaremill%2Fokapi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/softwaremill%2Fokapi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/softwaremill%2Fokapi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/softwaremill%2Fokapi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/softwaremill","download_url":"https://codeload.github.com/softwaremill/okapi/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/softwaremill%2Fokapi/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34657902,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-22T02:00:06.391Z","response_time":106,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kafka","kotlin","mysql","outbox-pattern","postgresql","spring-boot","transactional-outbox"],"created_at":"2026-06-22T17:02:34.818Z","updated_at":"2026-06-22T17:02:35.638Z","avatar_url":"https://github.com/softwaremill.png","language":"Kotlin","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Okapi\n\n[![Ideas, suggestions, problems, questions](https://img.shields.io/badge/Discourse-ask%20question-blue)](https://softwaremill.community/c/open-source/11)\n[![CI](https://github.com/softwaremill/okapi/workflows/CI/badge.svg)](https://github.com/softwaremill/okapi/actions?query=workflow%3A%22CI%22)\n[![Kotlin](https://img.shields.io/badge/dynamic/toml?url=https%3A%2F%2Fraw.githubusercontent.com%2Fsoftwaremill%2Fokapi%2Frefs%2Fheads%2Fmain%2Fgradle%2Flibs.versions.toml\u0026query=%24.versions.kotlin\u0026logo=kotlin\u0026label=kotlin\u0026color=blue)](https://kotlinlang.org)\n[![JVM](https://img.shields.io/badge/JVM-21-orange.svg?logo=openjdk)](https://www.java.com)\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE)\n\nKotlin library implementing the **transactional outbox pattern** — reliable message delivery alongside local database operations.\n\nMessages are stored in a database table within the same transaction as your business operation, then asynchronously delivered to external transports (HTTP webhooks, Kafka). This guarantees **at-least-once delivery** without distributed transactions.\n\n## Quick Start (Spring Boot)\n\nAdd dependencies using the BOM for version alignment:\n\n```kotlin\ndependencies {\n    implementation(platform(\"com.softwaremill.okapi:okapi-bom:$okapiVersion\"))\n    implementation(\"com.softwaremill.okapi:okapi-core\")\n    implementation(\"com.softwaremill.okapi:okapi-postgres\")\n    implementation(\"com.softwaremill.okapi:okapi-http\")\n    implementation(\"com.softwaremill.okapi:okapi-spring-boot\")\n}\n```\n\nProvide a `MessageDeliverer` bean — this tells okapi how to deliver messages.\n`ServiceUrlResolver` maps the logical service name (set per message) to a base URL:\n\n```kotlin\n@Bean\nfun httpDeliverer(): HttpMessageDeliverer =\n    HttpMessageDeliverer(ServiceUrlResolver { serviceName -\u003e\n        when (serviceName) {\n            \"notification-service\" -\u003e \"https://notifications.example.com\"\n            else -\u003e error(\"Unknown service: $serviceName\")\n        }\n    })\n```\n\nPublish inside any `@Transactional` method — inject `SpringOutboxPublisher` via constructor:\n\n```kotlin\n@Service\nclass OrderService(\n    private val orderRepository: OrderRepository,\n    private val springOutboxPublisher: SpringOutboxPublisher\n) {\n    @Transactional\n    fun placeOrder(order: Order) {\n        orderRepository.save(order)\n        springOutboxPublisher.publish(\n            OutboxMessage(\"order.created\", order.toJson()),\n            httpDeliveryInfo {\n                serviceName = \"notification-service\"\n                endpointPath = \"/webhooks/orders\"\n            }\n        )\n    }\n}\n```\n\nAutoconfiguration handles scheduling, retries, and delivery automatically. For Micrometer metrics, also add `okapi-micrometer` — see [Observability](#observability).\n\n**Using Kafka instead of HTTP?** Swap the deliverer bean and delivery info:\n\n```kotlin\n@Bean\nfun kafkaDeliverer(producer: KafkaProducer\u003cString, String\u003e): KafkaMessageDeliverer =\n    KafkaMessageDeliverer(producer)\n```\n```kotlin\nspringOutboxPublisher.publish(\n    OutboxMessage(\"order.created\", order.toJson()),\n    kafkaDeliveryInfo { topic = \"order-events\" }\n)\n```\n\n**Using MySQL instead of PostgreSQL?** Replace `okapi-postgres` with `okapi-mysql` in your dependencies — no code changes needed.\n\n\u003e **Note:** Spring and Kafka versions are not forced by okapi — you control them.\n\u003e Okapi uses plain JDBC internally — it works with any `PlatformTransactionManager` (JPA, JDBC, jOOQ, Exposed, etc.).\n\n`okapi-spring-boot` requires a `TransactionRunner` bean to bracket each scheduler tick in a transaction. The autoconfiguration derives one from any `PlatformTransactionManager` on the classpath (`spring-boot-starter-jdbc` or `spring-boot-starter-data-jpa` provide one out of the box) — no extra wiring needed in typical setups. If your application has no `PlatformTransactionManager` (single-instance, no transaction infrastructure) you must opt in explicitly:\n\n```kotlin\n@Bean\nfun outboxTransactionRunner(): TransactionRunner = object : TransactionRunner {\n    override fun \u003cT\u003e runInTransaction(block: () -\u003e T): T = block()\n}\n```\n\nWithout a `TransactionRunner` each scheduler tick runs in auto-commit, which can cause duplicate delivery across instances — see Advanced below.\n\nAdvanced setups — multiple DataSources, JTA/Exposed PTMs, qualifier precedence — see [Advanced: transactions \u0026 multi-DataSource](#advanced-transactions--multi-datasource) below.\n\n## Advanced: transactions \u0026 multi-DataSource\n\nWithout bracketing, `FOR UPDATE SKIP LOCKED` collapses to the single SELECT statement under JDBC auto-commit, which silently allows duplicate delivery across processor instances. This opt-in is intentionally manual to keep accidental misconfiguration out of multi-instance deployments.\n\n**Multi-DataSource contexts.** If your application has multiple `DataSource` beans and uses a `PlatformTransactionManager` from which okapi cannot extract a `DataSource` (JTA, Exposed's `SpringTransactionManager`, JPA without a JDBC `DataSource`), the autoconfiguration refuses to start until you set `okapi.transaction-manager-qualifier` to the bean name of the PTM that brackets the outbox `DataSource`. `okapi.datasource-qualifier` alone is not sufficient: it picks the outbox `DataSource` but does not constrain which PTM brackets it. Alternative escape hatch: supply your own `@Bean TransactionRunner`. Single-DataSource setups and PTMs whose `DataSource` can be introspected (`DataSourceTransactionManager`, `JpaTransactionManager`, `HibernateTransactionManager`) are unaffected.\n\nWhen `okapi.transaction-manager-qualifier` is set, it takes precedence over any auto-wired `TransactionTemplate` — including the one Spring Boot's `TransactionAutoConfiguration` registers around the `@Primary` `PlatformTransactionManager`. If the qualifier names a different PTM than that auto-TT wraps, okapi builds a fresh `TransactionTemplate` around the qualified PTM (so the qualifier's intent is honoured) and any custom timeout/isolation/propagation on the auto-wired TT is not inherited — a WARN is logged in that case.\n\n**Constructing schedulers directly (non-autoconfig usage).** When wiring `OutboxProcessorScheduler` / `OutboxPurgerScheduler` manually (Ktor, custom Spring contexts without autoconfig, etc.), supply a `TransactionRunner` explicitly — the parameter is required, with no default:\n\n```kotlin\nOutboxProcessorScheduler(\n    outboxProcessor = processor,\n    transactionRunner = SpringTransactionRunner(template), // or your framework's equivalent\n    config = OutboxSchedulerConfig(...),\n)\n```\n\n## How It Works\n\nOkapi implements the [transactional outbox pattern](https://softwaremill.com/microservices-101/) (see also: [microservices.io description](https://microservices.io/patterns/data/transactional-outbox.html)):\n\n1. Your application writes an `OutboxMessage` to the outbox table **in the same database transaction** as your business operation\n2. A background `OutboxScheduler` polls for pending messages and delivers them to the configured transport (HTTP, Kafka)\n3. Failed deliveries are retried according to a configurable `RetryPolicy` (max attempts, backoff)\n\n**Delivery guarantees:**\n\n- **At-least-once delivery** — okapi guarantees every message will be delivered, but duplicates are possible (e.g., after a crash between delivery and status update). Consumers should handle idempotency, for example by checking the `OutboxId` returned by `publish()`.\n- **Concurrent processing** — multiple processors can run in parallel using `FOR UPDATE SKIP LOCKED`, so messages are never processed twice simultaneously.\n- **Delivery result classification** — each transport classifies errors as `Success`, `RetriableFailure`, or `PermanentFailure`. For example, HTTP 429 is retriable while HTTP 400 is permanent.\n\n## Database migrations\n\nOkapi ships Liquibase changelogs that create the outbox table and its indexes:\n\n- `classpath:com/softwaremill/okapi/db/postgres/changelog.xml` — PostgreSQL (from `okapi-postgres`)\n- `classpath:com/softwaremill/okapi/db/mysql/changelog.xml` — MySQL (from `okapi-mysql`)\n\nWhen `okapi-spring-boot` is on the classpath, these run automatically against the configured `DataSource` on application startup. Without Spring Boot, point your own Liquibase setup at the paths above and pass an `outboxTable` change-log parameter (see below).\n\n### Configuration\n\nOkapi's table names are fixed under the `okapi_` prefix so its schema stays out of the way of any pre-existing tables in the host application (`outbox`, `databasechangelog`, etc.):\n\n| Table | Purpose |\n|-------|---------|\n| `okapi_outbox` | Domain table holding outbox entries (created by the bundled Liquibase changesets, queried by `PostgresOutboxStore` / `MysqlOutboxStore`). |\n| `okapi_databasechangelog` | Liquibase changeset history for okapi (configurable). |\n| `okapi_databasechangeloglock` | Liquibase concurrency lock for okapi (configurable). |\n\nThe Liquibase tracking-table names are configurable in case the host application wants to share them with its own Liquibase setup:\n\n| Property | Default | Description |\n|----------|---------|-------------|\n| `okapi.liquibase.changelog-table` | `okapi_databasechangelog` | Liquibase changeset history for okapi |\n| `okapi.liquibase.changelog-lock-table` | `okapi_databasechangeloglock` | Liquibase concurrency lock for okapi |\n\nThese properties affect the autoconfigured `okapiPostgresLiquibase` / `okapiMysqlLiquibase` beans only. If you run Liquibase yourself, configure the table names there directly. The domain table name (`okapi_outbox`) is fixed.\n\n### Upgrading from 0.2.x\n\nReleases up to 0.2.x wrote to shared tables `databasechangelog` / `databasechangeloglock` and the domain table `outbox`. From 0.3.0 these are renamed to `okapi_*`. Two upgrade paths:\n\n**Stay on the existing changelog tables** (simplest for the Liquibase tracking pair, zero-downtime) — opt out of the new defaults:\n\n```yaml\nokapi:\n  liquibase:\n    changelog-table: databasechangelog\n    changelog-lock-table: databasechangeloglock\n```\n\nThe domain table `outbox` cannot be opted out via configuration — see the migration steps below.\n\n**Migrate to dedicated tables** — run before the first 0.3.0 startup (PostgreSQL syntax shown):\n\n```sql\n-- Outbox domain table: rename in place. Indexes follow the table.\nALTER TABLE outbox RENAME TO okapi_outbox;\nALTER INDEX idx_outbox_status_last_attempt RENAME TO idx_okapi_outbox_status_last_attempt;\nALTER INDEX idx_outbox_status_created_at  RENAME TO idx_okapi_outbox_status_created_at;\n\n-- Liquibase tracking: split okapi rows into the new tables.\nCREATE TABLE okapi_databasechangelog (LIKE databasechangelog INCLUDING ALL);\nCREATE TABLE okapi_databasechangeloglock (LIKE databasechangeloglock INCLUDING ALL);\nINSERT INTO okapi_databasechangelog\n    SELECT * FROM databasechangelog WHERE filename LIKE '%com/softwaremill/okapi/%';\nINSERT INTO okapi_databasechangeloglock SELECT * FROM databasechangeloglock;\nDELETE FROM databasechangelog WHERE filename LIKE '%com/softwaremill/okapi/%';\n```\n\nWithout one of these steps, Liquibase will see an empty changelog table on the first 0.3.0 startup and try to re-run okapi's migrations — which fails if rows already exist under the legacy `outbox` table while okapi now writes to `okapi_outbox`.\n\nFull release history: [CHANGELOG.md](CHANGELOG.md).\n\n## Observability\n\nAdd `okapi-micrometer` alongside `okapi-spring-boot` (from the Quick Start above) to get Micrometer metrics:\n\n```kotlin\nimplementation(\"com.softwaremill.okapi:okapi-micrometer\")\n```\n\nWith Spring Boot Actuator and a Prometheus registry (`micrometer-registry-prometheus`) on the classpath, metrics are automatically exposed on `/actuator/prometheus`. They are also visible via `/actuator/metrics`.\n\n| Metric | Type | Description |\n|--------|------|-------------|\n| `okapi.entries.delivered` | Counter | Successfully delivered entries |\n| `okapi.entries.retry.scheduled` | Counter | Failed attempts rescheduled for retry |\n| `okapi.entries.failed` | Counter | Permanently failed entries |\n| `okapi.batch.duration` | Timer | Processing time per batch |\n| `okapi.entries.count` | Gauge | Current entry count (tag: `status=pending\\|delivered\\|failed`) |\n| `okapi.entries.lag.seconds` | Gauge | Age of oldest entry in seconds (tag: `status`) |\n\n### Configuration\n\n| Property | Default | Description |\n|----------|---------|-------------|\n| `okapi.metrics.refresh-interval` | `PT15S` (15s) | How often gauge metrics poll the outbox store. Each refresh runs one transaction with two queries. |\n\n### Multi-instance deployments\n\nCounters and timers (`okapi.entries.delivered`, `okapi.entries.retry.scheduled`, `okapi.entries.failed`, `okapi.batch.duration`) report work performed by **each instance** — aggregate with `sum`:\n\n```promql\nsum(rate(okapi_entries_delivered_total[5m]))\n```\n\nGauges (`okapi.entries.count`, `okapi.entries.lag.seconds`) reflect the **shared outbox state** and are reported identically by every instance. Aggregate with `max by (status)`, not `sum`:\n\n```promql\nmax by (status) (okapi_entries_count)\n```\n\nPolling cost per instance is `2 queries / okapi.metrics.refresh-interval` (default `2 queries / 15s`).\n\n### Without Spring Boot\n\n`okapi-micrometer` has no Spring dependency. Construct the beans manually and pass a `MeterRegistry`. `MicrometerOutboxMetrics` requires a `TransactionRunner` for Exposed-backed stores — see the class KDoc.\n\nFor periodic gauge refresh, use the framework-agnostic `OutboxMetricsRefresher` (single daemon thread):\n\n```kotlin\nval listener = MicrometerOutboxListener(meterRegistry)\nval metrics = MicrometerOutboxMetrics(store, meterRegistry, transactionRunner)\n\nval refresher = OutboxMetricsRefresher(metrics, Duration.ofSeconds(15))\nrefresher.start()\n// on application shutdown:\nrefresher.close()\n```\n\nOr call `metrics.refresh()` from your own scheduler (Ktor coroutine, `ScheduledExecutorService`, etc.) — `refresh()` is thread-safe.\n\n### Custom listener\n\nImplement `OutboxProcessorListener` to react to delivery events (logging, alerting, custom metrics). `OutboxProcessor` accepts a single listener; to combine multiple, implement a composite that delegates to each.\n\n## Modules\n\n```mermaid\ngraph BT\n    PG[okapi-postgres] --\u003e CORE[okapi-core]\n    MY[okapi-mysql] --\u003e CORE\n    HTTP[okapi-http] --\u003e CORE\n    KAFKA[okapi-kafka] --\u003e CORE\n    MICRO[okapi-micrometer] --\u003e CORE\n    EXP[okapi-exposed] --\u003e CORE\n    SPRING[okapi-spring-boot] --\u003e CORE\n    SPRING -.-\u003e|compileOnly| PG\n    SPRING -.-\u003e|compileOnly| MY\n    SPRING -.-\u003e|compileOnly| MICRO\n    BOM[okapi-bom]\n\n    style CORE fill:#4a9eff,color:#fff\n    style BOM fill:#888,color:#fff\n```\n\n| Module | Purpose |\n|--------|---------|\n| `okapi-core` | Transport/storage-agnostic orchestration, scheduling, retry policy, `ConnectionProvider` interface |\n| `okapi-exposed` | Exposed ORM integration — `ExposedConnectionProvider`, `ExposedTransactionRunner`, `ExposedTransactionContextValidator` |\n| `okapi-postgres` | PostgreSQL storage via plain JDBC (`FOR UPDATE SKIP LOCKED`) |\n| `okapi-mysql` | MySQL 8+ storage via plain JDBC |\n| `okapi-http` | HTTP webhook delivery (JDK HttpClient) |\n| `okapi-kafka` | Kafka topic publishing |\n| `okapi-micrometer` | Micrometer metrics (counters, timers, gauges) |\n| `okapi-spring-boot` | Spring Boot autoconfiguration (auto-detects store, transports, and metrics) |\n| `okapi-bom` | Bill of Materials for version alignment |\n\n## Compatibility\n\n| Dependency | Supported Versions | Notes |\n|---|---|---|\n| Java | 21+ | Required |\n| Spring Boot | 3.5.x, 4.0.x | `okapi-spring-boot` module |\n| Kafka Clients | 3.9.x, 4.x | `okapi-kafka` — you provide `kafka-clients` |\n| Exposed | 1.x | `okapi-exposed` module — for Ktor/standalone apps |\n\n## Performance\n\nThroughput baseline (single instance, sync sequential delivery, MacBook M3 Max, JDK 25 LTS, April 2026):\n\n| Transport | batchSize=10 | batchSize=100 |\n|-----------|--------------|----------------|\n| Kafka (`acks=all`, localhost broker) | ~110 msg/s | ~115 msg/s |\n| HTTP @ webhook latency 20 ms | ~33 msg/s | ~36 msg/s |\n| HTTP @ webhook latency 100 ms | ~9 msg/s | ~9 msg/s |\n\nThese numbers reflect the current sync-sequential delivery model. Throughput is bounded by per-message round-trip time × batch size.\n\nFull methodology, raw JMH results, and reproduction instructions: [`benchmarks/`](benchmarks/).\n\n## Build\n\n```sh\n./gradlew build                  # Build all modules\n./gradlew test                   # Run tests (Docker required — Testcontainers)\n./gradlew ktlintFormat           # Format code\n./gradlew :okapi-benchmarks:jmh  # Run JMH benchmarks (~30 min, see benchmarks/README.md)\n```\n\nRequires JDK 21.\n\n## Contributing\n\nAll suggestions welcome :)\n\nTo compile and test, run:\n\n```sh\n./gradlew build\n./gradlew ktlintFormat   # Mandatory before committing\n```\n\nSee the list of [issues](https://github.com/softwaremill/okapi/issues) and pick one! Or report your own.\n\nIf you are having doubts on the _why_ or _how_ something works, don't hesitate to ask a question on [Discourse](https://softwaremill.community/c/open-source/11) or via GitHub. This probably means that the documentation or code is unclear and can be improved for the benefit of all.\n\nTests use [Testcontainers](https://www.testcontainers.org/) — Docker must be running.\n\nWhen you have a PR ready, take a look at our [\"How to prepare a good PR\" guide](https://softwaremill.community/t/how-to-prepare-a-good-pr-to-a-library/448). Thanks! :)\n\n## Project sponsor\n\nWe offer commercial development services. [Contact us](https://softwaremill.com) to learn more about us!\n\n## Copyright\n\nCopyright (C) 2026 SoftwareMill [https://softwaremill.com](https://softwaremill.com).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsoftwaremill%2Fokapi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsoftwaremill%2Fokapi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsoftwaremill%2Fokapi/lists"}