https://github.com/thanos/kathikon
Kathikon is a BEAM-native durable job queue and task execution platform for Elixir. Built on Mnesia and OTP, it provides distributed workers, retries, scheduling, cron, uniqueness, rate limiting, batches, and operational visibility without requiring PostgreSQL, Redis, or external brokers.
https://github.com/thanos/kathikon
background-jobs beam cron distributed-computing distributed-systems distributed-workers elixir fault-tolerance high-availability job-queue mnesia observability orchestration otp rate-limiting scheduling task-queue telemetry workflow
Last synced: 3 days ago
JSON representation
Kathikon is a BEAM-native durable job queue and task execution platform for Elixir. Built on Mnesia and OTP, it provides distributed workers, retries, scheduling, cron, uniqueness, rate limiting, batches, and operational visibility without requiring PostgreSQL, Redis, or external brokers.
- Host: GitHub
- URL: https://github.com/thanos/kathikon
- Owner: thanos
- License: mit
- Created: 2026-06-16T15:28:35.000Z (9 days ago)
- Default Branch: main
- Last Pushed: 2026-06-18T17:03:21.000Z (7 days ago)
- Last Synced: 2026-06-22T03:21:51.379Z (4 days ago)
- Topics: background-jobs, beam, cron, distributed-computing, distributed-systems, distributed-workers, elixir, fault-tolerance, high-availability, job-queue, mnesia, observability, orchestration, otp, rate-limiting, scheduling, task-queue, telemetry, workflow
- Language: Elixir
- Homepage:
- Size: 96.7 KB
- Stars: 1
- Watchers: 0
- Forks: 1
- Open Issues: 15
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Kathikon
**Kathikon** (Greek: καθήκον — duty, obligation) is a BEAM-native durable job queue and task execution platform for Elixir.
Jobs are treated as durable obligations that must eventually be fulfilled: completed, retried, cancelled, or discarded — but never silently lost.
Kathikon uses **Mnesia** as its coordination store and **OTP** as its execution substrate. No PostgreSQL, Redis, RabbitMQ, or external brokers are required.
## Status
**v0.1.0 — Phase 1: Durable Job Queue**
- Job insertion and queue execution
- Retries with exponential backoff
- Scheduling (`schedule_in`, `schedule_at`)
- Priorities
- Telemetry
- Pruning / retention
## Installation
```elixir
def deps do
[
{:kathikon, "~> 0.1.0"}
]
end
```
## Quick start
Define a worker:
```elixir
defmodule MyApp.EmailWorker do
use Kathikon.Worker
@impl true
def perform(%Kathikon.Job{args: %{"email" => email}}) do
MyApp.Mailer.deliver(email)
:ok
end
end
```
Configure queues:
```elixir
# config/config.exs
config :kathikon,
queues: [
default: [concurrency: 10],
emails: [concurrency: 5]
]
```
Enqueue jobs:
```elixir
{:ok, job} = Kathikon.insert(MyApp.EmailWorker, %{"email" => "user@example.com"})
{:ok, job} =
Kathikon.insert(MyApp.EmailWorker, %{"email" => "user@example.com"},
queue: :emails,
priority: 5,
schedule_in: 60,
max_attempts: 10
)
```
Cancel a pending job:
```elixir
{:ok, job} = Kathikon.cancel(job_id)
```
## Architecture
```
Kathikon.Supervisor
├── Registry
├── Kathikon.Queue (DynamicSupervisor)
│ └── Kathikon.Dispatcher (one per queue)
├── Kathikon.Scheduler
└── Kathikon.Pruner
```
| Module | Role |
|--------|------|
| `Kathikon.Job` | Job struct and state machine |
| `Kathikon.Worker` | Worker behaviour (`perform/1`) |
| `Kathikon.Storage` | Storage facade over `Kathikon.Backend.Storage` |
| `Kathikon.Dispatcher` | Claims and executes jobs per queue |
| `Kathikon.Scheduler` | Promotes scheduled jobs to available |
| `Kathikon.Pruner` | Enforces retention on terminal jobs |
| `Kathikon.Backend.Storage` | Storage behaviour (default: `…Mnesia`) |
| `Kathikon.Telemetry` | `[:kathikon, ...]` telemetry events |
## Job states
```
scheduled → available → executing → completed
↘ ↘ retryable → ...
cancelled discarded
```
## Telemetry
Kathikon emits standard telemetry events. See [Telemetry guide](docs/guides/telemetry-and-observability.md) for the full list. Highlights:
- Job: `[:kathikon, :job, :insert]`, `:start`, `:stop`, `:sleep`, `:retry`, `:discard`, `:cancel`, `:prune`
- Runtime: `[:kathikon, :scheduler, :tick]`, `[:kathikon, :pruner, :tick]`, `[:kathikon, :dispatcher, :poll]`
Attach the default logger in development:
```elixir
Kathikon.Telemetry.attach_default_logger()
```
## Configuration
| Key | Default | Description |
|-----|---------|-------------|
| `:queues` | `[default: [concurrency: 10]]` | Queue names and concurrency |
| `:poll_interval` | `200` | Dispatcher poll interval (ms) |
| `:scheduler_interval` | `1000` | Scheduler tick interval (ms) |
| `:prune_interval` | `60000` | Pruner tick interval (ms) |
| `:retention_period` | `7 days` | How long to keep terminal jobs (ms) |
| `:max_attempts` | `20` | Default retry limit |
| `:mnesia_copies` | `:auto` | Mnesia storage: `:ram`, `:disc`, or `:auto` (`ram` on `nonode@nohost` and Livebook nodes) |
## Roadmap
| Phase | Focus |
|-------|-------|
| 1 | Durable job queue (current) |
| 2 | Distributed coordination, leases, lifeline |
| 3 | Cron, uniqueness, dynamic queues |
| 4 | Rate limits, pause/resume |
| 5 | Batches |
| 6 | Observability APIs |
| 7 | Workflows and DAGs |
| 8 | Optional LiveView dashboard |
## Documentation
Generate HTML docs with [ExDoc](https://github.com/elixir-lang/ex_doc):
```bash
mix docs
open doc/index.html
```
- **[Documentation index](docs/documentation.md)** — guides and module reference (source)
- [Quick start](docs/guides/quick-start.md)
- [Module reference](docs/reference/modules.md)
- [Configuration](docs/guides/configuration.md)
- [Interactive demo (Livebook)](livebooks/kathikon_demo.livemd)
## License
MIT. See [LICENSE](LICENSE).