{"id":51586025,"url":"https://github.com/waddie/skivi","last_synced_at":"2026-07-11T10:32:17.544Z","repository":{"id":359995361,"uuid":"1063613178","full_name":"waddie/skivi","owner":"waddie","description":"A Clojure-based job queue system for PostgreSQL.","archived":false,"fork":false,"pushed_at":"2026-05-24T13:15:27.000Z","size":140,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-24T15:13:59.289Z","etag":null,"topics":["clojure","clojure-library","cron","job-queue","job-scheduler","postgresql"],"latest_commit_sha":null,"homepage":"","language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/waddie.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"COPYING.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-24T21:50:05.000Z","updated_at":"2026-05-24T13:15:31.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/waddie/skivi","commit_stats":null,"previous_names":["waddie/skivi"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/waddie/skivi","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/waddie%2Fskivi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/waddie%2Fskivi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/waddie%2Fskivi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/waddie%2Fskivi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/waddie","download_url":"https://codeload.github.com/waddie/skivi/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/waddie%2Fskivi/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35360371,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-11T02:00:05.354Z","response_time":104,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clojure","clojure-library","cron","job-queue","job-scheduler","postgresql"],"created_at":"2026-07-11T10:32:17.453Z","updated_at":"2026-07-11T10:32:17.536Z","avatar_url":"https://github.com/waddie.png","language":"Clojure","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Clojars Project](https://img.shields.io/clojars/v/dev.skivi/skivi.svg)](https://clojars.org/dev.skivi/skivi)\n\n# Skivi\n\nSkivi is a PostgreSQL-backed job queue library for Clojure, in the spirit of\n[Graphile Worker](https://github.com/graphile/worker). It runs inside your\nexisting database, creating its own schema there rather than requiring a\nseparate service. This means you can enqueue jobs within the same transaction\nthat triggers them, with no extra infrastructure to operate.\n\n## Installation\n\n### deps.edn\n\n```clojure\n{:deps {dev.skivi/skivi {:mvn/version \"RELEASE\"}}}\n```\n\n## How it works\n\nA system is created from configuration, a task registry, and an optional list of\ncron entries, then started:\n\n```clojure\n(def system\n  (-\u003e (skivi/create-system config tasks)\n      (skivi/start!)))\n\n;; Enqueue from anywhere in your application\n(skivi/add-job system \"send-email\" {:to \"user@example.com\" :subject \"Hello\"})\n```\n\n`start!` runs pending database migrations automatically (this can be disabled\nwith `{:migrate? false}`), then starts the worker pool and any cron scheduler.\n`stop!` drains in-flight jobs gracefully before closing connections.\n\nThe worker pool polls the database for available jobs, claiming them with\nadvisory locks to prevent double-processing. Each job is dispatched to its\nregistered handler function. A normal return marks the job complete; any\nuncaught exception triggers a retry with exponential backoff. After exhausting\nits allowed attempts, a job is marked exhausted and recorded in history.\n\n## Task handlers\n\nTask handlers are plain Clojure functions keyed by string identifier. There are\ntwo ways to register them.\n\n**Programmatic registry** – pass a map to `create-system`:\n\n```clojure\n(def tasks\n  {\"send-email\"   (fn [{:keys [job]}] (send! (:payload job)))\n   \"resize-image\" (fn [{:keys [job]}] (resize! (:payload job)))})\n\n(skivi/create-system config tasks)\n```\n\n**File-based loading** – set `:task-directory` in the worker config and drop\n`.clj` files there. Each file must return a registry map as its last expression:\n\n```clojure\n;; tasks/email.clj\n{\"send-email\" (fn [{:keys [job]}]\n                (send! (:payload job)))}\n```\n\n```edn\n;; skivi-config.edn\n{:worker {:task-directory \"tasks\"\n          :file-extensions [\".clj\"]   ; optional, defaults to [\".clj\"]\n          ...}}\n```\n\nSkivi loads all matching files at startup, sorted alphabetically, and merges\ntheir registries. If you also pass a programmatic registry, it wins on any\nidentifier that appears in both. A job whose task identifier has no registered\nhandler is immediately exhausted rather than retried – a missing handler is a\nconfiguration error, not a transient failure.\n\n## Database schema\n\nSkivi creates all its tables, functions, and triggers in a dedicated PostgreSQL\nschema within your existing application database. The default schema name is\n`skivi`, though it is configurable if you need multiple instances in the same\ndatabase or prefer a different name:\n\n```edn\n;; skivi-config.edn\n{:database {:connection-string \"postgresql://localhost:5432/my_app\"\n            :schema-name \"jobs\"}}\n```\n\n## Scheduling\n\nCron entries are defined as data and passed to `create-system`:\n\n```clojure\n(def crontabs\n  [{:identifier \"daily-report\"\n    :schedule   \"0 2 * * *\"\n    :spec       {:queue-name \"maintenance\" :max-attempts 3}}\n   {:identifier \"hourly-cleanup\"\n    :schedule   \"0 * * * *\"}])\n\n(skivi/create-system config tasks crontabs)\n```\n\nEach cron entry uses `unsafe_dedupe` by default: if the previous run’s job is\nstill queued or in flight, the new enqueue is silently skipped. Last-execution\nstate is persisted to the database, so the scheduler survives restarts without\ndouble-firing.\n\n## Monitoring\n\nThe event emitter lets you attach handlers to specific event types or to all\nevents:\n\n```clojure\n(monitoring/on (:emitter system) :all\n  (fn [{:keys [type data]}]\n    (log/info \"event\" (assoc data :event/type type))))\n\n(monitoring/on (:emitter system) :job/exhausted\n  (fn [{:keys [data]}]\n    (alert! \"Job exhausted after all attempts\" data)))\n```\n\nStandard event types include `:job/completed`, `:job/failed`, `:job/exhausted`,\n`:job/partial-success`, `:pool/start`, `:pool/stop`, and `:cron/fired`. The\nemitter also maintains an in-memory ring buffer of recent events, useful in\ntests via `collecting-emitter`.\n\n## Payload validation\n\nTask payloads are validated before they reach the database. You can declare\n`malli` schemas or `clojure.spec` specs per task identifier in configuration,\nand validation errors surface at enqueue time rather than during execution.\n\n## License\n\nSkivi is dual-licensed. Open-source use is covered by the [GNU Affero General\nPublic License v3.0](https://www.gnu.org/licenses/agpl-3.0.en.html) or later.\nCommercial licenses are available if the AGPL is not suitable for your use case.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwaddie%2Fskivi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwaddie%2Fskivi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwaddie%2Fskivi/lists"}