{"id":14155625,"url":"https://github.com/geofmureithi/apalis","last_synced_at":"2025-04-11T20:38:48.438Z","repository":{"id":38388586,"uuid":"499804228","full_name":"geofmureithi/apalis","owner":"geofmureithi","description":"Simple, extensible multithreaded background job and message processing library for Rust ","archived":false,"fork":false,"pushed_at":"2024-05-22T19:12:03.000Z","size":1450,"stargazers_count":385,"open_issues_count":19,"forks_count":33,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-05-22T19:44:00.699Z","etag":null,"topics":["background-jobs","job-scheduler","message-queue","mysql","postgres","redis","rust","sqlite"],"latest_commit_sha":null,"homepage":"https://crates.io/crates/apalis","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"geofmureithi-zz/apalis","license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/geofmureithi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["geofmureithi"],"patreon":null,"open_collective":null,"ko_fi":"geofmureithi","tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"lfx_crowdfunding":null,"custom":null}},"created_at":"2022-06-04T11:18:05.000Z","updated_at":"2024-07-13T11:20:10.409Z","dependencies_parsed_at":"2023-02-03T15:16:16.708Z","dependency_job_id":"e68fb114-50d1-4937-a07d-eb11f722c5ee","html_url":"https://github.com/geofmureithi/apalis","commit_stats":{"total_commits":199,"total_committers":7,"mean_commits":"28.428571428571427","dds":0.2613065326633166,"last_synced_commit":"2079bb178c3413b6802ac0127bb91a5d93ec87c8"},"previous_names":[],"tags_count":31,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/geofmureithi%2Fapalis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/geofmureithi%2Fapalis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/geofmureithi%2Fapalis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/geofmureithi%2Fapalis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/geofmureithi","download_url":"https://codeload.github.com/geofmureithi/apalis/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248478161,"owners_count":21110648,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["background-jobs","job-scheduler","message-queue","mysql","postgres","redis","rust","sqlite"],"created_at":"2024-08-17T08:04:31.702Z","updated_at":"2025-04-11T20:38:48.214Z","avatar_url":"https://github.com/geofmureithi.png","language":"Rust","funding_links":["https://github.com/sponsors/geofmureithi","https://ko-fi.com/geofmureithi"],"categories":["sqlite","Rust"],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003eapalis\u003c/h1\u003e\n\u003cdiv align=\"center\"\u003e\n \u003cstrong\u003e\n   Simple, extensible multithreaded background job and messages processing library for Rust\n \u003c/strong\u003e\n\u003c/div\u003e\n\n\u003cbr /\u003e\n\n\u003cdiv align=\"center\"\u003e\n  \u003c!-- Crates version --\u003e\n  \u003ca href=\"https://crates.io/crates/apalis\"\u003e\n    \u003cimg src=\"https://img.shields.io/crates/v/apalis.svg?style=flat-square\"\n    alt=\"Crates.io version\" /\u003e\n  \u003c/a\u003e\n  \u003c!-- Downloads --\u003e\n  \u003ca href=\"https://crates.io/crates/apalis\"\u003e\n    \u003cimg src=\"https://img.shields.io/crates/d/apalis.svg?style=flat-square\"\n      alt=\"Download\" /\u003e\n  \u003c/a\u003e\n  \u003c!-- docs.rs docs --\u003e\n  \u003ca href=\"https://docs.rs/apalis\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/docs-latest-blue.svg?style=flat-square\"\n      alt=\"docs.rs docs\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/geofmureithi/apalis/actions\"\u003e\n    \u003cimg src=\"https://img.shields.io/github/actions/workflow/status/geofmureithi/apalis/ci.yaml?branch=master\u0026style=flat-square\"\n      alt=\"CI\" /\u003e\n  \u003c/a\u003e\n\u003c/div\u003e\n\u003cbr/\u003e\n\n## Features\n\n- Simple and predictable job handling model.\n- Jobs handlers with a macro free API.\n- Take full advantage of the [tower] ecosystem of\n  middleware, services, and utilities.\n- Runtime agnostic - Use tokio, smol etc.\n- Optional Web interface to help you manage your jobs.\n\napalis job processing is powered by [`tower::Service`] which means you have access to the [tower] middleware.\n\napalis has support for:\n\n| Source       | Crate                                                                                                                 | Example                                                                                                                                                                                  |\n| ------------ | --------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Cron Jobs    | \u003ca href=\"https://docs.rs/apalis-cron\"\u003e\u003cimg src=\"https://img.shields.io/crates/v/apalis-cron?style=flat-square\"\u003e\u003c/a\u003e   | \u003ca href=\"https://github.com/geofmureithi/apalis/tree/master/examples/async-std-runtime\"\u003e\u003cimg src=\"https://img.shields.io/badge/-basic_example-black?style=flat-square\u0026logo=github\"/\u003e\u003c/a\u003e |\n| Redis        | \u003ca href=\"https://docs.rs/apalis-redis\"\u003e\u003cimg src=\"https://img.shields.io/crates/v/apalis-redis?style=flat-square\"\u003e\u003c/a\u003e | \u003ca href=\"https://github.com/geofmureithi/apalis/tree/master/examples/redis\"\u003e\u003cimg src=\"https://img.shields.io/badge/-basic_example-black?style=flat-square\u0026logo=redis\"/\u003e\u003c/a\u003e              |\n| Sqlite       | \u003ca href=\"https://docs.rs/apalis-sql\"\u003e\u003cimg src=\"https://img.shields.io/crates/v/apalis-sql?style=flat-square\"\u003e\u003c/a\u003e     | \u003ca href=\"https://github.com/geofmureithi/apalis/tree/master/examples/sqlite\"\u003e\u003cimg src=\"https://img.shields.io/badge/-sqlite_example-black?style=flat-square\u0026logo=sqlite\"/\u003e\u003c/a\u003e           |\n| Postgres     | \u003ca href=\"https://docs.rs/apalis-sql\"\u003e\u003cimg src=\"https://img.shields.io/crates/v/apalis-sql?style=flat-square\"\u003e\u003c/a\u003e     | \u003ca href=\"https://github.com/geofmureithi/apalis/tree/master/examples/postgres\"\u003e\u003cimg src=\"https://img.shields.io/badge/-postgres_example-black?style=flat-square\u0026logo=postgres\"/\u003e\u003c/a\u003e     |\n| MySQL        | \u003ca href=\"https://docs.rs/apalis-sql\"\u003e\u003cimg src=\"https://img.shields.io/crates/v/apalis-sql?style=flat-square\"\u003e\u003c/a\u003e     | \u003ca href=\"https://github.com/geofmureithi/apalis/tree/master/examples/mysql\"\u003e\u003cimg src=\"https://img.shields.io/badge/-mysql_example-black?style=flat-square\u0026logo=mysql\"/\u003e\u003c/a\u003e              |\n| Amqp         | \u003ca href=\"https://docs.rs/apalis-amqp\"\u003e\u003cimg src=\"https://img.shields.io/crates/v/apalis-amqp?style=flat-square\"\u003e\u003c/a\u003e   | \u003ca href=\"https://github.com/geofmureithi/apalis-amqp/tree/master/examples/basic.rs\"\u003e\u003cimg src=\"https://img.shields.io/badge/-rabbitmq_example-black?style=flat-square\u0026logo=github\"/\u003e\u003c/a\u003e  |\n| From Scratch | \u003ca href=\"https://docs.rs/apalis-core\"\u003e\u003cimg src=\"https://img.shields.io/crates/v/apalis-core?style=flat-square\"\u003e\u003c/a\u003e   |                                                                                                                                                                                          |\n\n## Getting Started\n\nTo get started, just add to Cargo.toml\n\n```toml\n[dependencies]\napalis = { version = \"0.5\", features = [\"redis\"] } # Backends available: postgres, sqlite, mysql, amqp\n```\n\n## Usage\n\n```rust\nuse apalis::prelude::*;\nuse apalis::redis::RedisStorage;\nuse serde::{Deserialize, Serialize};\nuse anyhow::Result;\n\n#[derive(Debug, Deserialize, Serialize)]\nstruct Email {\n    to: String,\n}\n\nimpl Job for Email {\n    const NAME: \u0026'static str = \"apalis::Email\";\n}\n\n/// A function that will be converted into a service.\nasync fn send_email(job: Email, data: Data\u003cusize\u003e) -\u003e Result\u003c(), Error\u003e {\n  /// execute job\n  Ok(())\n}\n\n#[tokio::main]\nasync fn main() -\u003e Result\u003c()\u003e {\n    std::env::set_var(\"RUST_LOG\", \"debug\");\n    env_logger::init();\n    let redis_url = std::env::var(\"REDIS_URL\").expect(\"Missing env variable REDIS_URL\");\n    let storage = RedisStorage::new(redis).await?;\n    Monitor::new()\n        .register_with_count(2, {\n            WorkerBuilder::new(format!(\"email-worker\"))\n                .data(0usize)\n                .with_storage(storage)\n                .build_fn(send_email)\n        })\n        .run()\n        .await\n}\n\n```\n\nThen\n\n```rust\n//This can be in another part of the program or another application eg a http server\nasync fn produce_route_jobs(storage: \u0026RedisStorage\u003cEmail\u003e) -\u003e Result\u003c()\u003e {\n    let mut storage = storage.clone();\n    storage\n        .push(Email {\n            to: \"test@example.com\".to_string(),\n        })\n        .await?;\n}\n\n```\n\n## Feature flags\n\n- _tracing_ (enabled by default) — Support Tracing 👀\n- _redis_ — Include redis storage\n- _postgres_ — Include Postgres storage\n- _sqlite_ — Include SQlite storage\n- _mysql_ — Include MySql storage\n- _cron_ — Include cron job processing\n- _sentry_ — Support for Sentry exception and performance monitoring\n- _prometheus_ — Support Prometheus metrics\n- _retry_ — Support direct retrying jobs\n- _timeout_ — Support timeouts on jobs\n- _limit_ — 💪 Limit the amount of jobs\n- _filter_ — Support filtering jobs based on a predicate\n\n## Storage Comparison\n\nSince we provide a few storage solutions, here is a table comparing them:\n\n| Feature         | Redis | Sqlite | Postgres | Sled | Mysql | Mongo | Cron |\n| :-------------- | :---: | :----: | :------: | :--: | :---: | :---: | :--: |\n| Scheduled jobs  |   ✓   |   ✓    |    ✓     |  x   |   ✓   |   x   |  ✓   |\n| Retry jobs      |   ✓   |   ✓    |    ✓     |  x   |   ✓   |   x   |  ✓   |\n| Persistence     |   ✓   |   ✓    |    ✓     |  x   |   ✓   |   x   | BYO  |\n| Rerun Dead jobs |   ✓   |   ✓    |    ✓     |  x   |   ✓   |   x   |  x   |\n\n## How apalis works\n\nHere is a basic example of how the core parts integrate\n\n```mermaid\nsequenceDiagram\n    participant App\n    participant Worker\n    participant Backend\n\n    App-\u003e\u003e+Backend: Add job to queue\n    Backend--\u003e\u003e+Worker: Job data\n    Worker-\u003e\u003e+Backend: Update job status to 'Running'\n    Worker-\u003e\u003e+App: Started job\n    loop job execution\n        Worker--\u003e\u003e-App: Report job progress\n    end\n    Worker-\u003e\u003e+Backend: Update job status to 'completed'\n```\n\n## External examples\n\n- [Shuttle](https://github.com/shuttle-hq/shuttle-examples/tree/main/shuttle-cron): Using apalis-cron with [shuttle.rs](https://shuttle.rs)\n- [Actix-Web](https://github.com/actix/examples/tree/master/background-jobs): Using apalis-redis with actix-web\n\n## Projects using apalis\n\n- [Ryot](https://github.com/IgnisDa/ryot): A self hosted platform for tracking various facets of your life - media, fitness etc.\n- [Summarizer](https://github.com/akhildevelops/summarizer): Podcast summarizer\n- [Universal Inbox](https://github.com/universal-inbox/universal-inbox): Universal Inbox is a solution that centralizes all your notifications and tasks in one place to create a unique inbox.\n\n## Resources\n\n- [Background job processing with rust using actix and redis](https://mureithi.me/blog/background-job-processing-with-rust-actix-redis)\n\n### Web UI\n\nIf you are running [apalis Board](https://github.com/geofmureithi/apalis-board), you can easily manage your jobs. See a working [rest API example here](https://github.com/geofmureithi/apalis/tree/master/examples/rest-api)\n\n\u003cimg src=\"https://github.com/geofmureithi/apalis-board/raw/master/screenshots/workers.png\" width=\"100%\"\u003e\n\n## Thanks to\n\n- [tower] - Tower is a library of modular and reusable components for building robust networking clients and servers.\n- [redis-rs](https://github.com/mitsuhiko/redis-rs) - Redis library for rust\n- [sqlx](https://github.com/launchbadge/sqlx) - The Rust SQL Toolkit\n\n## Roadmap\n\nv 0.5\n\n- [x] Refactor the crates structure\n- [x] Mocking utilities\n- [ ] Support for SurrealDB and Mongo\n- [ ] Lock free for Postgres\n- [x] Add more utility layers\n- [x] Use extractors in job fn structure\n- [x] Polish up documentation\n- [ ] Improve and standardize apalis Board\n- [ ] Benchmarks\n\nv 0.4\n\n- [x] Move from actor based to layer based processing\n- [x] Graceful Shutdown\n- [x] Allow other types of executors apart from Tokio\n- [x] Mock/Test Worker\n- [x] Improve monitoring\n- [x] Add job progress via layer\n- [x] Add more sources\n\nv 0.3\n\n- [x] Standardize API (Storage, Worker, Data, Middleware, Context )\n- [x] Introduce SQL\n- [x] Implement layers for Sentry and Tracing.\n- [x] Improve documentation\n- [x] Organized modules and features.\n- [x] Basic Web API Interface\n- [x] Sql Examples\n- [x] Sqlx migrations\n\nv 0.2\n\n- [x] Redis Example\n- [x] Actix Web Example\n\n## Contributing\n\nPlease read [CONTRIBUTING.md](CONTRIBUTING.md) for details on our code of conduct, and the process for submitting pull requests to us.\n\n## Versioning\n\nWe use [SemVer](http://semver.org/) for versioning. For the versions available, see the [tags on this repository](https://github.com/geofmureithi/apalis/tags).\n\n## Authors\n\n- **Njuguna Mureithi** - _Initial work_ - [Njuguna Mureithi](https://github.com/geofmureithi)\n\nSee also the list of [contributors](https://github.com/geofmureithi/apalis/contributors) who participated in this project.\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE.md](LICENSE.md) file for details\n\n[`tower::Service`]: https://docs.rs/tower/latest/tower/trait.Service.html\n[tower]: https://crates.io/crates/tower\n[`actix`]: https://crates.io/crates/actix\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgeofmureithi%2Fapalis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgeofmureithi%2Fapalis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgeofmureithi%2Fapalis/lists"}