Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/insipx/coil
Futures-based background work queue built with SQLx and Postgres
https://github.com/insipx/coil
async-await async-std asyncronous-jobs job-queue postgres rust smol sqlx task-queue tokio
Last synced: about 2 months ago
JSON representation
Futures-based background work queue built with SQLx and Postgres
- Host: GitHub
- URL: https://github.com/insipx/coil
- Owner: insipx
- License: gpl-3.0
- Created: 2020-08-02T17:11:56.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2023-03-06T22:04:47.000Z (almost 2 years ago)
- Last Synced: 2024-10-29T01:16:01.068Z (about 2 months ago)
- Topics: async-await, async-std, asyncronous-jobs, job-queue, postgres, rust, smol, sqlx, task-queue, tokio
- Language: Rust
- Homepage:
- Size: 288 KB
- Stars: 12
- Watchers: 2
- Forks: 3
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Coil
An async task queue built with SQLx, Postgres, and Rayon
[](https://coveralls.io/github/insipx/coil?branch=master)
Coil is heavily inspired by and takes heavily from [swirl](https://github.com/sgrif/swirl). In many places of the codebase, code is very similiar.
Supports synchronous and asynchronous jobs. Synchronous jobs will be spawned into a threadpool managed by [`rayon`](https://github.com/rayon-rs/rayon). Async jobs will be spawned onto an executor. The only requirement is that the executor implements the futures `Spawn` trait. This way, `coil` supports `Tokio`, `smol`, and `async-std`.
† This software is alpha, and not intended for production use yet. Use at your own risk.
†† This software is sublicensed as GPLv3. Portions from `swirl` are licensed under MIT. See the [License](#license) section
---
### Examples
```rust
struct Size {
width: u32,
height: u32
}
#[coil::background_task]
async fn resize_image(id: u32, size: Size) -> Result<(), Error> {
// some work
}
```
With an environment
```rust
struct Size {
width: u32,
height: u32
}
struct Environment {
file_server_private_key: String,
http_client: http_lib::Client,
conn: sqlx::PgPool
}
#[coil::background_task]
async fn resize_image(env: &Environment, id: u32, size: Size) -> Result<(), Error> {
// some work
}
```
```rust
resize_image_with_env("tohru".to_string(), Size { height: 32, width: 32 }).enqueue(&pool).await;
let runner = coil::RunnerBuilder::new(env, Executor, pool)
.num_threads(8)
.build()
.unwrap();
runner.run_all_pending_tasks().await.unwrap()
```
### Differences from [`swirl`](https://github.com/sgrif/swirl)
- Supports asynchronous jobs/executors
- Supports jobs with generic arguments
- Serializes data into Postgres with Messagepack instead of JSON
- In asynchronous jobs, database queries will be run asynchronously with SQLx
- Migrations are included in the binary and exposed via a `migrate` fn.
- Enqueue is an `async fn`
## License
This program includes code from the `Swirl` library, used under the [MIT License](https://github.com/sgrif/swirl/blob/master/LICENSE-MIT) or [https://opensource.org/licenses/MIT](https://opensource.org/licenses/MIT)
This program is sublicensed under [GPLv3](https://github.com/insipx/coil/blob/master/LICENSE). An original MIT license copy for `Swirl` is provided in the source.