Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/procrastinate-org/procrastinate
PostgreSQL-based Task Queue for Python
https://github.com/procrastinate-org/procrastinate
async postgresql python sync task-queue
Last synced: 6 days ago
JSON representation
PostgreSQL-based Task Queue for Python
- Host: GitHub
- URL: https://github.com/procrastinate-org/procrastinate
- Owner: procrastinate-org
- License: mit
- Created: 2019-02-14T11:30:10.000Z (almost 6 years ago)
- Default Branch: main
- Last Pushed: 2024-10-28T02:21:55.000Z (3 months ago)
- Last Synced: 2024-10-29T15:19:08.779Z (3 months ago)
- Topics: async, postgresql, python, sync, task-queue
- Language: Python
- Homepage: https://procrastinate.readthedocs.io/
- Size: 15 MB
- Stars: 859
- Watchers: 38
- Forks: 53
- Open Issues: 49
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE.md
- Code of conduct: CODE_OF_CONDUCT.md
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project
README
# Procrastinate: PostgreSQL-based Task Queue for Python
[](https://pypi.org/pypi/procrastinate)
[](https://pypi.org/pypi/procrastinate)
[](https://github.com/procrastinate-org/procrastinate/)
[](https://github.com/procrastinate-org/procrastinate/actions?workflow=CI)
[](https://procrastinate.readthedocs.io/en/stable/badge=stable)
[](https://htmlpreview.github.io/?https://github.com/procrastinate-org/procrastinate/blob/python-coverage-comment-action-data/htmlcov/index.html)
[](https://github.com/procrastinate-org/procrastinate/blob/main/LICENSE)
[](https://github.com/procrastinate-org/procrastinate/blob/main/CODE_OF_CONDUCT.md)
[](https://discord.gg/JWZeNq6P6Z)
**Procrastinate is looking for** [additional maintainers!](https://github.com/procrastinate-org/procrastinate/discussions/748)
Procrastinate is an open-source Python 3.8+ distributed task processing
library, leveraging PostgreSQL to store task definitions, manage locks and
dispatch tasks. It can be used within both sync and async code,
has [Django](howto/django/configuration) integration, and is easy to use with ASGI frameworks.
It supports periodic tasks, retries, arbitrary task locks etc.
In other words, from your main code, you call specific functions (tasks) in a
special way and instead of being run on the spot, they're scheduled to
be run elsewhere, now or in the future.
Here's an example (if you want to run the code yourself, head to [Quickstart]):
```python
# mycode.py
import procrastinate
# Make an app in your code
app = procrastinate.App(connector=procrastinate.SyncPsycopgConnector())
# Then define tasks
@app.task(queue="sums")
def sum(a, b):
with open("myfile", "w") as f:
f.write(str(a + b))
with app.open():
# Launch a job
sum.defer(a=3, b=5)
# Somewhere in your program, run a worker (actually, it's usually a
# different program than the one deferring jobs for execution)
app.run_worker(queues=["sums"])
```
The worker will run the job, which will create a text file
named `myfile` with the result of the sum `3 + 5` (that's `8`).
Similarly, from the command line:
```bash
export PROCRASTINATE_APP="mycode.app"
# Launch a job
procrastinate defer mycode.sum '{"a": 3, "b": 5}'
# Run a worker
procrastinate worker -q sums
```
Lastly, you can use Procrastinate asynchronously too (actually, it's the
recommended way to use it):
```python
import asyncio
import procrastinate
# Make an app in your code
app = procrastinate.App(connector=procrastinate.PsycopgConnector())
# Define tasks using coroutine functions
@app.task(queue="sums")
async def sum(a, b):
await asyncio.sleep(a + b)
async with app.open_async():
# Launch a job
await sum.defer_async(a=3, b=5)
# Somewhere in your program, run a worker (actually, it's often a
# different program than the one deferring jobs for execution)
await app.run_worker_async(queues=["sums"])
```
There are quite a few interesting features that Procrastinate adds to the mix.
You can head to the Quickstart section for a general tour or
to the How-To sections for specific features. The Discussion
section should hopefully answer your questions. Otherwise,
feel free to open an [issue](https://github.com/procrastinate-org/procrastinate/issues).
*Note to my future self: add a quick note here on why this project is named*
"[Procrastinate]" ;) .
## Where to go from here
The complete [docs] is probably the best place to learn about the project.
If you encounter a bug, or want to get in touch, you're always welcome to open a
[ticket].
[docs]: https://procrastinate.readthedocs.io/
[procrastinate]: https://en.wikipedia.org/wiki/Procrastination
[quickstart]: https://procrastinate.readthedocs.io/en/stable/quickstart.html
[ticket]: https://github.com/procrastinate-org/procrastinate/issues/new