https://github.com/ymichael/async-py
Poor man's python task queue.
https://github.com/ymichael/async-py
Last synced: over 1 year ago
JSON representation
Poor man's python task queue.
- Host: GitHub
- URL: https://github.com/ymichael/async-py
- Owner: ymichael
- License: mit
- Created: 2014-09-20T14:04:37.000Z (over 11 years ago)
- Default Branch: master
- Last Pushed: 2014-09-21T06:46:47.000Z (over 11 years ago)
- Last Synced: 2025-01-11T04:41:13.201Z (over 1 year ago)
- Language: Python
- Size: 148 KB
- Stars: 2
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# async-py
Poor man's implementation of a redis backed python task queue.
The basic idea is to save the 'fully qualified' module path and function
name as well as the pickled arguments to a queue somewhere. Worker threads
then pick off tasks from this queue and process them asynchronously.
The way to make a function "async" is via a decorator.
When the resulting function is called, it now requires the time taken to pickle
the arguments and add the resulting string to the queue.
## usage
1. Run redis (so tasks can be added to the queue)
2. Run worker thread(s) to start picking off tasks from the queue.
3. Add the decorator to functions that do not need to run inline.
```py
import async
# Configure your redis connection and queue name etc.
delay = async.get_async_decorator()
def slow_func():
...
@delay
def slow_func_async():
slow_func()
```
## Running the worker
1. Run from the root folder of your application.
2. Specify redis connection information (optional)
3. Specify queue name (to allow multiple applications to use async-py on the same instance of redis)
```sh
$ py async/worker.py -h
usage: worker.py [-h] -q QUEUE [-r HOST] [-p PORT] [-d DB]
async-py task worker
optional arguments:
-h, --help show this help message and exit
Worker configuration:
-q QUEUE, --queue QUEUE
The async-py task queue to pull tasks from.
Redis connection information:
-r HOST, --host HOST Redis host.
-p PORT, --port PORT Redis port.
-d DB, --db DB Redis db.
```