https://github.com/noxdafox/pebble
Multi threading and processing eye-candy.
https://github.com/noxdafox/pebble
asyncio decorators multiprocessing pool python threading
Last synced: about 1 month ago
JSON representation
Multi threading and processing eye-candy.
- Host: GitHub
- URL: https://github.com/noxdafox/pebble
- Owner: noxdafox
- License: lgpl-3.0
- Created: 2013-10-16T05:56:17.000Z (over 11 years ago)
- Default Branch: master
- Last Pushed: 2025-03-16T15:59:59.000Z (3 months ago)
- Last Synced: 2025-04-14T16:53:48.455Z (2 months ago)
- Topics: asyncio, decorators, multiprocessing, pool, python, threading
- Language: Python
- Homepage:
- Size: 860 KB
- Stars: 581
- Watchers: 9
- Forks: 55
- Open Issues: 1
-
Metadata Files:
- Readme: README.rst
- License: LICENSE
- Code of conduct: .github/CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
Pebble
======
Pebble provides a neat API to manage threads and processes within an application.
:Source: https://github.com/noxdafox/pebble
:Documentation: https://pebble.readthedocs.io
:Download: https://pypi.org/project/Pebble/
|build badge| |docs badge| |downloads badge|
.. |build badge| image:: https://github.com/noxdafox/pebble/actions/workflows/action.yml/badge.svg
:target: https://github.com/noxdafox/pebble/actions/workflows/action.yml
:alt: Build Status
.. |docs badge| image:: https://readthedocs.org/projects/pebble/badge/?version=latest
:target: https://pebble.readthedocs.io
:alt: Documentation Status
.. |downloads badge| image:: https://img.shields.io/pypi/dm/pebble
:target: https://pypistats.org/packages/pebble
:alt: PyPI - Downloads
Examples
--------
Run a job in a separate thread and wait for its results.
.. code:: python
from pebble import concurrent
@concurrent.thread
def function(foo, bar=0):
return foo + bar
future = function(1, bar=2)
result = future.result() # blocks until results are ready
Same code with AsyncIO support.
.. code:: python
import asyncio
from pebble import asynchronous
@asynchronous.thread
def function(foo, bar=0):
return foo + bar
async def asynchronous_function():
result = await function(1, bar=2) # blocks until results are ready
print(result)
asyncio.run(asynchronous_function())
Run a function with a timeout of ten seconds and deal with errors.
.. code:: python
from pebble import concurrent
from concurrent.futures import TimeoutError
@concurrent.process(timeout=10)
def function(foo, bar=0):
return foo + bar
future = function(1, bar=2)
try:
result = future.result() # blocks until results are ready
except TimeoutError as error:
print("Function took longer than %d seconds" % error.args[1])
except Exception as error:
print("Function raised %s" % error)
print(error.traceback) # traceback of the function
Pools support workers restart, timeout for long running tasks and more.
.. code:: python
from pebble import ProcessPool
from concurrent.futures import TimeoutError
TIMEOUT_SECONDS = 3
def function(foo, bar=0):
return foo + bar
def task_done(future):
try:
result = future.result() # blocks until results are ready
except TimeoutError as error:
print("Function took longer than %d seconds" % error.args[1])
except Exception as error:
print("Function raised %s" % error)
print(error.traceback) # traceback of the function
with ProcessPool(max_workers=5, max_tasks=10) as pool:
for index in range(0, 10):
future = pool.schedule(function, index, bar=1, timeout=TIMEOUT_SECONDS)
future.add_done_callback(task_done)