https://github.com/aio-libs/janus
Thread-safe asyncio-aware queue for Python
https://github.com/aio-libs/janus
asyncio mypy queue threadsafe
Last synced: 13 days ago
JSON representation
Thread-safe asyncio-aware queue for Python
- Host: GitHub
- URL: https://github.com/aio-libs/janus
- Owner: aio-libs
- License: apache-2.0
- Created: 2015-06-08T15:45:28.000Z (almost 10 years ago)
- Default Branch: master
- Last Pushed: 2025-03-17T04:09:00.000Z (29 days ago)
- Last Synced: 2025-03-17T11:04:18.070Z (29 days ago)
- Topics: asyncio, mypy, queue, threadsafe
- Language: Python
- Homepage:
- Size: 626 KB
- Stars: 855
- Watchers: 21
- Forks: 53
- Open Issues: 3
-
Metadata Files:
- Readme: README.rst
- Changelog: CHANGES.rst
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
- best-of-python - GitHub - 21% open · ⏱️ 06.06.2024): (Data Structures)
README
=======
janus
=======
.. image:: https://github.com/aio-libs/janus/actions/workflows/ci.yml/badge.svg
:target: https://github.com/aio-libs/janus/actions/workflows/ci.yml
.. image:: https://codecov.io/gh/aio-libs/janus/branch/master/graph/badge.svg
:target: https://codecov.io/gh/aio-libs/janus
.. image:: https://img.shields.io/pypi/v/janus.svg
:target: https://pypi.python.org/pypi/janus
.. image:: https://badges.gitter.im/Join%20Chat.svg
:target: https://gitter.im/aio-libs/Lobby
:alt: Chat on Gitter
Mixed sync-async queue, supposed to be used for communicating between
classic synchronous (threaded) code and asynchronous (in terms of
`asyncio `_) one.
Like `Janus god `_ the queue
object from the library has two faces: synchronous and asynchronous
interface.
Synchronous is fully compatible with `standard queue
`_, asynchronous one
follows `asyncio queue design
`_.
Usage
=====
Three queues are available:
* ``Queue``
* ``LifoQueue``
* ``PriorityQueue``
Each has two properties: ``sync_q`` and ``async_q``.
Use the first to get synchronous interface and the second to get asynchronous
one.
Example
-------
.. code:: python
import asyncio
import janus
def threaded(sync_q: janus.SyncQueue[int]) -> None:
for i in range(100):
sync_q.put(i)
sync_q.join()
async def async_coro(async_q: janus.AsyncQueue[int]) -> None:
for i in range(100):
val = await async_q.get()
assert val == i
async_q.task_done()
async def main() -> None:
queue: janus.Queue[int] = janus.Queue()
loop = asyncio.get_running_loop()
fut = loop.run_in_executor(None, threaded, queue.sync_q)
await async_coro(queue.async_q)
await fut
await queue.aclose()
asyncio.run(main())
Limitations
===========
This library is built using a classic thread-safe design. The design is
time-tested, but has some limitations.
* Once you are done working with a queue, you must properly close it using
``aclose()``. This is because this library creates new tasks to notify other
threads. If you do not properly close the queue,
`asyncio may generate error messages
`_.
* The library has quite good performance only when used as intended, that is,
for communication between synchronous code and asynchronous one.
For sync-only and async-only cases, use queues from
`queue `_ and
`asyncio queue `_ modules,
otherwise `the slowdown can be significant
`_.
* You cannot use queues for communicating between two different event loops
because, like all asyncio primitives, they bind to the current one.
Development status is production/stable. The ``janus`` library is maintained to
support the latest versions of Python and fixes, but no major changes will be
made. If your application is performance-sensitive, or if you need any new
features such as ``anyio`` support, try the experimental
`culsans `_ library as an alternative.
Communication channels
======================
GitHub Discussions: https://github.com/aio-libs/janus/discussions
Feel free to post your questions and ideas here.
*gitter chat* https://gitter.im/aio-libs/Lobby
License
=======
``janus`` library is offered under Apache 2 license.
Thanks
======
The library development is sponsored by DataRobot (https://datarobot.com)