{"id":18223999,"url":"https://github.com/x42005e1f/culsans","last_synced_at":"2025-04-10T18:15:17.800Z","repository":{"id":260812598,"uuid":"882412521","full_name":"x42005e1f/culsans","owner":"x42005e1f","description":"Thread-safe async-aware queue for Python","archived":false,"fork":false,"pushed_at":"2025-02-19T22:54:48.000Z","size":218,"stargazers_count":21,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-24T15:52:34.561Z","etag":null,"topics":["async-await","asyncio","concurrency","eventlet","gevent","greenlet","library","mypy","python","queue","trio"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/culsans/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/x42005e1f.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSES/0BSD.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-02T18:30:44.000Z","updated_at":"2025-03-24T13:34:58.000Z","dependencies_parsed_at":"2024-12-09T17:24:42.866Z","dependency_job_id":"4d665a50-7056-4035-ba7f-b5a0e01940d6","html_url":"https://github.com/x42005e1f/culsans","commit_stats":null,"previous_names":["x42005e1f/culsans"],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/x42005e1f%2Fculsans","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/x42005e1f%2Fculsans/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/x42005e1f%2Fculsans/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/x42005e1f%2Fculsans/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/x42005e1f","download_url":"https://codeload.github.com/x42005e1f/culsans/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248269603,"owners_count":21075783,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["async-await","asyncio","concurrency","eventlet","gevent","greenlet","library","mypy","python","queue","trio"],"created_at":"2024-11-04T01:05:46.176Z","updated_at":"2025-04-10T18:15:17.788Z","avatar_url":"https://github.com/x42005e1f.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"..\n  SPDX-FileCopyrightText: 2024 Ilya Egorov \u003c0x42005e1f@gmail.com\u003e\n  SPDX-License-Identifier: CC-BY-4.0\n\n=======\nculsans\n=======\n\nMixed sync-async queue, supposed to be used for communicating between classic\nsynchronous (threaded) code and asynchronous one, between two asynchronous\ncodes in different threads, and for any other combination that you want. Based\non the `queue \u003chttps://docs.python.org/3/library/queue.html\u003e`_ module. Built\non the `aiologic \u003chttps://github.com/x42005e1f/aiologic\u003e`_ package. Inspired\nby the `janus \u003chttps://github.com/aio-libs/janus\u003e`_ library.\n\nLike `Culsans god \u003chttps://en.wikipedia.org/wiki/Culsans\u003e`_, the queue object\nfrom the library has two faces: synchronous and asynchronous interface. Unlike\n`Janus library \u003chttps://github.com/aio-libs/janus\u003e`_, synchronous interface\nsupports `eventlet \u003chttps://github.com/eventlet/eventlet\u003e`_,\n`gevent \u003chttps://github.com/gevent/gevent\u003e`_, and\n`threading \u003chttps://docs.python.org/3/library/threading.html\u003e`_, while\nasynchronous interface supports\n`asyncio \u003chttps://docs.python.org/3/library/asyncio.html\u003e`_,\n`trio \u003chttps://github.com/python-trio/trio\u003e`_, and\n`anyio \u003chttps://github.com/agronholm/anyio\u003e`_.\n\nSynchronous is fully compatible with\n`standard queue \u003chttps://docs.python.org/3/library/queue.html\u003e`_, asynchronous\none follows\n`asyncio queue design \u003chttps://docs.python.org/3/library/asyncio-queue.html\u003e`_.\n\nInstallation\n============\n\nInstall from `PyPI \u003chttps://pypi.org/project/culsans/\u003e`_ (recommended):\n\n.. code:: console\n\n    pip install culsans\n\nOr from `GitHub \u003chttps://github.com/x42005e1f/culsans\u003e`_:\n\n.. code:: console\n\n    pip install git+https://github.com/x42005e1f/culsans.git\n\nYou can also use other package managers, such as\n`uv \u003chttps://github.com/astral-sh/uv\u003e`_.\n\nUsage\n=====\n\nThree queues are available:\n\n* ``Queue``\n* ``LifoQueue``\n* ``PriorityQueue``\n\nEach has two properties: ``sync_q`` and ``async_q``.\n\nUse the first to get synchronous interface and the second to get asynchronous\none.\n\nExample\n-------\n\n.. code:: python\n\n    import anyio\n    import culsans\n\n\n    def sync_run(sync_q: culsans.SyncQueue[int]) -\u003e None:\n        for i in range(100):\n            sync_q.put(i)\n        else:\n            sync_q.join()\n\n\n    async def async_run(async_q: culsans.AsyncQueue[int]) -\u003e None:\n        for i in range(100):\n            value = await async_q.get()\n\n            assert value == i\n\n            async_q.task_done()\n\n\n    async def main() -\u003e None:\n        queue: culsans.Queue[int] = culsans.Queue()\n\n        async with anyio.create_task_group() as tasks:\n            tasks.start_soon(anyio.to_thread.run_sync, sync_run, queue.sync_q)\n            tasks.start_soon(async_run, queue.async_q)\n\n        queue.shutdown()\n\n\n    anyio.run(main)\n\nExtras\n------\n\nBoth interfaces support some additional features that are not found in the\noriginal queues.\n\ngrowing \u0026 shrinking\n^^^^^^^^^^^^^^^^^^^\n\nYou can dynamically change the upperbound limit on the number of items that can\nbe placed in the queue with ``queue.maxsize = N``. If it increases (growing),\nthe required number of waiting putters will be woken up. If it decreases\n(shrinking), items exceeding the new limit will remain in the queue, but all\nputters will be blocked until enough items are retrieved from the queue. And if\n*maxsize* is less than or equal to zero, all putters will be woken up.\n\n.. code:: python\n\n    async with anyio.create_task_group() as tasks:\n        async_q = culsans.Queue(1).async_q\n\n        for i in range(4):\n            tasks.start_soon(async_q.put, i)\n\n        await anyio.sleep(1e-3)\n        assert async_q.qsize() == 1\n\n        async_q.maxsize = 2  # growing\n\n        await anyio.sleep(1e-3)\n        assert async_q.qsize() == 2\n\n        async_q.maxsize = 1  # shrinking\n\n        await anyio.sleep(1e-3)\n        assert async_q.qsize() == 2\n\n        async_q.get_nowait()\n\n        await anyio.sleep(1e-3)\n        assert async_q.qsize() == 1\n\n        async_q.maxsize = 0  # now the queue size is infinite\n\n        await anyio.sleep(1e-3)\n        assert async_q.qsize() == 3\n\npeek() \u0026 peek_nowait()\n^^^^^^^^^^^^^^^^^^^^^^\n\nIf you want to check the first item of the queue, but do not want to remove\nthat item from the queue, you can use the ``peek()`` and ``peek_nowait()``\nmethods instead of the ``get()`` and ``get_nowait()`` methods.\n\n.. code:: python\n\n    sync_q = culsans.Queue().sync_q\n\n    sync_q.put(\"spam\")\n\n    assert sync_q.peekable()\n    assert sync_q.peek() == \"spam\"\n    assert sync_q.peek_nowait() == \"spam\"\n    assert sync_q.qsize() == 1\n\nThese methods can be considered an implementation of partial compatibility with\n`gevent queues \u003chttps://www.gevent.org/api/gevent.queue.html\u003e`_.\n\nclear()\n^^^^^^^\n\nIn some scenarios it may be necessary to clear the queue. But it is inefficient\nto do this through a loop, and it causes additional difficulties when it is\nalso necessary to ensure that no new items can be added during the clearing\nprocess. For this purpose, there is an atomic method ``clear()`` that clears\nthe queue most efficiently.\n\n.. code:: python\n\n    async with anyio.create_task_group() as tasks:\n        async_q = culsans.Queue(3).async_q\n\n        for i in range(5):\n            tasks.start_soon(async_q.put, i)\n\n        await anyio.sleep(1e-3)\n        assert async_q.qsize() == 3\n\n        async_q.clear()  # clearing\n\n        await anyio.sleep(1e-3)\n        assert async_q.qsize() == 2\n        assert async_q.get_nowait() == 3\n        assert async_q.get_nowait() == 4\n\nRoughly equivalent to:\n\n.. code:: python\n\n    def clear(queue):\n        while True:\n            try:\n                queue.get_nowait()\n            except Empty:\n                break\n            else:\n                queue.task_done()\n\nSubclasses\n----------\n\nYou can create your own queues by inheriting from existing queue classes as if\nyou were using the ``queue`` module. For example, this is how you can create an\nunordered queue that contains only unique items:\n\n.. code:: python\n\n    from culsans import Queue\n\n\n    class UniqueQueue(Queue):\n        def _init(self, maxsize):\n            self.data = set()\n\n        def _qsize(self):\n            return len(self.data)\n\n        def _put(self, item):\n            self.data.add(item)\n\n        def _get(self):\n            return self.data.pop()\n\n        _peek = None\n\n        def _peekable(self):\n            return False\n\n        def _clear(self):\n            self.data.clear()\n\n.. code:: python\n\n    sync_q = UniqueQueue().sync_q\n\n    sync_q.put_nowait(23)\n    sync_q.put_nowait(42)\n    sync_q.put_nowait(23)\n\n    assert sync_q.qsize() == 2\n    assert sorted(sync_q.get_nowait() for _ in range(2)) == [23, 42]\n\nAll seven of these methods are called in exclusive access mode, so you can\nfreely create your subclasses without thinking about whether your methods are\nthread-safe or not.\n\nGreenlets\n---------\n\nLibraries such as ``eventlet`` and ``gevent`` use\n`greenlets \u003chttps://greenlet.readthedocs.io/en/latest/\u003e`_ instead of\n`tasks \u003chttps://anyio.readthedocs.io/en/stable/tasks.html\u003e`_.\nSince they do not use async-await syntax, their code is similar to synchronous\ncode. There are three ways that you can tell ``culsans`` that you want to use\ngreenlets instead of threads:\n\n* Set ``aiologic.lowlevel.current_green_library_tlocal.name``\n  (for the current thread).\n* Patch the ``threading`` module\n  (for the main thread).\n* Specify ``AIOLOGIC_GREEN_LIBRARY`` environment variable\n  (for all threads).\n\nThe value is the name of the library that you want to use.\n\nCheckpoints\n-----------\n\nSometimes it is useful when each asynchronous call switches execution to the\nnext task and checks for cancellation and timeouts. For example, if you want to\ndistribute CPU usage across all tasks. There are two ways to do this:\n\n* Set ``aiologic.lowlevel.\u003clibrary\u003e_checkpoints_cvar``\n  (for the current context).\n* Specify ``AIOLOGIC_\u003cLIBRARY\u003e_CHECKPOINTS`` environment variable\n  (for all contexts).\n\nThe value is ``True`` or ``False`` for the first way, and a non-empty or empty\nstring for the second.\n\nCheckpoints are enabled by default for the ``trio`` library.\n\nCompatibility\n=============\n\nThe interfaces are compliant with the Python API version 3.13, and the\n``culsans`` library itself is fully compatible with the ``janus`` library\nversion 2.0.0. If you are using ``janus`` in your application and want to\nswitch to ``culsans``, all you have to do is replace this:\n\n.. code:: python\n\n    import janus\n\nwith this:\n\n.. code:: python\n\n    import culsans as janus\n\nand everything will work!\n\nPerformance\n===========\n\nBeing built on the ``aiologic`` package, the ``culsans`` library has\nspeed advantages. When communication is performed within a single thread using\nthe asynchronous API, ``culsans.Queue`` is typically 2 times faster than\n``janus.Queue``:\n\n+-------------+-------------+-------------+-------------+-------------+\n|   python    |    janus    |   culsans   |  aiologic   |   asyncio   |\n+=============+=============+=============+=============+=============+\n| python3.8   |    x1.00    |    x2.26    |    x2.89    |    x1.96    |\n+-------------+-------------+-------------+-------------+-------------+\n| python3.9   |    x1.00    |    x2.24    |    x2.85    |    x1.96    |\n+-------------+-------------+-------------+-------------+-------------+\n| python3.10  |    x1.00    |    x2.31    |    x2.99    |    x1.84    |\n+-------------+-------------+-------------+-------------+-------------+\n| python3.11  |    x1.00    |    x2.35    |    x3.00    |    x1.83    |\n+-------------+-------------+-------------+-------------+-------------+\n| python3.12  |    x1.00    |    x2.45    |    x3.14    |    x1.77    |\n+-------------+-------------+-------------+-------------+-------------+\n| python3.13  |    x1.00    |    x2.61    |    x3.25    |    x1.75    |\n+-------------+-------------+-------------+-------------+-------------+\n| python3.13t |    x1.00    |    x2.35    |    x2.97    |    x1.98    |\n+-------------+-------------+-------------+-------------+-------------+\n| pypy3.8     |    x1.00    |    x2.76    |    x5.48    |    x1.88    |\n+-------------+-------------+-------------+-------------+-------------+\n| pypy3.9     |    x1.00    |    x2.81    |    x5.07    |    x1.86    |\n+-------------+-------------+-------------+-------------+-------------+\n| pypy3.10    |    x1.00    |    x2.38    |    x5.17    |    x1.81    |\n+-------------+-------------+-------------+-------------+-------------+\n| pypy3.11    |    x1.00    |    x2.48    |    x5.29    |    x1.88    |\n+-------------+-------------+-------------+-------------+-------------+\n\nAnd when communication is performed within two threads, they are the same:\n\n+-------------+-------------+-------------+-------------+-------------+\n|   python    |    janus    |   culsans   |  aiologic   |   asyncio   |\n+=============+=============+=============+=============+=============+\n| python3.8   |    x1.00    |   +8.70%    |    x1.51    |   +0.65%    |\n+-------------+-------------+-------------+-------------+-------------+\n| python3.9   |    x1.00    |   +7.60%    |   +36.58%   |   -5.69%    |\n+-------------+-------------+-------------+-------------+-------------+\n| python3.10  |    x1.00    |   +6.38%    |   +23.84%   |   -13.91%   |\n+-------------+-------------+-------------+-------------+-------------+\n| python3.11  |    x1.00    |   +5.86%    |   +22.55%   |   -20.12%   |\n+-------------+-------------+-------------+-------------+-------------+\n| python3.12  |    x1.00    |   +0.29%    |   +15.12%   |   -22.31%   |\n+-------------+-------------+-------------+-------------+-------------+\n| python3.13  |    x1.00    |   +6.00%    |   +17.36%   |   -17.74%   |\n+-------------+-------------+-------------+-------------+-------------+\n| python3.13t |    x1.00    |   +8.49%    |   +24.35%   |   -22.25%   |\n+-------------+-------------+-------------+-------------+-------------+\n| pypy3.8     |    x1.00    |   -0.12%    |   +12.64%   |   +0.83%    |\n+-------------+-------------+-------------+-------------+-------------+\n| pypy3.9     |    x1.00    |   +6.25%    |   +9.59%    |   -0.15%    |\n+-------------+-------------+-------------+-------------+-------------+\n| pypy3.10    |    x1.00    |   +11.37%   |   +8.19%    |   +1.10%    |\n+-------------+-------------+-------------+-------------+-------------+\n| pypy3.11    |    x1.00    |   +7.03%    |   +10.68%   |   -2.20%    |\n+-------------+-------------+-------------+-------------+-------------+\n\nHowever, on your hardware the performance results may be different, especially\nfor the PyPy case, which on older hardware may show a tenfold speedup or more\nin both tables, so you may find it useful to run benchmarks yourself to measure\nactual relative performance.\n\nCommunication channels\n======================\n\nGitHub Discussions: https://github.com/x42005e1f/culsans/discussions\n\nFeel free to post your questions and ideas here.\n\nSupport\n=======\n\nIf you like ``culsans`` and want to support its development, star `its\nrepository on GitHub \u003chttps://github.com/x42005e1f/culsans\u003e`_.\n\n.. image:: https://starchart.cc/x42005e1f/culsans.svg?variant=adaptive\n  :target: https://starchart.cc/x42005e1f/culsans\n\nLicense\n=======\n\nThe ``culsans`` library is `REUSE \u003chttps://reuse.software/\u003e`_-compliant and is\noffered under multiple licenses:\n\n* All original source code is licensed under `ISC \u003cLICENSES/ISC.txt\u003e`_.\n* All original test code is licensed under `0BSD \u003cLICENSES/0BSD.txt\u003e`_.\n* All documentation is licensed under `CC-BY-4.0 \u003cLICENSES/CC-BY-4.0.txt\u003e`_.\n* All configuration is licensed under `CC0-1.0 \u003cLICENSES/CC0-1.0.txt\u003e`_.\n* Some test code borrowed from\n  `python/cpython \u003chttps://github.com/python/cpython\u003e`_ is licensed under\n  `PSF-2.0 \u003cLICENSES/PSF-2.0.txt\u003e`_.\n\nFor more accurate information, check the individual files.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fx42005e1f%2Fculsans","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fx42005e1f%2Fculsans","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fx42005e1f%2Fculsans/lists"}