{"id":18728116,"url":"https://github.com/narrativescience-old/async-task-queue","last_synced_at":"2025-11-12T03:30:15.590Z","repository":{"id":57412288,"uuid":"249534202","full_name":"NarrativeScience-old/async-task-queue","owner":"NarrativeScience-old","description":"In-memory FIFO queue for concurrent task execution","archived":false,"fork":false,"pushed_at":"2020-03-28T03:49:17.000Z","size":21,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":14,"default_branch":"master","last_synced_at":"2025-02-04T09:20:31.249Z","etag":null,"topics":["async-queue","asyncio","semaphore"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NarrativeScience-old.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-03-23T20:14:18.000Z","updated_at":"2022-10-29T11:51:23.000Z","dependencies_parsed_at":"2022-08-27T23:50:58.211Z","dependency_job_id":null,"html_url":"https://github.com/NarrativeScience-old/async-task-queue","commit_stats":null,"previous_names":["narrativescience/async-task-queue"],"tags_count":3,"template":false,"template_full_name":"NarrativeScience-old/python-package-template","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NarrativeScience-old%2Fasync-task-queue","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NarrativeScience-old%2Fasync-task-queue/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NarrativeScience-old%2Fasync-task-queue/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NarrativeScience-old%2Fasync-task-queue/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NarrativeScience-old","download_url":"https://codeload.github.com/NarrativeScience-old/async-task-queue/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239599078,"owners_count":19665911,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["async-queue","asyncio","semaphore"],"created_at":"2024-11-07T14:19:47.502Z","updated_at":"2025-11-12T03:30:15.465Z","avatar_url":"https://github.com/NarrativeScience-old.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# async-task-queue\n\n[![CircleCI](https://circleci.com/gh/NarrativeScience/async-task-queue/tree/master.svg?style=shield)](https://circleci.com/gh/NarrativeScience/async-task-queue/tree/master) [![](https://img.shields.io/pypi/v/async-task-queue.svg)](https://pypi.org/pypi/async-task-queue/) [![License](https://img.shields.io/badge/License-BSD%203--Clause-blue.svg)](https://opensource.org/licenses/BSD-3-Clause)\n\nIn-memory FIFO queue for concurrent task execution. Used to execute tasks concurrently with optional control (via semaphore) over the max number of tasks running at the same time.\n\nFeatures:\n\n- Queue processing summary logging\n- Introspection of failed, retried, and succeeded tasks\n- Task retries (optional)\n- Task execution timeout (optional)\n- Queue processing with semaphore (optional)\n- Batch size control (optional)\n\nTOC:\n\n- [Installation](#installation)\n- [Guide](#guide)\n- [Development](#development)\n\n## Installation\n\nasync-task-queue requires Python 3.6 or above.\n\n```bash\npip install async-task-queue\n```\n\n## Guide\n\n```python\nimport logging\nfrom async_task_queue import AsyncTask, AsyncTaskQueue\n\n# Initialize a logger\nlogger = logging.getLogger(\"foo\")\n\n# Initialize an AsyncTaskQueue where:\n#   - At most 5 tasks are running concurrently\n#   - Number of tasks executing concurrently should be limited by a semaphore\n#   - Failed tasks should be retried (default behavior)\n#   - Executing the tasks queued should timeout and be cancelled after 5 minutes\ntask_queue = AsyncTaskQueue(\n    logger,\n    use_semaphore=True,\n    batch_size=5,\n    execution_timeout=300\n)\n\n# Add async tasks to the queue\ntask_queue.enqueue(\n    [\n        AsyncTask(some_coroutine, *args, **kwargs) for args, kwargs in some_args_kwargs\n    ]\n)\n\n# Start processing the queue\nawait task_queue.execute()\n```\n\n## Development\n\nTo develop async-task-queue, install dependencies and enable the pre-commit hook:\n\n```bash\npip install pre-commit tox\npre-commit install\n```\n\nTo run tests:\n\n```bash\ntox\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnarrativescience-old%2Fasync-task-queue","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnarrativescience-old%2Fasync-task-queue","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnarrativescience-old%2Fasync-task-queue/lists"}