{"id":13426994,"url":"https://github.com/litements/litequeue","last_synced_at":"2026-04-09T00:34:29.782Z","repository":{"id":39536420,"uuid":"335426292","full_name":"litements/litequeue","owner":"litements","description":"Queue built on top of SQLite","archived":false,"fork":false,"pushed_at":"2025-02-10T10:06:44.000Z","size":86,"stargazers_count":210,"open_issues_count":1,"forks_count":11,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-09-28T11:03:55.526Z","etag":null,"topics":["message","persitent","python","queue","sql","sqlite"],"latest_commit_sha":null,"homepage":"https://litements.polyrand.net/queue/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/litements.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-02-02T21:20:26.000Z","updated_at":"2025-09-26T01:07:50.000Z","dependencies_parsed_at":"2022-08-28T22:11:52.754Z","dependency_job_id":"a4816cb1-da03-43a2-ad9f-3500548a28c9","html_url":"https://github.com/litements/litequeue","commit_stats":{"total_commits":81,"total_committers":2,"mean_commits":40.5,"dds":"0.012345679012345734","last_synced_commit":"c768cce7cbbe938f10b38d48a72eb99cb6dddc8f"},"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"purl":"pkg:github/litements/litequeue","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/litements%2Flitequeue","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/litements%2Flitequeue/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/litements%2Flitequeue/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/litements%2Flitequeue/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/litements","download_url":"https://codeload.github.com/litements/litequeue/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/litements%2Flitequeue/sbom","scorecard":{"id":592713,"data":{"date":"2025-08-11","repo":{"name":"github.com/litements/litequeue","commit":"f23754713c30cc5b4e5379f1d544541eb1801bb6"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.5,"checks":[{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Code-Review","score":1,"reason":"Found 4/22 approved changesets -- score normalized to 1","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/pypi-publish.yaml:17: update your workflow using https://app.stepsecurity.io/secureworkflow/litements/litequeue/pypi-publish.yaml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/pypi-publish.yaml:19: update your workflow using https://app.stepsecurity.io/secureworkflow/litements/litequeue/pypi-publish.yaml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/tests.yaml:16: update your workflow using https://app.stepsecurity.io/secureworkflow/litements/litequeue/tests.yaml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/tests.yaml:17: update your workflow using https://app.stepsecurity.io/secureworkflow/litements/litequeue/tests.yaml/main?enable=pin","Info:   0 out of   4 GitHub-owned GitHubAction dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/pypi-publish.yaml:1","Warn: no topLevel permission defined: .github/workflows/tests.yaml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'main'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 12 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-20T22:22:49.549Z","repository_id":39536420,"created_at":"2025-08-20T22:22:49.549Z","updated_at":"2025-08-20T22:22:49.549Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31580183,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-08T14:31:17.711Z","status":"ssl_error","status_checked_at":"2026-04-08T14:31:17.202Z","response_time":54,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["message","persitent","python","queue","sql","sqlite"],"created_at":"2024-07-31T00:01:50.415Z","updated_at":"2026-04-09T00:34:29.748Z","avatar_url":"https://github.com/litements.png","language":"Python","readme":"# litequeue\n\n\u003e Queue implemented on top of SQLite\n\n## Why?\n\nYou can use this to implement a persistent queue. It also has extra timing\nmetrics for the messages/tasks, and the api to set a message as **done** lets\nyou specifiy the `message_id` to be set as done.\n\nSince it's all based on SQLite / SQL, it is easily extendable.\n\nMessages are always passed as strings, so you can use json data as messages.\nMessages are interpreted as tasks, so after you `pop` a message, you need to\nmark it as done when you finish processing it. When you run the `.prune()`\nmethod, it will remove all the finished tasks from the database.\n\n## Installation\n\nCreate a virtual environment if you are alredy not inside one and install the\npackage using pip:\n\n```\npython3 -m venv .venv\npython3 -m pip --require-virtualenv install --upgrade litequeue\n```\n\n## Quickstart\n\n```python\nfrom litequeue import LiteQueue\n\nq = LiteQueue(\":memory:\")\n\nq.put(\"hello\")\nq.put(\"world\")\n\n# Message object used by LiteQueue\n# Message(data='world', message_id=UUID('063e95f1-3d9f-7547-8000-c3eb531fff93'), status=\u003cMessageStatus.READY: 0\u003e, in_time=1676238611851409010, lock_time=None, done_time=None)\n\ntask = q.pop()\n\nprint(task)\n# Message(data='hello', message_id='063e95f1-3d9e-7bbc-8000-a6a18a5f65d1', status=1, in_time=1676238611851279408, lock_time=1676238623180543854, done_time=None)\n\nq.done(task.message_id)\n\nq.get(task.message_id)\n\n# Message(\n#     data='hello',\n#     message_id='063e95f1-3d9e-7bbc-8000-a6a18a5f65d1',\n#     status=2,                \u003c---- status is now 2 (DONE)\n#     in_time=1676238611851279408,\n#     lock_time=1676238623180543854,\n#     done_time=1676238641276753673\n# )\n\n```\n\nCheck out [the docs page](https://litements.polyrand.net/queue/) for more.\n\n## Differences with a normal Python `queue.Queue`\n\n- Persistence\n- Different API to set tasks as done (you tell it which `message_id` to set as done)\n- Timing metrics. As long as tasks are still in the queue or not pruned, you can see how long they have been there or how long they took to finish.\n- Easy to extend using SQL\n\n## Examples and bechmarks and bechmarks\n\nYou can have a look at the `test.py` file. The tests are short and showcase\ndifferent usage scenarios.\n\nThe `benchmark.ipynb` file contains some benchmarks comparing `litequeue` to\nthe built-in Python `queue.Queue`.\n\n## Multiple queues in the same DB file\n\nIn the `LiteQueue` class, the `filename_or_conn` parameter defines the SQLite\nfile that will be used to store the messages, the `queue_name` parameter is used\nto define the table name that will be used to store the messages.\n\nMultiple queues in the same SQLite is supported, but it's neither tested nor\nrecommended. But if you need it, you can use different `queue_name` values when\ninitializing the `LiteQueue` object to store multiple queues in the same DB\nfile.\n\n```python\nimport tempfile\nimport litequeue\n\n\nwith tempfile.TemporaryDirectory() as tmpdirname:\n\n    db_path = tmpdirname + \"/test.sqlite3\"\n\n    q1 = litequeue.LiteQueue(db_path, queue_name=\"q1\")\n    q2 = litequeue.LiteQueue(db_path, queue_name=\"q2\")\n\n    q1.put(\"a\")\n    q1.put(\"b\")\n\n    print(\"Q1 size\", q1.qsize())\n\n    print(\"Q2 size\", q2.qsize())\n\n    q2.put(\"c\")\n    q2.put(\"d\")\n\n    print(\"Q2 size\", q2.qsize())\n\n    print(q1.pop())\n    print(q1.peek())\n\n    print(q2.peek())\n    print(q2.pop())\n\n```\n\n## Meta\n\nRicardo Ander-Egg Aguilar – [@ricardoanderegg](https://twitter.com/ricardoanderegg) –\n\n- [ricardoanderegg.com](http://ricardoanderegg.com/)\n- [github.com/polyrand](https://github.com/polyrand/)\n\nDistributed under the MIT license. See `LICENSE` for more information.\n\n## Important changes\n\n- In version 0.6:\n  - The database schema has changed and the column `message` is now `data`.\n  - Time is still measured as an integer, but now it's using nanoseconds.\n  - Messages are represented as a frozen dataclass, not as a dictionary.\n  - Message IDs are uuidv7 strings.\n- In version 0.4 the database schema has changed and the column `task_id` is now `message_id`.\n\n## Contributing\n\nThe only hard rules for the project are:\n\n- No extra dependencies allowed\n- No extra files, everything must be inside `litequeue.py` file.\n- Tests must be inside the `test.py` file.\n- Files must be formatted using `black` and `isort`, using one import per line.\n","funding_links":[],"categories":["Jupyter Notebook"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flitements%2Flitequeue","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flitements%2Flitequeue","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flitements%2Flitequeue/lists"}