{"id":23312466,"url":"https://github.com/attumm/meesee","last_synced_at":"2025-08-22T14:30:53.991Z","repository":{"id":53109042,"uuid":"136083254","full_name":"Attumm/meesee","owner":"Attumm","description":"Task queue, Long lived workers for work based parallelization, with processes and Redis as back-end. For distributed computing.","archived":false,"fork":false,"pushed_at":"2024-08-19T08:00:49.000Z","size":113,"stargazers_count":18,"open_issues_count":1,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-20T09:13:54.549Z","etag":null,"topics":["distributed","distributed-computing","python","redis","task-queue","worker-pool","workers"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Attumm.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-06-04T21:10:42.000Z","updated_at":"2024-12-03T02:59:52.000Z","dependencies_parsed_at":"2024-08-19T09:27:58.699Z","dependency_job_id":"fed06188-59be-4089-8568-90b954725a31","html_url":"https://github.com/Attumm/meesee","commit_stats":{"total_commits":50,"total_committers":4,"mean_commits":12.5,"dds":0.28,"last_synced_commit":"7aed15e6b5643675f11161512922279f5468c8a2"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/Attumm/meesee","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Attumm%2Fmeesee","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Attumm%2Fmeesee/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Attumm%2Fmeesee/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Attumm%2Fmeesee/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Attumm","download_url":"https://codeload.github.com/Attumm/meesee/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Attumm%2Fmeesee/sbom","scorecard":{"id":17537,"data":{"date":"2025-08-11","repo":{"name":"github.com/Attumm/meesee","commit":"2986827727eb217c8c204e43563abbb7956beb53"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":4.4,"checks":[{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Code-Review","score":0,"reason":"Found 0/7 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:31: update your workflow using https://app.stepsecurity.io/secureworkflow/Attumm/meesee/ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:33: update your workflow using https://app.stepsecurity.io/secureworkflow/Attumm/meesee/ci.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:56: update your workflow using https://app.stepsecurity.io/secureworkflow/Attumm/meesee/ci.yml/main?enable=pin","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:39","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:40","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:44","Info:   0 out of   2 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   1 third-party GitHubAction dependencies pinned","Info:   0 out of   3 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Token-Permissions","score":10,"reason":"GitHub workflow tokens follow principle of least privilege","details":["Info: topLevel 'contents' permission set to 'read': .github/workflows/ci.yml:13","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'main'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 30 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-14T16:07:05.488Z","repository_id":53109042,"created_at":"2025-08-14T16:07:05.489Z","updated_at":"2025-08-14T16:07:05.489Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271652314,"owners_count":24797043,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-22T02:00:08.480Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["distributed","distributed-computing","python","redis","task-queue","worker-pool","workers"],"created_at":"2024-12-20T14:32:52.115Z","updated_at":"2025-08-22T14:30:53.640Z","avatar_url":"https://github.com/Attumm.png","language":"Python","readme":"# Meesee\n[![CI](https://github.com/Attumm/meesee/actions/workflows/ci.yml/badge.svg)](https://github.com/Attumm/meesee/actions/workflows/ci.yml)\n[![codecov](https://codecov.io/gh/Attumm/meesee/graph/badge.svg?token=upEkV8OYwI)](https://codecov.io/gh/Attumm/meesee)\n[![Downloads](https://static.pepy.tech/badge/meesee)](https://pepy.tech/project/meesee)\n\nMeesee is an task queue system featuring long-lived worker process parallelization through multiprocessing, with Redis as its backend. Engineered for reliability, efficiency, and ease of use, Meesee is tailored for distributed computing environments, particularly in big data and mission-critical software applications. By leveraging individual processes for each worker, Meesee circumvents Python's Global Interpreter Lock (GIL), making it ideal for compute intensive tasks.\n\n## Production Proven\n\n- In active production since 2018, powering the backends of at least three known companies\n- Instances have demonstrated exceptional uptime, running for years without requiring maintenance\n- The only identified scenario necessitating a restart is during network interface changes, which can leave workers connected to a non-functional network—an infrequent occurrence, but noteworthy for systems with multi-year uptime expectations\n- Meesee workers are designed to restart without data loss, ensuring continuity even during rare restart events\n\n\n## Core Design Principles\n\nMeesee was specifically developed to address the following critical challenges in distributed computing:\n\n1. **Long-Term Stability**: Ability to run for extended periods without maintenance or restarts\n2. **Zero Message Loss**: Ensuring no messages are lost during service restarts for maintenance or deployments\n3. **Optimized Performance**: Achieving surprising speed with minimal memory overhead for both client and Redis instances\n4. **Deployment Flexibility**: Capability to schedule messages even when workers are offline during deployment\n5. **Message Integrity Under Load**: Preventing message skips even during high-load scenarios\n6. **Simplicity in Complexity**: Providing an intuitive interface to minimize the learning curve, acknowledging that distributed computing is challenging enough on its own\n\n\n## Examples\nHow to [Examples](https://github.com/Attumm/meesee/tree/main/examples).\n\nCreate my_func that will \n1. print starting message.\n2. Sleep 1 second.\n3. print a ending message.\n\nLet's start 10 of those.\n\n\n```python\nimport time\nfrom meesee import startapp\n\ndef my_func(item, worker_id):\n    print(\"hello, look at me\")\n    time.sleep(1)\n    print('finished item', locals())\n\n\nstartapp(my_func, workers=10)\n```\n\nOpen another terminal, Let's produce some tasks\n```python\nfrom meesee import RedisQueue, config\n\ndef produce(items):\n    r = RedisQueue(**config)\n    for i in range(items):\n        r.send(i)\n\nproduce(10)\n\n```\n\nGreat, the placement of both scripts can be on any machine with connectivity to the redis instance.\n\n## Install\n\n```\n$ pip install meesee\n```\n\n## Example Usage\n\nLet's use Python to make writing workers and producers more fun.\nHere's a simple [example](https://github.com/Attumm/meesee/tree/main/examples/example_decorator_magic_simple.py) demonstrating how to use Meesee the pythonic way.\n\n```python\nfrom meesee import Meesee \n\nbox = Meesee()\n\n@box.worker()\ndef foobar(item, worker_id):\n    print('func: foobar, worker_id: {}, item: {}'.format(worker_id, item))\n\n@box.produce()\ndef produce_to_foobar(items):\n    return items\n\nif __name__ == '__main__':\n    items = [{\"name\": f\"name{i}\"} for i in range(10)]\n    produce_to_foobar(items)\n    box.push_button(workers=5, wait=1)\n```\n\nThis example demonstrates:\n1. Creating a Meesee instance\n2. Defining a worker function using the `@box.worker()` decorator\n3. Defining a producer function using the `@box.produce()` decorator\n4. Producing items to the queue\n5. Starting workers to process the items\n\n\n\nExample output\n```bash\nworker 1 started. foobar listening to foobar\nworker 2 started. foobar listening to foobar\nworker 3 started. foobar listening to foobar\nworker 4 started. foobar listening to foobar\nfunc: foobar, worker_id: 1, item: {\"name\": \"name0\"}\nfunc: foobar, worker_id: 1, item: {\"name\": \"name1\"}\nworker 5 started. foobar listening to foobar\nfunc: foobar, worker_id: 2, item: {\"name\": \"name4\"}\nfunc: foobar, worker_id: 3, item: {\"name\": \"name2\"}\nfunc: foobar, worker_id: 4, item: {\"name\": \"name3\"}\nfunc: foobar, worker_id: 1, item: {\"name\": \"name5\"}\nfunc: foobar, worker_id: 1, item: {\"name\": \"name6\"}\nfunc: foobar, worker_id: 3, item: {\"name\": \"name7\"}\nfunc: foobar, worker_id: 4, item: {\"name\": \"name8\"}\nfunc: foobar, worker_id: 2, item: {\"name\": \"name9\"}\ntimeout reached worker 5 stopped\ntimeout reached worker 2 stopped\ntimeout reached worker 1 stopped\ntimeout reached worker 4 stopped\ntimeout reached worker 3 stopped\nClean shut down\n```\n\nThis output shows:\n- Workers starting and listening to the 'foobar' queue\n- Items being processed by different workers\n- Workers shutting down after the timeout is reached\n\n## Usage explained\n\n\nProducers produce to workers, hence the name. They can either pass iterable values or iter themselves. For instance:\n\n```python\n@box.produce()\ndef produce():\n    return [1, 2, 3]\n\n# or \n\n@box.produce()\ndef produce_yield():\n    yield from [1, 2, 3]\n```\n\nWe can control which queue they will message to in two ways:\n\n1. Specify the queue in the decorator:\n```python\n@box.produce(queue=\"foobar\")\ndef produce_yield():\n    yield from [1, 2, 3]\n```\nThis will produce to the \"foobar\" queue.\n\n2. Use magic naming:\n```python\n@box.produce()\ndef produce_to_foobar():\n    yield from [1, 2, 3]\n```\nBy naming the function `produce_to_foobar`, the function will also send the data to the \"foobar\" queue.\n\nFor workers, they are special in that they will start during multiprocessing. Here's an example to start 5 workers. Since we only set up one worker, all workers will be of that type:\n\n```python\nbox.push_button(workers=5, wait=1)\n```\n\nThis will start 5 worker processes, each listening to the queue specified in the worker function.\n\n### Prerequisites\n\n#### Redis instance\n\nFor Docker\n```\n$ docker run --name some-redis -d redis\n```\n\n## Support and Resources\n\n- For feature requests, additional information or to report issues use github issues.\n- Explore our comprehensive [examples](https://github.com/Attumm/meesee/tree/main/examples) for in-depth usage scenarios and best practices.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fattumm%2Fmeesee","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fattumm%2Fmeesee","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fattumm%2Fmeesee/lists"}