{"id":18234101,"url":"https://github.com/chekart/workload","last_synced_at":"2026-02-28T08:35:09.270Z","repository":{"id":62589459,"uuid":"120618589","full_name":"chekart/workload","owner":"chekart","description":"Simple python task distribution","archived":false,"fork":false,"pushed_at":"2018-10-12T07:28:12.000Z","size":23,"stargazers_count":9,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-09-19T19:10:02.982Z","etag":null,"topics":["distributed","python","task-scheduler","workload"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chekart.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-02-07T13:27:44.000Z","updated_at":"2024-10-27T02:29:05.000Z","dependencies_parsed_at":"2022-11-03T17:55:44.449Z","dependency_job_id":null,"html_url":"https://github.com/chekart/workload","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/chekart/workload","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chekart%2Fworkload","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chekart%2Fworkload/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chekart%2Fworkload/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chekart%2Fworkload/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chekart","download_url":"https://codeload.github.com/chekart/workload/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chekart%2Fworkload/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29928881,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-27T19:37:42.220Z","status":"online","status_checked_at":"2026-02-28T02:00:07.010Z","response_time":90,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["distributed","python","task-scheduler","workload"],"created_at":"2024-11-04T17:03:42.139Z","updated_at":"2026-02-28T08:35:09.247Z","avatar_url":"https://github.com/chekart.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Workload\n\nSimple, clean, light python task distribution.\nNo magic, no configuration, no broker abstraction\n\nThe library uses Redis as broker\n\nUsage:\n\n### Create distributed job\n\n```python\nimport redis\nfrom workload import distributed\n\nREDIS_POOL = redis.ConnectionPool()\n\n@distributed('worker', redis_pool=REDIS_POOL)\ndef worker(job, country):\n    # do stuff\n    job.result('result') # add unique result\n    job.fanout(['task3', 'task4']) # schedule another tasks if needed\n    job.error('oops') # raise and catch exception\n```\n\n### Add workload to do\n\n```python\nfrom jobs import worker\n\nworker.distribute(['task1', 'task2'])\nworker.wait_results()\n```\n\n### Start worker\n\n```python\nfrom jobs import worker\n\nworker.start_processing(concurrency=10)\n```\n\n## Cycle\n\nworkload.cycle is a loop which starts deferred tasks at specified time or interval\n\nUsage:\n\n```python\nfrom workload.cycle import cycle\nfrom jobs import do_work\n\ncycle([\n    # Run job every hour\n    (cycle.interal(hours=1), do_work),\n    # Run job every day at midnight\n    (cycle.at(hour=0), do_work),\n])\n```\n\n## Admin\n\nIn order to monitor deferred and distributed jobs an autogenerated admin can be used.\n\nFrontend is written using jQuery and Bootstrap both served from corresponding official CDNs.\n\nSince we want to keep default install as clean as possible, to run the admin the following dependencies need to be installed manually:\n**falcon**\n**jinja2**\n\nThe result of the **create_admin_app** will be WSGI app which can be served by any WSGI application server\n(uWSGI, gunicorn, werkzeug to name a few)\n\nAdmin table fields description:\n\n#### Deferred section\n\n| Field       | Description                          |\n| ----------- | ------------------------------------ |\n| Name        | Fully qualified python function name |\n| Description | Function docstring                   |\n| Workload    | Number of queued deferred jobs       |\n\n#### Distributed section\n\n| Field       | Description                                             |\n| ----------- | ------------------------------------------------------- |\n| Name        | Fully qualified python function name                    |\n| Description | Function docstring                                      |\n| Workload    | Amount of workload to process                           |\n| Duration    | Current and last (smaller one) duration of job          |\n| Workers     | Amount of active workers processing the distributed job |\n\nUsage:\n\n```python\nfrom workload.admin import create_admin_app, START_DEFER\n\napp = create_admin_app(\n    '/admin',  # url prefix\n    [\n    # START_DEFER is a shortcut for lambda job: job.defer()\n    (deferred_worker, START_DEFER),\n    # without starting function job will be displayed without Start button\n    deferred_readonly_worker,\n    # starting function could be used for debug as well\n    (distributed_worker, lambda job: job.distribute(test_workload)),\n    ],\n    title='My admin',  # the name will be displayed at the top of the admin\n    redis_pool=REDIS_POOL,  # redis pool to display server info, can be ommited\n    credentials=('user', secret'),  # add credentials to enable basic auth\n)\n\n\n# to start local dev server\nif __name__ == '__main__':\n    from werkzeug.serving import run_simple\n    run_simple('localhost', 8000, app, use_reloader=True)\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchekart%2Fworkload","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchekart%2Fworkload","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchekart%2Fworkload/lists"}