{"id":18448698,"url":"https://github.com/mindee/tawazi","last_synced_at":"2025-04-04T21:05:33.774Z","repository":{"id":46241677,"uuid":"501272133","full_name":"mindee/tawazi","owner":"mindee","description":"A DAG Scheduler library written in pure python","archived":false,"fork":false,"pushed_at":"2025-03-18T12:32:47.000Z","size":2236,"stargazers_count":88,"open_issues_count":14,"forks_count":5,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-28T20:05:38.279Z","etag":null,"topics":["dag","graph","parallel","python"],"latest_commit_sha":null,"homepage":"https://mindee.github.io/tawazi/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mindee.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-06-08T13:52:13.000Z","updated_at":"2025-03-28T04:33:16.000Z","dependencies_parsed_at":"2024-03-14T15:04:53.314Z","dependency_job_id":"bcd34018-12a5-49f4-a017-aa4643fca190","html_url":"https://github.com/mindee/tawazi","commit_stats":{"total_commits":522,"total_committers":9,"mean_commits":58.0,"dds":0.4291187739463601,"last_synced_commit":"c0f0cc2b81cb44fcef13c2e53bac87ab4fd4a9a7"},"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mindee%2Ftawazi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mindee%2Ftawazi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mindee%2Ftawazi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mindee%2Ftawazi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mindee","download_url":"https://codeload.github.com/mindee/tawazi/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247249524,"owners_count":20908212,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dag","graph","parallel","python"],"created_at":"2024-11-06T07:16:47.193Z","updated_at":"2025-04-04T21:05:33.752Z","avatar_url":"https://github.com/mindee.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# tawazi\n\u003c!--Python badges --\u003e\n[![Python 3.9](https://img.shields.io/badge/python-3.9%20|%203.10%20|%203.11%20|%203.12|%203.13-blue.svg)](https://www.python.org/)\n[![Checked with mypy](http://www.mypy-lang.org/static/mypy_badge.svg)](http://mypy-lang.org/)\n[![CodeFactor](https://www.codefactor.io/repository/github/mindee/tawazi/badge)](https://www.codefactor.io/repository/github/mindee/tawazi)\n[![Downloads](https://img.shields.io/pypi/dm/tawazi)](https://pypi.org/project/tawazi/)\n\n\u003c!--Tawazi Badge--\u003e\n![Tawazi GIF](documentation/tawazi_GIF.gif)\n\n## Introduction\n\n\u003c!-- TODO: put a link explaining what a DAG is--\u003e\n\n\u003c!-- TODO: document that if you want to run DAG in a sync context, the DAG should be sync, if you want to run it in a async context, the DAG should be async--\u003e\n\n[Tawazi](https://pypi.org/project/tawazi/) facilitates **parallel** execution of functions using a [DAG](https://en.wikipedia.org/wiki/Directed_acyclic_graph) dependency structure.\n\n### Explanation\n\nConsider the function `f` that depends on the function `g` and `h`:\n```python\ndef g():\n    print(\"g\")\n    return \"g\"\ndef h():\n    print(\"h\")\n    return \"h\"\ndef f(g_var, h_var):\n    print(\"received\", g_var, h_var)\n    print(\"f\")\n    return \"f\"\n\ndef main():\n    f(g(), h())\n\nmain()\n```\nThe [DAG](https://en.wikipedia.org/wiki/Directed_acyclic_graph) described in `main` can be accelerated if `g` and `h` are executed in parallel. This is what [Tawazi](https://pypi.org/project/tawazi/) does by adding a decorator to the functions `g`, `h`, `f`, and `main`:\n\n```python\nfrom tawazi import dag, xn\n@xn\ndef g():\n    print(\"g\")\n    return \"g\"\n@xn\ndef h():\n    print(\"h\")\n    return \"h\"\n@xn\ndef f(g_var, h_var):\n    print(\"received\", g_var, h_var)\n    print(\"f\")\n    return \"f\"\n@dag(max_concurrency=2)\ndef main():\n    f(g(), h())\n\nmain()\n```\nThe total execution time of `main()` is **1 second instead of 2** which proves that the `g` and `h` have run in parallel, you can measure the speed up in the previous code:\n```python\nfrom time import sleep, time\nfrom tawazi import dag, xn\n@xn\ndef g():\n    sleep(1)\n    print(\"g\")\n    return \"g\"\n@xn\ndef h():\n    sleep(1)\n    print(\"h\")\n    return \"h\"\n@xn\ndef f(g_var, h_var):\n    print(\"received\", g_var, h_var)\n    print(\"f\")\n    return \"f\"\n\n@dag(max_concurrency=2)\ndef main():\n    f(g(), h())\n\nstart = time()\nmain()\nend = time()\nprint(\"time taken\", end - start)\n# h\n# g\n# received g h\n# f\n# time taken 1.004307508468628\n```\n\n### Features\n\nThis library satisfies the following:\n* robust, well tested\n* lightweight\n* Thread Safe\n* Few dependencies\n* Legacy Python versions support (in the future)\n* MyPy compatible\n* Many Python implementations support (in the future)\n\nIn [Tawazi](https://pypi.org/project/tawazi/), a computation sequence is referred to as `DAG`. The functions invoked inside the computation sequence are referred to as `ExecNode`s.\n\nCurrent features are:\n* Specifying the number of \"Threads\" that the `DAG` uses\n* setup `ExecNode`s: These nodes only run once per DAG instance\n* debug `ExecNode`s: These are nodes that run only if `RUN_DEBUG_NODES` environment variable is set\n* running a subgraph of the `DAG` instance\n* Excluding an `ExecNode` from running\n* caching the results of the execution of a `DAG` for faster subsequent execution\n* Priority Choice of each `ExecNode` for fine control of execution order\n* Per `ExecNode` choice of parallelization (i.e. An `ExecNode` is allowed to run in parallel with other `ExecNode`s or not)\n* and more!\n\n### Documentation\nYou can find the documentation here: [Tawazi](https://mindee.github.io/tawazi/).\n\nIn [this blog](https://blog.mindee.com/directed-acyclic-graph-dag-scheduler-library/) we also talk about the purpose of using `Tawazi` in more detail.\n\n**Note**: The library is still at an [advanced state of development](#future-developments). Breaking changes might happen on the minor version (v0.Minor.Patch). Please pin [Tawazi](https://pypi.org/project/tawazi/) to the __Minor Version__. Your contributions are highly welcomed.\n\n## Name explanation\nThe libraries name is inspired from the arabic word تَوَازٍ which means parallel.\n\n## Building the doc\nOnly the latest version's documentation is hosted. \n\nIf you want to check the documentation of a previous version please checkout the corresponding release, install the required packages and run: `mkdocs serve`\n\n\n## Developer mode\n```sh\npip install --upgrade pip\npip install flit wheel\n\ncd tawazi\nflit install -s --deps develop\n```\n\n## Future developments\n__This library is still in development. Breaking changes are expected.__\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmindee%2Ftawazi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmindee%2Ftawazi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmindee%2Ftawazi/lists"}