{"id":43601977,"url":"https://github.com/xianghuzhao/paradag","last_synced_at":"2026-02-04T06:05:50.228Z","repository":{"id":57450679,"uuid":"97564106","full_name":"xianghuzhao/paradag","owner":"xianghuzhao","description":"A robust DAG implementation for parallel execution","archived":false,"fork":false,"pushed_at":"2024-02-17T07:06:17.000Z","size":52,"stargazers_count":70,"open_issues_count":1,"forks_count":9,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-09-01T13:40:36.771Z","etag":null,"topics":["dag","directed-acyclic-graph","multithreading","parallel-programming"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xianghuzhao.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-07-18T06:57:41.000Z","updated_at":"2025-08-21T21:55:38.000Z","dependencies_parsed_at":"2024-06-21T14:04:41.798Z","dependency_job_id":"48c1db2b-a793-451e-8986-eda7acd50389","html_url":"https://github.com/xianghuzhao/paradag","commit_stats":{"total_commits":46,"total_committers":2,"mean_commits":23.0,"dds":"0.021739130434782594","last_synced_commit":"4d0810846ce94eed4cf70edda4f53e81d2eb47d3"},"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/xianghuzhao/paradag","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xianghuzhao%2Fparadag","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xianghuzhao%2Fparadag/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xianghuzhao%2Fparadag/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xianghuzhao%2Fparadag/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xianghuzhao","download_url":"https://codeload.github.com/xianghuzhao/paradag/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xianghuzhao%2Fparadag/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29072515,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-04T03:31:03.593Z","status":"ssl_error","status_checked_at":"2026-02-04T03:29:50.742Z","response_time":62,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dag","directed-acyclic-graph","multithreading","parallel-programming"],"created_at":"2026-02-04T06:05:49.139Z","updated_at":"2026-02-04T06:05:50.221Z","avatar_url":"https://github.com/xianghuzhao.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# paradag\n\n[![PyPI](https://badge.fury.io/py/paradag.svg)](https://pypi.org/project/paradag/)\n[![Travis CI Status](https://travis-ci.org/xianghuzhao/paradag.svg?branch=master)](https://travis-ci.org/xianghuzhao/paradag)\n[![Code Climate](https://codeclimate.com/github/xianghuzhao/paradag/badges/gpa.svg)](https://codeclimate.com/github/xianghuzhao/paradag)\n\n`paradag` a robust DAG package for easy parallel execution.\n`paradag` is implemented in pure python and totally independent of any\nthird party packages.\n\n[Directed acyclic graph (DAG)](https://en.wikipedia.org/wiki/Directed_acyclic_graph)\nis commonly used as a dependency graph. It could be used to describe the\ndependencies of tasks.\nThe order of task executions must comply with the dependencies,\nwhere tasks with direct or indirect path must run in sequence,\nand tasks without any connection could run in parallel.\n\n\n## Installation\n\n```shell\n$ pip install paradag\n```\n\n\n## Create a DAG\n\nBefore running tasks, first create a DAG,\nwith each vertex representing a task.\nThe vertex of DAG instance could be any\n[hashable object](https://docs.python.org/3/glossary.html#term-hashable),\nlike integer, string, tuple of hashable objects, instance of\nuser-defined class, etc.\n\n```python\nfrom paradag import DAG\n\nclass Vtx(object):\n    def __init__(self, v):\n        self.__value = v\n\nvtx = Vtx(999)\n\ndag = DAG()\ndag.add_vertex(123, 'abcde', 'xyz', ('a', 'b', 3), vtx)\n\ndag.add_edge(123, 'abcde')                  # 123 -\u003e 'abcde'\ndag.add_edge('abcde', ('a', 'b', 3), vtx)   # 'abcde' -\u003e ('a', 'b', 3), 'abcde' -\u003e vtx\n```\n\n`add_edge` accepts one starting vertex and one or more ending vertices.\nPlease pay attention not to make a cycle with `add_edge`,\nwhich will raise a `DAGCycleError`.\n\nThe common DAG properties are accessible:\n\n```python\nprint(dag.vertex_size())\nprint(dag.edge_size())\n\nprint(dag.successors('abcde'))\nprint(dag.predecessors(vtx))\n\nprint(dag.all_starts())\nprint(dag.all_terminals())\n```\n\n\n## Run tasks in sequence\n\nWrite your executor and optionally a selector.\nThe executor handles the real execution for each vertex.\n\n```python\nfrom paradag import dag_run\nfrom paradag import SequentialProcessor\n\nclass CustomExecutor:\n    def param(self, vertex):\n        return vertex\n\n    def execute(self, param):\n        print('Executing:', param)\n\nprint(dag_run(dag, processor=SequentialProcessor(), executor=CustomExecutor()))\n```\n\n`dag_run` is the core function for task scheduling.\n\n\n## Run tasks in parallel\n\nRun tasks in parallel is quite similar, while only change the processor\nto `MultiThreadProcessor`.\n\n```python\nfrom paradag import MultiThreadProcessor\n\ndag_run(dag, processor=MultiThreadProcessor(), executor=CustomExecutor())\n```\n\nThe default selector `FullSelector` will try to find as many tasks\nas possible which could run in parallel.\nThis could be adjusted with custom selector.\nThe following selector will only allow at most 4 tasks running in parallel.\n\n```python\nclass CustomSelector(object):\n    def select(self, running, idle):\n        task_number = max(0, 4-len(running))\n        return list(idle)[:task_number]\n\ndag_run(dag, processor=MultiThreadProcessor(), selector=CustomSelector(), executor=CustomExecutor())\n```\n\nOnce you are using `MultiThreadProcessor`, great attentions must be\npaid that `execute` of executor could run in parallel. Try not to modify\nany variables outside the `execute` function, and all parameters should\nbe passed by the `param` argument. Also make sure that the return values\ngenerated from `param` function are independent.\n\n\n## Get task running status\n\nThe executor could also implement the optional methods which could get\nthe task running status.\n\n```python\nclass CustomExecutor:\n    def param(self, vertex):\n        return vertex\n\n    def execute(self, param):\n        print('Executing:', param)\n\n    def report_start(self, vertices):\n        print('Start to run:', vertices)\n\n    def report_running(self, vertices):\n        print('Current running:', vertices)\n\n    def report_finish(self, vertices_result):\n        for vertex, result in vertices_result:\n            print('Finished running {0} with result: {1}'.format(vertex, result))\n\ndag_run(dag, processor=MultiThreadProcessor(), executor=CustomExecutor())\n```\n\n\n## Deliver result to descendants\n\nIn case the result for one task should be used for its descendants,\n`deliver` method could be implemented in executor.\n\n```python\nclass CustomExecutor:\n    def __init__(self):\n        self.__level = {}\n\n    def param(self, vertex):\n        return self.__level.get(vertex, 0)\n\n    def execute(self, param):\n        return param + 1\n\n    def report_finish(self, vertices_result):\n        for vertex, result in vertices_result:\n            print('Vertex {0} finished, level: {1}'.format(vertex, result))\n\n    def deliver(self, vertex, result):\n        self.__level[vertex] = result\n```\n\nThe result from parent will be delivered to the vertex before execution.\n\n\n## Topological sorting\n\n[Topological sorting](https://en.wikipedia.org/wiki/Topological_sorting)\ncould also be done by `paradag.dag_run` function.\nThe return value of `dag_run` could be considered as\nthe result of topological sorting.\n\nA simple topological sorting without any execution:\n\n```python\nfrom paradag import SingleSelector, RandomSelector, ShuffleSelector\n\ndag = DAG()\ndag.add_vertex(1, 2, 3, 4, 5)\ndag.add_edge(1, 4)\ndag.add_edge(4, 2, 5)\n\nprint(dag_run(dag))\nprint(dag_run(dag, selector=SingleSelector()))\nprint(dag_run(dag, selector=RandomSelector()))\nprint(dag_run(dag, selector=ShuffleSelector()))\n```\n\nThe solution for topological sorting is not necessarily unique,\nand the final orders may vary with different selectors.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxianghuzhao%2Fparadag","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxianghuzhao%2Fparadag","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxianghuzhao%2Fparadag/lists"}