{"id":13483648,"url":"https://github.com/pydoit/doit","last_synced_at":"2025-05-14T01:04:34.353Z","repository":{"id":14145457,"uuid":"16851127","full_name":"pydoit/doit","owner":"pydoit","description":"CLI task management \u0026 automation tool","archived":false,"fork":false,"pushed_at":"2024-07-04T15:59:14.000Z","size":8097,"stargazers_count":1929,"open_issues_count":99,"forks_count":181,"subscribers_count":49,"default_branch":"master","last_synced_at":"2025-04-02T01:09:48.135Z","etag":null,"topics":["build-automation","build-system","build-tool","cli","data-pipeline","data-science","hacktoberfest","python","task-runner","workflow","workflow-automation","workflow-management"],"latest_commit_sha":null,"homepage":"http://pydoit.org","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pydoit.png","metadata":{"files":{"readme":"README.rst","changelog":"CHANGES","contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"schettino72","patreon":null,"open_collective":"doit","ko_fi":null,"tidelift":null,"custom":null}},"created_at":"2014-02-14T22:21:23.000Z","updated_at":"2025-03-29T22:04:45.000Z","dependencies_parsed_at":"2023-01-16T20:01:03.030Z","dependency_job_id":"130a234d-9fa7-496d-8815-4c8047d40769","html_url":"https://github.com/pydoit/doit","commit_stats":{"total_commits":1399,"total_committers":68,"mean_commits":"20.573529411764707","dds":0.1536812008577555,"last_synced_commit":"00c136f5dfe7e9039d0fed6dddd6d45c84c307b4"},"previous_names":[],"tags_count":47,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pydoit%2Fdoit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pydoit%2Fdoit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pydoit%2Fdoit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pydoit%2Fdoit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pydoit","download_url":"https://codeload.github.com/pydoit/doit/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247962460,"owners_count":21024862,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["build-automation","build-system","build-tool","cli","data-pipeline","data-science","hacktoberfest","python","task-runner","workflow","workflow-automation","workflow-management"],"created_at":"2024-07-31T17:01:13.734Z","updated_at":"2025-04-09T02:11:12.041Z","avatar_url":"https://github.com/pydoit.png","language":"Python","readme":"================\nREADME\n================\n\n.. display some badges\n\n.. image:: https://img.shields.io/pypi/v/doit.svg\n    :target: https://pypi.python.org/pypi/doit\n\n.. image:: https://github.com/pydoit/doit/actions/workflows/ci.yml/badge.svg?branch=master\n    :target: https://github.com/pydoit/doit/actions/workflows/ci.yml?query=branch%3Amaster\n\n.. image:: https://codecov.io/gh/pydoit/doit/branch/master/graph/badge.svg?token=wxKa1h11zn\n    :target: https://codecov.io/gh/pydoit/doit\n\n.. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.4892136.svg\n   :target: https://doi.org/10.5281/zenodo.4892136\n\n\nFinancial contributions on `Open Collective \u003chttps://opencollective.com/doit/tiers\u003e`_\n\n\ndoit - automation tool\n======================\n\n*doit* comes from the idea of bringing the power of build-tools to execute any\nkind of task\n\n*doit* can be uses as a simple **Task Runner** allowing you to easily define ad hoc\ntasks, helping you to organize all your project related tasks in an unified\neasy-to-use \u0026 discoverable way.\n\n*doit* scales-up with an efficient execution model like a **build-tool**.\n*doit* creates a DAG (direct acyclic graph) and is able to cache task results.\nIt ensures that only required tasks will be executed and in the correct order\n(aka incremental-builds).\n\nThe *up-to-date* check to cache task results is not restricted to looking for\nfile modification on dependencies.  Nor it requires \"target\" files.\nSo it is also suitable to handle **workflows** not handled by traditional build-tools.\n\nTasks' dependencies and creation can be done dynamically during it is execution\nmaking it suitable to drive complex workflows and **pipelines**.\n\n*doit* is build with a plugin architecture allowing extensible commands, custom\noutput, storage backend and \"task loader\". It also provides an API allowing\nusers to create new applications/tools leveraging *doit* functionality like a framework.\n\n*doit* is a mature project being actively developed for more than 10 years.\nIt includes several extras like: parallel execution, auto execution (watch for file\nchanges), shell tab-completion, DAG visualisation, IPython integration, and more.\n\n\n\nSample Code\n===========\n\nDefine functions returning python dict with task's meta-data.\n\nSnippet from `tutorial \u003chttp://pydoit.org/tutorial-1.html\u003e`_:\n\n.. code:: python\n\n  def task_imports():\n      \"\"\"find imports from a python module\"\"\"\n      for name, module in PKG_MODULES.by_name.items():\n          yield {\n              'name': name,\n              'file_dep': [module.path],\n              'actions': [(get_imports, (PKG_MODULES, module.path))],\n          }\n\n  def task_dot():\n      \"\"\"generate a graphviz's dot graph from module imports\"\"\"\n      return {\n          'targets': ['requests.dot'],\n          'actions': [module_to_dot],\n          'getargs': {'imports': ('imports', 'modules')},\n          'clean': True,\n      }\n\n  def task_draw():\n      \"\"\"generate image from a dot file\"\"\"\n      return {\n          'file_dep': ['requests.dot'],\n          'targets': ['requests.png'],\n          'actions': ['dot -Tpng %(dependencies)s -o %(targets)s'],\n          'clean': True,\n      }\n\n\nRun from terminal::\n\n  $ doit list\n  dot       generate a graphviz's dot graph from module imports\n  draw      generate image from a dot file\n  imports   find imports from a python module\n  $ doit\n  .  imports:requests.models\n  .  imports:requests.__init__\n  .  imports:requests.help\n  (...)\n  .  dot\n  .  draw\n\n\nProject Details\n===============\n\n - Website \u0026 docs - http://pydoit.org\n - Project management on github - https://github.com/pydoit/doit\n - Discussion group - https://groups.google.com/forum/#!forum/python-doit\n - News/twitter - https://twitter.com/pydoit\n - Plugins, extensions and projects based on doit - https://github.com/pydoit/doit/wiki/powered-by-doit\n\nlicense\n=======\n\nThe MIT License\nCopyright (c) 2008-2021 Eduardo Naufel Schettino\n\nsee LICENSE file\n\n\ndevelopers / contributors\n==========================\n\nsee AUTHORS file\n\n\ninstall\n=======\n\n*doit* is tested on python 3.6 to 3.10.\n\nThe last version supporting python 2 is version 0.29.\n\n.. code:: bash\n\n $ pip install doit\n\n\ndependencies\n=============\n\n- cloudpickle\n- pyinotify (linux)\n- macfsevents (mac)\n\nTools required for development:\n\n- git * VCS\n- py.test * unit-tests\n- coverage * code coverage\n- sphinx * doc tool\n- pyflakes * syntax checker\n- doit-py * helper to run dev tasks\n\n\ndevelopment setup\n==================\n\nThe best way to setup an environment to develop *doit* itself is to\ncreate a virtualenv...\n\n.. code:: bash\n\n  doit$ virtualenv dev\n  doit$ source dev/bin/activate\n\ninstall ``doit`` as \"editable\", and add development dependencies\nfrom `dev_requirements.txt`:\n\n.. code:: bash\n\n  (dev) doit$ pip install --editable .\n  (dev) doit$ pip install --requirement dev_requirements.txt\n\n\n\ntests\n=======\n\nUse py.test - http://pytest.org\n\n.. code:: bash\n\n  $ py.test\n\n\n\ndocumentation\n=============\n\n``doc`` folder contains ReST documentation based on Sphinx.\n\n.. code:: bash\n\n doc$ make html\n\nThey are the base for creating the website. The only difference is\nthat the website includes analytics tracking.\nTo create it (after installing *doit*):\n\n.. code:: bash\n\n $ doit website\n\n\n\nspell checking\n--------------\n\nAll documentation is spell checked using the task `spell`:\n\n.. code:: bash\n\n  $ doit spell\n\nIt is a bit annoying that code snippets and names always fails the check,\nthese words must be added into the file `doc/dictionary.txt`.\n\nThe spell checker currently uses `hunspell`, to install it on debian based\nsystems install the hunspell package: `apt-get install hunspell`.\n\n\nprofiling\n---------\n\n.. code:: bash\n\n  python -m cProfile -o output.pstats `which doit` list\n\n  gprof2dot -f pstats output.pstats | dot -Tpng -o output.png\n\n\nreleases\n========\n\nUpdate version number at:\n\n- doit/version.py\n- setup.py\n- doc/conf.py\n- doc/index.html\n\n.. code:: bash\n\n   python setup.py sdist\n   python setup.py bdist_wheel\n   twine upload dist/doit-X.Y.Z.tar.gz\n   twine upload dist/doit-X.Y.Z-py3-none-any.whl\n\nRemember to push GIT tags::\n\n  git push --tags\n\n\n\ncontributing\n==============\n\nOn github create pull requests using a named feature branch.\n\nFinancial contribution to support maintenance welcome.\n\n.. image:: https://opencollective.com/doit/tiers/backers.svg?avatarHeight=50\n    :target: https://opencollective.com/doit/tiers\n","funding_links":["https://github.com/sponsors/schettino72","https://opencollective.com/doit","https://opencollective.com/doit/tiers"],"categories":["\u003ca id=\"tag-dev\" href=\"#tag-dev\"\u003eDev\u003c/a\u003e","Build Tools","Python"],"sub_categories":["\u003ca id=\"tag-dev.build\" href=\"#tag-dev.build\"\u003eBuild\u003c/a\u003e"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpydoit%2Fdoit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpydoit%2Fdoit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpydoit%2Fdoit/lists"}