{"id":17289062,"url":"https://github.com/chezou/tdworkflow","last_synced_at":"2025-07-13T15:38:30.575Z","repository":{"id":35690996,"uuid":"218262232","full_name":"chezou/tdworkflow","owner":"chezou","description":"Unofficial Treasure Workflow Client","archived":false,"fork":false,"pushed_at":"2024-04-20T19:19:52.000Z","size":102,"stargazers_count":8,"open_issues_count":0,"forks_count":5,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-06-26T16:08:51.214Z","etag":null,"topics":["digdag","python","workflow"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chezou.png","metadata":{"files":{"readme":"README.rst","changelog":"CHANGES.rst","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-10-29T10:34:33.000Z","updated_at":"2025-06-22T18:02:44.000Z","dependencies_parsed_at":"2024-01-14T17:45:09.599Z","dependency_job_id":"d3011e11-5845-47ff-b7b1-19abc1e1a282","html_url":"https://github.com/chezou/tdworkflow","commit_stats":{"total_commits":75,"total_committers":4,"mean_commits":18.75,"dds":"0.45333333333333337","last_synced_commit":"8a6c9867b928af788faede91236325173f8c3b34"},"previous_names":[],"tags_count":19,"template":false,"template_full_name":null,"purl":"pkg:github/chezou/tdworkflow","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chezou%2Ftdworkflow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chezou%2Ftdworkflow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chezou%2Ftdworkflow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chezou%2Ftdworkflow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chezou","download_url":"https://codeload.github.com/chezou/tdworkflow/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chezou%2Ftdworkflow/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265165210,"owners_count":23721327,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["digdag","python","workflow"],"created_at":"2024-10-15T10:33:24.577Z","updated_at":"2025-07-13T15:38:30.531Z","avatar_url":"https://github.com/chezou.png","language":"Python","readme":"tdwokflow\n=========\n\nUnofficial Treasure Workflow API client.\n\nInstallation\n------------\n\n.. code-block:: shell\n\n   pip install tdworkflow\n\nIf you want to use development version, run as follows:\n\n.. code-block:: shell\n\n   pip install git+https://github.com/chezou/tdworkflow.git\n\nUsage\n-----\n\n.. code-block:: python\n\n   import os\n\n   from tdworkflow.client import Client\n\n   apikey = os.getenv(\"TD_API_KEY\")\n   client = Client(site=\"us\", apikey=apikey)\n   # Or, write endpoint explicitly\n   # client = Client(endpoint=\"api-workflow.treasuredata.com\", apikey=apikey)\n\n   projects = client.projects(\"pandas-df\")\n\n   secrets = {\"td.apikey\": apikey, \"td.apiserver\": \"https://api.treasuredata.com\", \"test\": \"secret-foo\"}\n\n   client.set_secrets(projects[0], secrets)\n\n   client.secrets(projects[0])\n   # ['td.apikey', 'td.apiserver', \"test\"]\n   client.delete_secrets(projects[0], [\"test\", \"td.apiserver\"])\n\nUpload Project from GitHub\n^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nBefore executing the example code, you have to install git-python\n\n.. code-block:: shell\n\n   pip install gitpython\n\nClone example repository with git-python and upload a digdag project.\n\n.. code-block:: python\n\n   import tempfile\n   import os\n   import shutil\n\n   import tdworkflow\n\n   from git import Git\n\n   # Download example GitHub repositoory\n\n   tempdir = tempfile.gettempdir()\n\n   git_repo = \"https://github.com/treasure-data/treasure-boxes/\"\n\n   shutil.rmtree(os.path.join(tempdir, \"treasure-boxes\"))\n\n   try:\n       Git(tempdir).clone(git_repo)\n       print(\"Clone repository succeeded\")\n   except Exception:\n       print(\"Repository clone failed\")\n       raise\n\n   # Upload specific Workflow project\n\n   apikey = os.getenv(\"TD_API_KEY\")\n   site = \"us\"\n\n   target_box = os.path.join(\"integration-box\", \"python\")\n   target_path = os.path.join(tempdir, \"treasure-boxes\", target_box)\n\n   client = tdworkflow.client.Client(site=site, apikey=apikey)\n   project = client.create_project(\"my-project\", target_path)\n\nIf you want to open Treasure Workflow console on your browser, you can get the workflow URL as the following:\n\n.. code-block:: python\n\n   CONSOLE_URL = {\n       \"us\": \"https://console.treasuredata.com/app/workflows\",\n       \"eu01\": \"https://console.eu01.treasuredata.com/app/workflows\",\n       \"jp\": \"https://console.treasuredata.co.jp/app/workflows\",\n   }\n\n   workflows = client.project_workflows(project)\n   workflows = list(filter(lambda w: w.name != \"test\", workflows))\n   if workflows:\n       print(f\"Project created! Open {CONSOLE_URL[site]}/{workflows[0].id}/info on your browser and click 'New Run' button.\")\n   else:\n       print(\"Project creation failed.\")\n\nStart workflow session\n^^^^^^^^^^^^^^^^^^^^^^\n\nYou can start a workflow session by using ``Client.start_attempt``.\n\n.. code-block:: python\n\n   attempt = client.start_attempt(workflows[0])\n\n   # Wait attempt until finish. This may require few minutes.\n   attempt = client.wait_attempt(attempt)\n\n\nConnect to open source digdag\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nSince Treasure Workflow is hosted digdag, tdworkflow is compatible with open source digdag.\n\n.. note::\n   Open source digdag API may be different with Treasure Workflow API so that tdworkflow might not work with some API of opensource digdag.\n\nHere is the example code to connect local digdag server.\n\n.. code-block:: python\n\n    \u003e\u003e\u003e import tdworkflow\n    \u003e\u003e\u003e import requests\n    \u003e\u003e\u003e session = requests.Session()\n    \u003e\u003e\u003e client = tdworkflow.client.Client(\n    ... endpoint=\"localhost:65432\", apikey=\"\", _session=session, scheme=\"http\")\n    \u003e\u003e\u003e client.projects()\n    [Project(id=1, name='python-tdworkflow', revision='134fe2f9-ded3-4e7c-af8e-8a82d55d688b', archiveType='db', archiveMd5='5Lc6F6m3DtmBN4DA5MzK8A==', createdAt='2019-11-01T13:03:26Z', deletedAt=None, updatedAt='2019-11-01T13:03:26Z')]\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchezou%2Ftdworkflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchezou%2Ftdworkflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchezou%2Ftdworkflow/lists"}