{"id":13534485,"url":"https://github.com/d6t/d6tflow","last_synced_at":"2026-01-14T10:01:48.760Z","repository":{"id":52195503,"uuid":"168899027","full_name":"d6t/d6tflow","owner":"d6t","description":"Python library for building highly effective data science workflows","archived":false,"fork":false,"pushed_at":"2023-07-20T18:28:08.000Z","size":284,"stargazers_count":949,"open_issues_count":11,"forks_count":74,"subscribers_count":23,"default_branch":"master","last_synced_at":"2025-09-24T09:46:22.962Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://d6tflow.readthedocs.io/en/latest/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/d6t.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2019-02-03T01:51:22.000Z","updated_at":"2025-07-04T14:53:30.000Z","dependencies_parsed_at":"2022-09-09T12:13:24.122Z","dependency_job_id":"0ca64ff9-16ec-423f-b709-715a70b1c1aa","html_url":"https://github.com/d6t/d6tflow","commit_stats":{"total_commits":206,"total_committers":14,"mean_commits":"14.714285714285714","dds":0.6601941747572815,"last_synced_commit":"4d583b6d92876d26aa6f31d520afe8e79fc2d9a5"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/d6t/d6tflow","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/d6t%2Fd6tflow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/d6t%2Fd6tflow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/d6t%2Fd6tflow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/d6t%2Fd6tflow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/d6t","download_url":"https://codeload.github.com/d6t/d6tflow/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/d6t%2Fd6tflow/sbom","scorecard":{"id":315908,"data":{"date":"2025-08-11","repo":{"name":"github.com/d6t/d6tflow","commit":"9ee52f79c6deb6de061a41b72757a119b0eb5b79"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":1.7,"checks":[{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Code-Review","score":0,"reason":"Found 1/29 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 2 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Vulnerabilities","score":0,"reason":"422 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: PYSEC-2021-387 / GHSA-j8fq-86c5-5v2r","Warn: Project is vulnerable to: PYSEC-2024-159 / GHSA-8qch-vj6m-2694","Warn: Project is vulnerable to: PYSEC-2018-11 / GHSA-p69g-f978-xxv9","Warn: Project is vulnerable to: PYSEC-2020-73","Warn: Project is vulnerable to: GHSA-23hm-7w47-xw72","Warn: Project is vulnerable to: GHSA-2475-53vw-vp25","Warn: Project is vulnerable to: GHSA-247x-2f9f-5wp7","Warn: Project is vulnerable to: GHSA-24x4-6qmh-88qg","Warn: Project is vulnerable to: PYSEC-2021-227 / GHSA-24x6-8c7m-hv3f","Warn: Project is vulnerable to: PYSEC-2021-223 / GHSA-26j7-6w8w-7922","Warn: Project is vulnerable to: GHSA-278g-rq84-9hmg","Warn: Project is vulnerable to: GHSA-27j5-4p9v-pp67","Warn: Project is vulnerable to: GHSA-27qf-jwm8-g7f3","Warn: Project is vulnerable to: GHSA-27rc-728f-x5w2","Warn: Project is vulnerable to: PYSEC-2021-160 / GHSA-2cpx-427x-q2c6","Warn: Project is vulnerable to: PYSEC-2021-173 / GHSA-2gfx-95x2-5v3x","Warn: Project is vulnerable to: PYSEC-2021-391 / GHSA-2p25-55c9-h58q","Warn: Project is vulnerable to: GHSA-2p9q-h29j-3f5v","Warn: Project is vulnerable to: GHSA-2r2f-g8mw-9gvr","Warn: Project is vulnerable to: GHSA-2r8p-fg3c-wcj4","Warn: Project is vulnerable to: GHSA-2vv3-56qg-g2cf","Warn: Project is vulnerable to: GHSA-2wmv-37vq-52g5","Warn: Project is vulnerable to: PYSEC-2021-249 / GHSA-2xgj-xhgf-ggjv","Warn: Project is vulnerable to: GHSA-34f9-hjfq-rr8j","Warn: Project is vulnerable to: GHSA-368v-7v32-52fx","Warn: Project is vulnerable to: PYSEC-2021-199 / GHSA-36vm-xw34-x4pj","Warn: Project is vulnerable to: PYSEC-2021-418 / GHSA-374m-jm66-3vj8","Warn: Project is vulnerable to: GHSA-37jf-mjv6-xfqw","Warn: Project is vulnerable to: PYSEC-2021-170 / GHSA-393f-2jr3-cp69","Warn: Project is vulnerable to: GHSA-397c-5g2j-qxpv","Warn: Project is vulnerable to: PYSEC-2021-408 / GHSA-3ff2-r28g-w7h9","Warn: Project is vulnerable to: PYSEC-2021-206 / GHSA-3h8m-483j-7xxm","Warn: Project is vulnerable to: GHSA-3hxh-8cp2-g4hg","Warn: Project is vulnerable to: GHSA-3mw4-6rj6-74g5","Warn: Project is vulnerable to: PYSEC-2021-231 / GHSA-3qgw-p4fm-x7gf","Warn: Project is vulnerable to: PYSEC-2021-168 / GHSA-3qxp-qjq7-w4hf","Warn: Project is vulnerable to: PYSEC-2021-420 / GHSA-3rcw-9p9x-582v","Warn: Project is vulnerable to: PYSEC-2021-226 / GHSA-3w67-q784-6w7c","Warn: Project is vulnerable to: GHSA-428x-9xc2-m8mj","Warn: Project is vulnerable to: GHSA-43jf-985q-588j","Warn: Project is vulnerable to: GHSA-43q8-3fv7-pr5x","Warn: Project is vulnerable to: GHSA-44qp-9wwf-734r","Warn: Project is vulnerable to: PYSEC-2021-150 / GHSA-452g-f7fp-9jf7","Warn: Project is vulnerable to: GHSA-49rq-hwc3-x77w","Warn: Project is vulnerable to: PYSEC-2021-398 / GHSA-49rx-x2rw-pc6f","Warn: Project is vulnerable to: GHSA-4c4g-crqm-xrxw","Warn: Project is vulnerable to: PYSEC-2021-411 / GHSA-4f99-p9c2-3j8x","Warn: Project is vulnerable to: PYSEC-2021-184 / GHSA-4fg4-p75j-w5xj","Warn: Project is vulnerable to: PYSEC-2020-113 / GHSA-4g9f-63rx-5cw4","Warn: Project is vulnerable to: PYSEC-2021-179 / GHSA-4hrh-9vmp-2jgg","Warn: Project is vulnerable to: GHSA-4hvf-hxvg-f67v","Warn: Project is vulnerable to: PYSEC-2021-253 / GHSA-4hvv-7x94-7vq8","Warn: Project is vulnerable to: GHSA-4j82-5ccr-4r8v","Warn: Project is vulnerable to: PYSEC-2021-205 / GHSA-4p4p-www8-8fv9","Warn: Project is vulnerable to: GHSA-4pc4-m9mj-v2r9","Warn: Project is vulnerable to: GHSA-4v5p-v5h9-6xjx","Warn: Project is vulnerable to: PYSEC-2021-163 / GHSA-4vf2-4xcg-65cx","Warn: Project is vulnerable to: PYSEC-2021-233 / GHSA-4vrf-ff7v-hpgr","Warn: Project is vulnerable to: GHSA-4w68-4x85-mjj9","Warn: Project is vulnerable to: GHSA-4xfp-4pfp-89wg","Warn: Project is vulnerable to: PYSEC-2021-207 / GHSA-545v-42p7-98fq","Warn: Project is vulnerable to: GHSA-54ch-gjq5-4976","Warn: Project is vulnerable to: GHSA-54pp-c6pp-7fpx","Warn: Project is vulnerable to: GHSA-558h-mq8x-7q9g","Warn: Project is vulnerable to: PYSEC-2021-400 / GHSA-57wx-m983-2f88","Warn: Project is vulnerable to: GHSA-5889-7v45-q28m","Warn: Project is vulnerable to: PYSEC-2021-196 / GHSA-59q2-x2qc-4c97","Warn: Project is vulnerable to: PYSEC-2021-409 / GHSA-5crj-c72x-m7gq","Warn: Project is vulnerable to: GHSA-5f2r-qp73-37mr","Warn: Project is vulnerable to: PYSEC-2021-209 / GHSA-5gqf-456p-4836","Warn: Project is vulnerable to: GHSA-5hj3-vjjf-f5m7","Warn: Project is vulnerable to: PYSEC-2021-392 / GHSA-5hx2-qx8j-qjqm","Warn: Project is vulnerable to: GHSA-5qw5-89mw-wcg2","Warn: Project is vulnerable to: GHSA-5v77-j66x-4c4g","Warn: Project is vulnerable to: GHSA-5w96-866f-6rm8","Warn: Project is vulnerable to: GHSA-5wpj-c6f7-24x8","Warn: Project is vulnerable to: GHSA-5xwc-mrhx-5g3m","Warn: Project is vulnerable to: GHSA-627q-g293-49q7","Warn: Project is vulnerable to: PYSEC-2021-155 / GHSA-62gx-355r-9fhg","Warn: Project is vulnerable to: PYSEC-2020-118 / GHSA-63xm-rx5p-xvqr","Warn: Project is vulnerable to: GHSA-6445-fm66-fvq2","Warn: Project is vulnerable to: GHSA-647v-r7qq-24fh","Warn: Project is vulnerable to: GHSA-64jg-wjww-7c5w","Warn: Project is vulnerable to: GHSA-66vq-54fq-6jvv","Warn: Project is vulnerable to: GHSA-67pf-62xr-q35m","Warn: Project is vulnerable to: GHSA-689c-r7h2-fv9v","Warn: Project is vulnerable to: GHSA-68v3-g9cm-rmm6","Warn: Project is vulnerable to: PYSEC-2021-165 / GHSA-6f84-42vf-ppwp","Warn: Project is vulnerable to: PYSEC-2021-215 / GHSA-6f89-8j54-29xf","Warn: Project is vulnerable to: GHSA-6gmv-pjp9-p8w8","Warn: Project is vulnerable to: GHSA-6gv8-p3vj-pxvr","Warn: Project is vulnerable to: GHSA-6hg6-5c2q-7rcr","Warn: Project is vulnerable to: PYSEC-2021-401 / GHSA-6hpv-v2rx-c5g6","Warn: Project is vulnerable to: PYSEC-2021-171 / GHSA-6j9c-grc6-5m6g","Warn: Project is vulnerable to: GHSA-6p5r-g9mq-ggh2","Warn: Project is vulnerable to: PYSEC-2021-212 / GHSA-6qgm-fv6v-rfpv","Warn: Project is vulnerable to: GHSA-6wfh-89q8-44jq","Warn: Project is vulnerable to: GHSA-6x99-gv2v-q76v","Warn: Project is vulnerable to: GHSA-75c9-jrh4-79mc","Warn: Project is vulnerable to: PYSEC-2021-201 / GHSA-75f6-78jr-4656","Warn: Project is vulnerable to: GHSA-762h-vpvw-3rcx","Warn: Project is vulnerable to: PYSEC-2021-156 / GHSA-772j-h9xw-ffp5","Warn: Project is vulnerable to: PYSEC-2021-154 / GHSA-772p-x54p-hjrv","Warn: Project is vulnerable to: GHSA-77gp-3h4r-6428","Warn: Project is vulnerable to: PYSEC-2021-397 / GHSA-786j-5qwq-r36x","Warn: Project is vulnerable to: PYSEC-2021-216 / GHSA-79fv-9865-4qcv","Warn: Project is vulnerable to: GHSA-79h2-q768-fpxr","Warn: Project is vulnerable to: PYSEC-2021-213 / GHSA-7cqx-92hp-x6wh","Warn: Project is vulnerable to: GHSA-7fvx-3jfc-2cpc","Warn: Project is vulnerable to: GHSA-7ghq-fvr3-pj2x","Warn: Project is vulnerable to: GHSA-7j3m-8g3c-9qqq","Warn: Project is vulnerable to: GHSA-7jvm-xxmr-v5cw","Warn: Project is vulnerable to: PYSEC-2021-396 / GHSA-7pxj-m4jf-r6h2","Warn: Project is vulnerable to: PYSEC-2021-417 / GHSA-7r94-xv9v-63jw","Warn: Project is vulnerable to: PYSEC-2021-399 / GHSA-7v94-64hj-m82h","Warn: Project is vulnerable to: GHSA-7x4v-9gxg-9hwj","Warn: Project is vulnerable to: GHSA-7xwj-5r4v-429p","Warn: Project is vulnerable to: GHSA-828c-5j5q-vrjq","Warn: Project is vulnerable to: PYSEC-2021-211 / GHSA-828x-qc2p-wprq","Warn: Project is vulnerable to: GHSA-844w-j86r-4x2j","Warn: Project is vulnerable to: GHSA-84jm-4cf3-9jfm","Warn: Project is vulnerable to: PYSEC-2021-153 / GHSA-84mw-34w6-2q43","Warn: Project is vulnerable to: GHSA-87v6-crgm-2gfj","Warn: Project is vulnerable to: PYSEC-2021-174 / GHSA-8c89-2vwr-chcq","Warn: Project is vulnerable to: GHSA-8cxv-76p7-jxwr","Warn: Project is vulnerable to: GHSA-8fvv-46hw-vpg3","Warn: Project is vulnerable to: PYSEC-2021-197 / GHSA-8gv3-57p6-g35r","Warn: Project is vulnerable to: GHSA-8jj7-5vxc-pg2q","Warn: Project is vulnerable to: PYSEC-2021-251 / GHSA-8pmx-p244-g88h","Warn: Project is vulnerable to: GHSA-8r7c-3cm2-3h8f","Warn: Project is vulnerable to: GHSA-8rcj-c8pj-v3m3","Warn: Project is vulnerable to: PYSEC-2021-241 / GHSA-8rm6-75mf-7r7r","Warn: Project is vulnerable to: GHSA-8w5g-3wcv-9g2j","Warn: Project is vulnerable to: GHSA-8wwm-6264-x792","Warn: Project is vulnerable to: GHSA-93vr-9q9m-pj8p","Warn: Project is vulnerable to: GHSA-94mm-g2mv-8p7r","Warn: Project is vulnerable to: GHSA-95xm-g58g-3p88","Warn: Project is vulnerable to: GHSA-9697-98pf-4rw7","Warn: Project is vulnerable to: PYSEC-2020-258 / GHSA-977j-xj7q-2jr9","Warn: Project is vulnerable to: GHSA-97p7-w86h-vcf9","Warn: Project is vulnerable to: PYSEC-2021-236 / GHSA-97wf-p777-86jq","Warn: Project is vulnerable to: GHSA-98j8-c9q4-r38g","Warn: Project is vulnerable to: GHSA-98p5-x8x4-c9m5","Warn: Project is vulnerable to: GHSA-9942-r22v-78cp","Warn: Project is vulnerable to: GHSA-9c78-vcq7-7vxq","Warn: Project is vulnerable to: PYSEC-2021-238 / GHSA-9c84-4hx6-xmm4","Warn: Project is vulnerable to: GHSA-9c8h-2mv3-49ww","Warn: Project is vulnerable to: GHSA-9c8h-vvrj-w2p8","Warn: Project is vulnerable to: GHSA-9cr2-8pwr-fhfq","Warn: Project is vulnerable to: PYSEC-2021-410 / GHSA-9crf-c6qr-r273","Warn: Project is vulnerable to: GHSA-9fpg-838v-wpv7","Warn: Project is vulnerable to: GHSA-9gwq-6cwj-47h3","Warn: Project is vulnerable to: GHSA-9j4v-pp28-mxv7","Warn: Project is vulnerable to: PYSEC-2020-117 / GHSA-9mqp-7v2h-2382","Warn: Project is vulnerable to: GHSA-9p77-mmrw-69c7","Warn: Project is vulnerable to: GHSA-9px9-73fg-3fqp","Warn: Project is vulnerable to: GHSA-9v8w-xmr4-wgxp","Warn: Project is vulnerable to: PYSEC-2021-210 / GHSA-9vpm-rcf4-9wqw","Warn: Project is vulnerable to: GHSA-9vqj-64pv-w55c","Warn: Project is vulnerable to: GHSA-9w2p-5mgw-p94c","Warn: Project is vulnerable to: PYSEC-2021-220 / GHSA-9xh4-23q4-v6wr","Warn: Project is vulnerable to: PYSEC-2021-219 / GHSA-c45w-2wxr-pp53","Warn: Project is vulnerable to: GHSA-c545-c4f9-rf6v","Warn: Project is vulnerable to: GHSA-c582-c96p-r5cq","Warn: Project is vulnerable to: GHSA-c5x2-p679-95wc","Warn: Project is vulnerable to: GHSA-c6fh-56w7-fvjw","Warn: Project is vulnerable to: GHSA-c94w-c95p-phf8","Warn: Project is vulnerable to: PYSEC-2021-159 / GHSA-c968-pq7h-7fxv","Warn: Project is vulnerable to: PYSEC-2020-140 / GHSA-c9f3-9wfr-wgh7","Warn: Project is vulnerable to: GHSA-c9qf-r67m-p7cg","Warn: Project is vulnerable to: GHSA-cfpj-3q4c-jhvr","Warn: Project is vulnerable to: PYSEC-2021-230 / GHSA-cfx7-2xpc-8w4h","Warn: Project is vulnerable to: GHSA-cg88-rpvp-cjv5","Warn: Project is vulnerable to: GHSA-cgfm-62j4-v4rf","Warn: Project is vulnerable to: GHSA-ch4f-829c-v5pw","Warn: Project is vulnerable to: PYSEC-2021-246 / GHSA-cjc7-49v2-jp64","Warn: Project is vulnerable to: GHSA-cm5x-837x-jf3c","Warn: Project is vulnerable to: PYSEC-2021-414 / GHSA-cpf4-wx82-gxp6","Warn: Project is vulnerable to: PYSEC-2021-842 / GHSA-cq76-mxrc-vchh","Warn: Project is vulnerable to: PYSEC-2021-413 / GHSA-cqv6-3phm-hcwx","Warn: Project is vulnerable to: GHSA-cqvq-fvhr-v6hc","Warn: Project is vulnerable to: PYSEC-2021-240 / GHSA-crch-j389-5f84","Warn: Project is vulnerable to: GHSA-cv2p-32v3-vhwq","Warn: Project is vulnerable to: PYSEC-2020-134 / GHSA-cvpc-8phh-8f45","Warn: Project is vulnerable to: GHSA-cwpm-f78v-7m5c","Warn: Project is vulnerable to: PYSEC-2021-228 / GHSA-cwv3-863g-39vx","Warn: Project is vulnerable to: GHSA-f2vv-v9cg-qhh7","Warn: Project is vulnerable to: GHSA-f2w8-jw48-fr7j","Warn: Project is vulnerable to: GHSA-f49c-87jh-g47q","Warn: Project is vulnerable to: GHSA-f4rr-5m7v-wxcw","Warn: Project is vulnerable to: GHSA-f4w6-h4f5-wx45","Warn: Project is vulnerable to: PYSEC-2021-415 / GHSA-f54p-f6jp-4rhr","Warn: Project is vulnerable to: GHSA-f5cx-5wr3-5qrc","Warn: Project is vulnerable to: GHSA-f637-vh3r-vfh2","Warn: Project is vulnerable to: PYSEC-2021-187 / GHSA-f78g-q7r4-9wcv","Warn: Project is vulnerable to: GHSA-f7r5-q7cx-h668","Warn: Project is vulnerable to: GHSA-f8h4-7rgh-q2gm","Warn: Project is vulnerable to: GHSA-fcwc-p4fc-c5cc","Warn: Project is vulnerable to: GHSA-ffjm-4qwc-7cmf","Warn: Project is vulnerable to: GHSA-fhfc-2q7x-929f","Warn: Project is vulnerable to: GHSA-fpcp-9h7m-ffpx","Warn: Project is vulnerable to: PYSEC-2021-180 / GHSA-fphq-gw9m-ghrv","Warn: Project is vulnerable to: GHSA-fq6p-6334-8gr4","Warn: Project is vulnerable to: GHSA-fq86-3f29-px2c","Warn: Project is vulnerable to: GHSA-fqm2-gh8w-gr68","Warn: Project is vulnerable to: GHSA-fqxc-pvf8-2w9v","Warn: Project is vulnerable to: PYSEC-2021-404 / GHSA-fr77-rrx3-cp7g","Warn: Project is vulnerable to: GHSA-frqp-wp83-qggv","Warn: Project is vulnerable to: GHSA-fv25-wrff-wf86","Warn: Project is vulnerable to: GHSA-fv43-93gv-vm8f","Warn: Project is vulnerable to: GHSA-fx5c-h9f6-rv7c","Warn: Project is vulnerable to: GHSA-fxgc-95xx-grvq","Warn: Project is vulnerable to: PYSEC-2021-193 / GHSA-fxqh-cfjm-fp93","Warn: Project is vulnerable to: GHSA-g25h-jr74-qp5j","Warn: Project is vulnerable to: GHSA-g35r-369w-3fqp","Warn: Project is vulnerable to: GHSA-g468-qj8g-vcjc","Warn: Project is vulnerable to: PYSEC-2021-176 / GHSA-g4h2-gqm3-c9wq","Warn: Project is vulnerable to: PYSEC-2020-128 / GHSA-g7p5-5759-qv46","Warn: Project is vulnerable to: GHSA-g8wg-cjwc-xhhp","Warn: Project is vulnerable to: GHSA-g9fm-r5mm-rf9f","Warn: Project is vulnerable to: GHSA-g9h5-vr8m-x2h4","Warn: Project is vulnerable to: GHSA-gcvh-66ff-4mwm","Warn: Project is vulnerable to: GHSA-gf2j-f278-xh4v","Warn: Project is vulnerable to: GHSA-gf88-j2mg-cc82","Warn: Project is vulnerable to: GHSA-gf97-q72m-7579","Warn: Project is vulnerable to: GHSA-gh6x-4whr-2qv4","Warn: Project is vulnerable to: PYSEC-2021-393 / GHSA-gh8h-7j2j-qv4f","Warn: Project is vulnerable to: GHSA-gjh7-xx4r-x345","Warn: Project is vulnerable to: GHSA-gjqc-q9g6-q2j3","Warn: Project is vulnerable to: GHSA-gq2j-cr96-gvqx","Warn: Project is vulnerable to: PYSEC-2021-244 / GHSA-gv26-jpj9-c8gq","Warn: Project is vulnerable to: PYSEC-2021-198 / GHSA-gvm4-h8j3-rjrq","Warn: Project is vulnerable to: GHSA-gw97-ff7c-9v96","Warn: Project is vulnerable to: GHSA-h246-cgh4-7475","Warn: Project is vulnerable to: GHSA-h2wq-prv9-2f56","Warn: Project is vulnerable to: GHSA-h48f-q7rw-hvr7","Warn: Project is vulnerable to: PYSEC-2021-243 / GHSA-h4pc-gx2w-f2xv","Warn: Project is vulnerable to: GHSA-h5g4-ppwx-48q2","Warn: Project is vulnerable to: GHSA-h5vq-gw2c-pq47","Warn: Project is vulnerable to: PYSEC-2021-405 / GHSA-h67m-xg8f-fxcf","Warn: Project is vulnerable to: PYSEC-2020-125 / GHSA-h6fg-mjxg-hqq4","Warn: Project is vulnerable to: GHSA-h6gw-r52c-724r","Warn: Project is vulnerable to: GHSA-h6jh-7gv5-28vg","Warn: Project is vulnerable to: GHSA-h6q3-vv32-2cq5","Warn: Project is vulnerable to: GHSA-h7ff-cfc9-wmmh","Warn: Project is vulnerable to: PYSEC-2020-253 / GHSA-h98h-8mxr-m8gx","Warn: Project is vulnerable to: PYSEC-2021-190 / GHSA-h9px-9vqg-222h","Warn: Project is vulnerable to: PYSEC-2021-152 / GHSA-hc6c-75p4-hmq4","Warn: Project is vulnerable to: PYSEC-2020-255 / GHSA-hhvc-g5hv-48c6","Warn: Project is vulnerable to: PYSEC-2021-182 / GHSA-hmg3-c7xj-6qwm","Warn: Project is vulnerable to: GHSA-hp4c-x6r7-6555","Warn: Project is vulnerable to: GHSA-hpv4-7p9c-mvfr","Warn: Project is vulnerable to: GHSA-hq7g-wwwp-q46h","Warn: Project is vulnerable to: GHSA-hrg5-737c-2p56","Warn: Project is vulnerable to: GHSA-hwr7-8gxx-fj5p","Warn: Project is vulnerable to: GHSA-hx9q-2mx4-m4pg","Warn: Project is vulnerable to: GHSA-j3mj-fhpq-qqjj","Warn: Project is vulnerable to: GHSA-j43h-pgmg-5hjq","Warn: Project is vulnerable to: PYSEC-2021-169 / GHSA-j47f-4232-hvv8","Warn: Project is vulnerable to: GHSA-j5w9-hmfh-4cr6","Warn: Project is vulnerable to: PYSEC-2021-224 / GHSA-j7rm-8ww4-xx2g","Warn: Project is vulnerable to: PYSEC-2021-394 / GHSA-j86v-p27c-73fm","Warn: Project is vulnerable to: PYSEC-2021-419 / GHSA-j8c8-67vp-6mx7","Warn: Project is vulnerable to: PYSEC-2021-175 / GHSA-j8qc-5fqr-52fp","Warn: Project is vulnerable to: PYSEC-2021-237 / GHSA-j8qh-3xrq-c825","Warn: Project is vulnerable to: PYSEC-2021-242 / GHSA-jf7h-7m85-w2v2","Warn: Project is vulnerable to: PYSEC-2021-166 / GHSA-jfp7-4j67-8r3q","Warn: Project is vulnerable to: PYSEC-2021-189 / GHSA-jhq9-wm9m-cf89","Warn: Project is vulnerable to: GHSA-jjm6-4vf7-cjh4","Warn: Project is vulnerable to: PYSEC-2021-229 / GHSA-jjr8-m8g8-p6wv","Warn: Project is vulnerable to: GHSA-jq6x-99hj-q636","Warn: Project is vulnerable to: GHSA-jqm7-m5q7-3hm5","Warn: Project is vulnerable to: GHSA-jvhc-5hhr-w3v5","Warn: Project is vulnerable to: GHSA-jwf9-w5xm-f437","Warn: Project is vulnerable to: PYSEC-2021-402 / GHSA-m342-ff57-4jcc","Warn: Project is vulnerable to: PYSEC-2021-183 / GHSA-m34j-p8rj-wjxq","Warn: Project is vulnerable to: PYSEC-2021-172 / GHSA-m3f9-w3p3-p669","Warn: Project is vulnerable to: GHSA-m4hf-j54p-p353","Warn: Project is vulnerable to: PYSEC-2021-389 / GHSA-m539-j985-hcr8","Warn: Project is vulnerable to: PYSEC-2020-256 / GHSA-m648-33qf-v3gp","Warn: Project is vulnerable to: GHSA-m6cv-4fmf-66xf","Warn: Project is vulnerable to: GHSA-m6vp-8q9j-whx4","Warn: Project is vulnerable to: GHSA-m7fm-4jfh-jrg6","Warn: Project is vulnerable to: GHSA-mg66-qvc5-rm93","Warn: Project is vulnerable to: GHSA-mgmh-g2v6-mqw5","Warn: Project is vulnerable to: GHSA-mh3m-62v7-68xg","Warn: Project is vulnerable to: PYSEC-2021-254 / GHSA-mmq6-q8r3-48fm","Warn: Project is vulnerable to: PYSEC-2021-247 / GHSA-mq5c-prh3-3f3h","Warn: Project is vulnerable to: PYSEC-2021-195 / GHSA-mqh2-9wrp-vx84","Warn: Project is vulnerable to: GHSA-mv77-9g28-cwg3","Warn: Project is vulnerable to: PYSEC-2021-222 / GHSA-mv78-g7wq-mhp4","Warn: Project is vulnerable to: GHSA-mv8m-8x97-937q","Warn: Project is vulnerable to: GHSA-mw6j-hh29-h379","Warn: Project is vulnerable to: PYSEC-2020-131 / GHSA-mxjj-953w-2c2v","Warn: Project is vulnerable to: GHSA-p2xf-8hgm-hpw5","Warn: Project is vulnerable to: PYSEC-2021-185 / GHSA-p45v-v4pw-77jr","Warn: Project is vulnerable to: GHSA-p7hr-f446-x6qf","Warn: Project is vulnerable to: GHSA-p9rc-rmr5-529j","Warn: Project is vulnerable to: GHSA-pf36-r9c6-h97j","Warn: Project is vulnerable to: GHSA-pfjj-m3jj-9jc9","Warn: Project is vulnerable to: PYSEC-2021-843 / GHSA-pgcq-h79j-2f69","Warn: Project is vulnerable to: PYSEC-2021-200 / GHSA-ph87-fvjr-v33w","Warn: Project is vulnerable to: PYSEC-2021-235 / GHSA-pmpr-55fj-r229","Warn: Project is vulnerable to: GHSA-pqhm-4wvf-2jg8","Warn: Project is vulnerable to: GHSA-pqrv-8r2f-7278","Warn: Project is vulnerable to: PYSEC-2021-390 / GHSA-prcg-wp5q-rv7p","Warn: Project is vulnerable to: PYSEC-2021-203 / GHSA-pvrc-hg3f-58r6","Warn: Project is vulnerable to: GHSA-pxrw-j2fv-hx3h","Warn: Project is vulnerable to: PYSEC-2020-257 / GHSA-q263-fvxm-m5mw","Warn: Project is vulnerable to: GHSA-q2c3-jpmc-gfjx","Warn: Project is vulnerable to: GHSA-q3g3-h9r4-prrc","Warn: Project is vulnerable to: PYSEC-2020-130 / GHSA-q4qf-3fc6-8x34","Warn: Project is vulnerable to: GHSA-q5jv-m6qw-5g37","Warn: Project is vulnerable to: GHSA-q7f7-544h-67h9","Warn: Project is vulnerable to: GHSA-q85f-69q7-55h2","Warn: Project is vulnerable to: PYSEC-2020-127 / GHSA-q8gv-q7wr-9jf8","Warn: Project is vulnerable to: GHSA-qfpc-5pjr-mh26","Warn: Project is vulnerable to: PYSEC-2020-132 / GHSA-qh32-6jjc-qprm","Warn: Project is vulnerable to: GHSA-qhw4-wwr7-gjc5","Warn: Project is vulnerable to: PYSEC-2020-254 / GHSA-qhxx-j73r-qpm2","Warn: Project is vulnerable to: GHSA-qj5r-f9mv-rffh","Warn: Project is vulnerable to: GHSA-qjj8-32p7-h289","Warn: Project is vulnerable to: GHSA-qjqc-vqcf-5qvj","Warn: Project is vulnerable to: GHSA-qr82-2c78-4m8h","Warn: Project is vulnerable to: PYSEC-2021-252 / GHSA-qw5h-7f53-xrp6","Warn: Project is vulnerable to: GHSA-qx3f-p745-w4hr","Warn: Project is vulnerable to: GHSA-qxpx-j395-pw36","Warn: Project is vulnerable to: GHSA-r26c-679w-mrjm","Warn: Project is vulnerable to: PYSEC-2021-192 / GHSA-r35g-4525-29fq","Warn: Project is vulnerable to: GHSA-r4c4-5fpq-56wg","Warn: Project is vulnerable to: PYSEC-2021-161 / GHSA-r4pj-74mg-8868","Warn: Project is vulnerable to: GHSA-r6jx-9g48-2r5r","Warn: Project is vulnerable to: PYSEC-2021-202 / GHSA-r6pg-pjwc-j585","Warn: Project is vulnerable to: GHSA-rc9w-5c64-9vqq","Warn: Project is vulnerable to: GHSA-rcf8-g8jv-vg6p","Warn: Project is vulnerable to: PYSEC-2021-239 / GHSA-rf3h-xgv5-2q39","Warn: Project is vulnerable to: PYSEC-2021-416 / GHSA-rg3m-hqc5-344v","Warn: Project is vulnerable to: PYSEC-2021-245 / GHSA-rgvq-pcvf-hx75","Warn: Project is vulnerable to: GHSA-rh87-q4vg-m45j","Warn: Project is vulnerable to: GHSA-rhrq-64mq-hf9h","Warn: Project is vulnerable to: GHSA-rjx6-v474-2ch9","Warn: Project is vulnerable to: GHSA-rmg2-f698-wq35","Warn: Project is vulnerable to: PYSEC-2020-138 / GHSA-rrfp-j2mp-hq9c","Warn: Project is vulnerable to: GHSA-rrx2-r989-2c43","Warn: Project is vulnerable to: GHSA-rww7-2gpw-fv6j","Warn: Project is vulnerable to: GHSA-v3f7-j968-4h5f","Warn: Project is vulnerable to: PYSEC-2021-234 / GHSA-v52p-hfjf-wg88","Warn: Project is vulnerable to: GHSA-v5xg-3q2c-c2r4","Warn: Project is vulnerable to: GHSA-v6h3-348g-6h5x","Warn: Project is vulnerable to: PYSEC-2021-214 / GHSA-v6r6-84gr-92rm","Warn: Project is vulnerable to: GHSA-v768-w7m9-2vmm","Warn: Project is vulnerable to: GHSA-v7vw-577f-vp8x","Warn: Project is vulnerable to: GHSA-v82p-hv3v-p6qp","Warn: Project is vulnerable to: GHSA-vcjj-9vg7-vf68","Warn: Project is vulnerable to: PYSEC-2021-232 / GHSA-vf94-36g5-69v8","Warn: Project is vulnerable to: PYSEC-2021-225 / GHSA-vfr4-x8j2-3rf9","Warn: Project is vulnerable to: GHSA-vgvh-2pf4-jr2x","Warn: Project is vulnerable to: GHSA-vjg4-v33c-ggc4","Warn: Project is vulnerable to: GHSA-vm7x-4qhj-rrcq","Warn: Project is vulnerable to: GHSA-vmjw-c2vp-p33c","Warn: Project is vulnerable to: PYSEC-2021-218 / GHSA-vq2r-5xvm-3hc3","Warn: Project is vulnerable to: PYSEC-2021-188 / GHSA-vqw6-72r7-fgw7","Warn: Project is vulnerable to: PYSEC-2021-250 / GHSA-vvg4-vgrv-xfr7","Warn: Project is vulnerable to: PYSEC-2021-406 / GHSA-vwhq-49r4-gj9v","Warn: Project is vulnerable to: GHSA-vxv8-r8q2-63xw","Warn: Project is vulnerable to: GHSA-w4xf-2pqw-5mq7","Warn: Project is vulnerable to: PYSEC-2020-129 / GHSA-w5gh-2wr2-pm6g","Warn: Project is vulnerable to: GHSA-w62h-8xjm-fv49","Warn: Project is vulnerable to: GHSA-w74j-v8xh-3w5h","Warn: Project is vulnerable to: GHSA-wc4g-r73w-x8mm","Warn: Project is vulnerable to: PYSEC-2021-157 / GHSA-wcv5-qrj6-9pfm","Warn: Project is vulnerable to: GHSA-wcv5-vrvr-3rx2","Warn: Project is vulnerable to: GHSA-wf5p-c75w-w3wh","Warn: Project is vulnerable to: PYSEC-2021-208 / GHSA-whr9-vfh2-7hm6","Warn: Project is vulnerable to: GHSA-wm93-f238-7v37","Warn: Project is vulnerable to: PYSEC-2021-204 / GHSA-wp3c-xw9g-gpcg","Warn: Project is vulnerable to: GHSA-wp77-4gmm-7cq8","Warn: Project is vulnerable to: GHSA-wq6q-6m32-9rv9","Warn: Project is vulnerable to: GHSA-wqmc-pm8c-2jhc","Warn: Project is vulnerable to: GHSA-wr9v-g9vf-c74v","Warn: Project is vulnerable to: PYSEC-2021-256 / GHSA-wvjw-p9f5-vq28","Warn: Project is vulnerable to: GHSA-wxjj-cgcx-r3vq","Warn: Project is vulnerable to: PYSEC-2021-407 / GHSA-x3v8-c8qx-3j3r","Warn: Project is vulnerable to: PYSEC-2021-164 / GHSA-x4g7-fvjj-prg8","Warn: Project is vulnerable to: GHSA-x4qx-4fjv-hmw6","Warn: Project is vulnerable to: PYSEC-2021-186 / GHSA-x83m-p7pv-ch8v","Warn: Project is vulnerable to: PYSEC-2021-217 / GHSA-x8h6-xgqx-jqgp","Warn: Project is vulnerable to: GHSA-x989-q2pq-4q5x","Warn: Project is vulnerable to: PYSEC-2020-133 / GHSA-x9j7-x98r-r4w2","Warn: Project is vulnerable to: PYSEC-2021-167 / GHSA-xcwj-wfcm-m23c","Warn: Project is vulnerable to: GHSA-xf83-q765-xm6m","Warn: Project is vulnerable to: PYSEC-2021-177 / GHSA-xgc3-m89p-vr3x","Warn: Project is vulnerable to: PYSEC-2021-162 / GHSA-xm2v-8rrw-w9pm","Warn: Project is vulnerable to: PYSEC-2020-126 / GHSA-xmq7-7fxm-rr79","Warn: Project is vulnerable to: PYSEC-2021-178 / GHSA-xqfj-35wv-m3cr","Warn: Project is vulnerable to: PYSEC-2021-255 / GHSA-xqfj-cr6q-pc8w","Warn: Project is vulnerable to: GHSA-xrp2-fhq4-4q3w","Warn: Project is vulnerable to: PYSEC-2021-395 / GHSA-xrqm-fpgr-6hhx","Warn: Project is vulnerable to: PYSEC-2021-221 / GHSA-xvjm-fvxx-q3hv","Warn: Project is vulnerable to: GHSA-xvwp-h6jv-7472","Warn: Project is vulnerable to: PYSEC-2021-194 / GHSA-xw93-v57j-fcgh","Warn: Project is vulnerable to: PYSEC-2020-139 / GHSA-xwhf-g6j5-j5gc","Warn: Project is vulnerable to: GHSA-xxcj-rhqg-m46g","Warn: Project is vulnerable to: PYSEC-2019-204","Warn: Project is vulnerable to: PYSEC-2019-205","Warn: Project is vulnerable to: PYSEC-2019-206","Warn: Project is vulnerable to: PYSEC-2019-207","Warn: Project is vulnerable to: PYSEC-2019-208","Warn: Project is vulnerable to: PYSEC-2019-210","Warn: Project is vulnerable to: PYSEC-2020-114","Warn: Project is vulnerable to: PYSEC-2020-115","Warn: Project is vulnerable to: PYSEC-2020-116","Warn: Project is vulnerable to: PYSEC-2020-119","Warn: Project is vulnerable to: PYSEC-2020-120","Warn: Project is vulnerable to: PYSEC-2020-121","Warn: Project is vulnerable to: PYSEC-2020-122","Warn: Project is vulnerable to: PYSEC-2020-123","Warn: Project is vulnerable to: PYSEC-2020-124","Warn: Project is vulnerable to: PYSEC-2021-151","Warn: Project is vulnerable to: PYSEC-2021-181","Warn: Project is vulnerable to: PYSEC-2021-191","Warn: Project is vulnerable to: PYSEC-2021-248"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-18T00:10:44.188Z","repository_id":52195503,"created_at":"2025-08-18T00:10:44.188Z","updated_at":"2025-08-18T00:10:44.188Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28416495,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T08:38:59.149Z","status":"ssl_error","status_checked_at":"2026-01-14T08:38:43.588Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T07:01:34.188Z","updated_at":"2026-01-14T10:01:48.742Z","avatar_url":"https://github.com/d6t.png","language":"Python","readme":"# Databolt Flow\n\nFor data scientists and data engineers, `d6tflow` is a python library which makes building complex data science workflows easy, fast and intuitive. It is **primarily designed for data scientists to build better models faster**. For data engineers, it can also be a lightweight alternative and help productionize data science models faster. Unlike other data pipeline/workflow solutions, `d6tflow` focuses on managing data science research workflows instead of managing production data pipelines. \n\n## Why use d6tflow?\n\nData science workflows typically look like this.\n\n![Sample Data Workflow](docs/d6tflow-docs-graph.png?raw=true \"Sample Data Workflow\")\n\nThe workflow involves chaining together parameterized tasks which pass multiple inputs and outputs between each other. The output data gets stored in multiple dataframes, files and databases but you have to manually keep track of where everything is. And often you want to rerun tasks with different parameters without inadvertently rerunning long-running tasks. The workflows get complex and your code gets messy, difficult to audit and doesn't scale well.\n\n`d6tflow` to the rescue! **With d6tflow you can easily chain together complex data flows and execute them. You can quickly load input and output data for each task.** It makes your workflow very clear and intuitive.\n\n#### Read more at:  \n[4 Reasons Why Your Machine Learning Code is Probably Bad](https://github.com/d6t/d6t-python/blob/master/blogs/reasons-why-bad-ml-code.rst)  \n[How d6tflow is different from airflow/luigi](https://github.com/d6t/d6t-python/blob/master/blogs/datasci-dags-airflow-meetup.md)\n\n![Badge](https://www.kdnuggets.com/images/tkb-1904-p.png \"Badge\")\n![Badge](https://www.kdnuggets.com/images/tkb-1902-g.png \"Badge\")\n\n## When to use d6tflow?\n\n* Data science: you want to build better models faster. Your workflow is EDA, feature engineering, model training and evaluation. d6tflow works with ANY ML library including sklearn, pytorch, keras\n* Data engineering: you want to build robust data pipelines using a lightweight yet powerful library. You workflow is load, filter, transform, join data in pandas, dask, pyspark, sql, athena\n\n## What can d6tflow do for you?\n\n* Data science  \n\t* Experiment management: easily manage workflows that compare different models to find the best one\n\t* Scalable workflows: build an efficient data workflow that support rapid prototyping and iterations\n\t* Cache data: easily save/load intermediary calculations to reduce model training time\n\t* Model deployment: d6tflow workflows are easier to deploy to production\n* Data engineering  \n\t* Build a data workflow made up of tasks with dependencies and parameters\n\t* Visualize task dependencies and their execution status\n\t* Execute tasks including dependencies\n\t* Intelligently continue workflows after failed tasks\n\t* Intelligently rerun workflow after changing parameters, code or data\n\t* Quickly share and hand off output data to others\n\n\n## Installation\n\nInstall with `pip install d6tflow`. To update, run `pip install d6tflow -U`.\n\nIf you are behind an enterprise firewall, you can also clone/download the repo and run `pip install .`\n\n**Python3 only** You might need to call `pip3 install d6tflow` if you have not set python 3 as default.\n\nTo install latest DEV `pip install git+git://github.com/d6t/d6tflow.git` or upgrade `pip install git+git://github.com/d6t/d6tflow.git -U --no-deps`\n\n## Example: Model Comparison\n\nBelow is an introductory example that gets training data, trains two models and compares their performance.  \n\n**[See the full ML workflow example here](http://tiny.cc/d6tflow-start-example)**  \n**[Interactive mybinder jupyter notebook](http://tiny.cc/d6tflow-start-interactive)**\n\n```python\n\nimport d6tflow\nimport sklearn.datasets, sklearn.ensemble, sklearn.linear_model\nimport pandas as pd\n\n\n# get training data and save it\nclass GetData(d6tflow.tasks.TaskPqPandas):\n    persist = ['x','y']\n\n    def run(self):\n        ds = sklearn.datasets.load_boston()\n        df_trainX = pd.DataFrame(ds.data, columns=ds.feature_names)\n        df_trainY = pd.DataFrame(ds.target, columns=['target'])\n        self.save({'x': df_trainX, 'y': df_trainY}) # persist/cache training data\n\n\n# train different models to compare\n@d6tflow.requires(GetData)  # define dependency\nclass ModelTrain(d6tflow.tasks.TaskPickle):\n    model = d6tflow.Parameter()  # parameter for model selection\n\n    def run(self):\n        df_trainX, df_trainY = self.inputLoad()  # quickly load input data\n\n        if self.model=='ols':  # select model based on parameter\n            model = sklearn.linear_model.LinearRegression()\n        elif self.model=='gbm':\n            model = sklearn.ensemble.GradientBoostingRegressor()\n\n        # fit and save model with training score\n        model.fit(df_trainX, df_trainY)\n        self.save(model)  # persist/cache model\n        self.saveMeta({'score': model.score(df_trainX, df_trainY)})  # save model score\n\n# goal: compare performance of two models\n# define workflow manager\nflow = d6tflow.WorkflowMulti(ModelTrain, {'model1':{'model':'ols'}, 'model2':{'model':'gbm'}})\nflow.reset_upstream(confirm=False) # DEMO ONLY: force re-run\nflow.run()  # execute model training including all dependencies\n\n'''\n===== Execution Summary =====\nScheduled 2 tasks of which:\n* 2 ran successfully:\n    - 1 GetData()\n    - 1 ModelTrain(model=ols)\nThis progress looks :) because there were no failed tasks or missing dependencies\n'''\n\nscores = flow.outputLoadMeta()  # load model scores\nprint(scores)\n# {'model1': {'score': 0.7406426641094095}, 'gbm': {'model2': 0.9761405838418584}}\n\n\n```\n\n\n## Example Library\n\n* [Minimal example](https://github.com/d6t/d6tflow/blob/master/docs/example-minimal.py)\n* [Rapid Prototyping for Quantitative Investing with d6tflow](https://github.com/d6tdev/d6tflow-binder-interactive/blob/master/example-trading.ipynb) \n* d6tflow with functions only: get the power of d6tflow with little change in code. **[Jupyter notebook example](https://github.com/d6t/d6tflow/blob/master/docs/example-functional.ipynb)**\n\n## Documentation\n\nLibrary usage and reference https://d6tflow.readthedocs.io\n\n## Getting started resources\n\n[Transition to d6tflow from typical scripts](https://d6tflow.readthedocs.io/en/latest/transition.html)\n\n[5 Step Guide to Scalable Deep Learning Pipelines with d6tflow](https://htmlpreview.github.io/?https://github.com/d6t/d6t-python/blob/master/blogs/blog-20190813-d6tflow-pytorch.html)\n\n[Data science project starter templates](https://github.com/d6t/d6tflow-template)\n\n## Collecting Errors Messages and Usage statistics\n\nWe have put a lot of effort into making this library useful to you. To help us make this library even better, it collects ANONYMOUS error messages and usage statistics. See [d6tcollect](https://github.com/d6t/d6tcollect) for details including how to disable collection. Collection is asynchronous and doesn't impact your code in any way.\n\nIt may not catch all errors so if you run into any problems or have any questions, please raise an issue on github.\n\n## How To Contribute\n\nThank you for considering to contribute to the project. First, fork the code repository and then pick an issue that is open. Afterwards follow these steps\n* Create a branch called \\[issue_no\\]\\_yyyymmdd\\_\\[feature\\]\n* Implement the feature\n* Write unit tests for the desired behaviour\n* Create a pull request to merge branch with master\n\nA similar workflow applies to bug-fixes as well. In the case of a fix, just change the feature name with the bug-fix name. And make sure the code passes already written unit tests.\n","funding_links":[],"categories":["Model and Data Versioning","Python","Data Pipeline","数据管道和流处理","Building","Data Pipelines \u0026 Streaming"],"sub_categories":["Workflows"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fd6t%2Fd6tflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fd6t%2Fd6tflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fd6t%2Fd6tflow/lists"}