{"id":19963261,"url":"https://github.com/bilbottom/dbt-py","last_synced_at":"2025-05-03T22:31:54.191Z","repository":{"id":215699152,"uuid":"739522084","full_name":"Bilbottom/dbt-py","owner":"Bilbottom","description":"Python wrapper for dbt-core to extend dbt with custom Python.","archived":false,"fork":false,"pushed_at":"2024-05-29T04:59:38.000Z","size":416,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-05-29T17:49:46.490Z","etag":null,"topics":["dbt","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Bilbottom.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-05T19:12:44.000Z","updated_at":"2024-06-18T06:32:38.678Z","dependencies_parsed_at":"2024-04-16T06:34:00.038Z","dependency_job_id":"d959b8d6-50f9-45bb-8417-03c82c8ab34c","html_url":"https://github.com/Bilbottom/dbt-py","commit_stats":null,"previous_names":["bilbottom/dbt-py"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Bilbottom%2Fdbt-py","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Bilbottom%2Fdbt-py/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Bilbottom%2Fdbt-py/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Bilbottom%2Fdbt-py/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Bilbottom","download_url":"https://codeload.github.com/Bilbottom/dbt-py/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224374655,"owners_count":17300691,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dbt","python"],"created_at":"2024-11-13T02:15:26.074Z","updated_at":"2025-05-03T22:31:54.180Z","avatar_url":"https://github.com/Bilbottom.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n[![Python](https://img.shields.io/badge/Python-3.11+-blue.svg)](https://www.python.org/downloads/)\n[![uv](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/uv/main/assets/badge/v0.json)](https://github.com/astral-sh/uv)\n[![dbt](https://img.shields.io/badge/dbt-1.5+-blue.svg)](https://github.com/dbt-labs/dbt-core/)\n[![tests](https://github.com/Bilbottom/dbt-py/actions/workflows/tests.yaml/badge.svg)](https://github.com/Bilbottom/dbt-py/actions/workflows/tests.yaml)\n[![coverage](coverage.svg)](https://github.com/dbrgn/coverage-badge)\n[![GitHub last commit](https://img.shields.io/github/last-commit/Bilbottom/dbt-py)](https://shields.io/badges/git-hub-last-commit)\n\n[![code style: prettier](https://img.shields.io/badge/code_style-prettier-ff69b4.svg?style=flat-square)](https://github.com/prettier/prettier)\n[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)\n[![pre-commit.ci status](https://results.pre-commit.ci/badge/github/Bilbottom/dbt-py/main.svg)](https://results.pre-commit.ci/latest/github/Bilbottom/dbt-py/main)\n\n\u003c/div\u003e\n\n---\n\n# dbt-π 🧬\n\nPython wrapper for [dbt-core](https://github.com/dbt-labs/dbt-core) to extend dbt with custom Python.\n\n## Shimmy shimmy shim 🕺🕺🕺\n\nThis package is a [shim](\u003chttps://en.wikipedia.org/wiki/Shim_(computing)\u003e) for [dbt-core](https://github.com/dbt-labs/dbt-core), inspired by (_cough_ stolen from _cough_) my old boss, [@darkdreamingdan](https://github.com/darkdreamingdan):\n\n- https://gist.github.com/darkdreamingdan/c5ded709a90fc3c5b420cee5f644f499\n\nBefore using this package, it's recommended to get up to speed with the Python modules that are already available in dbt:\n\n- https://docs.getdbt.com/reference/dbt-jinja-functions/modules\n\nThe existing Python modules are available in the dbt Jinja context under the `modules` object, for example:\n\n```jinja\n{{ modules.datetime.datetime.now() }}\n```\n\n## Installation ⬇️\n\nWhile in preview, this package is only available from GitHub:\n\n```\npip install git+https://github.com/Bilbottom/dbt-py@v0.0.4\n```\n\nThis will be made available on PyPI once it's ready for general use.\n\n## Usage 📖\n\nThis package adds a new executable, `dbt-py`, which injects your custom Python into dbt and then runs dbt. Either a custom module or a custom package can be injected. A custom module is the simplest to get started with.\n\nThe default module/package name is `custom` which would make custom Python available in the dbt Jinja context under the `modules.custom` object. This can be configured (see the [Configuration](#configuration-) section below).\n\n\u003e [!IMPORTANT]\n\u003e\n\u003e If you create the Python files in your dbt repo, you **must** install your own project as a package for your files to be found and imported by dbt.\n\u003e\n\u003e This is typically achieved with `pip install -e .`; please see the pip docs for more details:\n\u003e\n\u003e - [https://pip.pypa.io/en/stable/topics/local-project-installs/#editable-installs](https://pip.pypa.io/en/stable/topics/local-project-installs/#editable-installs)\n\u003e\n\u003e If you use a package manager such as [Poetry](https://python-poetry.org/) or [uv](https://docs.astral.sh/uv/), they should automatically install your project provided you configure the project metadata correctly.\n\u003e\n\u003e See the following repo for a minimal example that uses `requirements.txt` and [setuptools](https://setuptools.pypa.io/en/latest/):\n\u003e\n\u003e - [https://github.com/Bilbottom/dbt-py-test](https://github.com/Bilbottom/dbt-py-test)\n\n### Custom Module 🐍\n\nCreate a module called `custom.py` in the root of your dbt project. This module can contain any Python code you like, for example:\n\n```python\ndef salutation(name: str) -\u003e str:\n    return f\"Hello, {name}!\"\n```\n\nReference this module and function in the dbt Jinja context of a dbt model:\n\n```jinja\n{{ modules.custom.salutation(\"World\") }}\n```\n\nRather than run dbt with the `dbt` command, instead run it with `dbt-py`:\n\n```\ndbt-py clean\ndbt-py build\n```\n\nNote that `dbt-py` is a wrapper around `dbt` so all the usual dbt commands are available -- all the arguments passed to `dbt-py` are passed through to `dbt`, too.\n\n```\ndbt-py --help\ndbt-py run --select my_model\ndbt-py test --select tag:unit-test\n```\n\n### Custom Package 📦\n\nUsing a custom package is similar to using a custom module: create a package called `custom` in the root of your dbt project.\n\nThe submodules of this package will be available in the dbt Jinja context too. For example, suppose you have a package called `custom` with a submodule called `greetings`:\n\n```\ncustom/\n    __init__.py\n    greetings.py\n```\n\nIf the `greetings.py` submodule contains the same `salutation` function as above, then it can be referenced in the dbt Jinja context as follows:\n\n```jinja\n{{ modules.custom.greetings.salutation(\"World\") }}\n```\n\nAlternatively, you can expose the `salutation` function via the `__init__.py` file and then reference it directly via `custom`:\n\n```jinja\n{{ modules.custom.salutation(\"World\") }}\n```\n\n### Configuration 🛠️\n\nThe default module/package and Jinja context name is `custom` but both can be configured with the following environment variables:\n\n- `DBT_PY_PACKAGE_ROOT`: The Python-style ref to the custom module/package, e.g. `package.module.submodule`\n- `DBT_PY_PACKAGE_NAME`: The name to give the custom module/package in the dbt Jinja context, e.g. `custom_py`. Defaults to the value of `DBT_PY_PACKAGE_ROOT`\n\nIn particular, you can use the `DBT_PY_PACKAGE_ROOT` environment variable to reference a custom module/package that is not at the root of your dbt project.\n\n\u003e [!WARNING]\n\u003e\n\u003e If you set the `DBT_PY_PACKAGE_ROOT` environment variable to a name that already exists, this package will use the existing module/package rather than your custom one. Make sure that your custom module/package name does not clash with any existing modules/packages.\n\u003e\n\u003e This is likely to change in a future release, but for now you may choose to exploit this behaviour to use an existing module/package in your dbt Jinja context. For example, you could set `DBT_PY_PACKAGE_ROOT` to `math` and then reference the `math` standard library in your dbt Jinja context:\n\u003e\n\u003e ```jinja\n\u003e {{ modules.math.pi }}\n\u003e ```\n\n## Future Work 🚧\n\nThis is still in preview, and there are a few things to be added before it's ready for general use:\n\n- Support for importing any number of packages (currently only one package is supported)\n- Configuration via config files and CLI arguments (currently only environment variables are supported)\n- More robust testing\n\n## Contributing 🤝\n\nRaise an issue, or fork the repo and open a pull request.\n\nThis project uses [uv](https://github.com/astral-sh/uv/) and [pre-commit](https://pre-commit.com/). After cloning the repo, install the dependencies and enable pre-commit:\n\n```\nuv sync --all-groups\npre-commit install --install-hooks\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbilbottom%2Fdbt-py","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbilbottom%2Fdbt-py","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbilbottom%2Fdbt-py/lists"}