{"id":27933460,"url":"https://github.com/timescale/docker-dbt","last_synced_at":"2025-05-07T04:58:54.123Z","repository":{"id":120501885,"uuid":"610412141","full_name":"timescale/docker-dbt","owner":"timescale","description":"Dockerfiles for dbt","archived":false,"fork":false,"pushed_at":"2024-07-15T17:56:23.000Z","size":623,"stargazers_count":3,"open_issues_count":6,"forks_count":0,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-05-07T04:58:49.410Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/timescale.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2023-03-06T18:11:26.000Z","updated_at":"2024-11-02T13:20:50.000Z","dependencies_parsed_at":null,"dependency_job_id":"f554c84d-fc5b-4d57-9511-bf4d738f2d2c","html_url":"https://github.com/timescale/docker-dbt","commit_stats":null,"previous_names":["timescale/docker-dbt"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timescale%2Fdocker-dbt","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timescale%2Fdocker-dbt/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timescale%2Fdocker-dbt/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timescale%2Fdocker-dbt/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/timescale","download_url":"https://codeload.github.com/timescale/docker-dbt/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252816948,"owners_count":21808704,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-05-07T04:58:53.539Z","updated_at":"2025-05-07T04:58:54.114Z","avatar_url":"https://github.com/timescale.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# `popsql/dbt` docker images\n\nThis repo contains the code for generating our dbt Docker images, which are located on\nthe GitHub container registry.\n\nBased off of the official python image, we then add:\n\n- `dbt-core`\n- `dbt-rpc`\n- one or all of the `dbt` adapters (e.g. `dbt-postgres`)\n\nWhere each image is available for dbt 0.19 onward.\n\n## Use\n\nImages are published to the GitHub container registry, where we have one image that\ncontains all the adapters, and then a container per adapter:\n\n```bash\n# has all adapters\ndocker pull ghcr.io/popsql/dbt-full:${VERSION}\n# individual adapters\ndocker pull ghcr.io/popsql/dbt-${ADAPTER}:${VERSION}\n```\n\nEach image has both the dbt CLI as well as RPC programs available to use. The image\nsets the following environment variables by default:\n* `DBT_PROFILES_DIR=/.dbt` - where dbt should look for the profiles directory\n* `AWS_SHARED_CREDENTIALS_FILE=/.dbt/aws_credentials` - where boto3/AWS libraries will look for the AWS credentials file\n\nThe `DBT_PROFILES_DIR` variable can be overwritten when passing the `--profiles-dir` flag for a command.\n\nOn image start, the image entrypoint will do the following checks to setup the `/.dbt` directory:\n* If `DBT_PROFILES` is set:\n  * write the contents to `/.dbt/profiles.yml`\n  * If `AWS_CREDENTIALS` is set, write its contents to `/.dbt/aws_credentials`\n  * If `BQ_KEYFILE` is set, write its contents to `/.dbt/bq_keyfile.json`\n* else:\n  * If `~/.dbt/profiles.yml` exists, copy it to `/.dbt/profiles.yml`\n  * If `~/.aws/credentials` exists, copy it to `/.dbt/aws_credentials`\n\n## Development\n\nThe repo is structured such that under the `./requirements` folder, there is a folder\nthat contains each version of dbt we support. Within each folder, there is then\n`pyproject.toml` and `poetry.lock` files. Within each `pyproject.toml`, there is then\nthe list of adapters we support for that given version, along with the shared\ndependencies (e.g. `dbt-core` and `dbt-rpc`). After changing something within the\n`pyproject.toml` file, you will need to run `poetry lock` to update the `poetry.lock`\nfile (use `poetry lock --no-update` to avoid unnecessarily updating all dependencies\nin the lock file).\n\nAs part of our CD process, we then handle generating a per adapter requirements file\nfrom these two files.\n\n### Requirements\n\n* Python 3.8+\n* [poetry](https://python-poetry.org/)\n\n### Building Images Locally\n\nDocker images can be built locally via the `./bin/build.sh` script, which takes two\narguments: a version and optionally an adapter. For example, the following command\nbuilds the full image for dbt 1.4:\n\n```bash\n./bin/build.sh 1.4\n```\n\nand then to build the image for just Athena adapter for 1.4:\n\n```bash\n./bin/build.sh 1.4 athena\n```\n\nThe image is named and tagged in the same fashion as when it's built by the deploy\npipeline, meaning that it can be used locally in place of such an image until it's\npulled down from ghcr again.\n\n## Docker Platforms\n\nMost of the produced images should support the following platforms:\n\n- linux/amd64\n- linux/arm64\n\nSome images may not have a `linux/arm64` target if building for it is not possible, or\nvery ardous. For example, `dbt-snowflake \u003c= 1.1` requires building pyarrow from source\nwhich requires a bunch of additional packages and time, so we only have `linux/amd64`\nplatforms available there.\n\nThe information on which platforms to build for a given image is captured within our\n`deploy.yml` CD script.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimescale%2Fdocker-dbt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftimescale%2Fdocker-dbt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimescale%2Fdocker-dbt/lists"}