{"id":23238193,"url":"https://github.com/zkan/getting-started-with-analytics-engineering","last_synced_at":"2025-04-19T17:20:20.754Z","repository":{"id":144820516,"uuid":"551255273","full_name":"zkan/getting-started-with-analytics-engineering","owner":"zkan","description":"Getting Started with Analytics Engineering","archived":false,"fork":false,"pushed_at":"2024-07-25T12:52:45.000Z","size":393,"stargazers_count":10,"open_issues_count":0,"forks_count":147,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-16T08:09:34.359Z","etag":null,"topics":["analytics-engineering","dbt","postgresql","sql"],"latest_commit_sha":null,"homepage":"","language":"Makefile","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zkan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-10-14T04:36:08.000Z","updated_at":"2024-10-08T22:25:47.000Z","dependencies_parsed_at":null,"dependency_job_id":"7d4c27c6-cfcf-4e29-94fd-9b7944128e1d","html_url":"https://github.com/zkan/getting-started-with-analytics-engineering","commit_stats":{"total_commits":73,"total_committers":2,"mean_commits":36.5,"dds":"0.12328767123287676","last_synced_commit":"e30f7ec88b7edbbb4274d2047b9c03cbd5e8d1db"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zkan%2Fgetting-started-with-analytics-engineering","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zkan%2Fgetting-started-with-analytics-engineering/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zkan%2Fgetting-started-with-analytics-engineering/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zkan%2Fgetting-started-with-analytics-engineering/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zkan","download_url":"https://codeload.github.com/zkan/getting-started-with-analytics-engineering/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":235253779,"owners_count":18960482,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["analytics-engineering","dbt","postgresql","sql"],"created_at":"2024-12-19T04:17:00.233Z","updated_at":"2025-01-23T09:07:05.125Z","avatar_url":"https://github.com/zkan.png","language":"Makefile","readme":"# Getting Started with Analytics Engineering\n\n## Files/Folders and What They Do\n\n| Name | Description |\n| - | - |\n| `.devcontainer/devcontainer.json` | A file that tells VS Code how to access (or create) a development container with a well-defined tool and runtime stack |\n| `csv_files/` | A folder that contains CSV files used for testing `dbt seed` |\n| `setup/` | A folder that contains data and scripts used for initializing the Postgres database used in this project |\n| `yaml_files/` | A folder that contains the YAML file examples such as `profiles.yml` or `src.yml` |\n| `.gitignore` | A file that specifies intentionally untracked files that Git should ignore |\n| `LICENSE` | A license of this repo |\n| `Makefile` | A Makefile file which defines set of tasks to be executed |\n| `README.md` | README file that provides the setup instruction on this project |\n| `docker-compose.yml` | A Docker Compose file that runs a Postgres database and SQLPad used in this project |\n| `greenery-dbdiagram.txt` | A file that provides code for drawing an ER diagram on [dbdiagram.io](https://dbdiagram.io/home) |\n| `pyproject.toml` | A file that stores general information such as Python package dependencies for environment setup with Poetry |\n| `requirements.txt` | A file that contains Python package dependencies for this code repository |\n\n## Getting Started\n\nTo start the Docker compose:\n\n```sh\nmake up\n```\n\n### To Set Up and Activate Your Python Virtual Environment\n\n```bash\npython -m venv ENV\nsource ENV/bin/activate\npip install -r requirements.txt\n```\n\n### To Initialize A dbt Project\n\n```bash\ndbt init\n```\n\n**Note:** Let's specify the project name `greenery`.\n\n### To Set Up Your dbt Profile\n\n1. Change the directory to your dbt project.\n\n    ```bash\n    cd \u003cdbt_project_name\u003e\n    ```\n\n1. Run the following command to copy the profiles example file to the real\n   profiles file in the project folder.\n\n    ```bash\n    cp ../yaml_files/profiles.yml .\n    ```\n\n1. Edit the content in the `profiles.yml` file by changing the output and\n   target to your name (e.g., `dbt_john`), and save. See the example below.\n\n    ```yaml\n    greenery:\n\n      outputs:\n        dbt_zkan:\n          type: postgres\n          threads: 1\n          host: localhost\n          port: 5432\n          user: postgres\n          pass: \"{{ env_var('DBT_ENV_SECRET_PG_PASSWORD') }}\"\n          dbname: greenery\n          schema: dbt_zkan\n\n        prod:\n          type: postgres\n          threads: 1\n          host: localhost\n          port: 5432\n          user: postgres\n          pass: \"{{ env_var('DBT_ENV_SECRET_PG_PASSWORD') }}\"\n          dbname: greenery\n          schema: prod\n\n      target: dbt_zkan\n    ```\n\n1. Set the environment variable.\n\n    ```bash\n    export DBT_ENV_SECRET_PG_PASSWORD=postgres\n    ```\n\n1. We then should be able to use dbt now. :-)\n\n### To Debug The dbt Project\n\n```bash\nexport DBT_ENV_SECRET_PG_PASSWORD=postgres\ncd \u003cdbt_project_name\u003e\ndbt debug\n```\n\n### To Create Your Data Models\n\n```bash\nexport DBT_ENV_SECRET_PG_PASSWORD=postgres\ncd \u003cdbt_project_name\u003e\ndbt run\n```\n\n### To Test Your Data Models\n\n```bash\nexport DBT_ENV_SECRET_PG_PASSWORD=postgres\ncd \u003cdbt_project_name\u003e\ndbt test\n```\n\n### To Generate The dbt Documentation and Serve It\n\n```bash\nexport DBT_ENV_SECRET_PG_PASSWORD=postgres\ncd \u003cdbt_project_name\u003e\ndbt docs generate\ndbt docs serve\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzkan%2Fgetting-started-with-analytics-engineering","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzkan%2Fgetting-started-with-analytics-engineering","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzkan%2Fgetting-started-with-analytics-engineering/lists"}