{"id":16442261,"url":"https://github.com/sungchun12/sqlmesh-demos","last_synced_at":"2025-10-24T12:36:35.610Z","repository":{"id":238562543,"uuid":"780670258","full_name":"sungchun12/sqlmesh-demos","owner":"sungchun12","description":"SQLMesh project for live demos - provides instructions so you can run this on your own!","archived":false,"fork":false,"pushed_at":"2025-02-24T19:21:31.000Z","size":857,"stargazers_count":3,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-04T13:44:43.440Z","etag":null,"topics":["data","data-engineering","sql","sqlmesh"],"latest_commit_sha":null,"homepage":"https://sqlmesh.readthedocs.io/en/latest/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sungchun12.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":"audits/.gitkeep","citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-04-02T00:02:49.000Z","updated_at":"2025-02-07T18:58:40.000Z","dependencies_parsed_at":"2025-02-07T19:38:58.230Z","dependency_job_id":null,"html_url":"https://github.com/sungchun12/sqlmesh-demos","commit_stats":null,"previous_names":["sungchun12/sqlmesh-demos"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sungchun12%2Fsqlmesh-demos","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sungchun12%2Fsqlmesh-demos/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sungchun12%2Fsqlmesh-demos/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sungchun12%2Fsqlmesh-demos/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sungchun12","download_url":"https://codeload.github.com/sungchun12/sqlmesh-demos/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250993569,"owners_count":21519694,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data","data-engineering","sql","sqlmesh"],"created_at":"2024-10-11T09:16:50.040Z","updated_at":"2025-10-24T12:36:30.580Z","avatar_url":"https://github.com/sungchun12.png","language":"Python","readme":"\u003ch2 align=\"center\"\u003e\n\n![](sqlmesh_logo.png) Demos\n\n\u003c/h2\u003e\n\nThis is a simple, loveable, and complete SQLMesh demo project with the goal of running multiple, realistic scenarios very fast. \n\nThis repo is going to try something a bit novel compared to your standard demo projects. I'll take you through different stories that illustrate the data engineering workflow. It'll engage your inner dialogue along with look and feel of the development experience. The hope is for you to better internalize and translate these stories to your own reality. Don't worry, these stories won't be too long-winded!\n\nThis is intentionally similar to what people may have experienced when I created this github repo at dbt Labs: [here](https://github.com/dbt-labs/jaffle_shop_duckdb)\n\n\n## Basic Setup\n\nWhat you'll be using:\n\n- SQLMesh: transformation framework\n- BigQuery: data warehouse to run transforms against\n- DuckDB: local development testing\n- Postgres: Store SQLMesh metadata\n- SQLGlot: Python macros that compile to SQL\n- pytest: test Python macros\n\n\u003cdetails\u003e\n\n**Setup your virtual environment for SQLMesh:**\n\n```bash\ngit clone https://github.com/sungchun12/sqlmesh-demos.git # clone the repo\ncd sqlmesh-demos # go to the root directory\npython -m venv venv # create virtual environment\nsource venv/bin/activate # activate it\npip install -r requirements.txt # install dependencies\nsource venv/bin/activate # reactivate virtual environment\n```\n\n**Setup your BigQuery Service Account:**\n\n![service_account](./images/bigquery_service_account.png)\n\n1. Create a service account following these instructions: [here](https://cloud.google.com/iam/docs/service-accounts-create)\n2. Add permissions: `BigQuery Data Editor`, `BigQuery User`\n3. Download the service account json file\n4. Copy the contents of the service account file to your clipboard\n5. Export the credentials as an environment variable in your terminal: \n\n`export GOOGLE_CREDENTIALS=\u003cyour-service-account-key-contents\u003e`\n\n**Setup your Postgres Database:**\n\n1. Sign up for a free Neon account: [here](https://neon.tech/)\n2. Create a new database named: `sqlmesh_state_demo`\n![new_database](./images/new_database.png)\n3. Export your environment variables based on the newly created database connection string\n![new_database](./images/connection_string.png)\n\n```bash\n# examples based on the image above\nexport SQLMESH_STATE_HOST=ep-purple-heart-a6mqlep8.us-west-2.aws.neon.tech\nexport SQLMESH_STATE_USERNAME=\u003cyour-neon-username\u003e\nexport SQLMESH_STATE_PASSWORD=\u003cyour-password-blurred-in-the-image\u003e\n```\n\n**Verify SQLMesh can connect to BigQuery and Postgres:**\n\n```bash\nsqlmesh info # print info about a SQLMesh project\n\n# expected output\nModels: 15\nMacros: 1\nData warehouse connection succeeded\nState backend connection succeeded\nTest connection succeeded\n```\n\n\u003c/details\u003e\n\n## Story #1\n\nHow do I run SQLMesh as fast as possible and get the general look and feel?\n\nRun this command and watch the demo video!\n\n```bash\nsqlmesh plan # follow the instructions in the CLI prompt\n```\n\nTODO: add video\n\n\n\n## Story #2\n\nHow do I go about making a big breaking change to the logic of a core sql model?\n\nTODO\n\n### Credits\n\nPortions of this project are modifications based on work created and shared by dbt Labs and used according to terms described in the Apache License, Version 2.0. For the original work and its license, see: [here](https://github.com/dbt-labs/jaffle_shop_duckdb?tab=Apache-2.0-1-ov-file#readme)\n","funding_links":[],"categories":["Sample Projects"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsungchun12%2Fsqlmesh-demos","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsungchun12%2Fsqlmesh-demos","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsungchun12%2Fsqlmesh-demos/lists"}