{"id":19383800,"url":"https://github.com/airbytehq/airflow-summit-airbyte-2022","last_synced_at":"2025-08-13T18:05:02.235Z","repository":{"id":37541494,"uuid":"495609273","full_name":"airbytehq/airflow-summit-airbyte-2022","owner":"airbytehq","description":"git push your data stack with Airbyte, Airflow, and dbt  - 2022 Airflow Summit","archived":false,"fork":false,"pushed_at":"2023-05-12T19:25:36.000Z","size":36,"stargazers_count":53,"open_issues_count":3,"forks_count":20,"subscribers_count":13,"default_branch":"main","last_synced_at":"2025-04-23T21:38:10.393Z","etag":null,"topics":["airbyte","airflow","data-engineering","elt"],"latest_commit_sha":null,"homepage":"https://www.airbyte.com","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/airbytehq.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-05-23T23:54:23.000Z","updated_at":"2024-12-15T02:31:08.000Z","dependencies_parsed_at":"2024-11-10T09:28:02.886Z","dependency_job_id":null,"html_url":"https://github.com/airbytehq/airflow-summit-airbyte-2022","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/airbytehq/airflow-summit-airbyte-2022","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/airbytehq%2Fairflow-summit-airbyte-2022","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/airbytehq%2Fairflow-summit-airbyte-2022/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/airbytehq%2Fairflow-summit-airbyte-2022/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/airbytehq%2Fairflow-summit-airbyte-2022/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/airbytehq","download_url":"https://codeload.github.com/airbytehq/airflow-summit-airbyte-2022/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/airbytehq%2Fairflow-summit-airbyte-2022/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270286484,"owners_count":24558611,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-13T02:00:09.904Z","response_time":66,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["airbyte","airflow","data-engineering","elt"],"created_at":"2024-11-10T09:27:56.053Z","updated_at":"2025-08-13T18:05:02.186Z","avatar_url":"https://github.com/airbytehq.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# `git push` your data stack with Airbyte, Airflow and dbt\n\n_For Airflow Summit 2022, by [@marcosmarcm](https://github.com/marcosmarxm) and [@evantahler](https://github.com/evantahler) from [Airbyte](https://github.com/airbytehq)_\n\n[![CI](https://github.com/airbytehq/airflow-summit-airbyte-2022/actions/workflows/data-pipeline.yml/badge.svg)](https://github.com/airbytehq/airflow-summit-airbyte-2022/actions/workflows/data-pipeline.yml)\n\n## What:\n\nLinks:\n\n- [Slides](https://docs.google.com/presentation/d/17TuHlzgF3x_Q2NtkOq0O7SmR9e0MGqXMqlwVqJyUoKI)\n- [Speaker Page](https://airflowsummit.org/sessions/2022/git-push-your-data-stack-with-airbyte-airflow-and-dbt/)\n\nThis project configures a sample data stack orchestrated by Airflow, using Airbyte to Extract and Load data, and dbt to Transform it.\n\n## Running Locally:\n\n0.  Install [Docker Desktop](https://www.docker.com/products/docker-desktop/) and Python 3 (if you are on MacOS, you already have Python 3).\n\n1.  Create `{HOME}/.octavia` and add the following credentials for using a local postgres database managed by Docker:\n\n```\nPOSTGRES_HOST=host.docker.internal\nPOSTGRES_PASSWORD=password\nPOSTGRES_USERNAME=demo_user\nPOSTGRES_DATABASE=postgres\n```\n\n2. Create the profile dbt in `{HOME}/.dbt/profiles.yaml`\n\n```\nconfig:\n  partial_parse: true\n  printer_width: 120\n  send_anonymous_usage_stats: false\n  use_colors: true\nnormalize:\n  outputs:\n    prod:\n      dbname: postgres\n      host: host.docker.internal\n      pass: password\n      port: 5432\n      schema: public\n      threads: 8\n      type: postgres\n      user: demo_user\n  target: prod\n```\n\n3. Run the whole data stack using `./tools/start.sh`. This will install local requirements (PyYAML) and run everything though Docker. The script will exit when complete, but the Docker containers will remain running.\n\nIn your browser:\n\n- Visit http://localhost:8080/ to see the Airflow UI (user: `airflow`, password: `airflow`) and your completed DAG.\n- Visit http://localhost:8000/ to see the Airbyte UI and your completed Sync.\n- Visit your local postgres database (`localhost:5432`) with the `username=demo_user` and `password=password` to see the staged and transformed data.\n\n## Shut it down\n\nRun `./tools/stop.sh` to stop the Docker containers.\n\n## Testing\n\nThis repository is tested using Github Actions.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fairbytehq%2Fairflow-summit-airbyte-2022","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fairbytehq%2Fairflow-summit-airbyte-2022","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fairbytehq%2Fairflow-summit-airbyte-2022/lists"}