{"id":23112365,"url":"https://github.com/lixx21/dbt-shopping-data-transform","last_synced_at":"2025-04-03T23:45:15.903Z","repository":{"id":264457560,"uuid":"893432298","full_name":"lixx21/dbt-shopping-data-transform","owner":"lixx21","description":"This project leverages DBT (Data Build Tool) to transform raw shopping data into a well-structured, analytics-ready format","archived":false,"fork":false,"pushed_at":"2024-11-30T03:33:27.000Z","size":625,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-02-09T11:41:42.148Z","etag":null,"topics":["data-engineer","dbt","docker","etl-pipeline"],"latest_commit_sha":null,"homepage":"","language":"Dockerfile","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lixx21.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-24T12:42:43.000Z","updated_at":"2024-12-29T16:09:49.000Z","dependencies_parsed_at":null,"dependency_job_id":"65f67508-affc-491d-bb2a-bea2e583e275","html_url":"https://github.com/lixx21/dbt-shopping-data-transform","commit_stats":null,"previous_names":["lixx21/dbt-transform-data"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lixx21%2Fdbt-shopping-data-transform","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lixx21%2Fdbt-shopping-data-transform/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lixx21%2Fdbt-shopping-data-transform/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lixx21%2Fdbt-shopping-data-transform/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lixx21","download_url":"https://codeload.github.com/lixx21/dbt-shopping-data-transform/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247097974,"owners_count":20883127,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-engineer","dbt","docker","etl-pipeline"],"created_at":"2024-12-17T02:17:48.127Z","updated_at":"2025-04-03T23:45:15.897Z","avatar_url":"https://github.com/lixx21.png","language":"Dockerfile","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Shopping Data Transformation with DBT\n\nWelcome to the Shopping Data Transformation project! This project leverages DBT (Data Build Tool) to transform raw shopping data into a well-structured, analytics-ready format. The transformed data can be used for business intelligence, reporting, and data science workflows. This project demonstrates the creation of a data mart/data view using Pandas, PostgreSQL, DBT (Data Build Tool), and Docker. \n\n## Project Overview\n\nThe goal of this project is to:\n\n1. Extract and transform raw data into a structured format using Pandas.\n2. Load the transformed data into a PostgreSQL database.\n3. Model and optimize the data using DBT to create a data mart.\n4. Containerize the environment using Docker for easy deployment and reproducibility.\n\n## Features\n\n1. ETL Pipeline: Extract, transform, and load data using Python and Pandas.\n2. PostgreSQL Database: Central storage for the dataset.\n3. DBT Models: Clean and optimize data to create insightful views.\n4. Dockerized Setup: Simplified deployment with Docker and Docker Compose.\n\n## Dataset\nhttps://www.kaggle.com/datasets/bhadramohit/customer-shopping-latest-trends-dataset\n\nThis project demonstrates the creation of a data mart/data view using Pandas, PostgreSQL, DBT (Data Build Tool), and Docker. The dataset is sourced from Kaggle, containing customer shopping trends and purchase behaviors.\n\n## How to run\n1. git clone `https://github.com/lixx21/dbt-shopping-data-transform.git`\n2. `cd dbt-shopping-data-transform` \n3. run `docker-compose.yaml` using `docker-compose up --build -d` command. This command will automatically run PostgreSQL, PgAdmin (in localhost:5050) and ETL file in [load_data folder](/load_data)\n4. you can find the email and password of the PgAdmin and also username, password and main db name in `docker-compose.yaml` (feel free to adjust with your own account)\n\n## Example Data Mart\n\n![view-data-sum-itme-category.png](./images/view-data-sum-item-category.png)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flixx21%2Fdbt-shopping-data-transform","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flixx21%2Fdbt-shopping-data-transform","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flixx21%2Fdbt-shopping-data-transform/lists"}