{"id":23317937,"url":"https://github.com/kirbytes/ecommerce-analytic-platform","last_synced_at":"2026-04-13T07:01:25.898Z","repository":{"id":267325830,"uuid":"900899343","full_name":"Kirbytes/ecommerce-analytic-platform","owner":"Kirbytes","description":"DEMO for ISOM671 course project","archived":false,"fork":false,"pushed_at":"2024-12-09T17:22:24.000Z","size":2036,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-07T04:52:47.964Z","etag":null,"topics":["aws","dataengineering","frontend-web","nosql","python","sql"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Kirbytes.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-09T17:14:22.000Z","updated_at":"2024-12-09T17:25:13.000Z","dependencies_parsed_at":"2024-12-09T18:27:09.728Z","dependency_job_id":"bdbd6393-a55d-4c85-acb4-2996eb1904af","html_url":"https://github.com/Kirbytes/ecommerce-analytic-platform","commit_stats":null,"previous_names":["kirbytes/ecommerce-analytic-platform"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kirbytes%2Fecommerce-analytic-platform","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kirbytes%2Fecommerce-analytic-platform/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kirbytes%2Fecommerce-analytic-platform/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kirbytes%2Fecommerce-analytic-platform/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Kirbytes","download_url":"https://codeload.github.com/Kirbytes/ecommerce-analytic-platform/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247595381,"owners_count":20963943,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","dataengineering","frontend-web","nosql","python","sql"],"created_at":"2024-12-20T17:14:34.524Z","updated_at":"2026-04-13T07:01:25.762Z","avatar_url":"https://github.com/Kirbytes.png","language":"Python","readme":"# Alibaba E-commerce Analytics Repository\n\n## Project Description\nData warehouse and analytics dashboard for Alibaba e-commerce transaction data.\n\n## Directory Structure\n```\n.\n├── data/                           # Data files\n│   ├── raw/                       # Raw data\n│   │   └── round1_ijcai_18_train_20180301.txt\n│   └── processed/                 # Cleaned data\n│       ├── clicked_sample.csv     \n│       ├── advertising_item.csv\n│       ├── user_table.csv\n│       ├── context_table.csv\n│       └── shop_table.csv\n├── sql/                           # SQL scripts\n│   ├── dataloader.sql\n│   └── transform_data_for_dashboard.sql\n├── src/                           # Python source code\n│   ├── datacleaner.py\n│   ├── fact_sales_load.py\n│   └── dashboard.py\n└── README.md\n```\n\n## Data Pipeline\n\n### 1. Data Cleaning\n```python\n# src/datacleaner.py\n- Split raw data into normalized tables\n- Handle special delimiters\n- Export clean CSVs\n```\n\n### 2. Database Setup \n```sql\n-- sql/dataloader.sql\n- Create initial tables\n- Load dimensions\n- Set constraints\n```\n\n### 3. Fact Table ETL\nDue to the limited computational resources of the MySQL database, the fact table is generated using Python, then loaded into the database.\n```python\n# src/fact_sales_load.py\n- Aggregate metrics\n- Calculate conversions\n- Export fact table\n```\n\n### 4. Star Schema\n```sql\n-- sql/transform_data_for_dashboard.sql\n- Create dimensional model\n- Time dimension handling\n- Fact table structure\n```\n\n### 5. Dashboard\nThe dashboard in constructed using Tableau, with data source connected to MySQL.\n\n## Database Schema\n\n### Fact Table\n- **fact_sales**\n  - time_key (FK)\n  - item_id (FK)\n  - user_id (FK) \n  - shop_id (FK)\n  - total_views\n  - total_purchases\n  - conversion_rate\n\n### Dimension Tables\n- **dim_time**\n  - time_key (PK)\n  - full_datetime\n  - date_only\n  - hour_of_day\n  - is_peak_hour\n  - is_night\n  - day_of_week\n  - is_weekend\n  - time_segment\n\n- **dim_product**\n  - item_id (PK)\n  - category_list\n  - property_list\n  - brand_id\n  - city_id\n  - price_level\n  - sales_level\n  - collected_level\n  - pv_level\n\n- **dim_user**\n  - user_id (PK)\n  - gender_id\n  - age_level\n  - occupation_id\n  - star_level\n\n- **dim_shop**\n  - shop_id (PK)\n  - review_num_level\n  - review_positive_rate\n  - star_level\n  - score_service\n  - score_delivery\n  - score_description\n\n## Pipeline\nNote that file paths need to be adjusted to the local environment.\n```bash\n# Clean data\npython src/datacleaner.py\n\n# Generate fact table\npython src/fact_sales_load.py\n\n# Load database\nmysql -u root -p \u003c sql/dataloader.sql\nmysql -u root -p \u003c sql/transform_data_for_dashboard.sql\n\n\n# Launch dashboard\nstreamlit run src/dashboard.py\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkirbytes%2Fecommerce-analytic-platform","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkirbytes%2Fecommerce-analytic-platform","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkirbytes%2Fecommerce-analytic-platform/lists"}