{"id":30933929,"url":"https://github.com/shakeel-data/amazon-sales-forecasting-python-bigquery-ml","last_synced_at":"2026-05-09T05:34:23.118Z","repository":{"id":311624038,"uuid":"1042802831","full_name":"shakeel-data/amazon-sales-forecasting-python-bigquery-ml","owner":"shakeel-data","description":"An end-to-end analytics project using Python, SQL, \u0026 ML to forecast Amazon sales and segment customers. We build predictive models (LightGBM, Prophet) and clustering (KMeans) to deliver actionable insights for revenue growth and targeted marketing.","archived":false,"fork":false,"pushed_at":"2025-09-28T13:23:36.000Z","size":7984,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-28T15:42:23.505Z","etag":null,"topics":["bigquery","kmeans-clustring","lightgbm","linear-regression","prophet-facebook","scikit-learn"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/shakeel-data.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-08-22T15:46:21.000Z","updated_at":"2025-09-28T13:23:39.000Z","dependencies_parsed_at":"2025-08-27T21:32:54.324Z","dependency_job_id":null,"html_url":"https://github.com/shakeel-data/amazon-sales-forecasting-python-bigquery-ml","commit_stats":null,"previous_names":["shakeel-data/amazon-sales-forecasting","shakeel-data/amazon-sales-forecasting-python-bigquery-ml"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/shakeel-data/amazon-sales-forecasting-python-bigquery-ml","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shakeel-data%2Famazon-sales-forecasting-python-bigquery-ml","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shakeel-data%2Famazon-sales-forecasting-python-bigquery-ml/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shakeel-data%2Famazon-sales-forecasting-python-bigquery-ml/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shakeel-data%2Famazon-sales-forecasting-python-bigquery-ml/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/shakeel-data","download_url":"https://codeload.github.com/shakeel-data/amazon-sales-forecasting-python-bigquery-ml/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shakeel-data%2Famazon-sales-forecasting-python-bigquery-ml/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32808534,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-08T08:22:46.396Z","status":"online","status_checked_at":"2026-05-09T02:00:06.633Z","response_time":123,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bigquery","kmeans-clustring","lightgbm","linear-regression","prophet-facebook","scikit-learn"],"created_at":"2025-09-10T15:01:39.313Z","updated_at":"2026-05-09T05:34:23.102Z","avatar_url":"https://github.com/shakeel-data.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 📦 Data-Driven Amazon Sales Forecasting \u0026 Insights | Python, BigQuery, ML\n\n\u003cimg width=\"2048\" height=\"2048\" alt=\"Google_AI_Studio_2025-08-26T16_55_52 752Z new\" src=\"https://github.com/user-attachments/assets/86e5a2b1-37d4-41ae-8b67-67dd6c5f8159\" /\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://www.vectorlogo.zone/logos/python/python-icon.svg\" width=\"40\"/\u003e\n  \u003cimg src=\"https://www.vectorlogo.zone/logos/google_bigquery/google_bigquery-icon.svg\" width=\"40\"/\u003e\n  \u003cimg src=\"https://upload.wikimedia.org/wikipedia/commons/thumb/0/05/Scikit_learn_logo_small.svg/320px-Scikit_learn_logo_small.svg.png\" width=\"60\" alt=\"KMeans\"/\u003e\n  \u003cimg src=\"https://upload.wikimedia.org/wikipedia/commons/thumb/b/b2/SCIPY_2.svg/256px-SCIPY_2.svg.png\" width=\"40\" alt=\"Linear Regression\"/\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/microsoft/LightGBM/master/docs/logo/LightGBM_logo_black_text.svg\" width=\"150\" alt=\"LightGBM\"/\u003e\n  \u003cimg src=\"https://cdn.jsdelivr.net/gh/devicons/devicon/icons/facebook/facebook-original.svg\" width=\"40\" alt=\"Facebook Prophet\"/\u003e\n  \u003cimg src=\"https://github.com/user-attachments/assets/f0afa392-f2ae-4df3-9c0e-20d7a90a72c4\" width=\"40\" alt=\"KMeans Clustering\"/\u003e\n\u003c/p\u003e\n\nAmazon Sales Forecasting is crucial for accurately predicting future customer demand. This allows for optimized inventory management, preventing costly overstocking and lost sales from stockouts. As a result, businesses can significantly reduce waste, improve cash flow, and maximize profitability. Simultaneously, Customer Analytics uncovers distinct purchasing behaviors and segments the customer base. This enables highly targeted marketing campaigns and personalized experiences, boosting customer loyalty and satisfaction. Ultimately, this dual approach empowers data-driven strategic decisions that drive sustainable growth.\n\n## 📋 Project Overview\n\nThis project provides a full-stack analytics solution, beginning with raw data ingestion and concluding with actionable business strategies. It showcases a robust workflow that includes data cleaning, database normalization, advanced SQL querying, and the implementation of multiple machine learning models for both supervised and unsupervised tasks. The primary goal is to unlock data-driven insights to forecast future sales, understand customer behavior, and guide strategic decision-making.\n\n## 🎯 Business Objectives\n- **Forecast Future Sales**: Predict revenue trends using multiple robust ML models.\n- **Segment Customers**: Identify distinct customer personas based on purchasing behavior to enable targeted marketing.\n- **Analyze Performance**: Uncover sales patterns, seasonal impacts, and key growth drivers.\n- **Generate Strategic Insights**: Translate complex data into clear, actionable recommendations for business growth.\n\n## 📁 Data Sources\n- Kaggle\n  \u003ca href=\"https://github.com/shakeel-data/amazon-sales-forecasting/blob/main/Amazon_foodcategory_sales.csv\"\u003ecsv\u003c/a\u003e\n- Clean\n  - \u003ca href=\"https://github.com/shakeel-data/amazon-sales-forecasting/blob/main/customers.csv\"\u003ecustomers.csv\u003c/a\u003e\n  - \u003ca href=\"https://github.com/shakeel-data/amazon-sales-forecasting/blob/main/orders.csv\"\u003eorders.csv\u003c/a\u003e\n  - \u003ca href=\"https://github.com/shakeel-data/amazon-sales-forecasting/blob/main/products.csv\"\u003eproducts.csv\u003c/a\u003e\n  - \u003ca href=\"https://github.com/shakeel-data/amazon-sales-forecasting/blob/main/sales.csv\"\u003esales.csv\u003c/a\u003e\n- SQL\n  \u003ca href=\"https://github.com/shakeel-data/amazon-sales-forecasting/blob/main/Amazon-sales.sql\"\u003equeries\u003c/a\u003e\n- Python\n  \u003ca href=\"https://github.com/shakeel-data/amazon-sales-forecasting/blob/main/Amazon_Sales_Forecasting.ipynb\"\u003ecodes\u003c/a\u003e\n- Customer segment\n\u003ca href=\"https://github.com/shakeel-data/amazon-sales-forecasting/blob/main/customer_segments.csv\"\u003ecsv\u003c/a\u003e\n\n### Prerequisites\n- Python 3.8+\n- Access to a Google Cloud Platform (GCP) project with BigQuery enabled.\n- A GCP service account key (`.json` file) with BigQuery User \u0026 Data Editor roles.\n\n### Installation \u0026 Execution\n**Clone the repository:**\n```\ngit clone https://github.com/shakeel-data/amazon-sales-forecasting.git\ncd amazon-sales-forecasting\n```\n\n---\n## 📊 Project Breakdown – Simple Steps\nThe project follows a structured, multi-stage workflow designed to transform raw data into high-value business intelligence.\n\n| 🔢 Step | 🚀 Stage                       | 📝 Description                                                                                                     |\n|--------|------------------------------|-----------------------------------------------------------------------------------------------------------------|\n| 1      | **Data Ingestion \u0026 Cleaning** | Load raw CSV, handle missing values, correct data types, and assess quality using Python.                       |\n| 2      | **Database Normalization**    | Convert flat files into a **relational schema** (customers, products, orders, sales) for integrity \u0026 efficiency.|\n| 3      | **BigQuery Integration**      | Upload normalized tables via manually or python to **Google BigQuery** to serve as the single source of truth.                         |\n| 4      | **Advanced SQL Analysis**     | Run 20+ **SQL queries** for cohort analysis, RFM scores, and deep business insights.                           |\n| 5      | **Machine Learning Modeling** | - **Forecasting:** Linear Regression, LightGBM, Prophet\u003cbr\u003e - **Segmentation:** KMeans for customer groups |\n| 6      | **Insight \u0026 Strategy**        | Translate analytical findings into **strategic recommendations** for growth.                                   |\n---\n\n## 🔧 Project Workflow\n### 📥 Load Packages and Data Ingestion\n```python\nimport pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport lightgbm as lgb\n\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.linear_model import LinearRegression\nfrom sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score\n\nfrom sklearn.cluster import KMeans\nfrom sklearn.preprocessing import StandardScaler\nfrom sklearn.decomposition import PCA\n\nfrom prophet import Prophet\nfrom datetime import datetime\nimport warnings\nwarnings.filterwarnings('ignore')\n\n# For better model evaluation and visualization\nfrom sklearn.model_selection import cross_val_score, TimeSeriesSplit\nfrom sklearn.metrics import mean_absolute_percentage_error\n\n# For feature engineering (very useful for sales forecasting)\nfrom sklearn.preprocessing import LabelEncoder, PolynomialFeatures\n\n# For saving models and results\nimport pickle\nimport joblib\n\n# For enhanced plotting and statistical analysis\nimport plotly.express as px  # Interactive plots (optional)\nimport plotly.graph_objects as go  # Interactive plots (optional)\nfrom scipy import stats\n\n# Set random seed for reproducibility\nnp.random.seed(42)\n\n# Configure plotting style\nplt.style.use('seaborn-v0_8')  # Modern seaborn style\nplt.rcParams['figure.figsize'] = (12, 8)\nplt.rcParams['font.size'] = 11\n\nprint(\"All dependencies loaded successfully!\")\n```\n### Load and inspect the dataset\n```python\ndf = pd.read_csv('your-file-path')\n\nprint(f\"Dataset Shape: {df.shape}\")\n```\n\u003cimg width=\"1484\" height=\"39\" alt=\"image\" src=\"https://github.com/user-attachments/assets/c7a5e4e5-d874-463a-857e-7adc23e7a4ea\" /\u003e\n\n### Basic dataset information\n```python\nprint(\"Column Names and Data Types:\")\nprint(df.dtypes)\nprint(\"\\nFirst 5 rows:\")\ndf.head()\n```\n\u003cimg width=\"1500\" height=\"511\" alt=\"image\" src=\"https://github.com/user-attachments/assets/8e0778df-70b2-4e57-9ab4-251de4013dde\" /\u003e\n\n| Custkey  | DateKey    | Discount Amount | Invoice Date | Invoice Number | Item Class | Item Number | Item                      | Line Number | List Price | Order Number | Promised Delivery Date | Sales Amount | Sales Amount Based on List Price | Sales Cost Amount | Sales Margin Amount | Sales Price | Sales Quantity | Sales Rep | U/M |\n|----------|------------|-----------------|--------------|----------------|------------|-------------|---------------------------|-------------|------------|--------------|------------------------|--------------|----------------------------------|------------------|---------------------|-------------|----------------|-----------|-----|\n| 10016609 | 12/31/2019 | 398.73          | 2019/12/31   | 329568         | P01        | 15640       | Super Vegetable Oil       | 1000        | 163.47     | 122380       | 12/31/2019             | 418.62       | 817.35                           | 102.99           | 315.63              | 83.72400    | 5              | 176       | EA  |\n| 10016609 | 12/31/2019 | 268.67          | 2019/12/31   | 329569         | P01        | 31681       | Golden Fajita French Fries| 7000        | 275.37     | 123966       | 12/31/2019             | 282.07       | 550.74                           | 117.45           | 164.62              | 141.03500   | 2              | 176       | EA  |\n| 10016609 | 12/31/2019 | 398.73          | 2019/12/31   | 329569         | P01        | 15640       | Super Vegetable Oil       | 4000        | 163.47     | 123966       | 12/31/2019             | 418.62       | 817.35                           | 102.99           | 315.63              | 83.72400    | 5              | 176       | EA  |\n| 10016609 | 12/31/2019 | 466.45          | 2019/12/31   | 329569         | P01        | 13447       | High Top Oranges          | 3000        | 119.52     | 123966       | 12/31/2019             | 489.71       | 956.16                           | 213.29           | 276.42              | 61.21375    | 8              | 176       | EA  |\n| 10016609 | 12/31/2019 | 515.51          | 2019/12/31   | 329569         | P01        | 36942       | Tell Tale New Potatos     | 9000        | 264.18     | 123966       | 12/31/2019             | 541.21       | 1056.72                          | 290.56           | 250.65              | 135.30250   | 4              | 176       | EA  |\n\n\n### Data quality assessment\n```python\nprint(\"Missing Values:\")\nmissing_data = df.isnull().sum()\nmissing_percent = (missing_data / len(df)) * 100\nmissing_info = pd.DataFrame({\n    'Missing Count': missing_data,\n    'Missing %': missing_percent\n})\nprint(missing_info[missing_info['Missing Count'] \u003e 0])\n\nprint(f\"\\nDuplicate Rows: {df.duplicated().sum()}\")\nprint(f\"Unique Invoice Numbers: {df['Invoice Number'].nunique()}\")\nprint(f\"Total Rows: {len(df)}\")\n```\n\u003cimg width=\"1566\" height=\"230\" alt=\"image\" src=\"https://github.com/user-attachments/assets/87212729-b685-4b05-ac07-b5fc30abb614\" /\u003e\n\n### Create additional features for analysis\n```python\n# Extract date components\ndf['Year'] = df['Invoice Date'].dt.year\ndf['Month'] = df['Invoice Date'].dt.month\ndf['Quarter'] = df['Invoice Date'].dt.quarter\ndf['Day_of_Week'] = df['Invoice Date'].dt.dayofweek\ndf['Month_Year'] = df['Invoice Date'].dt.to_period('M')\n\n# Calculate profit margin percentage\ndf['Profit_Margin_Pct'] = ((df['Sales Amount'] - df['Sales Cost Amount']) / df['Sales Amount']) * 100\n\n# Calculate delivery days\ndf['Delivery_Days'] = (df['Promised Delivery Date'] - df['Invoice Date']).dt.days\n\nprint(\"Feature engineering completed!\")\n```\n\n## Exploratory Data Analysis (EDA)\n```python\n# Set up the plotting area\nfig, axes = plt.subplots(2, 2, figsize=(15, 12))\nfig.suptitle('Amazon Food Category Sales - Overview Dashboard', fontsize=16, fontweight='bold')\n\n# 1. Sales Trend Over Time\nmonthly_sales = df.groupby('Month_Year')['Sales Amount'].sum().reset_index()\nmonthly_sales['Month_Year'] = monthly_sales['Month_Year'].astype(str)\n\naxes[0,0].plot(monthly_sales['Month_Year'], monthly_sales['Sales Amount'],\n               marker='o', linewidth=2, markersize=6)\naxes[0,0].set_title('Monthly Sales Trend', fontweight='bold')\naxes[0,0].set_xlabel('Month')\naxes[0,0].set_ylabel('Sales Amount ($)')\naxes[0,0].tick_params(axis='x', rotation=45)\naxes[0,0].grid(True, alpha=0.3)\n\n# 2. Top 10 Products by Sales\ntop_products = df.groupby('Item')['Sales Amount'].sum().nlargest(10)\naxes[0,1].barh(range(len(top_products)), top_products.values)\naxes[0,1].set_yticks(range(len(top_products)))\naxes[0,1].set_yticklabels([item[:20] + '...' if len(item) \u003e 20 else item for item in top_products.index])\naxes[0,1].set_title('Top 10 Products by Sales', fontweight='bold')\naxes[0,1].set_xlabel('Sales Amount ($)')\n\n# 3. Sales by Item Class\nclass_sales = df.groupby('Item Class')['Sales Amount'].sum().sort_values(ascending=False)\naxes[1,0].pie(class_sales.values, labels=class_sales.index, autopct='%1.1f%%', startangle=90)\naxes[1,0].set_title('Sales Distribution by Item Class', fontweight='bold')\n\n# 4. Sales Rep Performance\nrep_performance = df.groupby('Sales Rep')['Sales Amount'].sum().sort_values(ascending=False).head(10)\naxes[1,1].bar(range(len(rep_performance)), rep_performance.values)\naxes[1,1].set_xticks(range(len(rep_performance)))\naxes[1,1].set_xticklabels(rep_performance.index, rotation=45, ha='right')\naxes[1,1].set_title('Top 10 Sales Rep Performance', fontweight='bold')\naxes[1,1].set_ylabel('Sales Amount ($)')\n\nplt.tight_layout()\nplt.show()\n```\n\u003cimg width=\"1488\" height=\"1180\" alt=\"image\" src=\"https://github.com/user-attachments/assets/3a01d229-4701-439a-9f02-428b65cb6b19\" /\u003e\n\n### Correlation Heatmap\n```python\n# 5. Correlation Heatmap\nplt.figure(figsize=(12, 12))\nnumeric_cols = df.select_dtypes(include=[np.number]).columns\ncorrelation_matrix = df[numeric_cols].corr()\n\nsns.heatmap(correlation_matrix, annot=True, cmap='coolwarm', center=0,\n            square=True, linewidths=0.5)\nplt.title('Correlation Matrix - Numeric Variables', fontweight='bold', pad=20)\nplt.tight_layout()\nplt.show()\n\nprint(\"Key Insight: Strong correlation between Sales Amount and List Price indicates pricing strategy effectiveness.\")\n```\n\u003cimg width=\"1144\" height=\"1144\" alt=\"image\" src=\"https://github.com/user-attachments/assets/0d52bba1-855e-425b-88ca-645b15129820\" /\u003e\n\n```python\n# 6. Seasonal Analysis\nplt.figure(figsize=(15, 5))\n\n# Monthly sales pattern\nplt.subplot(1, 3, 1)\nmonthly_avg = df.groupby('Month')['Sales Amount'].mean()\nplt.bar(monthly_avg.index, monthly_avg.values)\nplt.title('Average Sales by Month', fontweight='bold')\nplt.xlabel('Month')\nplt.ylabel('Average Sales ($)')\n\n# Quarterly sales\nplt.subplot(1, 3, 2)\nquarterly_sales = df.groupby('Quarter')['Sales Amount'].sum()\nplt.bar(['Q1', 'Q2', 'Q3', 'Q4'], quarterly_sales.values)\nplt.title('Sales by Quarter', fontweight='bold')\nplt.xlabel('Quarter')\nplt.ylabel('Total Sales ($)')\n\n# Day of week pattern\nplt.subplot(1, 3, 3)\ndow_sales = df.groupby('Day_of_Week')['Sales Amount'].mean()\ndays = ['Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat', 'Sun']\nplt.bar(days, dow_sales.values)\nplt.title('Average Sales by Day of Week', fontweight='bold')\nplt.xlabel('Day of Week')\nplt.ylabel('Average Sales ($)')\n\nplt.tight_layout()\nplt.show()\n\nprint(\"Key Insight: Clear seasonal patterns detected - Q4 shows highest sales, likely due to holiday shopping.\")\n```\n\u003cimg width=\"1489\" height=\"490\" alt=\"image\" src=\"https://github.com/user-attachments/assets/c9b0264e-04a6-44ac-97af-b7ab341a7e67\" /\u003e\n\n\n```python\n# 7. Profit Margin Analysis\nplt.figure(figsize=(12, 6))\n\nplt.subplot(1, 2, 1)\n# Profit margin distribution\nplt.hist(df['Profit_Margin_Pct'].dropna(), bins=30, alpha=0.7, edgecolor='black')\nplt.title('Profit Margin Distribution', fontweight='bold')\nplt.xlabel('Profit Margin (%)')\nplt.ylabel('Frequency')\nplt.axvline(df['Profit_Margin_Pct'].mean(), color='red', linestyle='--',\n            label=f'Mean: {df[\"Profit_Margin_Pct\"].mean():.1f}%')\nplt.legend()\n\nplt.subplot(1, 2, 2)\n# Profit margin by product class\nclass_margin = df.groupby('Item Class')['Profit_Margin_Pct'].mean().sort_values(ascending=False)\nplt.bar(range(len(class_margin)), class_margin.values)\nplt.xticks(range(len(class_margin)), class_margin.index, rotation=45, ha='right')\nplt.title('Average Profit Margin by Item Class', fontweight='bold')\nplt.ylabel('Profit Margin (%)')\n\nplt.tight_layout()\nplt.show()\n\nprint(\"Key Insight: Average profit margin is healthy, but varies significantly by product class.\")\n```\n\u003cimg width=\"1189\" height=\"590\" alt=\"image\" src=\"https://github.com/user-attachments/assets/1febaa5f-ed12-44b1-8eab-0c3b4fda8e4c\" /\u003e\n\n## Data Normalization and SQL Preparation\n```python\n# Create normalized tables for SQL analysis\nfrom faker import Faker\nfake = Faker()\n\nprint(\"CREATING NORMALIZED DATABASE TABLES\")\nprint(\"=\" * 45)\n\n# 1. Customers Table\nprint(\"Creating customers table with fake names...\")\ncustomers = df[['Custkey']].drop_duplicates().reset_index(drop=True)\ncustomers['customer_name'] = [fake.name() for _ in range(len(customers))]\ncustomers['email'] = [fake.email() for _ in range(len(customers))]\ncustomers['phone'] = [fake.phone_number() for _ in range(len(customers))]\ncustomers['address'] = [fake.address().replace('\\n', ', ') for _ in range(len(customers))]\ncustomers['registration_date'] = [fake.date_between(start_date='-2y', end_date='today') for _ in range(len(customers))]\n\nprint(f\"Customers table: {customers.shape}\")\n\n# 2. Products Table\nprint(\"Creating products table...\")\nproducts = df[['Item Number', 'Item', 'Item Class', 'List Price', 'U/M']].drop_duplicates().reset_index(drop=True)\nproducts.columns = ['product_id', 'product_name', 'category', 'list_price', 'unit_measure']\n\nprint(f\"Products table: {products.shape}\")\n\n# 3. Orders Table\nprint(\"Creating orders table...\")\norders = df[['Order Number', 'Custkey', 'Invoice Date', 'Promised Delivery Date', 'Sales Rep']].drop_duplicates().reset_index(drop=True)\norders.columns = ['order_id', 'customer_id', 'order_date', 'promised_delivery', 'sales_rep']\n\nprint(f\"Orders table: {orders.shape}\")\n\n# 4. Sales Table (fact table)\nprint(\"Creating sales table...\")\nsales = df[['Invoice Number', 'Order Number', 'Item Number', 'Line Number',\n             'Sales Quantity', 'Sales Price', 'Sales Amount', 'Discount Amount',\n             'Sales Cost Amount', 'Sales Margin Amount']].copy()\nsales.columns = ['invoice_number', 'order_id', 'product_id', 'line_number',\n                  'quantity', 'unit_price', 'sales_amount', 'discount_amount',\n                  'cost_amount', 'margin_amount']\n\nprint(f\"Sales table: {sales.shape}\")\n\n# Save all tables\nimport os\nos.makedirs('data/processed', exist_ok=True)\ncustomers.to_csv('data/processed/customers.csv', index=False)\nproducts.to_csv('data/processed/products.csv', index=False)\norders.to_csv('data/processed/orders.csv', index=False)\nsales.to_csv('data/processed/sales.csv', index=False)\n\nprint(\"\\n All normalized tables created and saved!\")\n```\n\u003cimg width=\"1716\" height=\"281\" alt=\"image\" src=\"https://github.com/user-attachments/assets/3ad32c3a-ab6b-4696-8d59-06efbba09f32\" /\u003e\n\n## 📦 BigQuery Integration\n### Solution 1\n**Set up GCP Credentials:**\nPlace your service account key `.json` file in the root directory and update the notebook code to reference its path.\n```python\nfrom google.cloud import bigquery\nimport os\n\n# 1. SETUP (Update these 2 lines with your info)\nos.environ['GOOGLE_APPLICATION_CREDENTIALS'] = 'your-json-file-path'\nclient = bigquery.Client(project='your-project-id')\n\n# 2. SIMPLE FUNCTIONS\ndef create_dataset():\n    dataset_id = 'your-dataset-id'\n    try:\n        client.get_dataset(dataset_id)\n        print(\"Dataset exists\")\n    except:\n        dataset = bigquery.Dataset(f\"{client.project}.{dataset_id}\")\n        dataset.location = \"US\"\n        client.create_dataset(dataset, exists_ok=True)\n        print(\"Dataset created\")\n\ndef upload_table(df, table_name):\n    table_id = f\"{client.project}.your-dataset-id.{table_name}\"\n    job_config = bigquery.LoadJobConfig(\n        write_disposition=\"WRITE_TRUNCATE\",\n        autodetect=True\n    )\n    job = client.load_table_from_dataframe(df, table_id, job_config=job_config)\n    job.result()\n    print(f\" {table_name}: {len(df)} rows uploaded\")\n\n# 3. RUN IT\nprint(\"Uploading to BigQuery...\")\n\n# Create dataset\ncreate_dataset()\n\n# Upload your 4 tables (these should already exist from your previous code)\nupload_table(customers, 'customers')\nupload_table(products, 'products') \nupload_table(orders, 'orders')\nupload_table(sales, 'sales')\n\nprint(\"\\n ALL DONE! Your tables are in BigQuery!\")\n\n# 4. TEST IT\ntest_query = \"\"\"\nSELECT COUNT(*) as customer_count \nFROM `your-project-id.your-dataset-id.customers`\n\"\"\"\n\nresult = client.query(test_query).to_dataframe()\nprint(f\"Test: Found {result.iloc[0]['customer_count']} customers\")\n```\n\n### Solution 2\nUpload the normalized 4 tables manually to Google BigQuery to serve as the single source of truth for all subsequent analysis\n\n## 🗄️ Database Schema\n\nThe initial flat CSV was normalized into a relational star schema to improve query performance and maintain data integrity.\n\n- **`customers` (Dimension)**: `customer_id` (PK), `customer_name`, `email`, `registration_date`\n- **`products` (Dimension)**: `product_id` (PK), `product_name`, `category`, `list_price`\n- **`orders` (Dimension)**: `order_id` (PK), `customer_id` (FK), `order_date`, `sales_rep`\n- **`sales` (Fact)**: `invoice_number` (PK), `order_id` (FK), `product_id` (FK), `quantity`, `sales_amount`\n\n## ⚙️ SQL Analysis Showcase\n\nThis project features over 20 advanced SQL queries. \nBelow are key highlights:\n### Customer Overview\n**Purpose: Basic customer information with aggregated metrics**\n```sql\nSELECT \n    customer_name,\n    email,\n    phone,\n    registration_date,\n    -- Calculate days since registration\n    DATE_DIFF(CURRENT_DATE(), registration_date, DAY) as days_as_customer\nFROM `your-project-id.your-dataset-id.customers`\nORDER BY registration_date DESC\nLIMIT 10;\n```\n\n### Product Catalog Summary\n**Purpose: Overview of products with pricing information**\n```sql\nSELECT \n    category,\n    product_name,\n    list_price,\n    unit_measure,\n    -- Price categories for analysis\n    CASE \n        WHEN list_price \u003c 50 THEN 'Budget'\n        WHEN list_price BETWEEN 50 AND 200 THEN 'Mid-Range'\n        WHEN list_price \u003e 200 THEN 'Premium'\n        ELSE 'Unknown'\n    END as price_category\nFROM `your-project-id.your-dataset-id.products`\nWHERE list_price IS NOT NULL\nORDER BY list_price DESC;\n```\n### Sales Performance by Month\n**Purpose: Track sales trends over time**\n```sql\nSELECT\n  EXTRACT(YEAR FROM o.order_date) AS year,\n  EXTRACT(MONTH FROM o.order_date) AS month,\n  COUNT(DISTINCT o.order_id) AS total_orders,\n  ROUND(SUM(s.sales_amount), 2) AS total_revenue,\n  ROUND(AVG(s.sales_amount), 2) AS avg_order_value\nFROM\n  `your-project-id.your-dataset-id.orders` AS o\n  JOIN\n  `your-project-id.your-dataset-id.sales` AS s\n  ON o.order_id = s.order_id\nGROUP BY year, month\nORDER BY year DESC, month DESC;\n```\n\n## JOIN Queries\n### Customer Purchase History (INNER JOIN)\n**Purpose: Show customers who have made purchases**\n```sql\nSELECT \n    c.customer_name,\n    c.email,\n    COUNT(DISTINCT o.order_id) as total_orders,\n    ROUND(SUM(s.sales_amount), 2) as lifetime_value,\n    MIN(o.order_date) as first_purchase,\n    MAX(o.order_date) as last_purchase\nFROM `your-project-id.your-dataset-id.customers` c\nINNER JOIN `your-project-id.your-dataset-id.orders` o ON c.Custkey = o.customer_id\nINNER JOIN `your-project-id.your-dataset-id.sales` s ON o.order_id = s.order_id\nGROUP BY c.customer_name, c.email\nORDER BY lifetime_value DESC\nLIMIT 20;\n```\n\n### All Customers with Purchase Status (LEFT JOIN)\n**Purpose: Include customers who haven't made purchases**\n```sql\nSELECT \n    c.customer_name,\n    c.email,\n    c.registration_date,\n    COALESCE(COUNT(DISTINCT o.order_id), 0) as total_orders,\n    COALESCE(ROUND(SUM(s.sales_amount), 2), 0) as lifetime_value,\n    CASE \n        WHEN o.customer_id IS NULL THEN 'No Purchases'\n        WHEN COUNT(DISTINCT o.order_id) = 1 THEN 'Single Purchase'\n        ELSE 'Repeat Customer'\n    END as customer_status\nFROM `your-project-id.your-dataset-id.customers` c\nLEFT JOIN `your-project-id.your-dataset-id.orders` o ON c.Custkey = o.customer_id\nLEFT JOIN `your-project-id.your-dataset-id.sales` s ON o.order_id = s.order_id\nGROUP BY c.customer_name, c.email, c.registration_date, o.customer_id\nORDER BY lifetime_value DESC;\n```\n\n### Product Performance Analysis (RIGHT JOIN)\n**Show all products with their sales performance**\n```sql\nSELECT \n    p.product_name,\n    p.category,\n    p.list_price,\n    COALESCE(COUNT(s.product_id), 0) as times_sold,\n    COALESCE(ROUND(SUM(s.sales_amount), 2), 0) as total_revenue,\n    COALESCE(ROUND(AVG(s.unit_price), 2), p.list_price) as avg_selling_price\nFROM `your-project-id.your-dataset-id.sales` s\nRIGHT JOIN `your-project-id.your-dataset-id.products` p ON s.product_id = p.product_id\nGROUP BY p.product_name, p.category, p.list_price\nORDER BY total_revenue DESC;\n```\n\n### Complete Order Details (FULL OUTER JOIN)\n**Purpose: Comprehensive view of all orders and potential data gaps**\n```sql\nSELECT \n    COALESCE(o.order_id, s.order_id) as order_id,\n    c.customer_name,\n    o.order_date,\n    p.product_name,\n    s.quantity,\n    s.unit_price,\n    s.sales_amount,\n    CASE \n        WHEN o.order_id IS NULL THEN 'Missing Order Info'\n        WHEN s.order_id IS NULL THEN 'Missing Sales Info'\n        ELSE 'Complete'\n    END as data_quality\nFROM `your-project-id.your-dataset-id.orders` o\nFULL OUTER JOIN `your-project-id.your-dataset-id.sales` s ON o.order_id = s.order_id\nLEFT JOIN `your-project-id.your-dataset-id.customers` c ON o.customer_id = c.Custkey\nLEFT JOIN `your-project-id.your-dataset-id.products` p ON s.product_id = p.product_id\nWHERE COALESCE(o.order_id, s.order_id) IS NOT NULL\nORDER BY COALESCE(o.order_date, '1900-01-01') DESC;\n```\n\n## Window Functions\n### Customer Ranking by Revenue\n**Purpose: Rank customers and show revenue percentiles**\n```sql\nSELECT \n    c.customer_name,\n    ROUND(SUM(s.sales_amount), 2) as total_revenue,\n    -- Ranking functions\n    ROW_NUMBER() OVER (ORDER BY SUM(s.sales_amount) DESC) as revenue_rank,\n    DENSE_RANK() OVER (ORDER BY SUM(s.sales_amount) DESC) as dense_rank,\n    -- Percentile calculation\n    PERCENT_RANK() OVER (ORDER BY SUM(s.sales_amount)) as revenue_percentile,\n    -- Revenue quartiles\n    NTILE(4) OVER (ORDER BY SUM(s.sales_amount)) as revenue_quartile\nFROM `your-project-id.your-dataset-id.customers` c\nJOIN `your-project-id.your-dataset-id.orders` o ON c.Custkey = o.customer_id\nJOIN `your-project-id.your-dataset-id.sales` s ON o.order_id = s.order_id\nGROUP BY c.customer_name\nORDER BY total_revenue DESC;\n```\n\n### Running Totals and Moving Averages\n**Purpose: Calculate cumulative sales and trends**\n```sql\nSELECT \n    order_date,\n    COUNT(DISTINCT order_id) as daily_orders,\n    ROUND(SUM(daily_revenue), 2) as daily_revenue,\n    -- Running totals\n    SUM(COUNT(DISTINCT order_id)) OVER (\n        ORDER BY order_date \n        ROWS UNBOUNDED PRECEDING\n    ) as cumulative_orders,\n    SUM(SUM(daily_revenue)) OVER (\n        ORDER BY order_date \n        ROWS UNBOUNDED PRECEDING\n    ) as cumulative_revenue,\n    -- 7-day moving average\n    ROUND(AVG(SUM(daily_revenue)) OVER (\n        ORDER BY order_date \n        ROWS BETWEEN 6 PRECEDING AND CURRENT ROW\n    ), 2) as revenue_7day_avg\nFROM (\n    SELECT \n        o.order_date,\n        o.order_id,\n        SUM(s.sales_amount) as daily_revenue\n    FROM `your-project-id.your-dataset-id.orders` o\n    JOIN `your-project-id.your-dataset-id.sales` s ON o.order_id = s.order_id\n    GROUP BY o.order_date, o.order_id\n) daily_data\nGROUP BY order_date\nORDER BY order_date;\n```\n\n### Product Sales Comparison\n**Purpose: Compare each product's performance to category average**\n```sql\nSELECT \n    p.category,\n    p.product_name,\n    COUNT(s.product_id) as units_sold,\n    ROUND(SUM(s.sales_amount), 2) as product_revenue,\n    -- Compare to category averages\n    ROUND(AVG(SUM(s.sales_amount)) OVER (PARTITION BY p.category), 2) as category_avg_revenue,\n    ROUND(SUM(s.sales_amount) - AVG(SUM(s.sales_amount)) OVER (PARTITION BY p.category), 2) as revenue_vs_category_avg,\n    -- Rank within category\n    ROW_NUMBER() OVER (PARTITION BY p.category ORDER BY SUM(s.sales_amount) DESC) as category_rank\nFROM `your-project-id.your-dataset-id.products` p\nJOIN `your-project-id.your-dataset-id.sales` s ON p.product_id = s.product_id\nGROUP BY p.category, p.product_name\nORDER BY p.category, product_revenue DESC;\n```\n\n## CTEs and Subqueries\n### Customer Segmentation with CTE\n**Purpose: Segment customers using RFM analysis**\n```sql\nWITH customer_metrics AS (\n    SELECT \n        c.Custkey, \n        c.customer_name,\n        -- Recency: Days since last purchase\n        DATE_DIFF(CURRENT_DATE(), MAX(o.order_date), DAY) as days_since_last_purchase,\n        -- Frequency: Number of orders\n        COUNT(DISTINCT o.order_id) as total_orders,\n        -- Monetary: Total spent\n        ROUND(SUM(s.sales_amount), 2) as total_spent\n    FROM `your-project-id.your-dataset-id.customers` c\n    JOIN `your-project-id.your-dataset-id.orders` o ON c.Custkey = o.customer_id\n    JOIN `your-project-id.your-dataset-id.sales` s ON o.order_id = s.order_id\n    GROUP BY c.Custkey, c.customer_name\n),\nrfm_scores AS (\n    SELECT \n        *,\n        -- RFM Scoring (1-5 scale)\n        NTILE(5) OVER (ORDER BY days_since_last_purchase DESC) as recency_score,\n        NTILE(5) OVER (ORDER BY total_orders) as frequency_score,\n        NTILE(5) OVER (ORDER BY total_spent) as monetary_score\n    FROM customer_metrics\n)\nSELECT \n    customer_name,\n    days_since_last_purchase,\n    total_orders,\n    total_spent,\n    recency_score,\n    frequency_score,\n    monetary_score,\n    -- Customer segments based on RFM\n    CASE \n        WHEN recency_score \u003e= 4 AND frequency_score \u003e= 4 AND monetary_score \u003e= 4 THEN 'Champions'\n        WHEN recency_score \u003e= 3 AND frequency_score \u003e= 3 AND monetary_score \u003e= 3 THEN 'Loyal Customers'\n        WHEN recency_score \u003e= 3 AND frequency_score \u003c= 2 THEN 'Potential Loyalists'\n        WHEN recency_score \u003c= 2 AND frequency_score \u003e= 3 THEN 'At Risk'\n        WHEN recency_score \u003c= 2 AND frequency_score \u003c= 2 AND monetary_score \u003e= 3 THEN 'Cannot Lose Them'\n        ELSE 'Others'\n    END as customer_segment\nFROM rfm_scores\nORDER BY monetary_score DESC, frequency_score DESC, recency_score DESC;\n```\n\n### Top Products by Category (Subqueries)\n**Purpose: Find best-selling products in each category**\n```sql\nSELECT \n    category,\n    product_name,\n    total_revenue,\n    units_sold,\n    category_rank\nFROM (\n    SELECT \n        p.category,\n        p.product_name,\n        ROUND(SUM(s.sales_amount), 2) as total_revenue,\n        SUM(s.quantity) as units_sold,\n        ROW_NUMBER() OVER (PARTITION BY p.category ORDER BY SUM(s.sales_amount) DESC) as category_rank\n    FROM `your-project-id.your-dataset-id.products` p\n    JOIN `your-project-id.your-dataset-id.sales` s ON p.product_id = s.product_id\n    GROUP BY p.category, p.product_name\n) ranked_products\nWHERE category_rank \u003c= 3  -- Top 3 products per category\nORDER BY category, category_rank;\n```\n\n### Monthly Growth Analysis with CTE\n**Purpose: Calculate month-over-month growth rates**\n```sql\nWITH monthly_sales AS (\n    SELECT \n        EXTRACT(YEAR FROM o.order_date) as year,\n        EXTRACT(MONTH FROM o.order_date) as month,\n        COUNT(DISTINCT o.order_id) as orders,\n        ROUND(SUM(s.sales_amount), 2) as revenue\n    FROM `your-project-id.your-dataset-id.orders` o\n    JOIN `your-project-id.your-dataset-id.sales` s ON o.order_id = s.order_id\n    GROUP BY year, month\n),\ngrowth_analysis AS (\n    SELECT \n        year,\n        month,\n        orders,\n        revenue,\n        -- Previous month values\n        LAG(orders) OVER (ORDER BY year, month) as prev_month_orders,\n        LAG(revenue) OVER (ORDER BY year, month) as prev_month_revenue\n    FROM monthly_sales\n)\nSELECT \n    year,\n    month,\n    orders,\n    revenue,\n    prev_month_orders,\n    prev_month_revenue,\n    -- Growth calculations\n    CASE \n        WHEN prev_month_orders IS NOT NULL THEN\n            ROUND(((orders - prev_month_orders) / prev_month_orders) * 100, 2)\n        ELSE NULL\n    END as order_growth_percent,\n    CASE \n        WHEN prev_month_revenue IS NOT NULL THEN\n            ROUND(((revenue - prev_month_revenue) / prev_month_revenue) * 100, 2)\n        ELSE NULL\n    END as revenue_growth_percent\nFROM growth_analysis\nORDER BY year, month;\n```\n\n\n## Advanced Analytics\n### Cohort Analysis - Customer Retention\n**Purpose: Track customer behavior over time**\n```sql\nWITH customer_orders AS (\n    SELECT \n        c.Custkey,\n        o.order_date,\n        ROW_NUMBER() OVER (PARTITION BY c.Custkey ORDER BY o.order_date) as order_sequence,\n        MIN(o.order_date) OVER (PARTITION BY c.Custkey) as first_order_date\n    FROM `your-project-id.your-dataset-id.customers` c\n    JOIN `your-project-id.your-dataset-id` o ON c.Custkey = o.customer_id\n),\ncohort_data AS (\n    SELECT \n        DATE_TRUNC(first_order_date, MONTH) as cohort_month,\n        DATE_DIFF(DATE_TRUNC(order_date, MONTH), DATE_TRUNC(first_order_date, MONTH), MONTH) as period_number,\n        Custkey\n    FROM customer_orders\n)\nSELECT \n    cohort_month,\n    period_number,\n    COUNT(DISTINCT Custkey) as customers,\n    -- Calculate retention rate\n    ROUND(\n        COUNT(DISTINCT Custkey) / \n        FIRST_VALUE(COUNT(DISTINCT Custkey)) OVER (\n            PARTITION BY cohort_month \n            ORDER BY period_number \n            ROWS UNBOUNDED PRECEDING\n        ) * 100, 2\n    ) as retention_rate\nFROM cohort_data\nGROUP BY cohort_month, period_number\nORDER BY cohort_month, period_number;\n```\n\n### ABC Analysis - Product Classification\n**Purpose: Classify products by revenue contribution**\n```sql\nWITH product_revenue AS (\n    SELECT \n        p.product_id,\n        p.product_name,\n        ROUND(SUM(s.sales_amount), 2) as total_revenue\n    FROM `your-project-id.your-dataset-id.products` p\n    JOIN `your-project-id.your-dataset-ids.sales` s ON p.product_id = s.product_id\n    GROUP BY p.product_id, p.product_name\n),\nrevenue_analysis AS (\n    SELECT \n        *,\n        SUM(total_revenue) OVER () as total_company_revenue,\n        SUM(total_revenue) OVER (ORDER BY total_revenue DESC ROWS UNBOUNDED PRECEDING) as cumulative_revenue\n    FROM product_revenue\n)\nSELECT \n    product_name,\n    total_revenue,\n    ROUND((total_revenue / total_company_revenue) * 100, 2) as revenue_percentage,\n    ROUND((cumulative_revenue / total_company_revenue) * 100, 2) as cumulative_percentage,\n    -- ABC Classification\n    CASE \n        WHEN ROUND((cumulative_revenue / total_company_revenue) * 100, 2) \u003c= 80 THEN 'A'\n        WHEN ROUND((cumulative_revenue / total_company_revenue) * 100, 2) \u003c= 95 THEN 'B'\n        ELSE 'C'\n    END as abc_category\nFROM revenue_analysis\nORDER BY total_revenue DESC;\n```\n\n### Sales Rep Performance Analysis\n**Purpose: Evaluate sales representative effectiveness**\n```sql\nSELECT \n    o.sales_rep,\n    COUNT(DISTINCT o.customer_id) as unique_customers,\n    COUNT(DISTINCT o.order_id) as total_orders,\n    ROUND(AVG(order_totals.order_value), 2) as avg_order_value,\n    ROUND(SUM(order_totals.order_value), 2) as total_sales,\n    -- Performance metrics\n    ROUND(SUM(order_totals.order_value) / COUNT(DISTINCT o.order_id), 2) as sales_per_order,\n    ROUND(SUM(order_totals.order_value) / COUNT(DISTINCT o.customer_id), 2) as sales_per_customer,\n    -- Ranking\n    RANK() OVER (ORDER BY SUM(order_totals.order_value) DESC) as sales_rank\nFROM `your-project-id.your-dataset-id.orders` o\nJOIN (\n    SELECT \n        order_id,\n        SUM(sales_amount) as order_value\n    FROM `your-project-id.your-dataset-id.sales`\n    GROUP BY order_id\n) order_totals ON o.order_id = order_totals.order_id\nWHERE o.sales_rep IS NOT NULL\nGROUP BY o.sales_rep\nORDER BY total_sales DESC;\n```\n\n\n## Business Intelligence Queries\n### Customer Lifetime Value Prediction\n**Purpose: Estimate future customer value**\n```sql\nWITH customer_behavior AS (\n    SELECT \n        c.Custkey,\n        c.customer_name,\n        c.registration_date,\n        COUNT(DISTINCT o.order_id) as total_orders,\n        ROUND(SUM(s.sales_amount), 2) as total_spent,\n        ROUND(AVG(s.sales_amount), 2) as avg_order_value,\n        DATE_DIFF(MAX(o.order_date), MIN(o.order_date), DAY) + 1 as customer_lifespan_days,\n        DATE_DIFF(CURRENT_DATE(), MAX(o.order_date), DAY) as days_since_last_order\n    FROM `your-project-id.your-dataset-id.customers` c\n    JOIN `your-project-id.your-dataset-id.orders` o ON c.Custkey = o.customer_id\n    JOIN `your-project-id.your-dataset-id.sales` s ON o.order_id = s.order_id\n    GROUP BY c.Custkey, c.customer_name, c.registration_date\n)\nSELECT \n    customer_name,\n    total_orders,\n    total_spent,\n    avg_order_value,\n    customer_lifespan_days,\n    -- Calculate purchase frequency\n    CASE \n        WHEN customer_lifespan_days \u003e 0 THEN \n            ROUND(total_orders / (customer_lifespan_days / 365.0), 2)\n        ELSE 0 \n    END as orders_per_year,\n    -- Estimated CLV (simplified)\n    CASE \n        WHEN customer_lifespan_days \u003e 0 THEN \n            ROUND(avg_order_value * (total_orders / (customer_lifespan_days / 365.0)) * 2, 2)\n        ELSE avg_order_value \n    END as estimated_2year_clv,\n    -- Customer status\n    CASE \n        WHEN days_since_last_order \u003c= 30 THEN 'Active'\n        WHEN days_since_last_order \u003c= 90 THEN 'At Risk'\n        ELSE 'Churned'\n    END as customer_status\nFROM customer_behavior\nORDER BY estimated_2year_clv DESC;\n```\n\n### Seasonal Sales Patterns\n**Purpose: Identify seasonal trends in sales**\n```sql\nSELECT \n    EXTRACT(MONTH FROM o.order_date) as month,\n    FORMAT_DATE('%B', DATE(2024, EXTRACT(MONTH FROM o.order_date), 1)) as month_name,\n    COUNT(DISTINCT o.order_id) as total_orders,\n    ROUND(SUM(s.sales_amount), 2) as total_revenue,\n    ROUND(AVG(s.sales_amount), 2) as avg_order_value,\n    -- Compare to annual average\n    ROUND(\n        (SUM(s.sales_amount) - AVG(SUM(s.sales_amount)) OVER ()) / \n        AVG(SUM(s.sales_amount)) OVER () * 100, 2\n    ) as variance_from_avg_percent\nFROM `your-project-id.your-dataset-id.orders` o\nJOIN `your-project-id.your-dataset-id.sales` s ON o.order_id = s.order_id\nGROUP BY month, month_name\nORDER BY month;\n```\n\n### Product Cross-Selling Analysis\n**Purpose: Find products frequently bought together**\n```sql\nWITH order_products AS (\n    SELECT \n        s1.order_id,\n        s1.product_id as product_a,\n        s2.product_id as product_b\n    FROM `your-project-id.your-dataset-id.sales` s1\n    JOIN `your-project-id.your-dataset-id.sales` s2 \n        ON s1.order_id = s2.order_id \n        AND s1.product_id \u003c s2.product_id  -- Avoid duplicates\n),\nproduct_pairs AS (\n    SELECT \n        product_a,\n        product_b,\n        COUNT(*) as times_bought_together\n    FROM order_products\n    GROUP BY product_a, product_b\n    HAVING COUNT(*) \u003e= 2  -- At least bought together twice\n)\nSELECT \n    pa.product_name as product_a_name,\n    pb.product_name as product_b_name,\n    pp.times_bought_together,\n    -- Calculate support (percentage of orders containing both items)\n    ROUND(\n        pp.times_bought_together / \n        (SELECT COUNT(DISTINCT order_id) FROM `your-project-id.your-dataset-id.sales`) * 100, 2\n    ) as support_percent\nFROM product_pairs pp\nJOIN `your-project-id.your-dataset-id.products` pa ON pp.product_a = pa.product_id\nJOIN `your-project-id.your-dataset-id.products` pb ON pp.product_b = pb.product_id\nORDER BY times_bought_together DESC\nLIMIT 20;\n```\n\n### Revenue Forecast Base Data\n**Purpose: Prepare data for revenue forecasting**\n```sql\nWITH daily_sales AS (\n    SELECT \n        o.order_date,\n        COUNT(DISTINCT o.order_id) as orders,\n        ROUND(SUM(s.sales_amount), 2) as revenue,\n        COUNT(DISTINCT o.customer_id) as unique_customers\n    FROM `your-project-id.your-dataset-id.orders` o\n    JOIN `your-project-id.your-dataset-id.sales` s ON o.order_id = s.order_id\n    GROUP BY o.order_date\n)\nSELECT \n    order_date,\n    orders,\n    revenue,\n    unique_customers,\n    -- 7-day moving averages for trend analysis\n    ROUND(AVG(orders) OVER (\n        ORDER BY order_date \n        ROWS BETWEEN 6 PRECEDING AND CURRENT ROW\n    ), 2) as orders_7day_avg,\n    ROUND(AVG(revenue) OVER (\n        ORDER BY order_date \n        ROWS BETWEEN 6 PRECEDING AND CURRENT ROW\n    ), 2) as revenue_7day_avg,\n    -- Week-over-week growth\n    ROUND(\n        (revenue - LAG(revenue, 7) OVER (ORDER BY order_date)) / \n        LAG(revenue, 7) OVER (ORDER BY order_date) * 100, 2\n    ) as wow_growth_percent\nFROM daily_sales\nORDER BY order_date DESC;\n```\n\n### Executive Dashboard Summary\n**Purpose: Key metrics for executive reporting**\n```sql\nWITH summary_stats AS (\n    SELECT \n        COUNT(DISTINCT c.Custkey) as total_customers,\n        COUNT(DISTINCT p.product_id) as total_products,\n        COUNT(DISTINCT o.order_id) as total_orders,\n        ROUND(SUM(s.sales_amount), 2) as total_revenue,\n        ROUND(AVG(s.sales_amount), 2) as avg_order_value,\n        COUNT(DISTINCT o.sales_rep) as active_sales_reps\n    FROM `your-project-id.your-dataset-id.customers` c\n    CROSS JOIN `your-project-id.your-dataset-id.products` p\n    LEFT JOIN `your-project-id.your-dataset-id.orders` o ON c.Custkey = o.customer_id\n    LEFT JOIN `your-project-id.your-dataset-id.sales` s ON o.order_id = s.order_id\n)\nSELECT \n    'Total Customers' as metric, CAST(total_customers as STRING) as value\nFROM summary_stats\nUNION ALL SELECT 'Total Products', CAST(total_products as STRING) FROM summary_stats\nUNION ALL SELECT 'Total Orders', CAST(total_orders as STRING) FROM summary_stats  \nUNION ALL SELECT 'Total Revenue', CONCAT('$', CAST(total_revenue as STRING)) FROM summary_stats\nUNION ALL SELECT 'Average Order Value', CONCAT('$', CAST(avg_order_value as STRING)) FROM summary_stats\nUNION ALL SELECT 'Active Sales Reps', CAST(active_sales_reps as STRING) FROM summary_stats;\n```\n\n## 🤖 Machine Learning - Sales Forecasting\n```python\n# Prepare data for ML\ndef prepare_ml_data():\n    \"\"\"Prepare aggregated data for machine learning\"\"\"\n\n    # Load the original data\n    df = pd.read_csv('your-file-path')\n\n    # Clean and convert dates\n    df['Invoice Date'] = pd.to_datetime(df['Invoice Date'])\n\n    # Create aggregated monthly data by product\n    ml_data = df.groupby([\n        df['Invoice Date'].dt.to_period('M'),\n        'Item',\n        'Item Class'\n    ]).agg({\n        'Sales Amount': 'sum',\n        'Sales Quantity': 'sum',\n        'Sales Price': 'mean',\n        'List Price': 'mean',\n        'Discount Amount': 'sum',\n        'Sales Margin Amount': 'sum'\n    }).reset_index()\n\n    # Rename columns\n    ml_data.columns = ['month', 'product', 'category', 'sales_amount',\n                      'quantity', 'avg_price', 'list_price', 'discount', 'margin']\n\n    # Convert period to datetime\n    ml_data['month'] = ml_data['month'].dt.to_timestamp()\n\n    # Create time-based features\n    ml_data['year'] = ml_data['month'].dt.year\n    ml_data['month_num'] = ml_data['month'].dt.month\n    ml_data['quarter'] = ml_data['month'].dt.quarter\n\n    # Create lag features (previous months)\n    ml_data = ml_data.sort_values(['product', 'month'])\n    ml_data['sales_lag_1'] = ml_data.groupby('product')['sales_amount'].shift(1)\n    ml_data['sales_lag_2'] = ml_data.groupby('product')['sales_amount'].shift(2)\n    ml_data['sales_lag_3'] = ml_data.groupby('product')['sales_amount'].shift(3)\n\n    # Calculate rolling averages\n    ml_data['sales_rolling_3'] = ml_data.groupby('product')['sales_amount'].rolling(3).mean().values\n\n    return ml_data\n\nml_data = prepare_ml_data()\nprint(f\"ML dataset shape: {ml_data.shape}\")\nml_data.head()\n```\n**ML dataset shape: (8137, 16)**\n\n| month       | product                | category | sales_amount | quantity | avg_price | list_price | discount | margin | year | month_num | quarter | sales_lag_1 | sales_lag_2 | sales_lag_3 | sales_rolling_3 |\n|-------------|-----------------------|----------|--------------|----------|-----------|------------|----------|--------|------|-----------|---------|-------------|-------------|-------------|----------------|\n| 2017-01-01  | American Beef Bologna | P01      | 229.76       | 20       | 11.488000 | 25.14      | 273.04   | 65.40  | 2017 | 1         | 1       | NaN         | NaN         | NaN         | NaN            |\n| 2017-02-01  | American Beef Bologna | P01      | 362.02       | 30       | 12.067333 | 25.14      | 392.18   | 115.48 | 2017 | 2         | 1       | 229.76      | NaN         | NaN         | NaN            |\n| 2017-08-01  | American Beef Bologna | P01      | 362.02       | 30       | 12.067333 | 25.14      | 392.18   | 115.48 | 2017 | 8         | 3       | 362.02      | 229.76      | NaN         | 317.933333     |\n| 2017-10-01  | American Beef Bologna | P01      | 239.33       | 20       | 11.966500 | 25.14      | 263.47   | 74.97  | 2017 | 10        | 4       | 362.02      | 362.02      | 229.76      | 321.123333     |\n| 2017-12-01  | American Beef Bologna | P01      | 239.33       | 20       | 11.966500 | 25.14      | 263.47   | 74.97  | 2017 | 12        | 4       | 239.33      | 362.02      | 362.02      | 280.226667     |\n\n\n## 📈 Linear Regression\n```python\nprint(\"LINEAR REGRESSION MODEL\")\nprint(\"-\" * 30)\n\n# Prepare features and target\nfeature_columns = ['year', 'month_num', 'quarter', 'avg_price', 'list_price',\n                  'sales_lag_1', 'sales_lag_2', 'sales_lag_3', 'sales_rolling_3']\n\n# Remove rows with NaN values (due to lag features)\nml_clean = ml_data.dropna()\n\nX = ml_clean[feature_columns]\ny = ml_clean['sales_amount']\n\n# Split the data\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n\n# Train Linear Regression\nlr_model = LinearRegression()\nlr_model.fit(X_train, y_train)\n\n# Make predictions\nlr_pred = lr_model.predict(X_test)\n\n# Calculate metrics\nlr_mae = mean_absolute_error(y_test, lr_pred)\nlr_mse = mean_squared_error(y_test, lr_pred)\nlr_rmse = np.sqrt(lr_mse)\nlr_r2 = r2_score(y_test, lr_pred)\n\nprint(f\"Linear Regression Results:\")\nprint(f\"MAE: ${lr_mae:,.2f}\")\nprint(f\"RMSE: ${lr_rmse:,.2f}\")\nprint(f\"R² Score: {lr_r2:.4f}\")\n\n# Feature importance\nfeature_importance = pd.DataFrame({\n    'feature': feature_columns,\n    'coefficient': lr_model.coef_,\n    'abs_coefficient': np.abs(lr_model.coef_)\n}).sort_values('abs_coefficient', ascending=False)\n\nprint(f\"\\nTop 5 Most Important Features:\")\nprint(feature_importance.head())\n```\n\u003cimg width=\"1669\" height=\"323\" alt=\"image\" src=\"https://github.com/user-attachments/assets/a3fdb5ce-7654-46a7-988a-67f0ff04ad4d\" /\u003e\n\n#### *Linear Regression (Baseline)*\n- **Performance**: R² = 0.72, RMSE = $1,247\n- **Purpose**: Establish performance baseline\n\n## ⚡ LightGBM\n```python\nprint(\"\\n LIGHTGBM MODEL\")\nprint(\"-\" * 30)\n\n# Import LightGBM\nimport lightgbm as lgb\n\n# Prepare dataset for LightGBM\nlgb_train = lgb.Dataset(X_train, label=y_train)\nlgb_eval = lgb.Dataset(X_test, label=y_test, reference=lgb_train)\n\n# Set parameters\nparams = {\n    'objective': 'regression',\n    'metric': ['l1', 'rmse'],\n    'learning_rate': 0.1,\n    'num_leaves': 31,\n    'feature_fraction': 0.8,\n    'bagging_fraction': 0.8,\n    'bagging_freq': 5,\n    'verbose': -1\n}\n\n# Train LightGBM model\nlgbm_model = lgb.train(params, lgb_train,\n                      num_boost_round=1000,\n                      valid_sets=[lgb_train, lgb_eval])\n\n# Make predictions\nlgbm_pred = lgbm_model.predict(X_test, num_iteration=lgbm_model.best_iteration)\n\n# Calculate metrics\nlgbm_mae = mean_absolute_error(y_test, lgbm_pred)\nlgbm_mse = mean_squared_error(y_test, lgbm_pred)\nlgbm_rmse = np.sqrt(lgbm_mse)\nlgbm_r2 = r2_score(y_test, lgbm_pred)\n\nprint(f\"LightGBM Results:\")\nprint(f\"MAE: ${lgbm_mae:,.2f}\")\nprint(f\"RMSE: ${lgbm_rmse:,.2f}\")\nprint(f\"R² Score: {lgbm_r2:.4f}\")\n\n# Feature importance\nlgbm_importance = pd.DataFrame({\n    'feature': feature_columns,\n    'importance': lgbm_model.feature_importance()\n}).sort_values('importance', ascending=False)\n\nprint(f\"\\nTop 5 Most Important Features:\")\nprint(lgbm_importance.head())\n```\n\u003cimg width=\"1568\" height=\"339\" alt=\"image\" src=\"https://github.com/user-attachments/assets/f9cf21aa-c3c7-4421-b0be-c6099286ae56\" /\u003e\n\n## 📊 Visualize Model Performance\n```python\nfig, axes = plt.subplots(2, 2, figsize=(15, 12))\nfig.suptitle('Sales Forecasting Model Performance', fontsize=16, fontweight='bold')\n\n# 1. Actual vs Predicted - Linear Regression\naxes[0,0].scatter(y_test, lr_pred, alpha=0.6, color='blue')\naxes[0,0].plot([y_test.min(), y_test.max()], [y_test.min(), y_test.max()], 'r--', lw=2)\naxes[0,0].set_xlabel('Actual Sales')\naxes[0,0].set_ylabel('Predicted Sales')\naxes[0,0].set_title(f'Linear Regression\\nR² = {lr_r2:.4f}')\n\n# 2. Actual vs Predicted - LightGBM\naxes[0,1].scatter(y_test, lgbm_pred, alpha=0.6, color='green')\naxes[0,1].plot([y_test.min(), y_test.max()], [y_test.min(), y_test.max()], 'r--', lw=2)\naxes[0,1].set_xlabel('Actual Sales')\naxes[0,1].set_ylabel('Predicted Sales')\naxes[0,1].set_title(f'LightGBM\\nR² = {lgbm_r2:.4f}')\n\n# 3. Feature Importance - LightGBM\ntop_features = lgbm_importance.head(8)\naxes[1,0].barh(range(len(top_features)), top_features['importance'], color='lightgreen')\naxes[1,0].set_yticks(range(len(top_features)))\naxes[1,0].set_yticklabels(top_features['feature'])\naxes[1,0].set_xlabel('Importance')\naxes[1,0].set_title('LightGBM Feature Importance')\n\n# 4. Model Comparison\nmodels = ['Linear Regression', 'LightGBM']\nmae_scores = [lr_mae, lgbm_mae]\nrmse_scores = [lr_rmse, lgbm_rmse]\n\nx = np.arange(len(models))\nwidth = 0.35\n\naxes[1,1].bar(x - width/2, mae_scores, width, label='MAE', alpha=0.8, color='skyblue')\naxes[1,1].bar(x + width/2, rmse_scores, width, label='RMSE', alpha=0.8, color='lightcoral')\naxes[1,1].set_xlabel('Models')\naxes[1,1].set_ylabel('Error')\naxes[1,1].set_title('Model Performance Comparison')\naxes[1,1].set_xticks(x)\naxes[1,1].set_xticklabels(models)\naxes[1,1].legend()\n\nplt.tight_layout()\nplt.show()\n\nprint(\"Key Insight: LightGBM typically outperforms Linear Regression with faster training and better handling of non-linear relationships.\")\n```\n\u003cimg width=\"1487\" height=\"1179\" alt=\"image\" src=\"https://github.com/user-attachments/assets/6be61dce-9c9e-4b54-b440-cf4be9006c0b\" /\u003e\n\n## 🔮 Facebook Prophet for Time Series Forecasting\n```python\nprint(\"\\nFACEBOOK PROPHET TIME SERIES FORECASTING\")\nprint(\"-\" * 50)\n\n# Prepare data for Prophet (requires specific column names)\ndef prepare_prophet_data():\n    # Aggregate data by month for overall sales\n    prophet_data = df.groupby(df['Invoice Date'].dt.to_period('M'))['Sales Amount'].sum().reset_index()\n    prophet_data['Invoice Date'] = prophet_data['Invoice Date'].dt.to_timestamp()\n\n    # Prophet requires 'ds' and 'y' columns\n    prophet_data.columns = ['ds', 'y']\n\n    return prophet_data\n\nprophet_data = prepare_prophet_data()\nprint(f\"Prophet dataset shape: {prophet_data.shape}\")\n\n# Initialize and fit Prophet model\nprophet_model = Prophet(\n    daily_seasonality=False,\n    weekly_seasonality=False,\n    yearly_seasonality=True,\n    changepoint_prior_scale=0.05\n)\n\nprophet_model.fit(prophet_data)\n\n# Create future dataframe for 12 months ahead\nfuture = prophet_model.make_future_dataframe(periods=12, freq='M')\nforecast = prophet_model.predict(future)\n\n# Calculate metrics on historical data\nhistorical_pred = forecast[forecast['ds'].isin(prophet_data['ds'])]\nprophet_mae = mean_absolute_error(prophet_data['y'], historical_pred['yhat'])\nprophet_rmse = np.sqrt(mean_squared_error(prophet_data['y'], historical_pred['yhat']))\nprophet_r2 = r2_score(prophet_data['y'], historical_pred['yhat'])\n\n\nprint(f\"Prophet Time Series Results:\")\nprint(f\"MAE: ${prophet_mae:,.2f}\")\nprint(f\"RMSE: ${prophet_rmse:,.2f}\")\nprint(f\"R² Score: {prophet_r2:.4f}\")\n```\n\u003cimg width=\"1774\" height=\"404\" alt=\"image\" src=\"https://github.com/user-attachments/assets/15466d4e-840d-4ab3-b9a7-a932ffb44aba\" /\u003e\n\n## 📶 Visualize Prophet Forecasting for 6 months\n```python\n# --- Plot 1: The Main Forecast ---\nfig, ax = plt.subplots(figsize=(15, 8))\nprophet_model.plot(forecast, ax=ax)\nax.set_title('Amazon Sales Forecast - Next 12 Months', fontweight='bold')\nax.set_xlabel('Date')\nax.set_ylabel('Sales Amount ($)')\nax.grid(True, alpha=0.3)\nplt.show()\n\n# --- Plot 2: The Components ---\n# Call this separately to let Prophet create its own multi-plot figure\nfig_components = prophet_model.plot_components(forecast)\nplt.show()\n\n# --- Future Predictions\nfuture_predictions = forecast[forecast['ds'] \u003e prophet_data['ds'].max()][['ds', 'yhat', 'yhat_lower', 'yhat_upper']]\nprint(f\"\\n SALES FORECAST FOR NEXT 6 MONTHS:\")\nprint(\"=\" * 50)\nfor _, row in future_predictions.head(6).iterrows():\n    print(f\"{row['ds'].strftime('%Y-%m')}: ${row['yhat']:,.0f} \"\n          f\"(Range: ${row['yhat_lower']:,.0f} - ${row['yhat_upper']:,.0f})\")\n```\n\u003cimg width=\"1254\" height=\"701\" alt=\"image\" src=\"https://github.com/user-attachments/assets/f284568f-22ef-411a-ad5d-318bac4256e9\" /\u003e\n\u003cimg width=\"887\" height=\"590\" alt=\"image\" src=\"https://github.com/user-attachments/assets/a7985089-5d7e-446c-bc1e-4c902eeb17da\" /\u003e\n\u003cimg width=\"1644\" height=\"205\" alt=\"image\" src=\"https://github.com/user-attachments/assets/dad6543b-e462-4f3a-a918-4dd3a169e007\" /\u003e\n\n#### *Facebook Prophet (Time-Series)*\n- **Performance**: 12-month forecast with seasonality\n- **Purpose**: Long-term forecasting with seasonal patterns\n\n## 👤 Customer Segmentation (Unsupervised ML)\n```python\nprint(\"CUSTOMER SEGMENTATION - K-MEANS CLUSTERING\")\nprint(\"=\" * 50)\n\n# Prepare customer behavior data\ndef prepare_segmentation_data():\n    # Calculate customer metrics\n    customer_behavior = df.groupby('Custkey').agg({\n        'Sales Amount': ['sum', 'mean', 'count'],\n        'Sales Quantity': 'sum',\n        'Discount Amount': 'sum',\n        'Sales Margin Amount': 'sum',\n        'Invoice Date': ['min', 'max']\n    }).reset_index()\n\n    # Flatten column names\n    customer_behavior.columns = [\n        'customer_id', 'total_spent', 'avg_order_value', 'order_frequency',\n        'total_quantity', 'total_discount', 'total_margin', 'first_purchase', 'last_purchase'\n    ]\n\n    # Calculate additional metrics\n    customer_behavior['customer_lifetime'] = (customer_behavior['last_purchase'] -\n                                            customer_behavior['first_purchase']).dt.days\n    customer_behavior['days_since_last_purchase'] = (df['Invoice Date'].max() -\n                                                   customer_behavior['last_purchase']).dt.days\n    customer_behavior['avg_discount_rate'] = customer_behavior['total_discount'] / customer_behavior['total_spent']\n    customer_behavior['profit_margin'] = customer_behavior['total_margin'] / customer_behavior['total_spent']\n\n    return customer_behavior\n\ncustomer_data = prepare_segmentation_data()\nprint(f\"Customer segmentation dataset shape: {customer_data.shape}\")\n\n# Select features for clustering\nclustering_features = ['total_spent', 'avg_order_value', 'order_frequency',\n                      'days_since_last_purchase', 'avg_discount_rate']\n\nX_cluster = customer_data[clustering_features].fillna(0)\n\n# Standardize the features\nscaler = StandardScaler()\nX_scaled = scaler.fit_transform(X_cluster)\n\nprint(f\"Features used for clustering: {clustering_features}\")\n```\n\u003cimg width=\"1688\" height=\"110\" alt=\"image\" src=\"https://github.com/user-attachments/assets/c377ac08-9339-4b00-9eb5-e34be0cabe6b\" /\u003e\n\n## 🔺Elbow Curve\n```python\n# Determine optimal number of clusters using Elbow Method\ninertias = []\nsilhouette_scores = []\nk_range = range(2, 11)\n\nfrom sklearn.metrics import silhouette_score\n\nfor k in k_range:\n    kmeans = KMeans(n_clusters=k, random_state=42, n_init=10)\n    kmeans.fit(X_scaled)\n    inertias.append(kmeans.inertia_)\n    silhouette_scores.append(silhouette_score(X_scaled, kmeans.labels_))\n\n# Plot elbow curve\nfig, axes = plt.subplots(1, 2, figsize=(15, 6))\n\naxes[0].plot(k_range, inertias, 'bo-')\naxes[0].set_xlabel('Number of Clusters (k)')\naxes[0].set_ylabel('Inertia')\naxes[0].set_title('Elbow Method for Optimal k')\naxes[0].grid(True, alpha=0.3)\n\naxes[1].plot(k_range, silhouette_scores, 'ro-')\naxes[1].set_xlabel('Number of Clusters (k)')\naxes[1].set_ylabel('Silhouette Score')\naxes[1].set_title('Silhouette Score vs Number of Clusters')\naxes[1].grid(True, alpha=0.3)\n\nplt.tight_layout()\nplt.show()\n\n# Choose optimal k (let's use k=4 based on elbow method)\noptimal_k = 4\nprint(f\"Selected k = {optimal_k} clusters\")\n```\n\u003cimg width=\"1489\" height=\"590\" alt=\"image\" src=\"https://github.com/user-attachments/assets/81fe68a5-fd6c-41c7-9388-95474badb6c7\" /\u003e\n\n\n## 🌿 K-Means clustering\n```python\n# Final clustering\nkmeans_final = KMeans(n_clusters=optimal_k, random_state=42, n_init=10)\ncluster_labels = kmeans_final.fit_predict(X_scaled)\n\n# Add cluster labels to customer data\ncustomer_data['cluster'] = cluster_labels\n\n# Analyze clusters\ncluster_summary = customer_data.groupby('cluster').agg({\n    'total_spent': ['mean', 'median'],\n    'avg_order_value': ['mean', 'median'],\n    'order_frequency': ['mean', 'median'],\n    'days_since_last_purchase': ['mean', 'median'],\n    'avg_discount_rate': ['mean', 'median'],\n    'customer_id': 'count'\n}).round(2)\n\nprint(\"CLUSTER ANALYSIS SUMMARY:\")\nprint(\"=\" * 40)\nprint(cluster_summary)\n```\n\u003cimg width=\"1535\" height=\"553\" alt=\"image\" src=\"https://github.com/user-attachments/assets/b9f77457-ec3b-4478-ad21-7472ec0afe02\" /\u003e\n\n## 🧊 Visualize customer clusters\n```python\n# PCA for visualization\npca = PCA(n_components=2)\nX_pca = pca.fit_transform(X_scaled)\n\nfig, axes = plt.subplots(2, 2, figsize=(15, 12))\nfig.suptitle('Customer Segmentation Analysis', fontsize=16, fontweight='bold')\n\n# 1. PCA visualization of clusters\nscatter = axes[0,0].scatter(X_pca[:, 0], X_pca[:, 1], c=cluster_labels, cmap='viridis', alpha=0.6)\naxes[0,0].set_xlabel(f'PC1 ({pca.explained_variance_ratio_[0]:.1%} variance)')\naxes[0,0].set_ylabel(f'PC2 ({pca.explained_variance_ratio_[1]:.1%} variance)')\naxes[0,0].set_title('Customer Clusters (PCA Visualization)')\nplt.colorbar(scatter, ax=axes[0,0])\n\n# 2. Total Spent vs Order Frequency\nfor cluster in range(optimal_k):\n    cluster_data = customer_data[customer_data['cluster'] == cluster]\n    axes[0,1].scatter(cluster_data['total_spent'], cluster_data['order_frequency'],\n                     label=f'Cluster {cluster}', alpha=0.7)\naxes[0,1].set_xlabel('Total Spent ($)')\naxes[0,1].set_ylabel('Order Frequency')\naxes[0,1].set_title('Total Spent vs Order Frequency')\naxes[0,1].legend()\n\n# 3. Cluster sizes\ncluster_counts = customer_data['cluster'].value_counts().sort_index()\naxes[1,0].bar(cluster_counts.index, cluster_counts.values)\naxes[1,0].set_xlabel('Cluster')\naxes[1,0].set_ylabel('Number of Customers')\naxes[1,0].set_title('Customer Distribution by Cluster')\n\n# 4. Average Order Value by Cluster\ncluster_aov = customer_data.groupby('cluster')['avg_order_value'].mean()\naxes[1,1].bar(cluster_aov.index, cluster_aov.values)\naxes[1,1].set_xlabel('Cluster')\naxes[1,1].set_ylabel('Average Order Value ($)')\naxes[1,1].set_title('Average Order Value by Cluster')\n\nplt.tight_layout()\nplt.show()\n```\n\u003cimg width=\"1490\" height=\"1180\" alt=\"image\" src=\"https://github.com/user-attachments/assets/514df4eb-b879-45ba-9292-901f0d9c7263\" /\u003e\n\n## 💼 Business insights and cluster interpretation\n```python\nprint(\"BUSINESS INSIGHTS FROM CUSTOMER SEGMENTATION\")\nprint(\"=\" * 55)\n\ncluster_insights = {}\nfor cluster in range(optimal_k):\n    cluster_data = customer_data[customer_data['cluster'] == cluster]\n\n    insights = {\n        'size': len(cluster_data),\n        'avg_spent': cluster_data['total_spent'].mean(),\n        'avg_orders': cluster_data['order_frequency'].mean(),\n        'avg_order_value': cluster_data['avg_order_value'].mean(),\n        'days_since_last': cluster_data['days_since_last_purchase'].mean(),\n        'discount_rate': cluster_data['avg_discount_rate'].mean()\n    }\n\n    cluster_insights[cluster] = insights\n\n# Define cluster names based on characteristics\ncluster_names = {\n    0: \"Budget Conscious\",\n    1: \"VIP Customers\",\n    2: \"Regular Customers\",\n    3: \"At-Risk Customers\"\n}\n\n# Sort clusters by total spending for better naming\nsorted_clusters = sorted(cluster_insights.items(), key=lambda x: x[1]['avg_spent'], reverse=True)\n\nprint(\"Cluster Characteristics:\")\nprint(\"-\" * 25)\n\nfor i, (cluster, data) in enumerate(sorted_clusters):\n    if data['avg_spent'] \u003e 1000:\n        segment = \"VIP Customers\"\n        recommendation = \"Offer premium services and exclusive deals\"\n    elif data['avg_spent'] \u003e 500:\n        segment = \"Loyal Customers\"\n        recommendation = \"Implement loyalty program and cross-sell\"\n    elif data['days_since_last'] \u003e 90:\n        segment = \"At-Risk Customers\"\n        recommendation = \"Re-engagement campaign with special offers\"\n    else:\n        segment = \"Regular Customers\"\n        recommendation = \"Encourage higher order values\"\n\n    print(f\"\\n Cluster {cluster} - {segment}:\")\n    print(f\"   Size: {data['size']} customers ({data['size']/len(customer_data)*100:.1f}%)\")\n    print(f\"   Avg Spent: ${data['avg_spent']:,.2f}\")\n    print(f\"   Avg Orders: {data['avg_orders']:.1f}\")\n    print(f\"   Avg Order Value: ${data['avg_order_value']:,.2f}\")\n    print(f\"   Days Since Last Purchase: {data['days_since_last']:.0f}\")\n    print(f\"   Strategy: {recommendation}\")\n\n# Save segmented customer data\ncustomer_data.to_csv('data/processed/customer_segments.csv', index=False)\nprint(f\"\\n Customer segmentation analysis completed! Data saved to 'customer_segments.csv'\")\n```\n\u003cimg width=\"1342\" height=\"672\" alt=\"image\" src=\"https://github.com/user-attachments/assets/16b3156e-d676-4a7e-8540-b66d012c0bf9\" /\u003e\n\n## 🪟 COMPLETE MODEL COMPARISON DASHBOARD\n```python\n# Create subplot grid (2 rows, 3 columns)\nfig, axes = plt.subplots(2, 3, figsize=(18, 12))\nfig.suptitle('Amazon Sales Forecasting - Complete Model Analysis Dashboard', fontsize=20, fontweight='bold')\n\n# Model performance data\nforecasting_models = ['Linear Regression', 'LightGBM', 'Prophet']\nr2_scores = [lr_r2, lgbm_r2, prophet_r2]\nmae_scores = [lr_mae, lgbm_mae, prophet_mae] \ncolors = ['#3498db', '#2ecc71', '#f39c12']\n\n# Plot 1: R² Score Comparison\naxes[0,0].bar(forecasting_models, r2_scores, color=colors, alpha=0.8)\naxes[0,0].set_title('R² Score Comparison', fontweight='bold')\naxes[0,0].set_ylabel('R² Score')\naxes[0,0].set_ylim(0, 1)\nfor i, v in enumerate(r2_scores):\n    axes[0,0].text(i, v + 0.02, f'{v:.3f}', ha='center', fontweight='bold')\n\n# Plot 2: MAE Comparison  \naxes[0,1].bar(forecasting_models, mae_scores, color=colors, alpha=0.8)\naxes[0,1].set_title('MAE Comparison', fontweight='bold')\naxes[0,1].set_ylabel('Mean Absolute Error ($)')\nfor i, v in enumerate(mae_scores):\n    axes[0,1].text(i, v + max(mae_scores)*0.02, f'${v:,.0f}', ha='center', fontweight='bold')\n\n# Plot 3: Best Model Actual vs Predicted\nbest_model_idx = np.argmax(r2_scores)\nbest_pred = [lr_pred, lgbm_pred, historical_pred['yhat']][best_model_idx]\nbest_name = forecasting_models[best_model_idx]\n\n# Select appropriate actual values based on best model\nactual_values = y_test if best_name != 'Prophet' else prophet_data['y']\n\naxes[0,2].scatter(actual_values, best_pred, alpha=0.6, color=colors[best_model_idx], s=50)\naxes[0,2].plot([actual_values.min(), actual_values.max()], [actual_values.min(), actual_values.max()], 'r--', lw=2)\naxes[0,2].set_xlabel('Actual Sales ($)')\naxes[0,2].set_ylabel('Predicted Sales ($)')\naxes[0,2].set_title(f'Best Model: {best_name}\\nR² = {r2_scores[best_model_idx]:.3f}', fontweight='bold')\n\n# Plot 4: LightGBM Feature Importance\ntop_features = lgbm_importance.head(6)\naxes[1,0].barh(top_features['feature'], top_features['importance'], color='#2ecc71', alpha=0.8)\naxes[1,0].set_title('LightGBM Feature Importance', fontweight='bold')\naxes[1,0].set_xlabel('Importance Score')\n\n# Plot 5: Prophet Forecast with Confidence Intervals\naxes[1,1].plot(forecast['ds'], forecast['yhat'], color='#f39c12', linewidth=2, label='Forecast')\naxes[1,1].fill_between(forecast['ds'], forecast['yhat_lower'], forecast['yhat_upper'], \n                      alpha=0.3, color='#f39c12')\naxes[1,1].plot(prophet_data['ds'], prophet_data['y'], 'o', color='#3498db', alpha=0.6, label='Historical')\naxes[1,1].set_title('Prophet Time Series Forecast', fontweight='bold')\naxes[1,1].set_xlabel('Date')\naxes[1,1].set_ylabel('Sales ($)')\naxes[1,1].legend()\n\n# Plot 6: K-Means Customer Segmentation (PCA Visualization)\npca_2d = PCA(n_components=2)\ncustomer_pca = pca_2d.fit_transform(X_scaled)\nscatter = axes[1,2].scatter(customer_pca[:, 0], customer_pca[:, 1], \n                           c=cluster_labels, cmap='viridis', alpha=0.6, s=50)\naxes[1,2].set_title('K-Means Customer Segmentation\\n(PCA Visualization)', fontweight='bold')\naxes[1,2].set_xlabel('First Principal Component')\naxes[1,2].set_ylabel('Second Principal Component')\n\n# Add cluster centers to PCA plot\ncenters_pca = pca_2d.transform(kmeans_final.cluster_centers_)\naxes[1,2].scatter(centers_pca[:, 0], centers_pca[:, 1], \n                 c='red', marker='x', s=200, linewidths=3, label='Centroids')\naxes[1,2].legend()\n\nplt.tight_layout()\nplt.show()\n\n# Performance Summary Table\nprint(\"\\n\" + \"=\"*60)\nprint(\"MODEL PERFORMANCE SUMMARY\")  \nprint(\"=\"*60)\n\nsummary_data = {\n    'Model': ['Linear Regression', 'LightGBM', 'Prophet', 'K-Means'],\n    'Type': ['Supervised', 'Supervised', 'Time Series', 'Unsupervised'],\n    'R² Score': [f'{lr_r2:.3f}', f'{lgbm_r2:.3f}', f'{prophet_r2:.3f}', 'N/A'],\n    'MAE': [f'${lr_mae:,.0f}', f'${lgbm_mae:,.0f}', f'${prophet_mae:,.0f}', 'N/A'],\n    'Best For': ['Baseline', 'Accuracy', 'Seasonality', 'Segmentation']\n}\n\nsummary_df = pd.DataFrame(summary_data)\nprint(summary_df.to_string(index=False))\n\n# Key Insights\nprint(\"\\nKEY INSIGHTS:\")\nprint(f\"• Best performing model: {best_name} (R² = {r2_scores[best_model_idx]:.3f})\")\nprint(f\"• Lowest prediction error: {forecasting_models[np.argmin(mae_scores)]} (MAE = ${min(mae_scores):,.0f})\")\nprint(f\"• Customer segments identified: {optimal_k} distinct groups\")\nprint(f\"• Prophet captures seasonality with {prophet_r2:.1%} accuracy\")\n\n# Business Impact Calculations\nimprovement_over_baseline_r2 = ((max(r2_scores) - min(r2_scores)) / min(r2_scores)) * 100 if min(r2_scores) != 0 else float('inf')\nerror_reduction_mae = ((max(mae_scores) - min(mae_scores)) / max(mae_scores)) * 100 if max(mae_scores) != 0 else 0\n\nprint(f\"\\nBUSINESS IMPACT:\")\nprint(f\"• {improvement_over_baseline_r2:.1f}% improvement in prediction accuracy (based on R²)\")\nprint(f\"• ${max(mae_scores) - min(mae_scores):,.0f} reduction in forecast error\")\nprint(f\"• {error_reduction_mae:.1f}% decrease in prediction uncertainty\")\nprint(\"=\"*60)\n```\n\u003cimg width=\"1787\" height=\"1179\" alt=\"image\" src=\"https://github.com/user-attachments/assets/6830778f-970a-4e77-a20e-7226a5e71a21\" /\u003e\n\u003cimg width=\"1624\" height=\"466\" alt=\"image\" src=\"https://github.com/user-attachments/assets/1fe756e9-f93d-45b6-8136-2f93259bc087\" /\u003e\n\n## 💡 Key Insights \u0026 Business Impact\n\n|  Area                |  Insight                                                                 |  Strategic Recommendation                                                                                 |\n|------------------------|--------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------|\n| 📈 **Sales Performance** | Q4 sales are **35% higher** than other quarters due to seasonality.       | Optimize **inventory and marketing** spend in Q3 to prepare for Q4 surge.                                   |\n| 🛒 **Product Analysis**   | Premium organic products have the **highest margins (avg. 23%)**.        | Expand **premium product line** and feature these items in campaigns.                                       |\n| 👥 **Customer Behavior**  | **23%** of customers are \"At-Risk\" (90+ days no purchase).              | Launch **personalized re-engagement campaigns** to win them back.                                          |\n| 🔮 **Forecasting**        | LightGBM predicts **8% sales growth** over next 6 months (R² = 0.88). | Adjust **financial targets and resource allocation** to align with predicted growth.                       |\n\n## 📊 Visualizations Created\n\n1. **Sales Trend Analysis** - Monthly revenue patterns\n2. **Product Performance** - Top sellers and category breakdown\n3. **Customer Behavior** - Purchase patterns and segmentation\n4. **Seasonality Charts** - Quarterly and monthly trends\n5. **Correlation Heatmap** - Feature relationships\n6. **Forecasting Plots** - Actual vs predicted with confidence intervals\n7. **Cluster Visualization** - Customer segments in 2D space\n8. **Final Dashbaord** - Comparing all models `Linear Regression`, `LightGBM`, `Prophet`, `KMeans`\n\n## ⚙ Tools and Technologies\n- **Kaggle** – Data Source\n- **Jupyter Notebbok** – Interactive environment for coding and presenting analysis\n- **Python** – Data analysis, Manipulation and Visualization\n  - Libraries: `numpy`, `pandas`, `matplotlib`, `seaborn`\n- **Big Query** – Database management used for data storage and queries\n  - Libraries: `bigquery`, `os`\n- **Machine Learning** – Model development and evaluation\n  - Scikit-learn: `train_test_split`, `StandardScaler`\n  - **Models**: `LinearRegression`, `LightGBM`, `Prophet`, `KMeans`\n\n## 🎯 Conclusion\nThis project successfully demonstrates a complete data analytics workflow by transforming raw Amazon sales data into actionable business intelligence. By leveraging Python, advanced SQL, and machine learning, we developed a robust sales forecasting model with 87% accuracy and segmented customers into four distinct personas. The key findings reveal significant seasonal trends and identify high-value customer groups, providing a clear roadmap for strategic decisions. The insights derived empower the business to optimize inventory, personalize marketing efforts, and ultimately drive significant revenue growth.\n\n### 📊 Key Achievements\n- ✅ 87% prediction accuracy in sales forecasting → confident business planning\n- ✅ 4 distinct customer segments → targeted marketing worth millions in potential revenue\n- ✅ Production-ready architecture → supports immediate deployment and scaling\n\n### 🔗 Future Adaptability\n- The solution is modular and scalable, making it easy to adapt to other industries.\n- Can integrate with real-time dashboards for faster decision-making.\n- Provides a framework for leveraging AI/ML to gain a competitive edge.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshakeel-data%2Famazon-sales-forecasting-python-bigquery-ml","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshakeel-data%2Famazon-sales-forecasting-python-bigquery-ml","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshakeel-data%2Famazon-sales-forecasting-python-bigquery-ml/lists"}