Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/adesoji1/googlesheettomysql

Export sales data from Google Sheet to a relational DBSM
https://github.com/adesoji1/googlesheettomysql

dataengineering dataingestion googlesheetsapi mysql-database pythonscripting sql

Last synced: 17 days ago
JSON representation

Export sales data from Google Sheet to a relational DBSM

Awesome Lists containing this project

README 

        
To export the sales data from the Google Sheet to a relational DBMS, we need to perform the following steps:



1. **Read the data from Google Sheets**: Use Google Sheets API to access the data programmatically.

2. **Process the data**: Extract relevant fields and format them as needed for the relational DBMS.

3. **Insert the data into the DBMS**: Use an appropriate database client to insert the data into the DBMS.



Thia is a step-by-step guide to achieve this:



### Step 1: Set Up Google Sheets API



First, enable the Google Sheets API and obtain the credentials JSON file.



1. Go to the [Google Cloud Console](https://console.cloud.google.com/).

2. Create a new project or select an existing project.

3. Go to the **API & Services** > **Library**.

4. Enable the **Google Sheets API**.

5. Go to **API & Services** > **Credentials**.

6. Create **OAuth client ID** credentials.

7. Download the JSON file with your credentials.



### Step 2: Install Required Libraries



Install the necessary Python libraries:



```sh

pip install gspread oauth2client mysql-connector-python

```



### Step 3: Create a Python Script



Create a Python script to read the data from Google Sheets and insert it into a MySQL database.



```python

import gspread

from oauth2client.service_account import ServiceAccountCredentials

import mysql.connector

from mysql.connector import errorcode



# Define the scope and authenticate using the credentials JSON

scope = ["https://spreadsheets.google.com/feeds", "https://www.googleapis.com/auth/drive"]

credentials = ServiceAccountCredentials.from_json_keyfile_name('path/to/your/credentials.json', scope)

client = gspread.authorize(credentials)



# Open the Google Sheet by name

sheet = client.open("Control Dashboard 2024").worksheet("ZU")



# Fetch all rows

rows = sheet.get_all_values()



# Connect to the MySQL database

try:

    cnx = mysql.connector.connect(user='your_username', password='your_password',

                                  host='your_host', database='your_database')

    cursor = cnx.cursor()

except mysql.connector.Error as err:

    if err.errno == errorcode.ER_ACCESS_DENIED_ERROR:

        print("Something is wrong with your user name or password")

    elif err.errno == errorcode.ER_BAD_DB_ERROR:

        print("Database does not exist")

    else:

        print(err)



# Iterate through the rows and insert into the database

for row in rows[1:]:  # Skip the header row

    c_name = row[0]

    c_country = row[1]

    c_segment = row[2]

    s_type = row[8]

    s_consultant = row[11]

    

    # Insert into customer table

    cursor.execute("INSERT INTO customer (c_name, c_country, c_segment) VALUES (%s, %s, %s)", 

                   (c_name, c_country, c_segment))

    

    # Get the last inserted id for customer

    customer_id = cursor.lastrowid

    

    # Insert into sale table

    cursor.execute("INSERT INTO sale (customer_id, s_type, s_consultant) VALUES (%s, %s, %s)", 

                   (customer_id, s_type, s_consultant))

    

    # Get the last inserted id for sale

    sale_id = cursor.lastrowid

    

    # Insert into revenue table for each month and currency

    for i, col in enumerate(['p', 'q', 'r', 'x', 'y', 'z', 'af', 'ag', 'ah', 'an', 'ao', 'ap', 'av', 'aw', 'ax', 'bd', 'be', 'bf', 'bl', 'bm', 'bn', 'bt', 'bu', 'bv', 'cb', 'cc', 'cd', 'cj', 'ck', 'cl', 'cr', 'cs', 'ct', 'cz', 'da', 'db']):

        r_value = row[sheet.find(col).col - 1]

        r_year = '2024'

        r_month = str((i // 3) + 1).zfill(2)

        r_currency = ['CHF', 'EUR', 'GBP'][i % 3]

        

        cursor.execute("INSERT INTO revenue (sale_id, r_year, r_month, r_value, r_currency) VALUES (%s, %s, %s, %s, %s)",

                       (sale_id, r_year, r_month, r_value, r_currency))



# Commit the transaction

cnx.commit()



# Close the connection

cursor.close()

cnx.close()

```



### Step 4: Create the Database Schema



Run the following SQL commands to create the database schema:



```sql

CREATE DATABASE your_database;

USE your_database;



CREATE TABLE customer (

    id INT AUTO_INCREMENT PRIMARY KEY,

    c_name VARCHAR(255),

    c_country VARCHAR(255),

    c_segment VARCHAR(255)

);



CREATE TABLE sale (

    id INT AUTO_INCREMENT PRIMARY KEY,

    customer_id INT,

    s_type VARCHAR(255),

    s_consultant VARCHAR(255),

    FOREIGN KEY (customer_id) REFERENCES customer(id)

);



CREATE TABLE revenue (

    id INT AUTO_INCREMENT PRIMARY KEY,

    sale_id INT,

    r_year VARCHAR(4),

    r_month VARCHAR(2),

    r_value DECIMAL(10, 2),

    r_currency VARCHAR(3),

    FOREIGN KEY (sale_id) REFERENCES sale(id)

);

```



### Summary



1. Set up Google Sheets API and obtain credentials.

2. Install necessary libraries.

3. Create a Python script to read data from Google Sheets and insert it into a MySQL database.

4. Create the required database schema.