https://github.com/topunix/qlik-mariadb-google-drive-sync

MariaDB to Qlik Sense Sync: Python script for seamless data synchronization between MariaDB and Qlik Sense, utilizing Google Drive as an intermediary storage.
https://github.com/topunix/qlik-mariadb-google-drive-sync

Last synced: 4 days ago
JSON representation

MariaDB to Qlik Sense Sync: Python script for seamless data synchronization between MariaDB and Qlik Sense, utilizing Google Drive as an intermediary storage.

• Host: GitHub
• URL: https://github.com/topunix/qlik-mariadb-google-drive-sync
• Owner: topunix
• Created: 2024-02-20T19:50:05.000Z (9 months ago)
• Default Branch: main
• Last Pushed: 2024-02-26T18:51:47.000Z (9 months ago)
• Last Synced: 2024-02-27T19:54:05.870Z (9 months ago)
• Language: Python
• Size: 11.7 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# MariaDB to Qlik Sense Data Synchronization via Google Drive

## Overview

This repository contains a Python script designed to facilitate data synchronization from a MariaDB database to Qlik Sense using Google Drive as an intermediary storage. Due to Qlik Sense's limitation in connecting directly to MariaDB, this script provides a workaround by creating a database dump in XML format, uploading it to Google Drive, and allowing Qlik Sense to read from the file stored in Google Drive.

**UPDATE: December 2023.** All ODBC data sources, except ServiceNow, are now supported with Qlik Data Gateway - Direct Access.

## Workflow

The script follows a three-step workflow:

1. **Delete Old Database Dump:**

   - Checks for an existing file named 'my-dump.xml' on Google Drive.

   - If found, deletes the old dump file.

2. **Upload New Database Dump:**

   - Uploads a new version of the 'my-dump.xml' file to Google Drive using the Google Drive API.

3. **Delete Local Database Dump:**

   - Deletes the local copy of the 'my-dump.xml' file from the machine.

## Requirements

- **Service Account Credentials:**

  - A JSON key file for a service account with appropriate permissions to access Google Drive. This file should be named 'client_cred.json' and placed in the same directory as the script.

- **Parent Folder ID:**

  - Replace the placeholder in the script with the actual ID of the parent folder in Google Drive where you want to store the database dump. You can obtain the folder ID by right-clicking on the folder in Google Drive and selecting "Share." The ID will be in the URL as `https://drive.google.com/drive/folders/YOUR_FOLDER_ID`.

- **Python Libraries:**

  - Ensure the following Python libraries are installed:

    - `oauth2client`

    - `google-api-python-client`

  You can install them using the following command:

  ```bash

  pip install oauth2client google-api-python-client