https://github.com/maxime-cllt/datastorm

Application to insert CSV file into a specified database with optimised columns lenght
https://github.com/maxime-cllt/datastorm

application cpp csvtodb csvtosql dbms ihm mariadb mysql qt-gui qt6 sql sqlite

Last synced: 5 days ago
JSON representation

Application to insert CSV file into a specified database with optimised columns lenght

• Host: GitHub
• URL: https://github.com/maxime-cllt/datastorm
• Owner: Maxime-Cllt
• Created: 2024-08-02T16:57:38.000Z (2 months ago)
• Default Branch: main
• Last Pushed: 2024-09-14T09:56:31.000Z (19 days ago)
• Last Synced: 2024-09-28T07:41:30.534Z (5 days ago)
• Topics: application, cpp, csvtodb, csvtosql, dbms, ihm, mariadb, mysql, qt-gui, qt6, sql, sqlite
• Language: C++
• Homepage:
• Size: 295 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        




DataStorm









## Description

DataStorm is a software that insert into a database the data of a CSV or Excel file. The software is able to insert the

data into a new table or into an existing table and optimise the type of the columns to reduce the size of the database.

Using C++ and Qt, the software can insert the data very efficiently and

into different database management systems like MySQL, PostgreSQL or SQLite.

## Features

 Insert data into a new table 


 Insert data into an existing table 


 Optimise the type of the columns (VARCHAR(MAX_LENGHT)) 


 From CSV file 


 Don't require ANY privilege to insert the data 


## Platforms & Requirements













































## Requirements drivers

These drivers are required to connect to one of the databases:

- mysql-connector for qt

- postgresql-connector for qt

- sqlite-connector for qt

- odbc-connector for qt

## Installation

To install the software, you need to follow the steps below:

1. Clone the repository:

```bash

git clone https://github.com/Maxime-Cllt/DataStorm.git

```

2. Create a build directory:

```bash

mkdir build

cd build

```

3. Generate the build files:

```bash

cmake ..

```

4. Compile the software:

```bash

make

```

5. Run the software:

```bash

./DataStorm

```

## Notes





Does not use LOAD DATA INFILE because it's required to have the FILE privilege to use it.





Does not use the COPY command because it's required to have the SUPERUSER privilege to use it.





Use the INSERT command to insert the data into the database despite not being the most efficient way.





Does not use the BULK INSERT command because it's only available for SQL Server.





## Benchmark



    

        Size (Ko)

        Time to import (ms)

        DBMS

    

        2345

        132

        SQLite

    

    

        2345

        893

        MySQL

    

    

        21 105

        1148

        SQLite

    

        21 105

        7623

        MySQL