https://github.com/suniljeurkar/panda_sql
CSV to MySQL Importer 🛠️ This Python script automates the process of importing a CSV file into a MySQL database by dynamically detecting column names, creating the table, and inserting data while handling missing values (`NULL`). It eliminates the need for manual SQL queries, making data migration seamless and efficient. 🚀
https://github.com/suniljeurkar/panda_sql
csv migration sql
Last synced: 5 months ago
JSON representation
CSV to MySQL Importer 🛠️ This Python script automates the process of importing a CSV file into a MySQL database by dynamically detecting column names, creating the table, and inserting data while handling missing values (`NULL`). It eliminates the need for manual SQL queries, making data migration seamless and efficient. 🚀
- Host: GitHub
- URL: https://github.com/suniljeurkar/panda_sql
- Owner: suniljeurkar
- Created: 2025-03-13T05:42:01.000Z (about 1 year ago)
- Default Branch: master
- Last Pushed: 2025-03-13T05:53:01.000Z (about 1 year ago)
- Last Synced: 2025-07-23T13:30:51.044Z (8 months ago)
- Topics: csv, migration, sql
- Language: Python
- Homepage:
- Size: 6.84 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: ReadMe.md
Awesome Lists containing this project
README
### 📄 **README.md**
```markdown
# 🛠 CSV to MySQL Importer
This Python script **automates the process of importing a CSV file into a MySQL database**.
It dynamically:
- Reads the CSV structure
- Creates the MySQL table based on the columns
- Inserts the data while handling `NULL` values correctly
---
## 🚀 Features
- Automated Table Creation: Dynamically detects column names and data types.
- Handles Missing Data: Converts `NaN` values from CSV to `NULL` in MySQL.
- Optimized Bulk Insert: Uses batch processing for faster performance.
- Minimal Configuration: No need to write SQL queries manually.
---
## 🔧 Setup Instructions
### 1️⃣ **Install Dependencies**
Ensure Python and MySQL are installed, then install the required libraries:
```sh
pip install mysql-connector-python pandas
```
### 2️⃣ Setup MySQL Database
Create a database in MySQL:
```sql
CREATE DATABASE northwind_mysql;
```
### 3️⃣ Configure MySQL Connection
Modify the `conn` settings in the script if needed:
```python
conn = mysql.connector.connect(
host="localhost",
user="root",
password="", # Change if necessary
database="northwind_mysql"
)
```
### 4️⃣ Run the Script
To execute the script:
```sh
python insert_SQL_data.py
```
---
## 📂 File Structure
```
📂 CSV_to_MySQL
│── customers.csv # CSV File to import
│── insert_SQL_data.py # Python script for importing
│── sql_create_columns.py # Python script for creating columns
│── README.md # Documentation
```
---
## 📝 Usage Guide
1️⃣ **Place the CSV file (`customers.csv`) in the script directory.**
2️⃣ **Run the script** to automatically create the table and import data.
3️⃣ **Verify the data** in MySQL:
```sql
SELECT * FROM Customers LIMIT 5;
```
---
## ❌ Troubleshooting
| Issue | Solution |
|--------|---------|
| `Unknown column 'nan' in 'field list'` | Ensure `NaN` values are handled (already fixed in the script). |
| `Access denied for user 'root'` | Check MySQL credentials in `conn`. |
| `Table already exists` | Drop existing table or modify script. |
---
## 📌 Future Improvements
- ✅ Batch insert for performance optimization
- ✅ Support for multiple CSV imports
- ✅ Flask API for remote CSV upload & database insertion
---
## 📜 License
This project is open-source and free to use.
---
💡 **Contributions & feedback are welcome!** Submit a pull request or open an issue. 🚀
```