An open API service indexing awesome lists of open source software.

https://github.com/kasrakhaksar/py-auto-migrate

A Powerful Database Migration Tool To Transfer Data 🚀
https://github.com/kasrakhaksar/py-auto-migrate

ai cli clickhouse database dynamodb elasticsearch mariadb mongodb mysql oracle postgresql pypi-package python redis sqlserver

Last synced: 20 days ago
JSON representation

A Powerful Database Migration Tool To Transfer Data 🚀

Awesome Lists containing this project

README

          


🚀 Py-Auto-Migrate


The Universal Database Migration Tool

Seamlessly transfer data between any database.



PyPI - Version


GitHub Repo


Stars


License


Latest Release

---

## Why Py-Auto-Migrate?

Migrating data between different database systems is often a tedious and error-prone task. **Py-Auto-Migrate** is here to change that. It's a powerful, flexible, and easy-to-use Python tool that automates the entire process.

- **Universal Connector**: Supports a vast range of databases, from SQL to NoSQL.
- **Zero Configuration**: Point to your source and target, and let the tool handle schema detection, data type mapping, and destination creation.
- **Blazing Fast**: Optimized for performance, even with large datasets.
- **Safe & Reliable**: Built-in checks ensure data integrity throughout the migration.

---

## Installation

Get started in seconds with `pip`.

```bash
pip install py-auto-migrate
```

### Prefer a Standalone Shell?

Don't have Python? No problem! Download the dedicated **PAM-Shell** for your OS from the [Releases page](https://github.com/kasrakhaksar/py-auto-migrate/releases). It's a ready-to-run executable with the same powerful features.

---

## Quick Start

Using Py-Auto-Migrate is as simple as running one command.

### Basic Command Structure

```bash
py-auto-migrate migrate --source --target
```

| Argument | Description |
| -------------- | -------------------------------------------------------------------------------------------- |
| `--source` | **Required.** Connection URI for the source database. |
| `--target` | **Required.** Connection URI for the target database. |
| `--table` | **Optional.** Migrate a specific table. If omitted, **all** data is migrated. |
| `--ai-ask` | **Optional.** Natural language query for **AI-powered**. |
| `--ai-model` | **Optional.** OpenAI model (default: gpt-3.5-turbo). |

### Real-World Examples

**1. Migrate an entire database from MongoDB to MySQL:**

```bash
py-auto-migrate migrate \
--source "mongodb://user:pass@localhost:27017/source_db" \
--target "mysql://user:pass@localhost:3306/target_db"
```

**2. Migrate a single PostgreSQL table to a new MongoDB collection:**

```bash
py-auto-migrate migrate \
--source "postgresql://user:pass@localhost:5432/mydb" \
--target "mongodb://user:pass@localhost:27017/mydb" \ --table users
```

> **The Magic:** If the target database or table doesn't exist, **Py-Auto-Migrate automatically creates it for you!** It intelligently maps source data types to the appropriate target schema.

**3. Query using AI (Target must be a relational database):**

```bash
export OPENAI_API_KEY="your-api-key"
```

> Before using AI, make sure the **OPENAI_API_KEY** has been exported.

```bash
py-auto-migrate migrate \
--source "oracle://user:password@host:port/service_name" \
--target "mssql://user:password@host:port/database" \
--ai-ask "Please select and insert only names that start with 'S', then sort them." /
--ai-model "gpt-3.5-turbo"
```

---

## Supported Databases

We support a wide and growing range of databases.

| Category | Databases |
| -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Relational** | [![MySQL](https://img.shields.io/badge/MySQL-4479A1?style=flat-square&logo=mysql&logoColor=white)](https://www.mysql.com/) [![PostgreSQL](https://img.shields.io/badge/PostgreSQL-4169E1?style=flat-square&logo=postgresql&logoColor=white)](https://www.postgresql.org/) [![Oracle](https://img.shields.io/badge/Oracle-F80000?style=flat-square&logo=oracle&logoColor=white)](https://www.oracle.com/database/) [![SQL Server](https://img.shields.io/badge/SQL%20Server-CC2927?style=flat-square&logo=microsoftsqlserver&logoColor=white)](https://www.microsoft.com/en-us/sql-server) [![MariaDB](https://img.shields.io/badge/MariaDB-003545?style=flat-square&logo=mariadb&logoColor=white)](https://mariadb.org/) [![SQLite](https://img.shields.io/badge/SQLite-003B57?style=flat-square&logo=sqlite&logoColor=white)](https://www.sqlite.org/) |
| **NoSQL** | [![MongoDB](https://img.shields.io/badge/MongoDB-47A248?style=flat-square&logo=mongodb&logoColor=white)](https://www.mongodb.com/) [![Redis](https://img.shields.io/badge/Redis-DC382D?style=flat-square&logo=redis&logoColor=white)](https://redis.io/) [![DynamoDB](https://img.shields.io/badge/DynamoDB-4053D6?style=flat-square&logo=amazondynamodb&logoColor=white)](https://aws.amazon.com/dynamodb/) [![Elasticsearch](https://img.shields.io/badge/Elasticsearch-005571?style=flat-square&logo=elasticsearch&logoColor=white)](https://www.elastic.co/elasticsearch/) |
| **Analysis** | [![ClickHouse](https://img.shields.io/badge/ClickHouse-FFCC00?style=flat-square&logo=clickhouse&logoColor=black)](https://clickhouse.com/) |

---

Have a feature request? [Open an issue](https://github.com/kasrakhaksar/py-auto-migrate/issues) and let us know!