https://github.com/sweatnessstrong/oracle-mssql-databasemigration-pipeline-using-mage

This project is the database migration pipline project using mage.ai.
https://github.com/sweatnessstrong/oracle-mssql-databasemigration-pipeline-using-mage

database-management database-migrations etl-pipeline mage mssql mssql-database mssql-server oracle oracle-database python vm

Last synced: 19 days ago
JSON representation

This project is the database migration pipline project using mage.ai.

• Host: GitHub
• URL: https://github.com/sweatnessstrong/oracle-mssql-databasemigration-pipeline-using-mage
• Owner: Sweatnessstrong
• Created: 2024-04-14T04:43:29.000Z (10 months ago)
• Default Branch: master
• Last Pushed: 2024-04-14T04:58:09.000Z (10 months ago)
• Last Synced: 2024-11-17T19:49:32.504Z (3 months ago)
• Topics: database-management, database-migrations, etl-pipeline, mage, mssql, mssql-database, mssql-server, oracle, oracle-database, python, vm
• Language: Python
• Homepage:
• Size: 18.6 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: ReadMe.md

Awesome Lists containing this project

README

        
This is ETL-Pipeline project to migrate the databse from oracle to mssql.

Althout Airflow is a good for this, Mage has advantage better that one.

I am sure Mage is good next age framework

You can check about Mage using following link: https://docs.mage.ai/integrations/databases.

I,e, Airflow is ideal for organizations that require comprehensive control over complex data pipelines, emphasizing robust orchestration and detailed monitoring. Mage.ai is ideal for teams focusing on machine learning, needing efficient and automated data preparation.

- Ready Credential and Connectting to another SQL server.

 If you want to migrate the data to another SQL server  you must get the credential for the server..

 Then you must edit the io_config.yaml file  with the information.

If you complete this step, you can access another SQL server now.

- Getting table structure by change value of veriable in loader and Running loader

 And you must get the table structure according to text file.

 To do this, you can chage the value of "owner_name"  that you want to migrate owner of tables in load_oracle loader.

 If you set this and run this loader, then you will get the structures of orqacle database table contain table_name and data_type.

-Changing path of input file and Running exporter

 Then  you must change the value of file_name and file_path in oracle_data_exporter.

 If you set this values in correctly, you can migrate the data by running the exporter.