{"id":25931209,"url":"https://github.com/iambotcoder/aws-rds-migration","last_synced_at":"2025-06-21T18:06:15.521Z","repository":{"id":277982387,"uuid":"932615137","full_name":"iambotcoder/aws-rds-migration","owner":"iambotcoder","description":"Migrated MariaDB from a local instance to Amazon RDS for improved scalability and performance. ","archived":false,"fork":false,"pushed_at":"2025-02-17T11:14:05.000Z","size":31,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-06-21T18:06:10.245Z","etag":null,"topics":["amazon-web-services","aws","rds-database","rds-mariadb"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/iambotcoder.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-02-14T07:53:27.000Z","updated_at":"2025-02-17T11:14:08.000Z","dependencies_parsed_at":"2025-02-17T11:38:56.300Z","dependency_job_id":null,"html_url":"https://github.com/iambotcoder/aws-rds-migration","commit_stats":null,"previous_names":["iambotcoder/aws-rds-migration"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/iambotcoder/aws-rds-migration","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iambotcoder%2Faws-rds-migration","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iambotcoder%2Faws-rds-migration/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iambotcoder%2Faws-rds-migration/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iambotcoder%2Faws-rds-migration/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/iambotcoder","download_url":"https://codeload.github.com/iambotcoder/aws-rds-migration/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iambotcoder%2Faws-rds-migration/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261170425,"owners_count":23119513,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["amazon-web-services","aws","rds-database","rds-mariadb"],"created_at":"2025-03-03T23:58:28.767Z","updated_at":"2025-06-21T18:06:10.505Z","avatar_url":"https://github.com/iambotcoder.png","language":null,"readme":"\n\n# 🚀 **MariaDB Migration to Amazon RDS**\n\n---\n\n## 📖 Overview\n\nIn this project, we set up an Amazon RDS MariaDB instance, migrated data from an EC2-hosted MariaDB database, and monitored the instance using CloudWatch metrics. This project covers the following tasks:\n\n1. Creating an Amazon RDS MariaDB instance using the AWS CLI\n2. Migrating data from an EC2-hosted MariaDB database to Amazon RDS\n3. Monitoring the RDS instance through the Amazon RDS Console and CloudWatch\n\n---\n\n## 📑 Table of Contents  \n\n1. [Prerequisites](#-prerequisites)  \n2. [Architecture](#-architecture)  \n3. [Task 1: Creating an Amazon RDS instance by using the AWS CLI](#-task-1-creating-an-amazon-rds-instance-by-using-the-aws-cli)  \n4. [Task 2: Migrating application data to the Amazon RDS instance](#-task-2-migrating-application-data-to-the-amazon-rds-instance)  \n5. [Task 3: Configuring the website to use the Amazon RDS instance](#-task-3-configuring-the-website-to-use-the-amazon-rds-instance)  \n6. [Task 4: Monitoring the Amazon RDS Database](#-task-4-monitoring-the-amazon-rds-database)  \n7. [Cleaning Up Resources](#-cleaning-up-resources)  \n8. [Conclusion](#-conclusion)  \n9. [References](#-references)  \n\n---\n\n\n## 🔑 Prerequisites\n\nBefore starting the project, ensure you have the following:\n\n- **AWS CLI** installed and configured with the appropriate permissions\n- **Amazon EC2 instance** running with MariaDB installed\n- **Amazon RDS MariaDB instance** created using the AWS CLI\n- **CloudWatch** enabled for the RDS instance\n- Access to **MariaDB client** to interact with the database\n\n---\n\n## 🗺️ Architecture\n\n![Screenshot 2025-02-17 154111](https://github.com/user-attachments/assets/2309f2f3-4aa8-4b07-83dc-821863e1da50)\n\n\n\n### 🔄 Workflow:\n\n1. Set up an Amazon RDS MariaDB instance using the AWS CLI.\n2. Migrate data from a MariaDB database on an EC2 instance to the Amazon RDS MariaDB instance.\n3. Use CloudWatch to monitor the RDS instance’s health and metrics.\n\n---\n\n## 📝 Task 1: Creating an Amazon RDS instance by using the AWS CLI\n\n\n### **Task 1.1: Connecting to the CLI Host instance**\n1. Open the EC2 Management Console.\n2. In the navigation pane, select **Instances**.\n3. Select the **CLI Host instance**.\n4. Choose **Connect** and then click **Connect** on the EC2 Instance Connect tab.\n\n### **Task 1.2: Configuring the AWS CLI**\n1. Connect to the EC2 instance.\n2. Run the command to configure AWS CLI:\n\n   ```bash\n   aws configure\n   ```\n\n   When prompted, enter:\n   - AWS Access Key ID: `\u003cAccessKey\u003e`\n   - AWS Secret Access Key: `\u003cSecretKey\u003e`\n   - Default region name: `\u003cLabRegion\u003e`\n   - Default output format: `json`\n\n### **Task 1.3: Creating prerequisite components**\n\n#### **1. Create the CafeDatabaseSG security group**\nRun the following command to create a security group:\n\n```bash\naws ec2 create-security-group \\\n--group-name CafeDatabaseSG \\\n--description \"Security group for Cafe database\" \\\n--vpc-id \u003cCafeInstance VPC ID\u003e\n```\n\n- Replace `\u003cCafeInstance VPC ID\u003e` with your VPC ID.\n- Save the `GroupId` from the output.\n\n#### **2. Add inbound rule to the security group**\nRun the following command to allow inbound traffic on TCP port 3306:\n\n```bash\naws ec2 authorize-security-group-ingress \\\n--group-id \u003cCafeDatabaseSG Group ID\u003e \\\n--protocol tcp --port 3306 \\\n--source-group \u003cCafeSecurityGroup Group ID\u003e\n```\n\n- Replace `\u003cCafeDatabaseSG Group ID\u003e` and `\u003cCafeSecurityGroup Group ID\u003e` with their respective values.\n\n#### **3. Verify inbound rule**\nRun this command to verify that the inbound rule is applied:\n\n```bash\naws ec2 describe-security-groups \\\n--query \"SecurityGroups[*].[GroupName,GroupId,IpPermissions]\" \\\n--filters \"Name=group-name,Values='CafeDatabaseSG'\"\n```\n\n#### **4. Create CafeDB Private Subnet 1**\nRun this command to create the first private subnet:\n\n```bash\naws ec2 create-subnet \\\n--vpc-id \u003cCafeInstance VPC ID\u003e \\\n--cidr-block 10.200.2.0/23 \\\n--availability-zone \u003cCafeInstance Availability Zone\u003e\n```\n\n- Replace `\u003cCafeInstance VPC ID\u003e` and `\u003cCafeInstance Availability Zone\u003e` with the appropriate values.\n\n#### **5. Create CafeDB Private Subnet 2**\nRun this command to create the second private subnet:\n\n```bash\naws ec2 create-subnet \\\n--vpc-id \u003cCafeInstance VPC ID\u003e \\\n--cidr-block 10.200.10.0/23 \\\n--availability-zone \u003cavailability-zone\u003e\n```\n\n- Replace `\u003cCafeInstance VPC ID\u003e` and `\u003cavailability-zone\u003e` with the appropriate values.\n\n#### **6. Create CafeDB Subnet Group**\nRun this command to create the DB subnet group:\n\n```bash\naws rds create-db-subnet-group \\\n--db-subnet-group-name \"CafeDB Subnet Group\" \\\n--db-subnet-group-description \"DB subnet group for Cafe\" \\\n--subnet-ids \u003cCafe Private Subnet 1 ID\u003e \u003cCafe Private Subnet 2 ID\u003e \\\n--tags \"Key=Name,Value= CafeDatabaseSubnetGroup\"\n```\n\n- Replace `\u003cCafe Private Subnet 1 ID\u003e` and `\u003cCafe Private Subnet 2 ID\u003e` with the respective subnet IDs.\n\n### **Task 1.4: Creating the Amazon RDS MariaDB instance**\n\nRun the following command to create the RDS MariaDB instance:\n\n```bash\naws rds create-db-instance \\\n--db-instance-identifier CafeDBInstance \\\n--engine mariadb \\\n--engine-version 10.5.13 \\\n--db-instance-class db.t3.micro \\\n--allocated-storage 20 \\\n--availability-zone \u003cCafeInstance Availability Zone\u003e \\\n--db-subnet-group-name \"CafeDB Subnet Group\" \\\n--vpc-security-group-ids \u003cCafeDatabaseSG Group ID\u003e \\\n--no-publicly-accessible \\\n--master-username root --master-user-password 'Re:Start!9'\n```\n\n- Replace `\u003cCafeInstance Availability Zone\u003e` and `\u003cCafeDatabaseSG Group ID\u003e` with the respective values.\n\n### **Monitor the status of the RDS instance**\nTo check the status of the database, run:\n\n```bash\naws rds describe-db-instances \\\n--db-instance-identifier CafeDBInstance \\\n--query \"DBInstances[*].[Endpoint.Address,AvailabilityZone,PreferredBackupWindow,BackupRetentionPeriod,DBInstanceStatus]\"\n```\n\nRepeat the command until the status shows **available**. Once available, you will find the endpoint address in the output.\n\n### **Final output:**\n- RDS Instance Database Endpoint Address: `\u003ccafedbinstance.xxxxxxx.us-west-2.rds.amazonaws.com\u003e`\n\nThese steps should help you create the Amazon RDS instance using AWS CLI! Let me know if you need further clarification.\n\n---\n\n## 📝 Task 2: Migrating application data to the Amazon RDS instance.\n\n#### 1. **Connect to the EC2 Instance (CafeInstance)**:\n\n   - First, you'll connect to the CafeInstance (EC2 instance) using EC2 Instance Connect. This instance will interact with the Amazon RDS instance through the MySQL protocol, as it is allowed by the associated security group (CafeDatabaseSG).\n\n#### 2. **Create a Backup Using `mysqldump`**:\n   - On the CafeInstance, use the `mysqldump` utility to create a backup of the local MySQL database (`cafe_db`).\n   - The command:\n     ```bash\n     mysqldump --user=root --password='Re:Start!9' --databases cafe_db --add-drop-database \u003e cafedb-backup.sql\n     ```\n   - This command:\n     - Uses `mysqldump` to create an SQL backup.\n     - The `--databases` option specifies the database to back up (`cafe_db`).\n     - `--add-drop-database` ensures that a `DROP DATABASE` statement is included to remove the database before restoring.\n     - `\u003e cafedb-backup.sql` redirects the output into a `.sql` file (`cafedb-backup.sql`).\n\n#### 3. **Review the Backup File**:\n   - Open the `cafedb-backup.sql` file to verify its contents.\n   - To view the file using the Linux `less` command:\n     ```bash\n     less cafedb-backup.sql\n     ```\n   - Navigate with the arrow keys, `Page Up/Down`, and press `q` to exit.\n\n#### 4. **Restore Backup to Amazon RDS**:\n   - To restore the backup file to the Amazon RDS database, use the `mysql` command. This command connects to the RDS instance and runs the SQL statements from the backup file.\n   - Command format:\n     ```bash\n     mysql --user=root --password='Re:Start!9' --host=\u003cRDS Instance Database Endpoint Address\u003e \u003c cafedb-backup.sql\n     ```\n   - Replace `\u003cRDS Instance Database Endpoint Address\u003e` with the actual endpoint address of the Amazon RDS instance.\n\n#### 5. **Verify Data Migration**:\n   - After restoring the backup, verify that the data was correctly transferred to the Amazon RDS instance.\n   - Connect to the Amazon RDS instance using the `mysql` command:\n     ```bash\n     mysql --user=root --password='Re:Start!9' --host=\u003cRDS Instance Database Endpoint Address\u003e cafe_db\n     ```\n   - Query the `product` table to ensure the data has been restored:\n     ```sql\n     select * from product;\n     ```\n   - Ensure the data returned matches the expected rows.\n\n#### 6. **Exit MySQL Session**:\n   - After verifying the data, exit the MySQL session:\n     ```bash\n     exit\n     ```\n\n#### 7. **Leave the CafeInstance SSH Window Open**:\n   - Keep the SSH window open for potential future use.\n\n---\n\n## 📝 Task 3: Configuring the website to use the Amazon RDS instance.\n\n#### 1. **Access the AWS Systems Manager Console**:\n   - In the AWS Management Console, search for and select **Systems Manager** to open the AWS Systems Manager Console.\n\n#### 2. **Navigate to Parameter Store**:\n   - In the left navigation pane, click on **Parameter Store** under **Application Management**.\n\n#### 3. **Edit the `dbUrl` Parameter**:\n   - In the **My Parameters** section, locate the `/cafe/dbUrl` parameter.\n   - Click on the parameter name to view its details.\n   - Select the **Edit** button to modify the value of the `dbUrl` parameter.\n\n#### 4. **Update the Database URL**:\n   - In the **Value** field, replace the current value with the endpoint address of the Amazon RDS instance that you recorded earlier.\n   - This action updates the website’s configuration to point to the RDS database instead of the local database.\n\n#### 5. **Save Changes**:\n   - After updating the database URL, click on **Save changes** to save the new value.\n\n#### 6. **Test the Website**:\n   - Open a new browser window and enter the URL for the café website (e.g., **CafeInstanceURL**) that you copied earlier.\n   - The homepage of the website should load successfully, confirming that the connection to the RDS instance is working.\n\n#### 7. **Verify the Data**:\n   - Click on the **Order History** tab on the website.\n   - Compare the number of orders displayed on the website with the number of orders you recorded before migrating the database to Amazon RDS.\n   - Both numbers should be the same, confirming that the data has been successfully migrated and is accessible from the new RDS instance.\n\n#### 8. **Place New Orders (Optional)**:\n   - To further verify the functionality, place some new orders on the website.\n   - Ensure that the orders are placed successfully and that the data is being written to the Amazon RDS instance.\n\n#### 9. **Close the Browser Tab**:\n   - Once the testing is complete, you can close the browser tab.\n\n--- \n\n\n## 📝 Task 4: Monitoring the Amazon RDS Database\n\n#### 1. **Access the RDS Management Console**:\n   - In the AWS Management Console, search for and select **RDS** to open the Amazon RDS Management Console.\n\n#### 2. **Select the Database Instance**:\n   - In the left navigation pane, choose **Databases**.\n   - From the list of available database instances, select the **cafedbinstance**.\n\n#### 3. **Monitor Performance Metrics**:\n   - Once the database instance details are displayed, navigate to the **Monitoring** tab.\n   - The **Monitoring** tab shows key metrics provided by Amazon RDS through CloudWatch, such as:\n     - **CPUUtilization**: The percentage of CPU being used by the database instance.\n     - **DatabaseConnections**: The number of active database connections.\n     - **FreeStorageSpace**: The available storage space in the database.\n     - **FreeableMemory**: The amount of free memory (RAM) available.\n     - **WriteIOPS**: The average number of write operations per second.\n     - **ReadIOPS**: The average number of read operations per second.\n   - Some of these metrics may appear across multiple pages, so use the pagination options to explore more metrics.\n\n#### 4. **Monitor Database Connections**:\n   - To track the number of active database connections, observe the **DatabaseConnections** metric.\n   - The graph will show the number of active connections over time.\n\n#### 5. **Establish a Database Connection (Optional)**:\n   - To create a connection to the RDS instance, use an interactive SQL session.\n   - Open the terminal on your CafeInstance (EC2 instance) and enter the following command, replacing `\u003cRDS Instance Database Endpoint Address\u003e` with the actual endpoint address of your Amazon RDS instance:\n     ```bash\n     mysql --user=root --password='Re:Start!9' --host=\u003cRDS Instance Database Endpoint Address\u003e cafe_db\n     ```\n   - This command establishes a connection to the RDS instance, and the **DatabaseConnections** graph should now reflect one open connection.\n\n#### 6. **Execute a Query**:\n   - Once connected, run the following SQL query to retrieve data from the `product` table:\n     ```sql\n     select * from product;\n     ```\n   - Verify that the query returns the expected data.\n\n#### 7. **Close the Database Connection**:\n   - To close the database connection, type `exit` in the SQL session:\n     ```bash\n     exit\n     ```\n   - After closing the connection, wait for about a minute, then click **Refresh** in the **DatabaseConnections** graph.\n   - The graph should show that the number of active connections has decreased to zero.\n\n#### 8. **Explore Additional Metrics (Optional)**:\n   - If time allows, explore the other available CloudWatch graphs to monitor additional performance metrics for the RDS instance.\n\n--- \n\n\n## 🗑️ Cleaning Up Resources\n\nTo clean up all resources, delete them in the following order:\n\n1. **RDS Instance**:\n   - Go to **RDS Dashboard** → **Databases**.\n   - Select your database and click **Delete**.\n   \n2. **EC2 Instance**:\n   - Go to **EC2 Dashboard** → **Instances**.\n   - Select the EC2 instance and click **Terminate**.\n\n3. **Backup Files** (if any):\n   - Delete any files from your local system or S3 bucket.\n\n---\n\n---\n\n## 📸 Outputs \u0026 Screenshots  \n\nThis section provides the expected outputs and screenshots for each task to help visualize the steps and verify successful execution.  \n\n### 🖼️ Task 1: Creating an Amazon RDS instance  \n\n✅ Screenshot: RDS instance details in AWS Management Console\n  \n![Screenshot 2025-02-16 225558](https://github.com/user-attachments/assets/e5277d70-0696-4fa7-9aa8-7c86fe6bcf01)\n\n### 🖼️ Task 2: Migrating application data to Amazon RDS    \n\n✅ Screenshot: Query results from the RDS instance\n  \n![Screenshot 2025-02-17 163850](https://github.com/user-attachments/assets/38af7309-d956-411d-abd2-b58e50da0ad1)\n\n\n### 🖼️ Task 3: Configuring the website to use Amazon RDS  \n\n✅ Screenshot: Application configuration file update\n  \n![Screenshot 2025-02-17 163840](https://github.com/user-attachments/assets/7ae765bc-6d6d-4c4c-8caa-ed481e6f2667)\n\n✅ Screenshot: Web application running with RDS connection\n  \n![Screenshot 2025-02-16 222451](https://github.com/user-attachments/assets/86fa7ecb-61c3-40b4-825a-67dc639ec6a9)\n\n\n### 🖼️ Task 4: Monitoring the Amazon RDS Database  \n\n✅ Screenshot: CloudWatch metrics (CPUUtilization, DatabaseConnections, FreeStorageSpace)\n \n![Screenshot 2025-02-17 163811](https://github.com/user-attachments/assets/896a5989-aabb-430e-9ff3-c5c21f465fa6)\n\n\n\n## ✅ Conclusion\n\n- Accessed the RDS Management Console and reviewed key database metrics.\n- Monitored the **DatabaseConnections** metric to observe live connections.\n- Used an interactive SQL session to test the connection and query the database.\n- Learned how to use CloudWatch to monitor Amazon RDS performance and health metrics.\n\n\n---\n\n## References\n\nThis project was inspired by AWS RDS migration tutorials and documentation.\n\n---\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiambotcoder%2Faws-rds-migration","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fiambotcoder%2Faws-rds-migration","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiambotcoder%2Faws-rds-migration/lists"}