https://github.com/oracle-quickstart/oci-oss-mirror-maker
A quickstart to demonstrate MirorMaker2 with OSS
https://github.com/oracle-quickstart/oci-oss-mirror-maker
Last synced: 8 months ago
JSON representation
A quickstart to demonstrate MirorMaker2 with OSS
- Host: GitHub
- URL: https://github.com/oracle-quickstart/oci-oss-mirror-maker
- Owner: oracle-quickstart
- License: upl-1.0
- Archived: true
- Created: 2022-02-24T04:47:13.000Z (almost 4 years ago)
- Default Branch: main
- Last Pushed: 2022-04-07T04:38:44.000Z (over 3 years ago)
- Last Synced: 2025-02-19T21:12:48.654Z (10 months ago)
- Language: HCL
- Size: 42 KB
- Stars: 0
- Watchers: 1
- Forks: 1
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# oci-oss-mirrormaker
Mirror Maker 2 [MM2] is an open-source tool which allows users to geo replicate streams from one region to another. Following use-cases can be met with MM2
1. Active-Passive setup
2. Active-Active setup
3. Migrating from on-premise to OSS
4. Multi cloud setup
This quick start will get you started with the Mirror Maker 2 on OCI.
## Prerequisites
- Permission to `manage` the following types of resources in your Oracle Cloud Infrastructure tenancy: `vcns`, `internet-gateways`, `route-tables`, `network-security-groups`, `subnets`, `OSS`, and `instances`.
- Quota to create the following resources: 1 VCN, 1 subnets, 1 Internet Gateway, 1 NAT Gateway, 2 route rules, 2 OSS stream pools, 1 OSS Connect Harness and 1 compute instances.
If you don't have the required permissions and quota, contact your tenancy administrator. See [Policy Reference](https://docs.cloud.oracle.com/en-us/iaas/Content/Identity/Reference/policyreference.htm), [Service Limits](https://docs.cloud.oracle.com/en-us/iaas/Content/General/Concepts/servicelimits.htm), [Compartment Quotas](https://docs.cloud.oracle.com/iaas/Content/General/Concepts/resourcequotas.htm).
## Deploy Using the Terraform CLI
### Clone the Module
Now, you'll want a local copy of this repo. You can make that with the commands:
git clone https://github.com/oracle-quickstart/oci-oss-mirrormaker.git
cd oci-oss-mirrormaker
ls
### Set Up and Configure Terraform
1. Complete the prerequisites described [here](https://github.com/cloud-partners/oci-prerequisites).
2. Create a `terraform.tfvars` file, and specify the following variables:
```
# Authentication
tenancy_ocid = ""
current_user_ocid = ""
fingerprint = ""
private_key_path = ""
# Region
region = ""
targetRegion = ""
# Compartment
compartment_ocid = ""
generate_public_ssh_key = false
public_ssh_key = ""
private_ssh_key_path = ""
````
### Create the Resources
Run the following commands:
terraform init
terraform plan
terraform apply
### Destroy the Deployment
When you no longer need the deployment, you can run this command to destroy the resources:
terraform destroy
## Quick Start Architecture
Mirror Maker 2 makes use of the Kafka connect ecosystem to instantiate three connectors, which are:-
1. MirrorSourceConnector
2. MirrorCheckpointConnector
3. MirrorHeartbeatConnector
We will be creating the following resources for this Quick start
1. Stream pool in source and target regions
2. Kafka Connect Harness in the target region
3. VCN and instance in the public subnet of VCN.
4. And finally we install and run MM2 on the instance.
> **_NOTE:_** This setup is only for OSS capability demonstration purpose. In production one would use private endpoints for streams and regions would be connected by the DRG or a VPN/Fastconnect.
### Limitations of the MM2 with OSS
- OSS doesn’t support compacted topics yet. So creation of offsets, heartbeat and checkpointing topics by mirror maker will throw an exception. But when Auto Create Topics in enabled in the Stream Pool then at the time of publishing data these topics will be auto created although they will not be compacted.
- sync.group.offsets.enabled=true is not supported in OSS
- sync.topic.acls.enabled=true is not supported in OSS