https://github.com/vutrandn94/minio-site-replication

Deploy MinIO: Site Replication
https://github.com/vutrandn94/minio-site-replication

docker docker-compose minio minio-site-replicate

Last synced: 2 months ago
JSON representation

Deploy MinIO: Site Replication

Host: GitHub
URL: https://github.com/vutrandn94/minio-site-replication
Owner: vutrandn94
Created: 2025-02-14T07:11:58.000Z (over 1 year ago)
Default Branch: main
Last Pushed: 2025-02-14T09:47:10.000Z (over 1 year ago)
Last Synced: 2025-10-19T02:34:55.175Z (8 months ago)
Topics: docker, docker-compose, minio, minio-site-replicate
Homepage:
Size: 691 KB
Stars: 1
Watchers: 1
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: README.md

Awesome Lists containing this project

README

# minio-site-replication
Deploy MinIO: Site Replication

![Alt Text](site-replicate-diagram.png)

## Requirement
- Format disk XFS for high performance
- Minimum: 2 nodes (2 servers)
- Deployment environment: Docker
- Deployment tools: Docker Compose

## Information about the servers deploying the lab

| Hostname | IP Address | Description |
| :--- | :--- | :--- |
| minio01 | 172.31.47.91 | |
| minio02 | 172.31.37.11 | |
| minio03 | 172.31.36.54 | Add Later |

**Setting hosts file (Set on all nodes)**
```
## MINIO
172.31.47.91 minio01
172.31.37.11 minio02
172.31.36.54 minio03
```

## Add site replication (2 node minio01 - minio02)
**docker-compose.yml in minio01 (Config in node "minio01"):**
```
services:
minio01:
image: 'quay.io/minio/minio:RELEASE.2025-01-20T14-49-07Z'
restart: always
environment:
MINIO_ROOT_USER: "root"
MINIO_ROOT_PASSWORD: "Enjoyday"
TZ: "Asia/Ho_Chi_Minh"
MC_HOST_minio01: "http://root:Enjoyday@minio01:9000"
MC_HOST_minio02: "http://root:Enjoyday@minio02:9000"
MC_HOST_minio03: "http://root:Enjoyday@minio03:9000"
command: server /data --console-address ":9001"
ports:
- 9000:9000
- 9001:9001
volumes:
- /mnt/data:/data
networks:
- minio-net
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
interval: 1m
timeout: 10s
retries: 3
start_period: 1m
networks:
minio-net:
driver: bridge
```

**docker-compose.yml in minio02 (Config in node "minio02"):**
```
services:
minio02:
image: 'quay.io/minio/minio:RELEASE.2025-01-20T14-49-07Z'
restart: always
environment:
MINIO_ROOT_USER: "root"
MINIO_ROOT_PASSWORD: "Enjoyday"
TZ: "Asia/Ho_Chi_Minh"
MC_HOST_minio01: "http://root:Enjoyday@minio01:9000"
MC_HOST_minio02: "http://root:Enjoyday@minio02:9000"
MC_HOST_minio03: "http://root:Enjoyday@minio03:9000"
command: server /data --console-address ":9001"
ports:
- 9000:9000
- 9001:9001
volumes:
- /mnt/data:/data
networks:
- minio-net
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
interval: 1m
timeout: 10s
retries: 3
start_period: 1m
networks:
minio-net:
driver: bridge
```

**Deploy service (Execute on all nodes):**
```
# docker-compose up -d
```

**Add site replication (Execute on minio01/minio02):**
```
root@minio01:~/minio-deploy# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b47c3db88b69 quay.io/minio/minio:RELEASE.2025-01-20T14-49-07Z "/usr/bin/docker-ent…" About a minute ago Up About a minute (healthy) 0.0.0.0:9000-9001->9000-9001/tcp minio-deploy-minio01-1
```
```
root@minio01:~/minio-deploy# docker exec -it minio-deploy-minio01-1 bash
```
```
bash-5.1# mc admin replicate add minio01 minio02
mc: Configuration written to `/tmp/.mc/config.json`. Please update your access credentials.
mc: Successfully created `/tmp/.mc/share`.
mc: Initialized share uploads `/tmp/.mc/share/uploads.json` file.
mc: Initialized share downloads `/tmp/.mc/share/downloads.json` file.
Requested sites were configured for replication successfully.
```
**Site replication configuration verify:**
```
bash-5.1# mc admin replicate info minio01
SiteReplication enabled for:

Policy replication status:
● 5/5 Policies in sync

User replication status:
No Users present

Group replication status:
No Groups present

ILM Expiry Rules replication status:
No ILM Expiry Rules present

Object replication status:
Replication status since 8 minutes
Summary:
Replicated: 0 objects (0 B)
Queued: ● 0 objects, (0 B) (avg: 0 objects, 0 B; max: 0 objects, 0 B)
Received: 0 objects (0 B)
```
**Check replicate status in node minio02:**
```
bash-5.1# mc admin replicate status minio02
Bucket replication status:
No Buckets present

Policy replication status:
● 5/5 Policies in sync

User replication status:
No Users present

Group replication status:
No Groups present

ILM Expiry Rules replication status:
No ILM Expiry Rules present

**Test replicate**
- **Create bucket in node minio01/minio02 and check the remaining node:**

![Alt Text](bucket-replicate-1.png)

![Alt Text](bucket-replicate-2.png)

- **Upload item object in node minio01/minio02 and verify object metadata in remaining node:**

![Alt Text](object-replicate-1.png)

![Alt Text](metric.png)

## (Optinal) Add site replication to expand (add node minio03)
> [!NOTE]
> Node need to add more to expand replicate size, there must be no data (Empty data). The data will automatically sync to the new node according to the default mode "asynchronous" (MinIO strongly recommends). Can change mode with "mc admin replicate update --mode" (https://min.io/docs/minio/linux/reference/minio-mc-admin/mc-admin-replicate.html#mc.admin.replicate.update.-mode)

**Execute on minio01/minio02:**
```
root@minio01:~/minio-deploy# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b47c3db88b69 quay.io/minio/minio:RELEASE.2025-01-20T14-49-07Z "/usr/bin/docker-ent…" 33 minutes ago Up 32 minutes (healthy) 0.0.0.0:9000-9001->9000-9001/tcp minio-deploy-minio01-1
```
```
root@minio01:~/minio-deploy# docker exec -it minio-deploy-minio01-1 bash
```
```
bash-5.1# mc admin replicate add minio01 minio02 minio03
Requested sites were configured for replication successfully.
```
**Site replication configuration verify:**
```
bash-5.1# mc admin replicate info minio01
SiteReplication enabled for:

**Test replicate:**

![Alt Text](expand.png)

## (Optinal) Remove a Site from Replication
> [!NOTE]
> If the node that needs to be removed crashes or cannot connect. The removal process when executing the "mc admin replicate remove" command will take longer (because it can't connect to the node that needs to be removed to clear the Replicate Policy). Please wait until the statement is complete.

```
Syntax:
mc admin replicate rm ALIAS PEER_TO_REMOVE --force

- Replace ALIAS with the alias of any peer site in the replication configuration.
- Replace PEER_TO_REMOVE with the alias of the peer site to remove.
- All healthy peers in the site replication configuration update to remove the specified peer automatically.
- MinIO requires the --force flag to remove the peer from the site replication configuration.
```

**Execution in a different node than the node to be removed (example remove node minio02):**
```
root@minio01:~/minio-deploy# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b47c3db88b69 quay.io/minio/minio:RELEASE.2025-01-20T14-49-07Z "/usr/bin/docker-ent…" 55 minutes ago Up 55 minutes (healthy) 0.0.0.0:9000-9001->9000-9001/tcp minio-deploy-minio01-1
```
```
root@minio01:~/minio-deploy# docker exec -it minio-deploy-minio01-1 bash
```
```
bash-5.1# mc admin replicate remove minio01 minio02 --force
Following site(s) [minio02] were removed successfully
```

**Site replication configuration verify:**
```
bash-5.1# mc admin replicate info minio01
SiteReplication enabled for:

**Test replicate (after remove node minio02):**

![Alt Text](remove-node.png)

## (Optinal) Some related documents and commands
- **Document:**
```
https://min.io/docs/minio/linux/operations/install-deploy-manage/multi-site-replication.html
https://min.io/docs/minio/linux/reference/minio-mc-admin/mc-admin-replicate.html#mc.admin.replicate.update.-mode
https://min.io/docs/minio/linux/reference/minio-mc-admin/mc-admin-trace.html
```

- **Commands:**
```
# mc admin trace --call storage

# mc admin replicate status --all

# mc admin replicate status --bucket

# mc admin replicate resync
```

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

https://github.com/vutrandn94/minio-site-replication

Awesome Lists containing this project

README