Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/akmalovaa/meilisearch-backup
meilisearch backup s3
https://github.com/akmalovaa/meilisearch-backup
backup docker docker-compose dump meilisearch minio s3 script
Last synced: 5 days ago
JSON representation
meilisearch backup s3
- Host: GitHub
- URL: https://github.com/akmalovaa/meilisearch-backup
- Owner: akmalovaa
- Created: 2024-02-13T09:18:25.000Z (12 months ago)
- Default Branch: main
- Last Pushed: 2024-02-14T07:53:57.000Z (12 months ago)
- Last Synced: 2024-02-14T10:30:16.628Z (11 months ago)
- Topics: backup, docker, docker-compose, dump, meilisearch, minio, s3, script
- Language: Python
- Homepage:
- Size: 2.57 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Meilisearch
Meilisearch is a RESTful search API
- https://www.meilisearch.com/
- https://github.com/meilisearch/meilisearch
## Meilisearch-backup
Simple script to backup dump file on s3 storage
Feature:
- containerized
- automatic scheduled
- easy restore dump file
- use in docker-compose or k8s sidecar container
- save the latest 3 versions by default
- customizable
This is a python container running in docker-compose or cronjob next to **meilisearch**
Containers share a common `/tmp/dumps` directory
### Backup
Backup is done automatically every day in `MEILI_BACKUP_TIME` or you can run `python meilisearch_backup/action/backup.py` manually when connecting to the container
S3 stores the 3 latest versions, you can change the variable `s3_number_of_buckets`
### Restore (only manual)
you can run `python meilisearch_backup/action/restore.py` manually when connecting to the container
```bash
docker-compose exec meilisearch-backup bash
python restore.py
```
In the meilisearch container, make sure the dump is downloaded `/tmp/dumps/meilisearch_backup.dump`
To restore, delete the current data and restart
```bash
docker-compose exec meilisearch sh
rm -rf data.ms
reboot
```
### Dev/Test docker-compose
`docker-compose.yaml` is used for develop and test local,
Build
```bash
task build
```
Run docker-compose
```bash
task up
```
You need to fill in some data in the **meilisearch** database to check it. Because the minimum size of the dump file = 500Kb
you can use the `test` directory to add data
Run backup
```bash
task run-backup
```
You can check in S3 minio gui that the dump file is loaded `http://localhost:9001`
### Settings
All script settings can be found in the file - `meilisearch_backup/meilisearch_backup/settings.py`
| Variables | Default | Descriptions |
|----------|----------|----------|
| meili_url | http://meilisearch:7700 | Meilisearch url |
| meili_master_key | DEVLOCALMASTERKEY | Meilisearch MASTER KEY |
| meili_dump_dir | /tmp/dumps | dumps dir |
| meili_backup_time | 01:00:00 | Every day at this time does a dump |
| meili_max_dump_create_time_sec | 3600 | Maximum time for dump task execution |
| meili_min_dump_size_bytes | 500000 | 500kb Minimum dump file size (for protection in case of data loss to avoid overwriting current dump files) |
| s3_backup | True | To disable sending to S3 (will only create dump in local filesystem) |
| s3_endpoint | http://minio:9000| S3 url |
| s3_access_key | minioadmin | S3 access key |
| s3_secret_key | minioadmin | S3 secret key |
| s3_meilisearch_bucket | meilisearch-backup | S3 bucket name |
| s3_backup_name | meilisearch_backup.dump | recovery dump file name |
| s3_number_of_saved_buckets | 3 | number of files to be saved S3 |
| number_of_saved_dumps | 3 | number of files to be saved in the local file system |
---
k8s sidecar container example:
```YAML
containers:
- name: meilisearch-backup
image: meilisearch-backup:1.1.0
command:
- python
args:
- '-m'
- meilisearch_backup
resources:
limits:
cpu: 200m
memory: 256Mi
requests:
cpu: 100m
memory: 128Mi
envFrom:
- configMapRef:
name: meilisearch-environment
volumeMounts:
- name: tmp
mountPath: /tmp
```