Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/integrii/go-redis-migrator
A cluster aware Redis key migrator. Moves keys from one cluster or host to another cluster or host.
https://github.com/integrii/go-redis-migrator
cluster go migrate redis
Last synced: about 8 hours ago
JSON representation
A cluster aware Redis key migrator. Moves keys from one cluster or host to another cluster or host.
- Host: GitHub
- URL: https://github.com/integrii/go-redis-migrator
- Owner: integrii
- License: unlicense
- Created: 2015-07-08T07:18:37.000Z (over 9 years ago)
- Default Branch: master
- Last Pushed: 2021-07-05T07:00:41.000Z (over 3 years ago)
- Last Synced: 2024-06-19T00:28:43.621Z (5 months ago)
- Topics: cluster, go, migrate, redis
- Language: Go
- Homepage:
- Size: 51.8 KB
- Stars: 34
- Watchers: 4
- Forks: 8
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
A cluster aware Redis key migrator. Moves keys from one cluster or host to another cluster or host.
## Details
It is possible to use the -keyFilter flag to match a subset of keys you wish to migrate. It is also possible to fetch a list of all keys with this program, create a file with a subset of keys to migrate, and then feed that list back into this program for selective migration. This is useful if you have some keys hanging around that you do not want to migrate and want to prune during the migration.
go-redis-migrator is especially useful for populating test data in development instances, executing production migrations, or moving from a non-clustered to a clustered deployment.
## Setup
- Make sure you have golang installed on your operating system.
- Make sure you have your `$GOPATH` environment variable setup properly.
- Download it: `go get github.com/integrii/go-redis-migrator`
- Build and install it: `go install github.com/integrii/go-redis-migrator`
- Finally, run the binary to see the help: `$GOPATH/bin/go-redis-migrator`
## Example
```
./go-redis-migrator -copyData -sourceHosts=172.31.37.164:6379,172.31.37.162:6379,172.31.37.168:6379,172.31.37.170:6379,172.31.37.169:6379 -destinationHosts=172.31.34.231:6379,172.31.34.228:6379,172.31.34.227:6379,172.31.34.230:6379,172.31.34.229:6379,172.31.34.226:6379 -keyFile=keyList.txt
Migrated 57 keys.
```
## Speed Test
```
time ./go-redis-migrator -sourceHosts=127.0.0.1:6379 -destinationHosts=127.0.0.1:6380 -copyData
Migrated 22000 keys.
real 0m3.455s
user 0m0.812s
sys 0m1.475s
```
**6,367 keys/sec!**
## More Examples
#### Fetching all keys from a host:
`./go-redis-migrator -getKeys -sourceHosts=127.0.0.1:6379`
#### Fetch a filtered key list from a host:
`./go-redis-migrator -getKeys -keyFilter=sessions:\* -sourceHosts=127.0.0.1:6379`
#### Fetch a filtered key list from a cluster:
`./go-redis-migrator -getKeys -keyFilter=sessions:\* -sourceHosts=127.0.0.1:6379,127.0.0.1:6380`
#### From a host to a host:
`./go-redis-migrator -copyData -sourceHosts=127.0.0.1:6379 -destinationHosts=192.168.0.1:6379`
#### From a cluster to a cluster:
`./go-redis-migrator -copyData -sourceHosts=127.0.0.1:6379,127.0.0.1:6380 -destinationHosts=192.168.0.1:6379,192.168.0.1:6380`
#### From a cluster to a single host:
`./go-redis-migrator -copyData -sourceHosts=127.0.0.1:6379,127.0.0.1:6380 -destinationHosts=192.168.0.1:6379`
#### From a single host to a cluster:
`./go-redis-migrator -copyData -sourceHosts=127.0.0.1:6379 -destinationHosts=192.168.0.1:6379,192.168.0.1:6380`
#### Copying keys specified in a file from a host to a host:
`./go-redis-migrator -copydata=true -sourceHosts=127.0.0.1:6379 -destinationHosts=192.168.0.1:6379 -keyFile=./onlyMoveTheseKeys.txt`
## CLI help
Simply run the binary to get the following help:
```
- Redis Key Migrator -
https://github.com/integrii/go-redis-migrator
Migrates all or some of the keys from a Redis host or cluster to a specified host or cluster. Supports migrating TTLs.
go-redis-migrator can also be used to list the keys in a cluster. Run with the -getKey and -sourceHosts= flags to do so.
You must run this program with an operation before it will do anything.
Flags:
-getKeys=false: Fetches and prints keys from the source host.
-copyData=false: Copies all keys in a list specified by -keyFile= to the destination cluster from the source cluster.
-keyFile="": The file path which contains the list of keys to migrate. One per line.
-keyFilter="*": The filter for which keys to migrate. Does not work when -keyFile is also specified.
-destinationHosts="": A list of source cluster host:port servers seperated by spaces. Use a single host without a ',' if there is no cluster. EX) 127.0.0.1:6379,127.0.0.1:6380
-sourceHosts="": A list of source cluster host:port servers seperated by commas. Use a single host without a ',' if there is no cluster. EX) 127.0.0.1:6379,127.0.0.1:6380
```