{"id":13458675,"url":"https://github.com/leonchen83/redis-replicator","last_synced_at":"2026-03-05T20:03:22.188Z","repository":{"id":55005922,"uuid":"65626089","full_name":"leonchen83/redis-replicator","owner":"leonchen83","description":"Redis replication tool. support sync, psync, psync2. can parse rdb, aof, mixed rdb and aof files. support redis-7.2","archived":false,"fork":false,"pushed_at":"2024-02-21T09:51:58.000Z","size":10599,"stargazers_count":969,"open_issues_count":6,"forks_count":273,"subscribers_count":78,"default_branch":"master","last_synced_at":"2024-10-29T03:33:21.365Z","etag":null,"topics":["aof","parser","psync2","rdb","redis","redis-replication","replication"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/leonchen83.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"custom":["https://www.paypal.com/paypalme/leonchen83"]}},"created_at":"2016-08-13T16:20:35.000Z","updated_at":"2024-10-25T08:08:53.000Z","dependencies_parsed_at":"2023-02-19T14:46:00.333Z","dependency_job_id":"88849859-6c96-444d-8e8e-bf19cf4368d0","html_url":"https://github.com/leonchen83/redis-replicator","commit_stats":null,"previous_names":[],"tags_count":69,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leonchen83%2Fredis-replicator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leonchen83%2Fredis-replicator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leonchen83%2Fredis-replicator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leonchen83%2Fredis-replicator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/leonchen83","download_url":"https://codeload.github.com/leonchen83/redis-replicator/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245298148,"owners_count":20592550,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aof","parser","psync2","rdb","redis","redis-replication","replication"],"created_at":"2024-07-31T09:00:55.070Z","updated_at":"2026-03-05T20:03:22.177Z","avatar_url":"https://github.com/leonchen83.png","language":"Java","funding_links":["https://www.paypal.com/paypalme/leonchen83"],"categories":["Java","数据库中间件"],"sub_categories":[],"readme":"Table of Contents ([中文说明](./README.zh_CN.md))\n=================\n\n* [1. Redis-replicator](#1-redis-replicator)\n    * [1.1. Brief Introduction](#11-brief-introduction)\n    * [1.2. Chat with Author](#12-chat-with-author)\n    * [1.3. Contact the Author](#13-contact-the-author)\n* [2. Installation](#2-installation)\n    * [2.1. Requirements](#21-requirements)\n    * [2.2. Maven Dependency](#22-maven-dependency)\n    * [2.3. Install from Source Code](#23-install-from-source-code)\n    * [2.4. Select a Version](#24-select-a-version)\n* [3. Simple Usage](#3-simple-usage)\n    * [3.1. Basic Usage](#31-basic-usage)\n    * [3.2. Backup Remote RDB Snapshot](#32-backup-remote-rdb-snapshot)\n    * [3.3. Backup Remote Commands](#33-backup-remote-commands)\n    * [3.4. Convert RDB to Dump Format](#34-convert-rdb-to-dump-format)\n    * [3.5. RDB Check](#35-rdb-check)\n    * [3.6. Scan and PSYNC](#36-scan-and-psync)\n    * [3.7. Other Examples](#37-other-examples)\n* [4. Advanced Topics](#4-advanced-topics)\n    * [4.1. Command Extension](#41-command-extension)\n        * [4.1.1. Write a Command](#411-write-a-command)\n        * [4.1.2. Write a Command Parser](#412-write-a-command-parser)\n        * [4.1.3. Register the Parser](#413-register-the-parser)\n        * [4.1.4. Handle Command Event](#414-handle-command-event)\n        * [4.1.5. Putting It All Together](#415-putting-it-all-together)\n    * [4.2. Module Extension](#42-module-extension)\n        * [4.2.1. Compile Redis Test Modules](#421-compile-redis-test-modules)\n        * [4.2.2. Uncomment in redis.conf](#422-uncomment-in-redisconf)\n        * [4.2.3. Write a Module Parser](#423-write-a-module-parser)\n        * [4.2.4. Write a Command Parser](#424-write-a-command-parser)\n        * [4.2.5. Register Parsers and Handle Events](#425-register-parsers-and-handle-events)\n        * [4.2.6. Putting It All Together](#426-putting-it-all-together)\n    * [4.3. Stream](#43-stream)\n    * [4.4. Write Your Own RDB Parser](#44-write-your-own-rdb-parser)\n    * [4.5. Redis URI](#45-redis-uri)\n* [5. Other Topics](#5-other-topics)\n    * [5.1. Built-in Command Parsers](#51-built-in-command-parsers)\n    * [5.2. EOFException](#52-eofexception)\n    * [5.3. Trace Event Log](#53-trace-event-log)\n    * [5.4. SSL Connection](#54-ssl-connection)\n    * [5.5. Authentication](#55-authentication)\n    * [5.6. Avoid Full Sync](#56-avoid-full-sync)\n    * [5.7. Lifecycle Events](#57-lifecycle-events)\n    * [5.8. Handle Huge Key-Value Pairs](#58-handle-huge-key-value-pairs)\n    * [5.9. Redis 6 Support](#59-redis-6-support)\n        * [5.9.1. SSL Support](#591-ssl-support)\n        * [5.9.2. ACL Support](#592-acl-support)\n    * [5.10. Redis 7 Support](#510-redis-7-support)\n        * [5.10.1. Function](#5101-function)\n    * [5.11. Redis 7.4 Support](#511-redis-74-support)\n        * [5.11.1. TTL Hash](#5111-ttl-hash)\n* [6. Contributors](#6-contributors)\n* [7. Consulting](#7-consulting)\n* [8. References](#8-references)\n* [9. Supported By](#9-supported-by)\n    * [9.1. 宁文君](#91-宁文君)\n    * [9.2. YourKit](#92-yourkit)\n    * [9.3. IntelliJ IDEA](#93-intellij-idea)\n    * [9.4. Redisson](#94-redisson)\n\n# 1. Redis-replicator\n\n\u003ca href=\"https://www.paypal.com/paypalme/leonchen83\" target=\"_blank\"\u003e\u003cimg src=\"https://github.com/leonchen83/share/blob/master/other/buymeacoffee.jpg?raw=true\" alt=\"Buy Me A Coffee\" style=\"height: 41px !important;width: 174px !important;box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;-webkit-box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;\" \u003e\u003c/a\u003e\n\n## 1.1. Brief Introduction\n[![Java CI with Maven](https://github.com/leonchen83/redis-replicator/actions/workflows/maven.yml/badge.svg?branch=master)](https://github.com/leonchen83/redis-replicator/actions/workflows/maven.yml)\n[![Coverage Status](https://coveralls.io/repos/github/leonchen83/redis-replicator/badge.svg?branch=master)](https://coveralls.io/github/leonchen83/redis-replicator?branch=master)\n[![Maven Central](https://maven-badges.herokuapp.com/maven-central/com.moilioncircle/redis-replicator/badge.svg)](https://maven-badges.herokuapp.com/maven-central/com.moilioncircle/redis-replicator)\n[![Javadocs](http://www.javadoc.io/badge/com.moilioncircle/redis-replicator.svg)](http://www.javadoc.io/doc/com.moilioncircle/redis-replicator)\n[![Hex.pm](https://img.shields.io/hexpm/l/plug.svg?maxAge=2592000)](https://github.com/leonchen83/redis-replicator/blob/master/LICENSE)\n[![LICENSE](https://img.shields.io/badge/license-Anti%20996-blue.svg?style=flat-square)](./ANTI-996-LICENSE)\n\nRedis Replicator is an implementation of the Redis Replication protocol written in Java. It can parse, filter, and broadcast RDB and AOF events in real-time. It can also synchronize Redis data to a local cache or a database. In this document, `Command` refers to writable commands (e.g., `set`, `hmset`) and excludes readable commands (e.g., `get`, `hmget`). Supports Redis 8.4.x and older versions.\n\n## 1.2. Chat with Author\n\n[![Join the chat at https://gitter.im/leonchen83/redis-replicator](https://badges.gitter.im/leonchen83/redis-replicator.svg)](https://gitter.im/leonchen83/redis-replicator?utm_source=badge\u0026utm_medium=badge\u0026utm_campaign=pr-badge\u0026utm_content=badge)\n\n## 1.3. Contact the Author\n\n**chen.bao.yi@gmail.com**\n\n# 2. Installation\n## 2.1. Requirements\n- **Compile:** JDK 9+\n- **Runtime:** JDK 8+\n- **Maven:** 3.3.1+\n- **Redis:** 2.6 - 8.4\n\n## 2.2. Maven Dependency\n```xml\n\u003cdependency\u003e\n    \u003cgroupId\u003ecom.moilioncircle\u003c/groupId\u003e\n    \u003cartifactId\u003eredis-replicator\u003c/artifactId\u003e\n    \u003cversion\u003e3.11.0\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\n## 2.3. Install from Source Code\n\n```bash\n# Step 1: Install JDK 11+ for compilation\n# Step 2: Clone the repository\ngit clone https://github.com/leonchen83/redis-replicator.git\n# Step 3: Navigate to the project directory\ncd redis-replicator\n# Step 4: Build the project\nmvn clean install package -DskipTests\n```\n\n## 2.4. Select a Version\n\n| **Redis Version** | **redis-replicator Version** |\n|-------------------|------------------------------|\n| [2.6, 8.4.x]      | [3.11.0,     ]               |\n| [2.6, 8.2.x]      | [3.10.0,3.10.0]              |\n| [2.6, 8.0.x]      | [3.9.0, 3.9.0]               |\n| [2.6, 7.2.x]      | [3.8.0, 3.8.1]               |\n| [2.6, 7.0.x]      | [3.6.4, 3.7.0]               |\n| [2.6, 7.0.x-RC2]  | [3.6.2, 3.6.3]               |\n| [2.6, 7.0.0-RC1]  | [3.6.0, 3.6.1]               |\n| [2.6, 6.2.x]      | [3.5.2, 3.5.5]               |\n| [2.6, 6.2.0-RC1]  | [3.5.0, 3.5.1]               |\n| [2.6, 6.0.x]      | [3.4.0, 3.4.4]               |\n| [2.6, 5.0.x]      | [2.6.1, 3.3.3]               |\n| [2.6, 4.0.x]      | [2.3.0, 2.5.0]               |\n| [2.6, 4.0-RC3]    | [2.1.0, 2.2.0]               |\n| [2.6, 3.2.x]      | [1.0.18] (not supported)     |\n\n\n# 3. Simple Usage\n\n## 3.1. Basic Usage\n\n```java\nReplicator replicator = new RedisReplicator(\"redis://127.0.0.1:6379\");\nreplicator.addEventListener(new EventListener() {\n    @Override\n    public void onEvent(Replicator replicator, Event event) {\n        if (event instanceof KeyStringValueString) {\n            KeyStringValueString kv = (KeyStringValueString) event;\n            System.out.println(new String(kv.getKey()));\n            System.out.println(new String(kv.getValue()));\n        } else {\n            // ...\n        }\n    }\n});\nreplicator.open();\n```\n\n## 3.2. Backup Remote RDB Snapshot\n\nSee [RdbBackupExample.java](./examples/com/moilioncircle/examples/backup/RdbBackupExample.java)\n\n## 3.3. Backup Remote Commands\n\nSee [CommandBackupExample.java](./examples/com/moilioncircle/examples/backup/CommandBackupExample.java)\n\n## 3.4. Convert RDB to Dump Format\n\nYou can use `DumpRdbVisitor` to convert an RDB file to the Redis [DUMP](https://redis.io/commands/dump) format.\n\n```java\nReplicator r = new RedisReplicator(\"redis:///path/to/dump.rdb\");\nr.setRdbVisitor(new DumpRdbVisitor(r));\nr.addEventListener(new EventListener() {\n    @Override\n    public void onEvent(Replicator replicator, Event event) {\n        if (!(event instanceof DumpKeyValuePair)) return;\n        DumpKeyValuePair dkv = (DumpKeyValuePair) event;\n        byte[] serialized = dkv.getValue();\n        // We can use the Redis RESTORE command to migrate this serialized value to another Redis instance.\n    }\n});\nr.open();\n```\n\n## 3.5. RDB Check\n\nYou can use `SkipRdbVisitor` to check the correctness of an RDB file.\n\n```java\nReplicator r = new RedisReplicator(\"redis:///path/to/dump.rdb\");\nr.setRdbVisitor(new SkipRdbVisitor(r));\nr.open();\n```\n\n## 3.6. Scan and PSYNC\n\nBy default, redis-replicator uses the `PSYNC` command, pretending to be a replica, to receive commands. An example is as follows:\n```java\nReplicator r = new RedisReplicator(\"redis://127.0.0.1:6379\");\nr.addEventListener(new EventListener() {\n    @Override\n    public void onEvent(Replicator replicator, Event event) {\n        System.out.println(event);\n    }\n});\n\nr.open();\n```\n\nHowever, on some cloud services, the `PSYNC` command is prohibited. In such cases, you can use the `SCAN` command instead:\n```java\nReplicator r = new RedisReplicator(\"redis://127.0.0.1:6379?enableScan=yes\u0026scanStep=256\");\nr.addEventListener(new EventListener() {\n    @Override\n    public void onEvent(Replicator replicator, Event event) {\n        System.out.println(event);\n    }\n});\n\nr.open();\n```\n\n## 3.7. Other Examples\n\nSee [examples](./examples/com/moilioncircle/examples/README.md)\n\n# 4. Advanced Topics\n\n## 4.1. Command Extension\n\n### 4.1.1. Write a Command\n```java\n@CommandSpec(command = \"APPEND\")\npublic static class YourAppendCommand extends AbstractCommand {\n    private final String key;\n    private final String value;\n\n    public YourAppendCommand(String key, String value) {\n        this.key = key;\n        this.value = value;\n    }\n    \n    public String getKey() {\n        return key;\n    }\n    \n    public String getValue() {\n        return value;\n    }\n}\n```\n\n### 4.1.2. Write a Command Parser\n```java\npublic class YourAppendParser implements CommandParser\u003cYourAppendCommand\u003e {\n    @Override\n    public YourAppendCommand parse(Object[] command) {\n        return new YourAppendCommand(new String((byte[]) command[1], UTF_8), new String((byte[]) command[2], UTF_8));\n    }\n}\n```\n\n### 4.1.3. Register the Parser\n```java\nReplicator replicator = new RedisReplicator(\"redis://127.0.0.1:6379\");\nreplicator.addCommandParser(CommandName.name(\"APPEND\"), new YourAppendParser());\n```\n\n### 4.1.4. Handle Command Event\n```java\nreplicator.addEventListener(new EventListener() {\n    @Override\n    public void onEvent(Replicator replicator, Event event) {\n        if(event instanceof YourAppendCommand){\n            YourAppendCommand appendCommand = (YourAppendCommand)event;\n            // Your code goes here\n        }\n    }\n});\n```\n\n### 4.1.5. Putting It All Together\n\nSee [CommandExtensionExample.java](./examples/com/moilioncircle/examples/extension/CommandExtensionExample.java)\n\n## 4.2. Module Extension\n### 4.2.1. Compile Redis Test Modules\n```bash\ncd /path/to/redis-4.0-rc2/src/modules\nmake\n```\n### 4.2.2. Uncomment in redis.conf\n\n```\nloadmodule /path/to/redis-4.0-rc2/src/modules/hellotype.so\n```\n### 4.2.3. Write a Module Parser\n```java\npublic class HelloTypeModuleParser implements ModuleParser\u003cHelloTypeModule\u003e {\n    @Override\n    public HelloTypeModule parse(RedisInputStream in, int version) throws IOException {\n        DefaultRdbModuleParser parser = new DefaultRdbModuleParser(in);\n        int elements = parser.loadUnsigned(version).intValue();\n        long[] ary = new long[elements];\n        int i = 0;\n        while (elements-- \u003e 0) {\n            ary[i++] = parser.loadSigned(version);\n        }\n        return new HelloTypeModule(ary);\n    }\n}\n\npublic class HelloTypeModule implements Module {\n    private final long[] value;\n\n    public HelloTypeModule(long[] value) {\n        this.value = value;\n    }\n\n    public long[] getValue() {\n        return value;\n    }\n}\n```\n### 4.2.4. Write a Command Parser\n```java\npublic class HelloTypeParser implements CommandParser\u003cHelloTypeCommand\u003e {\n    @Override\n    public HelloTypeCommand parse(Object[] command) {\n        String key = new String((byte[]) command[1], Constants.UTF_8);\n        long value = Long.parseLong(new String((byte[]) command[2], Constants.UTF_8));\n        return new HelloTypeCommand(key, value);\n    }\n}\n\n@CommandSpec(command = \"hellotype.insert\")\npublic class HelloTypeCommand extends AbstractCommand {\n    private final String key;\n    private final long value;\n\n    public long getValue() {\n        return value;\n    }\n\n    public String getKey() {\n        return key;\n    }\n\n    public HelloTypeCommand(String key, long value) {\n        this.key = key;\n        this.value = value;\n    }\n}\n```\n### 4.2.5. Register Parsers and Handle Events\n\n```java\npublic static void main(String[] args) throws IOException {\n    Replicator replicator = new RedisReplicator(\"redis://127.0.0.1:6379\");\n    replicator.addCommandParser(CommandName.name(\"hellotype.insert\"), new HelloTypeParser());\n    replicator.addModuleParser(\"hellotype\", 0, new HelloTypeModuleParser());\n    replicator.addEventListener(new EventListener() {\n        @Override\n        public void onEvent(Replicator replicator, Event event) {\n            if (event instanceof KeyStringValueModule) {\n                System.out.println(event);\n            }\n            \n            if (event instanceof HelloTypeCommand) {\n                System.out.println(event);\n            }\n        }\n    });\n    replicator.open();\n}\n```\n\n### 4.2.6. Putting It All Together\n\nSee [ModuleExtensionExample.java](./examples/com/moilioncircle/examples/extension/ModuleExtensionExample.java)\n\n## 4.3. Stream\n\nSince Redis 5.0, a new data structure called `STREAM` has been added. Redis-replicator parses `STREAM` data as follows:\n\n```java\nReplicator r = new RedisReplicator(\"redis://127.0.0.1:6379\");\nr.addEventListener(new EventListener() {\n    @Override\n    public void onEvent(Replicator replicator, Event event) {\n        if (event instanceof KeyStringValueStream) {\n            KeyStringValueStream kv = (KeyStringValueStream)event;\n            // Key\n            String key = kv.getKey();\n            \n            // Stream\n            Stream stream = kv.getValueAsStream();\n            // Last stream ID\n            stream.getLastId();\n            \n            // Entries\n            NavigableMap\u003cStream.ID, Stream.Entry\u003e entries = stream.getEntries();\n            \n            // Optional: Groups\n            for (Stream.Group group : stream.getGroups()) {\n                // Group PEL (Pending Entries List)\n                NavigableMap\u003cStream.ID, Stream.Nack\u003e gpel = group.getPendingEntries();\n                \n                // Consumers\n                for (Stream.Consumer consumer : group.getConsumers()) {\n                    // Consumer PEL (Pending Entries List)\n                    NavigableMap\u003cStream.ID, Stream.Nack\u003e cpel = consumer.getPendingEntries();\n                }\n            }\n        }\n    }\n});\nr.open();\n```\n\n## 4.4. Write Your Own RDB Parser\n\n*   Write a `YourRdbVisitor` that extends `RdbVisitor`.\n*   Register your `RdbVisitor` with the `Replicator` using the `setRdbVisitor` method.\n\n## 4.5. Redis URI\n\nBefore version 2.4.0, `RedisReplicator` was constructed as follows:\n\n```java\nReplicator replicator = new RedisReplicator(\"127.0.0.1\", 6379, Configuration.defaultSetting());\nReplicator replicator = new RedisReplicator(new File(\"/path/to/dump.rdb\"), FileType.RDB, Configuration.defaultSetting());\nReplicator replicator = new RedisReplicator(new File(\"/path/to/appendonly.aof\"), FileType.AOF, Configuration.defaultSetting());\nReplicator replicator = new RedisReplicator(new File(\"/path/to/appendonly.aof\"), FileType.MIXED, Configuration.defaultSetting());\n```\n\nSince version 2.4.0, we have introduced the Redis URI concept to simplify the `RedisReplicator` construction process:\n\n```java\nReplicator replicator = new RedisReplicator(\"redis://127.0.0.1:6379\");\nReplicator replicator = new RedisReplicator(\"redis:///path/to/dump.rdb\");\nReplicator replicator = new RedisReplicator(\"redis:///path/to/appendonly.aof\");\n\n// Configuration setting example\nReplicator replicator = new RedisReplicator(\"redis://127.0.0.1:6379?authPassword=foobared\u0026readTimeout=10000\u0026ssl=yes\");\nReplicator replicator = new RedisReplicator(\"redis:///path/to/dump.rdb?rateLimit=1000000\");\nReplicator replicator = new RedisReplicator(\"rediss://user:pass@127.0.0.1:6379?rateLimit=1000000\");\n```\n\n# 5. Other Topics\n\n## 5.1. Built-in Command Parsers\n\n| **Command**  | **Command**    | **Command**        | **Command**  | **Command**   | **Command**          |\n|--------------|----------------|--------------------|--------------|---------------|----------------------|\n| **PING**     | **APPEND**     | **SET**            | **SETEX**    | **MSET**      | **DEL**              |\n| **SADD**     | **HMSET**      | **HSET**           | **LSET**     | **EXPIRE**    | **EXPIREAT**         |\n| **GETSET**   | **HSETNX**     | **MSETNX**         | **PSETEX**   | **SETNX**     | **SETRANGE**         |\n| **HDEL**     | **UNLINK**     | **SREM**           | **LPOP**     | **LPUSH**     | **LPUSHX**           |\n| **LREM**     | **RPOP**       | **RPUSH**          | **RPUSHX**   | **ZREM**      | **ZINTERSTORE**      |\n| **INCR**     | **DECR**       | **INCRBY**         | **PERSIST**  | **SELECT**    | **FLUSHALL**         |\n| **FLUSHDB**  | **HINCRBY**    | **ZINCRBY**        | **MOVE**     | **SMOVE**     | **BRPOPLPUSH**       |\n| **PFCOUNT**  | **PFMERGE**    | **SDIFFSTORE**     | **RENAMENX** | **PEXPIREAT** | **SINTERSTORE**      |\n| **ZADD**     | **BITFIELD**   | **SUNIONSTORE**    | **RESTORE**  | **LINSERT**   | **ZREMRANGEBYLEX**   |\n| **GEOADD**   | **PEXPIRE**    | **ZUNIONSTORE**    | **EVAL**     | **SCRIPT**    | **ZREMRANGEBYRANK**  |\n| **PUBLISH**  | **BITOP**      | **SETBIT**         | **SWAPDB**   | **PFADD**     | **ZREMRANGEBYSCORE** |\n| **RENAME**   | **MULTI**      | **EXEC**           | **LTRIM**    | **RPOPLPUSH** | **SORT**             |\n| **EVALSHA**  | **ZPOPMAX**    | **ZPOPMIN**        | **XACK**     | **XADD**      | **XCLAIM**           |\n| **XDEL**     | **XGROUP**     | **XTRIM**          | **XSETID**   | **COPY**      | **LMOVE**            |\n| **BLMOVE**   | **ZDIFFSTORE** | **GEOSEARCHSTORE** | **FUNCTION** | **SPUBLISH**  | **HPERSIST**         |\n| **HSETEX**   | **HPEXPIREAT** | **XACKDEL**        | **XDELEX**   | **MSETEX**    |                      |\n\n## 5.2. EOFException\n\nWhen event consumption is too slow and the backlog of events exceeds the Redis backlog limit, Redis will actively disconnect from the slave. When Redis-replicator reconnects, it will perform a full synchronization. To avoid this situation, you need to set the parameter `client-output-buffer-limit slave 0 0 0`.\n\nFor more details, please refer to [redis.conf](https://raw.githubusercontent.com/antirez/redis/3.0/redis.conf).\n\n```\nclient-output-buffer-limit slave 0 0 0\n```\n**WARNING: This setting may cause the Redis server to run out of memory in some cases.**\n\n## 5.3. Trace Event Log\n\n*   Set the log level to **debug**.\n*   If you are using Log4j2, add a logger as shown below:\n\n```xml\n\u003cLogger name=\"com.moilioncircle\" level=\"debug\"\u003e\n    \u003cAppenderRef ref=\"YourAppender\"/\u003e\n\u003c/Logger\u003e\n```\n\n```java\nConfiguration.defaultSetting().setVerbose(true);\n// As a Redis URI parameter\n\"redis://127.0.0.1:6379?verbose=yes\"\n```\n\n## 5.4. SSL Connection\n\n```java\nSystem.setProperty(\"javax.net.ssl.keyStore\", \"/path/to/keystore\");\nSystem.setProperty(\"javax.net.ssl.keyStorePassword\", \"password\");\nSystem.setProperty(\"javax.net.ssl.keyStoreType\", \"your_type\");\n\nSystem.setProperty(\"javax.net.ssl.trustStore\", \"/path/to/truststore\");\nSystem.setProperty(\"javax.net.ssl.trustStorePassword\", \"password\");\nSystem.setProperty(\"javax.net.ssl.trustStoreType\", \"your_type\");\n\nConfiguration.defaultSetting().setSsl(true);\n\n// Optional settings\nConfiguration.defaultSetting().setSslSocketFactory(sslSocketFactory);\nConfiguration.defaultSetting().setSslParameters(sslParameters);\nConfiguration.defaultSetting().setHostnameVerifier(hostnameVerifier);\n\n// As a Redis URI parameter\n\"redis://127.0.0.1:6379?ssl=yes\"\n\"rediss://127.0.0.1:6379\"\n```\n\nIf you prefer not to use `System.setProperty`, you can configure it programmatically as follows:\n\n```java\nRedisSslContextFactory factory = new RedisSslContextFactory();\nfactory.setKeyStorePath(\"/path/to/redis/tests/tls/redis.p12\");\nfactory.setKeyStoreType(\"pkcs12\");\nfactory.setKeyStorePassword(\"password\");\n\nfactory.setTrustStorePath(\"/path/to/redis/tests/tls/redis.p12\");\nfactory.setTrustStoreType(\"pkcs12\");\nfactory.setTrustStorePassword(\"password\");\n\nSslConfiguration ssl = SslConfiguration.defaultSetting().setSslContextFactory(factory);\nReplicator replicator = new RedisReplicator(\"rediss://127.0.0.1:6379\", ssl);\n```\n\n## 5.5. Authentication\n\n```java\nConfiguration.defaultSetting().setAuthUser(\"default\");\nConfiguration.defaultSetting().setAuthPassword(\"foobared\");\n\n// As a Redis URI parameter\n\"redis://127.0.0.1:6379?authPassword=foobared\u0026authUser=default\"\n\"redis://default:foobared@127.0.0.1:6379\"\n```\n\n## 5.6. Avoid Full Sync\n\nAdjust the Redis server settings as follows:\n\n```\nrepl-backlog-size\nrepl-backlog-ttl\nrepl-ping-slave-period\n```\nThe `repl-ping-slave-period` **MUST** be less than `Configuration.getReadTimeout()`. The default `Configuration.getReadTimeout()` is 60 seconds.\n\n## 5.7. Lifecycle Events\n\n```java\nReplicator replicator = new RedisReplicator(\"redis://127.0.0.1:6379\");\nfinal long start = System.currentTimeMillis();\nfinal AtomicInteger acc = new AtomicInteger(0);\nreplicator.addEventListener(new EventListener() {\n    @Override\n    public void onEvent(Replicator replicator, Event event) {\n        if(event instanceof PreRdbSyncEvent) {\n            System.out.println(\"pre rdb sync\");\n        } else if(event instanceof PostRdbSyncEvent) {\n            long end = System.currentTimeMillis();\n            System.out.println(\"time elapsed:\" + (end - start));\n            System.out.println(\"rdb event count:\" + acc.get());\n        } else {\n            acc.incrementAndGet();\n        }\n    }\n});\nreplicator.open();\n```\n\n## 5.8. Handle Huge Key-Value Pairs\n\nAs mentioned in [4.4. Write Your Own RDB Parser](#44-write-your-own-rdb-parser), this tool has a built-in [Iterable Rdb Parser](./src/main/java/com/moilioncircle/redis/replicator/rdb/iterable/ValueIterableRdbVisitor.java) to handle huge key-value pairs.\nFor more details, please refer to:\n[1] [HugeKVFileExample.java](./examples/com/moilioncircle/examples/huge/HugeKVFileExample.java)\n[2] [HugeKVSocketExample.java](./examples/com/moilioncircle/examples/huge/HugeKVSocketExample.java)\n\n## 5.9. Redis 6 Support\n\n### 5.9.1. SSL Support\n\n```bash\ncd /path/to/redis\n./utils/gen-test-certs.sh\ncd tests/tls\nopenssl pkcs12 -export -CAfile ca.crt -in redis.crt -inkey redis.key -out redis.p12\ncd /path/to/redis\n./src/redis-server --tls-port 6379 --port 0 --tls-cert-file ./tests/tls/redis.crt \\\n     --tls-key-file ./tests/tls/redis.key --tls-ca-cert-file ./tests/tls/ca.crt \\\n     --tls-replication yes --bind 0.0.0.0 --protected-mode no\n```\n\n```java\nSystem.setProperty(\"javax.net.ssl.keyStore\", \"/path/to/redis/tests/tls/redis.p12\");\nSystem.setProperty(\"javax.net.ssl.keyStorePassword\", \"password\");\nSystem.setProperty(\"javax.net.ssl.keyStoreType\", \"pkcs12\");\n\nSystem.setProperty(\"javax.net.ssl.trustStore\", \"/path/to/redis/tests/tls/redis.p12\");\nSystem.setProperty(\"javax.net.ssl.trustStorePassword\", \"password\");\nSystem.setProperty(\"javax.net.ssl.trustStoreType\", \"pkcs12\");\n\nReplicator replicator = new RedisReplicator(\"rediss://127.0.0.1:6379\");\n```\n\n### 5.9.2. ACL Support\n\n```java\nReplicator replicator = new RedisReplicator(\"redis://user:pass@127.0.0.1:6379\");\n```\n\n## 5.10. Redis 7 Support\n\n### 5.10.1. Function\n\nSince Redis 7.0, `FUNCTION` is supported, and its structure is stored in the RDB file. You can use the following method to parse a `FUNCTION`.\n\n```java\nReplicator replicator = new RedisReplicator(\"redis://127.0.0.1:6379\");\nreplicator.addEventListener(new EventListener() {\n    @Override\n    public void onEvent(Replicator replicator, Event event) {\n        if (event instanceof Function) {\n            Function function = (Function) event;\n            function.getCode();\n                \n            // Your code goes here\n        }\n    }\n});\nreplicator.open();\n```\n\nYou can also parse a `FUNCTION` into serialized data and use `FUNCTION RESTORE` to restore it to a target Redis instance.\n\n```java\nReplicator replicator = new RedisReplicator(\"redis://127.0.0.1:6379\");\nreplicator.setRdbVisitor(new DumpRdbVisitor(replicator));\nreplicator.addEventListener(new EventListener() {\n    @Override\n    public void onEvent(Replicator replicator, Event event) {\n        if (event instanceof DumpFunction) {\n            DumpFunction function = (DumpFunction) event;\n            byte[] serialized = function.getSerialized();\n            // Your code goes here\n            // You can use FUNCTION RESTORE to restore the serialized data to a target Redis instance\n        }\n    }\n});\nreplicator.open();\n```\n\n## 5.11. Redis 7.4 Support\n\n### 5.11.1. TTL Hash\n\nSince Redis 7.4, `TTL HASH` is supported, and its structure is stored in the RDB file. You can use the following method to parse a `TTL HASH`.\n\n```java\nReplicator replicator = new RedisReplicator(\"redis://127.0.0.1:6379\");\nreplicator.addEventListener(new EventListener() {\n    @Override\n    public void onEvent(Replicator replicator, Event event) {\n        if (event instanceof KeyStringValueTTLHash) {\n            KeyStringValueTTLHash skv = (KeyStringValueTTLHash) event;\n            // Key\n            byte[] key = skv.getKey();\n            \n            // TTL Hash\n            Map\u003cbyte[], TTLValue\u003e ttlHash = skv.getValue();\n            for (Map.Entry\u003cbyte[], TTLValue\u003e entry : ttlHash.entrySet()) {\n                System.out.println(\"field: \" + Strings.toString(entry.getKey()));\n                System.out.println(\"value: \" + Strings.toString(entry.getValue().getValue()));\n                System.out.println(\"field ttl: \" + entry.getValue().getExpires());\n            }\n        }\n    }\n});\nreplicator.open();\n```\n\n# 6. Contributors\n* [Leon Chen](https://github.com/leonchen83)\n* [Adrian Yao](https://github.com/adrianyao89)\n* [Trydofor](https://github.com/trydofor)\n* [Argun](https://github.com/Argun)\n* [Sean Pan](https://github.com/XinYang-Pan)\n* [René Kerner](https://github.com/rk3rn3r)\n* [Maplestoria](https://github.com/maplestoria)\n* Special thanks to [Kevin Zheng](https://github.com/KevinZheng001)\n\n# 7. Consulting\n\nCommercial support for `redis-replicator` is available. The following services are currently offered:\n*   Onsite consulting: $10,000 per day\n*   Onsite training: $10,000 per day\n\nYou may also contact Baoyi Chen directly at [chen.bao.yi@gmail.com](mailto:chen.bao.yi@gmail.com).\n\n# 8. References\n* [rdb.c](https://github.com/antirez/redis/blob/unstable/src/rdb.c)\n* [Redis RDB File Format](https://github.com/leonchen83/redis-replicator/wiki/RDB-dump-data-format)\n* [Redis Protocol specification](http://redis.io/topics/protocol)\n* [Redis Replication](http://redis.io/topics/replication)\n* [Redis-replicator Design and Implementation](https://github.com/leonchen83/mycode/blob/master/redis/redis-share/Redis-replicator%E8%AE%BE%E8%AE%A1%E4%B8%8E%E5%AE%9E%E7%8E%B0.md)\n\n# 9. Supported By\n\n## 9.1. 宁文君\n\nJanuary 27, 2023, was a sad day as I lost my mother, 宁文君. She was always encouraging and supportive of my work on this tool. Every time a company started using it, she would get as excited as a child and motivate me to continue. Without her, I could not have maintained this tool for so many years. Even though I haven't achieved much, she was always proud of me. R.I.P, and may God bless her.\n\n## 9.2. YourKit\n\n![YourKit](https://www.yourkit.com/images/yklogo.png)\nYourKit is kindly supporting this open source project with its full-featured Java Profiler.\nYourKit, LLC is the creator of innovative and intelligent tools for profiling\nJava and .NET applications. Take a look at YourKit's leading software products:\n\u003ca href=\"http://www.yourkit.com/java/profiler/index.jsp\"\u003eYourKit Java Profiler\u003c/a\u003e and\n\u003ca href=\"http://www.yourkit.com/.net/profiler/index.jsp\"\u003eYourKit .NET Profiler\u003c/a\u003e.\n\n## 9.3. IntelliJ IDEA\n\n[IntelliJ IDEA](https://www.jetbrains.com/?from=redis-replicator) is a Java Integrated Development Environment (IDE) for developing computer software.\nIt is developed by JetBrains (formerly known as IntelliJ), and is available as an Apache 2 Licensed community edition,\nand in a proprietary commercial edition. Both can be used for commercial development.\n\n## 9.4. Redisson\n\n[Redisson](https://github.com/redisson/redisson), a Redis-based In-Memory Data Grid for Java, offers distributed objects and services (`BitSet`, `Set`, `Multimap`, `SortedSet`, `Map`, `List`, `Queue`, `BlockingQueue`, `Deque`, `BlockingDeque`, `Semaphore`, `Lock`, `AtomicLong`, `CountDownLatch`, `Publish / Subscribe`, `Bloom filter`, `Remote service`, `Spring cache`, `Executor service`, `Live Object service`, `Scheduler service`) backed by a Redis server. Redisson provides a more convenient and easier way to work with Redis. Redisson objects provide a separation of concerns, allowing you to focus on data modeling and application logic.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fleonchen83%2Fredis-replicator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fleonchen83%2Fredis-replicator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fleonchen83%2Fredis-replicator/lists"}