https://github.com/wgrape/parseaof
A simple and fast tool to parse the AOF file of redis / 一个简单快速的解析Redis AOF文件的工具
https://github.com/wgrape/parseaof
aof go redis
Last synced: about 1 year ago
JSON representation
A simple and fast tool to parse the AOF file of redis / 一个简单快速的解析Redis AOF文件的工具
- Host: GitHub
- URL: https://github.com/wgrape/parseaof
- Owner: WGrape
- License: mit
- Created: 2021-11-30T16:59:48.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2023-10-31T15:11:55.000Z (over 2 years ago)
- Last Synced: 2025-04-24T00:03:56.564Z (about 1 year ago)
- Topics: aof, go, redis
- Language: Go
- Homepage:
- Size: 16 MB
- Stars: 21
- Watchers: 2
- Forks: 3
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
## Content
- [Content](#content)
- [1、Introduction](#1introduction)
- [(1) Features](#1-features)
- [(2) Architecture](#2-architecture)
- [2、Build](#2build)
- [3、Usage](#3usage)
- [(1) The input file](#1-the-input-file)
- [(2) The output file](#2-the-output-file)
- [(3) Example](#3-example)
- [4、Performance](#4performance)
- [(1) Testing](#1-testing)
## 1、Introduction
A simple and fast tool to parse the AOF file of redis
### (1) Features
- Code is clean, simple and easy to customize
- Speed up parsing through multiple goroutines
- A list of commands will be generated after parsing for log querying
### (2) Architecture
## 2、Build
```bash
git clone https://github.com/WGrape/parseAOF
cd parseAOF
go mod download
make build
```
## 3、Usage
Run the binary under `bin` dir `parseAOF__` with the path of the aof file
```bash
./bin/parseAOF_macos_arm64 -i ~/Download/appendonly.aof -r 8
./bin/parseAOF_macos_arm64 -h
parse redis aof to readable
Usage:
parseAOF [flags]
Flags:
-h, --help help for parseAOF
-i, --input string input AOF file path
-o, --output string output dir path
-r, --routines int max goroutines (default 8)
```
### (1) The input file
> Here's an example input file [./data/appendonly.aof](./data/appendonly.aof) for you to test
Before running, pass the path of the aof file to the ```start.sh``` script, the content is as follows
```text
*2
$6
SELECT
$1
0
... ...
```
### (2) The output file
> Here's an example output file [./data/aof.merged](./data/aof.merged) for you to test
After the parsing is complete, the file [aof.merged](./data/aof.merged) will be generated in the directory of ```data```, the content is as follows
```text
--------------------parseAOF | version=0.5.0--------------------
SELECT 0
set key1 1
set key2 2
set key3 3
sadd key4 1 2 3 4
lpush key5 1 2 3 4 5
zadd key6 1 2 3 4 5 6
```
### (3) Example
## 4、Performance
- The average speed to parse is ```50000 lines/s```
- The maximum size of the aof is 1GB
### (1) Testing
| Id | Lines | Size | Cost | CPU |
| --- | :----: | :---: | :---: | :---: |
| 1 | 1,2301,117 | 39MB | 3m50s | <=65% |
| 2 | 3,435,263 | 13MB | 1m12s | <=65% |
| 3 | 357,850 | 8.6MB | 3.47s | <=113% |