https://github.com/long2ice/fettler

Auto refresh cache of redis with MySQL binlog
https://github.com/long2ice/fettler

binlog cache mysql redis

Last synced: 23 days ago
JSON representation

Auto refresh cache of redis with MySQL binlog

• Host: GitHub
• URL: https://github.com/long2ice/fettler
• Owner: long2ice
• License: apache-2.0
• Created: 2021-03-02T14:46:27.000Z (over 3 years ago)
• Default Branch: master
• Last Pushed: 2021-04-07T07:59:29.000Z (over 3 years ago)
• Last Synced: 2024-10-04T13:13:20.706Z (about 2 months ago)
• Topics: binlog, cache, mysql, redis
• Language: Python
• Homepage: https://github.com/long2ice/fettler
• Size: 167 KB
• Stars: 4
• Watchers: 3
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Fettler

[![image](https://img.shields.io/pypi/v/fettler.svg?style=flat)](https://pypi.python.org/pypi/fettler)

[![image](https://img.shields.io/github/license/long2ice/fettler)](https://github.com/long2ice/fettler)

[![pypi](https://github.com/long2ice/fettler/actions/workflows/pypi.yml/badge.svg)](https://github.com/long2ice/fettler/actions/workflows/pypi.yml)

[![ci](https://github.com/long2ice/fettler/actions/workflows/ci.yml/badge.svg)](https://github.com/long2ice/fettler/actions/workflows/ci.yml)

## Introduction

`Fettler` is a service that help you refresh redis cache automatically. By listening on MySQL binlog, you can refresh

redis cache in a timely manner and devoid of sensation or consciousness.

![architecture](./images/architecture.png)

## Install

Just install from pypi:

```shell

> pip install fettler

```

## Usage

### Config file

The example can be found in [config.yml](./config.yml).

### Run services

First you should run the services, which include `producer`, `consumer`.

#### Use `docker-compose`(recommended)

```shell

docker-compose up -d --build

```

Then the services is running.

#### Run manual

##### Run producer

The producer listens on MySQL binlog and send data changes to redis message queue.

```shell

> fettler produce

2021-03-17 23:10:23.228 | INFO     | fettler.producer:run:36 - Start producer success, listening on binlog from schemas ['test']....

```

##### Run consumer

The consumer consume message queue and delete invalid caches by data changes from binlog and refresh policy registered.

```shell

> fettler consume

2021-03-17 23:10:36.953 | INFO     | fettler.consumer:run:21 - Start consumer success, waiting for data changes and delete invalid caches...

```

##### Run both producer and consumer in one command

If you only need one consumer, you can just run the producer and consumer in one command.

```shell

> fettler start

2021-03-17 23:10:05.226 | INFO     | fettler.consumer:run:21 - Start consumer success, waiting for data changes and delete invalid caches...

2021-03-17 23:10:05.230 | INFO     | fettler.producer:run:36 - Start producer success, listening on binlog from schemas ['test']....

```

## Register cache refresh policy

See [examples](./examples) to see how to add cache refresh policy in you application.

## License

This project is licensed under the [Apache-2.0](./LICENSE) License.