https://github.com/zendesk/maxwell

Maxwell's daemon, a mysql-to-json kafka producer
https://github.com/zendesk/maxwell

Last synced: 6 days ago
JSON representation

Maxwell's daemon, a mysql-to-json kafka producer

• Host: GitHub
• URL: https://github.com/zendesk/maxwell
• Owner: zendesk
• License: other
• Created: 2015-02-09T22:06:13.000Z (almost 10 years ago)
• Default Branch: master
• Last Pushed: 2024-10-21T20:58:37.000Z (22 days ago)
• Last Synced: 2024-10-22T15:53:46.916Z (21 days ago)
• Language: Java
• Homepage: https://maxwells-daemon.io/
• Size: 25.1 MB
• Stars: 4,026
• Watchers: 528
• Forks: 1,009
• Open Issues: 242
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-kafka - Maxwell
• awesome-kafka - Maxwell's daemon - a mysql-to-json kafka producer

README

        
This is __Maxwell's daemon__, a [change data capture](https://www.confluent.io/blog/how-change-data-capture-works-patterns-solutions-implementation/) application 

that reads MySQL binlogs and writes data changes as JSON to Kafka, Kinesis, and other streaming platforms.

[↓ Download](https://github.com/zendesk/maxwell/releases/download/v1.41.2/maxwell-1.41.2.tar.gz) \|

[⚝ Source / Community](https://github.com/zendesk/maxwell) \|

[☝ Getting Started](/quickstart) \|

[☷ Reference](/config)

__What's it for?__

- ETL of all sorts

- maintaining an audit log of all changes to your database

- cache building/expiring

- search indexing 

- inter-service communication

__It goes like this:__

```

  mysql> update `test`.`maxwell` set mycol = 55, daemon = 'Stanislaw Lem';

  maxwell -> kafka: 

  {

    "database": "test",

    "table": "maxwell",

    "type": "update",

    "ts": 1449786310,

    "data": { "id":1, "daemon": "Stanislaw Lem", "mycol": 55 },

    "old": { "mycol":, 23, "daemon": "what once was" }

  }

```