https://github.com/zendesk/maxwell

Maxwell's daemon, a mysql-to-json kafka producer
https://github.com/zendesk/maxwell

Last synced: about 2 months ago
JSON representation

Maxwell's daemon, a mysql-to-json kafka producer

• Host: GitHub
• URL: https://github.com/zendesk/maxwell
• Owner: zendesk
• License: other
• Created: 2015-02-09T22:06:13.000Z (over 9 years ago)
• Default Branch: master
• Last Pushed: 2024-03-27T07:55:40.000Z (3 months ago)
• Last Synced: 2024-04-09T16:18:43.811Z (3 months ago)
• Language: Java
• Homepage: https://maxwells-daemon.io/
• Size: 24.5 MB
• Stars: 3,906
• Watchers: 521
• Forks: 992
• Open Issues: 220
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • License: LICENSE

Lists

• awesome-kafka - Maxwell
• awesome-kafka - Maxwell's daemon - a mysql-to-json kafka producer
• awesome-stars - zendesk/maxwell - to-json kafka producer (Java)
• awesome-stars - maxwell - to-json kafka producer | zendesk | 3002 | (Java)
• awesome-kafka - Maxwell

README

        
This is __Maxwell's daemon__, a [change data capture](https://www.confluent.io/blog/how-change-data-capture-works-patterns-solutions-implementation/) application 

that reads MySQL binlogs and writes data changes as JSON to Kafka, Kinesis, and other streaming platforms.

[↓ Download](https://github.com/zendesk/maxwell/releases/download/v1.41.0/maxwell-1.41.0.tar.gz) \|

[⚝ Source / Community](https://github.com/zendesk/maxwell) \|

[☝ Getting Started](/quickstart) \|

[☷ Reference](/config)

__What's it for?__

- ETL of all sorts

- maintaining an audit log of all changes to your database

- cache building/expiring

- search indexing 

- inter-service communication

__It goes like this:__

```

  mysql> update `test`.`maxwell` set mycol = 55, daemon = 'Stanislaw Lem';

  maxwell -> kafka: 

  {

    "database": "test",

    "table": "maxwell",

    "type": "insert",

    "ts": 1449786310,

    "data": { "id":1, "daemon": "Stanislaw Lem", "mycol": 55 },

    "old": { "mycol":, 23, "daemon": "what once was" }

  }

```