{"id":13816151,"url":"https://github.com/nevill/zongji","last_synced_at":"2025-04-12T03:46:22.138Z","repository":{"id":12085867,"uuid":"14673734","full_name":"nevill/zongji","owner":"nevill","description":"A mysql binlog listener running on Node.js.","archived":false,"fork":false,"pushed_at":"2024-02-19T09:28:11.000Z","size":443,"stargazers_count":380,"open_issues_count":60,"forks_count":122,"subscribers_count":17,"default_branch":"master","last_synced_at":"2025-04-12T03:46:13.605Z","etag":null,"topics":["binlog","mysql"],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nevill.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2013-11-25T02:00:15.000Z","updated_at":"2025-04-11T07:15:09.000Z","dependencies_parsed_at":"2024-01-15T20:47:32.363Z","dependency_job_id":"09c68a44-886b-4f3e-ae0f-218fd70fb80a","html_url":"https://github.com/nevill/zongji","commit_stats":{"total_commits":194,"total_committers":15,"mean_commits":"12.933333333333334","dds":"0.38144329896907214","last_synced_commit":"94aaac19fec7039a6d5c1fd0d591931347cf6a1c"},"previous_names":[],"tags_count":18,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nevill%2Fzongji","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nevill%2Fzongji/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nevill%2Fzongji/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nevill%2Fzongji/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nevill","download_url":"https://codeload.github.com/nevill/zongji/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248514213,"owners_count":21116899,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["binlog","mysql"],"created_at":"2024-08-04T05:00:36.007Z","updated_at":"2025-04-12T03:46:22.111Z","avatar_url":"https://github.com/nevill.png","language":"JavaScript","funding_links":[],"categories":["JavaScript"],"sub_categories":[],"readme":"# ZongJi [![Build Status](https://travis-ci.org/nevill/zongji.svg?branch=master)](https://travis-ci.org/nevill/zongji)\nA MySQL binlog listener running on Node.js.\n\nZongJi (踪迹) is pronounced as `zōng jì` in Chinese.\n\nThis package is a pure JS implementation based on [`mysql`](https://github.com/mysqljs/mysql). It has been tested to work in MySQL 5.5, 5.6, and 5.7.\n\n# Latest Release\n\nThe latest release is v0.5.0, only supports Node.js from v8.\n\nv0.4.7 is the last release which supports Node.js v4.x.\n\n## Quick Start\n\n```javascript\nlet zongji = new ZongJi({ /* ... MySQL Connection Settings ... */ });\n\n// Each change to the replication log results in an event\nzongji.on('binlog', function(evt) {\n  evt.dump();\n});\n\n// Binlog must be started, optionally pass in filters\nzongji.start({\n  includeEvents: ['tablemap', 'writerows', 'updaterows', 'deleterows']\n});\n```\n\nFor a complete implementation see [`example.js`](example.js)...\n\n## Installation\n\n* Requires Node.js v8+\n\n  ```bash\n  $ npm install zongji\n  ```\n\n* Enable MySQL binlog in `my.cnf`, restart MySQL server after making the changes.\n  \u003e From [MySQL 5.6](https://dev.mysql.com/doc/refman/5.6/en/replication-options-binary-log.html), binlog checksum is enabled by default. Zongji can work with it, but it doesn't really verify it.\n\n  ```\n  # Must be unique integer from 1-2^32\n  server-id        = 1\n  # Row format required for ZongJi\n  binlog_format    = row\n  # Directory must exist. This path works for Linux. Other OS may require\n  #   different path.\n  log_bin          = /var/log/mysql/mysql-bin.log\n\n  binlog_do_db     = employees   # Optional, limit which databases to log\n  expire_logs_days = 10          # Optional, purge old logs\n  max_binlog_size  = 100M        # Optional, limit log size\n  ```\n* Create an account with replication privileges, e.g. given privileges to account `zongji` (or any account that you use to read binary logs)\n\n  ```sql\n  GRANT REPLICATION SLAVE, REPLICATION CLIENT, SELECT ON *.* TO 'zongji'@'localhost'\n  ```\n\n## ZongJi Class\n\nThe `ZongJi` constructor accepts one argument of either:\n\n* An object containing MySQL connection details in the same format as used by [package mysql](https://npm.im/mysql)\n* Or, a [mysql](https://npm.im/mysql) `Connection` or `Pool` object that will be used for querying column information.\n\nIf a `Connection` or `Pool` object is passed to the constructor, it will not be destroyed/ended by Zongji's `stop()` method.\n\nIf there is a `dateStrings` `mysql` configuration option in the connection details or connection, `ZongJi` will follow it.\n\nEach instance includes the following methods:\n\nMethod Name | Arguments | Description\n------------|-----------|------------------------\n`start`     | `options` | Start receiving replication events, see options listed below\n`stop`      | *None*    | Disconnect from MySQL server, stop receiving events\n`on`        | `eventName`, `handler` | Add a listener to the `binlog` or `error` event. Each handler function accepts one argument.\n\nSome events can be emitted in different phases:\n\nEvent Name | Description\n-----------|------------------------\n`ready`    | This event is occurred right after ZongJi successfully established a connection, setup slave status, and set binlog position.\n`binlog`   | Once a binlog is received and passes the filter, it will bubble up with this event.\n`error`    | Every error will be caught by this event.\n`stopped`  | Emitted when ZongJi connection is stopped (ZongJi#stop is called).\n\n**Options available:**\n\nOption Name | Type | Description\n------------|------|-------------------------------\n`serverId`  | `integer` | [Unique number (1 - 2\u003csup\u003e32\u003c/sup\u003e)](http://dev.mysql.com/doc/refman/5.0/en/replication-options.html#option_mysqld_server-id) to identify this replication slave instance. Must be specified if running more than one instance of ZongJi. Must be used in `start()` method for effect.\u003cbr\u003e**Default:** `1`\n`startAtEnd` | `boolean` | Pass `true` to only emit binlog events that occur after ZongJi's instantiation. Must be used in `start()` method for effect.\u003cbr\u003e**Default:** `false`\n`filename` | `string` | Begin reading events from this binlog file. If specified together with `position`, will take precedence over `startAtEnd`.\n`position` | `integer` | Begin reading events from this position. Must be included with `filename`.\n`includeEvents` | `[string]` | Array of event names to include\u003cbr\u003e**Example:** `['writerows', 'updaterows', 'deleterows']`\n`excludeEvents` | `[string]` | Array of event names to exclude\u003cbr\u003e**Example:** `['rotate', 'tablemap']`\n`includeSchema` | `object` | Object describing which databases and tables to include (Only for row events). Use database names as the key and pass an array of table names or `true` (for the entire database).\u003cbr\u003e**Example:** ```{ 'my_database': ['allow_table', 'another_table'], 'another_db': true }```\n`excludeSchema` | `object` | Object describing which databases and tables to exclude (Same format as `includeSchema`)\u003cbr\u003e**Example:** ```{ 'other_db': ['disallowed_table'], 'ex_db': true }```\n\n* By default, all events and schema are emitted.\n* `excludeSchema` and `excludeEvents` take precedence over `includeSchema` and `includeEvents`, respectively.\n\n**Supported Binlog Events:**\n\nEvent name  | Description\n------------|---------------\n`unknown`   | Catch any other events\n`query`     | [Insert/Update/Delete Query](http://dev.mysql.com/doc/internals/en/query-event.html)\n`intvar`    | [Autoincrement and LAST_INSERT_ID](https://dev.mysql.com/doc/internals/en/intvar-event.html)\n`rotate`    | [New Binlog file](http://dev.mysql.com/doc/internals/en/rotate-event.html) Not required to be included to rotate to new files, but it is required to be included in order to keep the `filename` and `position` properties updated with current values for [graceful restarting on errors](https://gist.github.com/numtel/5b37b2a7f47b380c1a099596c6f3db2f).\n`format`    | [Format Description](http://dev.mysql.com/doc/internals/en/format-description-event.html)\n`xid`       | [Transaction ID](http://dev.mysql.com/doc/internals/en/xid-event.html)\n`tablemap`  | Before any row event (must be included for any other row events)\n`writerows` | Rows inserted, row data array available as `rows` property on event object\n`updaterows` | Rows changed, row data array available as `rows` property on event object\n`deleterows` | Rows deleted, row data array available as `rows` property on event object\n\n**Event Methods**\n\nNeither method requires any arguments.\n\nName   | Description\n-------|---------------------------\n`dump` | Log a description of the event to the console\n`getEventName` | Return the name of the event\n\n## Important Notes\n\n* :star2: [All types allowed by `mysql`](https://github.com/mysqljs/mysql#type-casting) are supported by this package.\n* :speak_no_evil: 64-bit integer is supported via package big-integer(see #108). If an integer is within the safe range of JS number (-2^53, 2^53), a Number object will returned, otherwise, will return as String.\n* :point_right: `TRUNCATE` statement does not cause corresponding `DeleteRows` event. Use unqualified `DELETE FROM` for same effect.\n* When using fractional seconds with `DATETIME` and `TIMESTAMP` data types in MySQL \u003e 5.6.4, only millisecond precision is available due to the limit of Javascript's `Date` object.\n\n## Run Tests\n\n* install [Docker](https://www.docker.com/community-edition#download)\n* run `docker-compose up` and then `./docker-test.sh`\n\n## Reference\n\nI learnt many things from following resources while making ZongJi.\n\n* https://github.com/mysqljs/mysql\n* https://github.com/felixge/faster-than-c/\n* http://intuitive-search.blogspot.co.uk/2011/07/binary-log-api-and-replication-listener.html\n* https://github.com/Sannis/node-mysql-libmysqlclient\n* https://kkaefer.com/node-cpp-modules/\n* http://dev.mysql.com/doc/internals/en/replication-protocol.html\n* http://www.cs.wichita.edu/~chang/lecture/cs742/program/how-mysql-c-api.html\n* https://github.com/jeremycole/mysql_binlog (Ruby implemenation of MySQL binlog parser)\n* http://dev.mysql.com/doc/internals/en/date-and-time-data-type-representation.html\n\n## License\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnevill%2Fzongji","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnevill%2Fzongji","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnevill%2Fzongji/lists"}