{"id":18104695,"url":"https://github.com/springmt/redo-log-parser","last_synced_at":"2025-04-06T06:17:52.756Z","repository":{"id":66034640,"uuid":"364274012","full_name":"SpringMT/redo-log-parser","owner":"SpringMT","description":null,"archived":false,"fork":false,"pushed_at":"2021-05-18T15:43:12.000Z","size":14,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-12T11:52:00.294Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SpringMT.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-05-04T13:57:35.000Z","updated_at":"2021-05-18T15:43:15.000Z","dependencies_parsed_at":"2023-06-07T07:45:37.224Z","dependency_job_id":null,"html_url":"https://github.com/SpringMT/redo-log-parser","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SpringMT%2Fredo-log-parser","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SpringMT%2Fredo-log-parser/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SpringMT%2Fredo-log-parser/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SpringMT%2Fredo-log-parser/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SpringMT","download_url":"https://codeload.github.com/SpringMT/redo-log-parser/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247441058,"owners_count":20939239,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-31T22:15:34.722Z","updated_at":"2025-04-06T06:17:52.731Z","avatar_url":"https://github.com/SpringMT.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# redo-log-parser\n\n## redo log\n* WAL (Write-ahead logging): データファイルを書き込む前、redoログに書き込むのを保証する\n\nMySQL 5.6 https://dev.mysql.com/doc/refman/5.6/en/innodb-redo-log.html\nMySQL 8.0 https://dev.mysql.com/doc/refman/8.0/en/innodb-redo-log.html\n\nhttp://nippondanji.blogspot.com/2009/01/innodb.html より\n```\nLog sequence numberは、ログバッファへの更新が行われたトータルのバイト数、Log flushed up toはWALへの書き込みが行われたバイト数、Last checkpoint atは最後にチェックポイントが行われたバイト数である。innodb_flush_log_at_trx_commit=1ならば、Log sequence numberとLog flushed up toは非常に近い値になる。\n```\n\nLSNの関係とかは https://github.com/kazeburo/cloudforecast/blob/master/lib/CloudForecast/Data/Innodb5extend.pm ここをみると参考になる。\n\n### サイズ\nhttps://dev.mysql.com/doc/refman/8.0/ja/innodb-parameters.html#sysvar_innodb_log_file_size\n\nLog fileは512-byte block単位で書き込む。\n\nこれはOSのブロックサイズです。Linuxシステムプログラミング p66\n\n参照 : https://lansen.hatenadiary.org/entry/20100724/1279973697\n\n\n## spec\n\nhttps://dev.mysql.com/doc/dev/mysql-server/latest/PAGE_INNODB_REDO_LOG.html\n\nhttps://dev.mysql.com/doc/dev/mysql-server/latest/PAGE_INNODB_REDO_LOG_FORMAT.html\n\n* redoログのファイルを開く\n\n### redoログの構成\n\nこのあたりを読むとstartとかもあってわかりやすい\nhttps://github.com/mysql/mysql-server/blob/8.0/storage/innobase/arch/arch0log.cc\n\n#### ヘッダー\nhttps://dev.mysql.com/doc/dev/mysql-server/latest/PAGE_INNODB_REDO_LOG_FORMAT.html#subsect_redo_log_format_header\n\nhttps://github.com/mysql/mysql-server/blob/8.0/storage/innobase/include/log0log.ic#L296-L311\n\n```\n#define LOG_FILE_HDR_SIZE   (4 * OS_FILE_LOG_BLOCK_SIZE)\n```\n#### Log Block\nhttps://dev.mysql.com/doc/dev/mysql-server/latest/PAGE_INNODB_REDO_LOG_FORMAT.html#subsect_redo_log_format_blocks\n\n## MySQLの便利関数\n### mach_read_from_4\nhttps://github.com/mysql/mysql-server/blob/8.0/storage/innobase/include/mach0data.ic#L146\n\n## 気になるところ\nredo logを緊急でflushしなければならないか?\n\nhttps://github.com/mysql/mysql-server/blob/3e90d07c3578e4da39dc1bce73559bbdf655c28c/storage/innobase/log/log0chkp.cc#L797-L816\n\n\n## Rust周り\n### バイナリ読み込みで参考にしたところ\n* https://github.com/image-rs/image/blob/master/src/io/reader.rs\n* https://qiita.com/fujitayy/items/12a80560a356607da637#%E3%83%90%E3%82%A4%E3%83%88%E5%88%97%E3%82%92%E3%83%90%E3%83%83%E3%83%95%E3%82%A1%E3%81%AB%E8%AA%AD%E3%81%BF%E8%BE%BC%E3%81%BF%E3%81%AA%E3%81%8C%E3%82%89%E5%87%A6%E7%90%86%E3%81%97%E3%81%9F%E3%81%84\n\n* [Rust で [u8; n] から数値を復元する](https://o296.com/2020/08/09/rust-integer-from-raw-byte.html)\n\n### その他\n* [Rust エラー処理2020](https://cha-shu00.hatenablog.com/entry/2020/12/08/060000)\n\n\n## 参照\n* https://github.com/azrle/redo-log-reader\n* [トランザクション技術とリカバリとInnoDBパラメータを調べた](https://tanishiking24.hatenablog.com/entry/innodb-durability)\n* https://www.percona.com/blog/2014/03/28/innodb-redo-log-archiving/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspringmt%2Fredo-log-parser","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fspringmt%2Fredo-log-parser","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspringmt%2Fredo-log-parser/lists"}