{"id":18246589,"url":"https://github.com/douban/kenshin","last_synced_at":"2025-04-04T14:31:18.856Z","repository":{"id":70974965,"uuid":"56772692","full_name":"douban/Kenshin","owner":"douban","description":"Kenshin: A time-series database alternative to Graphite Whisper with 40x improvement in IOPS","archived":false,"fork":false,"pushed_at":"2017-08-10T05:01:41.000Z","size":5028,"stargazers_count":205,"open_issues_count":2,"forks_count":26,"subscribers_count":31,"default_branch":"master","last_synced_at":"2024-05-19T03:03:44.599Z","etag":null,"topics":["graphite","metrics","monitoring"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/douban.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2016-04-21T12:41:37.000Z","updated_at":"2024-04-02T09:22:53.000Z","dependencies_parsed_at":"2023-03-08T11:45:14.681Z","dependency_job_id":null,"html_url":"https://github.com/douban/Kenshin","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/douban%2FKenshin","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/douban%2FKenshin/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/douban%2FKenshin/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/douban%2FKenshin/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/douban","download_url":"https://codeload.github.com/douban/Kenshin/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247193981,"owners_count":20899405,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["graphite","metrics","monitoring"],"created_at":"2024-11-05T09:26:51.289Z","updated_at":"2025-04-04T14:31:17.205Z","avatar_url":"https://github.com/douban.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Kenshin\n=============\n\n[![travis-ci status](https://travis-ci.org/douban/Kenshin.svg)](https://travis-ci.org/douban/Kenshin)\n\n\u003e Kenshin ([るろうに剣心](http://zh.wikipedia.org/wiki/%E6%B5%AA%E5%AE%A2%E5%89%91%E5%BF%83))\n\nKenshin project consists of two major components:\n\n- `Kenshin` is a fixed-size time-series database format, similar in design to [Whisper](https://github.com/graphite-project/whisper), it's an alternative to Whisper for [Graphite](https://github.com/graphite-project) storage component. Whisper performs lots of tiny I/O operations on lots of different files, Kenshin is aiming to improve the I/O performance. For more design details please refer to [design docs](https://github.com/douban/Kenshin/wiki/design) (Chinese) and QCon 2016 Presentation [slide](https://github.com/zzl0/zzl0.github.com/raw/master/files/QCon-Kenshin.pdf).\n\n- `Rurouni-cache` is a storage agent that sits in front of kenshin to batch up writes to files to make them more sequential, rurouni-cache is to kenshin as carbon-cache is to whisper.\n\nKenshin is developing and maintaining by Douban Inc. Currently, it is working in production environment, powering all metrics (host, service, DAE app, user defined) in douban.com.\n\n\n### What's the performance of Kenshin?\n\n\nIn our environment, after using Kenshin, the IOPS is decreased by 97.5%, and the query latency is not significantly slower than Whisper.\n\n\u003cimg src=\"/img/kenshin-perf.png\" width=\"400\"/\u003e\n\n\nQuick Start\n------------------\n\nWe recommend using virtualenv when installing dependencies:\n\n    $ git clone https://github.com/douban/Kenshin.git\n    $ cd Kenshin\n    $ virtualenv venv\n    $ source venv/bin/activate\n    $ pip install -r requirements.txt\n\nTests can be run using nosetests:\n\n    $ nosetests -v\n\nSetup configuration\n\n    $ misc/init_setup_demo.sh\n\nSetup Kenshin\n\n    $ python setup.py build_ext --inplace \u0026\u0026 python setup.py install\n\nStart two rurouni-cache instances\n\n    $ python bin/rurouni-cache.py --debug --config=conf/rurouni.conf --instance=0 start\n    $ python bin/rurouni-cache.py --debug --config=conf/rurouni.conf --instance=1 start\n\nThen go to [Graphite-Kenshin](https://github.com/douban/graphite-kenshin) for starting Web instances.\n\nFAQ\n----------\n\n\n### Why don't you just use whisper?\n\nWhisper is great, and initially we did use it. Over time though, we ran into several issues:\n\n1. Whisper using a lot of IO. There are serval reasons:\n    - Using one file per metric.\n    - Realtime downsample feature (different data resolutions based on age) causes a lot of extra IO\n2. Carbon-cache \u0026 Carbon-relay is inefficient and even is cpu-bound. We didn't write our own carbon-relay, but replaced carbon-relay with [carbon-c-relay](https://github.com/grobian/carbon-c-relay).\n\n\n### Why did you totally rewrite whisper? Couldn't you just submit a patch?\n\nThe reason I didn't simply submit a patch for Whisper is that kenshin's design is incompatible with Whisper's design. Whisper using one file per metric. Kenshin on the other hand merge N metrics into one file.\n\n### How to intergrate Kenshin with Graphite-Web?\n\nWe use write a plugin for Graphite-API named [Graphite-Kenshin](https://github.com/douban/graphite-kenshin)\n\nAcknowledgments\n------------------\n\n- Thanks for the [Graphite](https://github.com/graphite-project) project.\n- Thanks to [@grobian](https://github.com/grobian) for the [carbon-c-relay](https://github.com/grobian/carbon-c-relay) project.\n\n\nContributors\n---------------\n\n- [@zzl0](https://github.com/zzl0)\n- [@mckelvin](https://github.com/mckelvin)\n- [@windreamer](https://github.com/windreamer)\n- [@youngsofun](https://github.com/youngsofun)\n- [@camper42](https://github.com/camper42)\n\nLicense\n-------\n\nKenshin is licensed under version 2.0 of the Apache License. See the [LICENSE](/LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdouban%2Fkenshin","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdouban%2Fkenshin","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdouban%2Fkenshin/lists"}