{"id":20263623,"url":"https://github.com/springerpe/graphite-du-report","last_synced_at":"2025-07-25T17:41:27.918Z","repository":{"id":80987274,"uuid":"94320546","full_name":"SpringerPE/graphite-du-report","owner":"SpringerPE","description":"graphite utility for quota metrics","archived":false,"fork":false,"pushed_at":"2017-09-27T10:51:24.000Z","size":14178,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-14T04:44:35.316Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SpringerPE.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-06-14T10:48:31.000Z","updated_at":"2017-06-14T10:48:54.000Z","dependencies_parsed_at":null,"dependency_job_id":"e7208607-f4ac-4ea6-b131-0b1b92b459f5","html_url":"https://github.com/SpringerPE/graphite-du-report","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SpringerPE%2Fgraphite-du-report","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SpringerPE%2Fgraphite-du-report/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SpringerPE%2Fgraphite-du-report/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SpringerPE%2Fgraphite-du-report/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SpringerPE","download_url":"https://codeload.github.com/SpringerPE/graphite-du-report/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241731744,"owners_count":20010781,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T11:35:44.626Z","updated_at":"2025-03-03T19:59:46.386Z","avatar_url":"https://github.com/SpringerPE.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Description\nA tool to expose and visualise disk usage for a graphite cluster.\n\n## Sources of inspiration\n\n- https://github.com/Civil/ch-flamegraphs\n- https://redis.io/topics/distlock\n\n## Structure of this repositories\n\nExecutables can be found under the `cmd` folder.\nThe `pkg` folder contains both shared packages, while application code is located under\nthe `pkg/apps` subfolder\n\n## Components\nThe application is split into four main components:\n- the `updater` fetches and process the data coming from a graphite cluster, `/metrics/details` endpoint.\n- the `worker` offers api and data endpoints accessible to web clients and renderers\n- the `renderer` renders the raw disk usage data into a SVG image\n- the `visualiser` visualises the raw disk usage data via an html document\n\n## Api endpoints\n### updater\nThe `updater` currently exposes:\n\n|Method|Endpoint|Parameters|Usage|\n|----------|----------|---------------|-----------------|\n|POST|`/populate`|None|Fetches data from graphite and populates the redis backend|\n|DELETE|`/cleanup`|None|Cleanups old metrics|\n\n### worker\nThe `worker` exposes:\n\n|Method|Endpoint|Parameters|Usage|\n|----------|----------|---------------|------------|\n|GET|`/`|None|information about the supported endpoints|\n|GET|`[/{worker.BasePath}]/size`|`path`: dot separated string|Gets the current usage in bytes for the metrics under `path`|\n|GET|`[/{worker.BasePath}]/folded`|None|Generates a folded format representation of the current disk usage|\n|GET|`[/{worker.BasePath}]/json`|None|Generates a json tree representation of the current disk usage|\n\n### renderer\nThe `renderer` exposes:\n\n|Method|Endpoint|Parameters|Usage|\n|----------|----------|---------------|------------|\n|GET|`[/{renderer.BasePath}]/flame`|None|Returns a renderered flame image|\n\n#### How to contribute your own renderer\n\nAt the moment only one renderer implementation is offered, which produces a `flame` svg\ngraph starting from a folded data representation.\n\nThe `worker` process is in charge of serving the raw binary data to be \nrendered.\n\nAdditional renderers can be developed as simple web apps, implementing a `GET`\nendpoint and returning an html snippet containing the rendered image or object.\n\n### visualiser\nThe `visualiser` exposes:\n\n|Method|Endpoint|Parameters|Usage|\n|----------|----------|---------------|------------|\n|GET|`[/{visualiser.BasePath}]/flame`|None|Returns a html page which contains the rendered disk usage stats|\n\n## Requirements\nThe current implementation makes use of a single Redis instance as a data/caching backend\n\n```\nUpdater  ---------\u003e    Redis  \u003c----------    Worker  \u003c------------ Renderer(s) \u003c---------- Visualiser\n```\n\nThe `worker` , `renderer` and `visualiser` processes need to be served from the same base url\nendpoint. \n\nIn particular the `worker` should be served under the `/` context path,\nwhile the `renderer` should be served under the `/renderer` context path and the `visualiser` from the `/visualise` context path ie.:\n\n- graphite-du.example.com (`worker`)\n- graphite-du.example.com/renderer (`renderer`)\n- graphite-du.example.com/visualiser (`visualiser`)\n\nThere are no limitations regarding the `updater`\n\n## Configuration\nThe configuration of the `updater`, `worker`, `renderer` and `visualiser` processes relies on kingpin. The following `cli` \nparameters and `envs` are defined:\n\n#### updater\n\n|Param|Env|Default|Meaning|\n|------|------|-------|-------|\n|profiling|ENABLE_PPROF|false|enable pprof profiling|\n|servers|GRAPHITE_SERVERS|127.0.0.1:8080|comma separated list of graphite carbonserver endpoint, exposing `/metrics/details` endpoint|\n|bind-address|BIND_ADDRESS|0.0.0.0|binding address for the process|\n|bind-port|PORT|6061|binding port for the process|\n|root-name|ROOT_NAME|root|name for the root of the filesystem tree|\n|redis-addr|REDIS_ADDR|localhost:6379|address and port for the redis datastore\n|redis-passwd|REDIS_PASSWD|password|password to access the redis datastore|\n|num-update-routines|UPDATE_ROUTINES|10|num of concurrent update routines|\n|num-bulk-updates|BULK_UPDATES|100|num of concurrent bulk operations for redis|\n|num-bulk-scans|BULK_SCANS|100|num of bulk scans for redis|\n\nin order to run the updater:\n\n```\n#RUN THE UPDATER\n./updater --servers localhost:8080 --root-name root --redis-addr localhost:6379 --redis-passwd password\n```\n\n#### worker\n\n|Param|Env|Default|Meaning|\n|------|------|-------|-------|\n|profiling|ENABLE_PPROF|false|enable pprof profiling|\n|bind-address|BIND_ADDRESS|0.0.0.0|binding address for the process|\n|bind-port|PORT|6062|binding port for the process|\n|root-name|ROOT_NAME|root|name for the root of the filesystem tree|\n|redis-addr|REDIS_ADDR|localhost:6379|address and port for the redis datastore\n|redis-passwd|REDIS_PASSWD|password|password to access the redis datastore|\n|num-update-routines|UPDATE_ROUTINES|10|num of concurrent update routines|\n|num-bulk-updates|BULK_UPDATES|100|num of concurrent bulk operations for redis|\n|num-bulk-scans|BULK_SCANS|100|num of bulk scans for redis|\n|base-path|BASE_PATH|worker|base context path for this component|\n\nin order to run the worker:\n```\n#RUN THE WORKER\n./worker --root-name root --redis-addr localhost:6379 --redis-passwd password\n```\n\n#### renderer\n\n|Param|Env|Default|Meaning|\n|------|------|-------|-------|\n|profiling|ENABLE_PPROF|false|enable pprof profiling|\n|bind-address|BIND_ADDRESS|0.0.0.0|binding address for the process|\n|bind-port|PORT|6062|binding port for the process|\n|base-path|BASE_PATH|renderer|base context path for this component|\n\nin order to run the renderer:\n```\n#RUN THE RENDERER\n./renderer\n```\n\n#### visualiser\n\n|Param|Env|Default|Meaning|\n|------|------|-------|-------|\n|profiling|ENABLE_PPROF|false|enable pprof profiling|\n|bind-address|BIND_ADDRESS|0.0.0.0|binding address for the process|\n|bind-port|PORT|6063|binding port for the process|\n|base-path|BASE_PATH|visualiser|base context path for this component|\n|renderer-path|RENDERER_PATH|renderer|base context path for the renderer component|\n\nin order to run the visualiser:\n```\n#RUN THE VISUALISER\n./visualiser\n```\n\n## Installation\n`go install github.com/SpringerPE/graphite-du-report/cmd/{worker,updater,renderer,visualiser}`, \nthe dependencies are vendored in a `vendor` folder.\n\nThey have been generated using the [dep](https://github.com/golang/dep) tool.\n\n## Run the tests\nFirst install `ginkgo` and `gomega`:\n\n```\ngo get github.com/onsi/ginkgo/ginkgo\ngo get github.com/onsi/gomega\n```\n\nthen simply run ```ginkgo -r```\n\n## Creating a testing environment\nIn order to run the `graphite-du-report` locally it is needed to provide two main dependencies:\n- a redis installation\n- a carbonserver, implementing the `metrics/details` endpoint. The directory `test` contains a mock carbonserver \nable to generate a well-formed details response. Just run `go run cmd/carbonserver_test/main.go`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspringerpe%2Fgraphite-du-report","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fspringerpe%2Fgraphite-du-report","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspringerpe%2Fgraphite-du-report/lists"}