{"id":19494523,"url":"https://github.com/sepich/thanos-kit","last_synced_at":"2025-04-25T20:31:40.102Z","repository":{"id":74619310,"uuid":"291474609","full_name":"sepich/thanos-kit","owner":"sepich","description":"Aftermarket Thanos block tools and utilities. Convert Mimir blocks to Thanos.","archived":false,"fork":false,"pushed_at":"2024-07-29T13:23:48.000Z","size":174,"stargazers_count":19,"open_issues_count":3,"forks_count":4,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-04T03:12:25.594Z","etag":null,"topics":["kit","mimir","thanos","tools","tsdb","utils"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sepich.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-08-30T13:22:22.000Z","updated_at":"2025-01-10T06:30:17.000Z","dependencies_parsed_at":"2024-11-10T21:31:27.161Z","dependency_job_id":"5f4e5a9d-24fb-41a3-8118-c83ed5106b6c","html_url":"https://github.com/sepich/thanos-kit","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sepich%2Fthanos-kit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sepich%2Fthanos-kit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sepich%2Fthanos-kit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sepich%2Fthanos-kit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sepich","download_url":"https://codeload.github.com/sepich/thanos-kit/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250890515,"owners_count":21503507,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kit","mimir","thanos","tools","tsdb","utils"],"created_at":"2024-11-10T21:30:02.387Z","updated_at":"2025-04-25T20:31:39.823Z","avatar_url":"https://github.com/sepich.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# thanos-kit\nTooling to work with Thanos blocks in object storage.\n\n- **ls** - List all blocks ULIDs in the bucket, also show ULID as time (same as `thanos tools bucket ls` but with mimir support)\n- **inspect** - Inspect all blocks in the bucket in detailed, table-like way (same as `thanos tools bucket inspect` but with mimir support)\n- **analyze** - Analyze churn, label pair cardinality for specific block. (same as `promtool tsdb analyze` but also show Labels suitable for block split)\n- **dump** - Dump samples from a TSDB to text format (same as `promtool tsdb dump` but to promtext format)\n- **import** - Import samples to TSDB blocks (same as `promtool tsdb create-blocks-from openmetrics` but from promtext format). Read more about [backfill](#backfill) below\n- **unwrap** - Split one TSDB block to multiple based on Label values. Read more [below](#unwrap)\n\nCli arguments are mostly the same as for `thanos`, help is available for each sub-command:\n```\n$ docker run sepa/thanos-kit -h\nusage: thanos-kit [\u003cflags\u003e] \u003ccommand\u003e [\u003cargs\u003e ...]\n\nTooling for Thanos blocks in object storage\n\nFlags:\n  -h, --help            Show context-sensitive help (also try --help-long and --help-man).\n      --version         Show application version.\n      --log.level=info  Log filtering level (info, debug)\n      --objstore.config-file=\u003cfile-path\u003e\n                        Path to YAML file that contains object store configuration. See format details: https://thanos.io/tip/thanos/storage.md/\n      --objstore.config=\u003ccontent\u003e\n                        Alternative to 'objstore.config-file' flag (mutually exclusive). Content of YAML file that contains object store configuration. See format details: https://thanos.io/tip/thanos/storage.md/\n\nCommands:\n  help [\u003ccommand\u003e...]\n    Show help.\n\n  ls [\u003cflags\u003e]\n    List all blocks in the bucket.\n\n  inspect [\u003cflags\u003e]\n    Inspect all blocks in the bucket in detailed, table-like way\n\n  analyze [\u003cflags\u003e] \u003cULID\u003e\n    Analyze churn, label pair cardinality and find labels to split on\n\n  dump [\u003cflags\u003e] \u003cULID\u003e...\n    Dump samples from a TSDB to text\n\n  import --input-file=INPUT-FILE --label=\u003cname\u003e=\"\u003cvalue\u003e\" [\u003cflags\u003e]\n    Import samples from text to TSDB blocks\n\n  unwrap [\u003cflags\u003e]\n    Split TSDB block to multiple blocks by Label\n```\n\n### Get it\nDocker images are available on [Docker Hub](https://hub.docker.com/repository/docker/sepa/thanos-kit/tags)\n\n### Backfill\n([Original PR](https://github.com/prometheus/prometheus/pull/7586))  \nSupported input format is Prometheus text format.\nYou are free to export/convert your existing data to this format, into one **time-sorted** text file.\n\n`metric{[labels]} value timestamp_ms`\n\nFor example:\n```ini\nk8s_ns_hourly_cost{namespace=\"kube-system\"} 5.7 1599771600000\nk8s_ns_hourly_cost{namespace=\"kube-system\"} 5.0 1599771630000\n...\n```\n\nNote, `value` can be mixed as normal or scientific number as per your preference.\n\nThis format is simple to produce, but not optimized or compressed, so it's normal if your data file is huge.  \nExample of a 19G OpenMetrics file, with ~20k timeseries and 200M data points (samples) on 2y period. Globally resolution is very low in this example.\nImport will take around 2h and uncompacted new TSDB blocks will be around 2.1G for 7600 blocks. When thanos-compact scan them, it starts automatically compacting them in the background. Once compaction is completed (~30min), TSDB blocks will be around 970M for 80 blocks.  \nThe size, and number of blocks depends on timeseries numbers and metrics resolution, but it gives you an order of sizes.\n\nApart from labels set for each metric in text file, you would also need to set Thanos Metadata Labels for the whole batch of blocks you are importing (consider this as prometheus `external_labels` which scraped the metrics from the text file)\n\nExample of command for importing data from `data.prom` (above) to GCS bucket `bucketname`:\n```bash\ndocker run -it --rm \\ \n    -v `pwd`:/work -w /work \\\n    -e GOOGLE_APPLICATION_CREDENTIALS=/work/svc.json \\ \n    sepa/thanos-kit import \\\n        --objstore.config='{type: GCS, config: {bucket: bucketname}}' \\\n        --input-file data.prom \\\n        --label=replica=\\\"prom-a\\\" \\ \n        --label=location=\\\"us-east1\\\"\n```\nPlease note that compactor has default `--consistency-delay=30m` which is based on file upload time (not ULID), so it could take some time before compactor would start processing these blocks.\n\n### Cache dir\nBy default, `thanos-kit` will cache blocks from object storage to `./data` directory, and the dir is not cleaned up on exit. This is to speed up subsequent runs, and to avoid deleting user data when `--data-dir=/tmp` is used for example.\n\nImportant note that `dump` command downloads specified blocks to cache dir, but then dump TSDB as a whole (including blocks already present there)\n\n### Unwrap\n\nThis could be useful for incorporating Mimir to Thanos world by replacing thanos-receive component. Currently Mimir could accept remote-write, and do instant queries via [sidecar](https://grafana.com/docs/mimir/latest/set-up/migrate/migrate-from-thanos-to-mimir-with-thanos-sidecar/) scheme or via [thanos-promql-connector](https://github.com/thanos-community/thanos-promql-connector). But long-term queries via thanos-store would not work with Mimir blocks, as they have no Thanos metadata set. \n\nConsider this example, we have 2 prometheuses in different locations configured like so:\n```yml\n# first\nglobal:\n  external_labels:\n    prometheus: A\n    location: dc1\n\n# second\nglobal:\n  external_labels:\n    prometheus: B\n    location: dc2\n```\n\nAnd they do remote-write to Mimir, let's take a look at produced block:\n```bash\n# thanos-kit analyze\nBlock ID: 01GXGKXC3PA1DE6QNAH2BM2P0R\nThanos Labels:\nLabel names appearing in all Series: [instance, job, prometheus, location]\n```\n\nThe block has no Thanos labels set, and each metric inside has labels `[prometheus, location]` coming from external_labels. We can split this block to 2 separate blocks for each original prometheus like this:\n```bash\n# thanos-kit unwrap --relabel-config='[{target_label: __meta_ext_labels, replacement: prometheus}]'\nuploaded block ulid=001\nuploaded block ulid=002\n\n# thanos-kit analyze 001\nBlock ID: 001\nThanos Labels: prometheus=A\nLabel names appearing in all Series: [instance, job, location]\n\n# thanos-kit analyze 002\nBlock ID: 002\nThanos Labels: prometheus=B\nLabel names appearing in all Series: [instance, job, location]\n```\nUnwrap command takes usual [relabel_config](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config) and can modify block data. But additionally special label `__meta_ext_labels` is parsed if assigned, and all label names in it are extracted to unique blocks. You can specify multiple values separated by `;`, in example above to split original Mimir block by both prometheus+location sets:\n```yaml\n  - |\n    --relabel-config=\n    - target_label: __meta_ext_labels\n      replacement: prometheus;location\n```\nAnd resulting blocks are the same, as would be shipped by `thanos-sidecar`. So they would be compacted without duplicates and wasted space on S3 by compactor afterward. \n\nThis could also be used for blocks produced by thanos-receive too, existing Thanos labels would be merged with extracted ones. Smaller blocks ease thanos-compactor/store sharding, and helps to have different retentions.\n\n### Alternatives\n- [thanos tools bucket](https://thanos.io/tip/components/tools.md/#bucket)\n- [promtool tsdb](https://prometheus.io/docs/prometheus/latest/command-line/promtool/#promtool-tsdb)\n- [mimirtool](https://grafana.com/docs/mimir/latest/manage/tools/mimirtool/#backfill)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsepich%2Fthanos-kit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsepich%2Fthanos-kit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsepich%2Fthanos-kit/lists"}