{"id":21655232,"url":"https://github.com/pseitz/quickwit_perf","last_synced_at":"2026-05-10T17:05:29.948Z","repository":{"id":66217917,"uuid":"500696323","full_name":"PSeitz/quickwit_perf","owner":"PSeitz","description":"Build an quickwit index and track some statistics about it","archived":false,"fork":false,"pushed_at":"2022-06-08T11:15:55.000Z","size":16,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-25T06:25:34.685Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PSeitz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-06-07T05:04:34.000Z","updated_at":"2022-06-08T10:29:12.000Z","dependencies_parsed_at":null,"dependency_job_id":"ea5f7441-fead-4aa0-a0ce-8888de578bfc","html_url":"https://github.com/PSeitz/quickwit_perf","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PSeitz%2Fquickwit_perf","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PSeitz%2Fquickwit_perf/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PSeitz%2Fquickwit_perf/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PSeitz%2Fquickwit_perf/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PSeitz","download_url":"https://codeload.github.com/PSeitz/quickwit_perf/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244554122,"owners_count":20471173,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-25T08:30:54.231Z","updated_at":"2026-05-10T17:05:29.906Z","avatar_url":"https://github.com/PSeitz.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Quickwit Build Index\n\nTool to build quickwit indices, record some stats and put them into a file database `db.json`.\n\n### How to use\n\n\n1. Configure `build_index.toml`, which contains path to the data, path to the config, and _optionally_ an name:\n\nThe name or as fallback the index_id is used as \"index_id\" in the result stored in db.json.\n\n```toml\n[[indices]]\ndata_path = 'hdfs-log.json'\nindex_config = 'hdfs_index_config.yaml'\nname = 'hdfs'\n```\n\n\n2. \nThe tool will clone and compile quickwit. This can be skipped.\n\nExamples:\n```\ncargo run --release -- --machine-name c5n.2xlarge\n\ncargo run --release -- --machine-name c5n.2xlarge --quickwit-commit-hash 5e200a3\n\ncargo run --release -- --skip-quickwit-install --machine-name c5n.2xlarge\n```\n\n3. The compiled quickwit binary will be used to create the indices, gather some data. The results are appended to `db.json`.\n\n\n### Results\nOutput of the runs will be stored in `db.json`, which is a list of json serialized `BuildIndexResult`.\n\n\n### CLI Help\n\n\n```\n➜  track_index_size git:(master) cargo run --release -- --help\n   Compiling track_index_size v0.1.0 (/home/pascal/LinuxData/Development/track_index_size)\n    Finished release [optimized] target(s) in 1.67s\n     Running `target/release/track_index_size --help`\nUsage: track_index_size [--skip-quickwit-install \u003cskip-quickwit-install\u003e] [--build-indices-config-path \u003cbuild-indices-config-path\u003e] [--quickwit-commit-hash \u003cquickwit-commit-hash\u003e] --machine-name \u003cmachine-name\u003e\n\nOptions to configure a run.\n\nOptions:\n  --skip-quickwit-install\n                    skip quickwit installation\n  --build-indices-config-path\n                    the path to the config to build indices. See\n                    `BuildIndicesConfig` for parameters. Defaults to\n                    `build_index.toml`.\n  --quickwit-commit-hash\n                    optional quickwit_commit_hash to checkout after cloning.\n  --machine-name    the machine name. To differentiate different runners\n                    committing into the same db.json.\n  --help            display usage information\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpseitz%2Fquickwit_perf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpseitz%2Fquickwit_perf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpseitz%2Fquickwit_perf/lists"}