{"id":17570819,"url":"https://github.com/Quiq/influxdb-tools","last_synced_at":"2025-03-07T22:30:24.873Z","repository":{"id":91008082,"uuid":"103182306","full_name":"Quiq/influxdb-tools","owner":"Quiq","description":"InfluxDB Tools","archived":false,"fork":false,"pushed_at":"2023-10-18T13:07:35.000Z","size":38,"stargazers_count":51,"open_issues_count":1,"forks_count":25,"subscribers_count":8,"default_branch":"master","last_synced_at":"2024-10-21T22:21:57.910Z","etag":null,"topics":["backup","clickhouse","influxdb"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Quiq.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2017-09-11T20:04:33.000Z","updated_at":"2024-07-31T10:43:55.000Z","dependencies_parsed_at":null,"dependency_job_id":"b4b34966-151d-45b5-b90a-60275db36391","html_url":"https://github.com/Quiq/influxdb-tools","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Quiq%2Finfluxdb-tools","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Quiq%2Finfluxdb-tools/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Quiq%2Finfluxdb-tools/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Quiq%2Finfluxdb-tools/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Quiq","download_url":"https://codeload.github.com/Quiq/influxdb-tools/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242472668,"owners_count":20134006,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backup","clickhouse","influxdb"],"created_at":"2024-10-21T18:01:18.030Z","updated_at":"2025-03-07T22:30:24.580Z","avatar_url":"https://github.com/Quiq.png","language":"Python","readme":"# InfluxDB Tools\n\n## Scripts to migrate off influxdb to a better database :)\n\n* schema-influx-to-clickhouse.py - Generate table schemas for Clickhouse based on influxdb measurements.\n\n* line-protocol-to-clickhouse.py - Load influxdb line-protocol backup data into Clickhouse.\n\n## influx-backup.py\n\nInfluxDB backup/restore script using HTTP API and line-protocol format.\n\n* Use InfluxDB HTTP API\n* Backup raw data into text files in line-protocol format\n* Restore from a backup\n* Chunked read/write\n* Separate file for each measurement\n* Backup/restore individual measurements\n* Backup/restore specific retention\n* Incremental backups using \"since\", \"until\" date/time arguments\n* Delayed restore\n* Gzip support for backup/restore process\n\nIt is recommended to do a delayed restore using `--restore-chunk-delay`, `--restore-measurement-delay`\nso your InfluxDB instance does not run out of memory or IO pretty fast.\n\n### Usage\n```\nusage: influx-backup.py [-h] --url URL --user USER --dir DIR\n                        [--measurements MEASUREMENTS]\n                        [--from-measurement FROM_MEASUREMENT]\n                        [--retention RETENTION] [--gzip] [--dump]\n                        [--dump-db DUMP_DB] [--dump-since DUMP_SINCE]\n                        [--dump-until DUMP_UNTIL] [--restore]\n                        [--restore-db RESTORE_DB]\n                        [--restore-chunk-delay RESTORE_CHUNK_DELAY]\n                        [--restore-measurement-delay RESTORE_MEASUREMENT_DELAY]\n\nInfluxDB backup script\n\noptional arguments:\n  -h, --help            show this help message and exit\n  --url URL             InfluxDB URL including schema and port\n  --user USER           InfluxDB username. Password must be set as env var\n                        INFLUX_PW, otherwise will be asked.\n  --dir DIR             directory name for backup or restore form\n  --measurements MEASUREMENTS\n                        comma-separated list of measurements to dump/restore\n  --from-measurement FROM_MEASUREMENT\n                        dump/restore from this measurement and on (ignored\n                        when using --measurements)\n  --retention RETENTION\n                        retention to dump/restore\n  --gzip                dump/restore into/from gzipped files automatically\n  --dump                create a backup\n  --dump-db DUMP_DB     database to dump\n  --dump-since DUMP_SINCE\n                        start date in the format YYYY-MM-DD (starting\n                        00:00:00) or YYYY-MM-DDTHH:MM:SSZ\n  --dump-until DUMP_UNTIL\n                        end date in the format YYYY-MM-DD (exclusive)\n                        or YYYY-MM-DDTHH:MM:SSZ\n  --restore             restore from a backup\n  --restore-db RESTORE_DB\n                        database target of restore\n  --restore-chunk-delay RESTORE_CHUNK_DELAY\n                        restore delay in sec or subsec between chunks of 5000\n                        points\n  --restore-measurement-delay RESTORE_MEASUREMENT_DELAY\n                        restore delay in sec or subsec between measurements\n```\n\n### Examples\n\nDump `stats` db:\n```\n./influx-backup.py --url https://influxdb.localhost:8086 --user admin --dump --dump-db stats --dir stats\n```\nDump `heartbeat` measurement from `stats` db with data until 2017-09-01:\n```\n./influx-backup.py --url https://influxdb.localhost:8086 --user admin --dump --dump-db stats --dir stats \\\n    --dump-until 2017-09-01 --measurements heartbeat\n```\nNOTE: If you get `ChunkedEncodingError` on dump, try to limit the data set using \"since\", \"until\" arguments.\n\nRestore from `stats` dir into `stats_new` db:\n```\n./influx-backup.py --url https://influxdb.localhost:8086 --user admin --restore --restore-db stats_new \\\n    --dir stats\n```\nRestore only `heartbeat` measurement from `stats` dir into `stats_new` db:\n```\n./influx-backup.py --url https://influxdb.localhost:8086 --user admin --restore --restore-db stats_new \\\n    --dir stats --measurements heartbeat\n```\n","funding_links":[],"categories":["Integrations"],"sub_categories":["ETL and Data Processing"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FQuiq%2Finfluxdb-tools","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FQuiq%2Finfluxdb-tools","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FQuiq%2Finfluxdb-tools/lists"}