{"id":19758845,"url":"https://github.com/daq-tools/influxio","last_synced_at":"2026-04-01T21:57:28.457Z","repository":{"id":176496810,"uuid":"615645658","full_name":"daq-tools/influxio","owner":"daq-tools","description":"Import and export data into/from InfluxDB.","archived":false,"fork":false,"pushed_at":"2026-03-30T03:26:15.000Z","size":178,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-03-30T05:55:56.333Z","etag":null,"topics":["ilp","influxdb","influxdb-line-protocol","influxdb-tools","influxdb2","rdbms","sql","sqlalchemy","timeseries","timeseries-data","timeseries-database","timeseries-databases"],"latest_commit_sha":null,"homepage":"https://influxio.readthedocs.io/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/daq-tools.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-03-18T08:58:30.000Z","updated_at":"2026-03-30T03:26:18.000Z","dependencies_parsed_at":"2023-10-01T22:49:19.539Z","dependency_job_id":"53d7dd31-8f69-474b-95b1-08b433b2357f","html_url":"https://github.com/daq-tools/influxio","commit_stats":null,"previous_names":["daq-tools/influxio"],"tags_count":16,"template":false,"template_full_name":null,"purl":"pkg:github/daq-tools/influxio","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daq-tools%2Finfluxio","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daq-tools%2Finfluxio/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daq-tools%2Finfluxio/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daq-tools%2Finfluxio/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/daq-tools","download_url":"https://codeload.github.com/daq-tools/influxio/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daq-tools%2Finfluxio/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31292631,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-01T21:15:39.731Z","status":"ssl_error","status_checked_at":"2026-04-01T21:15:34.046Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ilp","influxdb","influxdb-line-protocol","influxdb-tools","influxdb2","rdbms","sql","sqlalchemy","timeseries","timeseries-data","timeseries-database","timeseries-databases"],"created_at":"2024-11-12T03:26:20.574Z","updated_at":"2026-04-01T21:57:28.451Z","avatar_url":"https://github.com/daq-tools.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# influxio\n\n[![Tests](https://github.com/daq-tools/influxio/actions/workflows/tests.yml/badge.svg)](https://github.com/daq-tools/influxio/actions/workflows/tests.yml)\n[![Test coverage](https://img.shields.io/codecov/c/gh/daq-tools/influxio.svg)](https://codecov.io/gh/daq-tools/influxio/)\n[![License](https://img.shields.io/github/license/daq-tools/influxio.svg)](https://github.com/daq-tools/influxio/blob/main/LICENSE)\n[![Downloads](https://pepy.tech/badge/influxio/month)](https://pypi.org/project/influxio/)\n\n[![Python versions](https://img.shields.io/pypi/pyversions/influxio.svg)](https://pypi.org/project/influxio/)\n[![Status](https://img.shields.io/pypi/status/influxio.svg)](https://pypi.org/project/influxio/)\n[![PyPI version](https://img.shields.io/pypi/v/influxio.svg)](https://pypi.org/project/influxio/)\n\n» [Documentation][project-documentation]\n| [Changelog][project-changelog]\n| [PyPI][project-pypi]\n| [Issues][project-issues]\n| [Source code][project-source]\n| [License][project-license]\n\n## About\n\nYou can use `influxio` to import and export data into/from InfluxDB.\nIt can be used both as a standalone program, and as a library.\n\n`influxio` is, amongst others, based on the excellent [dask], [fsspec],\n[influxdb-client], [influx-line], [line-protocol-parser], [pandas],\n[Polars], and [SQLAlchemy] packages.\n\n## Synopsis\n\n```shell\n# Export from API to database.\ninfluxio copy \\\n    \"http://example:token@localhost:8086/testdrive/demo\" \\\n    \"sqlite://export.sqlite?table=demo\"\n\n# Export from data directory to line protocol format.\ninfluxio copy \\\n    \"file:///path/to/influxdb/engine?bucket-id=372d1908eab801a6\u0026measurement=demo\" \\\n    \"file://export.lp\"\n```\n\n## Quickstart\n\nIf you are in a hurry, and want to run `influxio` without any installation,\njust use the OCI image on Podman or Docker.\n\n```shell\ndocker run --rm --network=host ghcr.io/daq-tools/influxio \\\n    influxio copy \\\n    \"http://example:token@localhost:8086/testdrive/demo\" \\\n    \"crate://crate@localhost:4200/testdrive/demo\"\n```\n\n## Setup\n\nInstall `influxio` from PyPI.\n\n```shell\npip install influxio\n```\n\n## Usage\n\nThis section outlines some example invocations of `influxio`, both on the\ncommand line, and per library use. Other than the resources available from\nthe web, testing data can be acquired from the repository's [testdata] folder.\n\n### Prerequisites\n\nFor properly running some of the example invocations outlined below, you will\nneed an InfluxDB and a CrateDB server. The easiest way to spin up those\ninstances is to use Podman or Docker.\n\nPlease visit the `docs/development.rst` documentation to learn about how to\nspin up corresponding sandbox instances on your workstation.\n\n### Command line use\n\n#### Help\n\n```shell\ninfluxio --help\ninfluxio info\ninfluxio copy --help\n```\n\n#### Import\n\nImport data from different sources into InfluxDB Server.\n\n```shell\n# From test data to API.\n# Choose one of dummy, mixed, dateindex, wide.\ninfluxio copy \\\n    \"testdata://dateindex/\" \\\n    \"http://example:token@localhost:8086/testdrive/demo\"\n\n# With selected amount of rows.\ninfluxio copy \\\n    \"testdata://dateindex/?rows=42\" \\\n    \"http://example:token@localhost:8086/testdrive/demo\"\n\n# With selected amount of rows and columns (only supported by certain test data sources).\ninfluxio copy \\\n    \"testdata://wide/?rows=42\u0026columns=42\" \\\n    \"http://example:token@localhost:8086/testdrive/demo\"\n\n# From line protocol file to InfluxDB API.\ninfluxio copy \\\n    \"file://tests/testdata/basic.lp\" \\\n    \"http://example:token@localhost:8086/testdrive/demo\"\n\n# From line protocol file to InfluxDB API.\ninfluxio copy \\\n    \"https://github.com/influxdata/influxdb2-sample-data/raw/master/air-sensor-data/air-sensor-data.lp\" \\\n    \"http://example:token@localhost:8086/testdrive/demo\"\n```\n\n#### Export from API\n\nExport data from InfluxDB Server into different sinks.\n\n```shell\n# From API to database file.\ninfluxio copy \\\n    \"http://example:token@localhost:8086/testdrive/demo\" \\\n    \"sqlite:///export.sqlite?table=demo\"\n\n# From API to database server.\ninfluxio copy \\\n    \"http://example:token@localhost:8086/testdrive/demo\" \\\n    \"crate://crate@localhost:4200/testdrive/demo\"\n\n# From API to line protocol file.\ninfluxio copy \\\n    \"http://example:token@localhost:8086/testdrive/demo\" \\\n    \"file://export.lp\"\n\n# From API to line protocol on stdout.\ninfluxio copy \\\n    \"http://example:token@localhost:8086/testdrive/demo\" \\\n    \"file://-?format=lp\"\n```\n\n#### Load from File\n\nLoad data from InfluxDB files into any SQL database supported by SQLAlchemy.\n\n```shell\n# From local line protocol file to SQLite.\ninfluxio copy \\\n    \"file://export.lp\" \\\n    \"sqlite:///export.sqlite?table=export\"\n\n# From local line protocol file to CrateDB.\ninfluxio copy \\\n    \"file://export.lp\" \\\n    \"crate://crate@localhost:4200/testdrive/demo\"\n\n# From remote line protocol file to SQLite.\ninfluxio copy \\\n    \"https://github.com/influxdata/influxdb2-sample-data/raw/master/air-sensor-data/air-sensor-data.lp\" \\\n    \"sqlite:///export.sqlite?table=air-sensor-data\"\n\n# From remote line protocol file to CrateDB.\ninfluxio copy \\\n    \"https://github.com/influxdata/influxdb2-sample-data/raw/master/air-sensor-data/air-sensor-data.lp\" \\\n    \"crate://crate@localhost:4200/testdrive/demo\"\n```\n\n#### Export from Cloud to Cloud\n\n```shell\n# From InfluxDB Cloud to CrateDB Cloud.\ninfluxio copy \\\n    \"https://8e9ec869a91a3517:T268DVLDHD8AJsjzOEluu...Pic4A==@eu-central-1-1.aws.cloud2.influxdata.com/testdrive/demo\" \\\n    \"crate://admin:dZ,Y18*Z...7)6LqB@green-shaak-ti.eks1.eu-west-1.aws.cratedb.net:4200/testdrive/demo?ssl=true\"\n\ncrash \\\n    --hosts 'https://admin:dZ,Y18*Z...7)6LqB@green-shaak-ti.eks1.eu-west-1.aws.cratedb.net:4200' \\\n    --command 'SELECT * FROM testdrive.demo;'\n```\n\n#### Export from data directory\n\n```shell\n# From InfluxDB data directory to line protocol file.\ninfluxio copy \\\n    \"file:///path/to/influxdb/engine?bucket-id=372d1908eab801a6\u0026measurement=demo\" \\\n    \"file://export.lp\"\n\n# From InfluxDB data directory to line protocol file, compressed with gzip.\ninfluxio copy \\\n    \"file:///path/to/influxdb/engine?bucket-id=372d1908eab801a6\u0026measurement=demo\" \\\n    \"file://export.lp.gz\"\n\n# From InfluxDB data directory to line protocol on stdout.\ninfluxio copy \\\n    \"file:///path/to/influxdb/engine?bucket-id=372d1908eab801a6\u0026measurement=demo\" \\\n    \"file://-?format=lp\"\n```\n\n#### OCI\n\nOCI images are available on the GitHub Container Registry (GHCR). In order to\nrun them on Podman or Docker, invoke:\n\n```shell\ndocker run --rm --network=host ghcr.io/daq-tools/influxio \\\n    influxio copy \\\n    \"http://example:token@localhost:8086/testdrive/demo\" \\\n    \"stdout://export.lp\"\n```\n\nIf you want to work with files on your filesystem, you will need to either\nmount the working directory into the container using the `--volume` option,\nor use the `--interactive` option to consume STDIN, like:\n\n```shell\ndocker run --rm --volume=$(pwd):/data ghcr.io/daq-tools/influxio \\\n    influxio copy \"file:///data/export.lp\" \"sqlite:///data/export.sqlite?table=export\"\n\ncat export.lp | \\\ndocker run --rm --interactive --network=host ghcr.io/daq-tools/influxio \\\n    influxio copy \"stdin://?format=lp\" \"crate://crate@localhost:4200/testdrive/export\"\n```\n\nIn order to always run the latest `nightly` development version, and to use a\nshortcut for that, this section outlines how to use an alias for `influxio`,\nand a variable for storing the input URL. It may be useful to save a few\nkeystrokes on subsequent invocations.\n\n```shell\ndocker pull ghcr.io/daq-tools/influxio:nightly\nalias influxio=\"docker run --rm --interactive ghcr.io/daq-tools/influxio:nightly influxio\"\nSOURCE=https://github.com/daq-tools/influxio/raw/main/tests/testdata/basic.lp\nTARGET=crate://crate@localhost:4200/testdrive/basic\n\ninfluxio copy \"${SOURCE}\" \"${TARGET}\"\n```\n\n### InfluxDB parameters\n\n#### `timeout`\n\nThe network timeout value is specified in seconds, the default value\nis 60 seconds. Both details deviate from the standard default setting\nof the underlying [InfluxDB client library][influxdb-client], which\nuses milliseconds, and a default value of 10_000 milliseconds.\n\nIf you need to adjust this setting, add the parameter `timeout` to\nthe InfluxDB URL like this:\n\n```shell\ninfluxio copy \\\n    \"http://example:token@localhost:8086/testdrive/demo?timeout=300\" \\\n    \"crate://crate@localhost:4200/testdrive/demo\"\n```\n\n### CrateDB parameters\n\n#### `if-exists`\n\nWhen targeting the SQLAlchemy database interface, the target table will be\ncreated automatically, if it does not exist. The `if-exists` URL query\nparameter can be used to configure this behaviour. The default value is\n`fail`.\n\n- fail: Raise a ValueError.\n- replace: Drop the table before inserting new values.\n- append: Insert new values to the existing table.\n\nExample usage:\n\n```shell\ninfluxio copy \\\n    \"http://example:token@localhost:8086/testdrive/demo\" \\\n    \"crate://crate@localhost:4200/testdrive/demo?if-exists=replace\"\n```\n\n## Project information\n\n### Contribute\n\nContributions of all kinds are much very welcome, in order to make the\nsoftware more solid.\n\nFor installing the project from source, please follow the [development]\ndocumentation.\n\n### Status\n\nBreaking changes should be expected until a 1.0 release, so version pinning\nis recommended, especially when you use it as a library.\n\n### Prior art\n\nThere are a few other projects which are aiming at similar goals.\n\n- [InfluxDB Fetcher]\n- [influxdb-write-to-postgresql] (IW2PG)\n- [Outflux]\n\n### Supported by\n\n[![JetBrains logo.](https://resources.jetbrains.com/storage/products/company/brand/logos/jetbrains.svg)](https://jb.gg/OpenSourceSupport)\n\nSpecial thanks to the people at JetBrains s.r.o. for supporting us with\nexcellent development tooling.\n\n\n[dask]: https://www.dask.org/\n[development]: https://influxio.readthedocs.io/development.html\n[fsspec]: https://pypi.org/project/fsspec/\n[influx]: https://docs.influxdata.com/influxdb/latest/reference/cli/influx/\n[influx-line]: https://github.com/functionoffunction/influx-line\n[influxd]: https://docs.influxdata.com/influxdb/latest/reference/cli/influxd/\n[influxdb fetcher]: https://github.com/hgomez/influxdb\n[influxdb line protocol]: https://docs.influxdata.com/influxdb/latest/reference/syntax/line-protocol/\n[influxdb-client]: https://github.com/influxdata/influxdb-client-python\n[influxdb-write-to-postgresql]: https://github.com/eras/influxdb-write-to-postgresql\n[line-protocol-parser]: https://github.com/Penlect/line-protocol-parser\n[outflux]: https://github.com/timescale/outflux\n[pandas]: https://pandas.pydata.org/\n[polars]: https://pola.rs/\n[project-changelog]: https://github.com/daq-tools/influxio/blob/main/CHANGES.rst\n[project-documentation]: https://influxio.readthedocs.io\n[project-issues]: https://github.com/daq-tools/influxio/issues\n[project-license]: https://github.com/daq-tools/influxio/blob/main/LICENSE\n[project-pypi]: https://pypi.org/project/influxio/\n[project-source]: https://github.com/daq-tools/influxio\n[sqlalchemy]: https://pypi.org/project/SQLAlchemy/\n[testdata]: https://github.com/daq-tools/influxio/tree/main/tests/testdata\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdaq-tools%2Finfluxio","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdaq-tools%2Finfluxio","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdaq-tools%2Finfluxio/lists"}