https://github.com/systemxlabs/datafusion-remote-table
A DataFusion table provider for executing SQL queries on remote databases.
https://github.com/systemxlabs/datafusion-remote-table
datafusion mysql oracle postgresql rust sql sqlite
Last synced: 9 months ago
JSON representation
A DataFusion table provider for executing SQL queries on remote databases.
- Host: GitHub
- URL: https://github.com/systemxlabs/datafusion-remote-table
- Owner: systemxlabs
- License: mit
- Created: 2025-03-06T05:54:54.000Z (11 months ago)
- Default Branch: master
- Last Pushed: 2025-04-25T02:20:19.000Z (9 months ago)
- Last Synced: 2025-04-25T03:30:37.290Z (9 months ago)
- Topics: datafusion, mysql, oracle, postgresql, rust, sql, sqlite
- Language: Rust
- Homepage:
- Size: 637 KB
- Stars: 5
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# datafusion-remote-table
[](https://crates.io/crates/datafusion-remote-table)
## Features
1. Execute SQL queries on remote databases and stream results as datafusion table provider
2. Support inferring schema or user specified schema
3. Support pushing down filters and limit to remote databases
4. Execution plan can be serialized for distributed execution
5. Record batches can be transformed before outputting to next plan node
## Usage
```rust
#[tokio::main]
pub async fn main() -> Result<(), Box> {
let options = ConnectionOptions::Postgres(PostgresConnectionOptions::new(
"localhost",
5432,
"user",
"password",
));
let remote_table = RemoteTable::try_new(options, "select * from supported_data_types").await?;
let ctx = SessionContext::new();
ctx.register_table("remote_table", Arc::new(remote_table))?;
ctx.sql("select * from remote_table").await?.show().await?;
Ok(())
}
```
## Supported databases
- [x] Postgres
- [x] Int2 / Int4 / Int8
- [x] Float4 / Float8 / Numeric
- [x] Char / Varchar / Text / Bpchar / Bytea
- [x] Date / Time / Timestamp / Timestamptz / Interval
- [x] Bool / Oid / Name / Json / Jsonb / Geometry(PostGIS)
- [x] Int2[] / Int4[] / Int8[]
- [x] Float4[] / Float8[]
- [x] Char[] / Varchar[] / Bpchar[] / Text[] / Bytea[]
- [x] MySQL
- [x] TinyInt (Unsigned) / Smallint (Unsigned) / MediumInt (Unsigned) / Int (Unsigned) / Bigint (Unsigned)
- [x] Float / Double / Decimal
- [x] Date / DateTime / Time / Timestamp / Year
- [x] Char / Varchar / Binary / Varbinary
- [x] TinyText / Text / MediumText / LongText
- [x] TinyBlob / Blob / MediumBlob / LongBlob
- [x] Json / Geometry
- [x] Oracle
- [x] Number / BinaryFloat / BinaryDouble / Float
- [x] Varchar2 / NVarchar2 / Char / NChar / Long / Clob / NClob
- [x] Raw / Long Raw / Blob
- [x] Date / Timestamp
- [x] Boolean
- [x] SQLite
- [x] Null / Integer / Real / Text / Blob
## Thanks
- [datafusion-table-providers](https://crates.io/crates/datafusion-table-providers)