https://github.com/unitycatalog/unitycatalog-rs
Open, Multi-modal Catalog for Data & AI, written in Rust
https://github.com/unitycatalog/unitycatalog-rs
rust unity-catalog unitycatalog
Last synced: 9 months ago
JSON representation
Open, Multi-modal Catalog for Data & AI, written in Rust
- Host: GitHub
- URL: https://github.com/unitycatalog/unitycatalog-rs
- Owner: unitycatalog
- License: apache-2.0
- Created: 2024-06-17T14:55:36.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-09-30T11:11:41.000Z (over 1 year ago)
- Last Synced: 2025-05-07T03:44:28.810Z (12 months ago)
- Topics: rust, unity-catalog, unitycatalog
- Language: Rust
- Homepage: https://unitycatalog.io/
- Size: 35.2 KB
- Stars: 79
- Watchers: 8
- Forks: 14
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
# unitycatalog-rs
[](https://github.com/unitycatalog/unitycatalog-rs/actions)   
**Unity Catalog Rust (UC-R)**
A native Rust implementation of the [Unity Catalog](https://github.com/unitycatalog/unitycatalog): an Open and Multimodal Catalog for Data & AI.
## Prerequisites
- Install Rust, e.g. as described [here](https://doc.rust-lang.org/cargo/getting-started/installation.html)
## [WIP] Quick Start
Use two terminal windows: one to run the Unity Catalog Rust server (UC-R) another to explore the content of the UC-R server using varios client tools.
### Run the server
In the first terminal window, start the UC server (from the repository root directory)
```
cargo run
```
### [TBD] Query with DuckDB
- [Install](https://duckdb.org/docs/installation/) DuckDB
- Start DuckDB (run `duckdb` in the terminal)
- Run the following commands in the DuckDB shell to install the required extensions
```
install uc_catalog from core_nightly;
load uc_catalog;
install delta;
load delta;
```
- Attach the unity catalog to DuckDB
```
ATTACH 'unity' AS unity (TYPE UC_CATALOG);
```
- Run the queries, for example
```
SHOW ALL TABLES;
SELECT * from unity.default.numbers;
```
## APIs and Compatibility
- Open API specification: The Unity Catalog Rest API is documented [here](https://github.com/unitycatalog/unitycatalog/tree/main/api).
- Compatibility and stability: The APIs are currently evolving and should not be assumed to be stable.
## Compiling and Testing
```
cargo test
```
## Why Rust?
Rust was chosen as a language for the alternative implementation for a few reasons:
- **Performance**: Rust compiles to native bytecode, enabling it to be used in performance-critical and constrained environments.
- **FFI**: As Rust uses the C ABI, bindings can be exposed to any language, like C, Go, etc.
- **Python & Rust**: Python and Rust are a strong pairing, especially in the AI ecosystem ([tokenizers](https://github.com/huggingface/tokenizers), [pydantic](https://github.com/pydantic/pydantic-core), etc).
- **Ecosystem**: Rust has a rich ecosystem of libraries, especially related to big data processing ([delta-rs](https://github.com/delta-io/delta-rs), [datafusion](https://github.com/apache/datafusion), etc).
- **Safety**: Rust's strong type system and ownership model make it difficult to write code that is unsafe or has undefined behavior. When dealing with data assets, especially when it comes to access control, this helps us ensure that we are not introducing security vulnerabilities.
- **We want to write it!**: Most importantly, we like writing Rust!
## Getting Involved
We encourage you to reach out and share your feedback or ideas by [raising an issue](issues/new).
See [CONTRIBUTING.md](CONTRIBUTING.md) if you are looking to contribute.