https://github.com/getdozer/dozer-python
https://github.com/getdozer/dozer-python
Last synced: about 1 year ago
JSON representation
- Host: GitHub
- URL: https://github.com/getdozer/dozer-python
- Owner: getdozer
- License: apache-2.0
- Created: 2023-02-24T08:53:12.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2023-11-03T09:14:59.000Z (over 2 years ago)
- Last Synced: 2025-03-26T05:41:56.445Z (over 1 year ago)
- Language: Python
- Size: 2.74 MB
- Stars: 6
- Watchers: 7
- Forks: 1
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
Connect any data source, combine them in real-time and instantly get low-latency gRPC and REST APIs.
⚡ All with just a simple configuration! ⚡️
## Overview
This repository is a python wrapper over gRPC APIs that are automatically when you run [Dozer](https://github.com/getdozer/dozer).
## Installation
```bash
poetry add git+ssh://git@github.com:getdozer/dozer-python.git#main
# or
pip install pydozer
```
## Dependencies
- [Dozer](https://github.com/getdozer/dozer)
- [Poetry](https://python-poetry.org/docs/)
### Querying
Intialize Users Endpoint
```python
api_client = ApiClient("users")
api_client.query()
# Query using $limit, $order_by, $filter
api_client.query({"$limit": 1})
#Count
api_client.count()
```
gRPC methods
```python
api_client.describe()
#Health
api_client.health()
```
### Ingestion
Initialize Ingestion Client
```
from dozer.ingest import IngestClient
ingestion_client = IngestClient()
```
Ingest a data frame
```python
df = pl.read_parquet('tests/trips_small.parquet')
ingestion_client.ingest_df('trips', df, seq_no=1)
```
Use [Arrow Format](https://github.com/apache/arrow) for ingestion
```python
ingestion_client.ingest_df_arrow('trips', df, seq_no=1)
```
Ingest raw records
```python
from pydozer.ingest_pb2 import IngestRequest
from pydozer.types_pb2 import Record, Value
user = IngestRequest(
schema_name="users",
typ=0,
old=None,
new=[Value(int_value=1), Value(string_value="vivek")],
seq_no=1
)
ingestor.ingest(user)
```
Ingest in Arrow format
Check out our [Docs](https://getdozer.io/docs/dozer/) for more information.
### Testing
```
pytest
```
## Contributing
Please refer to [Contributing](https://getdozer.io/docs/contributing/overview) for more details.