https://github.com/fbruzzesi/anyschema
From pydantic to any frame schema
https://github.com/fbruzzesi/anyschema
Last synced: 6 months ago
JSON representation
From pydantic to any frame schema
- Host: GitHub
- URL: https://github.com/fbruzzesi/anyschema
- Owner: FBruzzesi
- License: apache-2.0
- Created: 2025-01-09T13:01:32.000Z (9 months ago)
- Default Branch: main
- Last Pushed: 2025-02-01T21:26:06.000Z (8 months ago)
- Last Synced: 2025-04-15T02:58:06.735Z (6 months ago)
- Language: Python
- Size: 63.5 KB
- Stars: 11
- Watchers: 1
- Forks: 0
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
# anyschema: From pydantic to any frame schema
> [!CAUTION]
> `anyschema` is still in early development and not in pypi yet.
> If you are keen to try it out, it is possible to install it via pip anyway:
> `python -m pip install git+https://github.com/FBruzzesi/anyschema.git`
`anyschema` allows you to convert from a pydantic model to _any_ dataframe schema (by _"any"_ we intend those supported by Narwhals).
Let's see how it works in practice with an example:
```python
from anyschema import AnySchema
from pydantic import BaseModel
from pydantic import PositiveInt
class Student(BaseModel):
name: str
age: PositiveInt
classes: list[str]
schema = AnySchema(model=Student)
# Convert to pyarrow schema
pa_schema = schema.to_arrow()
type(pa_schema)
# pyarrow.lib.Schema
pa_schema
# name: string
# age: uint64
# classes: list
# child 0, item: string
pl_schema = schema.to_polars()
type(pl_schema)
# polars.schema.Schema
pl_schema
# Schema([('name', String), ('age', UInt64), ('classes', List(String))])
```
## Why does this exist?
The initial motivation to start this project has originated by listening to a
[Talk Python podcast episode](https://www.youtube.com/live/wuGirNCyTxA?t=2880s) in which the creator of
[LanceDB](https://github.com/lancedb/lancedb) was interviewed.
He mentioned that they need to convert from pydantic models to pyarrow schemas.
I thought that this could (easily?) be generalized to many other dataframe schema by translating to Narwhals first.
So the challenge was on!