{"id":48305990,"url":"https://github.com/altissimo-hq/firedantic-extras","last_synced_at":"2026-04-05T00:04:56.724Z","repository":{"id":348643282,"uuid":"1187734299","full_name":"altissimo-hq/firedantic-extras","owner":"altissimo-hq","description":"Add-on utilities for Firedantic: collection sync, FastAPI pagination, BigQuery schema generation","archived":false,"fork":false,"pushed_at":"2026-04-02T04:50:03.000Z","size":114,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-02T17:43:19.406Z","etag":null,"topics":["firedantic","firestore","pydantic","python","python-packages"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/altissimo-hq.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-21T04:50:41.000Z","updated_at":"2026-04-02T04:49:01.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/altissimo-hq/firedantic-extras","commit_stats":null,"previous_names":["altissimo-hq/firedantic-extras"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/altissimo-hq/firedantic-extras","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/altissimo-hq%2Ffiredantic-extras","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/altissimo-hq%2Ffiredantic-extras/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/altissimo-hq%2Ffiredantic-extras/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/altissimo-hq%2Ffiredantic-extras/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/altissimo-hq","download_url":"https://codeload.github.com/altissimo-hq/firedantic-extras/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/altissimo-hq%2Ffiredantic-extras/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31419552,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-04T20:09:54.854Z","status":"ssl_error","status_checked_at":"2026-04-04T20:09:44.350Z","response_time":60,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["firedantic","firestore","pydantic","python","python-packages"],"created_at":"2026-04-05T00:04:56.187Z","updated_at":"2026-04-05T00:04:56.709Z","avatar_url":"https://github.com/altissimo-hq.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Firedantic Extras\n\nAdd-on utilities for [Firedantic](https://github.com/altissimo-hq/firedantic) — the async-native Pydantic + Firestore ODM.\n\n[![License](https://img.shields.io/badge/license-BSD--3--Clause-blue.svg)](LICENSE)\n[![Python](https://img.shields.io/badge/python-3.10%2B-blue.svg)](https://www.python.org)\n\n## Overview\n\nFiredantic Extras is a companion library that provides higher-level utilities\nbuilt on top of Firedantic models. Each module solves a specific, recurring\nproblem that arises when using Firedantic in production:\n\n| Module                   | Purpose                                                               |\n| ------------------------ | --------------------------------------------------------------------- |\n| **`update_collection`**  | Batch-sync a list of models to a Firestore collection                 |\n| **`cursor_pagination`**  | Framework-agnostic cursor-based pagination for Firedantic models      |\n| **`query`**              | `count_model()` aggregation and `build_prefix_filters()` range search |\n| **`fastapi.pagination`** | FastAPI adapter (`PaginationParams`) for `cursor_paginate`            |\n| **`bigquery.schema`**    | Generate BigQuery table schemas from Firedantic model classes         |\n\n---\n\n## `cursor_pagination` — Cursor-Based Pagination\n\nEfficient, stable, bidirectional cursor pagination for any Firedantic model.\nWorks independently of any web framework — use it from Flask, FastAPI,\nbackground workers, or anywhere else.\n\n### Why?\n\n`Model.find()` returns every document in the collection. For large collections\nthis becomes slow and expensive. `cursor_paginate()` fetches only one page at a\ntime using Firestore's `start_after` cursor, so response time stays constant no\nmatter how many documents exist.\n\nKey design decisions:\n\n- **Document ID as cursor** — stable, type-safe, no field serialization needed.\n- **`__name__` tiebreaker** — a final `order_by(\"__name__\")` prevents silent\n  deduplication at page boundaries when the primary sort field has duplicates.\n- **Reversed sort for `prev`** — backward pagination reverses all sort\n  directions and uses `start_after` (instead of `end_before + limit_to_last`),\n  which is fully supported by both the Firestore SDK and the emulator.\n- **Sentinel row** — requests `limit + 1` rows to determine `has_next` /\n  `has_prev` without an extra `COUNT` query.\n\n### Quick Start\n\n```python\nfrom firedantic import Model\nfrom firedantic_extras import cursor_paginate\n\nclass Product(Model):\n    __collection__ = \"products\"\n    name: str\n    price: float\n    category: str\n\n# --- Page 1 (forward, no cursor) ---\npage = cursor_paginate(Product, limit=20, order_by=\"name\")\nfor p in page.items:\n    print(p.name)\n\nprint(page.has_next)    # True  — more items exist\nprint(page.has_prev)    # False — we're on the first page\n\n# --- Page 2 (next) ---\npage2 = cursor_paginate(\n    Product,\n    limit=20,\n    order_by=\"name\",\n    cursor=page.next_cursor,\n    direction=\"next\",\n)\n\n# --- Back to Page 1 (prev) ---\npage1_again = cursor_paginate(\n    Product,\n    limit=20,\n    order_by=\"name\",\n    cursor=page2.prev_cursor,\n    direction=\"prev\",\n)\n\n# --- Last page (no cursor, going backward) ---\nlast_page = cursor_paginate(Product, limit=20, order_by=\"name\", direction=\"prev\")\nprint(last_page.has_next)  # False — nothing after this\nprint(last_page.has_prev)  # True  — earlier pages exist\n```\n\n### Filtering\n\nPass a `filter_` dict using the same format as `Model.find()`:\n\n```python\n# Equality filter\npage = cursor_paginate(\n    Product,\n    limit=20,\n    order_by=\"name\",\n    filter_={\"category\": \"electronics\"},\n)\n\n# Comparison operators\npage = cursor_paginate(\n    Product,\n    limit=20,\n    order_by=[(\"price\", \"ASCENDING\"), (\"name\", \"ASCENDING\")],\n    filter_={\"price\": {\"\u003e=\": 10.0, \"\u003c\": 100.0}},\n)\n```\n\n### Compound Sort\n\nPass a list of `(field, direction)` tuples for multi-field ordering:\n\n```python\nfrom google.cloud.firestore_v1 import ASCENDING, DESCENDING\n\npage = cursor_paginate(\n    Product,\n    limit=20,\n    order_by=[(\"category\", ASCENDING), (\"price\", DESCENDING)],\n)\n```\n\n### Include Total Count\n\n```python\npage = cursor_paginate(Product, limit=20, order_by=\"name\", include_total=True)\nprint(page.total)  # e.g. 4231 — one extra server-side COUNT aggregation\n```\n\n### API Reference\n\n```python\ndef cursor_paginate(\n    model_class: type[BareModel],\n    *,\n    limit: int,\n    order_by: str | list[str | tuple[str, str]] | None = None,\n    cursor: str | None = None,\n    direction: Literal[\"next\", \"prev\"] = \"next\",\n    filter_: FilterDict | None = None,\n    include_total: bool = False,\n) -\u003e CursorPage[BareModel]:\n    ...\n```\n\n| Parameter       | Default  | Description                                                                                     |\n| --------------- | -------- | ----------------------------------------------------------------------------------------------- |\n| `model_class`   | _(req.)_ | The Firedantic model class to query                                                             |\n| `limit`         | _(req.)_ | Number of items per page (≥ 1)                                                                  |\n| `order_by`      | `None`   | Field name, or list of `(field, direction)` tuples. A `__name__` tiebreaker is always appended. |\n| `cursor`        | `None`   | Document ID from a previous page's `next_cursor` or `prev_cursor`                               |\n| `direction`     | `\"next\"` | `\"next\"` to go forward, `\"prev\"` to go backward                                                 |\n| `filter_`       | `None`   | Equality / comparison filters in Firedantic's `find()` format                                   |\n| `include_total` | `False`  | If `True`, runs an extra server-side `COUNT` aggregation and populates `CursorPage.total`       |\n\n```python\n@dataclass\nclass CursorPage(Generic[ModelT]):\n    items: list[ModelT]       # hydrated model instances for this page\n    has_next: bool            # True if a next page exists (going forward)\n    has_prev: bool            # True if a previous page exists (going backward)\n    next_cursor: str | None   # pass as cursor + direction=\"next\" to advance\n    prev_cursor: str | None   # pass as cursor + direction=\"prev\" to go back\n    total: int | None         # total doc count, only set when include_total=True\n```\n\n---\n\n## `query` — Count and Prefix Search\n\n### `count_model` — Server-Side Aggregation\n\nGet the number of documents matching a filter without fetching any data:\n\n```python\nfrom firedantic_extras import count_model\n\n# Count all documents in a collection\ntotal = count_model(Product)\n\n# Count with a filter\nelectronics_count = count_model(Product, filter_={\"category\": \"electronics\"})\n```\n\nUses Firestore's native `COUNT` aggregation — no documents are transferred.\n\n```python\ndef count_model(\n    model_class: type[BareModel],\n    *,\n    filter_: FilterDict | None = None,\n) -\u003e int:\n    ...\n```\n\n### `build_prefix_filters` — Prefix-Range Search\n\nGenerate a pair of Firedantic-compatible filters that implement a prefix search\nusing Firestore's range query pattern:\n\n```python\nfrom firedantic_extras import build_prefix_filters, cursor_paginate\n\n# Find all products whose name starts with \"lap\"\nfilters = build_prefix_filters(\"name\", \"lap\")\n# Returns: {\"name\": {\"\u003e=\": \"lap\", \"\u003c\": \"lap\\uf8ff\"}}\n\npage = cursor_paginate(\n    Product,\n    limit=20,\n    order_by=\"name\",\n    filter_=filters,\n)\n```\n\nThe upper bound uses the Unicode sentinel `\\uf8ff` (the highest character in\nthe Basic Multilingual Plane), so any string that starts with the prefix sorts\nbefore it.\n\n```python\ndef build_prefix_filters(field: str, prefix: str) -\u003e FilterDict:\n    ...\n```\n\n---\n\n## `fastapi.pagination` — FastAPI Adapter\n\n`PaginationParams` is a FastAPI dependency that extracts `cursor`, `direction`,\nand `limit` from query-string parameters, ready to pass straight to\n`cursor_paginate`.\n\n### Quick Start\n\n```python\nfrom fastapi import Depends, FastAPI\nfrom firedantic_extras import cursor_paginate, CursorPage\nfrom firedantic_extras.fastapi.pagination import PaginationParams\n\napp = FastAPI()\n\nclass Product(Model):\n    __collection__ = \"products\"\n    name: str\n    price: float\n    category: str\n\n@app.get(\"/products\")\ndef list_products(\n    pagination: PaginationParams = Depends(),\n    category: str | None = None,\n) -\u003e CursorPage[Product]:\n    filter_ = {\"category\": category} if category else None\n    return cursor_paginate(\n        Product,\n        limit=pagination.limit,\n        order_by=\"name\",\n        cursor=pagination.cursor,\n        direction=pagination.direction,\n        filter_=filter_,\n    )\n```\n\n**Request examples:**\n\n```http\nGET /products?limit=20\nGET /products?limit=20\u0026cursor=\u003cnext_cursor\u003e\u0026direction=next\nGET /products?limit=20\u0026cursor=\u003cprev_cursor\u003e\u0026direction=prev\n```\n\n**Response:**\n\n```json\n{\n  \"items\": [ ... ],\n  \"has_next\": true,\n  \"has_prev\": false,\n  \"next_cursor\": \"abc123\",\n  \"prev_cursor\": null,\n  \"total\": null\n}\n```\n\n### API\n\n```python\nclass PaginationParams:\n    def __init__(\n        self,\n        cursor: str | None = Query(default=None),\n        direction: Literal[\"next\", \"prev\"] = Query(default=\"next\"),\n        limit: int = Query(default=20, ge=1, le=500),\n    ) -\u003e None: ...\n```\n\n## Installation\n\n```bash\n# Core (includes update_collection)\npip install firedantic-extras\n\n# With FastAPI pagination support\npip install firedantic-extras[fastapi]\n\n# With BigQuery schema generation\npip install firedantic-extras[bigquery]\n\n# Everything\npip install firedantic-extras[all]\n```\n\n---\n\n## `update_collection` — Collection Sync\n\nSynchronize a Firestore collection to match a given list of Firedantic models.\nDocuments are added, updated, or deleted as needed, using batched writes that\nrespect Firestore's 500-document batch limit.\n\n### Why?\n\nManually diffing existing documents against a desired state is tedious and\nerror-prone. `CollectionSync` handles the full add/update/delete lifecycle\nin a single class, with support for dry-run mode, field-level diffing, and\nconfigurable sync keys.\n\nThe comparison is done against **raw Firestore data** (not re-hydrated\nmodels), so stale or extra fields stored in Firestore are visible and will\ntrigger updates — ensuring Firestore always converges to the exact shape\ndescribed by the model.\n\n### Quick Start\n\n```python\nfrom firedantic import Model\nfrom firedantic_extras import CollectionSync\n\nclass User(Model):\n    __collection__ = \"users\"\n    name: str\n    email: str\n    active: bool = True\n\ndesired = [\n    User(id=\"u1\", name=\"Alice\", email=\"alice@example.com\"),\n    User(id=\"u2\", name=\"Bob\",   email=\"bob@example.com\"),\n    User(id=\"u3\", name=\"Carol\", email=\"carol@example.com\"),\n]\n\n# Additive sync (default) — adds new docs, updates changed docs,\n# but does NOT delete docs missing from the list.\nresult = CollectionSync.sync(User, desired)\nprint(result.summary())\n# SyncResult(adds=3, updates=0, deletes=0, skips=0)\n\n# Full sync — also deletes docs not in the desired list.\nresult = CollectionSync.sync(User, desired, delete_items=True)\n\n# Dry run with field-level diff output — preview changes without writing.\nresult = CollectionSync.sync(\n    User, desired, delete_items=True, diff=True, dry_run=True,\n)\nprint(result.summary())\n# SyncResult(adds=0, updates=1, deletes=2, skips=0, DRY RUN)\n\n# Inspect field-level diffs for updated documents.\nfor key, doc_diff in result.diffs.items():\n    for change in doc_diff.changes:\n        print(f\"  {change.field}: {change.before!r} → {change.after!r}\")\n```\n\n### API Reference\n\n#### `CollectionSync`\n\n```python\nclass CollectionSync:\n    \"\"\"Reconcile a Firestore collection to match a desired list of models.\"\"\"\n\n    def __init__(\n        self,\n        model: type[BareModel],\n        items: Sequence[BareModel],\n        *,\n        delete_items: bool = False,\n        dry_run: bool = False,\n        diff: bool = False,\n        output_writer: Callable[[str], None] | None = print,\n        sync_key: str | None = None,\n        on_duplicate_keys: OnDuplicateKeys = \"raise\",\n        on_error: OnError = \"raise\",\n        chunk_size: int = 500,\n    ) -\u003e None: ...\n\n    def run(self) -\u003e SyncResult:\n        \"\"\"Execute the sync and return a SyncResult.\"\"\"\n\n    @classmethod\n    def sync(\n        cls,\n        model: type[BareModel],\n        items: Sequence[BareModel],\n        **kwargs,\n    ) -\u003e SyncResult:\n        \"\"\"Convenience class method — construct and run in one call.\"\"\"\n```\n\n\u003e **Note:** `UpdateCollection` is available as a backward-compatible alias\n\u003e for `CollectionSync`.\n\n#### Parameters\n\n\u003c!-- markdownlint-disable MD033 --\u003e\n\n| Parameter           | Default      | Description                                                                                                               |\n| ------------------- | ------------ | ------------------------------------------------------------------------------------------------------------------------- |\n| `model`             | _(required)_ | The Firedantic model class for the target collection                                                                      |\n| `items`             | _(required)_ | Desired state — every model that should exist after sync                                                                  |\n| `delete_items`      | `False`      | If `True`, documents not in `items` are deleted. \u003cbr\u003e**Safe default prevents accidental data loss.**                      |\n| `dry_run`           | `False`      | If `True`, logs planned changes without writing                                                                           |\n| `diff`              | `False`      | If `True`, collects field-level diffs for updates                                                                         |\n| `sync_key`          | `None`       | Field to match incoming items to existing docs. \u003cbr\u003e`None` uses the document ID; set to e.g. `\"email\"` for non-ID matches |\n| `on_duplicate_keys` | `\"raise\"`    | What to do when `sync_key` matches \u003e1 doc: `\"raise\"`, `\"skip\"`, or `\"update_all\"`                                         |\n| `on_error`          | `\"raise\"`    | Per-document error strategy: `\"raise\"`, `\"collect\"`, or `\"skip\"`                                                          |\n| `chunk_size`        | `500`        | Max operations per Firestore batch write (capped at 500)                                                                  |\n| `output_writer`     | `print`      | Callable for progress output; pass `None` to suppress                                                                     |\n\n\u003c!-- markdownlint-enable MD033 --\u003e\n\n#### `SyncResult`\n\n```python\n@dataclass\nclass SyncResult:\n    adds: int = 0\n    updates: int = 0\n    deletes: int = 0\n    skips: int = 0\n    diffs: dict[str, DocumentDiff]   # populated when diff=True\n    errors: list[SyncError]          # populated when on_error != \"raise\"\n    dry_run: bool = False\n\n    @property\n    def has_errors(self) -\u003e bool: ...\n\n    @property\n    def total_changes(self) -\u003e int:\n        \"\"\"adds + updates + deletes (excludes skips).\"\"\"\n\n    def summary(self) -\u003e str:\n        \"\"\"One-liner: 'SyncResult(adds=1, updates=2, ...)'\"\"\"\n```\n\n#### Supporting types\n\n```python\n@dataclass\nclass FieldDiff:\n    \"\"\"A single field-level change.\"\"\"\n    field: str\n    before: Any   # value in Firestore (_MISSING if absent)\n    after: Any    # value in desired model (_MISSING if absent)\n\n@dataclass\nclass DocumentDiff:\n    \"\"\"All field-level changes for one document.\"\"\"\n    doc_id: str\n    sync_key_value: str\n    changes: list[FieldDiff]\n\n@dataclass\nclass SyncError:\n    \"\"\"An error for one document (when on_error != 'raise').\"\"\"\n    sync_key_value: str\n    error: Exception\n```\n\n#### `build_sync_plan` (advanced)\n\nThe pure-function core of `CollectionSync`, exposed for testing and\ninspection without any Firestore I/O:\n\n```python\ndef build_sync_plan(\n    desired: dict[str, BareModel],\n    existing_models: dict[str, BareModel],\n    existing_raw: dict[str, dict[str, Any]],\n    doc_id_field: str,\n    delete_items: bool = False,\n    diff: bool = False,\n) -\u003e _SyncPlan:\n    \"\"\"Compute adds/updates/deletes/skips from pure data — no Firestore calls.\"\"\"\n```\n\n---\n\n## `bigquery.schema` — Schema Generation\n\nAutomatically generates BigQuery table schemas from Firedantic model classes,\nmapping Pydantic field types to their BigQuery equivalents.\n\n### Why?\n\nWhen you maintain Firedantic models as your source of truth for Firestore\ndocuments and also need to export that data to BigQuery, keeping schemas in\nsync manually is fragile. This module derives the BigQuery schema directly from\nyour model definitions — including `REQUIRED`/`NULLABLE` mode based on whether\nPydantic fields are required or optional, and `REPEATED` mode for list fields.\n\n### Quick Start\n\n```python\nfrom firedantic_extras.bigquery import model_to_bq_schema, schema_to_dict\n\nclass Sample(Model):\n    __collection__ = \"samples\"\n    sample_id: str          # required → REQUIRED\n    barcode: str\n    collected_at: datetime\n    results: dict[str, float]   # dict → JSON\n    tags: list[str]             # list[str] → REPEATED STRING\n    notes: str | None = None    # optional → NULLABLE\n\nschema = model_to_bq_schema(Sample)\n# [\n#   SchemaField(\"id\",           \"STRING\",    mode=\"NULLABLE\"),   ← always first\n#   SchemaField(\"sample_id\",    \"STRING\",    mode=\"REQUIRED\"),\n#   SchemaField(\"barcode\",      \"STRING\",    mode=\"REQUIRED\"),\n#   SchemaField(\"collected_at\", \"TIMESTAMP\", mode=\"REQUIRED\"),\n#   SchemaField(\"results\",      \"JSON\",      mode=\"NULLABLE\"),\n#   SchemaField(\"tags\",         \"STRING\",    mode=\"REPEATED\"),\n#   SchemaField(\"notes\",        \"STRING\",    mode=\"NULLABLE\"),\n# ]\n```\n\n### Type Mapping\n\n| Python / Pydantic Type        | BigQuery Type     | Mode                |\n| ----------------------------- | ----------------- | ------------------- |\n| `str`, `Enum`, `Literal[...]` | `STRING`          | `REQUIRED/NULLABLE` |\n| `int`                         | `INTEGER`         | `REQUIRED/NULLABLE` |\n| `float`, `Decimal`            | `FLOAT`/`NUMERIC` | `REQUIRED/NULLABLE` |\n| `bool`                        | `BOOLEAN`         | `REQUIRED/NULLABLE` |\n| `datetime`                    | `TIMESTAMP`       | `REQUIRED/NULLABLE` |\n| `date`                        | `DATE`            | `REQUIRED/NULLABLE` |\n| `time`                        | `TIME`            | `REQUIRED/NULLABLE` |\n| `bytes`                       | `BYTES`           | `REQUIRED/NULLABLE` |\n| `dict` / `dict[str, X]`       | `JSON`            | `NULLABLE`          |\n| `Any` / unknown               | `JSON`            | `NULLABLE`          |\n| Nested `BaseModel`            | `RECORD`          | `REQUIRED/NULLABLE` |\n| `list[scalar]`                | scalar type       | `REPEATED`          |\n| `list[BaseModel]`             | `RECORD`          | `REPEATED`          |\n| `list[dict]` / `list[Any]`    | `JSON`            | `NULLABLE`          |\n\n**Mode rules:**\n\n- Required Pydantic field (`field: T`) → `REQUIRED`\n- `Optional[T]` / `T | None` / field with a default → `NULLABLE`\n- `list[T]` → `REPEATED` (BQ does not support `REQUIRED` for repeated fields)\n- `id` is always `STRING NULLABLE` (first field, regardless of model definition)\n\n### Backward Compatibility — `json_fields`\n\nWhen migrating from hand-written schemas where nested objects were stored as\nJSON, use `json_fields` to keep specific fields as `JSON NULLABLE` regardless\nof what the model says:\n\n```python\n# populations: list[Population] would normally → REPEATED RECORD\n# but our existing BQ table has it as JSON — keep it for now\nschema = model_to_bq_schema(Kit, json_fields={\"populations\", \"acquired_from\"})\n```\n\nThis lets you migrate one table at a time without breaking existing queries.\n\n### Full API\n\n```python\ndef model_to_bq_schema(\n    model_class: type[BaseModel],\n    *,\n    json_fields: set[str] | None = None,\n    exclude_fields: set[str] | None = None,\n    extra_fields: list[SchemaField] | None = None,\n) -\u003e list[SchemaField]:\n    \"\"\"Generate a BigQuery schema from a Firedantic / Pydantic model.\n\n    Args:\n        model_class: The Pydantic model class to introspect.\n        json_fields: Field names to force to JSON NULLABLE (backward-compat).\n        exclude_fields: Field names to omit from the schema entirely.\n        extra_fields: Additional SchemaFields to append at the end\n            (e.g., load-time metadata columns not in the model).\n    \"\"\"\n\n\ndef models_to_bq_schemas(\n    model_classes: list[type[BaseModel]],\n    **kwargs,\n) -\u003e dict[str, list[SchemaField]]:\n    \"\"\"Generate schemas for multiple models, keyed by __collection__ name.\n\n    Args:\n        model_classes: Firedantic model classes (must have __collection__).\n        **kwargs: Forwarded to model_to_bq_schema.\n\n    Returns:\n        Dict mapping __collection__ names to their BigQuery schemas.\n    \"\"\"\n\n\ndef schema_to_dict(schema: list[SchemaField]) -\u003e list[dict]:\n    \"\"\"Serialise a schema to a JSON-serialisable list of dicts.\n\n    Output matches the BigQuery REST API representation and can be stored\n    in a JSON file or round-tripped via Client.schema_from_json().\n    \"\"\"\n\n\ndef compare_schemas(\n    a: list[SchemaField],\n    b: list[SchemaField],\n) -\u003e SchemaDiff:\n    \"\"\"Diff two BigQuery schemas at the top level (field names and BQ types).\n\n    Useful for verifying a model-derived schema against an existing live BQ\n    table schema before cutting over from hand-written definitions.\n\n    Returns a SchemaDiff with:\n      .only_in_a       — fields in a but not b\n      .only_in_b       — fields in b but not a\n      .type_mismatches — [(field, type_in_a, type_in_b), ...]\n      .is_equal        — True if schemas are identical\n    \"\"\"\n```\n\n### Migration Example for `json2bq`\n\n```python\nfrom firedantic_extras.bigquery.schema import model_to_bq_schema, compare_schemas\n\n# Map (dataset, table) to (ModelClass, fields_to_keep_as_json)\nMODEL_MAP = {\n    (\"darwinsark\", \"kits\"):    (Kit,    {\"populations\", \"acquired_from\"}),\n    (\"darwinsark\", \"animals\"): (Animal, {\"consent\", \"breeds\"}),\n    # tables without a model fall back to BQ autodetect (old behaviour)\n}\n\ndef create_schema(dataset_name, table_name):\n    entry = MODEL_MAP.get((dataset_name, table_name))\n    if entry is None:\n        return None\n    model_class, json_fields = entry\n    return model_to_bq_schema(model_class, json_fields=json_fields)\n\n# Verify new schema matches existing table before switching over:\nexisting = client.get_table(\"darwinsark.kits\").schema\ngenerated = create_schema(\"darwinsark\", \"kits\")\ndiff = compare_schemas(existing, generated)\nif not diff.is_equal:\n    print(\"Fields only in live table:\", diff.only_in_a)\n    print(\"Fields only in model:     \", diff.only_in_b)\n    print(\"Type mismatches:          \", diff.type_mismatches)\n```\n\n---\n\n## Development\n\n```bash\n# Clone and install\ngit clone https://github.com/altissimo-hq/firedantic-extras.git\ncd firedantic-extras\npoetry install --with dev --all-extras\n\n# Run unit tests (default — no emulator needed)\npoetry run pytest\n\n# Lint and format\npoetry run ruff check --fix .\npoetry run ruff format .\n\n# Pre-commit hooks (installed automatically)\npoetry run pre-commit run --all-files\n```\n\n### Integration Tests (Firestore Emulator)\n\nIntegration tests exercise the full Firestore round-trip and require the\n[Firebase Emulator Suite](https://firebase.google.com/docs/emulator-suite).\n\n```bash\n# Prerequisites: Firebase CLI (https://firebase.google.com/docs/cli)\nnpm install -g firebase-tools\n\n# Terminal 1 — start the emulator (Firestore on port 8686)\n./scripts/start_emulator.sh\n\n# Terminal 2 — run integration tests only\nFIRESTORE_EMULATOR_HOST=127.0.0.1:8686 poetry run pytest -m integration -v\n\n# Or run everything (unit + integration)\nFIRESTORE_EMULATOR_HOST=127.0.0.1:8686 poetry run pytest -m \"\" -v\n```\n\nThe default `pytest` command excludes integration tests via `addopts` in\n`pyproject.toml`, so `poetry run pytest` (and pre-commit) always runs\nfast, emulator-free unit tests.\n\n## License\n\nBSD 3-Clause. See [LICENSE](LICENSE) for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faltissimo-hq%2Ffiredantic-extras","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faltissimo-hq%2Ffiredantic-extras","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faltissimo-hq%2Ffiredantic-extras/lists"}