{"id":13904539,"url":"https://github.com/datacontract/datacontract-cli","last_synced_at":"2026-03-05T12:02:57.666Z","repository":{"id":195574478,"uuid":"670219860","full_name":"datacontract/datacontract-cli","owner":"datacontract","description":"Enforce Data Contracts","archived":false,"fork":false,"pushed_at":"2026-02-13T21:57:11.000Z","size":6083,"stargazers_count":805,"open_issues_count":87,"forks_count":201,"subscribers_count":14,"default_branch":"main","last_synced_at":"2026-02-13T23:27:01.360Z","etag":null,"topics":["datacontract","datamesh"],"latest_commit_sha":null,"homepage":"https://cli.datacontract.com","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/datacontract.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-07-24T14:54:16.000Z","updated_at":"2026-02-13T21:57:13.000Z","dependencies_parsed_at":"2023-09-29T22:12:08.016Z","dependency_job_id":"d912e455-a05f-475b-8cc6-585d91298b5b","html_url":"https://github.com/datacontract/datacontract-cli","commit_stats":null,"previous_names":["datacontract/datacontract.sh","datacontract/cli"],"tags_count":77,"template":false,"template_full_name":null,"purl":"pkg:github/datacontract/datacontract-cli","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datacontract%2Fdatacontract-cli","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datacontract%2Fdatacontract-cli/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datacontract%2Fdatacontract-cli/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datacontract%2Fdatacontract-cli/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/datacontract","download_url":"https://codeload.github.com/datacontract/datacontract-cli/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datacontract%2Fdatacontract-cli/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30123733,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-05T11:11:57.947Z","status":"ssl_error","status_checked_at":"2026-03-05T11:11:29.001Z","response_time":93,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["datacontract","datamesh"],"created_at":"2024-08-06T23:00:57.336Z","updated_at":"2026-03-05T12:02:57.656Z","avatar_url":"https://github.com/datacontract.png","language":"Python","funding_links":[],"categories":["🔐 Data Privacy \u0026 Governance","Data Contracts","Python"],"sub_categories":["📰 Blogs","Commercial / Managed"],"readme":"# Data Contract CLI\n\n\u003cp\u003e\n  \u003ca href=\"https://github.com/datacontract/datacontract-cli/actions/workflows/ci.yaml?query=branch%3Amain\"\u003e\n    \u003cimg alt=\"Test Workflow\" src=\"https://img.shields.io/github/actions/workflow/status/datacontract/datacontract-cli/ci.yaml?branch=main\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/datacontract/datacontract-cli\"\u003e\n    \u003cimg alt=\"Stars\" src=\"https://img.shields.io/github/stars/datacontract/datacontract-cli\" /\u003e\u003c/a\u003e\n  \u003ca href=\"https://datacontract.com/slack\" rel=\"nofollow\"\u003e\u003cimg src=\"https://img.shields.io/badge/slack-join_chat-white.svg?logo=slack\u0026amp;style=social\" alt=\"Slack Status\" data-canonical-src=\"https://img.shields.io/badge/slack-join_chat-white.svg?logo=slack\u0026amp;style=social\" style=\"max-width: 100%;\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\nThe `datacontract` CLI is an open-source command-line tool for working with [data contracts](https://datacontract.com).\nIt natively supports the [Open Data Contract Standard](https://bitol-io.github.io/open-data-contract-standard/latest/) to lint data contracts, connect to data sources and execute schema and quality tests, and export to different formats. \nThe tool is written in Python. \nIt can be used as a standalone CLI tool, in a CI/CD pipeline, or directly as a Python library.\n\n![Main features of the Data Contract CLI](datacontractcli.png)\n\n\n## Getting started\n\nLet's look at this data contract:\n[https://datacontract.com/orders-v1.odcs.yaml](https://datacontract.com/orders-v1.odcs.yaml)\n\nWe have a _servers_ section with endpoint details to a Postgres database, _schema_ for the structure and semantics of the data, _service levels_ and _quality_ attributes that describe the expected freshness and number of rows.\n\nThis data contract contains all information to connect to the database and check that the actual data meets the defined schema specification and quality expectations.\nWe can use this information to test if the actual data product is compliant to the data contract.\n\nLet's use [uv](https://docs.astral.sh/uv/) to install the CLI (or use the [Docker image](#docker)),\n```bash\n$ uv tool install --python python3.11 --upgrade 'datacontract-cli[all]'\n```\n\n\nNow, let's run the tests:\n\n```bash\n$ export DATACONTRACT_POSTGRES_USERNAME=datacontract_cli.egzhawjonpfweuutedfy\n$ export DATACONTRACT_POSTGRES_PASSWORD=jio10JuQfDfl9JCCPdaCCpuZ1YO\n$ datacontract test https://datacontract.com/orders-v1.odcs.yaml\n\n# returns:\nTesting https://datacontract.com/orders-v1.odcs.yaml\nServer: production (type=postgres, host=aws-1-eu-central-2.pooler.supabase.com, port=6543, database=postgres, schema=dp_orders_v1)\n╭────────┬──────────────────────────────────────────────────────────┬─────────────────────────┬─────────╮\n│ Result │ Check                                                    │ Field                   │ Details │\n├────────┼──────────────────────────────────────────────────────────┼─────────────────────────┼─────────┤\n│ passed │ Check that field 'line_item_id' is present               │ line_items.line_item_id │         │\n│ passed │ Check that field line_item_id has type UUID              │ line_items.line_item_id │         │\n│ passed │ Check that field line_item_id has no missing values      │ line_items.line_item_id │         │\n│ passed │ Check that field 'order_id' is present                   │ line_items.order_id     │         │\n│ passed │ Check that field order_id has type UUID                  │ line_items.order_id     │         │\n│ passed │ Check that field 'price' is present                      │ line_items.price        │         │\n│ passed │ Check that field price has type INTEGER                  │ line_items.price        │         │\n│ passed │ Check that field price has no missing values             │ line_items.price        │         │\n│ passed │ Check that field 'sku' is present                        │ line_items.sku          │         │\n│ passed │ Check that field sku has type TEXT                       │ line_items.sku          │         │\n│ passed │ Check that field sku has no missing values               │ line_items.sku          │         │\n│ passed │ Check that field 'customer_id' is present                │ orders.customer_id      │         │\n│ passed │ Check that field customer_id has type TEXT               │ orders.customer_id      │         │\n│ passed │ Check that field customer_id has no missing values       │ orders.customer_id      │         │\n│ passed │ Check that field 'order_id' is present                   │ orders.order_id         │         │\n│ passed │ Check that field order_id has type UUID                  │ orders.order_id         │         │\n│ passed │ Check that field order_id has no missing values          │ orders.order_id         │         │\n│ passed │ Check that unique field order_id has no duplicate values │ orders.order_id         │         │\n│ passed │ Check that field 'order_status' is present               │ orders.order_status     │         │\n│ passed │ Check that field order_status has type TEXT              │ orders.order_status     │         │\n│ passed │ Check that field 'order_timestamp' is present            │ orders.order_timestamp  │         │\n│ passed │ Check that field order_timestamp has type TIMESTAMPTZ    │ orders.order_timestamp  │         │\n│ passed │ Check that field 'order_total' is present                │ orders.order_total      │         │\n│ passed │ Check that field order_total has type INTEGER            │ orders.order_total      │         │\n│ passed │ Check that field order_total has no missing values       │ orders.order_total      │         │\n╰────────┴──────────────────────────────────────────────────────────┴─────────────────────────┴─────────╯\n🟢 data contract is valid. Run 25 checks. Took 3.938887 seconds.\n```\n\nVoilà, the CLI tested that the YAML itself is valid, all records comply with the schema, and all quality attributes are met.\n\nWe can also use the data contract metadata to export in many [formats](#format), e.g., to generate a SQL DDL:\n\n```bash\n$ datacontract export --format sql https://datacontract.com/orders-v1.odcs.yaml\n\n# returns:\n-- Data Contract: orders\n-- SQL Dialect: postgres\nCREATE TABLE orders (\n  order_id None not null primary key,\n  customer_id text not null,\n  order_total integer not null,\n  order_timestamp None,\n  order_status text\n);\nCREATE TABLE line_items (\n  line_item_id None not null primary key,\n  sku text not null,\n  price integer not null,\n  order_id None\n);\n```\n\nOr generate an HTML export:\n\n```bash\n$ datacontract export --format html --output orders-v1.odcs.html https://datacontract.com/orders-v1.odcs.yaml\n```\n\n[//]: # (which will create this [HTML export]\u0026#40;https://datacontract.com/examples/orders-latest/datacontract.html\u0026#41;.)\n\n\n## Usage\n\n```bash\n# create a new data contract from example and write it to odcs.yaml\n$ datacontract init odcs.yaml\n\n# lint the odcs.yaml\n$ datacontract lint odcs.yaml\n\n# execute schema and quality checks (define credentials as environment variables)\n$ datacontract test odcs.yaml\n\n# export data contract as html (other formats: avro, dbt, dbt-sources, dbt-staging-sql, jsonschema, odcs, rdf, sql, sodacl, terraform, ...)\n$ datacontract export --format html datacontract.yaml --output odcs.html\n\n# import sql (other formats: avro, glue, bigquery, jsonschema, excel ...)\n$ datacontract import --format sql --source my-ddl.sql --dialect postgres --output odcs.yaml\n\n# import from Excel template\n$ datacontract import --format excel --source odcs.xlsx --output odcs.yaml\n\n# export to Excel template  \n$ datacontract export --format excel --output odcs.xlsx odcs.yaml\n```\n\n## Programmatic (Python)\n```python\nfrom datacontract.data_contract import DataContract\n\ndata_contract = DataContract(data_contract_file=\"odcs.yaml\")\nrun = data_contract.test()\nif not run.has_passed():\n    print(\"Data quality validation failed.\")\n    # Abort pipeline, alert, or take corrective actions...\n```\n\n## How to\n\n- [How to integrate Data Contract CLI in your CI/CD pipeline as a GitHub Action](https://github.com/datacontract/datacontract-action/)\n- [How to run the Data Contract CLI API to test data contracts with POST requests](https://cli.datacontract.com/API)\n- [How to run Data Contract CLI in a Databricks pipeline](https://www.datamesh-architecture.com/howto/build-a-dataproduct-with-databricks#test-the-data-product)\n\n\n## Installation\n\nChoose the most appropriate installation method for your needs:\n\n### uv\n\nThe preferred way to install is [uv](https://docs.astral.sh/uv/):\n\n```\nuv tool install --python python3.11 --upgrade 'datacontract-cli[all]'\n```\n\n### uvx\n\nIf you have [uv](https://docs.astral.sh/uv/) installed, you can run datacontract-cli directly without installing:\n\n```\nuv run --with 'datacontract-cli[all]' datacontract --version\n```\n\n### pip\nPython 3.10, 3.11, and 3.12 are supported. We recommend using Python 3.11.\n\n```bash\npython3 -m pip install 'datacontract-cli[all]'\ndatacontract --version\n```\n\n### pip with venv\n\nTypically it is better to install the application in a virtual environment for your projects:\n\n```bash\ncd my-project\npython3.11 -m venv venv\nsource venv/bin/activate\npip install 'datacontract-cli[all]'\ndatacontract --version\n```\n\n### pipx\n\npipx installs into an isolated environment.\n\n```bash\npipx install 'datacontract-cli[all]'\ndatacontract --version\n```\n\n### Docker\n\nYou can also use our Docker image to run the CLI tool. It is also convenient for CI/CD pipelines.\n\n```bash\ndocker pull datacontract/cli\ndocker run --rm -v ${PWD}:/home/datacontract datacontract/cli\n```\n\nYou can create an alias for the Docker command to make it easier to use:\n\n```bash\nalias datacontract='docker run --rm -v \"${PWD}:/home/datacontract\" datacontract/cli:latest'\n```\n\n_Note:_ The output of Docker command line messages is limited to 80 columns and may include line breaks. Don't pipe docker output to files if you want to export code. Use the `--output` option instead.\n\n\n\n## Optional Dependencies (Extras)\n\nThe CLI tool defines several optional dependencies (also known as extras) that can be installed for using with specific servers types.\nWith _all_, all server dependencies are included.\n\n```bash\nuv tool install --python python3.11 --upgrade 'datacontract-cli[all]'\n```\n\nA list of available extras:\n\n| Dependency              | Installation Command                       |\n|-------------------------|--------------------------------------------|\n| Amazon Athena           | `pip install datacontract-cli[athena]`     |\n| Avro Support            | `pip install datacontract-cli[avro]`       |\n| Google BigQuery         | `pip install datacontract-cli[bigquery]`   |\n| Databricks Integration  | `pip install datacontract-cli[databricks]` |\n| DuckDB (local/S3/GCS/Azure file testing) | `pip install datacontract-cli[duckdb]` |\n| Iceberg                 | `pip install datacontract-cli[iceberg]`    |\n| Kafka Integration       | `pip install datacontract-cli[kafka]`      |\n| PostgreSQL Integration  | `pip install datacontract-cli[postgres]`   |\n| S3 Integration          | `pip install datacontract-cli[s3]`         |\n| Snowflake Integration   | `pip install datacontract-cli[snowflake]`  |\n| Microsoft SQL Server    | `pip install datacontract-cli[sqlserver]`  |\n| Trino                   | `pip install datacontract-cli[trino]`      |\n| Impala                  | `pip install datacontract-cli[impala]` \t   |\n| dbt                     | `pip install datacontract-cli[dbt]`        |\n| DBML                    | `pip install datacontract-cli[dbml]`       |\n| Parquet                 | `pip install datacontract-cli[parquet]`    |\n| RDF                     | `pip install datacontract-cli[rdf]`        |\n| API (run as web server) | `pip install datacontract-cli[api]`        |\n| protobuf                | `pip install datacontract-cli[protobuf]`   |\n\n\n## Documentation\n\nCommands\n\n- [init](#init)\n- [lint](#lint)\n- [test](#test)\n- [export](#export)\n- [import](#import)\n- [catalog](#catalog)\n- [publish](#publish)\n- [api](#api)\n\n### init\n```\n                                                                                                    \n Usage: datacontract init [OPTIONS] [LOCATION]                                                      \n                                                                                                    \n Create an empty data contract.                                                                     \n                                                                                                    \n                                                                                                    \n╭─ Arguments ──────────────────────────────────────────────────────────────────────────────────────╮\n│   location      [LOCATION]  The location of the data contract file to create.                    │\n│                             [default: datacontract.yaml]                                         │\n╰──────────────────────────────────────────────────────────────────────────────────────────────────╯\n╭─ Options ────────────────────────────────────────────────────────────────────────────────────────╮\n│ --template                       TEXT  URL of a template or data contract [default: None]        │\n│ --overwrite    --no-overwrite          Replace the existing datacontract.yaml                    │\n│                                        [default: no-overwrite]                                   │\n│ --debug        --no-debug              Enable debug logging [default: no-debug]                  │\n│ --help                                 Show this message and exit.                               │\n╰──────────────────────────────────────────────────────────────────────────────────────────────────╯\n\n```\n\n### lint\n```\n                                                                                                    \n Usage: datacontract lint [OPTIONS] [LOCATION]                                                      \n                                                                                                    \n Validate that the datacontract.yaml is correctly formatted.                                        \n                                                                                                    \n                                                                                                    \n╭─ Arguments ──────────────────────────────────────────────────────────────────────────────────────╮\n│   location      [LOCATION]  The location (url or path) of the data contract yaml.                │\n│                             [default: datacontract.yaml]                                         │\n╰──────────────────────────────────────────────────────────────────────────────────────────────────╯\n╭─ Options ────────────────────────────────────────────────────────────────────────────────────────╮\n│ --schema                         TEXT     The location (url or path) of the ODCS JSON Schema     │\n│                                           [default: None]                                        │\n│ --output                         PATH     Specify the file path where the test results should be │\n│                                           written to (e.g.,                                      │\n│                                           './test-results/TEST-datacontract.xml'). If no path is │\n│                                           provided, the output will be printed to stdout.        │\n│                                           [default: None]                                        │\n│ --output-format                  [junit]  The target format for the test results.                │\n│                                           [default: None]                                        │\n│ --debug            --no-debug             Enable debug logging [default: no-debug]               │\n│ --help                                    Show this message and exit.                            │\n╰──────────────────────────────────────────────────────────────────────────────────────────────────╯\n\n```\n\n### test\n```\n                                                                                                    \n Usage: datacontract test [OPTIONS] [LOCATION]                                                      \n                                                                                                    \n Run schema and quality tests on configured servers.                                                \n                                                                                                    \n                                                                                                    \n╭─ Arguments ──────────────────────────────────────────────────────────────────────────────────────╮\n│   location      [LOCATION]  The location (url or path) of the data contract yaml.                │\n│                             [default: datacontract.yaml]                                         │\n╰──────────────────────────────────────────────────────────────────────────────────────────────────╯\n╭─ Options ────────────────────────────────────────────────────────────────────────────────────────╮\n│ --schema                                               TEXT     The location (url or path) of    │\n│                                                                 the ODCS JSON Schema             │\n│                                                                 [default: None]                  │\n│ --server                                               TEXT     The server configuration to run  │\n│                                                                 the schema and quality tests.    │\n│                                                                 Use the key of the server object │\n│                                                                 in the data contract yaml file   │\n│                                                                 to refer to a server, e.g.,      │\n│                                                                 `production`, or `all` for all   │\n│                                                                 servers (default).               │\n│                                                                 [default: all]                   │\n│ --publish-test-results    --no-publish-test-results             Deprecated. Use publish          │\n│                                                                 parameter. Publish the results   │\n│                                                                 after the test                   │\n│                                                                 [default:                        │\n│                                                                 no-publish-test-results]         │\n│ --publish                                              TEXT     The url to publish the results   │\n│                                                                 after the test.                  │\n│                                                                 [default: None]                  │\n│ --output                                               PATH     Specify the file path where the  │\n│                                                                 test results should be written   │\n│                                                                 to (e.g.,                        │\n│                                                                 './test-results/TEST-datacontra… │\n│                                                                 [default: None]                  │\n│ --output-format                                        [junit]  The target format for the test   │\n│                                                                 results.                         │\n│                                                                 [default: None]                  │\n│ --logs                    --no-logs                             Print logs [default: no-logs]    │\n│ --ssl-verification        --no-ssl-verification                 SSL verification when publishing │\n│                                                                 the data contract.               │\n│                                                                 [default: ssl-verification]      │\n│ --debug                   --no-debug                            Enable debug logging             │\n│                                                                 [default: no-debug]              │\n│ --help                                                          Show this message and exit.      │\n╰──────────────────────────────────────────────────────────────────────────────────────────────────╯\n\n```\n\nData Contract CLI connects to a data source and runs schema and quality tests to verify that the data contract is valid.\n\n```bash\n$ datacontract test --server production datacontract.yaml\n```\n\nTo connect to the databases the `server` block in the datacontract.yaml is used to set up the connection.\nIn addition, credentials, such as username and passwords, may be defined with environment variables.\n\nThe application uses different engines, based on the server `type`.\nInternally, it connects with DuckDB, Spark, or a native connection and executes the most tests with _soda-core_ and _fastjsonschema_.\n\nCredentials are provided with environment variables.\n\nSupported server types:\n\n- [s3](#S3)\n- [athena](#athena)\n- [bigquery](#bigquery)\n- [azure](#azure)\n- [sqlserver](#sqlserver)\n- [oracle](#oracle)\n- [databricks](#databricks)\n- [databricks (programmatic)](#databricks-programmatic)\n- [dataframe (programmatic)](#dataframe-programmatic)\n- [snowflake](#snowflake)\n- [kafka](#kafka)\n- [postgres](#postgres)\n- [trino](#trino)\n- [impala](#impala)\n- [api](#api)\n- [local](#local)\n\nSupported formats:\n\n- parquet\n- json\n- csv\n- delta\n- iceberg (coming soon)\n\nFeel free to create an [issue](https://github.com/datacontract/datacontract-cli/issues), if you need support for an additional type and formats.\n\n#### S3\n\nData Contract CLI can test data that is stored in S3 buckets or any S3-compliant endpoints in various formats.\n\n- CSV\n- JSON\n- Delta\n- Parquet\n- Iceberg (coming soon)\n\n##### Examples\n\n###### JSON\n\ndatacontract.yaml\n```yaml\nservers:\n  production:\n    type: s3\n    endpointUrl: https://minio.example.com # not needed with AWS S3\n    location: s3://bucket-name/path/*/*.json\n    format: json\n    delimiter: new_line # new_line, array, or none\n```\n\n###### Delta Tables\n\ndatacontract.yaml\n```yaml\nservers:\n  production:\n    type: s3\n    endpointUrl: https://minio.example.com # not needed with AWS S3\n    location: s3://bucket-name/path/table.delta # path to the Delta table folder containing parquet data files and the _delta_log\n    format: delta\n```\n\n##### Environment Variables\n\n| Environment Variable                | Example                         | Description                            |\n|-------------------------------------|---------------------------------|----------------------------------------|\n| `DATACONTRACT_S3_REGION`            | `eu-central-1`                  | Region of S3 bucket                    |\n| `DATACONTRACT_S3_ACCESS_KEY_ID`     | `AKIAXV5Q5QABCDEFGH`            | AWS Access Key ID                      |\n| `DATACONTRACT_S3_SECRET_ACCESS_KEY` | `93S7LRrJcqLaaaa/XXXXXXXXXXXXX` | AWS Secret Access Key                  |\n| `DATACONTRACT_S3_SESSION_TOKEN`     | `AQoDYXdzEJr...`                | AWS temporary session token (optional) |\n\n\n#### Athena\n\nData Contract CLI can test data in AWS Athena stored in S3.\nSupports different file formats, such as Iceberg, Parquet, JSON, CSV...\n\n##### Example\n\ndatacontract.yaml\n```yaml\nservers:\n  athena:\n    type: athena\n    catalog: awsdatacatalog # awsdatacatalog is the default setting\n    schema: icebergdemodb   # in Athena, this is called \"database\"\n    regionName: eu-central-1\n    stagingDir: s3://my-bucket/athena-results/\nmodels:\n  my_table: # corresponds to a table or view name\n    type: table\n    fields:\n      my_column_1: # corresponds to a column\n        type: string\n        config:\n          physicalType: varchar\n```\n\n##### Environment Variables\n\n| Environment Variable                | Example                         | Description                            |\n|-------------------------------------|---------------------------------|----------------------------------------|\n| `DATACONTRACT_S3_REGION`            | `eu-central-1`                  | Region of Athena service               |\n| `DATACONTRACT_S3_ACCESS_KEY_ID`     | `AKIAXV5Q5QABCDEFGH`            | AWS Access Key ID                      |\n| `DATACONTRACT_S3_SECRET_ACCESS_KEY` | `93S7LRrJcqLaaaa/XXXXXXXXXXXXX` | AWS Secret Access Key                  |\n| `DATACONTRACT_S3_SESSION_TOKEN`     | `AQoDYXdzEJr...`                | AWS temporary session token (optional) |\n\n\n#### Google Cloud Storage (GCS)\n\nThe [S3](#S3) integration also works with files on Google Cloud Storage through its [interoperability](https://cloud.google.com/storage/docs/interoperability).\nUse `https://storage.googleapis.com` as the endpoint URL.\n\n##### Example\n\ndatacontract.yaml\n```yaml\nservers:\n  production:\n    type: s3\n    endpointUrl: https://storage.googleapis.com\n    location: s3://bucket-name/path/*/*.json # use s3:// schema instead of gs://\n    format: json\n    delimiter: new_line # new_line, array, or none\n```\n\n##### Environment Variables\n\n| Environment Variable                | Example        | Description                                                                              |\n|-------------------------------------|----------------|------------------------------------------------------------------------------------------|\n| `DATACONTRACT_S3_ACCESS_KEY_ID`     | `GOOG1EZZZ...` | The GCS [HMAC Key](https://cloud.google.com/storage/docs/authentication/hmackeys) Key ID |\n| `DATACONTRACT_S3_SECRET_ACCESS_KEY` | `PDWWpb...`    | The GCS [HMAC Key](https://cloud.google.com/storage/docs/authentication/hmackeys) Secret |\n\n\n#### BigQuery\n\nWe support authentication to BigQuery using Service Account Key or Application Default Credentials (ADC). ADC supports Workload Identity Federation (WIF), GCE metadata server, and `gcloud auth application-default login`. The used Service Account should include the roles:\n* BigQuery Job User\n* BigQuery Data Viewer\n\nWhen no `DATACONTRACT_BIGQUERY_ACCOUNT_INFO_JSON_PATH` is set, the CLI falls back to ADC/WIF automatically via Soda's `use_context_auth`.\n\n##### Example\n\ndatacontract.yaml\n```yaml\nservers:\n  production:\n    type: bigquery\n    project: datameshexample-product\n    dataset: datacontract_cli_test_dataset\nmodels:\n  datacontract_cli_test_table: # corresponds to a BigQuery table\n    type: table\n    fields: ...\n```\n\n##### Environment Variables\n\n| Environment Variable                         | Example                   | Description                                             |\n|----------------------------------------------|---------------------------|---------------------------------------------------------|\n| `DATACONTRACT_BIGQUERY_ACCOUNT_INFO_JSON_PATH` | `~/service-access-key.json` | Service Account key JSON file. If not set, ADC/WIF is used automatically. |\n| `DATACONTRACT_BIGQUERY_IMPERSONATION_ACCOUNT` | `sa@project.iam.gserviceaccount.com` | Optional. Service account to impersonate. Works with both key file and ADC auth. |\n\n\n#### Azure\n\nData Contract CLI can test data that is stored in Azure Blob storage or Azure Data Lake Storage (Gen2) (ADLS) in various formats.\n\n##### Example\n\ndatacontract.yaml\n```yaml\nservers:\n  production:\n    type: azure\n    location: abfss://datameshdatabricksdemo.dfs.core.windows.net/inventory_events/*.parquet\n    format: parquet\n```\n\n##### Environment Variables\n\nAuthentication works with an Azure Service Principal (SPN) aka App Registration with a secret.\n\n| Environment Variable               | Example                                | Description                                          |\n|------------------------------------|----------------------------------------|------------------------------------------------------|\n| `DATACONTRACT_AZURE_TENANT_ID`     | `79f5b80f-10ff-40b9-9d1f-774b42d605fc` | The Azure Tenant ID                                  |\n| `DATACONTRACT_AZURE_CLIENT_ID`     | `3cf7ce49-e2e9-4cbc-a922-4328d4a58622` | The ApplicationID / ClientID of the app registration |\n| `DATACONTRACT_AZURE_CLIENT_SECRET` | `yZK8Q~GWO1MMXXXXXXXXXXXXX`            | The Client Secret value                              |\n\n\n\n#### Sqlserver\n\nData Contract CLI can test data in MS SQL Server (including Azure SQL, Synapse Analytics SQL Pool).\n\n##### Example\n\ndatacontract.yaml\n```yaml\nservers:\n  production:\n    type: sqlserver\n    host: localhost\n    port: 5432\n    database: tempdb\n    schema: dbo\n    driver: ODBC Driver 18 for SQL Server\nmodels:\n  my_table_1: # corresponds to a table\n    type: table\n    fields:\n      my_column_1: # corresponds to a column\n        type: varchar\n```\n\n##### Environment Variables\n\n| Environment Variable                              | Example| Description                                  |\n|---------------------------------------------------|--------|----------------------------------------------|\n| `DATACONTRACT_SQLSERVER_USERNAME`                 | `root` | Username                                     |\n| `DATACONTRACT_SQLSERVER_PASSWORD`                 | `toor` | Password                                     |\n| `DATACONTRACT_SQLSERVER_TRUSTED_CONNECTION`       | `True` | Use windows authentication, instead of login |\n| `DATACONTRACT_SQLSERVER_TRUST_SERVER_CERTIFICATE` | `True` | Trust self-signed certificate                |\n| `DATACONTRACT_SQLSERVER_ENCRYPTED_CONNECTION`     | `True` | Use SSL                                      |\n| `DATACONTRACT_SQLSERVER_DRIVER`                   | `ODBC Driver 18 for SQL Server` | ODBC driver name   |\n\n\n\n\n#### Oracle\n\nData Contract CLI can test data in Oracle Database.\n\n##### Example\n\ndatacontract.yaml\n```yaml\nservers:\n  oracle:\n    type: oracle\n    host: localhost\n    port: 1521\n    service_name: ORCL\n    schema: ADMIN\nmodels:\n  my_table_1: # corresponds to a table\n    type: table\n    fields:\n      my_column_1: # corresponds to a column\n        type: decimal\n        description: Decimal number\n      my_column_2: # corresponds to another column\n        type: text\n        description: Unicode text string\n        config:\n          oracleType: NVARCHAR2 # optional: can be used to explicitly define the type used in the database\n                                # if not set a default mapping will be used\n```\n\n##### Environment Variables\n\nThese environment variable specify the credentials used by the datacontract tool to connect to the database.\nIf you've started the database from a container, e.g. [oracle-free](https://hub.docker.com/r/gvenzl/oracle-free)\nthis should match either `system` and what you specified as `ORACLE_PASSWORD` on the container or\nalternatively what you've specified under `APP_USER` and `APP_USER_PASSWORD`.\nIf you require thick mode to connect to the database, you need to have an Oracle Instant Client\ninstalled on the system and specify the path to the installation within the environment variable\n`DATACONTRACT_ORACLE_CLIENT_DIR`.\n\n| Environment Variable                             | Example            | Description                                |\n|--------------------------------------------------|--------------------|--------------------------------------------|\n| `DATACONTRACT_ORACLE_USERNAME`                   | `system`           | Username                                   |\n| `DATACONTRACT_ORACLE_PASSWORD`                   | `0x162e53`         | Password                                   |\n| `DATACONTRACT_ORACLE_CLIENT_DIR`                 | `C:\\oracle\\client` | Path to Oracle Instant Client installation |\n\n\n\n\n#### Databricks\n\nWorks with Unity Catalog and Hive metastore.\n\nNeeds a running SQL warehouse or compute cluster.\n\n##### Example\n\ndatacontract.yaml\n```yaml\nservers:\n  production:\n    type: databricks\n    catalog: acme_catalog_prod\n    schema: orders_latest\nmodels:\n  orders: # corresponds to a table\n    type: table\n    fields: ...\n```\n\n##### Environment Variables\n\n| Environment Variable                      | Example                              | Description                                               |\n|-------------------------------------------|--------------------------------------|-----------------------------------------------------------|\n| `DATACONTRACT_DATABRICKS_TOKEN`           | `dapia00000000000000000000000000000` | The personal access token to authenticate                 |\n| `DATACONTRACT_DATABRICKS_HTTP_PATH`       | `/sql/1.0/warehouses/b053a3ffffffff` | The HTTP path to the SQL warehouse or compute cluster     |\n| `DATACONTRACT_DATABRICKS_SERVER_HOSTNAME` | `dbc-abcdefgh-1234.cloud.databricks.com` | The host name of the SQL warehouse or compute cluster |\n\n\n#### Databricks (programmatic)\n\nWorks with Unity Catalog and Hive metastore.\nWhen running in a notebook or pipeline, the provided `spark` session can be used.\nAn additional authentication is not required.\n\nRequires a Databricks Runtime with Python \u003e= 3.10.\n\n##### Example\n\ndatacontract.yaml\n```yaml\nservers:\n  production:\n    type: databricks\n    host: dbc-abcdefgh-1234.cloud.databricks.com # ignored, always use current host\n    catalog: acme_catalog_prod\n    schema: orders_latest\nmodels:\n  orders: # corresponds to a table\n    type: table\n    fields: ...\n```\n\n##### Installing on Databricks Compute\n\n**Important:** When using Databricks LTS ML runtimes (15.4, 16.4), installing via `%pip install` in notebooks can cause issues.\n\n**Recommended approach:** Use Databricks' native library management instead:\n\n1. **Create or configure your compute cluster:**\n   - Navigate to **Compute** in the Databricks workspace\n   - Create a new cluster or select an existing one\n   - Go to the **Libraries** tab\n\n2. **Add the datacontract-cli library:**\n   - Click **Install new**\n   - Select **PyPI** as the library source\n   - Enter package name: `datacontract-cli[databricks]`\n   - Click **Install**\n\n3. **Restart the cluster** to apply the library installation\n\n4. **Use in your notebook** without additional installation:\n   ```python\n   from datacontract.data_contract import DataContract\n\n   data_contract = DataContract(\n     data_contract_file=\"/Volumes/acme_catalog_prod/orders_latest/datacontract/datacontract.yaml\",\n     spark=spark)\n   run = data_contract.test()\n   run.result\n   ```\n\nDatabricks' library management properly resolves dependencies during cluster initialization, rather than at runtime in the notebook.\n\n#### Dataframe (programmatic)\n\nWorks with Spark DataFrames.\nDataFrames need to be created as named temporary views.\nMultiple temporary views are supported if your data contract contains multiple models.\n\nTesting DataFrames is useful to test your datasets in a pipeline before writing them to a data source.\n\n##### Example\n\ndatacontract.yaml\n```yaml\nservers:\n  production:\n    type: dataframe\nmodels:\n  my_table: # corresponds to a temporary view\n    type: table\n    fields: ...\n```\n\nExample code\n```python\nfrom datacontract.data_contract import DataContract\n\ndf.createOrReplaceTempView(\"my_table\")\n\ndata_contract = DataContract(\n  data_contract_file=\"datacontract.yaml\",\n  spark=spark,\n)\nrun = data_contract.test()\nassert run.result == \"passed\"\n```\n\n\n#### Snowflake\n\nData Contract CLI can test data in Snowflake.\n\n##### Example\n\ndatacontract.yaml\n```yaml\n\nservers:\n  snowflake:\n    type: snowflake\n    account: abcdefg-xn12345\n    database: ORDER_DB\n    schema: ORDERS_PII_V2\nmodels:\n  my_table_1: # corresponds to a table\n    type: table\n    fields:\n      my_column_1: # corresponds to a column\n        type: varchar\n```\n\n##### Environment Variables\nAll [parameters supported by Soda](https://docs.soda.io/soda/connect-snowflake.html), uppercased and prepended by `DATACONTRACT_SNOWFLAKE_` prefix.\nFor example:\n\n| Soda parameter       | Environment Variable                        |\n|----------------------|---------------------------------------------|\n| `username`           | `DATACONTRACT_SNOWFLAKE_USERNAME`           |\n| `password`           | `DATACONTRACT_SNOWFLAKE_PASSWORD`           |\n| `warehouse`          | `DATACONTRACT_SNOWFLAKE_WAREHOUSE`          |\n| `role`               | `DATACONTRACT_SNOWFLAKE_ROLE`               |\n| `connection_timeout` | `DATACONTRACT_SNOWFLAKE_CONNECTION_TIMEOUT` |\n\nBeware, that parameters:\n* `account`\n* `database`\n* `schema`\n\nare obtained from the `servers` section of the YAML-file.\nE.g. from the example above:\n```yaml\nservers:\n  snowflake:\n    account: abcdefg-xn12345\n    database: ORDER_DB\n    schema: ORDERS_PII_V2\n```\n\n\n#### Kafka\n\nKafka support is currently considered experimental.\n\n##### Example\n\ndatacontract.yaml\n```yaml\nservers:\n  production:\n    type: kafka\n    host: abc-12345.eu-central-1.aws.confluent.cloud:9092\n    topic: my-topic-name\n    format: json\n```\n\n##### Environment Variables\n\n| Environment Variable                | Example | Description                                                                      |\n|-------------------------------------|---------|----------------------------------------------------------------------------------|\n| `DATACONTRACT_KAFKA_SASL_USERNAME`  | `xxx`   | The SASL username (key).                                                         |\n| `DATACONTRACT_KAFKA_SASL_PASSWORD`  | `xxx`   | The SASL password (secret).                                                      |\n| `DATACONTRACT_KAFKA_SASL_MECHANISM` | `PLAIN` | Default `PLAIN`. Other supported mechanisms: `SCRAM-SHA-256` and `SCRAM-SHA-512` |\n\n\n#### Postgres\n\nData Contract CLI can test data in Postgres or Postgres-compliant databases (e.g., RisingWave).\n\n##### Example\n\ndatacontract.yaml\n```yaml\nservers:\n  postgres:\n    type: postgres\n    host: localhost\n    port: 5432\n    database: postgres\n    schema: public\nmodels:\n  my_table_1: # corresponds to a table\n    type: table\n    fields:\n      my_column_1: # corresponds to a column\n        type: varchar\n```\n\n##### Environment Variables\n\n| Environment Variable             | Example            | Description |\n|----------------------------------|--------------------|-------------|\n| `DATACONTRACT_POSTGRES_USERNAME` | `postgres`         | Username    |\n| `DATACONTRACT_POSTGRES_PASSWORD` | `mysecretpassword` | Password    |\n\n\n#### Trino\n\nData Contract CLI can test data in Trino.\n\n##### Example\n\ndatacontract.yaml\n```yaml\nservers:\n  trino:\n    type: trino\n    host: localhost\n    port: 8080\n    catalog: my_catalog\n    schema: my_schema\nmodels:\n  my_table_1: # corresponds to a table\n    type: table\n    fields:\n      my_column_1: # corresponds to a column\n        type: varchar\n      my_column_2: # corresponds to a column with custom trino type\n        type: object\n        config:\n          trinoType: row(en_us varchar, pt_br varchar)\n```\n\n##### Environment Variables\n\n| Environment Variable          | Example            | Description |\n|-------------------------------|--------------------|-------------|\n| `DATACONTRACT_TRINO_USERNAME` | `trino`            | Username    |\n| `DATACONTRACT_TRINO_PASSWORD` | `mysecretpassword` | Password    |\n\n\n#### Impala\n\nData Contract CLI can run Soda checks against an Apache Impala cluster.\n\n##### Example\n\ndatacontract.yaml\n```yaml\nservers:\n  impala:\n    type: impala\n    host: my-impala-host\n    port: 443\n    # Optional default database used for Soda scans\n    database: my_database\nmodels:\n  my_table_1: # corresponds to a table\n    type: table\n    # fields as usual …\n```\n\n##### Environment Variables\n\n| Environment Variable                      | Example               | Description                                               |\n|-------------------------------            |--------------------   |-------------                                              |\n| `DATACONTRACT_IMPALA_USERNAME`            | `analytics_user`      | Username used to connect to Impala                        |\n| `DATACONTRACT_IMPALA_PASSWORD`            | `mysecretpassword`    | Password for the Impala user                              |\n| `DATACONTRACT_IMPALA_USE_SSL`             | `true`                | Whether to use SSL; defaults to true if unset             |\n| `DATACONTRACT_IMPALA_AUTH_MECHANISM`      | `LDAP`                | Authentication mechanism; defaults to LDAP                |\n| `DATACONTRACT_IMPALA_USE_HTTP_TRANSPORT`  | `true`                | Whether to use the HTTP transport; defaults to true       |\n| `DATACONTRACT_IMPALA_HTTP_PATH`           | `cliservice`          | HTTP path for the Impala service; defaults to cliservice  |\n\n### Type-mapping note (logicalType → Impala type)\n\nIf `physicalType` is not specified in the schema, we recommend the following mapping from `logicalType` to Impala column types:\n\n|logicalType | Recommended Impala type |\n|------------|-------------------------|\n| `integer`  | `INT` or `BIGINT`       |\n| `number`   | `DOUBLE`/`decimal(..)`  |\n| `string`   | `STRING` or `VARCHAR`   |\n| `boolean`  | `BOOLEAN`               |\n| `date`     | `DATE`                  |\n| `datetime` | `TIMESTAMP`             |\n\nThis keeps the Impala schema compatible with the expectations of the Soda checks generated by datacontract-cli.\n\n#### API\n\nData Contract CLI can test APIs that return data in JSON format. \nCurrently, only GET requests are supported.\n\n##### Example\n\ndatacontract.yaml\n```yaml\nservers:\n  api:\n    type: \"api\"\n    location: \"https://api.example.com/path\"\n    delimiter: none # new_line, array, or none (default)\n\nmodels:\n  my_object: # corresponds to the root element of the JSON response\n    type: object\n    fields:\n      field1: \n        type: string\n      fields2: \n        type: number\n```\n\n##### Environment Variables\n\n| Environment Variable                    | Example          | Description                                       |\n|-----------------------------------------|------------------|---------------------------------------------------|\n| `DATACONTRACT_API_HEADER_AUTHORIZATION` | `Bearer \u003ctoken\u003e` | The value for the `authorization` header. Optional. |\n\n\n#### Local\n\nData Contract CLI can test local files in parquet, json, csv, or delta format.\n\n##### Example\n\ndatacontract.yaml\n```yaml\nservers:\n  local:\n    type: local\n    path: ./*.parquet\n    format: parquet\nmodels:\n  my_table_1: # corresponds to a table\n    type: table\n    fields:\n      my_column_1: # corresponds to a column\n        type: varchar\n      my_column_2: # corresponds to a column\n        type: string\n```\n\n\n### export\n```\n                                                                                                    \n Usage: datacontract export [OPTIONS] [LOCATION]                                                    \n                                                                                                    \n Convert data contract to a specific format. Saves to file specified by `output` option if present, \n otherwise prints to stdout.                                                                        \n                                                                                                    \n                                                                                                    \n╭─ Arguments ──────────────────────────────────────────────────────────────────────────────────────╮\n│   location      [LOCATION]  The location (url or path) of the data contract yaml.                │\n│                             [default: datacontract.yaml]                                         │\n╰──────────────────────────────────────────────────────────────────────────────────────────────────╯\n╭─ Options ────────────────────────────────────────────────────────────────────────────────────────╮\n│ *  --format                       [jsonschema|pydantic-model|sod  The export format.             │\n│                                   acl|dbt|dbt-sources|dbt-stagin  [default: None]                │\n│                                   g-sql|odcs|rdf|avro|protobuf|g  [required]                     │\n│                                   reat-expectations|avro-idl|sql                                 │\n│                                   |sql-query|mermaid|html|go|big                                 │\n│                                   query|dbml|spark|sqlalchemy|da                                 │\n│                                   ta-caterer|dcs|markdown|iceber                                 │\n│                                   g|custom|excel|dqx]                                            │\n│    --output                       PATH                            Specify the file path where    │\n│                                                                   the exported data will be      │\n│                                                                   saved. If no path is provided, │\n│                                                                   the output will be printed to  │\n│                                                                   stdout.                        │\n│                                                                   [default: None]                │\n│    --server                       TEXT                            The server name to export.     │\n│                                                                   [default: None]                │\n│    --schema-name                  TEXT                            The name of the schema to      │\n│                                                                   export, e.g., `orders`, or     │\n│                                                                   `all` for all schemas          │\n│                                                                   (default).                     │\n│                                                                   [default: all]                 │\n│    --schema                       TEXT                            The location (url or path) of  │\n│                                                                   the ODCS JSON Schema           │\n│                                                                   [default: None]                │\n│    --engine                       TEXT                            [engine] The engine used for   │\n│                                                                   great expection run.           │\n│                                                                   [default: None]                │\n│    --template                     PATH                            The file path or URL of a      │\n│                                                                   template. For Excel format:    │\n│                                                                   path/URL to custom Excel       │\n│                                                                   template. For custom format:   │\n│                                                                   path to Jinja template.        │\n│                                                                   [default: None]                │\n│    --debug          --no-debug                                    Enable debug logging           │\n│                                                                   [default: no-debug]            │\n│    --help                                                         Show this message and exit.    │\n╰──────────────────────────────────────────────────────────────────────────────────────────────────╯\n╭─ RDF Options ────────────────────────────────────────────────────────────────────────────────────╮\n│ --rdf-base        TEXT  [rdf] The base URI used to generate the RDF graph. [default: None]       │\n╰──────────────────────────────────────────────────────────────────────────────────────────────────╯\n╭─ SQL Options ────────────────────────────────────────────────────────────────────────────────────╮\n│ --sql-server-type        TEXT  [sql] The server type to determine the sql dialect. By default,   │\n│                                it uses 'auto' to automatically detect the sql dialect via the    │\n│                                specified servers in the data contract.                           │\n│                                [default: auto]                                                   │\n╰──────────────────────────────────────────────────────────────────────────────────────────────────╯\n\n```\n\n```bash\n# Example export data contract as HTML\ndatacontract export --format html --output datacontract.html\n```\n\nAvailable export options:\n\n| Type                 | Description                                             | Status  |\n|----------------------|---------------------------------------------------------|---------|\n| `html`               | Export to HTML                                          | ✅       |\n| `jsonschema`         | Export to JSON Schema                                   | ✅       |\n| `odcs`               | Export to Open Data Contract Standard (ODCS) V3         | ✅       |\n| `sodacl`             | Export to SodaCL quality checks in YAML format          | ✅       |\n| `dbt`                | Export to dbt models in YAML format                     | ✅       |\n| `dbt-sources`        | Export to dbt sources in YAML format                    | ✅       |\n| `dbt-staging-sql`    | Export to dbt staging SQL models                        | ✅       |\n| `rdf`                | Export data contract to RDF representation in N3 format | ✅       |\n| `avro`               | Export to AVRO models                                   | ✅       |\n| `protobuf`           | Export to Protobuf                                      | ✅       |\n| `terraform`          | Export to terraform resources                           | ✅       |\n| `sql`                | Export to SQL DDL                                       | ✅       |\n| `sql-query`          | Export to SQL Query                                     | ✅       |\n| `great-expectations` | Export to Great Expectations Suites in JSON Format      | ✅       |\n| `bigquery`           | Export to BigQuery Schemas                              | ✅       |\n| `go`                 | Export to Go types                                      | ✅       |\n| `pydantic-model`     | Export to pydantic models                               | ✅       |\n| `DBML`               | Export to a DBML Diagram description                    | ✅       |\n| `spark`              | Export to a Spark StructType                            | ✅       |\n| `sqlalchemy`         | Export to SQLAlchemy Models                             | ✅       |\n| `data-caterer`       | Export to Data Caterer in YAML format                   | ✅       |\n| `dcs`                | Export to Data Contract Specification in YAML format    | ✅       |\n| `markdown`           | Export to Markdown                                      | ✅       |\n| `iceberg`            | Export to an Iceberg JSON Schema Definition             | partial |\n| `excel`              | Export to ODCS Excel Template                           | ✅       |\n| `custom`             | Export to Custom format with Jinja                      | ✅       |\n| `dqx`                | Export to DQX in YAML format                            | ✅       |\n| Missing something?   | Please create an issue on GitHub                        | TBD     |\n\n#### SQL\n\nThe `export` function converts a given data contract into a SQL data definition language (DDL).\n\n```shell\ndatacontract export datacontract.yaml --format sql --output output.sql\n```\n\nIf using Databricks, and an error is thrown when trying to deploy the SQL DDLs with `variant` columns set the following properties.\n\n```shell\nspark.conf.set(“spark.databricks.delta.schema.typeCheck.enabled”, “false”)\n```\n\n#### Great Expectations\n\nThe `export` function transforms a specified data contract into a comprehensive Great Expectations JSON suite.\nIf the contract includes multiple models, you need to specify the names of the schema/models you wish to export.\n\n```shell\ndatacontract export datacontract.yaml --format great-expectations --model orders\n```\n\nThe export creates a list of expectations by utilizing:\n\n- The data from the Model definition with a fixed mapping\n- The expectations provided in the quality field for each model (find here the expectations gallery: [Great Expectations Gallery](https://greatexpectations.io/expectations/))\n\n##### Additional Arguments\n\nTo further customize the export, the following optional arguments are available:\n\n- **`suite_name`**: The name of the expectation suite. This suite groups all generated expectations and provides a convenient identifier within Great Expectations. If not provided, a default suite name will be generated based on the model name(s).\n\n- **`engine`**: Specifies the engine used to run Great Expectations checks. Accepted values are:\n  - `pandas` — Use this when working with in-memory data frames through the Pandas library.\n  - `spark` — Use this for working with Spark dataframes.\n  - `sql` — Use this for working with SQL databases.\n\n- **`sql_server_type`**: Specifies the type of SQL server to connect with when `engine` is set to `sql`.\n\n  Providing `sql_server_type` ensures that the appropriate SQL dialect and connection settings are applied during the expectation validation.\n\n#### RDF\n\nThe `export` function converts a given data contract into a RDF representation. You have the option to\nadd a base_url which will be used as the default prefix to resolve relative IRIs inside the document.\n\n```shell\ndatacontract export --format rdf --rdf-base https://www.example.com/ datacontract.yaml\n```\n\nThe data contract is mapped onto the following concepts of a yet to be defined Data Contract\nOntology named https://datacontract.com/DataContractSpecification/ :\n- DataContract\n- Server\n- Model\n\nHaving the data contract inside an RDF Graph gives us access the following use cases:\n- Interoperability with other data contract specification formats\n- Store data contracts inside a knowledge graph\n- Enhance a semantic search to find and retrieve data contracts\n- Linking model elements to already established ontologies and knowledge\n- Using full power of OWL to reason about the graph structure of data contracts\n- Apply graph algorithms on multiple data contracts (Find similar data contracts, find \"gatekeeper\"\ndata products, find the true domain owner of a field attribute)\n\n#### DBML\n\nThe export function converts the logical data types of the datacontract into the specific ones of a concrete Database\nif a server is selected via the `--server` option (based on the `type` of that server). If no server is selected, the\nlogical data types are exported.\n\n#### DBT \u0026 DBT-SOURCES\n\nThe export function converts the datacontract to dbt models in YAML format, with support for SQL dialects.\nIf a server is selected via the `--server` option (based on the `type` of that server) then the DBT column `data_types` match the expected data types of the server.\nIf no server is selected, then it defaults to `snowflake`.\n\n#### Spark\n\nThe export function converts the data contract specification into a StructType Spark schema. The returned value is a Python code picture of the model schemas.\nSpark DataFrame schema is defined as StructType. For more details about Spark Data Types please see [the spark documentation](https://spark.apache.org/docs/latest/sql-ref-datatypes.html)\n\n#### Avro\n\nThe export function converts the data contract specification into an avro schema. It supports specifying custom avro properties for logicalTypes and default values.\n\n##### Custom Avro Properties\n\nWe support a **config map on field level**. A config map may include any additional key-value pairs and support multiple server type bindings.\n\nTo specify custom Avro properties in your data contract, you can define them within the `config` section of your field definition. Below is an example of how to structure your YAML configuration to include custom Avro properties, such as `avroLogicalType` and `avroDefault`.\n\n\u003eNOTE: At this moment, we just support [logicalType](https://avro.apache.org/docs/1.11.0/spec.html#Logical+Types) and [default](https://avro.apache.org/docs/1.11.0/spec.htm)\n\n#### Example Configuration\n\n```yaml\nmodels:\n  orders:\n    fields:\n      my_field_1:\n        description: Example for AVRO with Timestamp (microsecond precision) https://avro.apache.org/docs/current/spec.html#Local+timestamp+%28microsecond+precision%29\n        type: long\n        example: 1672534861000000  # Equivalent to 2023-01-01 01:01:01 in microseconds\n        required: true\n        config:\n          avroLogicalType: local-timestamp-micros\n          avroDefault: 1672534861000000\n```\n\n#### Explanation\n\n- **models**: The top-level key that contains different models (tables or objects) in your data contract.\n- **orders**: A specific model name. Replace this with the name of your model.\n- **fields**: The fields within the model. Each field can have various properties defined.\n- **my_field_1**: The name of a specific field. Replace this with your field name.\n  - **description**: A textual description of the field.\n  - **type**: The data type of the field. In this example, it is `long`.\n  - **example**: An example value for the field.\n  - **required**: Is this a required field (as opposed to optional/nullable).\n  - **config**: Section to specify custom Avro properties.\n    - **avroLogicalType**: Specifies the logical type of the field in Avro. In this example, it is `local-timestamp-micros`.\n    - **avroDefault**: Specifies the default value for the field in Avro. In this example, it is 1672534861000000 which corresponds to ` 2023-01-01 01:01:01 UTC`.\n\n#### Data Caterer\n\nThe export function converts the data contract to a data generation task in YAML format that can be\ningested by [Data Caterer](https://github.com/data-catering/data-caterer). This gives you the\nability to generate production-like data in any environment based off your data contract.\n\n```shell\ndatacontract export datacontract.yaml --format data-caterer --model orders\n```\n\nYou can further customise the way data is generated via adding\n[additional metadata in the YAML](https://data.catering/setup/generator/data-generator/)\nto suit your needs.\n\n#### Iceberg\n\nExports to an [Iceberg Table Json Schema Definition](https://iceberg.apache.org/spec/#appendix-c-json-serialization).\n\nThis export only supports a single model export at a time because Iceberg's schema definition is for a single table and the exporter maps 1 model to 1 table, use the `--model` flag\nto limit your contract export to a single model.\n\n```bash\n $ datacontract export --format iceberg --model orders https://datacontract.com/examples/orders-latest/datacontract.yaml --output /tmp/orders_iceberg.json\n \n $ cat /tmp/orders_iceberg.json | jq '.'\n{\n  \"type\": \"struct\",\n  \"fields\": [\n    {\n      \"id\": 1,\n      \"name\": \"order_id\",\n      \"type\": \"string\",\n      \"required\": true\n    },\n    {\n      \"id\": 2,\n      \"name\": \"order_timestamp\",\n      \"type\": \"timestamptz\",\n      \"required\": true\n    },\n    {\n      \"id\": 3,\n      \"name\": \"order_total\",\n      \"type\": \"long\",\n      \"required\": true\n    },\n    {\n      \"id\": 4,\n      \"name\": \"customer_id\",\n      \"type\": \"string\",\n      \"required\": false\n    },\n    {\n      \"id\": 5,\n      \"name\": \"customer_email_address\",\n      \"type\": \"string\",\n      \"required\": true\n    },\n    {\n      \"id\": 6,\n      \"name\": \"processed_timestamp\",\n      \"type\": \"timestamptz\",\n      \"required\": true\n    }\n  ],\n  \"schema-id\": 0,\n  \"identifier-field-ids\": [\n    1\n  ]\n}\n```\n\n#### Custom\n\nThe export function converts the data contract specification into the custom format with Jinja. You can specify the path to a Jinja template with the `--template` argument, allowing you to output files in any format.\n\n```shell\ndatacontract export --format custom --template template.txt datacontract.yaml\n```\n\n##### Jinja variables\n\nYou can directly use the Data Contract Specification as template variables.\n\n```shell\n$ cat template.txt\ntitle: {{ data_contract.info.title }}\n\n$ datacontract export --format custom --template template.txt datacontract.yaml\ntitle: Orders Latest\n```\n\n##### Example Jinja Templates\n\n###### Customized dbt model\n\nYou can export the dbt models containing any logic.\n\nBelow is an example of a dbt staging layer that converts a field of `type: timestamp` to a `DATETIME` type with time zone conversion.\n\ntemplate.sql\n\n{% raw %}\n```sql\n{%- for model_name, model in data_contract.models.items() %}\n{#- Export only the first model #}\n{%- if loop.first -%}\nSELECT\n{%- for field_name, field in model.fields.items() %}\n  {%- if field.type == \"timestamp\" %}\n  DATETIME({{ field_name }}, \"Asia/Tokyo\") AS {{ field_name }},\n  {%- else %}\n  {{ field_name }} AS {{ field_name }},\n  {%- endif %}\n{%- endfor %}\nFROM\n  {{ \"{{\" }} ref('{{ model_name }}') {{ \"}}\" }} \n{%- endif %}\n{%- endfor %}\n```\n{% endraw %}\n\ncommand\n\n```shell\ndatacontract export --format custom --template template.sql --output output.sql datacontract.yaml\n```\n\noutput.sql\n\n```sql\nSELECT\n  order_id AS order_id,\n  DATETIME(order_timestamp, \"Asia/Tokyo\") AS order_timestamp,\n  order_total AS order_total,\n  customer_id AS customer_id,\n  customer_email_address AS customer_email_address,\n  DATETIME(processed_timestamp, \"Asia/Tokyo\") AS processed_timestamp,\nFROM\n  {{ ref('orders') }}\n```\n\n#### ODCS Excel Template\n\nThe `export` function converts a data contract into an ODCS (Open Data Contract Standard) Excel template. This creates a user-friendly Excel spreadsheet that can be used for authoring, sharing, and managing data contracts using the familiar Excel interface.\n\n```shell\ndatacontract export --format excel --output datacontract.xlsx datacontract.yaml\n```\n\nThe Excel format enables:\n- **User-friendly authoring**: Create and edit data contracts in Excel's familiar interface\n- **Easy sharing**: Distribute data contracts as standard Excel files\n- **Collaboration**: Enable non-technical stakeholders to contribute to data contract definitions\n- **Round-trip conversion**: Import Excel templates back to YAML data contracts\n\nFor more information about the Excel template structure, visit the [ODCS Excel Template repository](https://github.com/datacontract/open-data-contract-standard-excel-template).\n\n### import\n```\n                                                                                                    \n Usage: datacontract import [OPTIONS]                                                               \n                                                                                                    \n Create a data contract from the given source location. Saves to file specified by `output` option  \n if present, otherwise prints to stdout.                                                            \n                                                                                                    \n                                                                                                    \n╭─ Options ────────────────────────────────────────────────────────────────────────────────────────╮\n│ *  --format                                 [sql|avro|dbt|dbml|glue|  The format of the source   │\n│                                             jsonschema|json|bigquery  file.                      │\n│                                             |odcs|unity|spark|iceber  [default: None]            │\n│                                             g|parquet|csv|protobuf|e  [required]                 │\n│                                             xcel]                                                │\n│    --output                                 PATH                      Specify the file path      │\n│                                                                       where the Data Contract    │\n│                                                                       will be saved. If no path  │\n│                                                                       is provided, the output    │\n│                                                                       will be printed to stdout. │\n│                                                                       [default: None]            │\n│    --source                                 TEXT                      The path to the file that  │\n│                                                                       should be imported.        │\n│                                                                       [default: None]            │\n│    --dialect                                TEXT                      The SQL dialect to use     │\n│                                                                       when importing SQL files,  │\n│                                                                       e.g., postgres, tsql,      │\n│                                                                       bigquery.                  │\n│                                                                       [default: None]            │\n│    --glue-table                             TEXT                      List of table ids to       │\n│                                                                       import from the Glue       │\n│                                                                       Database (repeat for       │\n│                                                                       multiple table ids, leave  │\n│                                                                       empty for all tables in    │\n│                                                                       the dataset).              │\n│                                                                       [default: None]            │\n│    --bigquery-project                       TEXT                      The bigquery project id.   │\n│                                                                       [default: None]            │\n│    --bigquery-dataset                       TEXT                      The bigquery dataset id.   │\n│                                                                       [default: None]            │\n│    --bigquery-table                         TEXT                      List of table ids to       │\n│                                                                       import from the bigquery   │\n│                                                                       API (repeat for multiple   │\n│                                                                       table ids, leave empty for │\n│                                                                       all tables in the          │\n│                                                                       dataset).                  │\n│                                                                       [default: None]            │\n│    --unity-table-full-name                  TEXT                      Full name of a table in    │\n│                                                                       the unity catalog          │\n│                                                                       [default: None]            │\n│    --dbt-model                              TEXT                      List of models names to    │\n│                                                                       import from the dbt        │\n│                                                                       manifest file (repeat for  │\n│                                                                       multiple models names,     │\n│                                                                       leave empty for all models │\n│                                                                       in the dataset).           │\n│                                                                       [default: None]            │\n│    --dbml-schema                            TEXT                      List of schema names to    │\n│                                                                       import from the DBML file  │\n│                                                                       (repeat for multiple       │\n│                                                                       schema names, leave empty  │\n│                                                                       for all tables in the      │\n│                                                                       file).                     │\n│                                                                       [default: None]            │\n│    --dbml-table                             TEXT                      List of table names to     │\n│                                                                       import from the DBML file  │\n│                                                                       (repeat for multiple table │\n│                                                                       names, leave empty for all │\n│                                                                       tables in the file).       │\n│                                                                       [default: None]            │\n│    --iceberg-table                          TEXT                      Table name to assign to    │\n│                                                                       the model created from the │\n│                                                                       Iceberg schema.            │\n│                                                                       [default: None]            │\n│    --template                               TEXT                      The location (url or path) │\n│                                                                       of the ODCS template       │\n│                                                                       [default: None]            │\n│    --schema                                 TEXT                      The location (url or path) │\n│                                                                       of the ODCS JSON Schema    │\n│                                                                       [default: None]            │\n│    --owner                                  TEXT                      The owner or team          │\n│                                                                       responsible for managing   │\n│                                                                       the data contract.         │\n│                                                                       [default: None]            │\n│    --id                                     TEXT                      The identifier for the the │\n│                                                                       data contract.             │\n│                                                                       [default: None]            │\n│    --debug                    --no-debug                              Enable debug logging       │\n│                                                                       [default: no-debug]        │\n│    --help                                                             Show this message and      │\n│                                                                       exit.                      │\n╰──────────────────────────────────────────────────────────────────────────────────────────────────╯\n\n```\n\nExample:\n```bash\n# Example import from SQL DDL\ndatacontract import --format sql --source my_ddl.sql --dialect postgres\n# To save to file\ndatacontract import --format sql --source my_ddl.sql --dialect postgres --output datacontract.yaml\n```\n\nAvailable import options:\n\n| Type               | Description                                   | Status  |\n|--------------------|-----------------------------------------------|---------|\n| `avro`             | Import from AVRO schemas                      | ✅       |\n| `bigquery`         | Import from BigQuery Schemas                  | ✅       |\n| `csv`              | Import from CSV File                          | ✅       |\n| `dbml`             | Import from DBML models                       | ✅       |\n| `dbt`              | Import from dbt models                        | ✅       |\n| `excel`            | Import from ODCS Excel Template               | ✅       |\n| `glue`             | Import from AWS Glue DataCatalog              | ✅       |\n| `iceberg`          | Import from an Iceberg JSON Schema Definition | partial |\n| `jsonschema`       | Import from JSON Schemas                      | ✅       |\n| `parquet`          | Import from Parquet File Metadata             | ✅       |\n| `protobuf`         | Import from Protobuf schemas                  | ✅       |\n| `spark`            | Import from Spark StructTypes, Variant        | ✅       |\n| `sql`              | Import from SQL DDL                           | ✅       |\n| `unity`            | Import from Databricks Unity Catalog          | partial |\n| Missing something? | Please create an issue on GitHub              | TBD     |\n\n\n#### BigQuery\n\nBigQuery data can either be imported off of JSON Files generated from the table descriptions or directly from the Bigquery API. In case you want to use JSON Files, specify the `source` parameter with a path to the JSON File.\n\nTo import from the Bigquery API, you have to _omit_ `source` and instead need to provide `bigquery-project` and `bigquery-dataset`. Additionally you may specify `bigquery-table` to enumerate the tables that should be imported. If no tables are given, _all_ available tables of the dataset will be imported.\n\nFor providing authentication to the Client, please see [the google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc#how-to) or the one [about authorizing client libraries](https://cloud.google.com/bigquery/docs/authentication#client-libs).\n\nExamples:\n\n```bash\n# Example import from Bigquery JSON\ndatacontract import --format bigquery --source my_bigquery_table.json\n```\n\n```bash\n# Example import from Bigquery API with specifying the tables to import\ndatacontract import --format bigquery --bigquery-project \u003cproject_id\u003e --bigquery-dataset \u003cdataset_id\u003e --bigquery-table \u003ctableid_1\u003e --bigquery-table \u003ctableid_2\u003e --bigquery-table \u003ctableid_3\u003e\n```\n\n```bash\n# Example import from Bigquery API importing all tables in the dataset\ndatacontract import --format bigquery --bigquery-project \u003cproject_id\u003e --bigquery-dataset \u003cdataset_id\u003e\n```\n\n#### Unity Catalog\n```bash\n# Example import from a Unity Catalog JSON file\ndatacontract import --format unity --source my_unity_table.json\n```\n\n```bash\n# Example import single table from Unity Catalog via HTTP endpoint using PAT\nexport DATACONTRACT_DATABRICKS_SERVER_HOSTNAME=\"https://xyz.cloud.databricks.com\"\nexport DATACONTRACT_DATABRICKS_TOKEN=\u003ctoken\u003e\ndatacontract import --format unity --unity-table-full-name \u003ctable_full_name\u003e\n```\n Please refer to [Databricks documentation](https://docs.databricks.com/aws/en/dev-tools/auth/unified-auth) on how to set up a profile\n```bash\n# Example import single table from Unity Catalog via HTTP endpoint using Profile\nexport DATACONTRACT_DATABRICKS_PROFILE=\"my-profile\"\ndatacontract import --format unity --unity-table-full-name \u003ctable_full_name\u003e\n```\n\n#### dbt\n\nImporting from dbt manifest file.\nYou may give the `dbt-model` parameter to enumerate the tables that should be imported. If no tables are given, _all_ available tables of the database will be imported.\n\nExamples:\n\n```bash\n# Example import from dbt manifest with specifying the tables to import\ndatacontract import --format dbt --source \u003cmanifest_path\u003e --dbt-model \u003cmodel_name_1\u003e --dbt-model \u003cmodel_name_2\u003e --dbt-model \u003cmodel_name_3\u003e\n```\n\n```bash\n# Example import from dbt manifest importing all tables in the database\ndatacontract import --format dbt --source \u003cmanifest_path\u003e\n```\n\n#### Excel\n\nImporting from [ODCS Excel Template](https://github.com/datacontract/open-data-contract-standard-excel-template).\n\nExamples:\n\n```bash\n# Example import from ODCS Excel Template\ndatacontract import --format excel --source odcs.xlsx\n```\n\n#### Glue\n\nImporting from Glue reads the necessary Data directly off of the AWS API.\nYou may give the `glue-table` parameter to enumerate the tables that should be imported. If no tables are given, _all_ available tables of the database will be imported.\n\nExamples:\n\n```bash\n# Example import from AWS Glue with specifying the tables to import\ndatacontract import --format glue --source \u003cdatabase_name\u003e --glue-table \u003ctable_name_1\u003e --glue-table \u003ctable_name_2\u003e --glue-table \u003ctable_name_3\u003e\n```\n\n```bash\n# Example import from AWS Glue importing all tables in the database\ndatacontract import --format glue --source \u003cdatabase_name\u003e\n```\n\n#### Spark\n\nImporting from Spark table or view these must be created or accessible in the Spark context. Specify tables list in `source` parameter.  If the `source` tables are registered as tables in Databricks, and they have a table-level descriptions they will also be added to the Data Contract Specification.\n\n```bash\n# Example: Import Spark table(s) from Spark context\ndatacontract import --format spark --source \"users,orders\"\n```\n\n```bash\n# Example: Import Spark table\nDataContract.import_from_source(\"spark\", \"users\")\nDataContract.import_from_source(format = \"spark\", source = \"users\")\n\n# Example: Import Spark dataframe\nDataContract.import_from_source(\"spark\", \"users\", dataframe = df_user)\nDataContract.import_from_source(format = \"spark\", source = \"users\", dataframe = df_user)\n\n# Example: Import Spark table + table description\nDataContract.import_from_source(\"spark\", \"users\", description = \"description\") \nDataContract.import_from_source(format = \"spark\", source = \"users\", description = \"description\")\n\n# Example: Import Spark dataframe + table description\nDataContract.import_from_source(\"spark\", \"users\", dataframe = df_user, description = \"description\")\nDataContract.import_from_source(format = \"spark\", source = \"users\", dataframe = df_user, description = \"description\")\n```\n\n#### DBML\n\nImporting from DBML Documents.\n**NOTE:** Since DBML does _not_ have strict requirements on the types of columns, this import _may_ create non-valid datacontracts, as not all types of fields can be properly mapped. In this case you will have to adapt the generated document manually.\nWe also assume, that the description for models and fields is stored in a Note within the DBML model.\n\nYou may give the `dbml-table` or `dbml-schema` parameter to enumerate the tables or schemas that should be imported. \nIf no tables are given, _all_ available tables of the source will be imported. Likewise, if no schema is given, _all_ schemas are imported.\n\nExamples:\n\n```bash\n# Example import from DBML file, importing everything\ndatacontract import --format dbml --source \u003cfile_path\u003e\n```\n\n```bash\n# Example import from DBML file, filtering for tables from specific schemas\ndatacontract import --format dbml --source \u003cfile_path\u003e --dbml-schema \u003cschema_1\u003e --dbml-schema \u003cschema_2\u003e\n```\n\n```bash\n# Example import from DBML file, filtering for tables with specific names\ndatacontract import --format dbml --source \u003cfile_path\u003e --dbml-table \u003ctable_name_1\u003e --dbml-table \u003ctable_name_2\u003e\n```\n\n```bash\n# Example import from DBML file, filtering for tables with specific names from a specific schema\ndatacontract import --format dbml --source \u003cfile_path\u003e --dbml-table \u003ctable_name_1\u003e --dbml-schema \u003cschema_1\u003e\n```\n\n#### Iceberg\n\nImporting from an [Iceberg Table Json Schema Definition](https://iceberg.apache.org/spec/#appendix-c-json-serialization). Specify location of json files using the `source` parameter.\n\nExamples:\n\n```bash\ndatacontract import --format iceberg --source ./tests/fixtures/iceberg/simple_schema.json --iceberg-table test-table\n```\n\n#### CSV\n\nImporting from CSV File. Specify file in `source` parameter. It does autodetection for encoding and csv dialect\n\nExample:\n\n```bash\ndatacontract import --format csv --source \"test.csv\"\n```\n\n#### protobuf\n\nImporting from protobuf File. Specify file in `source` parameter. \n\nExample:\n\n```bash\ndatacontract import --format protobuf --source \"test.proto\"\n```\n\n\n### catalog\n```\n                                                                                                    \n Usage: datacontract catalog [OPTIONS]                                                              \n                                                                                                    \n Create a html catalog of data contracts.                                                           \n                                                                                                    \n                                                                                                    \n╭─ Options ────────────────────────────────────────────────────────────────────────────────────────╮\n│ --files                   TEXT  Glob pattern for the data contract files to include in the       │\n│                                 catalog. Applies recursively to any subfolders.                  │\n│                                 [default: *.yaml]                                                │\n│ --output                  TEXT  Output directory for the catalog html files. [default: catalog/] │\n│ --schema                  TEXT  The location (url or path) of the ODCS JSON Schema               │\n│                                 [default: None]                                                  │\n│ --debug     --no-debug          Enable debug logging [default: no-debug]                         │\n│ --help                          Show this message and exit.                                      │\n╰──────────────────────────────────────────────────────────────────────────────────────────────────╯\n\n```\n\nExamples:\n\n```\n# create a catalog right in the current folder\ndatacontract catalog --output \".\"\n\n# Create a catalog based on a filename convention\ndatacontract catalog --files \"*.odcs.yaml\"\n```\n\n### publish\n```\n                                                                                                    \n Usage: datacontract publish [OPTIONS] [LOCATION]                                                   \n                                                                                                    \n Publish the data contract to the Entropy Data.                                                     \n                                                                                                    \n                                                                                                    \n╭─ Arguments ──────────────────────────────────────────────────────────────────────────────────────╮\n│   location      [LOCATION]  The location (url or path) of the data contract yaml.                │\n│                             [default: datacontract.yaml]                                         │\n╰──────────────────────────────────────────────────────────────────────────────────────────────────╯\n╭─ Options ────────────────────────────────────────────────────────────────────────────────────────╮\n│ --schema                                       TEXT  The location (url or path) of the ODCS JSON │\n│                                                      Schema                                      │\n│                                                      [default: None]                             │\n│ --ssl-verification    --no-ssl-verification          SSL verification when publishing the data   │\n│                                                      contract.                                   │\n│                                                      [default: ssl-verification]                 │\n│ --debug               --no-debug                     Enable debug logging [default: no-debug]    │\n│ --help                                               Show this message and exit.                 │\n╰──────────────────────────────────────────────────────────────────────────────────────────────────╯\n\n```\n\n### api\n```\n                                                                                                    \n Usage: datacontract api [OPTIONS]                                                                  \n                                                                                                    \n Start the datacontract CLI as server application with REST API.                                    \n                                                                                                    \n The OpenAPI documentation as Swagger UI is available on http://localhost:4242. You can execute the \n commands directly from the Swagger UI.                                                             \n To protect the API, you can set the environment variable DATACONTRACT_CLI_API_KEY to a secret API  \n key. To authenticate, requests must include the header 'x-api-key' with the correct API key. This  \n is highly recommended, as data contract tests may be subject to SQL injections or leak sensitive   \n information.                                                                                       \n To connect to servers (such as a Snowflake data source), set the credentials as environment        \n variables as documented in https://cli.datacontract.com/#test                                      \n It is possible to run the API with extra arguments for `uvicorn.run()` as keyword arguments, e.g.: \n `datacontract api --port 1234 --root_path /datacontract`.                                          \n                                                                                                    \n╭─ Options ────────────────────────────────────────────────────────────────────────────────────────╮\n│ --port                   INTEGER  Bind socket to this port. [default: 4242]                      │\n│ --host                   TEXT     Bind socket to this host. Hint: For running in docker, set it  │\n│                                   to 0.0.0.0                                                     │\n│                                   [default: 127.0.0.1]                                           │\n│ --debug    --no-debug             Enable debug logging [default: no-debug]                       │\n│ --help                            Show this message and exit.                                    │\n╰──────────────────────────────────────────────────────────────────────────────────────────────────╯\n\n```\n\n## Integrations\n\n| Integration           | Option                       | Description                                                                                                   |\n|-----------------------|------------------------------|---------------------------------------------------------------------------------------------------------------|\n| Entropy Data          | `--publish`                  | Push full results to the [Entropy Data API](https://api.entropy-data.com/swagger/index.html)                  |\n\n### Integration with Entropy Data\n\nIf you use [Entropy Data](https://entropy-data.com/), you can use the data contract URL to reference to the contract and append the `--publish` option to send and display the test results. Set an environment variable for your API key.\n\n```bash\n# Fetch current data contract, execute tests on production, and publish result to entropy data\n$ EXPORT ENTROPY_DATA_API_KEY=xxx\n$ datacontract test https://demo.entropy-data.com/demo279750347121/datacontracts/4df9d6ee-e55d-4088-9598-b635b2fdcbbc/datacontract.yaml \\\n --server production \\\n --publish https://api.entropy-data.com/api/test-results\n```\n\n## Best Practices\n\nWe share best practices in using the Data Contract CLI.\n\n### Data-first Approach\n\nCreate a data contract based on the actual data. This is the fastest way to get started and to get feedback from the data consumers.\n\n1. Use an existing physical schema (e.g., SQL DDL) as a starting point to define your logical data model in the contract. Double check right after the import whether the actual data meets the imported logical data model. Just to be sure.\n    ```bash\n    $ datacontract import --format sql --source ddl.sql\n    $ datacontract test\n    ```\n\n2. Add quality checks and additional type constraints one by one to the contract and make sure the\n   data still adheres to the contract.\n   ```bash\n   $ datacontract test\n   ```\n\n3. Validate that the `datacontract.yaml` is correctly formatted and adheres to the Data Contract Specification.\n   ```bash\n   $ datacontract lint\n   ```\n\n4. Set up a CI pipeline that executes daily for continuous quality checks. You can also report the\n   test results to tools like [Data Mesh Manager](https://datamesh-manager.com)\n   ```bash\n   $ datacontract test --publish https://api.datamesh-manager.com/api/test-results\n   ```\n\n### Contract-First\n\nCreate a data contract based on the requirements from use cases.\n\n1. Start with a `datacontract.yaml` template.\n   ```bash\n   $ datacontract init\n   ```\n\n2. Create the model and quality guarantees based on your business requirements. Fill in the terms,\n   descriptions, etc. Validate that your `datacontract.yaml` is correctly formatted.\n    ```bash\n    $ datacontract lint\n    ```\n\n3. Use the export function to start building the providing data product as well as the integration\n   into the consuming data products.\n    ```bash\n    # data provider\n    $ datacontract export --format dbt\n    # data consumer\n    $ datacontract export --format dbt-sources\n    $ datacontract export --format dbt-staging-sql\n    ```\n\n4. Test that your data product implementation adheres to the contract.\n    ```bash\n    $ datacontract test\n    ```\n\n\n## Customizing Exporters and Importers\n\n### Custom Exporter\nUsing the exporter factory to add a new custom exporter\n```python\n\nfrom datacontract.data_contract import DataContract\nfrom datacontract.export.exporter import Exporter\nfrom datacontract.export.exporter_factory import exporter_factory\n\n\n# Create a custom class that implements export method\nclass CustomExporter(Exporter):\n    def export(self, data_contract, model, server, sql_server_type, export_args) -\u003e dict:\n        result = {\n            \"title\": data_contract.info.title,\n            \"version\": data_contract.info.version,\n            \"description\": data_contract.info.description,\n            \"email\": data_contract.info.contact.email,\n            \"url\": data_contract.info.contact.url,\n            \"model\": model,\n            \"model_columns\": \", \".join(list(data_contract.models.get(model).fields.keys())),\n            \"export_args\": export_args,\n            \"custom_args\": export_args.get(\"custom_arg\", \"\"),\n        }\n        return result\n\n\n# Register the new custom class into factory\nexporter_factory.register_exporter(\"custom_exporter\", CustomExporter)\n\n\nif __name__ == \"__main__\":\n    # Create a DataContract instance\n    data_contract = DataContract(\n        data_contract_file=\"/path/datacontract.yaml\"\n    )\n    # Call export\n    result = data_contract.export(\n        export_format=\"custom_exporter\", model=\"orders\", server=\"production\", custom_arg=\"my_custom_arg\"\n    )\n    print(result)\n\n```\nOutput\n```python\n{\n 'title': 'Orders Unit Test',\n 'version': '1.0.0',\n 'description': 'The orders data contract',\n 'email': 'team-orders@example.com',\n 'url': 'https://wiki.example.com/teams/checkout',\n 'model': 'orders',\n 'model_columns': 'order_id, order_total, order_status',\n 'export_args': {'server': 'production', 'custom_arg': 'my_custom_arg'},\n 'custom_args': 'my_custom_arg'\n}\n```\n\n### Custom Importer\nUsing the importer factory to add a new custom importer\n```python\n\nfrom datacontract.model.data_contract_specification import DataContractSpecification, Field, Model\nfrom datacontract.data_contract import DataContract\nfrom datacontract.imports.importer import Importer\nfrom datacontract.imports.importer_factory import importer_factory\n\nimport json\n\n# Create a custom class that implements import_source method\nclass CustomImporter(Importer):\n    def import_source(\n        self, data_contract_specification: DataContractSpecification, source: str, import_args: dict\n    ) -\u003e dict:\n        source_dict = json.loads(source)\n        data_contract_specification.id = source_dict.get(\"id_custom\")\n        data_contract_specification.info.title = source_dict.get(\"title\")\n        data_contract_specification.info.version = source_dict.get(\"version\")\n        data_contract_specification.info.description = source_dict.get(\"description_from_app\")\n\n        for model in source_dict","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatacontract%2Fdatacontract-cli","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatacontract%2Fdatacontract-cli","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatacontract%2Fdatacontract-cli/lists"}