{"id":28404665,"url":"https://github.com/sodadata/prefect-soda-core","last_synced_at":"2025-08-16T09:07:44.046Z","repository":{"id":55071786,"uuid":"522871278","full_name":"sodadata/prefect-soda-core","owner":"sodadata","description":null,"archived":false,"fork":false,"pushed_at":"2023-12-15T11:08:54.000Z","size":1275,"stargazers_count":16,"open_issues_count":9,"forks_count":4,"subscribers_count":11,"default_branch":"main","last_synced_at":"2025-08-10T13:47:12.287Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sodadata.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null}},"created_at":"2022-08-09T08:48:18.000Z","updated_at":"2024-01-30T10:08:00.000Z","dependencies_parsed_at":"2023-01-30T21:16:02.881Z","dependency_job_id":null,"html_url":"https://github.com/sodadata/prefect-soda-core","commit_stats":{"total_commits":17,"total_committers":5,"mean_commits":3.4,"dds":0.6470588235294117,"last_synced_commit":"b0cd520fb42f4c60285de026c8d77af8b50ac72a"},"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/sodadata/prefect-soda-core","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sodadata%2Fprefect-soda-core","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sodadata%2Fprefect-soda-core/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sodadata%2Fprefect-soda-core/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sodadata%2Fprefect-soda-core/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sodadata","download_url":"https://codeload.github.com/sodadata/prefect-soda-core/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sodadata%2Fprefect-soda-core/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270689909,"owners_count":24628762,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-16T02:00:11.002Z","response_time":91,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-01T20:37:29.892Z","updated_at":"2025-08-16T09:07:44.031Z","avatar_url":"https://github.com/sodadata.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# prefect-soda-core\n\n## Welcome!\n\nPrefect 2.0 collection for Soda Core\n\n## Getting Started\n\n### Python setup\n\nRequires an installation of Python 3.7+.\n\nWe recommend using a Python virtual environment manager such as pipenv, conda or virtualenv.\n\nThese tasks are designed to work with Prefect 2.0. For more information about how to use Prefect, please refer to the [Prefect documentation](https://orion-docs.prefect.io/).\n\n### Installation\n\n\n`prefect-soda-core` is based on `soda-core`.  \nAs `soda-core` requires you to specify the right option for your database, so does `prefect-soda-core`.  \nI.e. to use `prefect-soda-core` with Snowflake, run the following:\n\n```bash\npip install prefect-soda-core[snowflake]\n```\n\nYou can find the list of supported options in `setup.py`.\n\n**Please note that since this integration is built on top of Soda CLI, it is not possible to run data quality checks using Spark.**\n\n### Write and run a flow\n\n```python\nfrom prefect import flow\nfrom prefect.context import get_run_context\nfrom prefect_soda_core.soda_configuration import SodaConfiguration\nfrom prefect_soda_core.sodacl_check import SodaCLCheck\nfrom prefect_soda_core.tasks import soda_scan_execute\n\n\n@flow\ndef run_soda_scan():\n    soda_configuration_block = SodaConfiguration(\n        configuration_yaml_path=\"/path/to/config.yaml\"\n    )\n    soda_check_block = SodaCLCheck(\n        sodacl_yaml_path=\"/path/to/checks.yaml\"\n    )\n    \n    # Using the flow_run_name as the name of the file to store the scan results\n    flow_run_name = get_run_context().flow_run.name\n    scan_results_file_path = f\"{flow_run_name}.json\"\n    \n    return soda_scan_execute(\n        data_source_name=\"my_datasource\",\n        configuration=soda_configuration_block,\n        checks=soda_check_block,\n        variables={\"var\": \"value\"},\n        scan_results_file=scan_results_file_path,\n        verbose=True,\n        return_scan_result_file_content=False,\n        shell_env={\"SNOWFLAKE_PASSWORD\": \"********\"}\n    )\n\nrun_soda_scan()\n```\n\n## Resources\n\nIf you encounter any bugs while using `prefect-soda-core`, feel free to open an issue in the [prefect-soda-core](https://github.com/sodadata/prefect-soda-core) repository.\n\nIf you have any questions or issues while using `prefect-soda-core`, you can find help in either the [Prefect Discourse forum](https://discourse.prefect.io/) or the [Prefect Slack community](https://prefect.io/slack).\n\n## Development\n\nIf you'd like to install a version of `prefect-soda-core` for development, clone the repository and perform an editable install with `pip`:\n\n```bash\ngit clone https://github.com/sodadata/prefect-soda-core.git\n\ncd prefect-soda-core/\n\npip install -e \".[dev]\"\n\n# Install linting pre-commit hooks\npre-commit install\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsodadata%2Fprefect-soda-core","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsodadata%2Fprefect-soda-core","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsodadata%2Fprefect-soda-core/lists"}