{"id":20908668,"url":"https://github.com/ottogroup/bquest","last_synced_at":"2025-08-14T08:20:54.449Z","repository":{"id":173340195,"uuid":"650521622","full_name":"ottogroup/bquest","owner":"ottogroup","description":"Effortlessly validate and test your Google BigQuery queries with the power of pandas DataFrames in Python.","archived":false,"fork":false,"pushed_at":"2025-06-02T07:27:29.000Z","size":2238,"stargazers_count":18,"open_issues_count":4,"forks_count":0,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-07-04T12:35:36.818Z","etag":null,"topics":["bigquery","google-big-query","google-cloud","integration","testing"],"latest_commit_sha":null,"homepage":"https://ottogroup.github.io/bquest/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ottogroup.png","metadata":{"files":{"readme":"README.rst","changelog":"CHANGELOG.rst","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-06-07T08:44:20.000Z","updated_at":"2025-06-02T07:12:41.000Z","dependencies_parsed_at":"2023-12-19T12:13:24.134Z","dependency_job_id":"3ac7d6ad-3311-475b-8ee8-fb7a3cdbc468","html_url":"https://github.com/ottogroup/bquest","commit_stats":{"total_commits":45,"total_committers":8,"mean_commits":5.625,"dds":0.3555555555555555,"last_synced_commit":"73f0ab6ee6b3f80b532963822d72f971c77b4c0e"},"previous_names":["ottogroup/bquest"],"tags_count":13,"template":false,"template_full_name":null,"purl":"pkg:github/ottogroup/bquest","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ottogroup%2Fbquest","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ottogroup%2Fbquest/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ottogroup%2Fbquest/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ottogroup%2Fbquest/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ottogroup","download_url":"https://codeload.github.com/ottogroup/bquest/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ottogroup%2Fbquest/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270385487,"owners_count":24574564,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-14T02:00:10.309Z","response_time":75,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bigquery","google-big-query","google-cloud","integration","testing"],"created_at":"2024-11-18T14:08:23.091Z","updated_at":"2025-08-14T08:20:54.392Z","avatar_url":"https://github.com/ottogroup.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":".. image:: https://raw.githubusercontent.com/ottogroup/bquest/main/docs/assets/logo.svg\n    :alt: BQuest Logo\n\nBQuest\n######\n\nEffortlessly validate and test your Google BigQuery queries with the power of pandas DataFrames in Python.\n\nWe would like to thank `Mike Czech \u003chttps://github.com/mikeczech\u003e`_ who is the original inventor of bquest!\n\n**Warning**\n\nThis library is a work in progress!\n\nBreaking changes should be expected until a 1.0 release, so version pinning is recommended.\n\n.. image:: https://github.com/ottogroup/bquest/workflows/Tests/badge.svg\n   :target: https://github.com/ottogroup/bquest/actions?workflow=Tests\n   :alt: CI: Overall outcome\n.. image:: https://github.com/ottogroup/bquest/actions/workflows/pages/pages-build-deployment/badge.svg?branch=gh-pages\n   :target: https://github.com/ottogroup/bquest/actions/workflows/pages/pages-build-deployment\n   :alt: CD: gh-pages documentation\n.. image:: https://img.shields.io/pypi/v/bquest.svg\n   :target: https://pypi.org/project/bquest/\n   :alt: PyPI version\n.. image:: https://img.shields.io/pypi/status/bquest.svg\n   :target: https://pypi.python.org/pypi/bquest/\n   :alt: Project status (alpha, beta, stable)\n.. image:: https://static.pepy.tech/personalized-badge/bquest?period=month\u0026units=international_system\u0026left_color=grey\u0026right_color=blue\u0026left_text=PyPI%20downloads/month\n   :target: https://pepy.tech/project/bquest\n   :alt: PyPI downloads\n.. image:: https://img.shields.io/github/license/ottogroup/bquest\n   :target: https://github.com/ottogroup/bquest/blob/main/LICENSE\n   :alt: Project license\n.. image:: https://img.shields.io/pypi/pyversions/bquest.svg\n   :target: https://pypi.python.org/pypi/bquest/\n   :alt: Python version compatibility\n.. image:: https://img.shields.io/badge/code%20style-black-000000.svg\n   :target: https://github.com/psf/black\n   :alt: Documentation: Black\n\nOverview\n********\n\n* Use BQuest in combination with your favorite testing framework (e.g. pytest).\n* Create temporary test tables from JSON_ or `pandas DataFrame`_.\n* Run BQ configurations and plain SQL queries on your test tables and check the result.\n\n.. _JSON: https://cloud.google.com/bigquery/docs/loading-data\n.. _pandas DataFrame: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html\n\nInstallation\n************\n\nVia PyPi (standard):\n\n.. code-block:: bash\n\n    pip install bquest\n\n\nVia Github (most recent):\n\n.. code-block:: bash\n\n    pip install git+https://github.com/ottogroup/bquest\n\n\nBQuest also requires a dedicated BigQuery dataset for storing test tables, e.g.\n\n.. code-block:: yaml\n\n    resource \"google_bigquery_dataset\" \"bquest\" {\n      dataset_id    = \"bquest\"\n      friendly_name = \"bquest\"\n      description   = \"Source tables for bquest tests\"\n      location      = \"EU\"\n      default_table_expiration_ms = 3600000\n    }\n\nWe recommend setting an `expiration time`_ for tables in the bquest dataset to assure removal of those test tables upon\ntest execution.\n\n.. _`expiration time`: https://www.terraform.io/docs/providers/google/r/bigquery_dataset.html#default_table_expiration_ms\n\nExample\n*******\n\nGiven a pandas DataFrame\n\n.. list-table::\n   :widths: 30 30 30\n   :header-rows: 1\n\n   * - foo\n     - weight\n     - prediction_date\n   * - bar\n     - 23\n     - 20190301\n   * - my\n     - 42\n     - 20190301\n\nand its table definition\n\n.. code-block:: python\n\n    from bquest.tables import BQTableDefinitionBuilder\n\n    table_def_builder = BQTableDefinitionBuilder(GOOGLE_PROJECT_ID, dataset=\"bquest\", location=\"EU\")\n    table_definition = table_def_builder.from_df(\"abc.feed_latest\", df)\n\nyou can use the config file *./abc/config.py*\n\n.. code-block:: json-object\n\n    {\n        \"query\": \"\"\"\n            SELECT\n                foo,\n                PARSE_DATE('%Y%m%d', prediction_date)\n            FROM\n                `{source_table}`\n            WHERE\n                weight \u003e {THRESHOLD}\n        \"\"\",\n        \"start_date\": \"prediction_date\",\n        \"end_date\": \"prediction_date\",\n        \"source_tables\": {\"source_table\": \"abc.feed_latest\"},\n        \"feature_table_name\": \"abc.myid\",\n    }\n\nand the runner\n\n.. code-block:: python\n\n    from bquest.runner import BQConfigFileRunner, BQConfigRunner\n\n    runner = BQConfigFileRunner(\n        BQConfigRunner(bq_client, bq_executor_func),\n        \"config/bq_config\",\n    )\n\n    result_df = runner.run_config(\n        \"20190301\",\n        \"20190308\",\n        [table_definition],\n        \"abc/config.py\",\n        templating_vars={\"THRESHOLD\": \"30\"},\n    )\n\nto assert the result table\n\n.. code-block:: python\n\n    assert result_df.shape == (1, 2)\n    assert result_df.iloc[0][\"foo\"] == \"my\"\n\nTesting\n*******\n\nFor the actual testing bquest relies on an accessible BigQuery project which can be configured\nwith the gcloud_ client. The corresponding ``GOOGLE_PROJECT_ID`` is extracted from this project\nand used with pandas-gbq_ to write temporary tables to the bquest dataset that has to be pre-\nconfigured before testing on that project.\n\nFor Github CI we have configured an identity provider in our testing project which allows\nonly core members of this repository to access the testing projects' resources.\n\n.. _gcloud: https://cloud.google.com/sdk/docs/install?hl=de\n.. _pandas-gbq: https://github.com/googleapis/python-bigquery-pandas\n\nImportant Links\n***************\n\n- Full documentation: https://ottogroup.github.io/bquest/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fottogroup%2Fbquest","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fottogroup%2Fbquest","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fottogroup%2Fbquest/lists"}