{"id":15060768,"url":"https://github.com/mlabarrere/pygquery","last_synced_at":"2026-02-04T10:39:34.669Z","repository":{"id":53548099,"uuid":"324533022","full_name":"mlabarrere/PygQuery","owner":"mlabarrere","description":"🐷 Multitread your data with Google BigQuery","archived":false,"fork":false,"pushed_at":"2021-03-22T18:45:36.000Z","size":41,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-05-18T07:47:20.169Z","etag":null,"topics":["bigquery","dataframe","google-bigquery","multithreading","pandas","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mlabarrere.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null},"funding":{"github":["mlabarrere"]}},"created_at":"2020-12-26T10:39:49.000Z","updated_at":"2024-03-12T12:46:54.000Z","dependencies_parsed_at":"2022-08-29T14:41:15.037Z","dependency_job_id":null,"html_url":"https://github.com/mlabarrere/PygQuery","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/mlabarrere/PygQuery","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mlabarrere%2FPygQuery","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mlabarrere%2FPygQuery/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mlabarrere%2FPygQuery/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mlabarrere%2FPygQuery/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mlabarrere","download_url":"https://codeload.github.com/mlabarrere/PygQuery/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mlabarrere%2FPygQuery/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262854366,"owners_count":23375112,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bigquery","dataframe","google-bigquery","multithreading","pandas","python"],"created_at":"2024-09-24T23:04:20.112Z","updated_at":"2026-02-04T10:39:34.606Z","avatar_url":"https://github.com/mlabarrere.png","language":"Python","funding_links":["https://github.com/sponsors/mlabarrere"],"categories":[],"sub_categories":[],"readme":"# 🐷 PygQuery\n\nMulti-treaded wrapper to read and write Pandas dataframes with Google BigQuery without the hassle of the heavy BigQuery API.\n\nPygQuery is multi-treaded by design, meaning that any SQL request is a thread of its own. The advantage of it is that you can send multiple requests in parallel, and have awaiting inbound data ready for later.\n\n### Install\nOn CLI, just type: \n```shell\npip install pygquery\n```\n\n### Read Data\n\nLet's import the module first\n```python\nfrom pygquery.bigquery import BigQueryReader\n```\n\nThe module takes 3 arguments as an input:\n1. `request` : A string of your query. E.g. `\"\"\"SELECT * FROM myproject.dataset.table\"\"\"`\n2. `project` : The string of the project you are currently gathering data from\n3. `api_key_path` : a path of the G Service Account key, you can create one in the IAM tab of your GCP interface\n\nLet's instantiate our data reader:\n```python\nreader_dict = {\n  'request' : \"\"\"SELECT * FROM myproject.dataset.table\"\"\",\n  'project' : 'myproject',\n  'api_key_path' : 'folder/key.json'\n}\n\n# If there any error in your query at the instantiation stage, BigQuery will let you know\nmy_request = BigQueryReader(**reader_dict) \n```\nNow you have an object ready to be launched. If the line of code above executes, you know that:\n1. There is no error in the SQL\n2. There is no credentials failure\n\n\nLet's fire up this object:\n\n```python\n\nmy_request.start() # Launch the thread for downloading data\n\n\"# ... Do other things while data is downloading, like launching another request ... #\"\n\nmy_request.join() # Tell to Python to wait for your download to complete\n\nmy_data = myRequest.data # Get your data\n```\n\nEt voilà! You have your data in Pandas `DataFrame` format ready to be crunched.\n```python\nmy_data.info()\nmy_data.head()\n\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmlabarrere%2Fpygquery","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmlabarrere%2Fpygquery","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmlabarrere%2Fpygquery/lists"}