{"id":22414804,"url":"https://github.com/alessio-siciliano/bigquery-advanced-utils","last_synced_at":"2026-04-28T20:33:47.849Z","repository":{"id":268178752,"uuid":"898684522","full_name":"Alessio-Siciliano/bigquery-advanced-utils","owner":"Alessio-Siciliano","description":"BigQuery-advanced-utils is a lightweight utility library that extends the official Google BigQuery Python client. It simplifies tasks like query management, data processing, and automation. Aimed at developers and data scientists, the project is open to contributions to improve and enhance its functionality.","archived":false,"fork":false,"pushed_at":"2024-12-30T09:45:35.000Z","size":62,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-27T03:44:20.180Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Alessio-Siciliano.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-04T20:56:06.000Z","updated_at":"2024-12-30T09:44:02.000Z","dependencies_parsed_at":null,"dependency_job_id":"c53a6508-05b1-4b1d-8797-a42bcdffbb70","html_url":"https://github.com/Alessio-Siciliano/bigquery-advanced-utils","commit_stats":null,"previous_names":["alessio-siciliano/bigquery-advanced-utils"],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Alessio-Siciliano%2Fbigquery-advanced-utils","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Alessio-Siciliano%2Fbigquery-advanced-utils/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Alessio-Siciliano%2Fbigquery-advanced-utils/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Alessio-Siciliano%2Fbigquery-advanced-utils/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Alessio-Siciliano","download_url":"https://codeload.github.com/Alessio-Siciliano/bigquery-advanced-utils/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245779444,"owners_count":20670684,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bigquery","datatransfer","google-cloud","python"],"created_at":"2024-12-05T15:09:26.893Z","updated_at":"2026-04-28T20:33:47.805Z","avatar_url":"https://github.com/Alessio-Siciliano.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# bigquery-advanced-utils\n\n![Total Downloads](https://static.pepy.tech/badge/bigquery-advanced-utils?textLength=250)\n![License](https://img.shields.io/badge/license-GNU%20GPL%20v3-blue)\n![Version](https://img.shields.io/github/v/release/Alessio-Siciliano/bigquery-advanced-utils)\n[![codecov](https://codecov.io/gh/Alessio-Siciliano/bigquery-advanced-utils/graph/badge.svg?token=TA7EPVWA7E)](https://codecov.io/gh/Alessio-Siciliano/bigquery-advanced-utils)\n![PyPI](https://img.shields.io/pypi/v/bigquery-advanced-utils)\n![Issues](https://img.shields.io/github/issues/Alessio-Siciliano/bigquery-advanced-utils)\n![Contributors](https://img.shields.io/github/contributors/Alessio-Siciliano/bigquery-advanced-utils)\n![PyLint](https://img.shields.io/github/actions/workflow/status/Alessio-Siciliano/bigquery-advanced-utils/pylint.yml?branch=main\u0026label=PyLint\u0026logo=python)\n![Black](https://img.shields.io/github/actions/workflow/status/Alessio-Siciliano/bigquery-advanced-utils/black_formatter.yml?branch=main\u0026label=Black\u0026logo=python)\n\n**BigQuery-advanced-utils** is a lightweight utility library that extends the official Google BigQuery Python client.  \nIt simplifies tasks like **query management**, *data processing*, and *automation*.  \n\nAimed at **developers** and **data scientists**, the project is open to *contributions* to improve and enhance its functionality.\n\nFor full documentation, please visit: [BigQuery-Advanced-Utils Documentation](https://alessio-siciliano.github.io/bigquery-utils/)\n\n## Why This Library?\n\nI created **bigquery-advanced-utils** because I often found myself facing complex or uncommon tasks when working with BigQuery, and there was little or no support available online. Rather than spending time reinventing the wheel, I decided to create this library to help others avoid the same challenges. I hope that everyone can contribute in the same spirit, so feel free to get involved and make this library even better!\n\n## Requirements\n- Python 3.10+\n\n## Installation 📦\n\n### Install via pip (recommended)\n\nRun the following command in your terminal:\n```bash\npip install bigquery-advanced-utils\n```\n### Install in a Virtual Environment\n\n1. Create a virtual environment:\n```bash\npython -m venv venv\n```\n2. Activate the environment and install:\n```bash\nsource venv/bin/activate  # on macOS/Linux  \nvenv\\Scripts\\activate     # on Windows  \npip install bigquery-advanced-utils\n```\n\n## Usage Examples 🚀\n### Quick Start\n\n```python\nfrom bigquery_advanced_utils.bigquery import BigQueryClient\n\n# Initialize helper with your project ID\nhelper = BigQueryClient(project_id=\"your_project_id\")\n\n# Load data from CSV\nhelper.load_data_from_csv(file_path=\"your_data.csv\")\n```\n### Data Validation\n```python\ntest_functions=[\n    partial(\n        # Check if any null values exist in the \"age\" column\n        b.check_no_nulls, columns_to_test=[\"age\"],\n    ),\n    partial(\n        # Ensure values in the \"email\" column are unique\n        b.check_unique_column, columns_to_test=[\"email\"]\n    )\n]\n```\n### Search by table or owner in Datatransfer\n```python\nfrom bigquery_advanced_utils.datatransfer import DataTransferClient\n\nhelper = DataTransferClient()\n# Call the function with two parameters: owner email and project id\nlist_transfer_config_by_owner_email(owner_email=\"my-email@email.com\", project_id=\"my-project\")\n\n# Get the scheduled queries by the name of a table (it's case sensitive) \nlist_transfer_configs_by_table(table_id=\"my-table\", project_id=\"my-project\")\n```\n\n## Planned features 🚧\n- A new query builder.\n- Custom data transformation and processing functions.\n- Exclusive features with datatransfer.\n- Utility functions to manipulate strings and query.\n\n## Contributing\n\nWe are always open to contributions! Whether you have a bug fix, a feature request, or a general improvement to make, your help is appreciated. Here are some ways you can contribute:\n\n- **Bug reports**: Help us catch issues before they affect users.\n- **New features**: Suggest new functionalities that could improve the usability of the package.\n- **Code improvements**: Review the code and suggest optimizations or fixes.\n\nPlease follow the [contributing guide](CONTRIBUTING.md) for more details on how to get started.\n\n## License\nThis project is licensed under the **GNU General Public License**. See the [LICENSE](LICENSE) file for details.\n\n\n## Contact\nFor questions or feedback, feel free to open an issue or reach out to me.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falessio-siciliano%2Fbigquery-advanced-utils","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falessio-siciliano%2Fbigquery-advanced-utils","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falessio-siciliano%2Fbigquery-advanced-utils/lists"}