Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/poogles/pytest-bq
pytest fixtures for a local bigquery suitable for local development.
https://github.com/poogles/pytest-bq
bigquery bigquery-emulator pytest
Last synced: about 1 month ago
JSON representation
pytest fixtures for a local bigquery suitable for local development.
- Host: GitHub
- URL: https://github.com/poogles/pytest-bq
- Owner: Poogles
- License: mit
- Created: 2024-05-08T15:10:02.000Z (6 months ago)
- Default Branch: main
- Last Pushed: 2024-05-08T18:36:40.000Z (6 months ago)
- Last Synced: 2024-10-12T21:02:32.842Z (about 1 month ago)
- Topics: bigquery, bigquery-emulator, pytest
- Language: Python
- Homepage:
- Size: 38.1 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
## Pytest BQ
This is a pytest plugin in similar vein to [pytest-postgres](https://github.com/ClearcodeHQ/pytest-postgresql) and [pytest-kafka](https://pypi.org/project/pytest-kafka/). It runs a local version of bigquery that you can use throughout your test suite.
This would have been much more painful without [Mirakuru](https://github.com/ClearcodeHQ/mirakuru)
and [bigquery-emulator](https://github.com/goccy/bigquery-emulator); this is a simple wrapper around
those tools.
### Installation
This tool requires you to have a copy of the `bigquery-emulator` binary somewhere on your path.
Depending upon your architecture you'll need a different version of the tool.
```sh
wget https://github.com/goccy/bigquery-emulator/releases/download/v0.6.1/bigquery-emulator-linux-amd64
mv bigquery-emulator-linux-amd64 /usr/local/bin/bigquery-emulator
```
To install this library:
```sh
pip install pytest-bq
```
### Demo
```python
# conftest.py
from pytest_bq.factories import client as bq_client
from pytest_bq.factories import proc as bq_process
# Create a process and a local client that targets that process.
bq_proc = bq_process.bq_proc(executable='/usr/local/bin/bigquery-emulator', project_id='test')
bqlocal = bq_client.bqlocal("bq_proc")
# tests/test_bq.py
from google.cloud import bigquery
from pytest_bq.executor.process import BQExecutor
def test_can_list_datasets(bq_proc: BQExecutor, bqlocal: bigquery.Client) -> None:
"""MVP to ensure everything works."""
test_dataset = "test_base"
bqlocal.create_dataset(test_dataset)
datasets = [x.full_dataset_id for x in bqlocal.list_datasets()]
assert test_dataset in datasets
```
### Contributing
PRs are accepted.
```sh
# Install the dependencies with:
pip install .[test]
# Install pre-commit hooks.
pre-commit install
# Validate everything passes.
pre-commit run --all
# Run the tests.
pytest tests/
```
### TODOs
* Validate `data_from_yaml` files.
* Tools for dummy data generation.