https://github.com/itallix/python_bdd_testcontainers_demo

Demo code for testing data migration from MS SQL Server into Google Datastore with pytest_bdd and testcontainers
https://github.com/itallix/python_bdd_testcontainers_demo

bdd pytest-bdd python testcontainers

Last synced: about 2 months ago
JSON representation

Demo code for testing data migration from MS SQL Server into Google Datastore with pytest_bdd and testcontainers

• Host: GitHub
• URL: https://github.com/itallix/python_bdd_testcontainers_demo
• Owner: itallix
• License: mit
• Created: 2022-10-04T12:04:50.000Z (over 2 years ago)
• Default Branch: main
• Last Pushed: 2022-10-04T12:09:27.000Z (over 2 years ago)
• Last Synced: 2024-10-31T20:44:00.603Z (3 months ago)
• Topics: bdd, pytest-bdd, python, testcontainers
• Language: Python
• Homepage:
• Size: 54.7 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# `bdd_testcontainers_demo`

![BDD Tests](https://github.com/itallix/py_bdd_testcontainers_demo/actions/workflows/test.yaml/badge.svg)

The purpose of this code is to demonstrate the technique of testing the data

migration scenario with [`pytest-bdd`](https://github.com/pytest-dev/pytest-bdd)

 and [`testcontainers`](https://github.com/testcontainers/testcontainers-python).

When the nature of the source and target data is quite different (e.g. SQL -> NoSQL),

and the mapping between those requires building additional logic. Then testing and

having fast development feedback loop becomes very important.

As a source for data migration here, we use MSSQL DB and as a target - Google Datastore.

However, this testing recipe could be easily adapted to many types of data sources/targets.

## Mapping from MSSSQL to Datastore

The following schema shows high level overview of the mapping that we are

going to use in our example:

![Mapping](./doc/sql_to_datastore_mapping.svg)

-   Source data lives in the SQL DB and is scattered in various tables

-   Target data should live in the NoSQL store and capture all source data in

the single kind

-   Mapping between these two assumes building some SQL query and transforming

some data

## Running tests

### Prerequisites

-   Make sure that you have [Poetry](https://python-poetry.org/docs/) installed 

for dependency management

-   Depending on your system, `pyodbc` might require additional binary dependency,

see more [here](https://pypi.org/project/pyodbc/)

-   Initialize an environment with `poetry install`

```bash

poetry run pytest tests/bdd

```