https://github.com/infinityworks/iw-tech-test-snowflake
https://github.com/infinityworks/iw-tech-test-snowflake
Last synced: 5 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/infinityworks/iw-tech-test-snowflake
- Owner: infinityworks
- License: mit
- Created: 2020-12-04T11:01:56.000Z (over 5 years ago)
- Default Branch: main
- Last Pushed: 2023-10-03T00:54:25.000Z (over 2 years ago)
- Last Synced: 2024-04-17T18:16:50.940Z (about 2 years ago)
- Language: Python
- Size: 14.6 KB
- Stars: 3
- Watchers: 5
- Forks: 13
- Open Issues: 11
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Snowflake Data Test - Starter Project
### Prerequisites
#### Python 3.8.* or later.
See installation instructions at: https://www.python.org/downloads/
Check you have python3 installed:
```bash
python3 --version
```
### Dependencies and data
#### Creating a virtual environment
Ensure your pip (package manager) is up to date:
```bash
pip3 install --upgrade pip
```
To check your pip version run:
```bash
pip3 --version
```
Create the virtual environment in the root of the cloned project:
```bash
python3 -m venv .venv
```
#### Activating the newly created virtual environment
You always want your virtual environment to be active when working on this project.
```bash
source ./.venv/bin/activate
```
#### Installing Python requirements
This will install some of the packages you might find useful:
```bash
pip3 install -r ./requirements.txt
```
#### Generating the data
A data generator is included as part of the project in `./input_data_generator/main_data_generator.py`
This allows you to generate a configurable number of months of data.
Although the technical test specification mentions 6 months of data, it's best to generate
less than that initially to help improve the debugging process.
To run the data generator use:
```bash
python ./input_data_generator/main_data_generator.py
```
This should produce customers, products and transaction data under `./input_data/starter`
#### Getting started
Please save Snowflake model code in `snowflake` and infrastructure code in `infra` folder.
Update this README as code evolves.