https://github.com/dimajix/flowman-tutorial

Tutorial for Flowman
https://github.com/dimajix/flowman-tutorial

Last synced: 7 days ago
JSON representation

Tutorial for Flowman

• Host: GitHub
• URL: https://github.com/dimajix/flowman-tutorial
• Owner: dimajix
• License: apache-2.0
• Created: 2022-03-20T09:18:18.000Z (almost 3 years ago)
• Default Branch: main
• Last Pushed: 2023-07-21T12:02:16.000Z (over 1 year ago)
• Last Synced: 2024-11-09T17:38:41.083Z (2 months ago)
• Language: Shell
• Size: 112 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Flowman Tutorial

Welcome to this tutorial for developing data transformation applications with Flowman. In a sequence of multiple

lessons, you will start learning the basic concepts of Flowman and later get known to more advanced features of

Flowman. All lessons will use a subset of a publicly available data set about weather data. This data is taken from

[Global Hourly - Integrated Surface Database (ISD)](https://www.ncdc.noaa.gov/isd).

## Running Flowman in Docker

The simplest way to follow the tutorial is to use the provided `docker-compose.yml` which will start a Docker container

containing Flowman with all lessons mounted as a volume.

### Start MS SQL Server

For some lessons, you will need an MS SQL Server. This is provided as a simple docker container and can be started

as follows:

```shell

docker-compose up -d sqlserver

```

### Start Flowman Container

Once the MS SQL Server is up, you need to start the Flowman container, which will mount all lessons in a volume

under `/home/flowman/lessons`. All `README.md` files assume that you changed into the directory `/home/flowman`.

```shell

docker-compose run --rm flowman bash

cd /home/flowman

```