{"id":21514911,"url":"https://github.com/getindata/ververica-platform-flink-workshop","last_synced_at":"2026-03-19T20:41:50.889Z","repository":{"id":46013093,"uuid":"413721512","full_name":"getindata/ververica-platform-flink-workshop","owner":"getindata","description":"All tools for local Ververica Platform setup with Flink SQL","archived":false,"fork":false,"pushed_at":"2021-11-20T09:32:19.000Z","size":30,"stargazers_count":1,"open_issues_count":0,"forks_count":3,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-01-24T02:31:04.750Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/getindata.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-10-05T07:45:53.000Z","updated_at":"2022-08-08T06:16:44.000Z","dependencies_parsed_at":"2022-09-21T04:10:51.825Z","dependency_job_id":null,"html_url":"https://github.com/getindata/ververica-platform-flink-workshop","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/getindata%2Fververica-platform-flink-workshop","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/getindata%2Fververica-platform-flink-workshop/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/getindata%2Fververica-platform-flink-workshop/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/getindata%2Fververica-platform-flink-workshop/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/getindata","download_url":"https://codeload.github.com/getindata/ververica-platform-flink-workshop/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244066192,"owners_count":20392407,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-23T23:53:29.896Z","updated_at":"2026-01-04T05:04:00.201Z","avatar_url":"https://github.com/getindata.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Ververica Platform \u0026 Flink Workshop\n\n## Overview\n\nThis repo is designed to be used as the base for demo for any workshop with Ververica Platform and Flink.\n\n### TODO \n\n- [] Flink jobs\n  - [ ] Easily customizable templates\n  - [ ] Examples of jobs upgrade\n\n- [] Scripts\n  - [ ] Setup Minikube locally\n  - [ ] Setup all components\n  - [ ] Manage jobs by passing arguments to script\n\n## Setup environment locally\n\n### Requirements\n\n- Installed a hypervisor (like Virtualbox) or a container runtime (like Docker)\n- At least 8 GB RAM and 4 CPU cores\n\n### Tutorial\n\n1. Install Minikube.\n\n**MacOS**\n```\nbrew install minikube\n```\n\n**Windows**\n\nChocolatey:\n```\nchoco install minikube\n```\n\nPowerShell:\n```\nNew-Item -Path 'c:\\' -Name 'minikube' -ItemType Directory -Force\nInvoke-WebRequest -OutFile 'c:\\minikube\\minikube.exe' -Uri 'https://github.com/kubernetes/minikube/releases/latest/download/minikube-windows-amd64.exe' -UseBasicParsing\n\nRUN AS ADMINISTRATOR\n\n$oldPath = [Environment]::GetEnvironmentVariable('Path', [EnvironmentVariableTarget]::Machine)\nif ($oldPath.Split(';') -inotcontains 'C:\\minikube'){ `\n  [Environment]::SetEnvironmentVariable('Path', $('{0};C:\\minikube' -f $oldPath), [EnvironmentVariableTarget]::Machine) `\n}\n```\n\n**Linux**\n```\ncurl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64\nsudo install minikube-linux-amd64 /usr/local/bin/minikube\n```\n\n2. Start Minikube from terminal.\n```\nminikube start --memory=8G --cpus=4\n```\n\n3. Check if the Minikube cluster started correctly.\n\n4. Install Helm\n\n**MacOS**\n```\nbrew install helm\n```\n\n**Windows**\n\nChocolatey\n```\nchoco install helm\n```\n\nPowerShell\n```\nNew-Item -Path 'c:\\' -Name 'minikube' -ItemType Directory -Force\nInvoke-WebRequest -OutFile 'c:\\minikube\\minikube.exe' -Uri 'https://github.com/kubernetes/minikube/releases/latest/download/minikube-windows-amd64.exe' -UseBasicParsing\n\nRUN AS ADMINISTRATOR\n\n$oldPath = [Environment]::GetEnvironmentVariable('Path', [EnvironmentVariableTarget]::Machine)\nif ($oldPath.Split(';') -inotcontains 'C:\\minikube'){ `\n  [Environment]::SetEnvironmentVariable('Path', $('{0};C:\\minikube' -f $oldPath), [EnvironmentVariableTarget]::Machine) `\n}\n```\n\n**Linux**\n```\ncurl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64\nsudo install minikube-linux-amd64 /usr/local/bin/minikube\n```\n\n5. Create namespaces for Ververica.\n```\n   kubectl create namespace vvp\n   kubectl create namespace vvp-jobs\n```\n\n6. Install MinIO as our local blob storage.\n``` \nhelm repo add stable https://charts.helm.sh/stable\nhelm --namespace vvp \\\n    install minio stable/minio \\\n    --values vvp-values/minio-values.yaml\n```\n\n7. Install Ververica Platform.\n``` \nhelm repo add ververica https://charts.ververica.com\nhelm --namespace vvp \\\n    install vvp ververica/ververica-platform \\\n    --values vvp-values/ververica-values.yaml\n```\n\n8. Forward port with Ververica Platform.\n``` \nkubectl -n vvp port-forward services/vvp-ververica-platform 8080:80\n```\n\n9. Create default deployment target that defines basic deployment target with the name of target namespace (we want to add it in the namespace called vvp-jobs.\n``` \ncurl localhost:8080/api/v1/namespaces/default/deployment-targets \\\n    -X POST \\\n    -H \"Content-Type: application/yaml\" \\\n    -H \"Accept: application/yaml\" \\\n    --data-binary @deployments/deployment-target.yaml\n```\n\n10. Create  first Flink job in Ververica (Top Speed Windowing).\n```\n    curl localhost:8080/api/v1/namespaces/default/deployments \\\n        -X POST \\\n        -H \"Content-Type: application/yaml\" \\\n        -H \"Accept: application/yaml\" \\\n        --data-binary @deployments/top-speed/deployment.yaml\n```\n11. Update job by using the following command.\n```\n    curl localhost:8080/api/v1/namespaces/default/deployments/top-speed-windowing \\\n        -X PATCH \\\n        -H \"Content-Type: application/yaml\" \\\n        -H \"Accept: application/yaml\" \\\n        --data-binary @deployments/top-speed/deployment.yaml\n```\n\n### Monitoring stack\n\n1. Get Helm chart from Grafana to set up Grafana, Prometheus, Grafana, Loki; and then create a namespace.\n\n```shell script\nhelm repo add grafana https://grafana.github.io/helm-charts\nhelm repo update\nkubectl create ns monitoring\n```\n\n2. Install the Helm chart\n\n```shell script\nhelm -n monitoring upgrade --install loki grafana/loki-stack  --set grafana.enabled=true,prometheus.enabled=true,prometheus.alertmanager.persistentVolume.enabled=false,prometheus.server.persistentVolume.enabled=false\n```\n\n3. Check if all pods are up and running.\n\n```shell script\nkubectl -n monitoring get po\n```\n\n4. Go to Grafana and get the password to panel\n\n```shell script\nkubectl get secret -n monitoring loki-grafana -o jsonpath=\"{.data.admin-password}\" | base64 --decode ; echo\n```\n\n5. Forward the port and access Grafana.\n\n```shell script\nkubectl port-forward -n monitoring service/loki-grafana 3000:80\n```\n\n6. Login to 127.0.0.1:3000 with username: admin and copied password.\n\n## Credits\n\n[Ververica Platform](https://www.ververica.com/platform)\n[Flink SQL Cookbook](https://github.com/ververica/flink-sql-cookbook)\n[wuchong - Flink SQL Demo](https://github.com/wuchong/flink-sql-demo)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgetindata%2Fververica-platform-flink-workshop","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgetindata%2Fververica-platform-flink-workshop","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgetindata%2Fververica-platform-flink-workshop/lists"}