https://github.com/hayatoy/dataflow-tutorial

Cloud Dataflow Tutorial for Beginners
https://github.com/hayatoy/dataflow-tutorial

bigquery cloud-dataflow google-cloud-platform google-cloud-storage

Last synced: 2 months ago
JSON representation

Cloud Dataflow Tutorial for Beginners

• Host: GitHub
• URL: https://github.com/hayatoy/dataflow-tutorial
• Owner: hayatoy
• License: apache-2.0
• Created: 2017-05-31T13:01:46.000Z (over 7 years ago)
• Default Branch: master
• Last Pushed: 2022-03-11T04:36:44.000Z (almost 3 years ago)
• Last Synced: 2023-02-26T20:51:36.154Z (almost 2 years ago)
• Topics: bigquery, cloud-dataflow, google-cloud-platform, google-cloud-storage
• Language: Python
• Homepage:
• Size: 14.6 KB
• Stars: 24
• Watchers: 1
• Forks: 9
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Cloud Dataflow (Python!) Tutorial for Beginners

## How to use

- Create Billing account on Google Cloud Platform

- Enable Dataflow API

- Open Datalab

  

Recommended Datalab settings;  

`datalab create dftutorial --disk-size-gb 10 --no-create-repository --no-backups`

## If you want to use Jupyter Notebook..

- Install Dataflow Python SDK  

`pip install google-cloud-dataflow`

- Set up default credential. just run the following command.  

`gcloud auth application-default login`