{"id":24088002,"url":"https://github.com/tinybirdco/gcs-cloud-function","last_synced_at":"2025-07-15T05:46:35.991Z","repository":{"id":113972327,"uuid":"350765536","full_name":"tinybirdco/gcs-cloud-function","owner":"tinybirdco","description":"Guide and scripts to automate csv import to Tinybird from Google Cloud Storage using cloud functions","archived":false,"fork":false,"pushed_at":"2021-11-11T17:04:40.000Z","size":5,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-02-27T05:25:32.664Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tinybirdco.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-03-23T15:36:21.000Z","updated_at":"2021-11-11T17:04:43.000Z","dependencies_parsed_at":null,"dependency_job_id":"d9adcb37-6db8-46f8-ae4b-77392dcd302f","html_url":"https://github.com/tinybirdco/gcs-cloud-function","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/tinybirdco/gcs-cloud-function","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinybirdco%2Fgcs-cloud-function","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinybirdco%2Fgcs-cloud-function/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinybirdco%2Fgcs-cloud-function/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinybirdco%2Fgcs-cloud-function/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tinybirdco","download_url":"https://codeload.github.com/tinybirdco/gcs-cloud-function/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinybirdco%2Fgcs-cloud-function/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265411684,"owners_count":23760653,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-10T03:56:52.940Z","updated_at":"2025-07-15T05:46:35.983Z","avatar_url":"https://github.com/tinybirdco.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# GCS + cloud function -\u003e Automatically ingesting to Tinybird from Google Cloud Storage\n\nThis repository contains a instructions to configure a Google Cloud function that is automatically triggered each time a new CSV file is uploaded to a given Google Cloud Storage bucket.\n\nThe Google Cloud function function appends the contents of the CSV file to a Tinybird Data Source.\n\n## How to deploy the Cloud Function\n\nInstall the `gcloud` command line tool and run these commands:\n\n```sh\n# edit the .env.yaml file to set your Tinybird token available from https://ui.tinybird.co/tokens\ncp .env.yaml.sample .env.yaml\n\n# set some environment variables before deploying\nPROJECT_NAME=\u003cthe_GCP_project_name\u003e\nSERVICE_ACCOUNT_NAME=\u003cservice_account_name@project_name.iam.gserviceaccount.com\u003e\nBUCKET_NAME=\u003cbucket_name\u003e\nREGION=\u003cregion\u003e\nTB_FUNCTION_NAME=\u003cname_of_the_function\u003e\n\n# grant permissions to deploy the cloud function and read from storage to the service account\ngcloud projects add-iam-policy-binding $PROJECT_NAME --member serviceAccount:$SERVICE_ACCOUNT_NAME --role roles/storage.admin\ngcloud projects add-iam-policy-binding $PROJECT_NAME --member serviceAccount:$SERVICE_ACCOUNT_NAME --role roles/iam.serviceAccountTokenCreator\ngcloud projects add-iam-policy-binding $PROJECT_NAME --member serviceAccount:$SERVICE_ACCOUNT_NAME --role roles/editor\n\n# deploy the cloud function\ngcloud functions deploy $TB_FUNCTION_NAME \\\n--runtime python38 \\\n--trigger-resource $BUCKET_NAME \\\n--trigger-event google.storage.object.finalize \\\n--region $REGION \\\n--env-vars-file .env.yaml \\\n--service-account $SERVICE_ACCOUNT_NAME\n```\n\nOnce finished, go to the Google Cloud console and check the cloud function is there and the `TB_HOST`, `TB_TOKEN`, and `FILE_REGEXP` variables required to run the script are available.\n\n![](output.gif)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftinybirdco%2Fgcs-cloud-function","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftinybirdco%2Fgcs-cloud-function","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftinybirdco%2Fgcs-cloud-function/lists"}