https://github.com/zkan/running-bigquery-query-from-airflow-using-bigqueryexecuteoperator

Running BigQuery Query from Airflow using BigQueryExecuteOperator
https://github.com/zkan/running-bigquery-query-from-airflow-using-bigqueryexecuteoperator

airflow bigquery sql

Last synced: about 1 month ago
JSON representation

Running BigQuery Query from Airflow using BigQueryExecuteOperator

• Host: GitHub
• URL: https://github.com/zkan/running-bigquery-query-from-airflow-using-bigqueryexecuteoperator
• Owner: zkan
• License: mit
• Created: 2021-07-30T13:42:19.000Z (almost 5 years ago)
• Default Branch: main
• Last Pushed: 2021-07-30T15:09:11.000Z (almost 5 years ago)
• Last Synced: 2025-02-12T01:40:21.303Z (over 1 year ago)
• Topics: airflow, bigquery, sql
• Language: Python
• Homepage:
• Size: 16.6 KB
• Stars: 1
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# Running BigQuery Query from Airflow using BigQueryExecuteOperator

```py

from airflow.providers.google.cloud.operators.bigquery import BigQueryExecuteQueryOperator

find_top_five_coffee_brands = BigQueryExecuteQueryOperator(

    task_id="find_top_five_coffee_brands",

    sql="""

    SELECT

      Brand,

      COUNT(Brand) AS BrandCount

    FROM

      `dataength.personal.me_and_coffee`

    GROUP BY

      Brand

    ORDER BY

      BrandCount DESC

    LIMIT 5

    """,

    destination_dataset_table=f"dataength.personal.top_five_coffee_brands",

    write_disposition="WRITE_TRUNCATE",

    gcp_conn_id="my_bigquery_connection",

    use_legacy_sql=False,

)

```