https://github.com/memsjava/bigquery-helper

A helper package for Google BigQuery operations
https://github.com/memsjava/bigquery-helper

bigquery google pandas-dataframe

Last synced: about 1 month ago
JSON representation

A helper package for Google BigQuery operations

• Host: GitHub
• URL: https://github.com/memsjava/bigquery-helper
• Owner: memsjava
• License: mit
• Created: 2024-08-29T12:01:54.000Z (3 months ago)
• Default Branch: main
• Last Pushed: 2024-08-29T12:27:01.000Z (3 months ago)
• Last Synced: 2024-09-30T15:05:55.535Z (about 2 months ago)
• Topics: bigquery, google, pandas-dataframe
• Language: Python
• Homepage: https://pypi.org/project/bigquery-helper/
• Size: 7.81 KB
• Stars: 7
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# BigQuery Helper

BigQuery Helper is a Python package that simplifies working with Google BigQuery. It provides an easy-to-use interface for common BigQuery operations such as executing queries, inserting rows, and working with DataFrames.

## Installation

You can install BigQuery Helper using pip:

```

pip install bigquery-helper

```

## Usage

To use BigQuery Helper, you'll need a Google Cloud project and credentials. Here's how to get started:

1. Set up a Google Cloud project and enable the BigQuery API.

2. Create a service account and download the JSON key file.

3. Set the `GOOGLE_APPLICATION_CREDENTIALS` environment variable to the path of your JSON key file:

   ```

   export GOOGLE_APPLICATION_CREDENTIALS="/path/to/your/keyfile.json"

   ```

   Alternatively, you can pass the credentials directly when initializing the BigQueryHelper.

Here's a quick example of how to use BigQuery Helper:

```python

from bigquery_helper import BigQueryHelper

# Initialize BigQueryHelper

bq_helper = BigQueryHelper(project_id="your-project-id", credentials="/path/to/your/keyfile.json")

# Execute a query and get results as a DataFrame

query = "SELECT * FROM `your_dataset.your_table` LIMIT 10"

df = bq_helper.query_to_dataframe(query)

print(df)

# Insert rows into a table

rows_to_insert = [

    {"column1": "value1", "column2": 123},

    {"column1": "value2", "column2": 456}

]

bq_helper.insert_rows("your_dataset.your_table", rows_to_insert)

# Upload a DataFrame to BigQuery

import pandas as pd

df_to_upload = pd.DataFrame({"column1": ["a", "b"], "column2": [1, 2]})

bq_helper.dataframe_to_table(df_to_upload, "your_dataset.new_table", if_exists="replace")

# Update a column in a table

bq_helper.update_column("your_dataset.your_table", "id_column", "status_column", [1, 2, 3])

# Get schema of a table

schema = bq_helper.get_schema("your_dataset.your_table")

print(schema)

```

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

## License

This project is licensed under the MIT License.