Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/prologuetech/laravel-big
Google BigQuery for Laravel
https://github.com/prologuetech/laravel-big
bigquery google google-bigquery laravel laravel-5-package php
Last synced: 2 months ago
JSON representation
Google BigQuery for Laravel
- Host: GitHub
- URL: https://github.com/prologuetech/laravel-big
- Owner: prologuetech
- License: mit
- Created: 2017-06-26T20:01:46.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2018-08-16T18:30:22.000Z (over 6 years ago)
- Last Synced: 2024-11-08T11:15:35.131Z (3 months ago)
- Topics: bigquery, google, google-bigquery, laravel, laravel-5-package, php
- Language: PHP
- Homepage: http://prologuetechnology.com
- Size: 44.9 KB
- Stars: 15
- Watchers: 6
- Forks: 11
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# Google BigQuery for Laravel
This package aims to wrap laravel functionality around Google's BigQuery.
## Install
Via Composer
``` bash
$ composer require prologuetech/big
```
## Setup
Publish our config file into your application:
``` bash
php artisan vendor:publish --provider="Prologuetech\Big\BigServiceProvider"
```
You should have a `config/prologue-big.php` file to configure defaults.
### Laravel 5.4.x
Older versions of Laravel require you to add our big service provider to your application providers array in `config/app.php`:
``` php
Prologuetech\Big\BigServiceProvider::class,
```
You now have access to a familiar laravel experience, enjoy!
## Google Authentication
The Google SDK supports Application Default Credentials (ADC) and thus this package does as well. You may leave your `auth_file` field inside of your config file `null` to use ADC. Credentials fetcher is not currently supported but may be added in the future.
For more information see the [adc docs](https://cloud.google.com/docs/authentication/production#auth-cloud-implicit-php).
## How to use
### Configuration
By default we use the following global config options with BigQuery.
```php
$this->options = [
'useLegacySql' => false,
'useQueryCache' => false,
];
```
### Tables
When creating tables in BQ we automatically flip a Eloquent model schema for you. Let's cover an example of archiving data
from our events table into BQ using laravel's chunk method.
```php
$datasetId = 'test';
$tableId = 'events';
// Create our BQ helper
$big = new Big();
// Create table, we will pass in a mocked model to mutate into BQ schema
// Note: create table will only make a new table if it does not exist
/** @var Google\Cloud\BigQuery\Table $table */
$table = $big->createFromModel($datasetId, $tableId, new Event());
// Let's stream our events into BQ in large chunks
// Note: notArchived() is a simple scope, use whatever scopes you have on your model
Event::notArchived()->chunk(1000, function ($events) use ($big, $table) {
// Prepare our rows
$rows = $big->prepareData($events);
// Stream into BQ, you may also pass in any options with a 3rd param.
// Note: By default we use: 'ignoreUnknownValues' => true
$big->insert($table, $rows);
// Get our current id's
/** @var Illuminate\Support\Collection $events */
$ids = $events->pluck('id')->toArray();
// Update these event's as processed
Event::whereIn('id', $ids)->update([
'system_processed' => 1
]);
});
```
That's it! You now have a replica of your events table in BigQuery, enjoy!
### Queries
Instantiating ```Big``` will automatically setup a Google ServiceBuilder and give us direct access to ```BigQuery``` through
our internals via ```$big->query```. However there are many helpers built into Big that make interacting with BigQuery a
piece of cake (or a tasty carrot if you're into that kind of thing).
For example when running a query on BigQuery we must use the reload method in a loop to poll results. Big comes with a
useful method ```run``` so all you need to do is this:
``` php
$query = 'SELECT count(id) FROM test.events';
$big = new Big();
$results = $big->run($query);
```
When using ```run``` we automatically poll BigQuery and return all results as a laravel collection object for you so you
can enjoy your results as a refreshing cup of Laravel.
## Change log
Please see [CHANGELOG](CHANGELOG.md) for more information on what has changed recently.
## License
The MIT License (MIT). Please see [License File](LICENSE.md) for more information.