Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/bankyadam/not-so-bigquery
An emulator for the Google BigQuery, that can be run locally, backed by PostgreSQL.
https://github.com/bankyadam/not-so-bigquery
bigquery development devtool emulator sql
Last synced: about 1 month ago
JSON representation
An emulator for the Google BigQuery, that can be run locally, backed by PostgreSQL.
- Host: GitHub
- URL: https://github.com/bankyadam/not-so-bigquery
- Owner: bankyadam
- License: mit
- Created: 2020-08-21T20:36:22.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2023-02-18T23:38:04.000Z (almost 2 years ago)
- Last Synced: 2023-04-13T20:31:06.923Z (over 1 year ago)
- Topics: bigquery, development, devtool, emulator, sql
- Language: TypeScript
- Homepage:
- Size: 1.47 MB
- Stars: 22
- Watchers: 3
- Forks: 2
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Not-So-BigQuery
An emulator for Google BigQuery, that can be run locally, backed by PostgreSQL.
## Compatibility
### API Endpoints
More info [here](docs/API_SUPPORT.md).
### Functions
More info [here](docs/FUNCTION_SUPPORT.md).
### SQL Compatibility
Since BigQuery uses its own SQL implementation, there can be and there are functionalities that will not work. To
achieve most of the features that BigQuery provides, Not-So-BigQuery uses its own query parser to translate to a
PostgreSQL-compatible query.
#### Supported BigQuery syntax
Link to the full query statement syntax: https://cloud.google.com/bigquery/docs/reference/standard-sql/query-syntax
```
query_statement:
query_expr
query_expr:
[ WITH with_query_name AS ( query_expr ) [, ...] ]
select | ( query_expr )
[ ORDER BY expression [{ ASC | DESC }] [, ...] ]
[ LIMIT count [ OFFSET skip_rows ] ]
select:
SELECT [{ ALL | DISTINCT }]
{ [ expression. ]*
| expression [ [ AS ] alias ] } [, ...]
[ FROM from_item [, ...] ]
[ WHERE expression ]
[ GROUP BY expression [, ...] ]
[ HAVING expression ]
from_item: {
table_name [ [ AS ] alias ]
join |
( query_expr ) [ [ AS ] alias ] |
with_query_name [ [ AS ] alias ]
}
join:
from_item [ join_type ] JOIN from_item
[ ON expression | USING ( join_column [, ...] ) ]
join_type:
{ INNER | CROSS | FULL [OUTER] | LEFT [OUTER] | RIGHT [OUTER] }
```
Where the `expression` is the following:
```
expression:
literal_value |
[ [ [ [ project_name . ] dataset_name . ] table_name . ] column_name ] |
unary_operator expression |
expression binary_operator expression |
function_name ( [ expression [, ...] | * ] ) |
( expression [, ...] ) |
CAST ( expression AS type_name ) |
EXTRACT ( part FROM date_expression ) |
expression IS [ NOT ] NULL |
expression IS [ NOT ] expression |
expression [ NOT ] IN ( { expression | select } )
literal_value:
numeric_literal |
string_literal |
NULL | TRUE | FALSE
unary_operator:
NOT
binary_operator:
AND | OR |
< | <= | > | >= | = | <> | !=
```
## Usage from source
### To run the app
```bash
docker-compose up app
```
This will start Not-So-BigQuery in a container, and expose port 5443 on your host.
After this, you can configure your library to use `http://localhost:5443/` as the API endpoint to reach BigQuery.
### NodeJS
```javascript
const bq = new BigQuery({ apiEndpoint: 'localhost:5443' });
```
### Ruby
```ruby
bigquery = Google::Cloud::Bigquery.new(endpoint: 'http://localhost:5443/')
```
### To clear the storage
```bash
docker-compose rm -fs postgres && docker volume rm -f not-so-bigquery_db-data
```
## Contributions
Contributions greatly appreciated.