https://github.com/dataform-co/dataform-bq-audit-logs
https://github.com/dataform-co/dataform-bq-audit-logs
Last synced: about 2 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/dataform-co/dataform-bq-audit-logs
- Owner: dataform-co
- License: mit
- Created: 2020-01-17T15:55:13.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2023-07-17T09:04:09.000Z (almost 3 years ago)
- Last Synced: 2025-01-12T05:11:23.620Z (over 1 year ago)
- Language: JavaScript
- Size: 28.3 KB
- Stars: 1
- Watchers: 3
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
**** NOTE: This package was created before the "Jobs" information schema tables were available in BigQuery. ****
**** We recommended that you use the information schema tables instead of this package ****
**** Learn more: https://cloud.google.com/bigquery/docs/information-schema-jobs ****
This package creates a simple dataset to aid analysis of BigQuery usage logs.
To set up BigQuery audit data logging, follow the instructions [here](https://dataform.co/blog/exporting-bigquery-usage-logs).
## Supported warehouses
- BigQuery
## Installation
Add the package to your `package.json` file in your Dataform project. You can find the most up to package version on the [releases page](https://github.com/dataform-co/dataform-bq-audit-logs/releases).
## Configure the package
Create a new JS file in your `definitions/` folder and create the BigQuery table with the following snippet:
```js
const bigQueryLogs = require("dataform-bigquery-logs");
const bigQueryLogsModels = bigQueryLogs({
logsSchema: "stackdriver_logs",
defaultConfig: {
tags: ["bq_audit_logs"],
type: "view"
},
});
```
For more advanced uses cases, see the [example.js](https://github.com/dataform-co/dataform-bq-audit-logs/blob/master/definitions/example.js).
## Data models
This output of this package is the following data model (configurable as tables or views).
### `bigquery_compute_logs`
One row for every query log sent from BigQuery to stackdriver. The schema of this table is significantly simplified as compared to the raw logs table!
