{"id":19163834,"url":"https://github.com/elbwalker/sgtm-tag-bigquery","last_synced_at":"2026-06-17T16:30:18.492Z","repository":{"id":110077802,"uuid":"573128532","full_name":"elbwalker/sgtm-tag-bigquery","owner":"elbwalker","description":null,"archived":false,"fork":false,"pushed_at":"2024-01-23T17:45:44.000Z","size":46,"stargazers_count":6,"open_issues_count":0,"forks_count":1,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-01-03T21:46:52.059Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Smarty","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/elbwalker.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-12-01T19:00:49.000Z","updated_at":"2024-01-12T18:50:58.000Z","dependencies_parsed_at":"2025-01-03T21:46:23.067Z","dependency_job_id":"6e91eee3-63f7-48d5-824e-3832d46e810c","html_url":"https://github.com/elbwalker/sgtm-tag-bigquery","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elbwalker%2Fsgtm-tag-bigquery","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elbwalker%2Fsgtm-tag-bigquery/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elbwalker%2Fsgtm-tag-bigquery/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elbwalker%2Fsgtm-tag-bigquery/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/elbwalker","download_url":"https://codeload.github.com/elbwalker/sgtm-tag-bigquery/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240245887,"owners_count":19771029,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T09:16:51.560Z","updated_at":"2026-06-17T16:30:18.460Z","avatar_url":"https://github.com/elbwalker.png","language":"Smarty","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"left\"\u003e\n  \u003ca href=\"https://elbwalker.com\"\u003e\n    \u003cimg title=\"elbwalker\" src='https://www.elbwalker.com/elbwalker.png' width=\"300px\"/\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n# elbwalker to BigQuery Server Side Google Tag Manager Custom Tag Template\nThe sGTM BigQuery tag lets you send walker.js events from server-side Google Tag Manager to BigQuery.\n\n[![Beta Status](https://img.shields.io/badge/Status-Beta-yellow.svg)](https://shields.io/)\n\n# Important Note\nThis is a preview release of the *walker.js to BigQuery* tag template for server-side GTM. It uses the `x-elb-event` key in the server-side event model that usually gets populated by handling incoming walker.js events with the [walker.js Custom Client Template for server-side GTM](https://github.com/elbwalker/sgtm-client-template). \n\n## Creating a BigQuery table for walker.js events\nThe BigQuery table that this tag template expects has a similar structure to all walker.js events - including entity, action, data, context, nested, version, timestamp, group, and everything else specified in the [Walker.js documentation](https://docs.elbwalker.com/). A `server_timestamp` is added as an alternative to the existing `timestamp` information from the browser. \n\nAdditionally, you can define a set of key/value pairs that get sent along with the walker.js event and get stored in a `additional_data` record. Values will be available as `string_value`, `float_value`, or `int_value`. You can use them to add a user-defined set of headers, cookies, or attributes of the request or event. \n\n### Schema\nCreate a new table in a new or existing dataset with the following schema that can be pasted as JSON: \n\n#### Schema Update 2024 (walkerOS 2.0.0 or older)\n```\n[\n    {\n        \"name\": \"timestamp\",\n        \"type\": \"INTEGER\",\n        \"mode\": \"REQUIRED\",\n        \"description\": \"walker event timestamp\"\n    },\n    {\n        \"name\": \"event\",\n        \"type\": \"STRING\",\n        \"mode\": \"REQUIRED\",\n        \"description\": \"Event name as entity and action\"\n    },\n    {\n        \"name\": \"data\",\n        \"type\": \"JSON\",\n        \"mode\": \"NULLABLE\",\n        \"description\": \"All entity properties\"\n    },\n    {\n        \"name\": \"context\",\n        \"type\": \"JSON\",\n        \"mode\": \"NULLABLE\",\n        \"description\": \"The context object\"\n    },\n    {\n        \"name\": \"globals\",\n        \"type\": \"JSON\",\n        \"mode\": \"NULLABLE\",\n        \"description\": \"All global context properties set\"\n    },\n    {\n        \"name\": \"custom\",\n        \"type\": \"JSON\",\n        \"mode\": \"NULLABLE\",\n        \"description\": \"The custom object\"\n    },\n    {\n        \"name\": \"user\",\n        \"type\": \"RECORD\",\n        \"mode\": \"NULLABLE\",\n        \"description\": \"Information about the user\",\n        \"fields\": [\n            {\n                \"name\": \"session\",\n                \"type\": \"STRING\",\n                \"mode\": \"NULLABLE\",\n                \"description\": \"session value (e.g. encrypted temporary fingerprint)\"\n            },\n            {\n                \"name\": \"device\",\n                \"type\": \"STRING\",\n                \"mode\": \"NULLABLE\",\n                \"description\": \"The user's cookie id\"\n            },\n            {\n                \"name\": \"id\",\n                \"type\": \"STRING\",\n                \"mode\": \"NULLABLE\",\n                \"description\": \"The user's hashed id\"\n            }\n        ]\n    },\n    {\n        \"name\": \"nested\",\n        \"type\": \"JSON\",\n        \"mode\": \"NULLABLE\",\n        \"description\": \"All nested entities within the mother entity\"\n    },\n    {\n        \"name\": \"consent\",\n        \"type\": \"JSON\",\n        \"mode\": \"NULLABLE\",\n        \"description\": \"The user's consent states for the event (analytics, marketing)\"\n    },\n    {\n        \"name\": \"id\",\n        \"type\": \"STRING\",\n        \"mode\": \"REQUIRED\",\n        \"description\": \"Identifier out of timestamp, group \u0026 count of the event\"\n    },\n    {\n        \"name\": \"entity\",\n        \"type\": \"STRING\",\n        \"mode\": \"REQUIRED\",\n        \"description\": \"Name of the entity\"\n    },\n    {\n        \"name\": \"action\",\n        \"type\": \"STRING\",\n        \"mode\": \"REQUIRED\",\n        \"description\": \"Name of the action\"\n    },\n    {\n        \"name\": \"trigger\",\n        \"type\": \"STRING\",\n        \"mode\": \"NULLABLE\",\n        \"description\": \"Name of the trigger that fired an event\"\n    },\n    {\n        \"name\": \"timing\",\n        \"type\": \"FLOAT\",\n        \"mode\": \"NULLABLE\",\n        \"description\": \"Seconds that have passed since initialization until the event occurs\"\n    },\n    {\n        \"name\": \"group\",\n        \"type\": \"STRING\",\n        \"mode\": \"REQUIRED\",\n        \"description\": \"Random group id for all events on a page\"\n    },\n    {\n        \"name\": \"count\",\n        \"type\": \"INTEGER\",\n        \"mode\": \"REQUIRED\",\n        \"description\": \"Incremental counter of the events on a page\"\n    },\n    {\n        \"name\": \"version\",\n        \"type\": \"RECORD\",\n        \"mode\": \"NULLABLE\",\n        \"description\": \"Used versions for event collection\",\n        \"fields\": [\n            {\n                \"name\": \"tagging\",\n                \"type\": \"STRING\",\n                \"mode\": \"REQUIRED\",\n                \"description\": \"The tagging configuration used on the client-side\"\n            },\n            {\n                \"name\": \"client\",\n                \"type\": \"STRING\",\n                \"mode\": \"REQUIRED\",\n                \"description\": \"The used walker.js version on the client-side\"\n            },\n            {\n                \"name\": \"server\",\n                \"type\": \"STRING\",\n                \"mode\": \"NULLABLE\",\n                \"description\": \"version of the sending tag on the server-side\"\n            }\n        ]\n    },\n    {\n        \"name\": \"source\",\n        \"type\": \"RECORD\",\n        \"mode\": \"NULLABLE\",\n        \"description\": \"Origins of the event\",\n        \"fields\": [\n            {\n                \"name\": \"type\",\n                \"type\": \"STRING\",\n                \"mode\": \"NULLABLE\",\n                \"description\": \"Source type of the event\"\n            },\n            {\n                \"name\": \"id\",\n                \"type\": \"STRING\",\n                \"mode\": \"NULLABLE\",\n                \"description\": \"Source id of the event's origin\"\n            },\n            {\n                \"name\": \"previous_id\",\n                \"type\": \"STRING\",\n                \"mode\": \"NULLABLE\",\n                \"description\": \"Previous source id of the event's origin\"\n            }\n        ]\n    },\n    {\n        \"name\": \"server_timestamp\",\n        \"type\": \"INTEGER\",\n        \"mode\": \"REQUIRED\",\n        \"description\": \"Timestamp when the sGTM processed the event in ms\"\n    },\n    {\n        \"name\": \"additional_data\",\n        \"type\": \"JSON\",\n        \"mode\": \"NULLABLE\",\n        \"description\": \"Optionally captured but relevant event information in addition\"\n    }\n]\n```\n\n#### Older Schema (still compatible with any incoming walker.js / walkerOS event)\n**Note**: There is an option to store additional data in the BQ table in two different ways:\n \n- store additional data as JSON\n- store additional data as RECORD\n\nThe default is JSON, since all other objects inside a walker.js event end up as a JSON field as well. If you want to use a record instead, change the field definition for `additional_data` as described below. \n\n```\n[\n    {\n        \"name\": \"timestamp\",\n        \"type\": \"INTEGER\",\n        \"mode\": \"REQUIRED\",\n        \"description\": \"walker event timestamp\"\n    },\n    {\n        \"name\": \"event\",\n        \"type\": \"STRING\",\n        \"mode\": \"REQUIRED\",\n        \"description\": \"Event name as entity and action\"\n    },\n    {\n        \"name\": \"data\",\n        \"type\": \"JSON\",\n        \"mode\": \"NULLABLE\",\n        \"description\": \"All entity properties\"\n    },\n    {\n        \"name\": \"context\",\n        \"type\": \"JSON\",\n        \"mode\": \"NULLABLE\",\n        \"description\": \"The context object\"\n    },\n    {\n        \"name\": \"globals\",\n        \"type\": \"JSON\",\n        \"mode\": \"NULLABLE\",\n        \"description\": \"All global context properties set\"\n    },\n    {\n        \"name\": \"user\",\n        \"type\": \"RECORD\",\n        \"mode\": \"NULLABLE\",\n        \"description\": \"Information about the user\",\n        \"fields\": [\n            {\n                \"name\": \"session\",\n                \"type\": \"STRING\",\n                \"mode\": \"NULLABLE\",\n                \"description\": \"session value (e.g. encrypted temporary fingerprint)\"\n            },\n            {\n                \"name\": \"device\",\n                \"type\": \"STRING\",\n                \"mode\": \"NULLABLE\",\n                \"description\": \"The user's cookie id\"\n            },\n            {\n                \"name\": \"id\",\n                \"type\": \"STRING\",\n                \"mode\": \"NULLABLE\",\n                \"description\": \"The user's hashed id\"\n            }\n        ]\n    },\n    {\n        \"name\": \"nested\",\n        \"type\": \"JSON\",\n        \"mode\": \"NULLABLE\",\n        \"description\": \"All nested entities within the mother entity\"\n    },\n    {\n        \"name\": \"consent\",\n        \"type\": \"JSON\",\n        \"mode\": \"NULLABLE\",\n        \"description\": \"The user's consent states for the event (analytics, marketing)\"\n    },\n    {\n        \"name\": \"id\",\n        \"type\": \"STRING\",\n        \"mode\": \"REQUIRED\",\n        \"description\": \"Identifier out of timestamp, group \u0026 count of the event\"\n    },\n    {\n        \"name\": \"entity\",\n        \"type\": \"STRING\",\n        \"mode\": \"REQUIRED\",\n        \"description\": \"Name of the entity\"\n    },\n    {\n        \"name\": \"action\",\n        \"type\": \"STRING\",\n        \"mode\": \"REQUIRED\",\n        \"description\": \"Name of the action\"\n    },\n    {\n        \"name\": \"trigger\",\n        \"type\": \"STRING\",\n        \"mode\": \"NULLABLE\",\n        \"description\": \"Name of the trigger that fired an event\"\n    },\n    {\n        \"name\": \"timing\",\n        \"type\": \"FLOAT\",\n        \"mode\": \"NULLABLE\",\n        \"description\": \"Seconds that have passed since initialization until the event occurs\"\n    },\n    {\n        \"name\": \"group\",\n        \"type\": \"STRING\",\n        \"mode\": \"REQUIRED\",\n        \"description\": \"Random group id for all events on a page\"\n    },\n    {\n        \"name\": \"count\",\n        \"type\": \"INTEGER\",\n        \"mode\": \"REQUIRED\",\n        \"description\": \"Incremental counter of the events on a page\"\n    },\n    {\n        \"name\": \"version\",\n        \"type\": \"RECORD\",\n        \"mode\": \"NULLABLE\",\n        \"description\": \"Used versions for event collection\",\n        \"fields\": [\n            {\n                \"name\": \"config\",\n                \"type\": \"STRING\",\n                \"mode\": \"REQUIRED\",\n                \"description\": \"The tagging configuration used on the client-side\"\n            },\n            {\n                \"name\": \"walker\",\n                \"type\": \"STRING\",\n                \"mode\": \"REQUIRED\",\n                \"description\": \"The used walker.js version on the client-side\"\n            },\n            {\n                \"name\": \"server\",\n                \"type\": \"STRING\",\n                \"mode\": \"NULLABLE\",\n                \"description\": \"version of the sending tag on the server-side\"\n            }\n        ]\n    },\n    {\n        \"name\": \"source\",\n        \"type\": \"RECORD\",\n        \"mode\": \"NULLABLE\",\n        \"description\": \"Origins of the event\",\n        \"fields\": [\n            {\n                \"name\": \"type\",\n                \"type\": \"STRING\",\n                \"mode\": \"NULLABLE\",\n                \"description\": \"Source type of the event\"\n            },\n            {\n                \"name\": \"id\",\n                \"type\": \"STRING\",\n                \"mode\": \"NULLABLE\",\n                \"description\": \"Source id of the event's origin\"\n            },\n            {\n                \"name\": \"previous_id\",\n                \"type\": \"STRING\",\n                \"mode\": \"NULLABLE\",\n                \"description\": \"Previous source id of the event's origin\"\n            }\n        ]\n    },\n    {\n        \"name\": \"server_timestamp\",\n        \"type\": \"INTEGER\",\n        \"mode\": \"REQUIRED\",\n        \"description\": \"Timestamp when the sGTM processed the event in ms\"\n    },\n    {\n        \"name\": \"additional_data\",\n        \"type\": \"JSON\",\n        \"mode\": \"NULLABLE\",\n        \"description\": \"Optionally captured but relevant event information in addition\"\n    }\n]\n\n```\n\nIf you prefer a RECORD over JSON, swap out the definition for `additional_data` and use this block instead: \n\n```\n  {\n    \"name\": \"additional_data\",\n    \"mode\": \"REPEATED\",\n    \"type\": \"RECORD\",\n    \"description\": \"Optionally captured but relevant event information in addition\",\n    \"fields\": [\n      {\n        \"name\": \"key\",\n        \"mode\": \"NULLABLE\",\n        \"type\": \"STRING\",\n        \"fields\": []\n      },\n      {\n        \"name\": \"value\",\n        \"mode\": \"NULLABLE\",\n        \"type\": \"RECORD\",\n        \"fields\": [\n          {\n            \"name\": \"string_value\",\n            \"mode\": \"NULLABLE\",\n            \"type\": \"STRING\"\n          },\n          {\n            \"name\": \"float_value\",\n            \"mode\": \"NULLABLE\",\n            \"type\": \"STRING\"\n          },\n          {\n            \"name\": \"int_value\",\n            \"mode\": \"NULLABLE\",\n            \"type\": \"STRING\"\n          }\n        ]\n      }\n    ]\n  }\n\n```\n\nMake sure to select *RECORD* as *Format / Type* for all additional data\n\n### Partitioning\nDepending on your use case you should consider partitioning. The example SQL queries can be used with either version and have to be adjusted to use the proper project/table name either way. If you prefer to use a timestamp instead of ingestion time, prefer `server_timestamp` over the browser `timestamp` to set a common time base for all events.  \n\n## Using The Tag Template\n- add the template to your sGTM in the *templates\" section as a new tag template by creating a new template and importing this `.tpl` file (use the three-dot-menu in GTM Template Editor). \n\n- Add a new tag to sGTM with this template and enter **project id** (if using a different project), **database id**, and **table id**. #\n\n- Incoming walker.js events originate from the `x-elb-event` key in the event model if you keep the *Event Source* setting on *Automatic*. In case a different type of client is used or enhanced event processing inside a custom variable before sending data to BigQuery is required, use a *Custom Variable* as source.  \n\n- The tag can optionally add additional parameters (like IP address, user agent, headers, cookie values, or others) to the BigQuery table in a `additional_data` record. \n\n![image](https://user-images.githubusercontent.com/15323700/205728127-5294143f-33df-498e-967e-13670fea0c28.png)\n\n- Send events to your endpoint like demonstrated in the [example code from the sGTM client repository](https://github.com/elbwalker/sgtm-client-template/tree/main/example) or use a client-side GTM and install the [walker.js Event Custom Tag Template](https://github.com/elbwalker/sgtm-client-template). Trigger your walker.js to BigQuery tag whenever a walker.js event arrives. \n\n## BigQuery Results\nAll Events will get pushed to the new table in BigQuery. Data like `globals`, `data`, `context`, `nested` and other objects will be stored in JSON fields and can be accessed in different ways. You can use `JSON_EXTRACT` as well as simply access nested information in a JSON field by using dot notation. The following query shows how to extract selected fields using both methods and creating a simple \"GA-like\" list of all page view events. \n\n```\nSELECT\n  timestamp AS event_timestamp,\n  \"page_view\" AS event_name,\n  JSON_VALUE(data.domain) AS page_hostname,\n  JSON_VALUE(data[\"title\"]) AS page_title,\n  /* optionally use JSON_EXTRACT instead of dot notation */\n  JSON_VALUE(JSON_EXTRACT(data, \"$.id\")) AS page_path,\n  user.device AS client_id, \n  user.session AS ga_session_id,\nFROM\n  `your-project.your-database.your-walker_events-table`\nWHERE\n  event = \"page view\"\nLIMIT\n  100\n  \n```\n\nThe results should look similar to this: \n\n![image](https://user-images.githubusercontent.com/15323700/205689109-7b69dbd1-32b5-4bab-b6a2-5200b1cc5812.png)\n\n## Release Notes\n*2024-01-23*\n- changed default BQ scheme to fit events from walkerOS / walker.js version 2.0.0\n- adjustments for sending version information to fit 2.x (still compatible with older versions)\n- processing for `custom` object from 2.x\n- note: template is compatible with events from 2.0.0 or older along with any BQ scheme describes above (new and old)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felbwalker%2Fsgtm-tag-bigquery","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Felbwalker%2Fsgtm-tag-bigquery","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felbwalker%2Fsgtm-tag-bigquery/lists"}