{"id":22793422,"url":"https://github.com/fivetran/dbt_fivetran_log","last_synced_at":"2026-04-03T00:25:54.267Z","repository":{"id":38303594,"uuid":"274959352","full_name":"fivetran/dbt_fivetran_log","owner":"fivetran","description":"Data models for Fivetran's internal log connector built using dbt.","archived":false,"fork":false,"pushed_at":"2024-05-17T13:43:20.000Z","size":2907,"stargazers_count":28,"open_issues_count":13,"forks_count":23,"subscribers_count":39,"default_branch":"main","last_synced_at":"2024-05-17T15:11:36.077Z","etag":null,"topics":["dbt","dbt-packages","fivetran"],"latest_commit_sha":null,"homepage":"https://fivetran.github.io/dbt_fivetran_log/","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fivetran.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-06-25T16:04:33.000Z","updated_at":"2024-05-29T23:38:40.631Z","dependencies_parsed_at":"2023-10-11T19:28:42.707Z","dependency_job_id":"efc3214c-9959-49d2-96ad-8abc059c27e6","html_url":"https://github.com/fivetran/dbt_fivetran_log","commit_stats":null,"previous_names":[],"tags_count":39,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fivetran%2Fdbt_fivetran_log","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fivetran%2Fdbt_fivetran_log/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fivetran%2Fdbt_fivetran_log/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fivetran%2Fdbt_fivetran_log/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fivetran","download_url":"https://codeload.github.com/fivetran/dbt_fivetran_log/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249266646,"owners_count":21240792,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dbt","dbt-packages","fivetran"],"created_at":"2024-12-12T03:19:50.536Z","updated_at":"2026-04-03T00:25:54.246Z","avatar_url":"https://github.com/fivetran.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003c!--section=\"fivetran-log_transformation_model\"--\u003e\n# Fivetran Platform dbt Package\n\nThis dbt package transforms data from the Fivetran Platform connector into analytics-ready tables.\n\n## Resources\n\n- Number of materialized models¹: 19\n- Connector documentation\n  - [Fivetran Platform connector documentation](https://fivetran.com/docs/logs/fivetran-platform)\n  - [Fivetran Platform ERD](https://fivetran.com/docs/logs/fivetran-platform#schemainformation)\n- dbt package documentation\n  - [GitHub repository](https://github.com/fivetran/dbt_fivetran_log)\n  - [dbt Docs](https://fivetran.github.io/dbt_fivetran_log/#!/overview)\n  - [DAG](https://fivetran.github.io/dbt_fivetran_log/#!/overview?g_v=1)\n  - [Changelog](https://github.com/fivetran/dbt_fivetran_log/blob/main/CHANGELOG.md)\n- dbt Core™ supported versions\n  - `\u003e=1.3.0, \u003c3.0.0`\n\n## What does this dbt package do?\nThis package enables you to better understand how you are spending money in Fivetran according to our [consumption-based pricing model](https://fivetran.com/docs/usage-based-pricing) and provides details about the performance and status of your Fivetran connections. It creates enriched models with metrics focused on consumption data, monthly active rows (MAR), credit consumption, connection events, schema changes, and audit logs.\n\n### Output schema\nFinal output tables are generated in the following target schema:\n\n```\n\u003cyour_database\u003e.\u003cconnector/schema_name\u003e_fivetran_platform\n```\n\n### Final output tables\n\nBy default, this package materializes the following final tables:\n\n| Table | Description |\n| :---- | :---- |\n| [fivetran_platform__connection_status](https://fivetran.github.io/dbt_fivetran_log/#!/model/model.fivetran_log.fivetran_platform__connection_status) | Provides a comprehensive view of each connection loading data into your destinations, enriched with detailed information about sync status, sync frequency, setup status, and connection health to monitor and troubleshoot your data pipeline performance. |\n| [fivetran_platform__mar_table_history](https://fivetran.github.io/dbt_fivetran_log/#!/model/model.fivetran_log.fivetran_platform__mar_table_history) | Tracks a table's monthly free, paid, and total volume breakdowns, with connection and destination details to analyze your data consumption patterns and costs at the table level over time. |\n| [fivetran_platform__usage_history](https://fivetran.github.io/dbt_fivetran_log/#!/model/model.fivetran_log.fivetran_platform__usage_history) | Summarizes each destination's monthly usage and active volume with calculated metrics for usage per million MAR and MAR per usage unit to track your Fivetran consumption costs and efficiency. Usage represents either dollar or credit amounts depending on your pricing model. Read more about the relationship between usage and MAR [here](https://www.fivetran.com/legal/service-consumption-table). |\n| [fivetran_platform__connection_daily_events](https://fivetran.github.io/dbt_fivetran_log/#!/model/model.fivetran_log.fivetran_platform__connection_daily_events) | Captures daily operational metrics for each connection including API calls made, schema changes implemented, and record modifications processed, starting from the connection setup date to provide insights into connection activity patterns and data processing volumes. |\n| [fivetran_platform__schema_changelog](https://fivetran.github.io/dbt_fivetran_log/#!/model/model.fivetran_log.fivetran_platform__schema_changelog) | Documents all schema changes made to your connections including table alterations, table creations, schema creations, and configuration changes with detailed metadata about each event to track data structure evolution and troubleshoot schema-related issues. |\n| [fivetran_platform__audit_table](https://fivetran.github.io/dbt_fivetran_log/#!/model/model.fivetran_log.fivetran_platform__audit_table) | Replaces the deprecated [`_fivetran_audit` table](https://fivetran.com/docs/getting-started/system-columns-and-tables#audittables) and tracks each table receiving data during connection syncs with comprehensive timestamps for connection and table-level sync progress plus detailed counts of records inserted, replaced, updated, and deleted to monitor data processing and sync performance. |\n| [fivetran_platform__audit_user_activity](https://fivetran.github.io/dbt_fivetran_log/#!/model/model.fivetran_log.fivetran_platform__audit_user_activity) | Records all user-triggered actions within your Fivetran account to provide a comprehensive audit trail that helps you trace user activities to specific [log events](https://fivetran.com/docs/logs#logeventlist) such as schema changes, sync frequency updates, manual syncs, connection failures, and other operational events for compliance and troubleshooting purposes. |\n\n¹ Each Quickstart transformation job run materializes these models if all components of this data model are enabled. This count includes all staging, intermediate, and final models materialized as `view`, `table`, or `incremental`.\n\n---\n\n## Prerequisites\n\nTo use this dbt package, you must have the following:\n\n- A Fivetran Platform connection syncing data into your destination.\n- A **BigQuery**, **Snowflake**, **Redshift**, **Postgres**, **Databricks**, or **SQL Server** destination.\n\n## How do I use the dbt package?\nYou can either add this dbt package in the Fivetran dashboard or import it into your dbt project:\n\n- To add the package in the Fivetran dashboard, follow our [Quickstart guide](https://fivetran.com/docs/transformations/data-models/quickstart-management).\n- To add the package to your dbt project, follow the setup instructions in the dbt package's [README file](https://github.com/fivetran/dbt_fivetran_log/blob/main/README.md#how-do-i-use-the-dbt-package) to use this package.\n\n\u003c!--section-end--\u003e\n\n### Install the Package\nInclude the following Fivetran Platform package version range in your `packages.yml`\n\u003e Check [dbt Hub](https://hub.getdbt.com/) for the latest installation instructions, or [read the dbt docs](https://docs.getdbt.com/docs/package-management) for more information on installing packages.\n\n\u003e dbt Core \u003e= 1.9.6 is required to run freshness tests out of the box. See other options [here](https://github.com/fivetran/dbt_fivetran_log/blob/main/CHANGELOG.md#breaking-change-for-dbt-core--196).\n\n```yml\npackages:\n  - package: fivetran/fivetran_log\n    version: [\"\u003e=2.5.0\", \"\u003c2.6.0\"]\n```\n\n\u003e Note that although the source connector is now \"Fivetran Platform\", the package retains the old name of \"fivetran_log\".\n\n#### Databricks Dispatch Configuration\nIf you are using a Databricks destination with this package you will need to add the below (or a variation of the below) dispatch configuration within your `dbt_project.yml`. This is required in order for the package to accurately search for macros within the `dbt-labs/spark_utils` then the `dbt-labs/dbt_utils` packages respectively.\n```yml\ndispatch:\n  - macro_namespace: dbt_utils\n    search_order: ['spark_utils', 'dbt_utils']\n```\n\n#### Database Incremental Strategies\nFor models in this package that are materialized incrementally, they are configured to work with the different strategies available to each supported warehouse.\n\nFor **BigQuery** and **Databricks All Purpose Cluster runtime** destinations, we have chosen `insert_overwrite` as the default strategy, which benefits from the partitioning capability.\n\nFor **Databricks SQL Warehouse** destinations, we have chosen `merge` as the default strategy.\n\nFor **Snowflake**, **Redshift**, and **Postgres** destinations, we have chosen `delete+insert` as the default strategy.\n\n\u003e Regardless of strategy, we recommend that users periodically run a `--full-refresh` to ensure a high level of data quality.\n\n### Define Database and Schema Variables\nBy default, this package will run using your target database and the `fivetran_log` schema. If this is not where your Fivetran Platform data is (perhaps your fivetran platform schema is `fivetran_platform`), add the following configuration to your root `dbt_project.yml` file:\n\n```yml\nvars:\n    fivetran_platform_database: your_database_name # default is your target.database\n    fivetran_platform_schema: your_schema_name # default is fivetran_log\n```\n\n### Disable Models for Non Existent Sources\nIf you do not leverage Fivetran RBAC, then you will not have the `user` or `destination_membership` source tables. The `user` and `destination_membership` are enabled by default. Therefore in order to switch the default configurations, you must add the following variable(s) to your root `dbt_project.yml` file for the respective source tables you wish to disable:\n\n```yml\nvars:\n    fivetran_platform_using_destination_membership: false # Default is true. This will disable only the destination membership logic\n    fivetran_platform_using_user: false # Default is true. This will disable only the user logic\n```\n\n#### Leveraging `CONNECTION` vs `CONNECTOR`  \nIn Q1 2025, the `CONNECTOR` source table was deprecated and replaced by `CONNECTION`, and `CONNECTION` is now the default source.\n\n- For **Quickstart users**, `CONNECTOR` will automatically be used if `CONNECTION` is not yet available.\n- For **dbt Core users**, if `CONNECTION` is not yet available in your connection, you can continue using `CONNECTOR` by adding the following variable to your root `dbt_project.yml` file:\n\n```yml\nvars:\n    fivetran_platform_using_connection: false # default: true\n```\n\n### (Optional) Additional Configurations\n\n#### Change the Build Schema\nBy default this package will build the Fivetran staging models within a schema titled (\u003ctarget_schema\u003e + `_stg_fivetran_platform`)  and the Fivetran Platform final models within your \u003ctarget_schema\u003e + `_fivetran_platform` in your target database. If this is not where you would like you Fivetran staging and final models to be written to, add the following configuration to your root `dbt_project.yml` file:\n\n```yml\nmodels:\n  fivetran_log:\n    +schema: my_new_final_models_schema # leave blank for just the target_schema\n    staging:\n      +schema: my_new_staging_models_schema # leave blank for just the target_schema\n```\n\n#### Change the Source Table References\nIf an individual source table has a different name than expected (see this projects [dbt_project.yml](https://github.com/fivetran/dbt_fivetran_log/blob/main/dbt_project.yml) variable declarations for expected names), provide the name of the table as it appears in your warehouse to the respective variable as identified below:\n```yml\nvars:\n    fivetran_platform_\u003cdefault_table_name\u003e_identifier: your_table_name \n```\n\n### (Optional) Orchestrate your models with Fivetran Transformations for dbt Core™\n\u003cdetails\u003e\u003csummary\u003eExpand for details\u003c/summary\u003e\n\u003cbr\u003e\n\nFivetran offers the ability for you to orchestrate your dbt project through [Fivetran Transformations for dbt Core™](https://fivetran.com/docs/transformations/dbt#transformationsfordbtcore). Refer to the linked docs for more information on how to setup your project for orchestration through Fivetran.\n\u003c/details\u003e\n\n## Does this package have dependencies?\nThis dbt package is dependent on the following dbt packages. These dependencies are installed by default within this package. For more information on the below packages, refer to the [dbt hub](https://hub.getdbt.com/) site.\n\u003e **If you have any of these dependent packages in your own `packages.yml` I highly recommend you remove them to ensure there are no package version conflicts.**\n```yml\npackages:\n    - package: fivetran/fivetran_utils\n      version: [\"\u003e=0.4.0\", \"\u003c0.5.0\"]\n\n    - package: dbt-labs/dbt_utils\n      version: [\"\u003e=1.0.0\", \"\u003c2.0.0\"]\n\n    - package: dbt-labs/spark_utils\n      version: [\"\u003e=0.3.0\", \"\u003c0.4.0\"]\n```\n\n\u003c!--section=\"fivetran-log_maintenance\"--\u003e\n## How is this package maintained and can I contribute?\n\n### Package Maintenance\nThe Fivetran team maintaining this package only maintains the [latest version](https://hub.getdbt.com/fivetran/fivetran_log/latest/) of the package. We highly recommend you stay consistent with the latest version of the package and refer to the [CHANGELOG](https://github.com/fivetran/dbt_fivetran_log/blob/main/CHANGELOG.md) and release notes for more information on changes across versions.\n\n### Contributions\nA small team of analytics engineers at Fivetran develops these dbt packages. However, the packages are made better by community contributions.\n\nWe highly encourage and welcome contributions to this package. Learn how to contribute to a package in dbt's [Contributing to an external dbt package article](https://discourse.getdbt.com/t/contributing-to-a-dbt-package/657).\n\n\u003c!--section-end--\u003e\n\n## Are there any resources available?\n- If you encounter any questions or want to reach out for help, see the [GitHub Issue](https://github.com/fivetran/dbt_fivetran_log/issues/new/choose) section to find the right avenue of support for you.\n- If you would like to provide feedback to the dbt package team at Fivetran, or would like to request a future dbt package to be developed, then feel free to fill out our [Feedback Form](https://www.surveymonkey.com/r/DQ7K7WW).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffivetran%2Fdbt_fivetran_log","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffivetran%2Fdbt_fivetran_log","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffivetran%2Fdbt_fivetran_log/lists"}