{"id":18995757,"url":"https://github.com/turbot/steampipe-plugin-databricks","last_synced_at":"2025-07-28T12:14:06.191Z","repository":{"id":180873071,"uuid":"662939522","full_name":"turbot/steampipe-plugin-databricks","owner":"turbot","description":"Use SQL to instantly query Databricks resources. Open source CLI. No DB required.","archived":false,"fork":false,"pushed_at":"2025-06-10T10:19:54.000Z","size":523,"stargazers_count":7,"open_issues_count":3,"forks_count":1,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-06-10T11:30:01.746Z","etag":null,"topics":["backup","databricks","etl","hacktoberfest","postgresql","postgresql-fdw","sql","sqlite","steampipe","steampipe-plugin","zero-etl"],"latest_commit_sha":null,"homepage":"https://hub.steampipe.io/plugins/turbot/databricks","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/turbot.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-07-06T08:01:38.000Z","updated_at":"2025-06-10T10:18:28.000Z","dependencies_parsed_at":"2023-10-01T18:41:58.953Z","dependency_job_id":"7e1c0630-e34f-4e7e-9d1e-f87cefb7d9b2","html_url":"https://github.com/turbot/steampipe-plugin-databricks","commit_stats":null,"previous_names":["turbot/steampipe-plugin-databricks"],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/turbot/steampipe-plugin-databricks","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/turbot%2Fsteampipe-plugin-databricks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/turbot%2Fsteampipe-plugin-databricks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/turbot%2Fsteampipe-plugin-databricks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/turbot%2Fsteampipe-plugin-databricks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/turbot","download_url":"https://codeload.github.com/turbot/steampipe-plugin-databricks/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/turbot%2Fsteampipe-plugin-databricks/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267514581,"owners_count":24099972,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-28T02:00:09.689Z","response_time":68,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backup","databricks","etl","hacktoberfest","postgresql","postgresql-fdw","sql","sqlite","steampipe","steampipe-plugin","zero-etl"],"created_at":"2024-11-08T17:32:40.897Z","updated_at":"2025-07-28T12:14:06.080Z","avatar_url":"https://github.com/turbot.png","language":"Go","readme":"![image](https://hub.steampipe.io/images/plugins/turbot/databricks-social-graphic.png)\n\n# Databricks Plugin for Steampipe\n\nUse SQL to query clusters, jobs, users, and more from Databricks.\n\n- **[Get started →](https://hub.steampipe.io/plugins/turbot/databricks)**\n- Documentation: [Table definitions \u0026 examples](https://hub.steampipe.io/plugins/turbot/databricks/tables)\n- Community: [Join #steampipe on Slack →](https://turbot.com/community/join)\n- Get involved: [Issues](https://github.com/turbot/steampipe-plugin-databricks/issues)\n\n## Quick start\n\n### Install\n\nDownload and install the latest Databricks plugin:\n\n```bash\nsteampipe plugin install databricks\n```\n\nConfigure your [credentials](https://hub.steampipe.io/plugins/turbot/databricks#credentials) and [config file](https://hub.steampipe.io/plugins/turbot/databricks#configuration).\n\nConfigure your account details in `~/.steampipe/config/databricks.spc`:\n\n```hcl\nconnection \"databricks\" {\n  plugin = \"databricks\"\n\n  # A connection profile specified within .databrickscfg to use instead of DEFAULT.\n  # This can also be set via the `DATABRICKS_CONFIG_PROFILE` environment variable.\n  # profile = \"databricks-dev\"\n\n  # The target Databricks account ID.\n  # This can also be set via the `DATABRICKS_ACCOUNT_ID` environment variable.\n  # See Locate your account ID: https://docs.databricks.com/administration-guide/account-settings/index.html#account-id.\n  # account_id = \"abcdd0f81-9be0-4425-9e29-3a7d96782373\"\n\n  # The target Databricks account SCIM token.\n  # See: https://docs.databricks.com/administration-guide/account-settings/index.html#generate-a-scim-token\n  # This can also be set via the `DATABRICKS_TOKEN` environment variable.\n  # account_token = \"dsapi5c72c067b40df73ccb6be3b085d3ba\"\n\n  # The target Databricks account console URL, which is typically https://accounts.cloud.databricks.com.\n  # This can also be set via the `DATABRICKS_HOST` environment variable.\n  # account_host = \"https://accounts.cloud.databricks.com/\"\n\n  # The target Databricks workspace Personal Access Token.\n  # This can also be set via the `DATABRICKS_TOKEN` environment variable.\n  # See: https://docs.databricks.com/dev-tools/auth.html#databricks-personal-access-tokens-for-users\n  # workspace_token = \"dapia865b9d1d41389ed883455032d090ee\"\n\n  # The target Databricks workspace URL.\n  # See https://docs.databricks.com/workspace/workspace-details.html#workspace-url\n  # This can also be set via the `DATABRICKS_HOST` environment variable.\n  # workspace_host = \"https://dbc-a1b2c3d4-e6f7.cloud.databricks.com\"\n\n  # The Databricks username part of basic authentication. Only possible when Host is *.cloud.databricks.com (AWS).\n  # This can also be set via the `DATABRICKS_USERNAME` environment variable.\n  # username = \"user@turbot.com\"\n\n  # The Databricks password part of basic authentication. Only possible when Host is *.cloud.databricks.com (AWS).\n  # This can also be set via the `DATABRICKS_PASSWORD` environment variable.\n  # password = \"password\"\n\n  # A non-default location of the Databricks CLI credentials file.\n  # This can also be set via the `DATABRICKS_CONFIG_FILE` environment variable.\n  # config_file_path = \"/Users/username/.databrickscfg\"\n  \n  # OAuth secret client ID of a service principal\n  # This can also be set via the `DATABRICKS_CLIENT_ID` environment variable.\n  # client_id = \"123-456-789\"\n\n  # OAuth secret value of a service principal\n  # This can also be set via the `DATABRICKS_CLIENT_SECRET` environment variable.\n  # client_secret = \"dose1234567789abcde\"\n}\n```\n\n- **[Detailed configuration guide →](https://hub.steampipe.io/plugins/turbot/databricks#quick-start)**\n\nOr through environment variables:\n\n```sh\nexport DATABRICKS_CONFIG_PROFILE=user1-test\nexport DATABRICKS_TOKEN=dsapi5c72c067b40df73ccb6be3b085d3ba\nexport DATABRICKS_HOST=https://accounts.cloud.databricks.com\nexport DATABRICKS_ACCOUNT_ID=abcdd0f81-9be0-4425-9e29-3a7d96782373\nexport DATABRICKS_USERNAME=user@turbot.com\nexport DATABRICKS_PASSWORD=password\nexport DATABRICKS_CLIENT_ID=123-456-789\nexport DATABRICKS_CLIENT_SECRET=dose1234567789abcde\n```\n\nRun steampipe:\n\n```shell\nsteampipe query\n```\n\nList details of your Databricks clusters:\n\n```sql\nselect\n  cluster_id,\n  title,\n  cluster_source,\n  creator_user_name,\n  driver_node_type_id,\n  node_type_id,\n  state,\n  start_time\nfrom\n  databricks_compute_cluster;\n```\n\n```\n+----------------------+--------------------------------+----------------+-------------------+---------------------+--------------+------------+---------------------------+\n| cluster_id           | title                          | cluster_source | creator_user_name | driver_node_type_id | node_type_id | state      | start_time                |\n+----------------------+--------------------------------+----------------+-------------------+---------------------+--------------+------------+---------------------------+\n| 1234-141524-10b6dv2h | [default]basic-starter-cluster | \"API\"          | user@turbot.com   | i3.xlarge           | i3.xlarge    | TERMINATED | 2023-07-21T19:45:24+05:30 |\n| 1234-061816-mvns8mxz | test-cluster-for-ml            | \"UI\"           | user@turbot.com   | i3.xlarge           | i3.xlarge    | TERMINATED | 2023-07-28T11:48:16+05:30 |\n+----------------------+--------------------------------+----------------+-------------------+---------------------+--------------+------------+---------------------------+\n```\n\n## Engines\n\nThis plugin is available for the following engines:\n\n| Engine        | Description\n|---------------|------------------------------------------\n| [Steampipe](https://steampipe.io/docs) | The Steampipe CLI exposes APIs and services as a high-performance relational database, giving you the ability to write SQL-based queries to explore dynamic data. Mods extend Steampipe's capabilities with dashboards, reports, and controls built with simple HCL. The Steampipe CLI is a turnkey solution that includes its own Postgres database, plugin management, and mod support.\n| [Postgres FDW](https://steampipe.io/docs/steampipe_postgres/overview) | Steampipe Postgres FDWs are native Postgres Foreign Data Wrappers that translate APIs to foreign tables. Unlike Steampipe CLI, which ships with its own Postgres server instance, the Steampipe Postgres FDWs can be installed in any supported Postgres database version.\n| [SQLite Extension](https://steampipe.io/docs/steampipe_sqlite/overview) | Steampipe SQLite Extensions provide SQLite virtual tables that translate your queries into API calls, transparently fetching information from your API or service as you request it.\n| [Export](https://steampipe.io/docs/steampipe_export/overview) | Steampipe Plugin Exporters provide a flexible mechanism for exporting information from cloud services and APIs. Each exporter is a stand-alone binary that allows you to extract data using Steampipe plugins without a database.\n| [Turbot Pipes](https://turbot.com/pipes/docs) | Turbot Pipes is the only intelligence, automation \u0026 security platform built specifically for DevOps. Pipes provide hosted Steampipe database instances, shared dashboards, snapshots, and more.\n\n## Developing\n\nPrerequisites:\n\n- [Steampipe](https://steampipe.io/downloads)\n- [Golang](https://golang.org/doc/install)\n\nClone:\n\n```sh\ngit clone https://github.com/turbot/steampipe-plugin-databricks.git\ncd steampipe-plugin-databricks\n```\n\nBuild, which automatically installs the new version to your `~/.steampipe/plugins` directory:\n\n```\nmake\n```\n\nConfigure the plugin:\n\n```\ncp config/* ~/.steampipe/config\nvi ~/.steampipe/config/databricks.spc\n```\n\nTry it!\n\n```\nsteampipe query\n\u003e .inspect databricks\n```\n\nFurther reading:\n\n- [Writing plugins](https://steampipe.io/docs/develop/writing-plugins)\n- [Writing your first table](https://steampipe.io/docs/develop/writing-your-first-table)\n\n## Open Source \u0026 Contributing\n\nThis repository is published under the [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0) (source code) and [CC BY-NC-ND](https://creativecommons.org/licenses/by-nc-nd/2.0/) (docs) licenses. Please see our [code of conduct](https://github.com/turbot/.github/blob/main/CODE_OF_CONDUCT.md). We look forward to collaborating with you!\n\n[Steampipe](https://steampipe.io) is a product produced from this open source software, exclusively by [Turbot HQ, Inc](https://turbot.com). It is distributed under our commercial terms. Others are allowed to make their own distribution of the software, but cannot use any of the Turbot trademarks, cloud services, etc. You can learn more in our [Open Source FAQ](https://turbot.com/open-source).\n\n## Get Involved\n\n**[Join #steampipe on Slack →](https://turbot.com/community/join)**\n\nWant to help but don't know where to start? Pick up one of the `help wanted` issues:\n\n- [Steampipe](https://github.com/turbot/steampipe/labels/help%20wanted)\n- [Databricks Plugin](https://github.com/turbot/steampipe-plugin-databricks/labels/help%20wanted)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fturbot%2Fsteampipe-plugin-databricks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fturbot%2Fsteampipe-plugin-databricks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fturbot%2Fsteampipe-plugin-databricks/lists"}