{"id":25405292,"url":"https://github.com/dacort/metabase-athena-driver","last_synced_at":"2025-10-27T01:05:17.612Z","repository":{"id":37832474,"uuid":"185419103","full_name":"dacort/metabase-athena-driver","owner":"dacort","description":"An Amazon Athena driver for Metabase 0.32 and later","archived":false,"fork":false,"pushed_at":"2022-12-08T18:15:27.000Z","size":146,"stargazers_count":224,"open_issues_count":2,"forks_count":32,"subscribers_count":22,"default_branch":"master","last_synced_at":"2025-03-02T13:11:18.424Z","etag":null,"topics":["amazon-athena","athena","athena-driver","aws","aws-athena","metabase","metabase-driver"],"latest_commit_sha":null,"homepage":"","language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dacort.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-05-07T14:30:58.000Z","updated_at":"2025-02-06T15:37:24.000Z","dependencies_parsed_at":"2022-09-16T08:41:15.029Z","dependency_job_id":null,"html_url":"https://github.com/dacort/metabase-athena-driver","commit_stats":null,"previous_names":[],"tags_count":20,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dacort%2Fmetabase-athena-driver","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dacort%2Fmetabase-athena-driver/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dacort%2Fmetabase-athena-driver/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dacort%2Fmetabase-athena-driver/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dacort","download_url":"https://codeload.github.com/dacort/metabase-athena-driver/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244066180,"owners_count":20392406,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["amazon-athena","athena","athena-driver","aws","aws-athena","metabase","metabase-driver"],"created_at":"2025-02-16T04:33:54.017Z","updated_at":"2025-10-27T01:05:17.487Z","avatar_url":"https://github.com/dacort.png","language":"Clojure","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Metabase Athena Driver\n\n\u003e **Note**: As of Metabase v0.45, [Metabase now supports Amazon Athena as an official data source](https://www.metabase.com/releases/Metabase-0.45)! 🎉 \u003cbr /\u003e This driver will no longer be actively maintained. Any future issues can be asked about on the [Metabase forum](https://discourse.metabase.com/) or with a detailed [bug report](https://github.com/metabase/metabase/issues).\n\n\u003cbr /\u003e\n\n[![CircleCI](https://img.shields.io/circleci/build/github/dacort/metabase-athena-driver)](https://circleci.com/gh/dacort/metabase-athena-driver)\n[![Latest Release](https://img.shields.io/github/v/release/dacort/metabase-athena-driver.svg?label=latest%20release\u0026include_prereleases)](https://github.com/dacort/metabase-athena-driver/releases)\n![Tested with Metabase v0.44.6](https://img.shields.io/badge/metabase-v0.44.6-blue?)\n[![GitHub license](https://img.shields.io/github/license/dacort/metabase-athena-driver)](https://raw.githubusercontent.com/dacort/metabase-athena-driver/master/LICENSE)\n\n## Installation\n\nBeginning with Metabase 0.32, drivers must be stored in a `plugins` directory in the same directory where `metabase.jar` is, or you can specify the directory by setting the environment variable `MB_PLUGINS_DIR`. There are a few options to get up and running with a custom driver.\n\n### Docker\n\nThis repository has an example [`Dockerfile`](./Dockerfile) you can use to build the Amazon Athena Metabase driver and run the most recent supported version of Metabase:\n\n```shell\ngit clone https://github.com/dacort/metabase-athena-driver.git\ncd metabase-athena-driver\ndocker build -t metabase/athena .\ndocker run --name metabase-athena -p 3000:3000 metabase/athena\n```\n\nThen open http://localhost:3000 and skip down to [Configuring](#Configuring)\n\n### Download Metabase Jar and Run\n\n1. Download a fairly recent Metabase binary release (jar file) from the [Metabase distribution page](https://metabase.com/start/jar.html).\n2. Download the Athena driver jar from this repository's [\"Releases\"](https://github.com/dacort/metabase-athena-driver/releases) page\n3. Create a directory and copy the `metabase.jar` to it.\n4. In that directory create a sub-directory called `plugins`.\n5. Copy the Athena driver jar to the `plugins` directory.\n6. Make sure you are the in the directory where your `metabase.jar` lives.\n7. Run `java -jar metabase.jar`.\n\nIn either case, you should see a message on startup similar to:\n\n```\n04-15 06:14:08 DEBUG plugins.lazy-loaded-driver :: Registering lazy loading driver :athena...\n04-15 06:14:08 INFO driver.impl :: Registered driver :athena (parents: [:sql-jdbc]) 🚚\n```\n\n## Configuring\n\nOnce you've started up Metabase, go to add a database and select \"Amazon Athena\".\n\nYou'll need to provide the AWS region, an access key and secret key, and an S3 bucket and prefix where query results will be written to.\n\nPlease note:\n\n- The provided bucket must be in the same region you specify.\n- If you do _not_ provide an access key, the [default credentials chain](https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/DefaultAWSCredentialsProviderChain.html) will be used.\n- The initial sync can take some time depending on how many databases and tables you have.\n\nIf you need an example IAM policy for providing read-only access to your customer-base, check out the [Example IAM Policy](#example-iam-policy) below.\n\nYou can provide additional options if necessary. For example, to disable result set streaming and enable `TRACE`-level debugging, use `UseResultsetStreaming=0;LogLevel=6`.\n\nResult set streaming is a performance optimization that streams results from Athena rather than using pagination logic, however it requries outbound access to TCP port 444 and not all organizations allow that.\n\nOther options can be found in the \"Driver Configuration Options\" section of the [Athena JDBC Driver Installation and Configuration\nGuide](https://s3.amazonaws.com/athena-downloads/drivers/JDBC/SimbaAthenaJDBC_2.0.13/docs/Simba+Athena+JDBC+Driver+Install+and+Configuration+Guide.pdf).\n\n## Contributing\n\n### Prerequisites\n\n- [Leiningen](https://leiningen.org/)\n- [Install metabase-core](https://github.com/metabase/metabase/wiki/Writing-a-Driver:-Packaging-a-Driver-\u0026-Metabase-Plugin-Basics#installing-metabase-core-locally)\n\n### Build from source\n\nThe entire jar can now be built from the included Dockerfile.\n\n1. Build the project and copy the jar from the export stage\n\n```shell\ndocker build --output jars --target stg_export .\n```\n\nYou should now have a `athena.metabase-driver.jar` file in the `jars/` directory.\n\n2. Download a fairly recent Metabase binary release (jar file) from the [Metabase distribution page](https://metabase.com/start/jar.html).\n\n3. Let's assume we download `metabase.jar` to `~/metabae/` and we built the project above. Copy the built jar to the Metabase plugins directly and run Metabase from there!\n   ```shell\n   TARGET_DIR=~/metabae\n   mkdir ${TARGET_DIR}/plugins/\n   cp jars/athena.metabase-driver.jar ${TARGET_DIR}/plugins/\n   cd ${TARGET_DIR}/\n   java -jar metabase.jar\n   ```\n\nYou should see a message on startup similar to:\n\n```\n2019-05-07 23:27:32 INFO plugins.lazy-loaded-driver :: Registering lazy loading driver :athena...\n2019-05-07 23:27:32 INFO metabase.driver :: Registered driver :athena (parents: #{:sql-jdbc}) 🚚\n```\n\n### Testing\n\nThere are two different sets of tests in the project.\n\n1. Unit tests, located in the `test_unit/` directory\n2. Integration tests, located in the standard `test/` directory\n\nThe reason they're split out is because the integration tests require us to [link the driver](https://github.com/metabase/metabase/wiki/Writing-a-Driver:-Adding-Test-Extensions,-Tests,-and-Setting-up-CI#file-organization) into the core Metabase code and run the full suite of tests there. I wanted to be able to have some lightweight unit tests that could be run without that overhead, so those are split out into the `test_unit/` directory.\n\nRunning the tests requires you to have the metabase source relevant to the version you're building against. To make this easier, you can also run tests from the Dockerfile.\n\n```shell\ndocker build -t metabase/athena-test --target stg_test .\ndocker run --rm --name mb-test metabase/athena-test\n```\n\n## Resources\n\n### Example IAM Policy\n\nThis policy provides read-only access. Note you need to specify any buckets you want the user to be able to query from _as well as_ the S3 bucket provided as part of the configuration where results are written to.\n\n```json\n{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n    {\n      \"Sid\": \"Athena\",\n      \"Effect\": \"Allow\",\n      \"Action\": [\n        \"athena:BatchGetNamedQuery\",\n        \"athena:BatchGetQueryExecution\",\n        \"athena:GetNamedQuery\",\n        \"athena:GetQueryExecution\",\n        \"athena:GetQueryResults\",\n        \"athena:GetQueryResultsStream\",\n        \"athena:GetWorkGroup\",\n        \"athena:ListDatabases\",\n        \"athena:ListDataCatalogs\",\n        \"athena:ListNamedQueries\",\n        \"athena:ListQueryExecutions\",\n        \"athena:ListTagsForResource\",\n        \"athena:ListWorkGroups\",\n        \"athena:ListTableMetadata\",\n        \"athena:StartQueryExecution\",\n        \"athena:StopQueryExecution\",\n        \"athena:CreatePreparedStatement\",\n        \"athena:DeletePreparedStatement\",\n        \"athena:GetPreparedStatement\"\n      ],\n      \"Resource\": \"*\"\n    },\n    {\n      \"Sid\": \"Glue\",\n      \"Effect\": \"Allow\",\n      \"Action\": [\n        \"glue:BatchGetPartition\",\n        \"glue:GetDatabase\",\n        \"glue:GetDatabases\",\n        \"glue:GetPartition\",\n        \"glue:GetPartitions\",\n        \"glue:GetTable\",\n        \"glue:GetTables\",\n        \"glue:GetTableVersion\",\n        \"glue:GetTableVersions\"\n      ],\n      \"Resource\": \"*\"\n    },\n    {\n      \"Sid\": \"S3ReadAccess\",\n      \"Effect\": \"Allow\",\n      \"Action\": [\"s3:GetObject\", \"s3:ListBucket\", \"s3:GetBucketLocation\"],\n      \"Resource\": [\n        \"arn:aws:s3:::bucket1\",\n        \"arn:aws:s3:::bucket1/*\",\n        \"arn:aws:s3:::bucket2\",\n        \"arn:aws:s3:::bucket2/*\"\n      ]\n    },\n    {\n      \"Sid\": \"AthenaResultsBucket\",\n      \"Effect\": \"Allow\",\n      \"Action\": [\n        \"s3:PutObject\",\n        \"s3:GetObject\",\n        \"s3:AbortMultipartUpload\",\n        \"s3:ListBucket\",\n        \"s3:GetBucketLocation\"\n      ],\n      \"Resource\": [\"arn:aws:s3:::bucket2\", \"arn:aws:s3:::bucket2/*\"]\n    }\n  ]\n}\n```\n\nIf your customer-base needs access to create tables for whatever reason, they will need additional AWS Glue permissions. Here is an example policy granting that. Note that the Resource is `*` so this will give Delete/Update permissions to any table.\n\n```json\n{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n    {\n      \"Sid\": \"VisualEditor0\",\n      \"Effect\": \"Allow\",\n      \"Action\": [\n        \"glue:BatchCreatePartition\",\n        \"glue:UpdateDatabase\",\n        \"glue:DeleteDatabase\",\n        \"glue:CreateTable\",\n        \"glue:CreateDatabase\",\n        \"glue:UpdateTable\",\n        \"glue:BatchDeletePartition\",\n        \"glue:BatchDeleteTable\",\n        \"glue:DeleteTable\",\n        \"glue:CreatePartition\",\n        \"glue:DeletePartition\",\n        \"glue:UpdatePartition\"\n      ],\n      \"Resource\": \"*\"\n    }\n  ]\n}\n```\n\n## Known Issues\n\n- Cannot specify a single database to sync\n- ~~Only native SQL queries are supported~~\n  - Native SQL Queries must not end with a semi-colon (`;`)\n  - Basic aggregations seem to work in the query builder\n  - ~~Parameterized queries are not supported~~\n- Sometimes, the initial database verification can time out\n  - If this happens, configure a higher timeout value with the `MB_DB_CONNECTION_TIMEOUT_MS` environment variable\n- ~~Heavily nested fields can result in a `StackOverflowError`~~\n  - ~~If this happens, increase the `-Xss` JVM parameter~~\n\n## Updated Dockerfile\n\n### Test image\n\n```shell\ndocker build -t metabase/athena-test --target stg_test .\ndocker run -it --rm --name mb-test metabase/athena-test\n```\n\n### Copy jars\n\n```shell\ndocker build --output jars --target stg_export .\n```\n\n### Run Metabase\n\n```shell\ndocker build -t metabase/athena .\ndocker run --rm --name metabase-athena -p 3000:3000 metabase/athena\n```\n\nIf you have an existing Metabase database you'd like to use, you can use the following command.\n\n```shell\ndocker run --rm  -p 3000:3000 \\\n  -v ~/metabase-data:/metabase-data \\\n  -e MB_DB_FILE=/metabase-data/metabase.db \\\n  --name metabase-athena metabase/athena\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdacort%2Fmetabase-athena-driver","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdacort%2Fmetabase-athena-driver","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdacort%2Fmetabase-athena-driver/lists"}