{"id":20291251,"url":"https://github.com/arsenvlad/docker-presto-adls-wasb","last_synced_at":"2025-08-24T06:45:00.406Z","repository":{"id":97982221,"uuid":"92454787","full_name":"arsenvlad/docker-presto-adls-wasb","owner":"arsenvlad","description":"Example of a single node Presto with Azure Data Lake Store (ADLS) and Azure Storage Blob (WASB) access via Hive metastore","archived":false,"fork":false,"pushed_at":"2020-06-16T15:46:03.000Z","size":174,"stargazers_count":19,"open_issues_count":1,"forks_count":16,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-11T11:54:52.684Z","etag":null,"topics":["azure-blob-storage","azure-data-lake","docker","presto"],"latest_commit_sha":null,"homepage":"","language":"Dockerfile","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/arsenvlad.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-05-26T00:03:36.000Z","updated_at":"2024-10-09T01:54:02.000Z","dependencies_parsed_at":"2023-05-03T10:00:51.106Z","dependency_job_id":null,"html_url":"https://github.com/arsenvlad/docker-presto-adls-wasb","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/arsenvlad/docker-presto-adls-wasb","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arsenvlad%2Fdocker-presto-adls-wasb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arsenvlad%2Fdocker-presto-adls-wasb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arsenvlad%2Fdocker-presto-adls-wasb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arsenvlad%2Fdocker-presto-adls-wasb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/arsenvlad","download_url":"https://codeload.github.com/arsenvlad/docker-presto-adls-wasb/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arsenvlad%2Fdocker-presto-adls-wasb/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271809687,"owners_count":24825772,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-24T02:00:11.135Z","response_time":111,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["azure-blob-storage","azure-data-lake","docker","presto"],"created_at":"2024-11-14T15:11:20.135Z","updated_at":"2025-08-24T06:45:00.308Z","avatar_url":"https://github.com/arsenvlad.png","language":"Dockerfile","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Example of a single node Presto with Azure Data Lake Store (ADLS) and Azure Blob Storage (WASB)\n\nClick to watch video\n[![Presto with ADLS and WASB](images/presto-adls-wasb-play.png)](https://youtu.be/hflThIJdkrI)\n\n## Start local Hive metastore and Presto containers\n\nClone this repo\n\n```git clone https://github.com/arsenvlad/docker-presto-adls-wasb```\n\nRun Hive and Presto containers using config specified in *env.conf.private*\n\n```docker-compose up --build```\n\nIn a separate terminal window, list currently running containers\n\n```docker ps```\n\n## Connect to Hive bash\n\nIn a separate terminal window, open interactive tty bash on the Hive container\n\n```docker exec -it dockerprestoadlswasb_hive_1 bash```\n\nIn the Hive container bash session, open Hive CLI pointing to itself as an external metastore. If you get an error saying \"Name node is in safe mode\", wait for a few minutes and try again.\n\n```hive --hiveconf hive.metastore.uris=thrift://localhost:9083```\n\nCreate table using Azure Storage Blobs (change the storage account name and container name to yours)\n\n```create table wasbtable1 (id int, name varchar(255)) row format delimited fields terminated by ',' stored as textfile location 'wasb://test-hive@avdatarepo1.blob.core.windows.net/wasbtable1';```\n\nCreate table using Azure Data Lake Store (change the ADLS account name to yours)\n\n```create table adltable1 (id int, name varchar(255)) row format delimited fields terminated by ',' stored as textfile location 'adl://avdatalake1.azuredatalakestore.net/adltable1';```\n\nConfirm you can see the tables\n```show tables;```\n\n## Connect to Presto bash\n\nIn a separate terminal window, open interactive tty bash on the Presto container\n\n```docker exec -it dockerprestoadlswasb_presto_1 bash```\n\nPresto is configured with a single node with Hive connector as described in [/etc/motd](files/motd.txt)\n\nUse Presto CLI to connect to the running Presto server\n\n```/opt/presto/presto --server http://localhost:8080```\n\nList shemas in Hive catalog\n\n```show schemas from hive;```\n\nList tables in the Hive default catalog\n\n```show tables from hive.default;```\n\nInsert data into the tables\n\n```\ninsert into hive.default.wasbtable1 (id, name) values (1,'1');\ninsert into hive.default.wasbtable1 (id, name) select id, name from hive.default.wasbtable1 union all select id, name from hive.default.wasbtable1 union all select id, name from hive.default.wasbtable1;\n\ninsert into hive.default.adltable1 (id, name) values (1,'1');\ninsert into hive.default.adltable1 (id, name) select id, name from hive.default.adltable1 union all select id, name from hive.default.adltable1 union all select id, name from hive.default.adltable1;\n```\n\nSelect from the table\n\n```select * from hive.default.adltable1;```\n\n## When using with HDInsight\n\nNOTE: To access Azure HDInsight Hive Thrift Service your Docker host VM must be within the same network.\n\nTo find the URLs of the HDInsight Hive Thrift Service (i.e. hive.metastore.uri), SSH into the HDInsight cluster and run this grep command:\n\n```echo $(grep -n1 \"hive.metastore.uri\" /etc/hive/conf/hive-site.xml | grep -o \"\u003cvalue\u003e.*/value\u003e\" | sed 's:\u003cvalue\u003e::g' | sed 's:\u003c/value\u003e::g')```\n\n## Presto with Azure Data Services\n\nSee [azure-data-services.md](azure-data-services.md) for an example showing how to configure Presto connectors to Azure Data Services to query and join data from Azure CosmosDB (using MongoDB API), Azure SQL Database, Azure MySQL, Azure PostgreSQL and store the joined results in Azure Blob Storage.\n\n[![Azure CosmosDB with MongoDB API, Azure SQL Database, Azure MySQL, Azure PostgreSQL](images/presto-azure-data-services-play.png)](https://youtu.be/XDfCK6Ejz-A)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farsenvlad%2Fdocker-presto-adls-wasb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Farsenvlad%2Fdocker-presto-adls-wasb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farsenvlad%2Fdocker-presto-adls-wasb/lists"}