{"id":19259820,"url":"https://github.com/unicef/magasin-drill","last_synced_at":"2025-04-21T16:31:06.215Z","repository":{"id":205916535,"uuid":"715029754","full_name":"unicef/magasin-drill","owner":"unicef","description":"Apache Drill  and Apache Zookeeper helm charts for kubernetes","archived":false,"fork":false,"pushed_at":"2024-12-02T13:23:20.000Z","size":1172,"stargazers_count":2,"open_issues_count":7,"forks_count":2,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-04-01T14:21:37.497Z","etag":null,"topics":["apache-drill","apache-zookeeper","charts","drill","drill-clusters","helm","helm-chart","kubernetes","magasin","zookeeper"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/unicef.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-06T10:42:00.000Z","updated_at":"2024-12-02T13:18:55.000Z","dependencies_parsed_at":"2023-11-28T06:23:15.728Z","dependency_job_id":"7cae694f-f735-45d6-91f2-96c37bc86261","html_url":"https://github.com/unicef/magasin-drill","commit_stats":null,"previous_names":["unicef/magasin-drill"],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unicef%2Fmagasin-drill","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unicef%2Fmagasin-drill/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unicef%2Fmagasin-drill/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unicef%2Fmagasin-drill/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/unicef","download_url":"https://codeload.github.com/unicef/magasin-drill/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250090764,"owners_count":21373250,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache-drill","apache-zookeeper","charts","drill","drill-clusters","helm","helm-chart","kubernetes","magasin","zookeeper"],"created_at":"2024-11-09T19:17:58.941Z","updated_at":"2025-04-21T16:31:06.200Z","avatar_url":"https://github.com/unicef.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Helm Charts for Apache Drill\n\n[![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/magasin-drill)](https://artifacthub.io/packages/search?repo=magasin-drill)\n\n\nThis repository contains a collection of files that can be used to deploy [Apache Drill](http://drill.apache.org/) on [Kubernetes](https://kubernetes.io/) using [Helm Charts](https://helm.sh/). Supports single-node and [cluster](http://drill.apache.org/docs/installing-drill-in-distributed-mode/) modes.\n\nThis is an extended version of the originally chart created in [github.com/Agirish/drill-helm-charts](https://github.com/Agirish/drill-helm-charts). \n\nThis extension has been created to fit the needs of [UNICEF's magasin](https://github.com/unicef/magasin), an end-to-end data AI/ML cloud data platform based.\n\n**Chart Versions**\n\n| Chart Version | Drill Version | Zookeeper Version | Release Date       | Release notes          |\n|---------------|---------------|-------------------|--------------------|------------------------|\n| 0.9.0         | 1.21.2        | 3.9.3             | 2024-12-02         | [Release notes v0.9.0](https://github.com/unicef/magasin-drill/releases/tag/releases%2Fv0.9.0) |\n| 0.8.0         | 1.21.2        | 3.9.2             | 2024-09-04         | [Release notes v0.8.0](https://github.com/unicef/magasin-drill/releases/tag/releases%2Fv0.8.0) | \n| 0.7.2         | 1.21.2        | 3.9.2             | 2024-08-07         | [Release notes v0.7.2](https://github.com/unicef/magasin-drill/releases/tag/releases%2Fv0.7.2) |\n| 0.7.1         | 1.21.1        | 3.9.1             | 2024-02-05         | [Release notes v0.7.1](https://github.com/unicef/magasin-drill/releases/tag/releases%2Fv0.7.1) |\n---\n\n\n\n## Pre-requisites\n- A Kubernetes Cluster.\n- [Helm](https://github.com/helm/helm#install) version 3.0 or greater.\n- [Kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) version 1.16.0 or greater.\n\n\n## Install\n\n### Option 1: Using the Helm repository (recommended)\n1. Add the helm repo\n    ```sh\n    helm repo add magasin-drill https://unicef.github.io/magasin-drill/\n    ```\n\n2. Intall the chart \n    ```sh\n    helm install drill magasin-drill/drill --namespace magasin-drill --create-namespace\n    ```\nThis will install Apache Drill in the `magasin-drill` namespace. If you skip `--namespace magasin-drill --create-namespace` the chart is installed in the `default` namespace. \n\n### Option 2: Cloning the repo\n\n1. Clone the repo\n    ```sh\n    git clone https://github.com/unicef/magasin-drill\n    ```\n\n2. Install the chart\n    ```sh\n    cd magasin-drill\n    helm install drill ./charts/drill --namespace magasin-drill --create-namespace\n    ```\n\nThis will install Apache Drill in the `magasin-drill` namespace. If you skip `--namespace magasin-drill --create-namespace` the chart is installed in the `default` namespace. \n\n\n## Usage\nYou can list the pods by running:\n\n```shell\n# kubectl get pods -n \u003cnamespace\u003e\nkubectl get pods -n magasin-drill\nNAME         READY   STATUS    RESTARTS   AGE\ndrillbit-0   1/1     Running   0          5m3s\ndrillbit-1   1/1     Running   0          5m3s\nzk-0         1/1     Running   0          5m3s \n```\n\nOnce all the pods  have the READY value as `1/1` and the `STATUS` is `Running`, you can launch the Drill Web UI using: \n\n```sh\nkubectl port-forward --namespace magasin-drill service/drill-service 8047:8047\n```\nThen open the browser in `http://localhost:8047` to access the Drill Web UI.\n\nOr you can download the [launch_ui.sh script from this repo](https://github.com/unicef/magasin-drill/blob/main/scripts/launch_ui.sh) and run it:\n\n\n```shell\n# ./scripts/launch_ui [-n \u003cnamespace\u003e]\n#  where namespace defaults to \"magasin-drill\"\n./scripts/launch_ui.sh \n```\nIt will output\n```\nForwarding from 127.0.0.1:8047 -\u003e 8047\nForwarding from [::1]:8047 -\u003e 8047\n```\nThen open `http://localhost:8047` in a browser.\n\nThe chart `appVersion` displays the drill and zookeeper versions, respectively.\nFor example:\n\n```shell\nhelm list -n magasin-drill\nNAME \tNAMESPACE    \tREVISION\tUPDATED                            \tSTATUS  \tCHART      \tAPP VERSION \ndrill\tmagasin-drill\t1       \t2023-11-20 09:12:48.88172 +0300 EAT\tdeployed\tdrill-0.7.2\t1.21.2-3.9.2\n```\n\n### Install cloning repo\nAlternatively, you can clone this repo and install the chart from the local directory:\n\n```sh\ngit clone https://github.com/unicef/magasin-drill\ncd magasin-drill\nhelm install drill ./charts/drill --namespace magasin-drill --create-namespace\n```\n\n\n### Customizing the setup\n\nHelm Charts use `values.yaml` for providing default values to 'variables' used in the chart templates.  Refer to the [charts/drill/values.yaml](charts/drillvalues.yaml) file for details on default values for the charts.\n\nYou can create a `values.yaml`` file to overrides some of the default values. For example, by default the helm chart launches 2 drillbits, but you can set it to 1.\n\n```yaml\n# values.yaml\ndrill:\n  count: 1\n```\n\nThen you add the `-f filename.yaml` or `--values filename.yaml` to the install command earlier \n\n```sh\nhelm install drill magasin-drill/drill -f values.yaml -n magasin-drill --create-namespace\n```\n\nIf the setup already exist you can use `upgrade`:\n\n```sh\nhelm upgrade drill magasin-drill/drill -n magasin-drill -f values.yaml\nRelease \"drill\" has been upgraded. Happy Helming!\nNAME: drill\nLAST DEPLOYED: Wed Nov  8 13:41:10 2023\nNAMESPACE: magasin-drill\nSTATUS: deployed\nREVISION: 2\nTEST SUITE: None\n```\n\nNow, the cluster only has a drillbit:\n```shell\nkubectl get pods -n magasin-drill\nNAME         READY   STATUS    RESTARTS       AGE\ndrillbit-0   0/1     Running   1 (22s ago)   156m\nzk-0         1/1     Running   0              157m\n```\n\n\n### Override the storage plugin configuration\n\nIt is possible to deploy drill with a custom set of storage plugins (ie. connected to a set of storages such as Azure Blobs, S3 Buckets, etc.).\n\nTo do that:\n\n1. Edit the configuration file: `storage-plugins-override.conf`.\n\n2. Enable the storage plugin config overriding by setting to `true` the `drill.volumes.drillStorage.override` section in your values file `values.yaml` file.\n\n```yaml\n# values.yaml\ndrill:\n  volumes: \n    drillStorage:\n      override: true \n```\n\n3. Create the `drill-storage-plugin-secret` secret. When a Drill chart is deployed, the files contained within this secret will be downloaded to each container and used by the drill-bit process during start-up. You can create the secret by running the script:\n\n```shell\n# ./scripts/create_secret.sh \u003cnamespace\u003e \n./scripts/create_secret.sh -n magasin-drill -f \u003cconfig-file\u003e\n```\nor manually\n\n```shell\nkubectl create secret generic drill-storage-plugin-secret --from-file=storage-plugins-override.conf --namespace magasin-drill\n```\n\n\n#### Override drill config\nYou can enable config overriding by modifying the values of the `drill.volumes.configOverrides` in your customized `values.yaml` file. Example:\n\n\n```yaml\n# values.yaml\ndrill:\n  volumes:\n    # Make true if you want to add any volumes. Else, make false.\n    add: true\n    # Drill Configuration can be overridden by mounting them on each Drill container.\n    configOverrides:\n      # Contents of drill-override.conf\n      # This file follows the HOCON format\n      # https://github.com/lightbend/config/blob/master/HOCON.md\n      # The available options are described: \n      # # https://drill.apache.org/docs/configuration-options-introduction/\n      #\n      # Alternatively on the web UI you can run the query:\n      #\n      # SELECT * FROM sys.boot;\n      #\n      drillOverrideConf: |\n        drill.exec.options.exec.errors.verbose: true\n      # Contents of drill-env.sh \n      drillEnvSh: |\n        echo \"Running drill-env.sh...\"\n        export DRILL_PID_DIR=\"/opt/drill\"\n        ps -ax\n        env \n        #...\n \n\n```\n\n#### Allow access to Drill web UI from outside of the cluster.\n\nBy default, you can only access the Drill Web UI by performing a port forward to your localhost `using scripts/launch_ui.sh` or using `kubectl`:\n\n`kubectl port-forward --namespace \u003cnamespace\u003e service/drill-service 8047:8047`\n\nFor cloud based deployments, you can setup a `LoadBalancer` type [service](https://kubernetes.io/docs/concepts/services-networking/service/). This allows to access the drill web UI from outside the cluster through a public IP. Basically, this type of service acts like a proxy which internally redirects the HTTP requests to any of the available drill pods.\n\nTo enable the service, modify the flag `exposeWebService` in your `values.yaml` in the `drill` section.\n\n```yaml\n# values.yaml\ndrill:\n  #...\n  #...\n  exposeWebService: true\n  webServiceType: LoadBalancer\n  #...\n  #...\n```\nand proceed to upgrade the setup.\n\nAlternatively, you can upgrade your setup just changing the value\n```shell\n helm upgrade drill magasin-drill/drill -n magasin-drill --set drill.exposeWebService=true\n```\nNow the drill-web-svc shall appear. \n```shell\n kubectl get services -n magasin-drill\nNAME            TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)                                  AGE\ndrill-service   ClusterIP      10.96.253.218   \u003cnone\u003e          8047/TCP,31010/TCP,31011/TCP,31012/TCP   28s\ndrill-web-svc   LoadBalancer   10.96.20.170    23.16.193.208   8047:31889/TCP,31010:31926/TCP           28s\nzk-service      ClusterIP      10.96.54.14     \u003cnone\u003e          2181/TCP,2888/TCP,3888/TCP               2\n```\n\nYou can open http://[external-ip]:8047 to access the cluster.\n\nExposing the cluster externally is **not recommended without enabling the authentication**. Anyone will be able to access your drill!!! \n\nYou can read more on [drill documentation about how to enable authentication](https://drill.apache.org/docs/securing-drill-introduction/)\n\n\n### Deploy multiple drill clusters\nKubernetes [Namespaces](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/) can be used when more that one Drill Cluster needs to be created. The `default` namespace is used by default. \n\n```shell\n# helm install \u003crelease-name\u003e repo/chart --namespace \u003cnamespace\u003e --create-namespace\nhelm install drill2 magasin-drill/drill --namespace magasin-drill2 --create-namespace\n```\n\nNote that installing the Drill Helm Chart also installs the dependent Zookeeper chart. So with current design, for each instance of a Drill cluster includes a single-node Zookeeper.\n\n```shell\nkubectl get pods -n magasin-drill2\nNAME                       READY   STATUS    RESTARTS   AGE\ndrill2bit-0                1/1     Running   0          51s\ndrill2bit-1                1/1     Running   0          51s\nzk-0                       1/1     Running   0          51s\n```\n\nYou can use the `launch_ui.sh` script with the `-n \u003cnamespace\u003e` as argument\n\n```shell\n./scripts/launch_ui.sh -n magasin-drill2\n```\n\n### Upgrading Drill Charts\n\nCurrently only scaling up/down the number of Drill pods is supported as part of Helm Chart upgrades. To resize a Drill Cluster, edit the `values.yaml` file and apply the changes as below:\n\n\n```shell\n# \nhelm upgrade drill1 magasin-drill/drill -f values.yaml\n\n# If cloned this repo\n# helm upgrade \u003crelease-name\u003e repo/chart/\nhelm upgrade drill1 magasin-drill/drill\n```\nAlternatively, provide the count as a part of the `upgrade` command:\n\n```shell\n# helm upgrade \u003crelease-name\u003e repo/chart --set drill.count=2\nhelm upgrade drill1 magasin-drill/drill --set drill.count=2\n```\n\n### Autoscaling Drill Clusters\nThe size of the Drill cluster (number of Drill Pod replicas / number of drill-bits) can not only be manually scaled up or down as shown above, but can also be autoscaled to simplify cluster management. When enabled, with a higher CPU utilization, more drill-bits are added automatically and as the cluster load goes down, so do the number of drill-bits in the Drill Cluster. The drill-bits deemed excessive [gracefully shut down](https://drill.apache.org/docs/stopping-drill/#gracefully-shutting-down-the-drill-process), by going into quiescent mode to permit running queries to complete.\n\nEnable autoscaling by editing the drill.autoscale section in `charts/drill/values.yaml` file.\n\n```yaml\n# values.yaml\ndrill:\n  #...\n  #..\n  autoscale:\n    # Flag to turn-on / turn-off this option\n    enabled: true\n    # Maximum number of Drill Pods\n    maxCount: 4\n    # Target CPU Utilization Percentage\n    cpuThreshold: 75\n```\n\nIf autoscaling is enabled:\n\n```shell\n# helm upgrade \u003crelease-name\u003e repo/chart --set drill.count=\u003cnew-min-count\u003e --set drill.autoscale.maxCount=\u003cnew-max-count\u003e\nhelm upgrade drill1 magasin-drill/drill --set drill.count=3 --set drill.autoscale.maxCount=6\n```\n\n\n### Package\nDrill Helm Charts can be packaged for distribution as follows:\n\n```sh\nhelm package charts/drill/\nSuccessfully packaged chart and saved it to: /xxx/magasin-drill/drill-1.0.0.tgz\n```\n\n### Uninstall\nDrill Helm Charts can be uninstalled as follows: \n\n```shell\n# helm [uninstall|delete] \u003cHELM_INSTALL_RELEASE_NAME\u003e -n \u003cnamespace\u003e\nhelm delete drill1\nhelm delete drill2\n```\nNote that `LoadBalancer` and a few other Kubernetes resources may take a while to terminate. Before re-installing Drill Helm Charts, please make sure to wait until all objects from any previous installation (in the same namespace) have terminated.\n\n## Building custom docker images\n\nThe [`docker/`](docker/) directory contains everything required to build and customize the Docker images of Apache Drill and Apache ZooKeeper included in the helm chart.\n\nAlready built docker images are available in [Docker Hub](https://hub.docker.com/u/merlos/). \n\nOnce you have updated the images, you need to replace the 'image' option in your `values.yaml` file.\n\nTo update the drill image edit the `drill.image` value:\n\n```yaml\ndrill:\n  #...\n  # Change the line below with your custom image\n  image: merlos/drill:1.21.1-multi-arch \n```\n\nAnd, in the zookeeper section:\n```yaml\nzookeper: \n  #...\n  #...\n  # Change the line below with your custom image\n  image: merlos/zookeeper:3.9.1-multi-arch\n```\n\n\n## Troubleshooting\n\nAssuming we have installed the helm chart in the `magasin-drill` namespace we'll explore how to fix an issue\n\nCheck the health of the pods:\n\n```shell\nkubectl get pods -n magasin-drill\nNAME         READY   STATUS             RESTARTS   AGE\ndrillbit-0   0/1     Running            0          4m25s\ndrillbit-1   0/1     Running            0          4m25s\nzk-0         1/1     Running            0          4m5s\n```\nIf after a few min the status is not Running as well as the readiness is \"1/1\", then you may be spotting a problem.\n\nIn that case, it is good to expand on the status of the pod. To do that you can use the describe pod kubectl command.\n\nGenerally you'll need to take a deeper look to the Events section:\n\n```shell\n# kubectl describe pod \u003cpodname\u003e -n \u003cnamespace\u003e \nkubectl describe pod drillbit-0 -n magasin-drill\nName:             drillbit-0\nNamespace:        magasin-drill\nPriority:         0\nService Account:  drill-sa\n...\n...\n...\n\nEvents:\n  Type     Reason     Age                From               Message\n  ----     ------     ----               ----               -------\n  Normal   Scheduled  2m                 default-scheduler  Successfully assigned magasin-drill/drillbit-0 to aks-systempool-32354579-vmss00000o\n  Normal   Pulling    2m                 kubelet            Pulling image \"busybox\"\n  Normal   Pulled     119s               kubelet            Successfully pulled image \"busybox\" in 756.658172ms (756.671572ms including waiting)\n  Normal   Created    119s               kubelet            Created container zk-available\n  Normal   Started    119s               kubelet            Started container zk-available\n  Normal   Pulling    118s               kubelet            Pulling image \"merlos/drill:1.21.1\"\n  Normal   Pulled     117s               kubelet            Successfully pulled image \"merlos/drill:1.21.1\" in 730.646907ms (730.656407ms including waiting)\n  Normal   Created    117s               kubelet            Created container drill-pod\n  Normal   Started    117s               kubelet            Started container drill-pod\n  Warning  Unhealthy  30s (x3 over 90s)  kubelet            Liveness probe failed: Running drill-env.sh...\n/opt/drill/drillbit.pid file is present but drillbit is not running.\n[WARN] Only DRILLBIT_MAX_PROC_MEM is defined. Auto-configuring for Heap \u0026 Direct memory\n[INFO] Attempting to start up Drill with the following settings\n  DRILL_HEAP=1G\n  DRILL_MAX_DIRECT_MEMORY=3G\n  DRILLBIT_CODE_CACHE_SIZE=768m\n  Warning  Unhealthy  25s (x4 over 90s)  kubelet  Readiness probe failed: Running drill-env.sh...\n/opt/drill/drillbit.pid file is present but drillbit is not running.\n[WARN] Only DRILLBIT_MAX_PROC_MEM is defined. Auto-configuring for Heap \u0026 Direct memory\n[INFO] Attempting to start up Drill with the following settings\n  DRILL_HEAP=1G\n  DRILL_MAX_DIRECT_MEMORY=3G\n  DRILLBIT_CODE_CACHE_SIZE=768m\n\n```\n\nIn this case, among the info, we see the line bellow, that tells us that the drillbit is not running. \n\n```\n/opt/drill/drillbit.pid file is present but drillbit is not running.\n```\n\nBut this is not enough info. Another thing we can try is to display the logs:\n\n```shell\n# kubectl logs \u003cpod\u003e -n \u003cnamespace\u003e -f\nkubectl logs drillbit-0 -n magasin-drill -f\n\nDefaulted container \"drill-pod\" out of: drill-pod, zk-available (init)\nsed: cannot rename /opt/drill/conf/sedQRvtNI: Device or resource busy\nsed: cannot rename /opt/drill/conf/sed6kHmUI: Device or resource busy\nRunning drill-env.sh...\n[WARN] Only DRILLBIT_MAX_PROC_MEM is defined. Auto-configuring for Heap \u0026 Direct memory\n[INFO] Attempting to start up Drill with the following settings\n  DRILL_HEAP=1G\n  DRILL_MAX_DIRECT_MEMORY=3G\n  DRILLBIT_CODE_CACHE_SIZE=768m\nStarting drillbit, logging to /var/log/drill/drillbit.out\nRunning drill-env.sh...\n[WARN] Only DRILLBIT_MAX_PROC_MEM is defined. Auto-configuring for Heap \u0026 Direct memory\n[INFO] Attempting to start up Drill with the following settings\n  DRILL_HEAP=1G\n  DRILL_MAX_DIRECT_MEMORY=3G\n  DRILLBIT_CODE_CACHE_SIZE=768m\ndrillbit is running.\n\n```\n\nThe `-f` argument will display new logs continuously. In this case, these logs do not give us any additional clue. So, next step is to look inside of the pod itself:\n\n```sh\n# kubectl exec \u003cpod\u003e -n \u003cnamespace\u003e -ti -- \u003ccommand\u003e \nkubectl exec drillbit-0 -n magasin-drill -ti -- /bin/bash\n```\n\nThis command will launch a shell within the pod. `-ti` is to allow interactive mode.\n\nWithin the shell we can explore the `/opt/drill/conf` and `/var/log/drill/` folders.\n\n```sh\n[root@drillbit-0 /]# pwd\n/\n[root@drillbit-0 /]# cd /opt/drill/conf/\n[root@drillbit-0 conf]# ls\ncore-site-example.xml  drill-metastore-override-example.conf  logback.xml\ndistrib-env.sh         drill-on-yarn-example.conf             saffron.properties\ndistrib-setup.sh       drill-override-example.conf            storage-plugins-override-example.conf\ndrill-am-log.xml       drill-override.conf                    yarn-client-log.xml\ndrill-distrib.conf     drill-setup.sh\ndrill-env.sh           drill-sqlline-override-example.conf\n[root@drillbit-0 conf]# \n```\n\nThe `drill-env.sh` is the custom script that is run prior to launching the pod. You can check that the contents are the ones you set in `values.yaml`.\nThe `drill-override.conf` is the custom configuration you set in `values.yaml`.\n\nIn the logs folder we have two files:\n```\nroot@drillbit-0 conf]# cd /var/log/drill/\n[root@drillbit-0 log]# ls\ndrillbit.log  drillbit.out  drillbit_queries.json\n[root@drillbit-0 log]# \n```\n\nWe can explore `drillbit.log` and `drillbit.out`\n\n```shell\n[root@drillbit-0 log]# cat drillbit.out \nOpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N\nException in thread \"main\" com.typesafe.config.ConfigException$Parse: drill-override.conf @ file:/opt/drill/conf/drill-override.conf: 5: unbalanced close brace '}' with no open brace (if you intended '}' to be part of a key or string value, try enclosing the key or value in double quotes)\n\tat com.typesafe.config.impl.ConfigDocumentParser$ParseContext.parseError(ConfigDocumentParser.java:201)\n\tat com.typesafe.config.impl.ConfigDocumentParser$ParseContext.parseError(ConfigDocumentParser.java:197)\n\tat com.typesafe.config.impl.ConfigDocumentParser$ParseContext.parseObject(ConfigDocumentParser.java:434)\n\tat com.typesafe.config.impl.ConfigDocumentParser$ParseContext.parse(ConfigDocumentParser.java:648)\n\tat com.typesafe.config.impl.ConfigDocumentParser.parse(ConfigDocumentParser.java:14)\n\tat com.typesafe.config.impl.Parseable.rawParseValue(Parseable.java:262)\n\tat com.typesafe.config.impl.Parseable.rawParseValue(Parseable.java:250)\n\tat com.typesafe.config.impl.Parseable.parseValue(Parseable.java:180)\n\tat com.typesafe.config.impl.Parseable.parseValue(Parseable.java:174)\n\tat com.typesafe.config.impl.Parseable.parseValue(Parseable.java:309)\n\tat com.typesafe.config.impl.Parseable$ParseableResources.rawParseValue(Parseable.java:738)\n\tat com.typesafe.config.impl.Parseable$ParseableResources.rawParseValue(Parseable.java:701)\n\tat com.typesafe.config.impl.Parseable.parseValue(Parseable.java:180)\n\tat com.typesafe.config.impl.Parseable.parseValue(Parseable.java:174)\n\tat com.typesafe.config.impl.Parseable.parse(Parseable.java:152)\n\tat com.typesafe.config.impl.SimpleIncluder.fromBasename(SimpleIncluder.java:172)\n\tat com.typesafe.config.impl.ConfigImpl.parseResourcesAnySyntax(ConfigImpl.java:133)\n\tat com.typesafe.config.ConfigFactory.parseResourcesAnySyntax(ConfigFactory.java:1083)\n\tat com.typesafe.config.ConfigFactory.load(ConfigFactory.java:118)\n\tat com.typesafe.config.ConfigFactory.load(ConfigFactory.java:79)\n\tat org.apache.drill.common.config.DrillConfig.create(DrillConfig.java:270)\n\tat org.apache.drill.common.config.DrillConfig.create(DrillConfig.java:165)\n\tat org.apache.drill.common.config.DrillConfig.create(DrillConfig.java:149)\n\tat org.apache.drill.exec.server.Drillbit.start(Drillbit.java:556)\n\tat org.apache.drill.exec.server.Drillbit.main(Drillbit.java:552)\n[root@drillbit-0 log]# command terminated with exit code 137\n```\n\nThere is one line:\n\n```\nException in thread \"main\" com.typesafe.config.ConfigException$Parse: drill-override.conf @ file:/opt/drill/conf/drill-override.conf: 5: unbalanced close brace '}' with no open brace (if you intended '}' to be part of a key or string value, try enclosing the key or value in double quotes)\n```\n\nIn this case the line tells us that there is a `}` in excess. Let's check it in the file:\n\n```sh\n[root@drillbit-0 /]# cat /opt/drill/conf/drill-override.conf \n\nexec.errors: { \n    verbose: true\n}\n}\n[root@drillbit-0 /]# \n```\nEffectively that is the case.\nNow, we just need to update the `values.yaml` file and install again the chart.\n\n### Error: ImagePullBackOff in minikube\n\nFor testing purposes you may be running your cluster in [minikube](https://minikube.sigs.k8s.io/). We have experimented  `Error: ImagePullBackOff` errors. \n\nFor example after running\n\n```shell\nkubectl describe pods zk-0 -n magasin-drill\n```\n\nYou may see something like:\n```\nEvents:\n  Type     Reason     Age                  From               Message\n  ----     ------     ----                 ----               -------\n  Normal   Scheduled  2m40s                default-scheduler  Successfully assigned magasin-drill/zk-0 to minikube\n  Warning  Failed     37s                  kubelet            Failed to pull image \"merlos/zookeeper:3.9.1\": rpc error: code = Unknown desc = context deadline exceeded\n  Warning  Failed     37s                  kubelet            Error: ErrImagePull\n  Normal   BackOff    37s                  kubelet            Back-off pulling image \"merlos/zookeeper:3.9.1\"\n  Warning  Failed     37s                  kubelet            Error: ImagePullBackOff\n  Normal   Pulling    22s (x2 over 2m40s)  kubelet            Pulling image \"merlos/zookeeper:3.9.1\"\n\n```\nIn this case, you can fix the issue by pulling the image manually using the following command:\n\n```shell\n# minikube image pull \u003cimage-name\u003e\nminikube image pull merlos/zookeeper:3.9.1\nminikube image pull merlos/drill:1.21.1\n```\nThese will download the images locally so they'll be available. Use `minikube image ls` to view the images available and `minikube image remove \u003cimage-name\u003e` to remove them.\n\n## The chart structure\nApache Drill Helm charts are organized as a collection of files inside of the `charts/drill/` directory. As Drill depends on [Apache Zookeeper](https://zookeeper.apache.org/) for cluster co-ordination, a zookeeper chart is inside the dependencies folder ['charts/drill/charts'](charts/drill/charts/). The Zookeeper chart follows a similar structure as the Drill chart.\n\n```\ncharts/drill/   \n  Chart.yaml    # A YAML file with information about the chart\n  values.yaml   # The default configuration values for this chart\n  charts/       # A directory containing the Zookeeper (zk) charts\n  templates/    # A directory of templates, when combined with values, will generate valid Kubernetes manifest files\n```\n\n#### Templates\nHelm Charts contain `templates` which are used to generate Kubernetes manifest files. These are YAML-formatted resource descriptions that Kubernetes can understand. These templates contain 'variables', values for which  are picked up from the `values.yaml` file.\n\nDrill Helm Charts contain the following templates:\n```\ncharts/drill/\n  ...\n  templates/\n    drill-rbac-*.yaml       # To enable RBAC for the Drill app\n    drill-service.yaml      # To create a Drill Service\n    drill-web-service.yaml  # To expose Drill's Web UI externally using a LoadBalancer. Works on cloud deployments only. \n    drill-statefulset.yaml  # To create a Drill cluster\n  charts/\n    zookeeper/\n      ...\n      templates/\n        zk-rbac.yaml        # To enable RBAC for the ZK app\n        zk-service.yaml     # To create a ZK Service\n        zk-statefulset.yaml # To create a ZK cluster. Currently only a single-node ZK (1 replica) is supported\n```\n\n## Contributing\n\nWe welcome contributions. To contribute, please fork this repository, make your changes, and submit a pull request.\n\nIf new values are added to the `values.yaml` file include the documentation on how to use them.\n\nIf you find a bug or have a feature request, please open an issue.\n\n\n\n## License \n\nApache License 2.0 UNICEF.org\n\nNote tha this is an updated version of the original: https://github.com/Agirish/drill-helm-charts\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Funicef%2Fmagasin-drill","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Funicef%2Fmagasin-drill","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Funicef%2Fmagasin-drill/lists"}