Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/metabase/mba

metabase assembler
https://github.com/metabase/mba

Last synced: about 1 year ago
JSON representation

metabase assembler

Awesome Lists containing this project

README 

          
* Overview

  Metabase Assembler is a development and repro tool based on

  docker-compose.

* Dependencies

  - docker & docker-compose

  - [[https://github.com/babashka/babashka/][babashka]] 2.12 or higher.



* Install

  - install babashka. [[https://github.com/babashka/babashka/releases/][Download]]

  - install docker && docker-compose

  - git clone https://github.com/metabase/mba

  - If you want it in your path: =ln -s ./mba/src/main.clj ~/bin/mba=

  - =mba --help= or =./mba/src/main.clj --help=



* BugBash Instructions

  If you're bugbashing:

  - =mba --mb dockerhub:metabase/metabase-head pull=. Will pull latest

    metabase-head image.

  - =mba --mb dockerhub:metabase/metabase-head up=. Will spin metabase linked to

    a postgresql and maildev. Open `localhost:3000` and `localhost:1080`.

  - When you're done, =mba --mb dockerhub:metabase/metabase-head down=. containers

    and data will be wiped.

  - Emergency hatch: =mba nuke= will kill everything mba related.



* Usage & Commands



  - =mba up= . creates the stack

  - =mba pull= . pulls images from the stack. (useful if you use =*-head=)

  - =mba shell=. gives a shell inside the main metabase container

  - =mba dbconsole=. console into the app-db

  - =mba run ls=. runs one-off commands in the mb container

  - =mba logs=. show logs

  - =mba compose logs -- --tail=100 -f=. same, using raw compose command

  - =mba logs postgres=. show logs and tail of postgres container

  - =mba compose logs -- --tail=100 -f postgres=. same, using raw compose command

  - =mba down=. destroys the stack

  - =mba ps=. show all mba containers running in the system.

  - =mba compose ps=. show running containers from this stack

  - =mba compose ...=. proxy to all docker-compose commands

  - =mba compose config=. show generated yml

  - =mba nuke=. kill all mba containers dead



  mba wraps docker-compose, and using the command =compose=, it will pass all

  further commands to the underlying docker-compose.



  =mba compose config= is equivalent to =docker-compose -f tmp.yml config= and

  will show the yml file that is generating with your current flags.



  By default, it will share ports. ==--publish false= will jail the

  whole stack not publishing any port.



  If current directory is not a metabase source dir, by default it

  will use dockerhub's =metabase/metabase= image. This WILL start

  automatically.



  If it is ran in a metabase source directory, it will run the

  .devcontainer image, so you should shell into it and run the

  commands manually to start the app.



  SUPERHACK: ENV vars are passed around at the moment, to have the commands available

  #+begin_src bash

    mba up

    mba shell

    lein repl     # or `eval $MBA_CLI`. lein repl as cider-jack-in would call it with

                  # fixed host 0.0.0.0 and port 7888 (which is the exported port by mba)



    mba shell

    yarn build-hot # or `eval $MBA_YARN_BUILD`, as a tunnable command to

                   # run the proper yarn build. It used to be a place to

                   # put special flags like --host 0.0.0.0, but now the

                   # flag is merged in master



  #+end_src

  so hacky, amazing.



  - mba --help may help you with the commands and option, but for the

    values, explore src/main.clj



  - to pass flags to docker-compose , or any command underlying mba,

    the way to pass as a parameter to inside is with =--=: =mba

    --app-db postgres compose logs -- -f=



* Flags

  **tl;dr**:

  #+begin_src bash

    mba --mb ~/prj/metabase \

        --app-db postgres:13 \

        --data-db mariadb \

        --proxy nginx\

        --env FOO=bar\

        --network n1\

        --prefix mb1\

        up

  #+end_src

** --mb / -M

   Main metabase app source or image.



   If mba runs in a metabase source directory, it will default to the current

   directory. Else, it defaults to dockerhub:metabase/metabase-enterprise docker

   image.



   Set it explicitly like the following:



   - =mba --help= will show the options accepted by mba.

   - =cd ~/src/metabase; mba -M ./=

   - =mba --mb ~/src/metabase/=

   - =mba --mb file:~/src/metabase/=

   - =mba --mb docker:metabase/metabase:v0.37=

   - =mba --mb git:ref=  <- not implemented



** --app-db / -d

   App db for metabase. Defaults to =postgres=. Valid options are:



   - =postgresql=

   - =postgres=

   - =postgres:13= Or wtv version

   - =mysql=

   - =mariadb=

   - =h2=



** --data-db / -D

   Datawarehouse db. Defaults to nil. Valid options are:



   - =postgresql=

   - =mariadb=

   - =mysql=

   - =mongodb=

   - =vertica=

   - =presto=

   - =sparksql=

   - =sqlserver=



** --proxy

    Proxy infrastructure. Defaults to nil. Valid options are:



   - =nginx=

   - =haproxy=

   - =envoy=



** --env / -e

   Injects env variables to the metabase container. Can be present

   multiple times. For example, this would start a metabase image with

   a custom h2 file:



  #+begin_src bash

   mba --mb dockerhub:metabase/metabase\

       --env MB_DB_FILE=/root/testdb \

       --app-db h2\

       up

  #+end_src



  Remember that =/root/testdb= means =~/.mba/.mba-home/= from your

  host.



** --publish

   Publish ports. Boolean. Defaults to true.



** --prefix / -p

   stack name. Defaults to "mba-[somehash]". Useful when running multiple

   instances.



** --network / -n

   Network name. By default it creates an anonymous network per stack,

   Fixing it is useful to make many metabase join the same network.



* ports

  By default, the stack will share ports:

  - 3000: backend

  - 8080: frontend

  - 7888: for lein repl

  - 1025: maildev mail port

  - 1080: maildev http

  - 8081 if --proxy is set



* networks

   By default it will create a new unnamed network, but we can tell

   mba to create a named network with =-n foo=. This and

   =docker-compose run= allows us to run different mb versions in the

   same environment to test upgrades and the like.



   #+begin_src bash

     # this will start the "main" one

     mba --mb docker:metabase/metabase:v0.37.9 --app-db postgres -n n1 up

     # attach a new container to the network. Can run migrations.

     # or hit http://metabase:3000/api/session/properties

     mba --mb file:~/workspace/metabase --app-db postgres -n n1 run metabase bash

   #+end_src



   Note: you can't =run bash= on a docker image that we provide (see

   https://github.com/metabase/metabase/issues/14528), but you can run

   normally, and then exec into it and run sh. That works ok.



* Troubleshooting

** I'm using docker:metabase/metabase and mba is using an old version

   Use =mba --mb docker:metabase/metabase:v0.37.9 up= or use the

   =pull= command: =mba --mb docker:metabase/metabase pull=



* modifications done (or to be done) to make it work as a dev env

** clojure

Add :port and :host.



  #+begin_src bash

  lein repl :headless :host 0.0.0.0 :port 7888

  #+end_src



** How to see actual Metabase queries sent to the database



  #+begin_src bash

   ./src/main.clj --app-db postgres --data-db postgres logs postgres | grep -A 5 "queryHash"

  #+end_src



** cypress

   TODO



** how to see what is the actual compose being built

  #+begin_src bash

   ./src/main.clj --data-db postgres --app-db postgres compose config | less

  #+end_src



* See also

  - Mba is heavily inspired on [[https://github.com/Kong/gojira][Gojira]]. Nearly an exact port, molded to

    cover a slightly different usecase.

  - https://evilmartians.com/chronicles/reusable-development-containers-with-docker-compose-and-dip

  - https://danlebrero.com/2017/09/25/how-do-docker-compose-development-environement/

  - https://opensource.com/article/20/4/how-containerize-build-system

  - https://github.com/microsoft/vscode-dev-containers/tree/master/containers/java-8

  - https://earthly.dev/