{"id":29953718,"url":"https://github.com/rumbledb/rumbledb-tutorial","last_synced_at":"2026-02-14T19:02:51.918Z","repository":{"id":88572944,"uuid":"548936236","full_name":"RumbleDB/rumbledb-tutorial","owner":"RumbleDB","description":"Tutorial for Declarative Amsterdam 2022","archived":false,"fork":false,"pushed_at":"2022-11-07T07:34:17.000Z","size":63,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-08-27T04:29:16.449Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JSONiq","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RumbleDB.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-10-10T12:20:30.000Z","updated_at":"2022-11-04T14:44:17.000Z","dependencies_parsed_at":"2023-03-13T18:22:45.126Z","dependency_job_id":null,"html_url":"https://github.com/RumbleDB/rumbledb-tutorial","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/RumbleDB/rumbledb-tutorial","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RumbleDB%2Frumbledb-tutorial","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RumbleDB%2Frumbledb-tutorial/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RumbleDB%2Frumbledb-tutorial/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RumbleDB%2Frumbledb-tutorial/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RumbleDB","download_url":"https://codeload.github.com/RumbleDB/rumbledb-tutorial/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RumbleDB%2Frumbledb-tutorial/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29452590,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-14T15:52:44.973Z","status":"ssl_error","status_checked_at":"2026-02-14T15:52:11.208Z","response_time":53,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-03T15:10:53.375Z","updated_at":"2026-02-14T19:02:51.913Z","avatar_url":"https://github.com/RumbleDB.png","language":"JSONiq","funding_links":[],"categories":[],"sub_categories":[],"readme":"# RumbleDB tutorial - Declarative Amsterdam 2022\n\n## Update: the links used in the slides\n\n\nRelease page with the jar: [https://github.com/RumbleDB/rumble/releases](https://github.com/RumbleDB/rumble/releases)\n\nPattern of pre-stored query solutions: [http://www.rumbledb.org/samples/1.jq](http://www.rumbledb.org/samples/1.jq)\n\nSmall dataset: [https://www.rumbledb.org/samples/git-archive.json](https://www.rumbledb.org/samples/git-archive.json)\n\nLarger dataset: [https://www.rumbledb.org/samples/git-archive-big.json](https://www.rumbledb.org/samples/git-archive-big.json)\n\nHuger dataset URI pattern: [https://data.gharchive.org/2022-11-01-0.json.gz](https://data.gharchive.org/2022-11-01-0.json.gz)\n\nGreat language game: [https://lars.yencken.org/datasets/great-language-game/](https://lars.yencken.org/datasets/great-language-game/)\n\n## What is RumbleDB?\n\nRumbleDB is an engine that allows you to query large, messy datasets with JSONiq.\n\nIt works on your laptop, smartly spreading computations on all your cores (by encapsulating Apache Spark in a way that is hidden from the user), but also works on a large cluster of machines (3, or 5, or 10, or 50, or 100, or 1000...) in the cloud.\n\nOn your laptop, it works with very small files in kB, all the way to 100s of GBs.\nOn a cluster, it works with very small files in kB, all the way to 100s of TBs.\n\nIt supports JSON, CSV, Parquet, Avro, ...\n\nIn this tutorial, we will simply work with your laptop.\n\n## Installation notes\n\nRumbleDB can be installed in four different ways:\n- As a standalone Java jar that contains all it needs (for use on a laptop)\n- As a thin jar for use with an existing Apache Spark installation (for use on a laptop or a cluster)\n- As a docker image (for use on a laptop)\n- As a brew recipe (for use on a laptop running macOS)\n\nFor this tutorial, we will use the standalone Java jar, because this is the most straightforward way to get started.\n\n## Installing Java\n\nFirst, you need to install Java. We will use Java 11.\n\n### macOS\n\nOn macOS, we recommend using the Adoptium distribution, which has an Apache 2.0 license. The Adoptium distribution can be installed from [this page](https://adoptium.net/en-GB/temurin/releases/?version=11). You need to download the .pkg version that corresponds to macOS and your type of CPU. For this you can use the filtering dropdowns by selecting \"macOS\" as the operating system, either x64 (for Intel-based Macs) or aarch64 (for the newer M1 or M2 Macs) as the architecture, \"JDK\" as the package type, and \"11\" as the version.\n\nOnce you have finished the installation, you can check that the correct version of Java is running:\n\n    java -version\n   \nwhich should display something like this (what is important is the 11, and not 8 or 17):\n\n    openjdk version \"11.0.16.1\" 2022-08-12\n    OpenJDK Runtime Environment Temurin-11.0.16.1+1 (build 11.0.16.1+1)\n    OpenJDK 64-Bit Server VM Temurin-11.0.16.1+1 (build 11.0.16.1+1, mixed mode)\n\nIn case a different version is displayed, you need to point JAVA_HOME to the right directory. For macOS, you can run:\n\n    export JAVA_HOME=`/usr/libexec/java_home -v 11` \n\nAnd then run\n\n    java -version\n    \nagain to check that this worked.\n\n### Ubuntu\n\nIn ubuntu, you can use the package manager like so:\n\n    sudo apt install openjdk-11-jdk\n    \nThen set JAVA_HOME with:\n\n    export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64\n    \nAnd then run\n\n    java -version\n    \nagain to check that this worked (it should say 11, not 8 or 17).\n\n### Windows\n\nOn Windows, we recommend using the Adoptium distribution, which has an Apache 2.0 license. The Adoptium distribution can be installed from [this page](https://adoptium.net/en-GB/temurin/releases/?version=11). You need to download the .msi file for Windows (x64). For this you can use the filtering dropdowns by selecting \"Windows\" as the operating system, x64 as the architecture (except if you have a very old laptop running in 32-bit mode), \"JDK\" as the package type, and \"11\" as the version.\n\nSimply run the installer and make sure to activate the setup of the PATH and JAVA_HOME directories\n\nOnce you have finished the installation, you can check that the correct version of Java is running by running in a command window:\n\n    java -version\n   \nwhich should display something like this (what is important is the 11, and not 8 or 17):\n\n    openjdk version \"11.0.16.1\" 2022-08-12\n    OpenJDK Runtime Environment Temurin-11.0.16.1+1 (build 11.0.16.1+1)\n    OpenJDK 64-Bit Server VM Temurin-11.0.16.1+1 (build 11.0.16.1+1, mixed mode)\n    \nagain to check that this worked.\n\n## Download the RumbleDB jar (all systems)\n\nThe RumbleDB jar can directly be downloaded from [this place](https://github.com/RumbleDB/rumble/releases/download/v1.19.0/rumbledb-1.19.0-standalone.jar).\n\nJust put it in a directory of your choice, specially created for the tutorial. In this directory, we will also put datasets and a jupyter notebook.\n\n## Checking that the RumbleDB jar gets correctly executed (all systems)\n\nTo check that it works, open a command window, go to the directory, and try to run the following command:\n\n    java -jar rumbledb-1.19.0-standalone.jar\n    \nwhich should do nothing else than display get-started instructions:\n\n        ____                  __    __     ____  ____ \n       / __ \\__  ______ ___  / /_  / /__  / __ \\/ __ )\n      / /_/ / / / / __ `__ \\/ __ \\/ / _ \\/ / / / __  |  The distributed JSONiq engine\n     / _, _/ /_/ / / / / / / /_/ / /  __/ /_/ / /_/ /   1.19.0 \"Tipuana Tipu\" beta\n    /_/ |_|\\__,_/_/ /_/ /_/_.___/_/\\___/_____/_____/  \n    \n    \n    \n    RumbleDB is a JSONiq engine that can be used both on your laptop or on a\n    cluster (e.g. with Amazon EMR or Azure HDInsight).\n    \n    This is the standalone jar that does not require the installation of Spark.\n    \n    If you need more control over Spark or use it on a cluster, we recommend using\n    the leaner jars instead, which you can download from www.rumbledb.org.\n    \n    If you do not want to install Spark, then you need to use the standalone jar\n    instead from www.rumbledb.org.\n    \n    Usage:\n    jar -jar \u003cpath to RumbleDB's jar\u003e \u003cmode\u003e \u003cparameters\u003e\n    \n    The first optional argument specifies the mode:\n    **** run ****\n    for directly running a query from an input file or (with -q) provided directly on the command line.\n    \n    It is the default mode.\n    \n    jar -jar rumbledb-1.19.0.jar run my-query.jq\n    jar -jar rumbledb-1.19.0.jar run -q '1+1'\n\n    You can specify an output path with -o like so:\n    jar -jar rumbledb-1.19.0.jar run -q '1+1' -o my-output.txt\n    \n    **** serve ****\n    for running as an HTTP server listening on the specified port (-p) and host (-h).\n    \n    jar -jar rumbledb-1.19.0.jar serve -p 9090\n    \n    RumbleDB also supports Apache Livy for use in Jupyter notebooks, which may be\n    even more convenient if you are using a cluster.\n    \n    **** repl ****\n    for shell mode.\n    \n    spark-submit rumbledb-1.19.0.jar repl\n    \n    \n    **** resource use configuration ****\n    \n    For a local use, you can control the number of cores, as well as allocated\n    memory, with:\n    \n    java -jar -Xmx10g rumbledb-1.19.0.jar repl\n    \n    More documentation on available CLI parameters is available on https://www.rumbledb.org/\n\n    \n## Run a hello-world query (all systems)\n\nYou can check that the 1+1 query works with:\n\n    java -jar rumbledb-1.19.0-standalone.jar run -q '1+1'\n    \nThis should return (possibly with some additional info or warning messages that you can safely ignore) 2, as expected.\n\nNext, you can check that the embedded Apache Spark works correctly with this query:\n\n    java -jar rumbledb-1.19.0-standalone.jar run -q 'for $i in parallelize(1 to 100, 10) return { \"foo\" : $i }'\n\nThis should return a sequence of 100 objects, looking like this (also possibly with some additional info or warning messages that you can safely ignore):\n\n    { \"foo\" : 1 }\n    { \"foo\" : 2 }\n    { \"foo\" : 3 }\n    { \"foo\" : 4 }\n    { \"foo\" : 5 }\n    { \"foo\" : 6 }\n    { \"foo\" : 7 }\n    { \"foo\" : 8 }\n    ...\n\nNext, you can check that RumbleDB manages to read files from the Web:\n\n    java -jar rumbledb-1.19.0-standalone.jar run -q 'json-file(\"https://www.rumbledb.org/download/movies.jsonl\")[1]'\n    \nShowing:\n\n    { \"genres\" : \"Thriller\", \"actors\" : [ { \"birth\" : \"1920\", \"name\" : \"Mickey Rooney\", \"death\" : \"2014\" }, { \"birth\" : \"1916\", \"name\" : \"Keenan Wynn\", \"death\" : \"1986\" } ], \"name\" : \"The Manipulator\", \"year\" : \"1971\", \"votes\" : \"248\", \"rating\" : \"3.8\" }\n\nAnd finally, you can download the actual dataset from [this place](https://www.rumbledb.org/download/movies.jsonl) to your directory (the same where you have the RumbleDB jar) and execute\n\n    java -jar rumbledb-1.19.0-standalone.jar run -q 'json-file(\"movies.jsonl\")[1]'\n    \nShowing:\n\n    { \"genres\" : \"Thriller\", \"actors\" : [ { \"birth\" : \"1920\", \"name\" : \"Mickey Rooney\", \"death\" : \"2014\" }, { \"birth\" : \"1916\", \"name\" : \"Keenan Wynn\", \"death\" : \"1986\" } ], \"name\" : \"The Manipulator\", \"year\" : \"1971\", \"votes\" : \"248\", \"rating\" : \"3.8\" }\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frumbledb%2Frumbledb-tutorial","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frumbledb%2Frumbledb-tutorial","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frumbledb%2Frumbledb-tutorial/lists"}