{"id":29190816,"url":"https://github.com/firebolt-db/firebolt-core","last_synced_at":"2025-07-02T00:11:50.630Z","repository":{"id":298580162,"uuid":"998882887","full_name":"firebolt-db/firebolt-core","owner":"firebolt-db","description":"Firebolt Core is a free, self-hosted edition of Firebolt's distributed query engine (https://www.firebolt.io/); it provides high-performance data warehousing capabilities that can be deployed anywhere from a single laptop to enterprise datacenters.","archived":false,"fork":false,"pushed_at":"2025-06-26T13:45:25.000Z","size":282,"stargazers_count":87,"open_issues_count":2,"forks_count":0,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-06-26T14:44:17.313Z","etag":null,"topics":["ai","analytics","big-data","cloud-native","database","gcs","iceberg","jdbc","parquet","postgresql","query-engine","s3","self-hosted","sql"],"latest_commit_sha":null,"homepage":"https://firebolt.io/core","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/firebolt-db.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-09T12:03:13.000Z","updated_at":"2025-06-26T13:39:09.000Z","dependencies_parsed_at":"2025-06-11T21:42:15.234Z","dependency_job_id":"a5fe581e-269f-442c-a675-1f7052751c33","html_url":"https://github.com/firebolt-db/firebolt-core","commit_stats":null,"previous_names":["firebolt-db/firebolt-core"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/firebolt-db/firebolt-core","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/firebolt-db%2Ffirebolt-core","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/firebolt-db%2Ffirebolt-core/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/firebolt-db%2Ffirebolt-core/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/firebolt-db%2Ffirebolt-core/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/firebolt-db","download_url":"https://codeload.github.com/firebolt-db/firebolt-core/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/firebolt-db%2Ffirebolt-core/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263052434,"owners_count":23406106,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","analytics","big-data","cloud-native","database","gcs","iceberg","jdbc","parquet","postgresql","query-engine","s3","self-hosted","sql"],"created_at":"2025-07-02T00:11:48.106Z","updated_at":"2025-07-02T00:11:50.599Z","avatar_url":"https://github.com/firebolt-db.png","language":"Shell","readme":"\u003cp align=\"center\"\u003e\n    \u003cimg alt=\"Firebolt Core logo\" src=\"static/core.png?raw=true\" width=\"400\" /\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n    \u003cb\u003eFirebolt Core\u003c/b\u003e is a free, self-hosted edition of \u003ca href=\"https://www.firebolt.io/\"\u003eFirebolt's high-performance distributed query engine\u003c/a\u003e, designed to power the data infrastructure behind today's most demanding applications.\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://discord.gg/UpMPDHActM\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/Discord-%235865F2.svg?logo=discord\u0026logoColor=white\" alt=\"Discord chat\"\n  /\u003e\u003c/a\u003e\n  \u003ca href=\"https://docs.firebolt.io/firebolt-core\" style=\"text-decoration: none\"\u003e\u003cimg\n    src=\"https://img.shields.io/badge/Core-docs-brightgreen\"\n    alt=\"Firebolt Core documentation\"\n  /\u003e\u003c/a\u003e\n  \u003ca href=\"https://docs.firebolt.io/FireboltCore/firebolt-core-operation.html\" style=\"text-decoration: none\"\u003e\u003cimg\n    src=\"https://img.shields.io/badge/deployment-guide-brightgreen\"\n    alt=\"Deployment and Operational Guide\"\n  /\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/firebolt-db/firebolt-core?tab=readme-ov-file#get-started\" style=\"text-decoration: none\"\u003e\u003cimg\n    src=\"https://img.shields.io/badge/release-preview%E2%80%93rc-brightgreen\"\n    alt=\"Release\"\n  /\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/firebolt-db/firebolt-core/issues\" style=\"text-decoration: none\"\u003e\u003cimg\n    src=\"https://img.shields.io/github/issues/firebolt-db/firebolt-core.svg\"\n    alt=\"GitHub Issues\"\n  /\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/firebolt-db/firebolt-core/stargazers\" style=\"text-decoration: none\"\u003e\u003cimg\n    src=\"https://img.shields.io/github/stars/firebolt-db/firebolt-core.svg\"\n    alt=\"GitHub Stars\"\n  /\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\u003e *Deploy anywhere, from a single laptop to your own datacenter*\n\n## Key Features\n\n* 🚀 **Powerful.** Firebolt Core ships with all key performance and usability features of Firebolt's [managed Cloud data warehouse](https://www.firebolt.io/), including a state-of-the-art query optimizer, distributed query execution engine, Iceberg support, and many more.\n* 🆓 **Free to use.** Firebolt Core is free to use, forever (see the [LICENSE](LICENSE.md) for details).\n* 📈 **No usage limits.** Firebolt Core has no usage limits. Process unlimited data, scale to as many nodes as you need, and run as many queries as you like.\n* 🛢️ **Postgres compliant.** Firebolt's SQL dialect is Postgres compliant. We offer powerful extensions for analytical workloads, such as lambda functions for array processing. For a complete reference, see the [SQL reference documentation](https://docs.firebolt.io/sql_reference/).\n* 🛠️ **Self-contained.** Firebolt Core comes packaged as a single Docker image (`ghcr.io/firebolt-db/firebolt-core:preview-rc`) that contains everything needed to run it.\n* 🏠 **Self-hosted.** You can deploy Firebolt Core anywhere you want, from your personal workstation to large on-premise clusters or VPCs.\n* 📊 **First-class support** with [documentation](https://docs.firebolt.io/firebolt-core), updates, and active [community support via GitHub Discussions](https://github.com/firebolt-db/firebolt-core/discussions). We encourage you to join the conversation!\n* 🤖 **AI-ready architecture** optimized for modern data and ML applications.\n* 🎯 **Designed for demanding applications.** Powering real-time analytics, embedded analytics, and large-scale data processing workloads.\n* 🔄 **Workload compatibility.** Many workloads run interchangeably with [managed Firebolt](https://www.firebolt.io/).\n\n## Get Started\n\nStart Core on your machine with:\n```bash\nbash \u003c(curl -s https://get-core.firebolt.io/)\n```\n\nIf you want to work with Docker directly, you can also run:\n\n```bash\ndocker run -i --rm \\\n        --ulimit memlock=8589934592:8589934592 \\\n        --security-opt seccomp=unconfined \\\n        -p 127.0.0.1:3473:3473 \\\n        -v ./firebolt-core-data:/firebolt-core/volume \\\n        ghcr.io/firebolt-db/firebolt-core:preview-rc\n```\n\n\u003e [!CAUTION]\n\u003e This will create a local `firebolt-core-data` directory, owned by root, where data, metadata, logs and diagnostic information are persisted.\n\nYou can also start a single node cluster by cloning this repository and then run the following command within the repository root directory:\n```bash\ndocker compose up\n```\n\nSee also:\n* [Get Started](https://docs.firebolt.io/FireboltCore/firebolt-core-get-started.html)\n\n### Multi-Node via Docker Compose\n\nUse this setup if you want to leverage the computing power of multiple hosts.\n\n1. Add in `config.json` one entry (either hostname or IP address) for each of the nodes; node 0 is the first node in the array:\n\n    ```json\n    {\n        \"nodes\": [\n            {\n                \"host\": \"ip-or-host-of-node-0\"\n            },\n            {\n                \"host\": \"ip-or-host-of-node-1\"\n            },\n            {\n                \"host\": \"ip-or-host-of-node-2\"\n            }\n        ]\n    }\n    ```\n\n1. Make sure that this repository and your `config.json` are present on each of the nodes.\n\n1. Activate node 0:\n\n   ```bash\n   docker compose up\n   ```\n\n1. Activate all further nodes by running on each host:\n\n   ```bash\n   NODE=1 docker compose -f compose.yaml -f compose.nodeN.yaml up\n   ```\n\n   Increase `NODE` for each further node.\n\nSee also:\n* [Deployment using Docker Compose](https://docs.firebolt.io/FireboltCore/firebolt-core-deployment-compose.html)\n* [Docker Compose](https://docs.docker.com/compose/)\n\n### Multi-Node via Kubernetes\n\nYou can deploy Firebolt Core on Kubernetes (v1.19+) by following these steps:\n\n1. Create a dedicated namespace:\n```bash\nkubectl create namespace firebolt-core\n```\n2. Customize the values for the chart (see [helm/README.md](helm/README.md)), for example by setting `nodesCount` to 3.\n3. Install the Helm chart in such namespace:\n```bash\nhelm install helm/ --generate-name --namespace firebolt-core\n```\n4. Verify that pods are running:\n```bash\nkubectl get pods\n```\nExpected output:\n```\nNAME                              READY   STATUS    RESTARTS   AGE\nhelm-1748880880-firebolt-core-0   0/1     Running   0          5m32s\nhelm-1748880880-firebolt-core-1   0/1     Running   0          5m32s\nhelm-1748880880-firebolt-core-2   0/1     Running   0          5m32s\n```\n\nSee also:\n* [helm/README.md](helm/README.md) for information on the values you can customize, including number of nodes.\n* [Deployment on Kubernetes](https://docs.firebolt.io/FireboltCore/firebolt-core-deployment-k8s.html)\n\n## Requirements\n\nSoftware for your host OS:\n\n* **[Docker Engine](https://docs.docker.com/engine/install/)**, with the **[Docker Compose plugin](https://docs.docker.com/compose/install/linux/)** if you want to use `docker compose`; if you use the [get-core.sh](get-core.sh) script Docker engine will be installed automatically.\n* **[cURL](https://curl.se/) or any other HTTP client** in order to send SQL queries to Firebolt Core.\n\nSoftware for your Docker host:\n* **Linux kernel version \u003e= 6.1**. Firebolt Core internally uses the `io_uring` kernel API for fast network and disk I/O, and some required features of this API have only been released in Linux 6.1.\n\nResources for each node (either a local machine or a VM instance):\n\n* **An amd64 CPU supporting at least SSE 4.2, or an arm64 CPU** All published Firebolt Core Docker images are multi-arch images suitable for both `amd64` and `arm64`.\n* **At least 16 GB of RAM** are recommended in order to run basic queries.\n* **At least 25 GB of SSD space** are recommended in order to run basic queries.\n* **At least 10 GBit/s of inter-node network bandwidth** is recommended for multi-node deployments.\n* TCP port `3473` open when using a single node.\n* TCP ports `3473`, `1717`, `3434`, `5678`, `6500`, `8122`, `16000` open when using multiple nodes.\n\n\u003e [!NOTE]\n\u003e There is no universally correct amount of RAM and disk space for running Firebolt Core, and the above are simply rough guidelines for running some simple queries as a way to get started. The ideal amount of RAM and disk space depends heavily on the specific workload that you are running against a Firebolt Core deployment (see [Deployment and Operational Guide](https://docs.firebolt.io/FireboltCore/firebolt-core-operation.html) for details).\n\n## Run Queries on Firebolt Core\n\nYou can submit queries to a Firebolt Core cluster using any HTTP client (like cURL), the official [Firebolt CLI](https://github.com/firebolt-db/fb-cli) as a standalone binary, or by invoking the CLI from within the Core Docker container.\n\n```bash\nSuggested change\n# Use the fbcli script available within the container\ndocker exec -ti firebolt-core fbcli \"SELECT 42;\"\n```\n\n```bash\n# Run the standalone fb CLI\nfb --core \"SELECT 42;\"\n```\n\n```bash\n# Use cURL \ncurl -s \"http://localhost:3473/?output_format=psql\" --data-binary \"SELECT 42\";\n```\n\nSee also:\n* [Example Queries](examples/README.md)\n* [Connect to Firebolt Core](https://docs.firebolt.io/FireboltCore/firebolt-core-connect.html)\n\n## Troubleshooting \u0026 Support\n\nDetailed information about Firebolt Core is available in the [documentation](https://docs.firebolt.io/FireboltCore/).\n\n* Encountering issues? Check the [Troubleshooting Guide](https://docs.firebolt.io/FireboltCore/firebolt-core-troubleshooting.html) for common problems and solutions and the [FAQs](https://docs.firebolt.io/FireboltCore/firebolt-core-faq.html)\n* For further assistance, join the [GitHub Discussions](https://github.com/firebolt-db/firebolt-core/discussions).\n* For best practices on securing your deployment and information on backing up, consult the [Operational Guide](https://docs.firebolt.io/FireboltCore/firebolt-core-operation.html).\n* Curious about what's next for Firebolt Core? Check out the [Roadmap](https://docs.firebolt.io/FireboltCore/firebolt-core-roadmap.html) to see planned features and improvements.\n\n## Verifying Firebolt Core Docker Images\n\nAll Firebolt Core published images are signed using [cosign](https://github.com/sigstore/cosign).\n\nTo verify:\n\n1. Download the public key: [cosign.pub](cosign.pub)\n1. Run:\n```console\ncosign verify --key cosign.pub ghcr.io/firebolt-db/firebolt-core:preview-rc\n```\n\nYou should see output confirming the signature is valid:\n```\nVerification for ghcr.io/firebolt-db/firebolt-core:preview-rc --\nThe following checks were performed on each of these signatures:\n  - The cosign claims were validated\n  - Existence of the claims in the transparency log was verified offline\n  - The signatures were verified against the specified public key\n[...]\n```\n\n## License\n\nSee [LICENSE](LICENSE.md) and [NOTICE](NOTICE.md).\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffirebolt-db%2Ffirebolt-core","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffirebolt-db%2Ffirebolt-core","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffirebolt-db%2Ffirebolt-core/lists"}