{"id":14155167,"url":"https://github.com/sebadob/hiqlite","last_synced_at":"2025-08-27T12:02:10.463Z","repository":{"id":249117858,"uuid":"830619308","full_name":"sebadob/hiqlite","owner":"sebadob","description":"Hiqlite - highly-available, embeddable, raft-based SQLite + cache","archived":false,"fork":false,"pushed_at":"2024-10-24T07:12:28.000Z","size":2949,"stargazers_count":33,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-10-25T02:54:52.318Z","etag":null,"topics":["database","high-availability","raft","rust","sqlite"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sebadob.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"sebadob","patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"lfx_crowdfunding":null,"polar":null,"buy_me_a_coffee":null,"thanks_dev":null,"custom":null}},"created_at":"2024-07-18T16:12:52.000Z","updated_at":"2024-10-24T07:12:30.000Z","dependencies_parsed_at":"2024-10-28T08:41:01.639Z","dependency_job_id":null,"html_url":"https://github.com/sebadob/hiqlite","commit_stats":null,"previous_names":["sebadob/hiqlite"],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sebadob%2Fhiqlite","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sebadob%2Fhiqlite/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sebadob%2Fhiqlite/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sebadob%2Fhiqlite/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sebadob","download_url":"https://codeload.github.com/sebadob/hiqlite/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":228821399,"owners_count":17977165,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","high-availability","raft","rust","sqlite"],"created_at":"2024-08-17T08:02:19.547Z","updated_at":"2025-08-27T12:02:10.447Z","avatar_url":"https://github.com/sebadob.png","language":"Rust","funding_links":["https://github.com/sponsors/sebadob"],"categories":["Applications","Rust","sqlite","Scalable SQLite","distribute"],"sub_categories":["Database"],"readme":"# Hiqlite\n\nHiqlite is an embeddable SQLite database that can form a Raft cluster to provide strong consistency, high availability\n(which is where `Hiqlite` derives from), replication, automatic leader fail-over and self-healing features.\n\n## Why\n\nWhy another SQLite replication solution? Other projects exist already that can do this. The problem is that none of\nthem checks all boxes. They either require an additional independent process running on the side which can do async\nreplication, need a special file system, have bad throughput / latency, or are running as a server.\n\nI don't think that running SQLite as a server is a good solution. Yes, it is very resource friendly, and it may be a\ngood choice when you are heavily resource constrained, but you lose its biggest strength when doing this: having\nall your data local, which makes reads superfast without network latency.\n\nHiqlite builds on top of `rusqlite` and provides an async wrapper around it. For the Raft logic, it builds on top of\n`openraft` while providing its own storage and network implementations.\n\n## Goal\n\nRust is such an efficient language that you most often only need a single process to achieve whatever you need, for most\napplications at least. An embedded SQLite makes everything very convenient. You get very fast local reads and at the\nsame time, it comes with the benefit that you don't have to manage an additional database, which you need to set up,\nconfigure and more importantly maintain. And embedded SQLite will bring database updates basically for free when you\nbuild a new application version.\n\nWhen configured correctly, SQLite offers very good performance and can handle most workloads these days. In very\nfirst benchmarks that I did to find out if the project makes sense at all, I got up to 24.5k single inserts / s on a\ncheap consumer grade M2 SSD. These tests were done on localhost with 3 different processes, but still with real\nnetworking in between them. On another machine with older SATA SSDs it reached up to 16.5k inserts / s.\n\nAt the end, the goal is that you can have the simplicity and all the advantages of an embedded SQLite while still being\nable to run your application highly available (which is almost always mandatory for me) and having automatic fail-over\nand self-healing capabilities in case of any errors or problems.\n\n## Currently implemented and working features\n\n- full Raft cluster setup\n- everything a Raft is expected to do (thanks to [openraft](https://github.com/datafuselabs/openraft))\n- persistent storage for Raft logs and SQLite state machine\n- \"magic\" auto setup, no need to do any manual init or management for the Raft\n- self-healing - each node can automatically recover from un-graceful shutdowns and even full data volume loss\n- automatic database migrations\n- fully authenticated networking\n- optional TLS everywhere for a zero-trust philosophy\n- fully encrypted backups to s3, cron job or manual (\n  with [s3-simple](https://github.com/sebadob/s3-simple) + [cryptr](https://github.com/sebadob/cryptr))\n- restore from remote backup (with log index roll-over)\n- strongly consistent, replicated `EXECUTE` queries\n    - on a leader node, the client will not even bother with using networking\n    - on a non-leader node, it will automatically switch over to a network connection so the request\n      is forwarded and initiated on the current Raft leader\n- strongly consistent, replicated `EXECUTE` queries with returning statement through the Raft\n    - you can either get a raw handle to the custom `RowOwned` struct\n    - or you can map the `RETURNING` statement to an existing struct\n- transaction executes\n- simple `String` batch executes\n- consistent read / select queries on leader\n- `query_as()` for local reads with auto-mapping to `struct`s implementing `serde::Deserialize`.\n- `query_map()` for local reads for `structs` that implement `impl\u003c'r\u003e From\u003chiqlite::Row\u003c'r\u003e\u003e` which is the\n  more flexible method with more manual work\n- in addition to SQLite, multiple in-memory K/V caches with optional independent TTL per entry per cache - K/V caches\n  are disk-backed and store their WAL file + Snapshots on disk, which means they are easy on your memory, and they can\n  rebuild their in-memory data after a restart\n- listen / notify to send real-time messages through the Raft\n- `dlock` feature provides access to distributed locks\n- `counters` feature provides distributed counters\n- standalone binary with the `server` feature which can run as a single node, cluster, or proxy to an existing cluster\n- integrated simple dashboard UI for debugging the database in production - pretty basic for now but it gets the job\n  done\n\n## Performance\n\nI added a [bench example](https://github.com/sebadob/hiqlite/tree/main/examples/bench) for easy testing on different\nhardware and setups. This example is very simple and it mostly cares about `INSERT` performance, which is usually the\nbottleneck when using Raft, because of 2 network round-trips for each write by design.\n\nThe performance can vary quite a bit, depending on your setup and hardware, of course. Quite a lot of work has been put\ninto performance tuning already and I would say, it will be able to handle everything you throw at it. When you reach\nthe threshold, you are probably in an area where you usually would not rely on a single database instance with something\nlike a Postgres anymore as well.  \nSSDs and fast memory make quite a big difference of course. Regarding the CPU, the whole system is designed to benefit\nmore from fewer cores with higher single core speed like Workstation CPU's or AMD Epyc 4004 series. The reason is the\nsingle writer at a time limitation from SQLite.\n\nJust to give you some raw numbers so you can get an idea how fast it currently is, some numbers below. These values were\ntaken using the [bench example](https://github.com/sebadob/hiqlite/tree/main/examples/bench).\n\nThe benchmarks activate the `jemalloc` feature, which is quite a bit faster than glibc `malloc` but is not supported on\nWindows MSVC target for instance. For cache performance, there are 2 different metrics. One uses a disk-backed cache,\nthe other one is fully in-memory. Disk-backed provides a lot more consistency and can even rebuild the whole in-memory\ncache from the WAL + Snapshot on disk, which means even a restart does not make you lose cached data. A pure in-memory\nversion will need to re-join and sync all data from the other nodes between restarts, but it will be a lot faster. The\ndisk-backed caches are limited by your disks IOPS and throughput only.\n\nWhen you take a look at the numbers below, you will see that the SQLite implementation can almost reach the physical\nlimits of the disk, when it has roughly the same throughput as the cache does. This is actually really impressive,\nconsidering that SQLite only allows a single writer at the same time.\n\nTest command (`-c` adjusted each time for different concurrency):\n\n```\ncargo run --release -- cluster -c 4 -r 100000\n```\n\n### Beefy Workstation\n\nAMD Ryzen 9950X, DDR5-5200 with highly optimized timings, M2 SSD Gen4\n\n**SQLite:**\n\n| Concurrency | 100k single `INSERT` | 100k transactional `INSERT` |\n|-------------|----------------------|-----------------------------| \n| 4           | ~29.000 / s          | ~710.000 / s                |\n| 16          | ~58.000 / s          | ~593.000 / s                |\n| 64          | ~91.000 / s          | ~528.000 / s                |\n\nFor a simple `SELECT`, we have 2 different metrics. By default, `hiqlite` caches all prepared statements.\nA simple `SELECT` with a fresh connection, which has not been prepared and cached yet, it took ~180-210 micros.\nOnce the connection has been used once and the statement has been cached, this drops down dramatically to\n6 -25 micros (hard to measure these short ones).\n\n**Cache (disk-backed):**\n\n| Concurrency | 100k single PUT | single entry GET |\n|-------------|-----------------|------------------| \n| 4           | ~35.000 / s     | ~6 micros        |\n| 16          | ~78.000 / s     |                  |\n| 64          | ~94.000 / s     |                  |\n\n**Cache (full in-memory):**\n\n| Concurrency | 100k single PUT |\n|-------------|-----------------| \n| 4           | ~89.000 / s     |\n| 16          | ~262.000 / s    |\n| 64          | ~504.000 / s    |\n\n### Older Workstation\n\nAMD Ryzen 3900X, DDR4-3000, 2x M2 SSD Gen3 as Raid 0\n\n**SQLite:**\n\n| Concurrency | 100k single `INSERT` | 100k transactional `INSERT` |\n|-------------|----------------------|-----------------------------| \n| 4           | ~9.100 / s           | ~388.000 / s                |\n| 16          | ~17.200 / s          | ~335.000 / s                |\n| 64          | ~27.300 / s          | ~299.000 / s                |\n\n**Cache (disk-backed):**\n\n| Concurrency | 100k single PUT | single entry GET |\n|-------------|-----------------|------------------| \n| 4           | ~10.200 / s     | ~14 micros       |\n| 16          | ~22.100 / s     |                  |\n| 64          | ~29.100 / s     |                  |\n\n**Cache (full in-memory):**\n\n| Concurrency | 100k single PUT |\n|-------------|-----------------| \n| 4           | ~24.700 / s     |\n| 16          | ~78.800 / s     |\n| 64          | ~177.000 / s    |\n\n## Crate Features\n\n### `default`\n\nBy default, the following features are enabled:\n\n- `auto-heal`\n- `backup`\n- `sqlite`\n- `toml`\n\n### `auto-heal`\n\nThis feature allows for auto-healing the State Machine (SQLite) in case of an un-graceful shutdown.\nTo reduce I/O and improve performance, Hiqlite does not write the `last_applied_log_id` from the Raft messages\ninto SQLite with each write. If it would do that, we would need to execute 1 extra query for each incoming\nrequest, which effectively would double the amount of I/O if we just think about single `EXECUTE` queries.\nInstead of doing that, it tracks the last applied ID in memory and only persists it into the DB in the\nfollowing situations:\n\n- a new snapshot creation has been triggered\n- a backup has been triggered\n- the metadata of the whole Raft changes (leader change, a node has joined, ...)\n- the node is being shut down\n\nTo make sure it would not start up a database where the last ID has not been persisted correctly, Hiqlite\ncreates a lock file at startup (like most other DB's). If this file exists with the next start, it means that\nthe application has been killed (host crashed, `kill -9`, ...), because otherwise it would remove the lock\nfile after the `last_applied_log_id` has been persisted correctly.\n\nThe `auto-heal` feature enabled the functionality to recover an un-graceful shutdown automatically by simply\ndeleting the whole existing SQLite and rebuilding it from the latest snapshot + raft logs to always reach a\nclean state.\n\nIf you have special needs, you may not want this. I can't think of a situation where it would make much sense\nto disable it, but you could do it.\n\n### `backup`\n\nThis feature allows the creation of automatic backups for disaster recovery. It pulls in `cron` as an additional\ndependency and enabled `sqlite` and `s3` features as well, because it does not make sense without these.\n\nWhen `backup` is enabled, you will get the (by default) nightly backup cron job and you can manually trigger\nbackup creation's via the `hiqlite::Client`. Backups without pushing them to an S3 storage don't make too much\nsense, because even when a cluster node would lose its whole volume, it would simply be rebuilt from the current\nraft leader via snapshot + log replication.\n\nBackups will be created locally first on each of the Raft nodes. Afterward, only the leader will encrypt the\nbackup and push it to the configured S3 bucket for disaster recovery.\n\nAuto-restoring from a backup on S3 storage will also be possible with this feature enabled. The likelihood that you\nneed to do this, is pretty low though.\n\n#### You lose a cluster node\n\nIf you lost a cluster node for whatever reason, you don't need a backup. Just shut down the node, get rid of any\npossibly left over data, and restart it. The node will join the cluster and fetch the latest snapshot + logs from\nthe current leader node.\n\n#### You lose the full cluster\n\nIf you end up in a situation where you lost the complete cluster, it is the only moment when you probably need\nrestore from backup as disaster recovery. The process is simple:\n\n1. Have the cluster shut down. This is probably the case anyway, if you need to restore from a backup.\n2. Provide a backup file name on S3 storage with the `HQL_BACKUP_RESTORE` value with prefix `s3:` (encrypted), or a file\n   on disk (plain sqlite file) with the prefix `file:`.\n3. Start up the cluster again.\n4. After the restart, make sure to remove the `HQL_BACKUP_RESTORE` env value.\n\n### `cache`\n\nThis feature will start another independent raft group (can run without `sqlite` enabled as well).\nThe `hiqlite::Client` will get new functions like `get()` and `put()`. The `cache` feature will build multiple\nraft-replicated, in-memory caches on all nodes. Basically an in-memory KV store with optional per cache per entry\nTTL for each key.\n\n### `dashboard`\n\nThis feature is the one that makes the crate size on crates.io that big. Hiqlite comes with pre-built, static\nHTML files to optionally serve a simple dashboard. With this dashboard, you have the possibility to run queries\nagainst your database, which typically is not that easy for a SQLite in production, which is probably deployed\ninside some container.\n\nThe dashboard will be served alongside the API HTTP server. It is very basic for now, but it gets the job done.\nIt will pull in quite a few extra dependencies and enable `sqlite` feature, because it does not work with the\n`cache` or other features currently.\n\n![dashboard screenshot](https://raw.githubusercontent.com/sebadob/hiqlite/main/dashboard/screenshot.png)\n\n### `dlock`\n\nThe `dlock` feature gives you access to distributed locks, synchronized over all Raft nodes. It depends on\nthe `cache` feature to work.\n\nIn some cases, you can't achieve what you need to do within a single query or inside a transaction. For instance,\nyou need to fetch data from the DB, compute stuff with it, and write something back to the DB while the data\non the DB must be locked the whole time. Because transactions with Hiqlite can't let you hold a lock directly\non the DB (because of the Raft replication), you get distributed locks.\n\nYou can lock any key, then do whatever you need, and as soon as the `Lock` you will get is being dropped, it will\nbe released automatically.\n\n**Important:**\nIn the current version, a distributed lock is only valid for max 10 seconds, to avoid issues with network segmentation\nor crashed nodes while they were holding some locks. If a lock is older than 10 seconds, it will be considered being\n\"dead\" in the current implementation to get rid of never-ending locks.\n\n### `full`\n\nThis feature will simply enable everything apart from the `server` feature:\n\n- auto-heal\n- backup\n- cache\n- dashboard\n- dlock\n- listen_notify\n- s3\n- shutdown-handle\n- sqlite\n- toml\n- webpki-roots\n\n### `jemalloc`\n\nThis feature enables the `jemallocator` instead of using the default glibc `malloc`. It is a lot more performant, solves\nsome issues with memory fragmentation and can be tuned for specific use cases. However, it does not work on Windows\nMSVC targets and out of the box, without any tuning, it will use a bit more memory than default `malloc`.\n\n### `listen_notify`\n\nSometimes, you need something simple like Postgres' listen / notify to send real time messages between nodes of your\ndeployment, without the need for message delivery guarantees or something like that. That is exactly what the\n`listen_notify` feature will let you do. It pulls in a few additional dependencies and enables the `cache` feature it\ndepends on.\n\nDepending on your setup, you will get different levels of message delivery guarantees. The classic Postgres listen /\nnotify will forward messages, if another connection is listening, and drop them if not, pretty simple.\nWith Hiqlite, if your node is a real Raft member, meaning it is not using a remote client, you will have a guaranteed\nonce delivery with any form of `listen()`. If however you have a remote client, which is connected to a remote Hiqlite\ncluster without a local replicated state, you will not receive missed messages, if you stopped listening for some time.\nIn this case, you will have the classic Postgres behavior.\n\n**Important:**\nIf you enabled this feature and you `notify()` via the `hiqlite::Client`, you must make sure to actually consume the\nmessages on each node. Behind the scenes, Hiqlite uses an unbound channel to never block these. This channel could fill\nup if you `notify()` without `listen()`.\n\n### `s3`\n\nYou would probably never just enable the `s3` feature on its own in the current implementation. It has been outsourced\nfor a possible future feature expansion. It depends on the `backup` feature and both will pull in each other as a\ndependency right now.\nThis feature will enable the possibility to push encrypted State Machine (SQLite) backups to a configured `s3` bucket.\n\n### `server`\n\nThis feature only exists to make it possible to run Hiqlite as a standalone DB / Cluster, if you really want this.\nIt will build a binary which spins up a cluster with the given configuration, or you you can use it to install Hiqlite\nto spin up instances easily with\n\n`cargo install hiqlite --features server`\n\nYou should never enable the `server` feature if you are using Hiqlite as a crate and run it inside your application,\nwhich should always be preferred, because it would make all operations a lot faster because of local data and less\nnetwork round-trips. Embedding Hiqlite is actually one of its biggest advantages over a server / client database like\nPostgres, which would never be able to even come close to the read and `SELECT` speeds of a local SQLite instance.\n\n### `shutdown-handle`\n\nAs mentioned in other places already, a Hiqlite node should always be shut down gracefully to prevent full State Machine\nrebuilds with each restart. Most applications already have some sort of shutdown handles or can listen automatically.\nIf you already have something like that, you can leave this feature disabled and simply call\n`hiqlite::Client.shutdown()`\nbefore exiting your `main()`.\nIn any other case, you can enable the `shutdown-handle` and register an automatic shutdown handle like shown in the\nexamples, which you can `.await` just before exiting your `main()`.\n\n### `sqlite`\n\nThis is the main feature for Hiqlite, the main reason why it has been created. The `sqlite` feature will spin up a\nRaft cluster which uses a `SQLite` instance as the State Machine.\n\nThis SQLite database will always be on disk and never in-memory only. Actually, the in-memory SQLite is slower than\non-disk with all the applied default optimizations. The reason is that an in-memory SQLite cannot use a WAL file. This\nmakes it slower than on-disk with a WAL file and proper `PRAGMA` settings in all of my tests.\nAnother issue with an in-memory SQLite is that you will get into problems with queries blocking each other all the time\nas soon as you have multiple connections for the same reason as above: no WAL file.\n\nThis has its own feature though, because you may only be interested in having an in-memory cache / KV store sometimes.\nIn this case, you can disable the default features and only enable `cache` or whatever you need. You would not even\nneed any volume attached to your container in that case.\n\n### `webpki-roots`\n\nThis feature will simply enable baked-in TLS ROOT CA's to be independent of any OS trust store, like for instance\nwhen you don't even have one inside your minimal docker container.\n\n## Standalone Server / Cluster\n\nEven though it is recommended to embed `hiqlite` into your application, you can run it standalone as well.\n\n### Local Start\n\nThe easiest way would be to install the binary with\n\n```\ncargo install hiqlite --features server\n```\n\nand then just execute it:\n\n```\nhiqlite -h\n```\n\nThe current implementation is still a bit basic, but it will help you to get it up and running. I suggest to start\nwith generating a template config file with\n\n```\nhiqlite generate-config -h\n```\n\nIf you want to just test it without TLS, add the `--insecure-cookie` option, and you may generate a testing password\nwith `-p`. Once you have you config, you can start a node with\n\n```\nhiqlite serve -h\n```\n\nThe `--node-id` must match a value from `HQL_NODES` inside your config. When you overwrite the node id at startup,\nyou can re-use the same config for multiple nodes.\n\n### Example Config\n\nTake a look at the [examples](https://github.com/sebadob/hiqlite/tree/main/examples) or the example\n[config](https://github.com/sebadob/hiqlite/blob/main/config) to get an idea about the possible config values.\nThe `NodeConfig` can be created programmatically or fully created either `from_toml()` or `from_env()` vars.\n\n### Cluster inside Kubernetes\n\nThere is no Helm chart or anything like that yet, but starting the Hiqlite server inside K8s is very simple.\n\n#### Namespace\n\nLet's run it inside a new namespace called `hiqlite`:\n\n```\nkubectl create ns hiqlite\n```\n\n#### Config\n\nCreate a secret for your config. Adapt the below values to your needs. To have an easy time with the `enc_keys`, since\nthe CLI does not provide a generator yet, you can copy the value from your `generate-config` step above and re-use the\nvalue here, or just re-use the below example values. You could also copy \u0026 paste your whole generated config into this\nsecret.\n\n```yaml\napiVersion: v1\nkind: Secret\nmetadata:\n  name: hiqlite-config\n  namespace: hiqlite\ntype: Opaque\nstringData:\n  hiqlite.toml: |\n    [hiqlite]\n    node_id_from = \"k8s\"\n    nodes = [\n        \"1 hiqlite-0.hiqlite-headless:8100 hiqlite-0.hiqlite-headless:8200\",\n        \"2 hiqlite-1.hiqlite-headless:8100 hiqlite-1.hiqlite-headless:8200\",\n        \"3 hiqlite-2.hiqlite-headless:8100 hiqlite-2.hiqlite-headless:8200\",\n    ]\n\n    secret_raft = \"SuperSecureSecret1337\"\n    secret_api = \"SuperSecureSecret1337\"\n\n    enc_keys = [ \"bVCyTsGaggVy5yqQ/UzluN29DZW41M3hTSkx6Y3NtZmRuQkR2TnJxUTYzcjQ=\" ]\n    enc_key_active = \"bVCyTsGaggVy5yqQ\"\n\n    # This is a base64 encoded Argon2ID hash for the password: 123SuperMegaSafe\n    password_dashboard = \"JGFyZ29uMmlkJHY9MTkkbT0zMix0PTIscD0xJE9FbFZURnAwU0V0bFJ6ZFBlSEZDT0EkTklCN0txTy8vanB4WFE5bUdCaVM2SlhraEpwaWVYOFRUNW5qdG9wcXkzQQ==\"\n\n    # necessary if you want to access the dashboard via plain HTTP for testing\n    insecure_cookie = true\n\n    s3_url = \"https://s3.example.com\"\n    s3_bucket = \"S3BucketName\"\n    s3_region = \"S3Region\"\n    s3_key = \"YourS3KeyId\"\n    s3_secret = \"YourS3Secret\"\n```\n\n#### StatefulSet\n\nThe last one for testing (leaving Ingress out for this simple example) will create a StatefulSet, a load balanced\nService you could access via a `NodePort` to reach the dashboard, and a headless Service to the nodes can create\ndirect connections to each other. Create an `sts.yaml`:\n\n```yaml\napiVersion: v1\nkind: Service\nmetadata:\n  name: hiqlite\n  namespace: hiqlite\nspec:\n  selector:\n    app: hiqlite\n  type: NodePort\n  ports:\n    - name: raft\n      protocol: TCP\n      port: 8100\n      targetPort: 8100\n    - name: api\n      protocol: TCP\n      port: 8200\n      targetPort: 8200\n---\napiVersion: v1\nkind: Service\nmetadata:\n  name: hiqlite-headless\n  namespace: hiqlite\nspec:\n  clusterIP: None\n  selector:\n    app: hiqlite\n  ports:\n    - name: raft\n      protocol: TCP\n      port: 8100\n      targetPort: 8100\n    - name: api\n      protocol: TCP\n      port: 8200\n      targetPort: 8200\n---\napiVersion: apps/v1\nkind: StatefulSet\nmetadata:\n  name: hiqlite\n  namespace: hiqlite\n  labels:\n    app: hiqlite\nspec:\n  replicas: 3\n  selector:\n    matchLabels:\n      app: hiqlite\n  serviceName: hiqlite-headless\n  template:\n    metadata:\n      labels:\n        app: hiqlite\n    spec:\n      containers:\n        - name: hiqlite\n          image: ghcr.io/sebadob/hiqlite:0.10.0\n          securityContext:\n            allowPrivilegeEscalation: false\n          ports:\n            - containerPort: 8100\n            - containerPort: 8200\n          volumeMounts:\n            - name: hiqlite-config\n              mountPath: /app/hiqlite.toml\n              subPath: hiqlite.toml\n              readOnly: true\n            - name: hiqlite-data\n              mountPath: /app/data\n          livenessProbe:\n            httpGet:\n              # You may need to adjust this, if you decide to start in https only\n              # mode or use another port\n              scheme: HTTP\n              port: 8200\n              #scheme: HTTPS\n              #port: 8443\n              path: /health\n              initialDelaySeconds: 30\n              periodSeconds: 30\n          resources:\n            requests:\n              memory: 32Mi\n              cpu: 100m\n      # add your image pull secrets name here in case you use a private container registry\n      #imagePullSecrets:\n      #  - name: harbor\n      volumes:\n        - name: hiqlite-config\n          secret:\n            secretName: hiqlite-config\n  volumeClaimTemplates:\n    - metadata:\n        name: hiqlite-data\n      spec:\n        accessModes:\n          - \"ReadWriteOnce\"\n        resources:\n          requests:\n            storage: 256Mi\n        # In case you want to specify the storage class.\n        # You should always(!) prefer local over some replicated abstraction layer.\n        # Hiqlite cares about replication itself already.\n        #storageClassName: local-path\n```\n\n#### Apply Files\n\nThe last step is to simply `kubectl apply -f` the `config.yaml` followed by the `sts.yaml` last. This should bring up a\n3 node, standalone Hiqlite cluster.\n\n## Limitations\n\nYou must not use non-deterministic functions inside your database-modifying statements. These are functions like `now()`\nor `random()` for instance. Apart from the fact, that you should generally avoid this on any database to shift the\nresource usage from the DB (which almost always will be the bottleneck when scaling) into your application code, which\nis typically easy to scale, the writer connection for the database will always `panic!` as a safety-net, if you use any\nof these functions.\n\nIf you really want to, you can use them on all non-modifying queries of course.\n\n## Known Issues\n\nThere are currently some known issues:\n\n1. When creating synthetic benchmarks for testing write throughput at the absolute max, you will see error logs because\n   of missed Raft heartbeats and leader switches, even though the network and everything else is fine. The reason is\n   that the Raft heartbeats in the current implementation come in-order with the Raft data replication. So, if you\n   generate an insane amount of Raft data which takes time to replicate, because you end up being effectively I/O\n   bound by your physical disk, these heartbeats can get lost, because they won't happen in-time. This issue will be\n   resolved with the next major release of `openraft`, where heartbeats will be sent separately from the main data\n   replication.\n2. In the current version, the logging output is very verbose on the `info` level. This is on purpose until everything\n   has been stabilized. In future versions, this will be reduced quite a bit.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsebadob%2Fhiqlite","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsebadob%2Fhiqlite","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsebadob%2Fhiqlite/lists"}