https://github.com/denoland/denokv
A self-hosted backend for Deno KV
https://github.com/denoland/denokv
database deno kv sqlite
Last synced: about 1 year ago
JSON representation
A self-hosted backend for Deno KV
- Host: GitHub
- URL: https://github.com/denoland/denokv
- Owner: denoland
- License: mit
- Created: 2023-10-18T07:04:47.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2025-03-14T06:59:43.000Z (over 1 year ago)
- Last Synced: 2025-04-11T01:41:57.274Z (about 1 year ago)
- Topics: database, deno, kv, sqlite
- Language: TypeScript
- Homepage: https://deno.com/kv
- Size: 1.38 MB
- Stars: 581
- Watchers: 22
- Forks: 30
- Open Issues: 37
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# denokv
A self-hosted backend for [Deno KV](https://deno.com/kv), the JavaScript first key-value database:
- Seamlessly integrated JavaScript APIs
- ACID transactions
- Multiple consistency levels for optimal performance for every usecase
Deno KV can be used with the built-in single instance database in the CLI,
useful for testing and development, with a hosted and [scalable backend](https://deno.com/blog/building-deno-kv) on
[Deno Deploy](https://deno.com/deploy), or with this self-hostable Deno KV
backend.
To run `denokv`, just run:
```sh
docker run -it --init -p 4512:4512 -v ./data:/data ghcr.io/denoland/denokv --sqlite-path /data/denokv.sqlite serve --access-token
```
Then run your Deno program and specify the access token in the
`DENO_KV_ACCESS_TOKEN` environment variable:
```ts
const kv = await Deno.openKv("http://localhost:4512");
```
The self-hosted `denokv` backend is built on the same robust SQLite backend as
the built-in single instance database in the CLI. It is designed to be run on a
VPS or Kubernetes cluster statefully, with Deno processes connecting via the
network using [KV Connect](https://docs.deno.com/kv/manual/on_deploy#connect-to-managed-databases-from-outside-of-deno-deploy).
The standalone `denokv` binary is designed to handle thousands of concurrent
requests, from hundreds of different Deno processes. It is built on top of the
robust SQLite database, and uses non-blocking IO to ensure excellent performance
even in the face of hundreds of concurrent connections.
Just like the Deno CLI, `denokv` is MIT licensed, free and open source.
Read more in [the announcement of self-hosted Deno KV](https://deno.com/blog/kv-is-open-source-with-continuous-backup).
## When should I use this?
If you need more than a single Deno process to access the same KV database, and
you are ok with running a server, keeping `denokv` updated, handling backups,
and performing regular maintenance, then this is for you.
You can use a hosted KV database on Deno Deploy if you don't want to self-host
and manage a `denokv` server.
If you are just need a backend for local development or testing, you can use the
Deno KV backend built into the Deno CLI. You can open a temporary in memory KV
database with `Deno.openKv(":memory:")` or a persistent database by specifying a
path like `Deno.openKv("./my-database.sqlite")`.
## How to run
### Docker on a VPS
> Ensure that you are running on a service that supports persistent storage, and
> does not perform auto-scaling beyond a single instance. This means you can not
> run `denokv` on Google Cloud Run or AWS Lambda.
Install Docker on your VPS and create a directory for the database to store data
in.
```sh
$ mkdir -p /data
```
Then run the `denokv` Docker image, mounting the `/data` directory as a volume
and specifying a random access token.
```sh
docker run -it --init -p 4512:4512 -v ./data:/data ghcr.io/denoland/denokv --sqlite-path /data/denokv.sqlite serve --access-token
```
You can now access the database from your Deno programs by specifying the access
token in the `DENO_KV_ACCESS_TOKEN` environment variable, and the host and port
of your VPS in the URL passed to `Deno.openKv`.
You should additionally add a HTTPS terminating proxy or loadbalancer in front
of `denokv` to ensure that all communication happens over TLS. Not using TLS can
pose a significant security risk. The HTTP protocol used by Deno KV is
compatible with any HTTP proxy, such as `caddy`, `nginx`, or a loadbalancer.
### Fly.io
You can easily host `denokv` on https://fly.io.
> Note: Fly.io is a paid service. You will need to add a credit card to your
> account to use it.
Sign up to Fly.io and
[install the `flyctl` CLI](https://fly.io/docs/hands-on/install-flyctl/).
Sign into the CLI with `flyctl auth login`.
Create a new app with `flyctl apps create`.
Create a `fly.toml` file with the following contents. Make sure to replace the
`` and `` placeholders with your app name and the region
you want to deploy to.
```toml
app = ""
primary_region = ""
[build]
image = "ghcr.io/denoland/denokv:latest"
[http_service]
internal_port = 4512
force_https = true
auto_stop_machines = true
auto_start_machines = true
min_machines_running = 0
[env]
DENO_KV_SQLITE_PATH="/data/denokv.sqlite3"
# access token is set via `flyctl secrets set`
[mounts]
destination = "/data"
source = "denokv_data"
```
Run `flyctl volumes create denokv_data` to create a volume to store the database
in.
Run `flyctl secrets set DENO_KV_ACCESS_TOKEN=` to set the access
token. Make sure to replace `` with a random string. Keep this
token secret, and don't share it with anyone. You will need this token to
connect to your database from Deno.
Run `flyctl deploy` to deploy your app.
You can now access the database from your Deno programs by specifying the access
token in the `DENO_KV_ACCESS_TOKEN` environment variable, and the URL provided
by `flyctl deploy` in the URL passed to `Deno.openKv`.
Be aware that with this configuration, your database can scale to 0 instances
when not in use. This means that the first request to your database after a
period of inactivity will be slow, as the database needs to be started. You can
avoid this by setting `min_machines_running` to `1`, and setting
`auto_stop_machines = false`.
### Install binary
You can download a prebuilt binary from the
[releases page](https://github.com/denoland/denokv/releases/tag/0.1.0) and place
it in your `PATH`.
You can also compile from source by running `cargo install denokv --locked`.
## How to connect
### Deno
To connect to a `denokv` server from Deno, use the `Deno.openKv` API:
```ts
const kv = await Deno.openKv("http://localhost:4512");
```
Make sure to specify your access token in the `DENO_KV_ACCESS_TOKEN` environment
variable.
## Advanced setup
### Running as a replica of a hosted KV database
`denokv` has a mode for running as a replica of a KV database hosted on Deno
Deploy through the S3 backup feature.
To run as a replica:
```sh
docker run -it --init -p 4512:4512 -v ./data:/data \
-e AWS_ACCESS_KEY_ID="" \
-e AWS_SECRET_ACCESS_KEY="" \
-e AWS_REGION="" \
ghcr.io/denoland/denokv --sqlite-path /data/denokv.sqlite serve \
--access-token --sync-from-s3 --s3-bucket your-bucket --s3-prefix some-prefix/6aea9765-2b1e-41c7-8904-0bdcd70b21d3/
```
To sync the local database from S3, without updating the snapshot:
```sh
denokv --sqlite-path /data/denokv.sqlite pitr sync --s3-bucket your-bucket --s3-prefix some-prefix/6aea9765-2b1e-41c7-8904-0bdcd70b21d3/
```
To list recoverable points:
```sh
denokv --sqlite-path /data/denokv.sqlite pitr list
```
To checkout the snapshot at a specific recoverable point:
```sh
denokv --sqlite-path /data/denokv.sqlite pitr checkout 0100000002c0f4c10000
```
### Continuous backup using LiteFS
TODO
## Other things in this repo
This repository contains two crates:
- `denokv_proto` (`/proto`): Shared interfaces backing KV, like definitions of
`Key`, `Database`, and `Value`.
- `denokv_sqlite` (`/sqlite`): An implementation of `Database` backed by SQLite.
- `denokv_remote` (`/remote`): An implementation of `Database` backed by a
remote KV database, acessible via the KV Connect protocol.
These crates are used by the `deno_kv` crate in the Deno repository to provide a
JavaScript API for interacting with Deno KV.
The Deno KV Connect protocol used for communication between Deno and a remote KV
database is defined in [`/proto/kv-connect.md`](./proto/kv-connect.md).