https://github.com/superfly/litevfs

LiteFS VFS SQLite extension for serverless environments
https://github.com/superfly/litevfs

Last synced: about 2 months ago
JSON representation

LiteFS VFS SQLite extension for serverless environments

• Host: GitHub
• URL: https://github.com/superfly/litevfs
• Owner: superfly
• License: apache-2.0
• Created: 2023-07-17T16:26:44.000Z (about 1 year ago)
• Default Branch: main
• Last Pushed: 2024-04-03T08:54:24.000Z (6 months ago)
• Last Synced: 2024-07-08T07:32:48.099Z (3 months ago)
• Language: Rust
• Size: 324 KB
• Stars: 44
• Watchers: 6
• Forks: 1
• Open Issues: 9
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome - superfly/litevfs - LiteFS VFS SQLite extension for serverless environments (others)

README

        
# LiteVFS - LiteFS VFS implementation for serverless environments (WIP)

LiteVFS is a Virtual Filesystem extension for SQLite that uses [LiteFS Cloud][litefs-cloud] as a backing store.

## SQLite CLI

To test with SQLite CLI:

1) Build the extension:

```

$ cargo build --release

```

1) Provide `LITEFS_CLOUD_TOKEN` env variable and load the extension

```

$ LITEFS_CLOUD_TOKEN= sqlite3

sqlite> .load target/release/liblitevfs.so

```

1) Open the database

```

sqlite> .open file:db1?vfs=litevfs

```

That's it. It should work now. The database is stored under `tmp` in a random directory.

To enable debug logging, run `sqlite3` binary like this:

```

$ RUST_LOG=trace sqlite3

```

The following environment variable are handled by LiteVFS:

 - `LITEFS_CLOUD_TOKEN` - LiteFS Cloud token (mandatory)

 - `LITEFS_CLOUD_CLUSTER` - LiteFS Cloud cluster (optional for cluster-scoped tokens, mandatory otherwise)

 - `LITEFS_CLOUD_HOST` - LiteFS Cloud host (optional, defaults to https://litefs.fly.io)

 - `LITEVFS_CACHE_DIR` - cache directory for databases (optional, random directory under `/tmp` if not specified)

 - `LITEVFS_LOG_FILE` - log into the given file instead of stderr

The same shared library can be loaded from any language using their SQLite bindings.

### Modifying the database through LiteVFS

In order to modify a database, the LiteVFS instance must host a write lease to the database. A write lease can be obtained

via a pragma statement:

```

sqlite> pragma litevfs_acquire_lease;

sqlite> 

sqlite> pragma litevfs_release_lease;

```

Only one LiteVFS instance can hold a write lease for speficic database at a time.

### Limitations

* Databases with `journal_mode=wal` cannot be modified via LiteVFS (but can be read)

* Databases with auto-vacuum cannon be opened via LiteVFS at all

* `VACUUM` is not supported

## Building LiteVFS for browsers

The build process uses Emscripten target, thus, Emscripten SDK needs to be installed and configured on the system.

Refer to Emscripted docs on how to do this. Alternatively, `devenv.nix` file in this repo includes all the

required dependencies.

To build simply do:

```sh

$ cargo xtask build-wasm

```

The command will build LiteVFS with Emscripten, download SQLite3 sources, build it with Emscripten and link it with LiteVFS.

At this point you should have `target/sqlite3-wasm/sqlite3.{js,wasm}` files.

Note that since LiteVFS uses synchronous Emscripten's FETCH API, SQLite3 can only be used from a Worker thread, not from the

main browser UI thread.

[litefs-cloud]: https://fly.io/docs/litefs/