https://github.com/cycneuramus/nomad-litestream-example

Nomad example job for deploying SQLite-backed applications with high availability
https://github.com/cycneuramus/nomad-litestream-example

high-availability litestream nomad s3 sqlite

Last synced: 11 days ago
JSON representation

Nomad example job for deploying SQLite-backed applications with high availability

• Host: GitHub
• URL: https://github.com/cycneuramus/nomad-litestream-example
• Owner: cycneuramus
• Created: 2024-04-10T14:05:48.000Z (7 months ago)
• Default Branch: master
• Last Pushed: 2024-04-11T08:30:59.000Z (7 months ago)
• Last Synced: 2024-04-11T18:41:19.675Z (7 months ago)
• Topics: high-availability, litestream, nomad, s3, sqlite
• Language: HCL
• Homepage:
• Size: 1.95 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Nomad + Litestream example

This repository contains a hacky [Nomad](https://www.nomadproject.io) job that deploys an SQLite backed application (in this case, [Navidrome](https://github.com/navidrome/navidrome)) and leverages [Litestream](https://litestream.io) to synchronize the database to, and restore it from, an S3 endpoint.

## Why?

The point of this example job is to demonstrate a gross but functional way of achieving high availability with SQLite-based applications. SQLite, being file-based, doesn't inherently support the distributed data synchronization required by multi-node deployments, and is notoriously unreliable over network filesystems (e.g. NFS).

So, by continuously streaming the SQLite database changes to an S3 endpoint and—this is the gross part—restoring its state from this endpoint on every redeploy, this example job remains effectively agnostic to the underlying database's physical location.

## Prerequisites

+ A Nomad cluster with Docker runtime support

+ An S3 bucket (selfhosted or otherwise) with read/write permissions

## Usage

+ In the `.env` file: fill out your S3 credentials and configuration details.

+ In `navidrome.hcl`: change `~/path/to/navidrome-dir` to the relevant path on your host(s)

+ On all nodes: `mkdir ~/path/to/navidrome-dir/db`.

+ `nomad run navidrome.hcl`.

**NOTE**: For the sake of this example, the `cache` and `music` directories in the Nomad job are simple bind-mounts, which technically defeats the purpose of this pseudo-stateless deployment, so it's up to you to configure the storage mechanism of your choosing here.