https://github.com/j-sephb-lt-n/cloud-sqlite

A serverless sqlite database on google cloud run
https://github.com/j-sephb-lt-n/cloud-sqlite

database gcp google-cloud microservice serverless sqlite

Last synced: 14 days ago
JSON representation

A serverless sqlite database on google cloud run

• Host: GitHub
• URL: https://github.com/j-sephb-lt-n/cloud-sqlite
• Owner: J-sephB-lt-n
• License: gpl-3.0
• Created: 2024-04-04T10:00:44.000Z (9 months ago)
• Default Branch: main
• Last Pushed: 2024-10-12T18:16:56.000Z (3 months ago)
• Last Synced: 2024-11-07T10:19:47.258Z (2 months ago)
• Topics: database, gcp, google-cloud, microservice, serverless, sqlite
• Language: Shell
• Homepage:
• Size: 27.3 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# serverless-SQLite-db-on-google-cloud

I love the idea of a persistent SQL database (with streaming replication) that is available on a serverless infrastructure (such as GCP Cloud Run or AWS Lambda).

With a SQLite database, [litestream](https://github.com/benbjohnson/litestream) provides streaming replication to cloud storage, and in theory GCP Cloud Run can just replicate from cloud storage when cold starting using the [litestream](https://github.com/benbjohnson/litestream) backup.

However, I'm not sure if the max_vms=1 on Cloud Run can be 100% be relied upon, and if Cloud Run happens to start 2 VMs, then the integrity of the data will be stuffed up.

Another problem is that SQLite is not designed to be hosted on a server and accessed by many users over a network, so this will also be a bit of a hack.

Here is some code illustrating how [litestream](https://github.com/benbjohnson/litestream) works (using a Compute Engine VM):

Create an E2-Micro with read/write access to Cloud Storage

Run the following on the VM:

```bash

# set up "db_admin" permission group #

sudo groupadd db_admin

sudo usermod --append --groups db_admin joseph_bolton # add myself to the group

su joseph_bolton # log in again to make the group change take effect

id joseph_bolton

sudo mkdir /var/lib/sqlite_databases/

sudo chgrp db_admin /var/lib/sqlite_databases/

sudo chmod g+rwx /var/lib/sqlite_databases/

# setup dev environment #

wget https://raw.githubusercontent.com/J-sephB-lt-n/my-personal-configs/main/setup_joes_dev_environment.sh

sudo bash setup_joes_dev_environment.sh

rm setup_joes_dev_environment.sh

# litestream setup #

wget https://github.com/benbjohnson/litestream/releases/download/v0.3.13/litestream-v0.3.13-linux-amd64.deb

sudo dpkg -i litestream-v0.3.13-linux-amd64.deb

sudo chgrp db_admin /etc/litestream.yml

sudo chmod g+w /etc/litestream.yml

echo "

dbs:

  - path: /var/lib/sqlite_databases/test_db.db

    replicas:

      - url: gcs://sqlite-db-litestream-backups/test_db.db

" > /etc/litestream.yml

sudo systemctl enable litestream

sudo systemctl start litestream

journalctl -u litestream -f # this shows the litestream logs

# create SQL databases #

sudo apt-get install sqlite3

sqlite3 /var/lib/sqlite_databases/test_db.db <