{"id":18684810,"url":"https://github.com/j-sephb-lt-n/cloud-sqlite","last_synced_at":"2025-09-18T17:06:22.679Z","repository":{"id":231556205,"uuid":"781926554","full_name":"J-sephB-lt-n/cloud-sqlite","owner":"J-sephB-lt-n","description":"A serverless sqlite database on google cloud run","archived":false,"fork":false,"pushed_at":"2024-10-12T18:16:56.000Z","size":28,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-12-28T00:21:09.174Z","etag":null,"topics":["database","gcp","google-cloud","microservice","serverless","sqlite"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/J-sephB-lt-n.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-04-04T10:00:44.000Z","updated_at":"2024-10-12T18:16:59.000Z","dependencies_parsed_at":"2024-05-17T20:48:11.123Z","dependency_job_id":"19f5e92e-36d4-4fcc-abe6-ac093d72bde6","html_url":"https://github.com/J-sephB-lt-n/cloud-sqlite","commit_stats":null,"previous_names":["j-sephb-lt-n/serverless-sqlite-db-on-google-cloud","j-sephb-lt-n/cloud-sqlite"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/J-sephB-lt-n%2Fcloud-sqlite","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/J-sephB-lt-n%2Fcloud-sqlite/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/J-sephB-lt-n%2Fcloud-sqlite/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/J-sephB-lt-n%2Fcloud-sqlite/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/J-sephB-lt-n","download_url":"https://codeload.github.com/J-sephB-lt-n/cloud-sqlite/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239539713,"owners_count":19655859,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","gcp","google-cloud","microservice","serverless","sqlite"],"created_at":"2024-11-07T10:19:25.628Z","updated_at":"2025-09-18T17:06:17.614Z","avatar_url":"https://github.com/J-sephB-lt-n.png","language":"Shell","readme":"# serverless-SQLite-db-on-google-cloud\n\nI love the idea of a persistent SQL database (with streaming replication) that is available on a serverless infrastructure (such as GCP Cloud Run or AWS Lambda).\n\nWith a SQLite database, [litestream](https://github.com/benbjohnson/litestream) provides streaming replication to cloud storage, and in theory GCP Cloud Run can just replicate from cloud storage when cold starting using the [litestream](https://github.com/benbjohnson/litestream) backup.\n\nHowever, I'm not sure if the max_vms=1 on Cloud Run can be 100% be relied upon, and if Cloud Run happens to start 2 VMs, then the integrity of the data will be stuffed up.\n\nAnother problem is that SQLite is not designed to be hosted on a server and accessed by many users over a network, so this will also be a bit of a hack.\n\nHere is some code illustrating how [litestream](https://github.com/benbjohnson/litestream) works (using a Compute Engine VM):\n\nCreate an E2-Micro with read/write access to Cloud Storage\n\nRun the following on the VM:\n\n```bash\n# set up \"db_admin\" permission group #\nsudo groupadd db_admin\nsudo usermod --append --groups db_admin joseph_bolton # add myself to the group\nsu joseph_bolton # log in again to make the group change take effect\nid joseph_bolton\n\nsudo mkdir /var/lib/sqlite_databases/\nsudo chgrp db_admin /var/lib/sqlite_databases/\nsudo chmod g+rwx /var/lib/sqlite_databases/\n\n# setup dev environment #\nwget https://raw.githubusercontent.com/J-sephB-lt-n/my-personal-configs/main/setup_joes_dev_environment.sh\nsudo bash setup_joes_dev_environment.sh\nrm setup_joes_dev_environment.sh\n\n# litestream setup #\nwget https://github.com/benbjohnson/litestream/releases/download/v0.3.13/litestream-v0.3.13-linux-amd64.deb\nsudo dpkg -i litestream-v0.3.13-linux-amd64.deb\nsudo chgrp db_admin /etc/litestream.yml\nsudo chmod g+w /etc/litestream.yml\necho \"\ndbs:\n  - path: /var/lib/sqlite_databases/test_db.db\n    replicas:\n      - url: gcs://sqlite-db-litestream-backups/test_db.db\n\" \u003e /etc/litestream.yml\nsudo systemctl enable litestream\nsudo systemctl start litestream\njournalctl -u litestream -f # this shows the litestream logs\n\n# create SQL databases #\nsudo apt-get install sqlite3\n\nsqlite3 /var/lib/sqlite_databases/test_db.db \u003c\u003cEOF\nCREATE TABLE users (user_id INTEGER PRIMARY KEY, first_name TEXT, last_name TEXT);\nINSERT INTO users (first_name, last_name) VALUES ('oscar', 'peterson');\nINSERT INTO users (first_name, last_name) VALUES ('bill', 'evans');\nPRAGMA busy_timeout = 5000;\nPRAGMA synchronous = NORMAL;\nPRAGMA wal_autocheckpoint = 0;\nEOF\n```\n\nTo create a new database from a litestream Cloud Storage backup:\n\n```bash\nlitestream restore -o /var/lib/db_restore_mydb1.db gcs://sqlite-db-litestream-backups/test_db.db\n```\n\nSet up the python flask/gunicorn app:\n\n```bash\nsudo apt-get install -y python3-pip\nsudo apt-get install -y python3.11-venv\npython3 -m venv venv\nsource venv/bin/activate\npip3 install Flask gunicorn\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fj-sephb-lt-n%2Fcloud-sqlite","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fj-sephb-lt-n%2Fcloud-sqlite","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fj-sephb-lt-n%2Fcloud-sqlite/lists"}