Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/mnapoli/sqlite-s3
Serverless dev database: SQLite backed by S3
https://github.com/mnapoli/sqlite-s3
s3 sqlite sqlite-s3
Last synced: 6 days ago
JSON representation
Serverless dev database: SQLite backed by S3
- Host: GitHub
- URL: https://github.com/mnapoli/sqlite-s3
- Owner: mnapoli
- License: mit
- Created: 2023-12-14T18:16:57.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2024-10-07T18:55:47.000Z (3 months ago)
- Last Synced: 2024-12-14T16:08:58.799Z (13 days ago)
- Topics: s3, sqlite, sqlite-s3
- Language: PHP
- Homepage:
- Size: 26.4 KB
- Stars: 93
- Watchers: 2
- Forks: 2
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
README
# Serverless dev database: SQLite backed by S3
## Why?
A "serverless" SQL database:
- For development and testing purposes
- Costs $0
- As simple to use as possible
- Ideal for [serverless environments](https://bref.sh/) like AWS Lambda
**Not for production use-cases**. It does not handle concurrent updates (in which case some data might be lost) and performances are not production-grade.
## How?
The SQLite database (a file) is stored on S3. The PHP class will transparently download the file locally on every request, and upload it back at the end.
This has two obvious implications:
1. If two concurrent requests download the database file, update it (separately), and upload it back, then the last to upload the modified file will overwrite the changes of the other request.
2. There is extra latency added to the request (and it wouldn't work well with huge databases).
That is why this solution is best for testing scenarios (e.g. testing a fully deployed application, where there is one test running at a time). It could also work for development environments with only one active user at a time, where an extra 50ms-100ms per request is acceptable.
## Setup
You will need an AWS S3 bucket (where the database will be stored). The S3 bucket must exist, but **the SQLite database file will automatically be created** if it doesn't.
Install the package with Composer:
```sh
composer require mnapoli/sqlite-s3
```
## Usage
### With Laravel
Update `.env` (or set environment variables) to set:
- `DB_CONNECTION=sqlite`
- `DB_DATABASE='s3://the-s3-bucket-name/a-file-name.sqlite'`
The `DB_DATABASE` usually contains a file name, but here it will contain a S3 URL. That URL will be automatically detected to retrieve the database from S3.
The database will be uploaded to S3 on every request. When running on AWS Lambda with Bref, it will be uploaded/synced on every AWS Lambda invocation too.
Outside of Lambda (for example in test code), call `DB::purge();` to force the database to be synced to S3.
### Generic PHP application
Instead of:
```php
$db = new PDO('sqlite:test-db.sqlite');
$db->exec('SELECT * FROM my-table');
```
Use:
```php
$db = new PDOSQLiteS3('the-s3-bucket-name', 'a-file-name.sqlite');
$db->exec('SELECT * FROM my-table');
```
The database will be uploaded back to S3 when the `$db` instance is destroyed (i.e. when the PDO connection is closed).
### Configuration
If needed, set the AWS region:
```php
$db = new PDOSQLiteS3('the-s3-bucket-name', 'a-file-name.sqlite', [
'region' => 'us-east-1',
]);
```
The AWS credentials will automatically be picked up by the AWS SDK. The [Async-AWS](https://async-aws.com/) library is used under the hood, check out [its documentation](https://async-aws.com/authentication/).
### Without PDO
If you are using the [`SQLite3` class](https://www.php.net/manual/en/class.sqlite3.php) directly, replace it with the `SQLiteS3` class:
```php
$db = new SQLiteS3('the-s3-bucket-name', 'a-file-name.sqlite');
```