Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/code402/datasette-lambda

Run Datasette on AWS serverless.
https://github.com/code402/datasette-lambda

asgi cloudformation datasette mangum sqlite

Last synced: about 1 month ago
JSON representation

Run Datasette on AWS serverless.

Awesome Lists containing this project

README 

        
# datasette-lambda



Run [Datasette](https://github.com/simonw/datasette) on AWS as a serverless application:






Sufficiently small databases (unzipped size up to ~250 MB, zipped size up to ~50 MB) will be inlined in the Lambda deployment package. Others will be published to S3 and fetched on Lambda startup.



You can see a demo using Datasette's fixtures db here: https://datasette-demo.code402.com/



## Getting started



### Creating



Clone the repo and run `./update-stack  [flags]  [ ...]`, e.g.:



```bash

git clone https://github.com/code402/datasette-lambda.git

cd datasette-lambda

./update-stack northwinds northwinds.db`

```



Some Datasette flags are supported:



- `--config key:value`, to set [config options](https://datasette.readthedocs.io/en/stable/config.html)

- `--cors`, to enable `Access-Control-Allow-Origin: *` headers on responses

- `--metadata `, to provide [metadata](https://datasette.readthedocs.io/en/stable/metadata.html)



And some non-Datasette flags are supported:



- `--domain example.com` or `--domain subdomain.example.com`, if `example.com` is a hosted zone in Route 53

  - register a `CNAME` record that points to the CloudFront distribution

  - register an SSL certificate for the domain (you'll have to ack a confirmation email from Amazon)

  - associate that certificate to the CloudFront distribution

- `--prefix some/path`, to mount the Datasette app at a path other than the root



A CloudFormation stack will be created (or updated) with an S3 bucket.



The stub code and SQLite database(s) will be uploaded to the S3 bucket.



A second CloudFormation stack will then be created (or updated) with the necessary

IAM roles, CloudFront, API Gateway and Lambda entities to expose your Datasette

instance to the web.



### Watching logs



`./tail-logs ` will watch the CloudWatch logs for the Lambda (NB: not the API Gateway) service - this can be useful for debugging runtime errors in Datasette itself.



### Destroying



Run `./delete-stack ` to tear down the infrastructure.



_Note: AWS has a rough edge with deleting Lambda@Edge functions. You will need to run `delete-stack`, then wait a period of time, and run it again for the entire stack to be successfully removed. Ref: [AWS docs](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-edge-delete-replicas.html)_



## Known issues / future work



- [x] Downloads from S3 should use an atomic fetch/rename to be robust against transient errors

- [x] We should embed the DB in the Lambda package itself, when possible, to avoid the coldstart S3 fetch

- [x] Repeated calls of update-stack should be robust against template-not-changed errors

- [x] Fix issue with `base_url` not always being respected in generated URLs (maybe issue in how we use Mangum?)

- [x] Be able to host multiple DBs

- [x] Use the passed-in name of the DB as the DB name

- [x] Create a CloudFront distribution

- [x] Optionally be able to use a custom domain name on CloudFront

- [x] Parity: Support CORS flag

- [x] Parity: Support metadata flag

- [x] Parity: Support config options

- [x] Fix wrong absolute URLs for facets/next page (ds.absolute_url)

- [ ] Use API Gateway's faster/cheaper HTTP APIs instead of REST APIs (requires [erm/mangum #94](https://github.com/erm/mangum/pull/94))



Maybe:



- [x] Be able to customize the "mount" point of the CloudFront distribution

- [ ] Add support into core datasette's `publish` command, fixing [#236](https://github.com/simonw/datasette/issues/236)