{"id":17946478,"url":"https://github.com/lloydmeta/miniaturs","last_synced_at":"2025-09-07T06:36:46.773Z","repository":{"id":259892077,"uuid":"879710877","full_name":"lloydmeta/miniaturs","owner":"lloydmeta","description":"Making images small, with Rust","archived":false,"fork":false,"pushed_at":"2025-04-29T00:20:23.000Z","size":107,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-13T02:04:06.570Z","etag":null,"topics":["aws-lambda","image","image-processing","lambda","rust","rust-lang","serverless","terraform"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lloydmeta.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-10-28T12:18:19.000Z","updated_at":"2025-04-29T00:19:23.000Z","dependencies_parsed_at":"2025-02-09T03:30:32.975Z","dependency_job_id":"939f0967-c3b7-4e34-8ddb-73b4e15afa06","html_url":"https://github.com/lloydmeta/miniaturs","commit_stats":null,"previous_names":["lloydmeta/miniaturs"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/lloydmeta/miniaturs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lloydmeta%2Fminiaturs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lloydmeta%2Fminiaturs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lloydmeta%2Fminiaturs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lloydmeta%2Fminiaturs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lloydmeta","download_url":"https://codeload.github.com/lloydmeta/miniaturs/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lloydmeta%2Fminiaturs/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259565541,"owners_count":22877346,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws-lambda","image","image-processing","lambda","rust","rust-lang","serverless","terraform"],"created_at":"2024-10-29T07:05:48.770Z","updated_at":"2025-06-13T02:04:08.977Z","avatar_url":"https://github.com/lloydmeta.png","language":"Rust","readme":"# miniaturs\n[![Continuous integration](https://github.com/lloydmeta/miniaturs/actions/workflows/ci.yaml/badge.svg)](https://github.com/lloydmeta/miniaturs/actions/workflows/ci.yaml)\n\nTiny HTTP image resizer.\n\n## Goals\n\n* Secure\n* Fast:\n  * Startup should be in the low 2-digit ms range (e.g., avoid \"oh, it's a lambda\")\n  * Processing should be quick\n* Cheap:\n  * Pay as little as possible and avoid unnecessary work\n  * Being fast can help minimise costs\n* Scalable: can handle lots of requests\n* Thumbor-ish\n* Fetch from remote urls, but be good net citizen (don’t make requests to third parties if we have it in cache)\n* Debuggable\n\nTo fulfil the above:\n\n* Runs in a Lambda\n  * Running on ARM, [because it's cheaper](https://blog.awsfundamentals.com/aws-lambda-pricing-a-complete-guide-to-understanding-the-cost-of-the-serverless-service#heading-architectures-arm-vs-x86-arm-is-cheaper): \"about 34% cheaper compared to the default x86 processors\"\n* Rust ⚡️\n* Caching in layers: CDN to protect the app, with S3 for storing images\n* Serverless, but built on HTTP framework ([cargo-lambda](https://www.cargo-lambda.info) on top of [axum](https://github.com/tokio-rs/axum))\n\nAn example Terraform config in `terraform/prod` is provided to show how to deploy at a subdomain using Cloudflare as our ([free!](https://www.cloudflare.com/en-gb/plans/free/)) CDN + WAF with AWS Lambda and S3 ([also free!](https://aws.amazon.com/free/))\n\n## Usage:\n\nWe only support resizing at the moment\n\n1. An \"image\" endpoint [a la Thumbor](https://thumbor.readthedocs.io/en/latest/usage.html#image-endpoint)\n    * `GET /{HMAC_signature}/-Wx-H/{image_url}`\n2. A \"metadata\" endpoint [a la Thumbor](https://thumbor.readthedocs.io/en/latest/usage.html#metadata-endpoint)\n    * `GET /{HMAC_signature}/meta/-Wx-H/{image_url}`\n    * Difference: target image size is _not_ returned (might change in the future)\n\n### Confguration\n\nminiaturs relies on environment variables for configuration. These include\n\n* `MINIATURS_SHARED_SECRET`   : required, used for signature verification\n* `UNPROCESSED_IMAGES_BUCKET` : required, bucket used for caching unprocessed images\n* `PROCESSED_IMAGES_BUCKET`   : required, bucket used for caching processed images\n* `REQUIRE_PATH_STYLE_S3`     : optional, whether to use \"path style\" S3 addressing (for local testing), defaults to false.\n* `MAX_RESIZE_TARGET_WIDTH`   : optional, max resize-to image width, defaults to 10,000 (pixels)\n* `MAX_RESIZE_TARGET_HEIGHT`  : optional, max resize-to image height, defaults to 10,000 (pixels)\n* `MAX_SOURCE_IMAGE_WIDTH`    : optional, max source image width, defaults to 10,000 (pixels)\n* `MAX_SOURCE_IMAGE_HEIGHT`   : optional, max source image height, defaults to 10,000 (pixels)\n* `MAX_IMAGE_DOWNLOAD_SIZE`   : optional, max source image download size (as reported by content-length header), defaults to 10mb (must be parseable by [bytesize](https://crates.io/crates/bytesize))\n* `MAX_IMAGE_FILE_SIZE`       : optional, max source (post-download) image size, defaults to 10mb (must be parseable by [bytesize](https://crates.io/crates/bytesize))\n\n## Flow\n\nFlow for an image resize request. The CDN/WAF graph is included since it's an important part of keeping costs low (and included in the prod terraform example), but is not a _must_.\n\n```mermaid\nflowchart\n   imageReq((\"Image request\"))\n\n\n   respEnd((Ok/Error response))\n\n   imageReq --\u003e cdnReq\n   cdnResp --\u003e respEnd\n\n   subgraph CDN/WAF\n      cdnReq[\\\"Request\"\\]\n      cdnResp[\\\"Ok/Error Response\"\\]\n      cdnCache[(\"CDN Cache\")]\n      wafBlocked{\"Rate limited?\"}\n      cdnCached{\"Cached?\"}\n      cache[\"Cache\"]\n\n      cdnReq --\u003e wafBlocked\n      wafBlocked --\u003e|yes| cdnResp\n\n      wafBlocked --\u003e|no| cdnCached\n\n      cdnCached \u003c--\u003e|fetch| cdnCache\n      cdnCached --\u003e|yes| cdnResp\n\n\n      cache --\u003e|cache| cdnCache\n      cache --\u003e cdnResp\n\n\n   end\n\n   subgraph Lambda\n\n      lambdaRequest[\\\"Request\"\\]\n      sigCheck{\"Signature OK\"?}\n      toOps[To Operations]\n      opsCheck{\"Operations OK?\"}\n\n      processedCache[(\"Processed cache\")]\n      fetchCachedResult[\"Fetch cached result\"]\n      opsResultCached{\"Operations Result Cached?\"}\n\n\n      rawCache[(\"Unprocessed cache\")]\n      fetchCachedRaw[\"Fetch cached unprocessed image\"]\n      rawCached{\"Unprocessed image Cached?\"}\n\n      fetchRaw[\"Fetch unprocessed image from remote\"]\n      rawImageValidation{\"Unprocessed image validations pass?\"}\n      cacheRaw[\"Cache unprocessed image\"]\n\n      processImage[\"Process image according to operations\"]\n      cacheResult[\"Cache processed result\"]\n      lambdaResponse[\\\"Ok/Error Response\"\\]\n\n\n      cdnCached --\u003e|no| lambdaRequest\n      lambdaRequest --\u003e sigCheck\n      sigCheck --\u003e|no| lambdaResponse\n      sigCheck --\u003e|yes| toOps\n\n      toOps --\u003e opsCheck\n      opsCheck --\u003e|no| lambdaResponse\n      opsCheck --\u003e|yes| fetchCachedResult\n      fetchCachedResult \u003c--\u003e|fetch| processedCache\n      fetchCachedResult --\u003e opsResultCached\n      opsResultCached --\u003e|yes| lambdaResponse\n\n      opsResultCached --\u003e|no| fetchCachedRaw\n      fetchCachedRaw \u003c--\u003e|fetch| rawCache\n      fetchCachedRaw --\u003e rawCached\n      rawCached --\u003e|no| fetchRaw\n      fetchRaw --\u003e rawImageValidation\n      rawImageValidation --\u003e|no| lambdaResponse\n      rawImageValidation --\u003e|yes| cacheRaw\n      cacheRaw --\u003e|cache| rawCache\n\n      cacheRaw --\u003e processImage\n      rawCached --\u003e|yes| processImage\n\n      processImage --\u003e cacheResult\n      cacheResult --\u003e|cache| processedCache\n\n      cacheResult --\u003e lambdaResponse\n      lambdaResponse --\u003e cache\n   end\n\n```\n\n## Development\n\n### Rust\n\nAssuming we have the [Rust toolbelt installed](https://doc.rust-lang.org/cargo/getting-started/installation.html#install-rust-and-cargo), the main thing we need is `cargo-lambda`\n\n```sh\n❯ brew tap cargo-lambda/cargo-lambda\n❯ brew install cargo-lambda\n```\n\n### AWS\n\n* `brew install awscli` to install the CLI\n* Log into your app\n\nEnsure:\n\n* `aws configure sso` is done\n* `.aws/config` has the correct profile configuration, with a `[profile ${PROFILE_NAME}]` line where `PROFILE_NAME` matches what is in `main.tf`\n\n#### Login for Terraform\n\n`aws sso login --profile ${PROFILE_NAME}`\n\n### Cloudflare\n\nEnsure `CLOUDFLARE_API_TOKEN` is defined in the environment (needed for Cloudflare provider and cache busting). It’ll need privileges for updating DNS and cache settings.\n\n## Deploying\n\n### Terraform\n\n* Use tfenv: https://formulae.brew.sh/formula/tfenv\n* Check what version is needed and install using ^\n\n* For local dev, `localstack` is used (see terraform/localdev/docker-compose.yaml), and `tflocal` is used (https://formulae.brew.sh/formula/terraform-local)\n  * `docker-compose` through official Docker _or_ Rancher is supported, but [enabling admin access](https://github.com/rancher-sandbox/rancher-desktop/issues/2534#issuecomment-1909912585) is needed for running tests with Rancher if simply setting the following in shell does not work\n\n    ```sh\n    export DOCKER_HOST=unix://$HOME/.rd/docker.sock\n    export TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE=/var/run/docker.sock\n    export TESTCONTAINERS_HOST_OVERRIDE=$(rdctl shell ip a show vznat | awk '/inet / {sub(\"/.*\",\"\"); print $2}')\n    ```\n\n### Per Environment\n\nUse `Makefile` targets.\n\n* For local dev:\n    * `make init_dev_env` (only needed if TF complains)\n    * `make start_dev_env provision_dev_env`\n    * `make begin_dev`\n    * `TO_SIGN=\"200x-100/https://beachape.com/images/octopress_with_container.png\" make signature_for_localstack` to get a signed path for dev env\n    * `TO_SIGN=\"200x-100/https://beachape.com/images/octopress_with_container.png\" make signature_for_dev` to get a signed path for dev\n* For prod:\n    * Copy and customise:\n      * `main.tf.example` to `main.tf`\n      * `terraform.tfvars.example` to `terraform.tfvars`\n    * `make plan_prod` to see changes\n    * `make provision_prod` to apply changes\n    * `TO_SIGN=\"200x-100/https://beachape.com/images/octopress_with_container.png\" make signature_for_prod` to get a signed path\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flloydmeta%2Fminiaturs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flloydmeta%2Fminiaturs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flloydmeta%2Fminiaturs/lists"}