Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/mbenoukaiss/vmod-impress

Optimizes images by resizing, compressing and converting them to the best format supported by the client
https://github.com/mbenoukaiss/vmod-impress

avif image optimization resize varnish vmod webp

Last synced: 3 months ago
JSON representation

Optimizes images by resizing, compressing and converting them to the best format supported by the client

Awesome Lists containing this project

README 

          
# Impress

A Varnish module that resizes, compresses and converts images on the fly to

the best format the requesting client supports (AVIF, WebP, JPEG), and that

also serves general static files (HTML, CSS, JS, JSON, fonts, raw images,

…) with per-extension in-process minification on cache miss. Optimized

image variants are written to a local cache directory; static-file output

is held in memory only as long as Varnish needs it, then re-derived on the

next miss. Both paths emit proper HTTP cache validators (`ETag`,

`Last-Modified`, `stale-while-revalidate`) so Varnish itself can

revalidate cheaply.



## Setting up the plugin



Download [`libvmod_impress.so`](https://github.com/mbenoukaiss/vmod-impress/releases/latest/download/libvmod_impress.so)

from the latest release and place it in your Varnish vmods directory

(typically `/usr/lib/varnish/vmods`).



Minimal VCL — the recommended setup configures the module entirely from

`vcl_init` using the builder API:



```vcl

vcl 4.1;



import impress;



backend default none;



sub vcl_init {

    new mod = impress.engine(

        url = "/media/{size}/{path}[.{ext}]",

        cache_directory = "/var/cache/varnish",

        default_format = "jpeg"

    );



    mod.add_root("/var/www/media");



    mod.add_extension("avif");

    mod.add_extension("webp");

    mod.add_extension("jpeg");



    mod.add_size(name = "low",     width = 300,  height = 300);

    mod.add_size(name = "medium",  width = 600,  height = 600);

    mod.add_size(name = "high",    width = 1200, height = 1200);

    mod.add_size(name = "product", width = 546,  height = 302,

                 pattern = "^products/", pre_optimize = true);



    mod.build();

}



sub vcl_recv {

    set req.backend_hint = mod.backend();

}

```



Configure Varnish to vary on / sanitize the `Accept` header for image URLs;

otherwise different clients may get the wrong format from a single cache key.



The instance variable is named `mod` because VCL forbids reusing the

imported VMOD name (`impress`) for a `new` declaration. Any name that

isn't already in scope works — pick whatever reads best for you.



## Configuration



The VMOD accepts configuration via three equivalent surfaces:



1. The VCL builder API (`impress.engine(...)` plus `add_*` / `set_*` /

   `build`). Keeps everything in VCL and reloads atomically with the rest

   of your VCL.

2. A JSON file passed to `impress.new(path)`. Useful when the config is

   long, generated by tooling, or shared across multiple VCLs.

3. A RON file passed to `impress.new(path)`. The original supported

   format; identical schema to JSON.



The schema is the same in all three cases. The reference tables below

describe the model abstractly, with the corresponding VCL builder method

in parentheses; the JSON/RON file shape is shown in the [JSON](#json-file-configuration)

and [RON](#ron-file-configuration) sections.



### `engine` constructor (`impress.engine(...)`)



The constructor takes the global, scalar settings. The collections

(`extensions`, `roots`, `sizes`, `static`, `logger`) are set via separate

methods so the module isn't constrained by VCL's lack of compound-type

literals.



| Argument | Required | Default | Description |

|---|---|---|---|

| `url` | yes | — | URL pattern with `{size}`, `{path}`, optional `{ext}`, and `[...]` for optional segments — see the URL pattern section. |

| `cache_directory` | yes | — | Directory where optimized variants are written. Layout is `//.`. |

| `default_format` | no | `"jpeg"` | Format served when the client's `Accept` header doesn't match any of the registered extensions. One of `jpeg`, `webp`, `avif`. |

| `quality_jpeg` | no | `90` | Default JPEG quality used by every `add_size` unless overridden. |

| `quality_webp` | no | `70` | Default WebP quality. |

| `quality_avif` | no | `40` | Default AVIF quality. |

| `cache_control` | no | `"public, max-age=86400, stale-while-revalidate=604800"` | Raw `Cache-Control` value used on optimized image responses and static-file responses. The image in-flight fallback (raw bytes served while optimization is running) is hardcoded to `"no-cache"` and is not configurable — caching it would pin the un-optimized variant at the HTTP layer until the next mtime. |

| `pre_optimizer_threads` | no | `1` | Threads in the optimization pool. |



### Builder methods



All methods are valid only between the constructor and `build()`. After

`build()` they error.



| Method | Description |

|---|---|

| `add_root(STRING path)` | Append a source-image directory. At least one root is required. Multiple roots supported; paths in URLs are matched against the first root that contains the requested file. |

| `add_extension(STRING format)` | Append an optimized format to the priority list. One of `jpeg`, `webp`, `avif`. At least one is required. |

| `add_size(STRING name, INT width, INT height, REAL quality_jpeg=…, REAL quality_webp=…, REAL quality_avif=…, STRING pattern=…, BOOL pre_optimize=…)` | Register a named size — see the table below. At least one size is required. |

| `add_static(STRING url, STRING root, STRING cache_control=…, BOOL optimize_html=true, BOOL optimize_css=true, BOOL optimize_js=false, BOOL optimize_json=true, INT optimize_max_bytes=…)` | Register a static-file route — see the table below. |

| `set_logger(STRING path, STRING level="info")` | Enable file logging. `level` is one of `off`, `error`, `warn`, `info`, `debug`, `trace` (case-insensitive). Calling `set_logger` more than once replaces the previous setting. |

| `build()` | Finalize: validate config, compile URL patterns, canonicalize static roots, set up logging, and create the backend. Must be called once at the end of `vcl_init`. |

| `backend()` | Return the backend handle. Use only in request-time subroutines (`vcl_recv` etc.) and only after `build()` has run in `vcl_init`. |



### URL pattern (`url`)



The `url` argument describes how to extract `size`, `path`, and optionally `ext` from the request URL.



- `{size}` (required) — matched against the names registered with `add_size`.

- `{path}` (required) — image identifier, used to locate the source under one of the registered roots.

- `{ext}` (optional) — explicit extension; usually wrapped in `[...]` so it can be omitted.

- `[...]` — wraps a portion of the URL that is optional.



Examples:



```

url = "/media/{size}/{path}[.{ext}]"

   matches /media/medium/products/logo

       and /media/medium/products/logo.jpg

```



### `add_size` parameters



| Parameter | Required | Description |

|---|---|---|

| `name` | yes | Name matched against `{size}` in the URL pattern. |

| `width` | yes | Maximum width of the resized variant. |

| `height` | yes | Maximum height of the resized variant. |

| `quality_jpeg` / `quality_webp` / `quality_avif` | no | Per-size quality override; falls back to the engine-level `quality_*`, then to a built-in default per format. |

| `pattern` | no | Regex that must match `{path}` for this size to apply. If absent, all paths match. |

| `pre_optimize` | no | If `true`, every matching image is optimized into all configured extensions at startup and on watcher modification, instead of lazily on first request. Recommended only with a `pattern` so you don't pre-optimize the whole tree. |



### Disk cleanup



Orphan cleanup runs unconditionally — there is no `cleanup` config block:



- A startup sweep walks `cache_directory` once after `load_images` and

  removes any cache file whose source no longer exists or whose

  size/extension is no longer in the current configuration.

- A periodic sweep thread runs the same logic every 24 hours so

  long-running varnishd instances reclaim disk space without a restart.



The cleaner removes only **orphans** — cache files for which the source has

been deleted, or that don't fit the current `add_size` / `add_extension`

configuration. There is no LRU or size-cap eviction; the source filesystem

is the source of truth for what should exist in the cache.



### `add_static` parameters (optional, repeatable)



Each call serves files from `root` under the URL pattern `url`. Routes

evaluate top-down in the order they were added; the first route whose URL

pattern matches owns the response — even if the file is missing or the

path-traversal guard trips, the route does not fall through to a later

route or to the image regex. Add narrow routes before broad ones.



| Parameter | Required | Default | Description |

|---|---|---|---|

| `url` | yes | — | URL pattern. Must contain `{path}`. `[...]` denotes optional segments. Example: `/assets/{path}`. |

| `root` | yes | — | Filesystem root the route serves from. Canonicalized once at `build()` time. Requests are joined under this root and rejected if the resolved path escapes (parent-dir, absolute, or symlink to outside-root). |

| `cache_control` | no | engine default | Per-route `Cache-Control` override. |

| `optimize_html` | no | `true` | Run minify-html on `text/html` responses. Also runs lightningcss / oxc on inline `` / `<script>`. |

| `optimize_css` | no | `true` | Run lightningcss on `text/css` responses. |

| `optimize_js` | no | `false` | Run oxc_minifier on JS responses. Defaults **off** because the crate is alpha and has shipped semantically-broken outputs on edge cases — opt in only after batch-testing against your asset corpus. |

| `optimize_json` | no | `true` | Run serde_json (parse + compact-serialize) on JSON responses. |

| `optimize_max_bytes` | no | `2_097_152` (2 MiB) | Files larger than this skip optimization and stream from disk regardless of toggles — keeps a single big file from OOMing a worker. `0` removes the cap. |



SVG is served unoptimized — `oxvg_optimiser` pins a `lightningcss` feature

that conflicts with the version `minify-html` requires, so the two crates

can't coexist in this dep graph.



When optimization runs, output bytes ship via an in-memory `MemoryTransfer`.

On no-improvement (output not strictly smaller than the input) we still

ship the in-heap bytes rather than re-opening the file — that avoids the

read-after-rename TOCTOU window. Files past `optimize_max_bytes` and files

with optimization toggled off stream directly from disk via `FileTransfer`.



ETag for static responses is hashed from

`(inode, body_len, mtime_secs, mime, is_optimized, STATIC_OPTIMIZER_VERSION)`.

The version constant is bumped manually after a minifier crate upgrade

that changes output bytes for unchanged sources, so clients holding an

old `If-None-Match` get a real 200 with the new bytes rather than a stale

304.



### `set_logger` parameters



| Parameter | Required | Description |

|---|---|---|

| `path` | yes | Log file path. |

| `level` | no | Minimum level of log; entries below are dropped. Default `info`. One of `off`, `error`, `warn`, `info`, `debug`, `trace`. |



### Full builder example



```vcl

sub vcl_init {

    new mod = impress.engine(

        url = "/media/{size}/{path}[.{ext}]",

        cache_directory = "/var/cache/varnish",

        default_format = "jpeg",

        quality_webp = 70,

        quality_avif = 40,

        cache_control = "public, max-age=86400, stale-while-revalidate=604800"

    );



    mod.add_root("/var/www/media");



    mod.add_extension("avif");

    mod.add_extension("webp");



    mod.add_size(name = "low",     width = 300,  height = 300,

                 quality_webp = 90, quality_jpeg = 100);

    mod.add_size(name = "medium",  width = 600,  height = 600);

    mod.add_size(name = "high",    width = 1200, height = 1200);

    mod.add_size(name = "product", width = 546,  height = 302,

                 pattern = "^products/", pre_optimize = true);



    mod.add_static(url = "/assets/{path}", root = "/var/www/static",

                   optimize_js = true);



    mod.set_logger(path = "/var/log/impress.log", level = "warn");



    mod.build();

}

```



## File-based configuration



If a VCL builder block doesn't fit your workflow — for example because

the configuration is generated by tooling, or shared between several

VCLs — pass a path to a JSON or RON file instead. Format is selected by

the file extension: `.json` is parsed as JSON, `.ron` as RON; any other

extension produces an error at `vcl.load`.



```vcl

sub vcl_init {

    new mod = impress.new("/etc/varnish/impress.json");

}



sub vcl_recv {

    set req.backend_hint = mod.backend();

}

```



The schema mirrors the builder API exactly: every field listed above maps

to a top-level key (or, for `static`, an array entry).



### JSON file configuration



```json

{

    "extensions": ["AVIF", "WEBP"],

    "default_format": "JPEG",

    "qualities": {"WEBP": 70, "AVIF": 40},

    "roots": ["/var/www/media"],

    "url": "/media/{size}/{path}[.{ext}]",

    "cache_directory": "/var/cache/varnish",

    "sizes": {

        "low":     {"width": 300,  "height": 300, "qualities": {"WEBP": 90, "JPEG": 100}},

        "medium":  {"width": 600,  "height": 600},

        "high":    {"width": 1200, "height": 1200},

        "product": {"width": 546,  "height": 302, "pattern": "^products/", "pre_optimize": true}

    },

    "cache_control": "public, max-age=86400, stale-while-revalidate=604800",

    "static": [

        {

            "url": "/assets/{path}",

            "root": "/var/www/static",

            "optimization": {"js": true}

        }

    ],

    "logger": {

        "path": "/var/log/impress.log",

        "level": "WARN"

    }

}

```



### RON file configuration



RON is also supported and uses the same schema. It permits trailing

commas and inline comments and is the original format supported by this

VMOD.



```ron

Config(

    extensions: [AVIF, WEBP],

    default_format: JPEG,

    qualities: {WEBP: 70, AVIF: 40},

    roots: ["/var/www/media"],

    url: "/media/{size}/{path}[.{ext}]",

    cache_directory: "/var/cache/varnish",

    sizes: {

        "low":     Size(width: 300,  height: 300, qualities: {WEBP: 90, JPEG: 100}),

        "medium":  Size(width: 600,  height: 600),

        "high":    Size(width: 1200, height: 1200),

        "product": Size(width: 546,  height: 302, pattern: "^products/", pre_optimize: true),

    },

    cache_control: "public, max-age=86400, stale-while-revalidate=604800",

    static: [

        StaticRoute(

            url: "/assets/{path}",

            root: "/var/www/static",

            optimization: Optimization(js: true),

        ),

    ],

    logger: Logger(

        path: "/var/log/impress.log",

        level: WARN,

    ),

)

```



## On-the-fly minifier (`impress_minify`)



The VMOD registers a Varnish Fetch Processor named `impress_minify` that

buffers a backend response, minifies it, and stores the minified bytes in

the cache. Subsequent cache hits don't run the filter.



Wire it up in `vcl_backend_response`:



```vcl

import impress;



sub vcl_backend_response {

    if (beresp.http.content-type ~ "^(text/(html|css|javascript)|application/(json|javascript))") {

        # Gunzip the backend body so the minifier sees plaintext; Varnish

        # re-gzips on the way to storage/clients.

        set beresp.do_gunzip = true;

        # Insert impress_minify *before* the trailing gzip filter — appending

        # would feed gzipped bytes into the minifier and silently no-op.

        set beresp.filters = regsuball(beresp.filters, "(^| )gzip( |$)", "\1impress_minify gzip\2");

        if (beresp.filters !~ "impress_minify") {

            set beresp.filters = beresp.filters + " impress_minify";

        }

    }

}

```



Supported types: `text/html`, `text/css`, `text/javascript`,

`application/javascript`, `application/json`.



The filter only engages on cacheable responses. First request to a fresh

URL pays a TTFB hit equal to the backend response time plus a few ms of

minify; cache hits are unaffected. Pair with

`Cache-Control: stale-while-revalidate=N` to amortize refreshes.



## Cache freshness model



The on-disk optimized cache is kept in sync with the source images via three

layers, no periodic disk-walking required:



1. **Live invalidation** — a `notify`-based file watcher tracks every

   configured root and re-optimizes / removes cached variants on source

   modify and delete events.

2. **Startup reconcile** — at VMOD load time, `load_images` walks the source

   roots, compares each cached variant's `mtime` to the source's `mtime`,

   and removes any cache file older than its source. Catches changes that

   happened during downtime or that the watcher missed.

3. **Lazy HTTP-level revalidation** — responses carry

   `Cache-Control: public, max-age=N, stale-while-revalidate=M`. After

   `max-age` expires, Varnish keeps serving stale to clients while firing

   an asynchronous background fetch through this VMOD; the backend computes

   an `ETag` from `(inode, size, mtime, is_optimized)` and returns 304 if

   the disk is unchanged or 200 with the new bytes if the watcher updated

   the file.



## Running the project



Start the dev container:



```shell

docker compose up -d

```



After every Rust or Varnish-config change, rebuild and reload:



```shell

docker exec vmod-impress /build.sh

```



The script compiles the cdylib, runs the test suite, copies the resulting

`.so` into Varnish's vmods directory, restarts varnishd, and tails the log.