https://github.com/willdollman/pixel-slicer
A fast and multithreaded media encoder built for the web
https://github.com/willdollman/pixel-slicer
av1 encoder ffmpeg image jpg video webm webp
Last synced: 7 months ago
JSON representation
A fast and multithreaded media encoder built for the web
- Host: GitHub
- URL: https://github.com/willdollman/pixel-slicer
- Owner: willdollman
- License: mit
- Created: 2020-06-17T23:22:55.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2022-05-30T21:48:59.000Z (over 3 years ago)
- Last Synced: 2025-01-21T01:41:25.826Z (9 months ago)
- Topics: av1, encoder, ffmpeg, image, jpg, video, webm, webp
- Language: Go
- Homepage:
- Size: 176 KB
- Stars: 1
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# pixel-slicer
A fast and multithreaded media encoder built for the web.
## Overview
pixel-slicer is an image and video encoder written in [Go](https://go.dev/).
It's designed to optimise your web media for every possible browser and screen size, upload your media wherever it needs to go, and to do it all as quickly as possible.
pixel-slicer uses the extremely efficient [libvips](https://github.com/libvips/libvips) for image encoding and [FFmpeg](https://github.com/FFmpeg/FFmpeg) for video encoding, though it's extendable and can work with any media encoder.
pixel-slicer's goal is to help you optimise media-heavy websites by encoding resources at multiple resolutions and formats, so the most optimised file can be served to each client tuned for their screen size and supported formats.
It was written to allow high performance sites packed with media to be served on a shoestring budget with pro performance - I wrote pixel-slicer as a backend processor for my own static [photolog site](https://photos.dollman.org/).
## Getting Started
### Docker
The easiest way to run pixel-slicer is via Docker, which includes all required media encoding libraries:
```
docker pull ghcr.io/willdollman/pixel-slicer:master
# Resize all media files in ~/input-images, and output to ~/output
docker run -v ~/input-images:/input -v ~/output:/output ghcr.io/willdollman/pixel-slicer:master
```
See [#Configuration](#configuration) for how to select image and video sizes and their output formats.
### Building from source
pixel-slicer relies on various media encoding libraries, which must be installed before running:
```
# macOS (using Homebrew)
brew install vips ffmpeg
# Linux
apt-get install libvips-dev ffmpeg
```
Then with Go installed you can download and build from source:
```
go install github.com/willdollman/pixel-slicer/cmd/...@latest
# Or clone and run
git clone git@github.com:willdollman/pixel-slicer.git && cd pixel-slicer
go run cmd/pixel-slicer/main.go
```
If installing from source fails on macOS, try setting `CGO_CFLAGS_ALLOW=-Xpreprocessor` to work around an issue in the macOS libvips library.
## Configuration
pixel-slicer configuration is stored as YAML, and contains conversion rules for each media type:
Sample Config
Save the below as `config.yaml` and pass `--config config.yaml`:
```
inputDir: sample-data/
outputDir: output-media/
moveProcessed: false # Move files to another directory once processed
processedDir: processed/
watch: false # Watch input directory for new files
# Upload all generated media to S3-compatible storage (when Enabled is set to true)
S3:
Enabled: false
Endpoint: https://s3.us-west-000.backblazeb2.com
Region: us-east-1
Bucket: media
# Convert images to JPG and WebP at a variety of sizes
ImageConfigurations:
- MaxWidth: 500
Quality: 80
FileType: jpg
- MaxWidth: 500
Quality: 80
FileType: webp
- MaxWidth: 1000
Quality: 75
FileType: jpg
- MaxWidth: 1000
Quality: 80
FileType: webp
- MaxWidth: 2000
Quality: 70
FileType: jpg
- MaxWidth: 2000
Quality: 75
FileType: webp
# Convert videos to H.264 and AV1 at two sizes, and include a JPG thumbnail
VideoConfigurations:
- MaxWidth: 500
Quality: 2
FileType: jpg
- MaxWidth: 360
Quality: 23
Preset: slow
FileType: mp4
- MaxWidth: 360
Quality: 40
Codec: av1
- MaxWidth: 720
Quality: 23
Preset: slow
FileType: mp4
- MaxWidth: 720
Quality: 40
Codec: av1
```
YAML config files can be passed with `--config `. All config file options can be overridden by command line flags.
Use `--help` to see a full list of configuration flags, but some useful flags are:
* `--dir `: process media in the given directory
* `--outputdir `: output resized and encoded media to the given directory
* `--watch`: watch the input directory for new files
* `--move-processed`: move processed files to a separate directory. Useful when used with `--watch`
* See `--help` for a full list
## Supported Output Formats
pixel-slicer is designed for the web, and out-of-the-box it supports a carefully considered selection of widely-supported and more modern high-efficiency formats.
| Format | Media Type | Browser Support | Description |
|--------|------------|----------|-------------|
| JPG | Image | [Universal](https://caniuse.com/jpg) | Supported everywhere, outdated efficiency |
| WebP | Image | [Modern, good](https://caniuse.com/webp) | 25-34% smaller than JPG|
| H.264 | Video | [Universal](https://caniuse.com/mpeg4) | Supported everywhere, outdated efficiency |
| H.265 | Video | [Poor; Apple platforms only](https://caniuse.com/?search=h265) | ~30-50% efficiency gain over H.264, not open source |
| VP9 | Video | [Medium; modern browsers excluding Apple](https://caniuse.com/?search=vp9) | ~30-50% efficiency gain over H.264, open source |
| AV1 | Video | [Medium; modern browsers excluding Apple](https://caniuse.com/?search=av1) | Successor to VP9; slow encoding speeds|
By making multiple forms of an image or video available [using source sets](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/source), a browser can select the most appropriate filetype to use.