Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/bertrandmartel/media-optimizer
Automatically optimize your images & videos hosted on AWS S3 when an S3 event is detected
https://github.com/bertrandmartel/media-optimizer
aws aws-cloudformation aws-ec2 aws-lambda aws-s3 aws-sdk go image-compression optimizer
Last synced: 4 days ago
JSON representation
Automatically optimize your images & videos hosted on AWS S3 when an S3 event is detected
- Host: GitHub
- URL: https://github.com/bertrandmartel/media-optimizer
- Owner: bertrandmartel
- License: mit
- Created: 2020-01-28T16:23:25.000Z (almost 5 years ago)
- Default Branch: master
- Last Pushed: 2020-01-29T17:51:34.000Z (almost 5 years ago)
- Last Synced: 2024-08-10T05:09:22.931Z (3 months ago)
- Topics: aws, aws-cloudformation, aws-ec2, aws-lambda, aws-s3, aws-sdk, go, image-compression, optimizer
- Language: Go
- Homepage:
- Size: 70.3 KB
- Stars: 5
- Watchers: 3
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
README
# Media Optimizer for AWS S3
[](https://github.com/bertrandmartel/media-optimizer/actions?workflow=build%20and%20deploy)
[](https://goreportcard.com/report/github.com/bertrandmartel/media-optimizer)
[](https://hub.docker.com/r/bertrandmartel/media-optimizer)
[](LICENSE.md)
Automatically optimize your images & videos hosted on AWS S3 when an S3 event is detected
This project is inspired by [image-optimizer](https://github.com/spatie/image-optimizer) & [Lambda ECS Worker Pattern](https://github.com/aws-samples/lambda-ecs-worker-pattern)
## Architecture
A tag is also applied to the optimized file so this file will be ignored in subsequent event
## How it works ?
* S3
* S3 is configured to trigger a specific lambda on objectCreate event
* Lambda
* catch the S3 Event
* push the event to a SQS queue
* EC2
* constantly listens to messages from SQS queue
* when a message arrives :
* get the S3 Event from the message
* get information about S3 object (content-type)
* download the S3 object
* executes optimization chain on object (one or more command)
* upload the optimized object to S3
## Run on AWS
In AWS dashboard, go to cloudformation & create a new stack from the template file [cloudformation.yml](https://github.com/bertrandmartel/media-optimizer/blob/master/cloudformation.yml)
In the parameters, change the bucket name & the subnet value (a private subnet is fine)
When the stack is up, go to the created S3 bucket & upload some images. After some moment, you will notice they will be automatically optimized
You can check that the new optimized image has the tag `optimizer_ignore` :
In CloudWatch you can get the logs of your Lambda & your docker container running the optimization program
## Optimization configuration
The optimization configuration is located in optimizer.json file. In the actual version, the JSON file need to be at the same level as the media-optimizer executable
### Current configuration
Currently using all the optimizers from [image-optimizer](https://github.com/spatie/image-optimizer) project + ffmpeg for mp4 video :
* image/png
* `pngquant --output [output_file] -f [intput_file]`
* `optipng -out [output_file] -i0 -o2 -clobber [intput_file]`
* image/jpeg
* `jpegoptim -d [output_directory] -m85 --strip-all --all-progressive [intput_file]`
* image/svg+xml
* `svgo -o [output_file] --disable={cleanupIDs,removeViewBox} [intput_file]`
* image/gif
* `gifsicle -o [output_file] -b -O3 [intput_file]`
* image/webp
* `cwebp -o [output_file] -m 6 -pass 10 -mt -q 80 [intput_file]`
* video/mp4
* `ffmpeg -i [input_file] -vcodec libx264 -crf 24 -y [output_file]`
You can add more format & more commands to the config file `optimizer.json`
### Configuration Format
```json
{
"optimizers": [{
"contentType": "image/png",
"exec": [{
"binary": "pngquant",
"outputFile": "--output",
"params": ["-f"]
},{
"binary":"optipng",
"outputFile": "-out",
"params": ["-i0","-o2", "-clobber"]
}]
.....
}
```
* `binary` is the command to execute
* `params` are the command parameters
* `outputFile` is the parameter for the output file
* `outputDirectory` is the parameter for the output directory if `outputFile` is not specified
* `inputFile` is the parameter for the input file if neither `outputFile` nor `outputDirectory` are specified. In case `inputFile` is specified the outputFile value will be appended to the parameters (at the end)
## Environment variables
Those variables configure S3 Object configuration & metadata :
| name | default value |
|-----------|-------------|
| OBJECT_ACL | private |
| OBJECT_CACHE_CONTROL | max-age=15552000 |
| OBJECT_STORAGE_CLASS | STANDARD |
| OBJECT_SERVER_SIDE_ENCRYPTION | none |
Note that if `AWS::Region` is not specified in your `~/.aws`, the variable `AWS_REGION` should be set (see [cloudformation.yml](https://github.com/bertrandmartel/media-optimizer/blob/master/cloudformation.yml))
## Run locally
```bash
make install
make run
```
## Run with docker (locally)
```bash
docker build . -t media-optimizer
docker run -v $HOME/.aws:/root/.aws -it media-optimizer
```
## Run with docker (DockerHub)
```bash
docker pull bertrandmartel/media-optimizer:latest
docker run -v $HOME/.aws:/root/.aws -it bertrandmartel/media-optimizer:latest
```
## Dependencies
* https://github.com/aws/aws-sdk-go
* https://github.com/aws/aws-lambda-go