{"id":13643830,"url":"https://github.com/danielgtaylor/jpeg-archive","last_synced_at":"2025-05-14T11:11:18.795Z","repository":{"id":12979344,"uuid":"15658042","full_name":"danielgtaylor/jpeg-archive","owner":"danielgtaylor","description":"Utilities for archiving JPEGs for long term storage.","archived":false,"fork":false,"pushed_at":"2025-03-03T18:05:11.000Z","size":2031,"stargazers_count":1176,"open_issues_count":45,"forks_count":124,"subscribers_count":46,"default_branch":"master","last_synced_at":"2025-04-01T04:51:15.837Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/danielgtaylor.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-01-05T20:16:37.000Z","updated_at":"2025-03-24T11:43:12.000Z","dependencies_parsed_at":"2024-08-02T01:17:37.492Z","dependency_job_id":"1c11dd9e-de52-4e3a-a04c-ff6391ed4434","html_url":"https://github.com/danielgtaylor/jpeg-archive","commit_stats":{"total_commits":155,"total_committers":20,"mean_commits":7.75,"dds":"0.44516129032258067","last_synced_commit":"5b56afa11872fc4b9f68c5a3dcf2ec5afe869504"},"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danielgtaylor%2Fjpeg-archive","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danielgtaylor%2Fjpeg-archive/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danielgtaylor%2Fjpeg-archive/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danielgtaylor%2Fjpeg-archive/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/danielgtaylor","download_url":"https://codeload.github.com/danielgtaylor/jpeg-archive/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247790360,"owners_count":20996510,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T01:01:53.321Z","updated_at":"2025-04-08T06:29:40.180Z","avatar_url":"https://github.com/danielgtaylor.png","language":"C","funding_links":[],"categories":["C"],"sub_categories":[],"readme":"JPEG Archive [![Build Status](http://img.shields.io/travis/danielgtaylor/jpeg-archive.svg?style=flat)](https://travis-ci.org/danielgtaylor/jpeg-archive) [![Build status](https://ci.appveyor.com/api/projects/status/1p7hrrq380xuqlyh?svg=true)](https://ci.appveyor.com/project/danielgtaylor/jpeg-archive) [![Version](http://img.shields.io/badge/version-2.2.0-blue.svg?style=flat)](https://github.com/danielgtaylor/jpeg-archive/releases) [![License](http://img.shields.io/badge/license-MIT-red.svg?style=flat)](http://dgt.mit-license.org/)\n============\nUtilities for archiving photos for saving to long term storage or serving over the web. The goals are:\n\n * Use a common, well supported format (JPEG)\n * Minimize storage space and cost\n * Identify duplicates / similar photos\n\nApproach:\n\n * Command line utilities and scripts\n * Simple options and useful help\n * Good quality output via sane defaults\n\nContributions to this project are very welcome.\n\nDownload\n--------\nYou can download the latest source and binary releases from the [JPEG Archive releases page](https://github.com/danielgtaylor/jpeg-archive/releases). Windows binaries for the latest commit are available from the [Windows CI build server](https://ci.appveyor.com/project/danielgtaylor/jpeg-archive/build/artifacts).\n\nIf you are looking for an easy way to run these utilities in parallel over many files to utilize all CPU cores, please also download [Ladon](https://github.com/danielgtaylor/ladon) or [GNU Parallel](https://www.gnu.org/software/parallel/). You can then use the `jpeg-archive` command below or use `ladon` directly. Example:\n\n```bash\n# Re-compress JPEGs and replace the originals\nladon \"Photos/**/*.jpg\" -- jpeg-recompress FULLPATH FULLPATH\n\n# Re-compress JPEGs into the new directory 'Comp'\nladon -m Comp/RELDIR \"Photos/**/*.jpg\" -- jpeg-recompress FULLPATH Comp/RELPATH\n```\n\nUtilities\n---------\nThe following utilities are part of this project. All of them accept a `--help` parameter to see the available options.\n\n### jpeg-archive\nCompress RAW and JPEG files in a folder utilizing all CPU cores. This is a simple shell script that uses the utilities below. It requires:\n\n* a POSIX-compatible shell such as Bash\n* [Ladon](https://github.com/danielgtaylor/ladon) or [GNU Parallel](https://www.gnu.org/software/parallel/)\n* [dcraw](http://www.cybercom.net/~dcoffin/dcraw/)\n* [exiftool](http://www.sno.phy.queensu.ca/~phil/exiftool/)\n* jpeg-recompress (part of this project)\n\n```bash\n# Compress a folder of images\ncd path/to/photos\njpeg-archive\n\n# Custom quality and metric\njpeg-archive --quality medium --method smallfry\n```\n\n### jpeg-recompress\nCompress JPEGs by re-encoding to the smallest JPEG quality while keeping _perceived_ visual quality the same and by making sure huffman tables are optimized. This is a __lossy__ operation, but the images are visually identical and it usually saves 30-70% of the size for JPEGs coming from a digital camera, particularly DSLRs. By default all EXIF/IPTC/XMP and color profile metadata is copied over, but this can be disabled to save more space if desired.\n\nThere is no need for the input file to be a JPEG. In fact, you can use `jpeg-recompress` as a replacement for `cjpeg` by using PPM input and the `--ppm` option.\n\nThe better the quality of the input image is, the better the output will be.\n\nSome basic photo-related editing options are available, such as removing fisheye lens distortion.\n\n#### Demo\nBelow are two 100% crops of [Nikon's D3x Sample Image 2](http://static.nikonusa.com/D3X_gallery/index.html). The left shows the original image from the camera, while the others show the output of `jpeg-recompress` with the `medium` quality setting and various comparison methods. By default SSIM is used, which lowers the file size by **88%**. The recompression algorithm chooses a JPEG quality of 80. By comparison the `veryhigh` quality setting chooses a JPEG quality of 93 and saves 70% of the file size.\n\n![JPEG recompression comparison](https://cloud.githubusercontent.com/assets/106826/3633843/5fde26b6-0eff-11e4-8c98-f18dbbf7b510.png)\n\nWhy are they different sizes? The default quality settings are set to average out to similar visual quality over large data sets. They may differ on individual photos (like above) because each metric considers different parts of the image to be more or less important for compression.\n\n#### Image Comparison Metrics\nThe following metrics are available when using `jpeg-recompress`. SSIM is the default.\n\nName     | Option        | Description\n-------- | ------------- | -----------\nMPE      | `-m mpe`      | Mean pixel error (as used by [imgmin](https://github.com/rflynn/imgmin))\nSSIM     | `-m ssim`     | [Structural similarity](http://en.wikipedia.org/wiki/Structural_similarity) **DEFAULT**\nMS-SSIM* | `-m ms-ssim`  | Multi-scale structural similarity (slow!) ([2008 paper](https://doi.org/10.1117/12.768060))\nSmallFry | `-m smallfry` | Linear-weighted BBCQ-like ([original project](https://github.com/dwbuiten/smallfry), [2011 BBCQ paper](http://spie.org/Publications/Proceedings/Paper/10.1117/12.872231))\n\n**Note**: The SmallFry algorithm may be [patented](http://www.jpegmini.com/main/technology) so use with caution.\n\n#### Subsampling\nThe JPEG format allows for subsampling of the color channels to save space. For each 2x2 block of pixels per color channel (four pixels total) it can store four pixels (all of them), two pixels or a single pixel. By default, the JPEG encoder subsamples the non-luma channels to two pixels (often referred to as 4:2:0 subsampling). Most digital cameras do the same because of limitations in the human eye. This may lead to unintended behavior for specific use cases (see [#12](https://github.com/danielgtaylor/jpeg-archive/issues/12) for an example), so you can use `--subsample disable` to disable this subsampling.\n\n#### Example Commands\n\n```bash\n# Default settings\njpeg-recompress image.jpg compressed.jpg\n\n# High quality example settings\njpeg-recompress --quality high --min 60 image.jpg compressed.jpg\n\n# Slow high quality settings (3-4x slower than above, slightly more accurate)\njpeg-recompress --accurate --quality high --min 60 image.jpg compressed.jpg\n\n# Use SmallFry instead of SSIM\njpeg-recompress --method smallfry image.jpg compressed.jpg\n\n# Use 4:4:4 sampling (disables subsampling).\njpeg-recompress --subsample disable image.jpg compressed.jpg\n\n# Remove fisheye distortion (Tokina 10-17mm on APS-C @ 10mm)\njpeg-recompress --defish 2.6 --zoom 1.2 image.jpg defished.jpg\n\n# Read from stdin and write to stdout with '-' as the filename\njpeg-recompress - - \u003cimage.jpg \u003ecompressed.jpg\n\n# Convert RAW to JPEG via PPM from stdin\ndcraw -w -q 3 -c IMG_1234.CR2 | jpeg-recompress --ppm - compressed.jpg\n\n# Disable progressive mode (not recommended)\njpeg-recompress --no-progressive image.jpg compressed.jpg\n\n# Disable all output except for errors\njpeg-recompress --quiet image.jpg compressed.jpg\n```\n\n### jpeg-compare\nCompare two JPEG photos to judge how similar they are. The `fast` comparison method returns an integer from 0 to 99, where 0 is identical. PSNR, SSIM, and MS-SSIM return floats but require images to be the same dimensions.\n\n```bash\n# Do a fast compare of two images\njpeg-compare image1.jpg image2.jpg\n\n# Calculate PSNR\njpeg-compare --method psnr image1.jpg image2.jpg\n\n# Calculate SSIM\njpeg-compare --method ssim image1.jpg image2.jpg\n```\n\n### jpeg-hash\nCreate a hash of an image that can be used to compare it to other images quickly.\n\n```bash\njpeg-hash image.jpg\n```\n\nBuilding\n--------\n### Dependencies\n * [mozjpeg](https://github.com/mozilla/mozjpeg)\n\n#### Ubuntu\nUbuntu users can install via `apt-get`:\n\n```bash\nsudo apt-get install build-essential autoconf pkg-config nasm libtool\ngit clone https://github.com/mozilla/mozjpeg.git\ncd mozjpeg\nautoreconf -fiv\n./configure --with-jpeg8\nmake\nsudo make install\n```\n\n#### Mac OS X\nMac users can install it via [Homebrew](http://brew.sh/):\n\n```bash\nbrew install mozjpeg\n```\n\n#### FreeBSD\n\n```bash\npkg install mozjpeg\ngit clone https://github.com/danielgtaylor/jpeg-archive.git\ncd jpeg-archive/\ngmake\nsudo gmake install\n```\n\n#### Windows\nThe `Makefile` should work with MinGW/Cygwin/etc and standard GCC. Patches welcome.\n\nTo get everything you need to build, install these:\n\n* [CMake](https://cmake.org/download/)\n* [NASM](https://www.nasm.us/)\n* [MinGW](https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win32/Personal%20Builds/mingw-builds/installer/mingw-w64-install.exe/download) (installed to e.g. `C:\\mingw`)\n* [Github for Windows](https://windows.github.com/)\n\nRun Github for windows. In the settings, set **Git Bash** as the shell. Open Git Shell from the start menu.\n\n```bash\n# Update PATH to include MinGW/NASM bin folder, location on your system may vary\nexport PATH=/c/mingw/mingw32/bin:/c/Program\\ Files \\(x68\\)/nasm:$PATH\n\n# Build mozjpeg or download https://www.dropbox.com/s/98jppfgds2xjblu/libjpeg.a\ngit clone https://github.com/mozilla/mozjpeg.git\ncd mozjpeg\ncmake -G \"MSYS Makefiles\" -D CMAKE_C_COMPILER=gcc.exe -D CMAKE_MAKE_PROGRAM=mingw32-make.exe  -D WITH_JPEG8=1\nmingw32-make\ncd ..\n\n# Build jpeg-archive\ngit clone https://github.com/danielgtaylor/jpeg-archive\ncd jpeg-archive\nCC=gcc mingw32-make\n```\n\nJPEG-Archive should now be built.\n\n### Compiling (Linux and Mac OS X)\nThe `Makefile` should work as-is on Ubuntu and Mac OS X. Other platforms may need to set the location of `libjpeg.a` or make other tweaks.\n\n```bash\nmake\n```\n\n### Installation\nInstall the binaries into `/usr/local/bin`:\n\n```bash\nsudo make install\n```\n\nLinks / Alternatives\n--------------------\n* https://github.com/rflynn/imgmin\n* https://news.ycombinator.com/item?id=803839\n\nLicense\n-------\n* JPEG-Archive is copyright \u0026copy; 2015 Daniel G. Taylor\n* Image Quality Assessment (IQA) is copyright 2011, Tom Distler (http://tdistler.com)\n* SmallFry is copyright 2014, Derek Buitenhuis (https://github.com/dwbuiten)\n\nAll are released under an MIT license.\n\nhttp://dgt.mit-license.org/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdanielgtaylor%2Fjpeg-archive","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdanielgtaylor%2Fjpeg-archive","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdanielgtaylor%2Fjpeg-archive/lists"}