https://github.com/cdown/jpgfromraw

Fast JPEG extractor from RAW files
https://github.com/cdown/jpgfromraw

Last synced: about 1 year ago
JSON representation

Fast JPEG extractor from RAW files

• Host: GitHub
• URL: https://github.com/cdown/jpgfromraw
• Owner: cdown
• License: mit
• Created: 2024-05-08T05:28:42.000Z (about 2 years ago)
• Default Branch: master
• Last Pushed: 2025-04-08T02:04:38.000Z (about 1 year ago)
• Last Synced: 2025-05-01T02:17:50.292Z (about 1 year ago)
• Language: Rust
• Homepage:
• Size: 110 KB
• Stars: 6
• Watchers: 1
• Forks: 2
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# jpgfromraw

jpgfromraw provides a much faster way to extract embedded JPEGs from RAW files

than exiftool's `-JpgFromRaw`. In a directory with 4000 files, jpgfromraw

extracts JPEGs about 15 times faster than exiftool:

    % rm -rf ~/jfr && mkdir -p ~/jfr/{jpgfromraw,exiftool}

    % sudo sh -c 'sync; echo 3 > /proc/sys/vm/drop_caches'

    % \time -v jpgfromraw /mnt/sdcard/DCIM/101MSDCF ~/jfr/jpgfromraw >/dev/null

        User time (seconds): 2.18

        System time (seconds): 20.99

        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:29.72

        File system inputs: 22956551

        File system outputs: 22855504

    % sudo sh -c 'sync; echo 3 > /proc/sys/vm/drop_caches'

    % \time -v exiftool -b -JpgFromRaw -ext arw -r /mnt/sdcard/DCIM/101MSDCF -w ~/jfr/exiftool/%f.jpg >/dev/null

        User time (seconds): 113.21

        System time (seconds): 58.15

        Elapsed (wall clock) time (h:mm:ss or m:ss): 7:52.87

        File system inputs: 316175244

        File system outputs: 22864400

The total size of the output JPEGs is 11.5GiB, so in terms of throughput,

jpgfromraw does ~386MiB/s, and exiftool does ~24MiB/s.

The key reason jpgfromraw is so much faster is because it very carefully avoids

overreading into the entire RAW file. exiftool does not do that and suffers

quite greatly in (useful) throughput as a result. This is achieved through

judicious use of `madvise` (and similar strategies on other platforms).

Other than that, jpgfromraw also processes multiple files concurrently, which

can help a lot on faster devices like CFexpress cards.