{"id":16916586,"url":"https://github.com/ternaus/imread_benchmark","last_synced_at":"2025-05-07T10:13:29.829Z","repository":{"id":62570825,"uuid":"207006805","full_name":"ternaus/imread_benchmark","owner":"ternaus","description":"I/O benchmark for different image processing python libraries.","archived":false,"fork":false,"pushed_at":"2024-12-24T14:08:01.000Z","size":416,"stargazers_count":31,"open_issues_count":1,"forks_count":2,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-12-27T05:06:31.075Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ternaus.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"ternaus","patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"lfx_crowdfunding":null,"custom":null}},"created_at":"2019-09-07T18:03:15.000Z","updated_at":"2024-12-13T14:10:41.000Z","dependencies_parsed_at":"2024-03-11T23:34:37.206Z","dependency_job_id":"d1f1354f-6b9d-427a-8755-95f8eb4bec09","html_url":"https://github.com/ternaus/imread_benchmark","commit_stats":{"total_commits":36,"total_committers":3,"mean_commits":12.0,"dds":"0.11111111111111116","last_synced_commit":"4084f700bacbb629f5354ff9e336a37edc20facb"},"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ternaus%2Fimread_benchmark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ternaus%2Fimread_benchmark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ternaus%2Fimread_benchmark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ternaus%2Fimread_benchmark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ternaus","download_url":"https://codeload.github.com/ternaus/imread_benchmark/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":232814810,"owners_count":18580529,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-13T19:28:39.520Z","updated_at":"2025-05-07T10:13:29.802Z","avatar_url":"https://github.com/ternaus.png","language":"Python","funding_links":["https://github.com/sponsors/ternaus"],"categories":[],"sub_categories":[],"readme":"# Image Loading Benchmark\n## Overview\n\nThis benchmark evaluates the efficiency of different libraries in loading JPG images\nand converting them into RGB numpy arrays, essential for neural network training\ndata preparation. The study compares traditional image processing libraries (Pillow, OpenCV),\nmachine learning frameworks (TensorFlow, PyTorch), and specialized decoders (jpeg4py, kornia-rs)\nacross different computing architectures.\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\u003cimg src=\"images/performance_darwin.png\" alt=\"Darwin Performance\" width=\"400\"/\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cimg src=\"images/performance_linux.png\" alt=\"Linux Performance\" width=\"400\"/\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\"\u003ePerformance on Apple Silicon (M4 Max)\u003c/td\u003e\n    \u003ctd align=\"center\"\u003ePerformance on Linux (AMD Threadripper)\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n## Important Note on Image Conversion\n\nIn the benchmark, it's crucial to standardize image formats for a fair comparison.\nDifferent libraries use different default formats: OpenCV (BGR), torchvision and\nTensorFlow (tensors). A conversion step to RGB numpy arrays is included for\nconsistency. Note that in typical use cases, torchvision and TensorFlow do not\nrequire this conversion.\n\n## Installation and Setup\n\nBefore running the benchmark, ensure your system is equipped with the necessary\ndependencies:\n\n### System Requirements\n\n```bash\n# On Ubuntu/Debian\nsudo apt-get install libturbojpeg\n```\n### Python Setup\n\nThe benchmark uses separate virtual environments for each library to avoid\ndependency conflicts. You'll need:\n\n```bash\n# Install uv for faster package installation\npip install uv\n```\n\n## Running the Benchmark\n\nTo reproduce the benchmarks, you'll need the ImageNet validation dataset:\n\nDownload the validation set (50,000 images, ~6.3GB):\n\n```bash\nwget https://image-net.org/data/ILSVRC/2012/ILSVRC2012_img_val.tar\n```\n\nCreate a directory and extract the images:\n\n```bash\nmkdir -p imagenet/val\ntar -xf ILSVRC2012_img_val.tar -C imagenet/val\n```\n\nThe benchmark script creates separate virtual environments for each library and\nruns tests independently:\n\n```bash\n# Make the script executable\nchmod +x run_benchmarks.sh\n\n# Show help and options\n./run_benchmarks.sh --help\n\n# Run benchmark with default settings (2000 images, 5 runs)\n./run_benchmarks.sh /path/to/images\n\n# Run with custom settings\n./run_benchmarks.sh /path/to/images 1000 3\n```\n\nThe script will:\n\n1. Create separate virtual environments for each library\n2. Install required dependencies using `uv`\n3. Run benchmarks independently\n4. Save results to OS-specific directories\n\n### Results Structure\n\nResults are saved in JSON format under:\n\n```text\noutput/\n├── linux/          # When run on Linux\n│   ├── opencv_results.json\n│   ├── pil_results.json\n│   └── ...\n└── darwin/         # When run on macOS\n    ├── opencv_results.json\n    ├── pil_results.json\n    └── ...\n```\n\n## Libraries Being Benchmarked\n\nEach library uses different underlying JPEG decoders and implementation approaches:\n\n### Direct libjpeg-turbo Users (Fastest)\n- jpeg4py (Linux only) - Direct libjpeg-turbo binding\n- kornia-rs - Modern Rust-based implementation\n- OpenCV (opencv-python-headless)\n- torchvision\n\n### Standard libjpeg Users\n- PIL (Pillow)\n- Pillow-SIMD (Linux only)\n- scikit-image\n- imageio\n\n### Machine Learning Framework Components\n- tensorflow\n- torchvision\n- kornia-rs\n\n\n## Performance Considerations\n\nSeveral factors influence real-world performance beyond raw decoding speed:\n\n### Memory Usage\n- Memory utilization varies significantly across libraries\n- Some implementations (like kornia-rs) have specific memory allocation optimizations\n- Consider available system resources when scaling to batch processing\n\n### System Integration\n- All benchmarks performed on NVMe SSDs to minimize I/O variance\n- Single-threaded performance reported\n- Multi-threading capabilities vary between libraries\n\n### Image Characteristics\n- Results based on typical ImageNet JPEG images (~500x400 pixels)\n- Performance scaling with image size varies between implementations\n- Compression ratio and JPEG encoding parameters can influence decoding speed\n\n## Recommendations\n\n### High-Performance Applications\n- Use kornia-rs or OpenCV for consistent cross-platform performance\n- On Linux, consider jpeg4py for maximum performance\n- Consider memory usage if processing many images simultaneously\n\n### Cross-Platform Development\n- kornia-rs provides the most consistent performance\n- OpenCV and torchvision offer good balance of features and speed\n- Test with representative image sizes and batching patterns\n\n### Feature-Rich Applications\n- When needing extensive image processing features, OpenCV remains a strong choice\n- Consider dependency size and installation complexity\n- Evaluate the full image processing pipeline, not just JPEG decoding\n\n\n## Citation\n\nIf you found this work useful, please cite:\n```bibtex\n@misc{iglovikov2025speed,\n      title={Need for Speed: A Comprehensive Benchmark of JPEG Decoders in Python},\n      author={Vladimir Iglovikov},\n      year={2025},\n      eprint={2501.13131},\n      archivePrefix={arXiv},\n      primaryClass={eess.IV},\n      doi={10.48550/arXiv.2501.13131}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fternaus%2Fimread_benchmark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fternaus%2Fimread_benchmark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fternaus%2Fimread_benchmark/lists"}