Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

Projects in Awesome Lists tagged with compression

A curated list of projects in awesome lists tagged with compression .

https://github.com/deepspeedai/deepspeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

billion-parameters compression data-parallelism deep-learning gpu inference machine-learning mixture-of-experts model-parallelism pipeline-parallelism pytorch trillion-parameters zero

Last synced: 15 Jan 2026

https://github.com/microsoft/DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

billion-parameters compression data-parallelism deep-learning gpu inference machine-learning mixture-of-experts model-parallelism pipeline-parallelism pytorch trillion-parameters zero

Last synced: 02 Apr 2025

https://github.com/deepspeedai/DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

billion-parameters compression data-parallelism deep-learning gpu inference machine-learning mixture-of-experts model-parallelism pipeline-parallelism pytorch trillion-parameters zero

Last synced: 19 Oct 2025

https://github.com/gchq/cyberchef

The Cyber Swiss Army Knife - a web app for encryption, encoding, compression and data analysis

compression data-analysis data-manipulation encoding encryption hashing parsing

Last synced: 12 May 2025

https://gchq.github.io/CyberChef/

The Cyber Swiss Army Knife - a web app for encryption, encoding, compression and data analysis

compression data-analysis data-manipulation encoding encryption hashing parsing

Last synced: 18 Mar 2025

https://github.com/gchq/CyberChef

The Cyber Swiss Army Knife - a web app for encryption, encoding, compression and data analysis

compression data-analysis data-manipulation encoding encryption hashing parsing

Last synced: 13 Mar 2025

https://github.com/leandromoreira/digital_video_introduction

A hands-on introduction to video technology: image, video, codec (av1, vp9, h265) and more (ffmpeg encoding). Translations: ๐Ÿ‡บ๐Ÿ‡ธ ๐Ÿ‡จ๐Ÿ‡ณ ๐Ÿ‡ฏ๐Ÿ‡ต ๐Ÿ‡ฎ๐Ÿ‡น ๐Ÿ‡ฐ๐Ÿ‡ท ๐Ÿ‡ท๐Ÿ‡บ ๐Ÿ‡ง๐Ÿ‡ท ๐Ÿ‡ช๐Ÿ‡ธ

adaptive-streaming arithmetic-coding audio av1 bitstream codec compression dash ffmpeg frame-types guide h264 h265 handson hls learning tutorial video video-codec vp9

Last synced: 13 May 2025

https://github.com/PaddlePaddle/PaddleNLP

๐Ÿ‘‘ Easy-to-use and powerful NLP and LLM library with ๐Ÿค— Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including ๐Ÿ—‚Text Classification, ๐Ÿ” Neural Search, โ“ Question Answering, โ„น๏ธ Information Extraction, ๐Ÿ“„ Document Intelligence, ๐Ÿ’Œ Sentiment Analysis etc.

bert compression distributed-training document-intelligence embedding ernie information-extraction llama llm neural-search nlp paddlenlp pretrained-models question-answering search-engine semantic-analysis sentiment-analysis transformers uie

Last synced: 18 Mar 2025

https://github.com/borgbackup/borg

Deduplicating archiver with compression and authenticated encryption.

backup borgbackup compression deduplication encryption python ssh

Last synced: 16 Mar 2026

https://github.com/lz4/lz4

Extremely Fast Compression algorithm

c compression lz4

Last synced: 13 May 2025

https://github.com/google/draco

Draco is a library for compressing and decompressing 3D geometric meshes and point clouds. It is intended to improve the storage and transmission of 3D graphics.

3d-graphics compression mesh point-cloud

Last synced: 12 May 2025

https://google.github.io/draco/

Draco is a library for compressing and decompressing 3D geometric meshes and point clouds. It is intended to improve the storage and transmission of 3D graphics.

3d-graphics compression mesh point-cloud

Last synced: 11 Jul 2025

https://github.com/spf13/afero

The Universal Filesystem Abstraction for Go

compression filesystem fs go golang network-file-system network-file-transfer vfs virtual

Last synced: 09 Sep 2025

https://github.com/zeux/meshoptimizer

Mesh optimization library that makes meshes smaller and faster to render

compression gltf gpu mesh-processing optimization simplification

Last synced: 12 May 2025

https://github.com/iridiumio/compactgui

Transparently compress active games and programs using Windows 10/11 APIs

compressing-folders compression dotnet6 game standalone vbnet windows windows-10 windows11 wpf

Last synced: 14 May 2025

https://github.com/IridiumIO/CompactGUI

Transparently compress active games and programs using Windows 10/11 APIs

compressing-folders compression dotnet6 game standalone vbnet windows windows-10 windows11 wpf

Last synced: 28 Mar 2025

https://github.com/klauspost/compress

Optimized Go Compression Packages

compression decompression deflate go golang gzip snappy zip zstandard zstd

Last synced: 09 Sep 2025

https://github.com/dropbox/lepton

Lepton is a tool and file format for losslessly compressing JPEGs by an average of 22%.

compression compression-algorithm decompression jpeg lossless-compression-algorithm

Last synced: 28 Sep 2025

https://github.com/randygaul/cute_headers

Collection of cross-platform one-file C/C++ libraries with no dependencies, primarily used for games

audio c collision collision-detection compression cross-platform game header library math opengl png

Last synced: 13 May 2025

https://github.com/RandyGaul/cute_headers

Collection of cross-platform one-file C/C++ libraries with no dependencies, primarily used for games

audio c collision collision-detection compression cross-platform game header library math opengl png

Last synced: 20 Mar 2025

https://github.com/peazip/peazip

Free Zip / Unzip software and Rar file extractor. Cross-platform file and archive manager. Features volume spanning, compression, authenticated encryption. Supports 7Z, 7-Zip sfx, ACE, ARJ, Brotli, BZ2, CAB, CHM, CPIO, DEB, GZ, ISO, JAR, LHA/LZH, NSIS, OOo, PAQ/LPAQ, PEA, QUAD, RAR, RPM, split, TAR, Z, ZIP, ZIPX, Zstandard.

archiver brotli compression encryption extractor paq peazip rar zip zstandard

Last synced: 06 Apr 2026

https://github.com/peazip/PeaZip

Free Zip / Unzip software and Rar file extractor. Cross-platform file and archive manager. Features volume spanning, compression, authenticated encryption. Supports 7Z, 7-Zip sfx, ACE, ARJ, Brotli, BZ2, CAB, CHM, CPIO, DEB, GZ, ISO, JAR, LHA/LZH, NSIS, OOo, PAQ/LPAQ, PEA, QUAD, RAR, RPM, split, TAR, Z, ZIP, ZIPX, Zstandard.

archiver brotli compression encryption extractor paq peazip rar zip zstandard

Last synced: 30 Mar 2025

https://github.com/mholt/archiver

Easily create & extract archives, and compress & decompress files of various formats

7zip archives brotli bzip2 compression decompression extract go golang gzip lz4 rar snappy streaming streams tar xz zip zstandard

Last synced: 10 Apr 2025

https://github.com/Lymphatus/caesium-image-compressor

Caesium is an image compression software that helps you store, send and share digital pictures, supporting JPG, PNG, WebP and TIFF formats. You can quickly reduce the file size (and resolution, if you want) by preserving the overall quality of the image.

caesium compression cross-platform image-compression libcaesium linux macos windows

Last synced: 14 Mar 2025

https://github.com/kurolabs/stegcloak

Hide secrets with invisible characters in plain text securely using passwords ๐Ÿง™๐Ÿปโ€โ™‚๏ธโญ

cipher compression cryptography data-exfiltration functional-programming hacking hacking-tool hacking-tools infosec javascript privacy ramdajs security security-tools steganography stego

Last synced: 14 May 2025

https://github.com/KuroLabs/stegcloak

Hide secrets with invisible characters in plain text securely using passwords ๐Ÿง™๐Ÿปโ€โ™‚๏ธโญ

cipher compression cryptography data-exfiltration functional-programming hacking hacking-tool hacking-tools infosec javascript privacy ramdajs security security-tools steganography stego

Last synced: 26 Mar 2025

https://github.com/apache/fory

A blazingly fast multi-language serialization framework powered by JIT and zero-copy.

compression cpp cross-language encoding fast golang hacktoberfest java javascript jit lightning marshalling multiple-language persistence python rpc rust serialization transfer zero-copy

Last synced: 06 Feb 2026

https://github.com/apache/fury

A blazingly fast multi-language serialization framework powered by JIT and zero-copy.

compression cpp cross-language encoding fast golang hacktoberfest java javascript jit lightning marshalling multiple-language persistence python rpc rust serialization transfer zero-copy

Last synced: 12 May 2025

https://github.com/Bilibili/boxing

Android multi-media selector based on MVP mode.

android cinderella-man compression kotlin-android media-picker mvp picture video

Last synced: 21 Apr 2025

https://github.com/bilibili/boxing

Android multi-media selector based on MVP mode.

android cinderella-man compression kotlin-android media-picker mvp picture video

Last synced: 15 May 2025

https://github.com/ouch-org/ouch

Painless compression and decompression in the terminal

cli compression decompression hacktoberfest rust

Last synced: 12 May 2025

https://github.com/marmelroy/Zip

Swift framework for zipping and unzipping files.

compression files swift unzip zip

Last synced: 06 Aug 2025

https://github.com/marmelroy/zip

Swift framework for zipping and unzipping files.

compression files swift unzip zip

Last synced: 09 Apr 2026

https://github.com/xiph/opus

Modern audio compression for the internet.

audio c codec compression

Last synced: 23 Apr 2025

https://github.com/adamhathcock/sharpcompress

SharpCompress is a fully managed C# library to deal with many compression types and formats.

7zip bzip2 c-sharp compression dotnet-core gzip lzip lzma sharpcompress tar unrar xz zip

Last synced: 16 Feb 2026

https://github.com/dblalock/bolt

10x faster matrix and vector operations

compression data-mining database machine-learning

Last synced: 15 May 2025

https://github.com/101arrowz/fflate

High performance (de)compression in an 8kB package

compression decompression high-performance pako tiny zlib

Last synced: 13 May 2025

https://github.com/megastep/makeself

A self-extracting archiving tool for Unix systems, in 100% shell script.

compression extract-archive linux makeself makeself-archives shell tar-archive unix

Last synced: 14 May 2025

https://github.com/onekey-sec/unblob

Extract files from any kind of container formats

archive compression extraction filesystem python

Last synced: 14 May 2025

https://github.com/mhx/dwarfs

A fast high compression read-only file system for Linux, Windows and macOS

archiving compression cpp deduplication dwarfs filesystem flac fuse fuse-filesystem linux lrzip lzma macfuse macos squashfs windows winfsp zpaq zstd

Last synced: 02 Apr 2026

https://github.com/quic/aimet

AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.

auto-ml compression deep-learning deep-neural-networks machine-learning network-compression network-quantization open-source opensource pruning quantization

Last synced: 02 Apr 2026

https://github.com/matthiasmullie/minify

CSS & JavaScript minifier, in PHP. Removes whitespace, strips comments, combines files (incl. @import statements and small assets in CSS files), and optimizes/shortens a few common programming patterns.

compression compressor css css-minifier javascript js js-minifier minifier minify php

Last synced: 15 May 2025

https://github.com/achno/gowall

A tool to convert a Wallpaper's color scheme / palette, OCR with VLM's Traditional & Hybrid, Image Compression ,color palette extraction, image upsacling with Adversarial Networks and more image processing features.

cli color-extractor color-palette color-scheme compression go image image-converter image-processing ocr pixel-art theme-switcher upscale

Last synced: 09 Apr 2026

https://github.com/leon406/toolsfx

่ทจๅนณๅฐๅฏ†็ ๅญฆๅทฅๅ…ท็ฎฑใ€‚ๅŒ…ๅซ็ผ–่งฃ็ ๏ผŒ็ผ–็ ่ฝฌๆข๏ผŒๅŠ ่งฃๅฏ†๏ผŒ ๅ“ˆๅธŒ๏ผŒMAC๏ผŒ็ญพๅ๏ผŒๅคงๆ•ฐ่ฟ็ฎ—๏ผŒๅŽ‹็ผฉ๏ผŒไบŒ็ปด็ ๅŠŸ่ƒฝ๏ผŒCTF็ญ‰ๅŠŸ่ƒฝใ€‚

aes base64 biginterger compression crypto cryptography ctf ctf-tools hex md5 rsa signature

Last synced: 14 May 2025

https://github.com/citusdata/cstore_fdw

Columnar storage extension for Postgres built as a foreign data wrapper. Check out https://github.com/citusdata/citus for a modernized columnar storage implementation built as a table access method.

columnar-storage columnar-store compression postgresql

Last synced: 01 Aug 2025

https://github.com/zlib-ng/zlib-ng

zlib replacement with optimizations for "next generation" systems.

c compression deflate inflate library optimized zlib zlib-license zlib-ng zlib-replacement

Last synced: 13 May 2025

https://github.com/Leon406/ToolsFx

ๅŸบไบŽkotlin+tornadoFx็š„่ทจๅนณๅฐๅฏ†็ ๅญฆๅทฅๅ…ท็ฎฑ.ๅŒ…ๅซ็ผ–่งฃ็ ,็ผ–็ ่ฝฌๆข,ๅŠ ่งฃๅฏ†, ๅ“ˆๅธŒ,MAC,็ญพๅ,ๅคงๆ•ฐ่ฟ็ฎ—,ๅŽ‹็ผฉ,ไบŒ็ปด็ ๅŠŸ่ƒฝ,ctf็ญ‰ๅฎž็”จๅŠŸ่ƒฝ,ๆ”ฏๆŒๆ’ไปถ

aes base64 biginterger compression crypto cryptography ctf ctf-tools hex md5 rsa signature toolset

Last synced: 11 Jul 2025

https://github.com/paddlepaddle/paddleslim

PaddleSlim is an open-source library for deep model compression and architecture search.

bert compression detection distillation ernie nas pruning quantization segmentation sparsity tensorrt transformer yolov5 yolov6 yolov7

Last synced: 14 May 2025

https://github.com/PaddlePaddle/PaddleSlim

PaddleSlim is an open-source library for deep model compression and architecture search.

bert compression detection distillation ernie nas pruning quantization segmentation sparsity tensorrt transformer yolov5 yolov6 yolov7

Last synced: 20 Mar 2025

https://github.com/donmccurdy/gltf-transform

glTF 2.0 SDK for JavaScript and TypeScript, on Web and Node.js.

3d compression glb gltf mesh-processing optimization scene-graph

Last synced: 13 Nov 2025

https://github.com/donmccurdy/glTF-Transform

glTF 2.0 SDK for JavaScript and TypeScript, on Web and Node.js.

3d compression glb gltf mesh-processing optimization scene-graph

Last synced: 24 Mar 2025

https://github.com/tensorflow/model-optimization

A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.

compression deep-learning keras machine-learning ml model-compression optimization pruning quantization quantized-networks quantized-neural-networks quantized-training sparsity tensorflow

Last synced: 12 May 2025

https://github.com/akai-katto/dandere2x

Dandere2x - Fast Waifu2x Video Upscaling.

compression fast faster video waifu2x waifu2x-video

Last synced: 15 May 2025

https://github.com/kuba--/zip

A portable, simple zip library written in C

c compression hacking miniz portable zip

Last synced: 14 Mar 2026

https://github.com/calibreapp/image-actions

A Github Action that automatically compresses JPEGs, PNGs and WebPs in Pull Requests.

compression github-action github-actions image-compression image-processing performance performance-monitoring performance-testing

Last synced: 14 May 2025

https://github.com/cyan4973/finitestateentropy

New generation entropy codecs : Finite State Entropy and Huff0

c compression entropy fse huffman

Last synced: 15 May 2025

https://github.com/vllm-project/llm-compressor

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

compression quantization sparsity

Last synced: 14 May 2025

https://github.com/Cyan4973/FiniteStateEntropy

New generation entropy codecs : Finite State Entropy and Huff0

c compression entropy fse huffman

Last synced: 15 Mar 2025

https://github.com/vurtun/lib

single header libraries for C/C++

ansi c c89 compression deflate header-only json multithreading webserver

Last synced: 14 May 2025

https://github.com/interdigitalinc/compressai

A PyTorch library and evaluation platform for end-to-end compression research

compression deep-learning deep-neural-networks machine-learning neural-network python pytorch

Last synced: 14 May 2025

https://github.com/zlib-ng/minizip-ng

Fork of the popular zip manipulation library found in the zlib distribution.

bzip2 c compression disk-splitting encryption linux lzma macos minizip pkware windows winzip-aes xz zip zlib

Last synced: 21 Jan 2026

https://github.com/InterDigitalInc/CompressAI

A PyTorch library and evaluation platform for end-to-end compression research

compression deep-learning deep-neural-networks machine-learning neural-network python pytorch

Last synced: 03 May 2025

https://github.com/gpuopen-tools/compressonator

Tool suite for Texture and 3D Model Compression, Optimization and Analysis using CPUs, GPUs and APUs

3d-models analysis astc bc1 bc6h bc7 compression compressonator decompression dxtc dxtc1 etc1 etc2 gui-application mipmaps sdk texture-compression

Last synced: 15 May 2025

https://github.com/GPUOpen-Tools/compressonator

Tool suite for Texture and 3D Model Compression, Optimization and Analysis using CPUs, GPUs and APUs

3d-models analysis astc bc1 bc6h bc7 compression compressonator decompression dxtc dxtc1 etc1 etc2 gui-application mipmaps sdk texture-compression

Last synced: 08 May 2025

https://github.com/GPUOpen-Tools/Compressonator

Tool suite for Texture and 3D Model Compression, Optimization and Analysis using CPUs, GPUs and APUs

3d-models analysis astc bc1 bc6h bc7 compression compressonator decompression dxtc dxtc1 etc1 etc2 gui-application mipmaps sdk texture-compression

Last synced: 15 Mar 2025

https://github.com/Freaky/Compactor

A user interface for Windows 10 filesystem compression

compression rust windows windows-10

Last synced: 28 Mar 2025

https://github.com/freaky/compactor

A user interface for Windows 10 filesystem compression

compression rust windows windows-10

Last synced: 12 Apr 2025

https://github.com/vortex-data/vortex

An extensible, state of the art columnar file format

array arrow compression python rust

Last synced: 10 Apr 2026

https://github.com/ma-lab-berkeley/crate

Code for CRATE (Coding RAte reduction TransformEr).

compression sparsification transformer-architecture white-box-architecture

Last synced: 14 May 2025

https://github.com/Ma-Lab-Berkeley/CRATE

Code for CRATE (Coding RAte reduction TransformEr).

compression sparsification transformer-architecture white-box-architecture

Last synced: 22 Jul 2025

https://github.com/arm-software/astc-encoder

The Arm ASTC Encoder, a compressor for the Adaptive Scalable Texture Compression data format.

astc astcenc compression image-compression

Last synced: 14 May 2025

https://github.com/mit-han-lab/gan-compression

[CVPR 2020] GAN Compression: Efficient Architectures for Interactive Conditional GANs

compression conditional-gans cyclegan gans gaugan image-to-image-translation pix2pix pytorch

Last synced: 16 May 2025

https://github.com/taoensso/nippy

Fast serialization library for Clojure

clojure compression edn encryption epl serialization taoensso

Last synced: 15 Jan 2026

https://github.com/horseee/LLM-Pruner

[NeurIPS 2023] LLM-Pruner: On the Structural Pruning of Large Language Models. Support Llama-3/3.1, Llama-2, LLaMA, BLOOM, Vicuna, Baichuan, TinyLlama, etc.

baichuan bloom chatglm compression language-model llama llama-2 llama3 llm neurips-2023 pruning pruning-algorithms vicuna

Last synced: 01 Oct 2025

https://github.com/bareos/bareos

Bareos is a cross-network Open Source backup solution (licensed under AGPLv3) which preserves, archives, and recovers data from all major operating systems.

archiving backup backup-solution backup-utility bareos ceph compression cross-platform disaster-recovery encrypt gluster mysql postgresql python recover restore s3 security vmware

Last synced: 13 May 2025

https://github.com/rustmailer/bichon

Bichon โ€“ A lightweight, high-performance Rust email archiver with WebUI

backup compression email email-archiver email-archiving imap mail oauth2 openapi rust search self-hosted selfhosted tantivy webui

Last synced: 22 Jan 2026

https://github.com/caoscott/srec

PyTorch Implementation of "Lossless Image Compression through Super-Resolution"

compression lossless neural-network pytorch

Last synced: 12 Apr 2025

https://github.com/caoscott/SReC

PyTorch Implementation of "Lossless Image Compression through Super-Resolution"

compression lossless neural-network pytorch

Last synced: 14 Jul 2025

https://github.com/y-scope/clp

Compressed Log Processor (CLP) is a free log management tool capable of compressing logs and searching the compressed logs without decompression.

analytics compression log-management log-parser logging search

Last synced: 21 Jan 2026

https://github.com/tr1s/tris-webpack-boilerplate

A Webpack boilerplate for static websites that has all the necessary modern tools and optimizations built-in. Score a perfect 10/10 on performance.

babel boilerplate compression es6 html offline offline-first optimization pwa sass scss starter static-site webpack webpack-boilerplate webpack4

Last synced: 16 May 2025

https://github.com/mxmlnkn/ratarmount

Access large archives as a filesystem efficiently, e.g., TAR, RAR, ZIP, GZ, BZ2, XZ, ZSTD archives

bz2 bzip2 compression filesystem fuse gz gzip mergefs mounting overlayfs python random-access rar tar tar-archive unionfs xz zip zstandard zstd

Last synced: 10 Apr 2025

https://github.com/horseee/llm-pruner

[NeurIPS 2023] LLM-Pruner: On the Structural Pruning of Large Language Models. Support Llama-3/3.1, Llama-2, LLaMA, BLOOM, Vicuna, Baichuan, TinyLlama, etc.

baichuan bloom chatglm compression language-model llama llama-2 llama3 llm neurips-2023 pruning pruning-algorithms vicuna

Last synced: 16 May 2025

https://github.com/blosc/c-blosc

A blocking, shuffling and loss-less compression library that can be faster than `memcpy()`.

c compression fast

Last synced: 14 May 2025

https://github.com/Blosc/c-blosc

A blocking, shuffling and loss-less compression library that can be faster than `memcpy()`.

c compression fast

Last synced: 20 Apr 2025

https://github.com/nih-at/libzip

A C library for reading, creating, and modifying zip archives.

archive compression encryption library zip

Last synced: 21 Feb 2026

https://github.com/eashish93/imgsquash

Simple image compression full website code written in node, react and next.js framework. Easy to deploy as a microservice.

compression image-compressor image-processing nextjs nodejs react

Last synced: 15 Mar 2025

https://github.com/inikep/lzbench

lzbench is an in-memory benchmark of open-source compressors

benchmark benchmarking brotli compression compressor lz4 zlib zstd

Last synced: 14 May 2025

https://github.com/Tebmer/Awesome-Knowledge-Distillation-of-LLMs

This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vertical Distillation of LLMs.

alignment compression data-augmentation data-synthesis feedback instruction-following kd knowledge-distillation large-language-model llm multi-modal self-distillation self-training supervised-finetuning survey

Last synced: 12 Apr 2025