Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

Projects in Awesome Lists tagged with data-compression

A curated list of projects in awesome lists tagged with data-compression .

https://github.com/matsui528/nanopq

Pure python implementation of product quantization for nearest neighbor search

approximate-nearest-neighbor-search data-compression nearest-neighbor-search product-quantization

Last synced: 24 Dec 2025

https://github.com/jermp/data_compression_course

A Crash Course on Data Compression.

data-compression data-compression-algorithms

Last synced: 24 Dec 2025

https://github.com/avaneev/lzav

Fast In-Memory Data Compression Algorithm (inline C/C++) 460+MB/s compress, 2500+MB/s decompress, ratio% better than LZ4, Snappy, and Zstd@-1

compress compression compression-algorithm compression-library compressor data-compression data-compression-algorithms lossless-compression lossless-compression-algorithm lossless-data-compression lz4 lz77 lzf snappy

Last synced: 10 Jul 2025

https://github.com/ayastreb/bandwidth-hero

:guardsman: Saves data by compressing images on web page

chrome-extension data-compression data-saving firefox-addon react semantic-ui web-extension webpack

Last synced: 08 Apr 2025

https://github.com/matsui528/rii

Fast and memory-efficient ANN with a subset-search functionality

approximate-nearest-neighbor-search data-compression nearest-neighbor-search product-quantization pybind11

Last synced: 30 Dec 2025

https://github.com/hellobertrand/zxc

High-performance asymmetric lossless compression library optimized for Content Delivery. Decodes 40% faster than LZ4 on ARM64.

apple-silicon arm64 c compression compression-library data-compression embedded high-performance lossless lossless-compression lz4

Last synced: 14 Feb 2026

https://github.com/einar-saukas/ZX0

Data compressor for 8-bit computers and low-end platforms

compression compressor data-compression

Last synced: 19 Nov 2025

https://github.com/maxim-zhao/bmp2tile

Converts images to Master System data formats

data-compression image-conversion sega-game-gear sega-master-system

Last synced: 02 Feb 2026

https://github.com/sbsaga/toon

TOON — Laravel AI package for compact, human-readable, token-efficient data format with JSON ⇄ TOON conversion for ChatGPT, OpenAI, and other LLM prompts.

ai ai-library api-optimization chatgpt compact-format data-compression developer-tool human-readable json laravel laravel-package large-language-models llm openai php php-library prompt-engineering serialization token-optimization toon

Last synced: 02 Mar 2026

https://github.com/jbruchon/lzjody

Lempel-Ziv-JodyBruchon compression - a mix of interesting compression methods

compression-algorithm data-compression lempel-ziv lz rle run-length-encoding

Last synced: 13 May 2025

https://github.com/wittline/wbz

A parallel implementation of the bzip2 data compressor in python, this data compression pipeline is using algorithms like Burrows–Wheeler transform (BWT) and Move to front (MTF) to improve the Huffman compression. For now, this tool only will be focused on compressing .csv files, and other files on tabular format.

big-data bigdata burrows-wheeler-transform csv csv-files data-compression data-engineering huffman-coding huffman-compression-algorithm move-to-front tabular-data

Last synced: 24 Aug 2025

https://github.com/01alchemist/as-lzma

LZMA Decoder written in AssemblyScript

assemblyscript data-compression lzma typescript webassembly

Last synced: 13 Jul 2025

https://github.com/01alchemist/AS-LZMA

LZMA Decoder written in AssemblyScript

assemblyscript data-compression lzma typescript webassembly

Last synced: 15 May 2025

https://github.com/hxndev/lzw-data-encryption-algorithm

LZW stands for “Lempel-Ziv-Welch”. The LZW algorithm is a very common compression technique. This algorithm is typically used in GIF and PDF. It is lossless, meaning no data is lost when compressing. The algorithm is simple to implement and has the potential for very high throughput in hardware implementations. It is the algorithm of the widely used Unix file compression utility compress, and is used in the GIF image format.

algorithm code compression cpp data-compression decoding decryption dictionary encoding encryption lzw project

Last synced: 18 Jul 2025

https://github.com/mkloubert/node-simple-socket

Wrapper for Node.js sockets that makes it easy to send data compressed and crypted (RSA / AES).

aes client-server communication data-compression downloader events node-socket receive-data rsa socket stream typescript2 uploader wrapper

Last synced: 25 Jul 2025

https://github.com/wittline/huffman-decoding

A New Approach for Efficient Sequential Decoding of Static Huffman Codes

burrows-wheeler-transform compression data-compression huffman-coding huffman-decoder move-to-front

Last synced: 06 Jan 2026

https://github.com/jonnor/acm2019-compress

Hardware compression for memory

data-compression memory-hierarchy

Last synced: 07 Oct 2025

https://github.com/khaledashrafh/lz-77

This repository contains a Java implementation of the LZ-77 data compression algorithm. LZ-77 is a lossless data compression method that replaces repetitive sequences of characters with references to previously occurring data. It achieves compression by storing the position and length of a match in a sliding window of previously processed data.

compression data-compression decompression lempel-ziv lempel-ziv-77 lz77 lz77-compression-algorithm multimedia

Last synced: 17 Oct 2025

https://github.com/khaledashrafh/lz-78

This repository contains Java code implementing the LZ-78 (Lempel-Ziv 78) data compression algorithm. The LZ-78 algorithm is a lossless data compression method that replaces repeated occurrences of data patterns with references to previously encountered patterns.

compression data-compression decompression lempel-ziv lempel-ziv-78 lz78 lz78-compression multimedia

Last synced: 17 Oct 2025

https://github.com/jeremy-rifkin/markov-huffman-coding

This is a proof of concept for an experimental data compression technique that applies Markov chains to Huffman coding.

compression data-compression

Last synced: 25 Aug 2025

https://github.com/mpetri/ans-large-alphabet

Large-Alphabet Semi-Static Entropy Coding Via Asymmetric Numeral Systems

compression data-compression entropy-coding research-paper

Last synced: 08 Mar 2026

https://github.com/arqonai/pixelog

Pixelog is a Go and React app that transforms files and folders into encrypted, QR-encoded video streams (.pixe) for portable, offline-friendly knowledge storage. Includes a modern web UI, CLI, Docker support, real-time WebSocket progress, and FFmpeg based video packaging.

ai cli context data-compression docker encryption ffmpeg file-conversion go golang llm memory microservices pixelog pwa qr qr-code react video-encoding vite

Last synced: 07 Oct 2025

https://github.com/oelin/polynomial-regression

Demonstrating how to use crude MDL to regularize polynomial regression models.

coding-theory data-compression data-science machine-learning minimum-description-length statistics

Last synced: 12 Mar 2025

https://github.com/gwlucastrig/gridfourc

A C-language API with tools for raster data including geophysical applications and digital elevation models

autonomous-underwater-vehicle bathymetry c data-compression digital-elevation-model geospatial raspberry-pi uav uuv

Last synced: 12 Jan 2026

https://github.com/thomasthaddeus/algorithmselector

Explore our extensive Algorithms Repository, featuring a diverse range of algorithms from computational methods to data structures, cryptographic techniques, and AI. Ideal for education, research, and practical application, each algorithm is well-documented, tested, and optimized for performance and readability.

ai algorithms computational-algorithms computer-science cryptography data-compression data-structures educational-resource game-theory graph-theory machine-learning network-flow open-source optimization python

Last synced: 01 Mar 2026

https://github.com/trizen/compression-util

Implementation of various techniques used in data compression.

bwt bzip2 compression data-compression deflate huffman lossless-data-compression lz77 lzss mtf rle

Last synced: 03 Mar 2025

https://github.com/hxndev/aes-encoding-algorithm

AES stands for Advanced Encryption Standard. It is mostly used as a replacement of DES. It has a bigger key size than DES which provides encryption at least 6 times faster. AES specifies a FIPS-approved cryptographic algorithm that can be used to protect electronic data. The AES algorithm is a symmetric block cipher that can encrypt (encipher) and decrypt (decipher) information. Encryption converts data to an unintelligible form called ciphertext; decrypting the ciphertext converts the data back into its original form, called plaintext. The AES algorithm can use cryptographic keys of 128, 192, and 256 bits to encrypt and decrypt data in blocks of 128 bits.

aes aes-cipher aes-encryption algorithm code compression cpp data-compression decryption encoding project

Last synced: 31 Mar 2025

https://github.com/indianwhocodes/adshuffmanproject

Comparative Evaluation of Data Compression through binary, four-way, and pairing heaps without any standard lib. container except vector.

academic academic-project binary-heap cpp data-compression data-structures heaps

Last synced: 03 Apr 2025

https://github.com/nurfawaiq/kompresi-huffman

Teknik Kompresi dengan Algoritma Huffman

data-compression huffman-algorithm java

Last synced: 18 Mar 2025

https://github.com/tallamjr/shannon

Implementation of various compression algorithms

compression compression-algorithms data-compression lossless lossy

Last synced: 04 Apr 2025

https://github.com/wittline/burrows-wheeler-transform

Implementation of the algorithm "Burrows Wheeler Transform" in python for data compression

burrows-wheeler-transform data-compression python

Last synced: 07 Nov 2025

https://github.com/ndsvw/huffman-coding-data-compression

Huffman code is a particular type of optimal prefix code that is commonly used for lossless data compression. It's usually implemented via a greedy approach.

compression compression-algorithm data-compression huffman huffman-coding java

Last synced: 10 Jun 2025

https://github.com/sirsayed98/arithmetic-algorithm

A python compression and decompression program based on Arithmetic Coding.

arithmetic-algorithm arithmetic-coding data-compression image-compression python

Last synced: 02 Jul 2025

https://github.com/sureshbeekhani/autoencoders

This project is a practice implementation of an autoencoder, The primary use case for this autoencoder is for anomaly detection in sales data, but it can be adapted for other purposes. The autoencoder compresses the input data into a lower-dimensional representation and then reconstructs the original input from this representation.

autoencoder convolutional-autoencoder data-compression data-reconstruction deep-learning denoising-autoencoder generative-models image-processing latent-space machine-learning neural-networks vanilla-autoencoder variational-autoencode

Last synced: 04 Sep 2025

https://github.com/kamionn/zippypack

🚀 Modern Rust compression tool with block-level deduplication and zstd compression. Superior ratios vs WinRAR/7-Zip with real-time progress and system image format.

archiver backup-tool cli-tool compression cross-platform data-compression deduplication file-archiver file-compression performance rust zippypack zstd

Last synced: 11 Oct 2025

https://github.com/steviecurran/gbt-scripts

IDL scripts for the reduction of Green Bank Telescope data

data-analysis data-compression data-visualization radio-astronomy spectroscopy

Last synced: 31 Jan 2026

https://github.com/fednazar/kitten

Lossless data compression algorithm, aimed at fast decompression speeds and better compression ratios than LZ4HC and other speed-oriented algorithms

compress compression compression-algorithm compressor data-compression game-development gamedev lossless-compression lossless-compression-algorithm lossless-data-compression lz77 lz77-compression-algorithm lzss lzss-compression-decompression

Last synced: 24 Aug 2025

https://github.com/mohanad-80/data_compression

Data compression algorithms implemented in different languages

algorithms data-compression

Last synced: 21 Aug 2025

https://github.com/oelin/coco

A library for creating self-extracting archives in Python.

data-compression data-science self-extracting-archive serialization

Last synced: 25 Dec 2025

https://github.com/israel-007/stashquiver

This library is set to support cache, API request, data compression, handle errors, format different data types, log data's, rate limiting e.t.c

api api-request api-resources api-response cache cache-storage caching data-compression format-response log-data rate-limiter rate-limiting

Last synced: 27 Mar 2025

https://github.com/12danielll/neural_networks_project

The project focuses on analyzing neural activity data to classify neuron types (spiny and aspiny). It integrates unsupervised learning methods (PCA, Autoencoders) and supervised learning models (Logistic Regression, MLP) to build accurate classifiers that effectively analyze neurons' electrical responses.

2d-and-3d-visualizations autoencoders classifier-evaluation cortical-neurons data-compression gradient-descent high-dimensional-neural-datasets logistic-regression mlp mlp-networks neural-classification neuron neuronal-network pca-analysis perceptron roc-auc stochastic-gradient-descent supervised-learning unsupervised-learning

Last synced: 16 Mar 2025

https://github.com/nurfawaiq/kompresi-shannonfano

Teknik Kompresi dengan Algoritma Shannon-Fano

data-compression java shannon-fano-algorithm

Last synced: 18 Mar 2025

https://github.com/wittline/move-to-front

Implementation of the algorithm "Move to front" in python for data compression

data-compression move-to-front python

Last synced: 24 Mar 2025

https://github.com/oelin/vl-vae

An implementation of VL-VAE in PyTorch.

data-compression progressive-decoding variational-autoencoder

Last synced: 25 Dec 2025

https://github.com/marcosd59/teoria-de-la-informacion

El repositorio incluye programas en Python para simular esquemas de comunicación, estudiar la entropía, y aplicar técnicas de compresión de datos como Huffman, Shannon Fano, LZW y RLE. Se enfoca en la teoría de la información y su aplicación en ingeniería de datos.

data-compression hashing information-recovery

Last synced: 07 Nov 2025

https://github.com/praeclarumjj3/atlas_evaluation

Submission for the Evaluation Exercise for ATLAS Encoders

autoencoder data-compression fastai pytorch

Last synced: 25 Dec 2025

https://github.com/nurfawaiq/kompresi-lzw

Teknik Kompresi dengan Algoritma LZW (Lempel-Ziv-Welch)

data-compression lzw-algorithm

Last synced: 18 Mar 2025

https://github.com/rossellequizon/codage-huffman

Compression et décompression de fichiers avec Huffman en C

algorithm c data-compression huffman-coding

Last synced: 05 Sep 2025

https://github.com/gmersy/hire

Hierarchical Residual Encoding for Multiresolution Time Series Compression

compression compression-algorithm data-compression storage

Last synced: 05 Jul 2025

https://github.com/oelin/implicit-language-models

Elucidation of implicit language models from common data compression algorithms.

beam-search data-compression data-science generative-models language-models machine-learning

Last synced: 01 Nov 2025

https://github.com/youssef-saaed/huffman-coding-algorithm

This repository contains a C implementation of the Huffman coding algorithm, which is an efficient method for lossless data compression. The code constructs a Huffman tree based on the frequency of characters in a given message and generates a unique binary code for each character, minimizing the overall space required to store the message.

algorithms data-compression discrete-mathematics huffman-coding lossless-compression-algorithm

Last synced: 22 Mar 2025

https://github.com/kevinbird61/data-compression

Some implementations of data compression algorithms.

data-compression

Last synced: 18 Jun 2025

https://github.com/guiribei/huffman_algorithm

A project to learn about data compression using huffman algorithm and shared memory operations.

bitwise bitwise-operators c data-compression linux memory-management

Last synced: 08 Oct 2025

https://github.com/antilneeraj/huffmancompressor

A simple, lossless text file compressor and decompressor built in Java using Huffman Coding. It shrinks files by giving shorter codes to frequent characters, and restores them exactly when decompressed. Features include a command‑line interface, clean modular design, and efficient bit‑level file handling

algorithms binary-compression cli-tool compression-algorithm data-compression education file-compression file-io gradle huffman-coding java lossless-compression open-source text-compression

Last synced: 09 Oct 2025

https://github.com/oelin/ar-mnist

An autoregressive model for compressing MNIST digits.

data-compression data-science machine-learning mnist

Last synced: 12 Mar 2025

https://github.com/neimat1/data-compression

Using some techniques to compress and decompress data

data-compression

Last synced: 20 Jun 2025

https://github.com/nazlizamanian/network-programming

Lab assigmenets completed in the course Network Programming course code TNPK18 using python.

data-compression entropy firebase-realtime-database network python regex socket

Last synced: 05 Apr 2025

https://github.com/crossbowerbt/shannon_compressor

An implementation in C of the Shannon's compressor.

data-compression data-compressor shannon shannon-entropy

Last synced: 24 Oct 2025

https://github.com/oelin/ami

🎄 A compression-based measure of mutual information between strings.

algorithmic-information-theory data-compression machine-learning mutual-information python

Last synced: 09 Sep 2025

https://github.com/rsheremeta/archiver

archiver written in Go (Golang) implements Shannon-Fano algorithm for text data compression

data-compression go golang shannon-fano shannon-fano-algorithm

Last synced: 25 Jun 2025

https://github.com/afa-farkhod/huffman-code

Huffman coding, Data compression

data-compression huffman-coding java

Last synced: 01 Sep 2025

https://github.com/oelin/rle

A simple run-length encoding library for Python.

data-compression python run-length-encoding

Last synced: 12 Mar 2025