{"id":15394898,"url":"https://github.com/cgbur/arcode-rs","last_synced_at":"2025-09-13T06:47:02.889Z","repository":{"id":34641062,"uuid":"180714704","full_name":"cgbur/arcode-rs","owner":"cgbur","description":"An arithmetic coder for Rust.","archived":false,"fork":false,"pushed_at":"2023-05-24T15:59:02.000Z","size":629,"stargazers_count":23,"open_issues_count":3,"forks_count":4,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-08-31T20:58:40.298Z","etag":null,"topics":["arithmetic-coding","cabac","data-compression","decoding","encoder","entropy-coding","lossless","lossless-compression-algorithm","lossless-data-compression"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cgbur.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-04-11T04:33:46.000Z","updated_at":"2025-08-29T14:56:34.000Z","dependencies_parsed_at":"2024-10-19T02:13:03.233Z","dependency_job_id":"05f9c7d1-0ea2-41bf-a288-42ef0a15acb4","html_url":"https://github.com/cgbur/arcode-rs","commit_stats":{"total_commits":82,"total_committers":6,"mean_commits":"13.666666666666666","dds":0.6097560975609756,"last_synced_commit":"e4fb8815051dad653363a1b333396ee267642af0"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/cgbur/arcode-rs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cgbur%2Farcode-rs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cgbur%2Farcode-rs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cgbur%2Farcode-rs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cgbur%2Farcode-rs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cgbur","download_url":"https://codeload.github.com/cgbur/arcode-rs/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cgbur%2Farcode-rs/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274930260,"owners_count":25375712,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-13T02:00:10.085Z","response_time":70,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["arithmetic-coding","cabac","data-compression","decoding","encoder","entropy-coding","lossless","lossless-compression-algorithm","lossless-data-compression"],"created_at":"2024-10-01T15:24:50.172Z","updated_at":"2025-09-13T06:47:02.866Z","avatar_url":"https://github.com/cgbur.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Arcode\nAn arithmetic coder for Rust.\n\n\n[![Crates.io](https://img.shields.io/crates/v/arcode?color=blueviolet)](https://crates.io/crates/arcode)\n[![Crates.io](https://img.shields.io/crates/l/arcode)](https://crates.io/crates/arcode)\n[![GitHub top language](https://img.shields.io/github/languages/top/cgbur/arcode-rs?color=orange)](https://crates.io/crates/arcode)\n\n## About\n\nThis crate provides the an efficient implementation of\nan [arithmetic encoder/decoder](https://en.wikipedia.org/wiki/Arithmetic_coding). This crate is based off the paper\nthat describes arithmetic coding found [here](https://web.stanford.edu/class/ee398a/handouts/papers/WittenACM87ArithmCoding.pdf).\nThis implementation features many readability and performance improvements, especially on the decoding side.\n\nThe goal of this project is not to provide an out-of-the-box compression solution.\nArithmetic coding ([entropy encoding](https://en.wikipedia.org/wiki/Entropy_encoding)) is the backbone of almost every\nmodern day compression scheme. This crate is meant to be included in future projects that rely on an efficient entropy\ncoder e.g. [PPM](https://en.wikipedia.org/wiki/Prediction_by_partial_matching), [LZ77/LZ78](https://en.wikipedia.org/wiki/LZ77_and_LZ78),\n[h265/HEVC](https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding).\n\n## Core components\n\nThere are a lot of structs available for use but for the average user there are only a few that will be used.\n- [Model](model/struct.Model.html) models of the probability of symbols. Counts can be adjusted\nas encoding is done to improve compression.\n- [Encoder](encode/struct.ArithmeticEncoder.html) encodes symbols given a source model and a symbol.\n- [Decoder](decode/struct.ArithmeticDecoder.html) decodes symbols given a source model and a bitstream.\n\n# Examples\nIn the git repository there is an [old_complex.rs](https://github.com/cgbur/arcode-rs/blob/master/example/example.rs)\nfile that does context switching on a per character basis. A simpler example can be found at [new_simple.rs](https://github.com/cgbur/arcode-rs/blob/master/tests/integration_test.rs)\n\n## Input and output bitstreams\nIn order for arithmetic coding to work streams need to be read a bit at a time (for decoding and for the encoders output).\nBecause of this, [BitBit](https://docs.rs/bitbit) is required. **Wrapping whatever your input is in a buffered reader/writer\nshould greatly improve performance.**\n\nUsing bitbit to create an input stream.\n```rust\nuse arcode::bitbit::{BitReader, MSB, BitWriter};\nuse std::io::Cursor;\n\nfn read_example() {\n  // normally you would have a Read type with a BufReader\n  let mut source = Cursor::new(vec![0u8; 4]);\n  let input: BitReader\u003c_, MSB\u003e = BitReader::new(\u0026mut source);\n}\n\nfn out_example() {\n  // once again would be Write type with a BufWriter\n  let compressed = Cursor::new(vec![]);\n  let mut compressed_writer = BitWriter::new(compressed);\n}\n```\n\n### Source Model(s)\nDepending on your application you could have one or many source models.\nThe source model is relied on by the encoder and the decoder. If the decoder ever becomes\nout of phase with the encoder you will be decoding nonsense.\n\n#### model::Builder\nIn order to make a source model you need to use the model::Builder struct.\n\n```rust\nuse arcode::{EOFKind, Model};\n\nfn source_model_example() {\n  // create a new model that has symbols 0-256\n  // 8 bit values + one EOF marker\n  let mut model_with_eof = Model::builder()\n    .num_symbols(256)\n    .eof(EOFKind::EndAddOne)\n    .build();\n  \n  // model for 8 bit 0 - 255, if we arent using\n  // the EOF flag we can set it to NONE or let it default\n  // to none as in the second example below.\n  let model_without_eof = Model::builder()\n    .num_symbols(256)\n    .eof(EOFKind::None)\n    .build();\n  let model_without_eof = Model::builder().num_symbols(256).build();\n\n  // we can also create a model for 0-255 using num_bits\n  let model_8_bit = Model::builder().num_bits(8).build();\n  \n  // update the probability of symbol 4.\n  model_with_eof.update_symbol(4);\n}\n```\n## Encode\nEncoding some simple input\n```rust\nuse arcode::bitbit::BitWriter;\nuse arcode::{ArithmeticEncoder, EOFKind, Model};\nuse std::io::{Cursor, Result};\n\n/// Encodes bytes and returns the compressed form\nfn encode(data: \u0026[u8]) -\u003e Result\u003cVec\u003cu8\u003e\u003e {\n  let mut model = Model::builder()\n    .num_bits(8)\n    .eof(EOFKind::EndAddOne)\n    .build();\n\n  // make a stream to collect the compressed data\n  let compressed = Cursor::new(vec![]);\n  let mut compressed_writer = BitWriter::new(compressed);\n  \n  let mut encoder = ArithmeticEncoder::new(48);\n\n  for \u0026sym in data {\n    encoder.encode(sym as u32, \u0026model, \u0026mut compressed_writer)?;\n    model.update_symbol(sym as u32);\n  }\n\n  encoder.encode(model.eof(), \u0026model, \u0026mut compressed_writer)?;\n  encoder.finish_encode(\u0026mut compressed_writer)?;\n  compressed_writer.pad_to_byte()?;\n\n  // retrieves the bytes from the writer. This will\n  // be cleaner when bitbit updates. Not necessary if\n  // using files or a stream\n  Ok(compressed_writer.get_ref().get_ref().clone())\n}\n\n```\n### Decode\n```rust\nuse arcode::bitbit::{BitReader, MSB};\nuse arcode::{ArithmeticDecoder, EOFKind, Model};\nuse std::io::{Cursor, Result};\n\n/// Decompresses the data\nfn decode(data: \u0026[u8]) -\u003e Result\u003cVec\u003cu8\u003e\u003e {\n  let mut model = Model::builder()\n    .num_bits(8)\n    .eof(EOFKind::EndAddOne)\n    .build();\n\n  let mut input_reader = BitReader::\u003c_, MSB\u003e::new(data);\n  let mut decoder = ArithmeticDecoder::new(48);\n  let mut decompressed_data = vec![];\n\n  while !decoder.finished() {\n    let sym = decoder.decode(\u0026model, \u0026mut input_reader)?;\n    model.update_symbol(sym);\n    decompressed_data.push(sym as u8);\n  }\n\n  decompressed_data.pop(); // remove the EOF\n\n  Ok(decompressed_data)\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcgbur%2Farcode-rs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcgbur%2Farcode-rs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcgbur%2Farcode-rs/lists"}