{"id":17644435,"url":"https://github.com/vigna/dsi-bitstream-rs","last_synced_at":"2025-05-07T01:40:40.789Z","repository":{"id":165610131,"uuid":"640910683","full_name":"vigna/dsi-bitstream-rs","owner":"vigna","description":"A Rust implementation of read/write bit streams supporting several types of instantaneous codes","archived":false,"fork":false,"pushed_at":"2025-03-17T14:23:07.000Z","size":4214,"stargazers_count":7,"open_issues_count":7,"forks_count":4,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-18T11:50:02.756Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vigna.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING","funding":null,"license":"LICENSE-Apache-2.0","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-15T11:38:48.000Z","updated_at":"2025-03-17T14:23:00.000Z","dependencies_parsed_at":"2023-10-11T20:34:23.176Z","dependency_job_id":"e67b7bdf-20c0-4e71-98f6-aa0bd343fab9","html_url":"https://github.com/vigna/dsi-bitstream-rs","commit_stats":null,"previous_names":[],"tags_count":16,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vigna%2Fdsi-bitstream-rs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vigna%2Fdsi-bitstream-rs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vigna%2Fdsi-bitstream-rs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vigna%2Fdsi-bitstream-rs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vigna","download_url":"https://codeload.github.com/vigna/dsi-bitstream-rs/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252797650,"owners_count":21805744,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-23T10:05:26.634Z","updated_at":"2025-05-07T01:40:40.781Z","avatar_url":"https://github.com/vigna.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# dsi-bitstream\n\n[![downloads](https://img.shields.io/crates/d/dsi-bitstream)](https://crates.io/crates/dsi-bitstream)\n[![dependents](https://img.shields.io/librariesio/dependents/cargo/dsi-bitstream)](https://crates.io/crates/dsi-bitstream/reverse_dependencies)\n![GitHub CI](https://github.com/vigna/dsi-bitstream-rs/actions/workflows/rust.yml/badge.svg)\n![license](https://img.shields.io/crates/l/dsi-bitstream)\n[![](https://tokei.rs/b1/github/vigna/dsi-bitstream-rs?type=Rust,Python)](https://github.com/vigna/dsi-bitstream-rs)\n[![Latest version](https://img.shields.io/crates/v/dsi-bitstream.svg)](https://crates.io/crates/dsi-bitstream)\n[![Documentation](https://docs.rs/dsi-bitstream/badge.svg)](https://docs.rs/dsi-bitstream)\n[![Coverage Status](https://coveralls.io/repos/github/vigna/dsi-bitstream-rs/badge.svg?branch=main)](https://coveralls.io/github/vigna/dsi-bitstream-rs?branch=main)\n\nA Rust implementation of bit streams supporting several types of instantaneous\ncodes for compression.\n\nThis library mimics the behavior of the analogous classes in the [DSI\nUtilities], but it aims at being much more flexible and (hopefully) efficient.\n\nThe two main traits are [`BitRead`] and [`BitWrite`], with which are associated\ntwo main implementations [`BufBitReader`] and [`BufBitWriter`]. Additional\ntraits make it possible to read and write instantaneous codes, like the\n[exponential Golomb codes] used in [H.264 (MPEG-4)] and [H.265].\n\n```rust\n# fn main() -\u003e Result\u003c(), Box\u003cdyn std::error::Error\u003e\u003e {\nuse dsi_bitstream::prelude::*;\n// To write a bit stream, we need first a WordWrite around an output backend\n// (in this case, a vector), which is word-based for efficiency.\n// It could be a file, etc. \nlet mut word_write = MemWordWriterVec::new(Vec::\u003cu64\u003e::new());\n// Let us create a little-endian bit writer. The write word size will be inferred.\nlet mut writer = BufBitWriter::\u003cLE, _\u003e::new(word_write);\n// Write 0 using 10 bits\nwriter.write_bits(0, 10)?;\n// Write 1 in unary code\nwriter.write_unary(0)?;\n// Write 2 in γ code\nwriter.write_gamma(1)?;\n// Write 3 in δ code\nwriter.write_delta(2)?;\nwriter.flush();\n\n// Let's recover the data\nlet data = writer.into_inner()?.into_inner();\n\n// Reading back the data is similar, but since a reader has a bit buffer\n// twice as large as the read word size, it is more efficient to use a \n// u32 as read word, so we need to transmute the data.\nlet data = unsafe { std::mem::transmute::\u003c_, Vec\u003cu32\u003e\u003e(data) };\nlet mut reader = BufBitReader::\u003cLE, _\u003e::new(MemWordReader::new(data));\nassert_eq!(reader.read_bits(10)?, 0);\nassert_eq!(reader.read_unary()?, 0);\nassert_eq!(reader.read_gamma()?, 1);\nassert_eq!(reader.read_delta()?, 2);\n\n# Ok(())\n# }\n```\n\nIn this case, the backend is already word-based, but if you have a byte-based\nbackend such as a file [`WordAdapter`] can be used to adapt it to a word-based\nbackend.\n\nYou can also use references to backends instead of owned values,\nbut this approach is less efficient:\n\n```rust\n# fn main() -\u003e Result\u003c(), Box\u003cdyn std::error::Error\u003e\u003e {\nuse dsi_bitstream::prelude::*;\nlet mut data = Vec::\u003cu64\u003e::new();\nlet mut word_write = MemWordWriterVec::new(\u0026mut data);\nlet mut writer = BufBitWriter::\u003cLE, _\u003e::new(word_write);\nwriter.write_bits(0, 10)?;\nwriter.write_unary(0)?;\nwriter.write_gamma(1)?;\nwriter.write_delta(2)?;\nwriter.flush();\ndrop(writer); // We must drop the writer release the borrow on data\n\nlet data = unsafe { std::mem::transmute::\u003c_, Vec\u003cu32\u003e\u003e(data) };\nlet mut reader = BufBitReader::\u003cLE, _\u003e::new(MemWordReader::new(\u0026data));\nassert_eq!(reader.read_bits(10)?, 0);\nassert_eq!(reader.read_unary()?, 0);\nassert_eq!(reader.read_gamma()?, 1);\nassert_eq!(reader.read_delta()?, 2);\n# Ok(())\n# }\n```\n\nPlease read the documentation of the [`traits`] module and the [`impls`] module\nfor more details.\n\n## Options\n\nThere are a few options to modify the behavior of the bit read/write traits:\n\n- Endianness can be selected using the [`BE`] or [`LE`] types as the first\n  parameter. The native endianness is usually the best choice, albeit sometimes\n  the lack of some low-level instructions (first bit set, last bit etc, etc.)\n  may make the non-native endianness more efficient.\n- Data is read from or written to the backend one word at a time, and the size\n  of the word can be selected using the second parameter, but it must match the\n  word size of the backend, so it is usually inferred. Currently, we suggest\n  `usize` for writing and a type that is half of `usize` for reading.\n\nMore in-depth (and much more complicated) tuning can be obtained by modifying\nthe default values for the parameters of instantaneous codes. Methods reading or\nwriting instantaneous codes are defined in supporting traits and usually have\nconst type parameters, in particular, whether to use decoding tables or not\n(e.g., [`GammaReadParam::read_gamma_param`]). Such traits are implemented for\n[`BitRead`]/[`BitWrite`]. The only exception is unary code, which is implemented\nby [`BitRead::read_unary`] and [`BitWrite::write_unary`].\n\nHowever, there are traits with non-parametric methods (e.g.,\n[`GammaRead::read_gamma`]) that are the standard entry points for the user.\nThese traits are implemented for [`BufBitReader`]/[`BufBitWriter`] depending on\na selector type implementing [`ReadParams`]/[`WriteParams`], respectively.\nThe default value for the parameter is\n[`DefaultReadParams`]/[`DefaultWriteParams`], which uses choices we tested on\nseveral platforms and that we believe are good defaults, but by passing a\ndifferent implementation of [`ReadParams`]/[`WriteParams`] you can change the\ndefault behavior. See [`params`] for more details.\n\nFinally, if you choose to use tables, the size of the tables is hardwired in the\nsource code (in particular, in the files `*_tables.rs` in the `codes` source\ndirectory) and can be changed only by regenerating the tables using the script\n`gen_code_tables.py` in the `python` directory. You will need to modify the\nvalues hardwired at the end of the script.\n\n## Dispatching\n\nWe provide several options to [dispatch] codes dynamically.\n\n## Benchmarks\n\nTo evaluate the performance on your hardware you can run the\nbenchmarks in the `benchmarks` directory, which test the speed of read/write\noperations under several combinations of parameters. Please refer to the crate\ndocumentation therein. The `svg` directory contains reference results of these\nbenchmarks of a few architectures.\n\n## Testing\n\nBesides unit tests, we provide zipped precomputed corpora generated by fuzzing.\nYou can run the tests on the zipped precomputed corpora by enabling the `fuzz`\nfeature:\n\n```shell\ncargo test --features fuzz\n```\n\nWhen the feature is enabled, tests will be also run on local corpora found in\nthe top-level `fuzz` directory, if any are present.\n\n## Acknowledgments\n\nThis software has been partially supported by project SERICS (PE00000014) under\nthe NRRP MUR program funded by the EU - NGEU, and by project ANR COREGRAPHIE,\ngrant ANR-20-CE23-0002 of the French Agence Nationale de la Recherche. Views and\nopinions expressed are however those of the authors only and do not necessarily\nreflect those of the European Union or the Italian MUR. Neither the European\nUnion nor the Italian MUR can be held responsible for them.\n\n[`BitRead`]: \u003chttps://docs.rs/dsi-bitstream/latest/dsi_bitstream/traits/trait.BitRead.html\u003e\n[`BitWrite`]: \u003chttps://docs.rs/dsi-bitstream/latest/dsi_bitstream/traits/trait.BitWrite.html\u003e\n[`BufBitReader`]: \u003chttps://docs.rs/dsi-bitstream/latest/dsi_bitstream/impls/struct.BufBitReader.html\u003e\n[`BufBitWriter`]: \u003chttps://docs.rs/dsi-bitstream/latest/dsi_bitstream/impls/struct.BufBitWriter.html\u003e\n[`ReadParams`]: \u003chttps://docs.rs/dsi-bitstream/latest/dsi_bitstream/codes/params/trait.ReadParams.html\u003e\n[`WriteParams`]: \u003chttps://docs.rs/dsi-bitstream/latest/dsi_bitstream/codes/params/trait.WriteParams.html\u003e\n[`GammaReadParam::read_gamma_param`]: \u003chttps://docs.rs/dsi-bitstream/latest/dsi_bitstream/codes/gamma/trait.GammaReadParam.html#tymethod.read_gamma_param\u003e\n[`WordAdapter`]: \u003chttps://docs.rs/dsi-bitstream/latest/dsi_bitstream/impls/struct.WordAdapter.html\u003e\n[`traits`]: \u003chttps://docs.rs/dsi-bitstream/latest/dsi_bitstream/traits/index.html\u003e\n[`impls`]: \u003chttps://docs.rs/dsi-bitstream/latest/dsi_bitstream/impls/index.html\u003e\n[`params`]: \u003chttps://docs.rs/dsi-bitstream/latest/dsi_bitstream/codes/params/index.html\u003e\n[`GammaRead::read_gamma`]: \u003chttps://docs.rs/dsi-bitstream/latest/dsi_bitstream/codes/gamma/trait.GammaRead.html#tymethod.read_gamma\u003e\n[`BE`]: \u003chttps://docs.rs/dsi-bitstream/latest/dsi_bitstream/traits/type.BE.html\u003e\n[`LE`]: \u003chttps://docs.rs/dsi-bitstream/latest/dsi_bitstream/traits/type.LE.html\u003e\n[`DefaultReadParams`]: \u003chttps://docs.rs/dsi-bitstream/latest/dsi_bitstream/codes/params/struct.DefaultReadParams.html\u003e\n[`DefaultWriteParams`]: \u003chttps://docs.rs/dsi-bitstream/latest/dsi_bitstream/codes/params/struct.DefaultWriteParams.html\u003e\n[`BitRead::read_unary`]: \u003chttps://docs.rs/dsi-bitstream/latest/dsi_bitstream/traits/trait.BitRead.html#method.read_unary\u003e\n[`BitWrite::write_unary`]: \u003chttps://docs.rs/dsi-bitstream/latest/dsi_bitstream/traits/trait.BitWrite.html#method.write_unary\u003e\n[DSI Utilities]: \u003chttps://dsiutils.di.unimi.it/\u003e\n[exponential Golomb codes]: \u003chttps://docs.rs/dsi-bitstream/latest/dsi_bitstream/codes/exp_golomb/index.html\u003e\n[H.264 (MPEG-4)]: \u003chttps://en.wikipedia.org/wiki/Advanced_Video_Coding\u003e\n[H.265]: \u003chttps://en.wikipedia.org/wiki/High_Efficiency_Video_Coding\u003e\n[dispatch]: \u003chttps://docs.rs/dsi-bitstream/latest/dsi_bitstream/dispatch/index.html\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvigna%2Fdsi-bitstream-rs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvigna%2Fdsi-bitstream-rs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvigna%2Fdsi-bitstream-rs/lists"}