{"id":13514885,"url":"https://github.com/bodgergely/nitro","last_synced_at":"2025-03-31T03:31:38.449Z","repository":{"id":148364847,"uuid":"169634134","full_name":"bodgergely/nitro","owner":"bodgergely","description":"nitro - blazingly fast compressor from the future","archived":false,"fork":false,"pushed_at":"2019-02-07T20:22:30.000Z","size":15,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-11-01T18:38:01.803Z","etag":null,"topics":["compression","compression-algorithm","compressor","encoder-decoder"],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bodgergely.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2019-02-07T20:02:40.000Z","updated_at":"2019-02-26T20:31:07.000Z","dependencies_parsed_at":"2023-06-19T14:28:34.914Z","dependency_job_id":null,"html_url":"https://github.com/bodgergely/nitro","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bodgergely%2Fnitro","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bodgergely%2Fnitro/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bodgergely%2Fnitro/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bodgergely%2Fnitro/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bodgergely","download_url":"https://codeload.github.com/bodgergely/nitro/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246413377,"owners_count":20773053,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["compression","compression-algorithm","compressor","encoder-decoder"],"created_at":"2024-08-01T05:01:03.152Z","updated_at":"2025-03-31T03:31:33.439Z","avatar_url":"https://github.com/bodgergely.png","language":"C++","funding_links":[],"categories":["C++"],"sub_categories":[],"readme":"# nitro - the blazingly fast compressor from the future\n\nImplemented in C++ nitro has the convenience of a standalone application but offers a library interface for its powerusers.\n\nUnparalleled speed/conversion ratio!\n\nnitro currently supports the simple 'block encoding' scheme.\n\n## Design decisions\n\nBlock encoding is simple to implement and offers good (but not the best by far) compression ratio if the\nnumber of unique symbols (8 bit value) is equal or less than 16. \n\n#### Algorithm:\n\t- Count the number of unique symbols (8 bit) values in the data\n\t- Use ceil(log2(count_unique_syms)) bits to represent each of those symbols\n\t- Build a symbol table which associates the symbols with their encodings (prepend to the output)\n\t- Iterate over the input and encode the symbols\n\nIt offers a rather straightforward way to 'index into' (in constant time) \nthe compressed representation to retrieve an original symbol.\nThis is possible because we are using identical block size (number of bits) to encode\nevery symbol thus making the index calculation simple.\n\nArithmetic encoding offers better compression ratios but more complicated to\nimplement and very quickly degrades if the number of unique characters are large.\nIt is the next candidate encoding scheme to be implemented in nitro.\n\nThe second BIG design constraint is the need to keep the whole input data in memory as nitro\ndoes the compression in memory without flushing to disk. This has the advantage of a rather \nsimplified and flexible interface for users since is not filestream bound.\n\n\n## Platform support\n- Linux/Windows\n\n\n## Install\n\n#### Build all\n\nRun build.sh to build everything. \n\n- shared library (libnitro.so)\n- test driver (testNitro)\n- nitro app (nitro)\n\n/bin and /lib will contain the binaries.\n\n#### Build separately\n\n- build_lib.sh\n- build_tests.sh\n- build_app.sh\n\n## Tests\n\nGoogle Test unit tests.\n\nAfter building the library and test driver you can run ./run_tests.sh\n\n## Requirements for building\n\n- Google Test for C++ (sudo apt-get install googletest)\n- gcc 7.3.0 or Visual Studio 2017 or any C++17 supporting compiler (C++14 should work too)\n- LD_LIBRARY_PATH should be set when using non system shared lib location.\n\n(You can export LD_LIBRARY_PATH into your environment too for convenience)\n\n## nitro library\n\n```\n#include \u003cnitro/nitro.h\u003e\n```\n\ngcc app.c -o app -lnitro\n \n\n## nitro app\n\nLD_LIBRARY_PATH=./lib ./bin/nitro -c genome.txt compressed.txt\n\n\n## Licence\nMIT\n\n## TODO\n\n- Set up cmake/premake/make and test on linux\n- Write Google Tests\n- Clean up includes\n- exception handling\n- split encoder and decoder up from .hpp to both .hpp and .cpp\n- Separation of client headers from implementation files\n- Clean up compiler warnings\n- Brute force memleak test - encoding/decoding to see if we have some memory leak\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbodgergely%2Fnitro","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbodgergely%2Fnitro","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbodgergely%2Fnitro/lists"}