{"id":19768690,"url":"https://github.com/siara-cc/firestorecompress","last_synced_at":"2025-02-28T04:22:41.046Z","repository":{"id":97919172,"uuid":"448092724","full_name":"siara-cc/FirestoreCompress","owner":"siara-cc","description":"Store compressed text in Firestore","archived":false,"fork":false,"pushed_at":"2022-01-14T20:18:15.000Z","size":9,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-01-11T00:33:24.244Z","etag":null,"topics":["compression","database-compression","firebase","firestore","firestore-database","javascript-library","string-compression","string-compression-algorithms","text-compression","unicode","utf-8"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/siara-cc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-01-14T20:02:36.000Z","updated_at":"2022-07-13T23:05:08.000Z","dependencies_parsed_at":"2023-03-13T16:08:21.725Z","dependency_job_id":null,"html_url":"https://github.com/siara-cc/FirestoreCompress","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/siara-cc%2FFirestoreCompress","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/siara-cc%2FFirestoreCompress/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/siara-cc%2FFirestoreCompress/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/siara-cc%2FFirestoreCompress/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/siara-cc","download_url":"https://codeload.github.com/siara-cc/FirestoreCompress/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241100674,"owners_count":19909762,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["compression","database-compression","firebase","firestore","firestore-database","javascript-library","string-compression","string-compression-algorithms","text-compression","unicode","utf-8"],"created_at":"2024-11-12T04:39:52.201Z","updated_at":"2025-02-28T04:22:41.040Z","avatar_url":"https://github.com/siara-cc.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Store compressed text in Firestore databases\n\nUnishox text compression can be used to store more data than the 1GB free limit in Firestore database by storing text content as a Blob.  For example, \n\n```Javascript\nvar usx = require(\"./unishox2.js\")\nvar my_str = \"The quick brown fox jumped over the lazy dog\";\nvar out_buf = new Uint8Array(100); // A buffer with arbitrary length\nvar out_len = usx.unishox2_compress_simple(my_str, my_str.length, out_buf);\nfirebase.firestore().collection(\"/my-collection\").add({\n  myCompressedBlob: firebase.firestore.Blob.fromUint8Array(out_buf.slice(0, out_len))\n});\n```\n\nThis library can be installed from npm package repository using `npm i unishox2.siara.cc`.  The source code can be found at https://github.com/siara-cc/unishox_js.\n\nSince the size of compressed content will be smaller than the input text, more than 1GB data can be compressed and stored into Firestore.  The compression ratio will depend on the type of text that is being compressed.\n\nUnishox can get good compression for any type of UTF-8 text compositions such as English sentences, Latin and other languages.  The picture shown below gives an idea of how much compression can be achieved for each language:\n\n![Promo Picture](https://github.com/siara-cc/unishox_js/blob/master/demo/Banner1.png?raw=true)\n\n# How it works\n\nUnishox is an hybrid encoder (entropy, dictionary and delta coding).  It works by assigning fixed prefix-free codes for each letter in the above Character Set (entropy coding).  It also encodes repeating letter sets separately (dictionary coding).  For Unicode characters, delta coding is used.\n\nThe model used for arriving at the prefix-free code is shown below:\n\n![Promo video](https://github.com/siara-cc/Unishox2/blob/master/promo/model.png?raw=true)\n\nThe complete specification can be found in this article: [Unishox 2 - Guaranteed Configurable Compression for Short Strings using Entropy, Dictionary and Delta encoding techniques](https://github.com/siara-cc/Unishox2/blob/master/Unishox_Article_2.pdf?raw=true).\n\n## Using it in your application\n\nTo compress and decompress strings in your application, import `unishox2.js`:\n\n```Javascript\nvar usx = require(\"./unishox2.js\");\n```\n\nThe function expects a Javascript `string` or `Uint8Array` as Input and Output an `Uint8Array`.\n\n### Simple API\n\nIn its simplest form, just pass the string you want to compress, its length and a UintArray to receive the compressed contents.  While the library can make the output array, it is faster to supply it.  Since this technology guarantees compression, the output buffer needs be only as lengthy as the input string 99.99% of times:\n\n```Javascript\nvar usx = require(\"./unishox2.js\")\nvar my_str = \"The quick brown fox jumped over the lazy dog\";\nvar out_buf = new Uint8Array(100); // A buffer with arbitrary length\nvar out_len = usx.unishox2_compress_simple(my_str, my_str.length, out_buf);\nvar out_str = usx.unishox2_decompress_simple(out_buf, out_len);\nconsole.log(out_str);\n```\n\nAs shown above, the original string can be obtained by passing the UintArray and length to `unishox2_decompress_simple`.\n\n# Issues\n\nIn case of any issues, please email the Author (Arundale Ramanathan) at arun@siara.cc or create GitHub issue.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsiara-cc%2Ffirestorecompress","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsiara-cc%2Ffirestorecompress","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsiara-cc%2Ffirestorecompress/lists"}