{"id":16444170,"url":"https://github.com/douglascdev/weird-sort-and-compress","last_synced_at":"2026-06-12T08:32:58.415Z","repository":{"id":134791752,"uuid":"444212148","full_name":"douglascdev/weird-sort-and-compress","owner":"douglascdev","description":"Weird Sort-and-Compress Thing","archived":false,"fork":false,"pushed_at":"2022-01-03T23:59:30.000Z","size":8,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-11-20T08:02:53.823Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/douglascdev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-01-03T22:04:52.000Z","updated_at":"2022-01-03T23:59:33.000Z","dependencies_parsed_at":"2023-05-30T00:15:49.040Z","dependency_job_id":null,"html_url":"https://github.com/douglascdev/weird-sort-and-compress","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/douglascdev/weird-sort-and-compress","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/douglascdev%2Fweird-sort-and-compress","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/douglascdev%2Fweird-sort-and-compress/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/douglascdev%2Fweird-sort-and-compress/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/douglascdev%2Fweird-sort-and-compress/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/douglascdev","download_url":"https://codeload.github.com/douglascdev/weird-sort-and-compress/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/douglascdev%2Fweird-sort-and-compress/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34236551,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-12T02:00:06.859Z","response_time":109,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-11T09:23:29.301Z","updated_at":"2026-06-12T08:32:58.397Z","avatar_url":"https://github.com/douglascdev.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Weird Sort-and-Compress Thing\nA weird integer sorting + compression algorithm inspired by a conversation with [Luthingx]( https://github.com/Luthingx ) (it probably already exists by some name I don't know yet). There's a lot still to improve about this algorithm, so be careful where you use it.\n\n\n## How it works\nHere's an example for the following list:\n\n    l = [1, 2, 2, 2, 3]\n\nThe algorithm starts with counting sort, creating a dictionary with each unique number as key and the number of occurences of it in the list as the value:\n\n    d = {1: 1, 2: 3, 3: 1}\n\nTo decrease the space needed to store the numbers in memory, we'll only store the first number and then the difference between each of the next numbers and the previous one:\n\n    d2 = [(1, 1), (1, 3), (1, 1))\n    \nNow, the minimum amount of memory we need to store every key that's in d2 is 1 bit, since 1 is the maximum difference between any subsequent elements. The same applies to the values, except that to store any value here we need 2 bits of memory, since the maximum value is 3(11 in binary).\nSo we know that we can store this list as a sequence of 3 bits elements, like this:\n\n    d2_bin = [\"101\", \"111\", 101\"]\n    \nWe can now return the list as a single number, along with a pair of integers containing the number of bits in each key and the number of bits in each value, allowing the value to be decompressed.\n\n## Memory efficiency\n\nHere's a list with the sum of the number of bits of all numbers in a list with 100 elements, generated with random values in the range 0 to 50 and generated 20 times, vs. the number of bits in the resulting compressed integer(taking as a premise that all numbers in the array are all actually stored in continuous memory, including duplicates):\n\n    467 =\u003e 208\n    486 =\u003e 230\n    490 =\u003e 221\n    491 =\u003e 216\n    493 =\u003e 222\n    491 =\u003e 221\n    494 =\u003e 230\n    485 =\u003e 235\n    494 =\u003e 217\n    490 =\u003e 252\n    461 =\u003e 265\n    476 =\u003e 241\n    492 =\u003e 247\n    487 =\u003e 230\n    474 =\u003e 246\n    460 =\u003e 222\n    484 =\u003e 216\n    486 =\u003e 203\n    484 =\u003e 222\n    485 =\u003e 231\n    \nAnd 1000 numbers from 0 to 50, also 20 times:\n\n    4724 =\u003e 358\n    4827 =\u003e 309\n    4818 =\u003e 308\n    4801 =\u003e 309\n    4763 =\u003e 309\n    4763 =\u003e 309\n    4801 =\u003e 359\n    4757 =\u003e 359\n    4766 =\u003e 309\n    4794 =\u003e 309\n    4769 =\u003e 309\n    4789 =\u003e 359\n    4887 =\u003e 359\n    4787 =\u003e 309\n    4761 =\u003e 309\n    4749 =\u003e 309\n    4844 =\u003e 308\n    4798 =\u003e 359\n    4799 =\u003e 308\n    4763 =\u003e 359\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdouglascdev%2Fweird-sort-and-compress","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdouglascdev%2Fweird-sort-and-compress","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdouglascdev%2Fweird-sort-and-compress/lists"}