{"id":13791182,"url":"https://github.com/dgraph-io/sroar","last_synced_at":"2025-05-12T09:34:10.866Z","repository":{"id":39685864,"uuid":"344952139","full_name":"dgraph-io/sroar","owner":"dgraph-io","description":"Serialized Roaring Bitmaps","archived":true,"fork":false,"pushed_at":"2023-03-29T06:52:40.000Z","size":210,"stargazers_count":274,"open_issues_count":0,"forks_count":46,"subscribers_count":28,"default_branch":"main","last_synced_at":"2024-11-18T05:38:42.336Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dgraph-io.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2021-03-05T22:40:00.000Z","updated_at":"2024-09-16T11:03:03.000Z","dependencies_parsed_at":"2024-04-02T12:54:53.597Z","dependency_job_id":"3c37fe96-b868-4663-801b-8e0616477392","html_url":"https://github.com/dgraph-io/sroar","commit_stats":null,"previous_names":["dgraph-io/roar"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dgraph-io%2Fsroar","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dgraph-io%2Fsroar/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dgraph-io%2Fsroar/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dgraph-io%2Fsroar/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dgraph-io","download_url":"https://codeload.github.com/dgraph-io/sroar/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253709564,"owners_count":21951184,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-03T22:00:56.995Z","updated_at":"2025-05-12T09:34:10.569Z","avatar_url":"https://github.com/dgraph-io.png","language":"Go","funding_links":[],"categories":["0x10. dgraph-io/sroar"],"sub_categories":[],"readme":"# sroar: Serialized Roaring Bitmaps\n\nsroar is a re-written version of Roaring Bitmaps in Go, with the aim to have\nequality between in-memory representation and on-disk representation. An\nsroar.Bitmap does not need to be marshalled or unmarshalled, as the underlying\nrepresetation is a byte slice. Therefore, it can be written to disk, brought to\nmemory, or shipped over the network immediately. This is needed in [Dgraph][], where\nwe need to deal with lots of bitmaps.\n\nsroar only implements array and bitmap containers. It does NOT implement run\ncontainers, which is an optimization that RoaringBitmaps has. Despite that, it\noutperforms RoaringBitmaps as shown in the Benchmarks section.\n\n[Dgraph]: https://github.com/dgraph-io/dgraph\n[Roaring]: https://github.com/RoaringBitmap/roaring\n\nThe code borrows concepts and code from [RoaringBitmaps][Roaring].\n\n## Benchmarks\n\nThe benchmarks were run:\n- Using real data set as described in [RoaringBitmaps][Roaring].\n- Only on the 64-bit version of roaring bitmaps (roaring64).\n- Only on `FastOr`, which is the more expensive operation than `And` or\n    equivalent.\n- On AMD Ryzen Threadripper 2950X 16-Core Processor.\n- Using Go benchmarks serially.\n\nBased on the benchmarks, sroar is:\n- 6.5x faster (-85% p50) for benchmarks \u003e1ms, uses\n- 15x (-93.5% p50) less memory for allocations \u003e1MB.\n- 25x fewer allocations.\n\nThe benchmark command and the results are:\n\n```\n$ go test -bench BenchmarkRealDataFastOr --run=XXX --count=5 --benchmem\n\nname CPU                                    old time/op    new time/op    delta\nRealDataFastOr/census1881-32                 302ms ± 2%       2ms ± 3%   -99.29%  (p=0.008 n=5+5)\nRealDataFastOr/wikileaks-noquotes-32        76.5ms ± 1%     0.9ms ± 1%   -98.83%  (p=0.008 n=5+5)\nRealDataFastOr/wikileaks-noquotes_srt-32    34.8ms ± 5%     1.0ms ± 2%   -97.07%  (p=0.008 n=5+5)\nRealDataFastOr/dimension_033-32             55.0ms ± 3%     2.7ms ± 0%   -95.16%  (p=0.016 n=5+4)\nRealDataFastOr/census1881_srt-32            36.8ms ± 3%     2.9ms ± 1%   -92.13%  (p=0.008 n=5+5)\nRealDataFastOr/dimension_003-32             50.4ms ± 1%    11.6ms ± 4%   -77.06%  (p=0.008 n=5+5)\nRealDataFastOr/dimension_008-32             10.0ms ± 2%     3.7ms ± 2%   -62.69%  (p=0.008 n=5+5)\nRealDataFastOr/weather_sept_85_srt-32       6.13ms ± 3%    2.72ms ± 2%   -55.66%  (p=0.008 n=5+5)\nRealDataFastOr/census-income-32             1.70ms ± 3%    1.05ms ± 1%   -38.53%  (p=0.008 n=5+5)\nRealDataFastOr/weather_sept_85-32           2.28ms ± 2%    4.07ms ± 2%   +78.52%  (p=0.008 n=5+5)\n\nRealDataFastOr/uscensus2000-32               556µs ± 2%     791µs ± 1%   +42.17%  (p=0.008 n=5+5)\nRealDataFastOr/census-income_srt-32          260µs ± 4%     986µs ± 2%  +279.09%  (p=0.008 n=5+5)\n\nname MEM_BYTES                             old alloc/op   new alloc/op   delta\nRealDataFastOr/census1881-32                 585MB ± 0%       1MB ± 0%   -99.75%  (p=0.008 n=5+5)\nRealDataFastOr/wikileaks-noquotes-32        76.3MB ± 0%     0.6MB ± 0%   -99.24%  (p=0.008 n=5+5)\nRealDataFastOr/wikileaks-noquotes_srt-32    22.8MB ± 0%     0.6MB ± 0%   -97.46%  (p=0.008 n=5+5)\nRealDataFastOr/census1881_srt-32            15.3MB ± 0%     1.4MB ± 0%   -90.58%  (p=0.008 n=5+5)\nRealDataFastOr/dimension_003-32             7.78MB ± 0%    1.44MB ± 0%   -81.49%  (p=0.008 n=5+5)\nRealDataFastOr/dimension_033-32             1.10MB ± 0%    1.44MB ± 0%   +30.92%  (p=0.008 n=5+5)\n\nRealDataFastOr/dimension_008-32              537kB ± 0%      97kB ± 0%   -81.94%  (p=0.008 n=5+5)\nRealDataFastOr/census-income-32              187kB ± 0%      70kB ± 0%   -62.86%  (p=0.008 n=5+5)\nRealDataFastOr/census-income_srt-32         99.1kB ± 0%    69.6kB ± 0%   -29.81%  (p=0.008 n=5+5)\nRealDataFastOr/weather_sept_85_srt-32        375kB ± 0%     292kB ± 0%   -21.95%  (p=0.008 n=5+5)\nRealDataFastOr/uscensus2000-32               169kB ± 0%     231kB ± 0%   +36.97%  (p=0.008 n=5+5)\nRealDataFastOr/weather_sept_85-32            169kB ± 0%     292kB ± 0%   +72.93%  (p=0.008 n=5+5)\n\nname MEM_ALLOCS                           old allocs/op  new allocs/op  delta\nRealDataFastOr/census1881_srt-32             29.7k ± 0%      0.0k ± 0%   -99.91%  (p=0.008 n=5+5)\nRealDataFastOr/wikileaks-noquotes_srt-32     6.06k ± 0%     0.02k ± 0%   -99.74%  (p=0.008 n=5+5)\nRealDataFastOr/dimension_003-32              4.57k ± 0%     0.03k ± 2%   -99.42%  (p=0.008 n=5+5)\nRealDataFastOr/dimension_033-32              4.33k ± 0%     0.03k ± 0%   -99.38%  (p=0.000 n=5+4)\nRealDataFastOr/uscensus2000-32               1.75k ± 0%     0.06k ± 0%   -96.85%  (p=0.008 n=5+5)\nRealDataFastOr/dimension_008-32                704 ± 0%        23 ± 3%   -96.79%  (p=0.008 n=5+5)\nRealDataFastOr/census-income-32                271 ± 0%         9 ± 0%   -96.68%  (p=0.008 n=5+5)\nRealDataFastOr/weather_sept_85_srt-32          248 ± 0%        14 ± 0%   -94.35%  (p=0.008 n=5+5)\nRealDataFastOr/weather_sept_85-32             81.0 ± 0%      14.0 ± 0%   -82.72%  (p=0.008 n=5+5)\nRealDataFastOr/census-income_srt-32           40.0 ± 0%       9.0 ± 0%   -77.50%  (p=0.008 n=5+5)\nRealDataFastOr/census1881-32                 54.5k ± 0%      0.0k ± 0%      ~     (p=0.079 n=4+5)\nRealDataFastOr/wikileaks-noquotes-32         39.2k ± 0%      0.0k ± 0%      ~     (p=0.079 n=4+5)\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdgraph-io%2Fsroar","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdgraph-io%2Fsroar","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdgraph-io%2Fsroar/lists"}