{"id":30227650,"url":"https://github.com/hogwai/benchmark-java-collection-alternatives","last_synced_at":"2025-08-14T18:09:45.131Z","repository":{"id":308621644,"uuid":"1033432674","full_name":"Hogwai/benchmark-java-collection-alternatives","owner":"Hogwai","description":"Benchmarking alternatives to java collections (WIP)","archived":false,"fork":false,"pushed_at":"2025-08-06T23:22:28.000Z","size":17,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-07T00:09:05.649Z","etag":null,"topics":["agrona","fastutil","java-collections","jmh-benchmark"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Hogwai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-08-06T20:05:44.000Z","updated_at":"2025-08-06T23:22:32.000Z","dependencies_parsed_at":"2025-08-07T00:19:14.735Z","dependency_job_id":null,"html_url":"https://github.com/Hogwai/benchmark-java-collection-alternatives","commit_stats":null,"previous_names":["hogwai/benchmark-java-collection-alternatives"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/Hogwai/benchmark-java-collection-alternatives","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Hogwai%2Fbenchmark-java-collection-alternatives","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Hogwai%2Fbenchmark-java-collection-alternatives/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Hogwai%2Fbenchmark-java-collection-alternatives/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Hogwai%2Fbenchmark-java-collection-alternatives/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Hogwai","download_url":"https://codeload.github.com/Hogwai/benchmark-java-collection-alternatives/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Hogwai%2Fbenchmark-java-collection-alternatives/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270460454,"owners_count":24587635,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-14T02:00:10.309Z","response_time":75,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agrona","fastutil","java-collections","jmh-benchmark"],"created_at":"2025-08-14T18:09:42.007Z","updated_at":"2025-08-14T18:09:45.116Z","avatar_url":"https://github.com/Hogwai.png","language":"Java","readme":"# benchmark-java-collection-alternatives\nFrench version: [README_FR.md](README_fr.md)\n\n## Table of Contents\n\n- [Description](#description)\n- [Prerequisites](#prerequisites)\n- [Installation](#installation)\n- [Running the Benchmarks](#running-the-benchmarks)\n- [Results](#results)\n  - [fastutil](#fastutil)\n    - [HashSet vs IntOpenHashSet](#hashset-vs-intopenhashset)\n    - [ArrayList vs IntArrayList](#arraylist-vs-intarraylist)\n    - [HashMap vs Int2IntOpenHashMap](#hashmap-vs-int2intopenhashmap)\n  - [agrona](#agrona)\n    - [ArrayList vs Agrona IntArrayList](#arraylist-vs-agrona-intarraylist)\n    - [HashMap vs Agrona Int2IntHashMap](#hashmap-vs-agrona-int2inthashmap)\n  - [eclipse-collections](#eclipse-collections)\n    - [HashSet vs Eclipse IntHashSet](#hashset-vs-eclipse-inthashset)\n    - [ArrayList vs Eclipse IntArrayList](#arraylist-vs-eclipse-intarraylist)\n    - [HashMap vs MutableIntIntMap](#hashmap-vs-mutableintintmap)\n\n---\n## Description\nThis repository contains JMH (Java Microbenchmark Harness) benchmarks comparing the performance of standard Java collections from `java.util` with their optimized equivalents ([fastutil](https://github.com/vigna/fastutil), [agrona](https://github.com/aeron-io/agrona), [eclipse-collections](https://github.com/eclipse-collections/eclipse-collections)).\n\n## Prerequisites\n- Java 21\n- Maven 3.6.3 or higher (or use the wrapper)\n\n## Installation\n- Clone the repository:\n```bash\ngit clone https://github.com/Hogwai/benchmark-java-collection-alternatives.git\n```\n- Download the dependencies:\n```bash\nmvn clean install\n```\n\n## Running the Benchmarks\n### IntelliJ\nTo run the benchmarks directly from IntelliJ, you need to install the [JMH Java Microbenchmark Harness](https://plugins.jetbrains.com/plugin/7529-jmh-java-microbenchmark-harness) plugin.\n\n## Results\n\n### fastutil\nTimes are in microseconds (µs/op), measured with JMH (5 warm-up iterations, 5 measurement iterations, 2 forks).\n\n#### HashSet vs IntOpenHashSet\nThe tables below present the benchmark results comparing `java.util.HashSet\u003cInteger\u003e` and `it.unimi.dsi.fastutil.ints.IntOpenHashSet` for the `add` and `contains` operations.\n\n##### `add` Operation\n| Size     | java.util.HashSet (µs/op) | fastutil.IntOpenHashSet (µs/op) | Gain (ratio) |\n|----------|---------------------------|---------------------------------|----------------|\n| 100      | 1.750 ± 0.098             | 0.359 ± 0.021                   | ~4.87x         |\n| 1000     | 17.030 ± 0.211            | 3.180 ± 0.207                   | ~5.35x         |\n| 10,000   | 142.175 ± 7.993           | 52.269 ± 8.926                  | ~2.72x         |\n| 100,000  | 2000.463 ± 76.154         | 739.544 ± 30.210                | ~2.70x         |\n\n##### `contains` Operation\n| Size     | java.util.HashSet (µs/op) | fastutil.IntOpenHashSet (µs/op) | Gain (ratio) |\n|----------|---------------------------|---------------------------------|----------------|\n| 100      | 0.216 ± 0.020             | 0.104 ± 0.005                   | ~2.08x         |\n| 1000     | 2.870 ± 0.400             | 1.035 ± 0.065                   | ~2.77x         |\n| 10,000   | 28.112 ± 3.126            | 12.116 ± 0.740                  | ~2.32x         |\n| 100,000  | 290.259 ± 28.938          | 189.197 ± 10.495                | ~1.53x         |\n\n##### Observations\n- **add**: `IntOpenHashSet` is **2.7 to 5.3x faster**, with the maximum gain at 1000 elements.\n- **contains**: `IntOpenHashSet` is **1.5 to 2.8x faster**, with decreasing advantage for larger sizes.\n\n#### ArrayList vs IntArrayList\nThe tables below present the benchmark results comparing `java.util.ArrayList\u003cInteger\u003e` and `it.unimi.dsi.fastutil.ints.IntArrayList` for the `add` and `get` operations.\n\n##### `add` Operation\n| Size     | java.util.ArrayList (µs/op) | fastutil.IntArrayList (µs/op) | Gain (ratio) |\n|----------|-----------------------------|-------------------------------|----------------|\n| 100      | 0.474 ± 0.020               | 0.115 ± 0.011                 | ~4.12x         |\n| 1000     | 5.160 ± 0.543               | 1.148 ± 0.054                 | ~4.49x         |\n| 10,000   | 52.625 ± 2.441              | 12.416 ± 0.239                | ~4.24x         |\n| 100,000  | 563.262 ± 17.496            | 108.858 ± 4.085               | ~5.17x         |\n\n##### `get` Operation\n| Size     | java.util.ArrayList (µs/op) | fastutil.IntArrayList (µs/op) | Gain (ratio) |\n|----------|-----------------------------|-------------------------------|----------------|\n| 100      | 1.251 ± 0.098               | 1.222 ± 0.023                 | ~1.02x         |\n| 1000     | 12.424 ± 0.320              | 12.194 ± 0.326                | ~1.02x         |\n| 10,000   | 129.480 ± 5.467             | 129.017 ± 11.554              | ~1.00x         |\n| 100,000  | 1622.351 ± 96.042           | 1811.133 ± 407.451            | ~0.90x         |\n\n##### Observations\n- **add**: `IntArrayList` is **4 to 5x faster**, with the maximum gain at 100,000 elements.\n- **get**: Performance is **nearly identical** (~1.0-1.02x), with slight variability at large scale where `ArrayList` may be marginally faster.\n\n#### HashMap vs Int2IntOpenHashMap\nThe tables below present the benchmark results comparing `java.util.HashMap\u003cInteger, Integer\u003e` and `it.unimi.dsi.fastutil.ints.Int2IntOpenHashMap` for the `put` and `get` operations.\n\n##### `put` Operation\n| Size     | java.util.HashMap (µs/op) | fastutil.Int2IntOpenHashMap (µs/op) | Gain (ratio) |\n|----------|---------------------------|-------------------------------------|----------------|\n| 100      | 1.837 ± 0.126             | 0.482 ± 0.028                       | ~3.81x         |\n| 1000     | 19.235 ± 1.882            | 4.870 ± 0.206                       | ~3.95x         |\n| 10,000   | 174.106 ± 6.519           | 76.930 ± 13.478                     | ~2.26x         |\n| 100,000  | 2751.915 ± 375.027        | 1122.377 ± 112.861                  | ~2.45x         |\n\n##### `get` Operation\n| Size     | java.util.HashMap (µs/op) | fastutil.Int2IntOpenHashMap (µs/op) | Gain (ratio) |\n|----------|---------------------------|-------------------------------------|----------------|\n| 100      | 1.420 ± 0.059             | 2.095 ± 0.089                       | ~0.68x         |\n| 1000     | 19.008 ± 2.295            | 18.672 ± 0.557                      | ~1.02x         |\n| 10,000   | 266.739 ± 19.672          | 208.464 ± 9.795                     | ~1.28x         |\n| 100,000  | 6693.834 ± 1182.978       | 2468.793 ± 89.490                   | ~2.71x         |\n\n##### Observations\n- **put**: `Int2IntOpenHashMap` is **2.3 to 3.9x faster**, with the maximum gain at 1000 elements.\n- **get**: Mixed results; `HashMap` is faster for 100 elements (~1.47x), but `Int2IntOpenHashMap` becomes **up to 2.7x faster** at 100,000 elements.\n\n### agrona\n#### ArrayList vs Agrona IntArrayList\n\nThe tables below present the benchmark results comparing `java.util.ArrayList\u003cInteger\u003e` and `org.agrona.collections.IntArrayList` for the `add` and `get` operations.\n\n##### Operation `add`\n| Size     | java.util.ArrayList (µs/op) | agrona.IntArrayList (µs/op) | Gain (ratio) |\n|----------|-----------------------------|-----------------------------|----------------|\n| 100      | 0.506 ± 0.031               | 0.113 ± 0.010               | ~4.48x         |\n| 1000     | 5.126 ± 0.283               | 1.440 ± 0.332               | ~3.56x         |\n| 10,000   | 57.441 ± 5.968              | 14.603 ± 1.973              | ~3.93x         |\n| 100,000  | 584.322 ± 23.313            | 112.660 ± 5.952             | ~5.19x         |\n\n##### Operation `get`\n| Size     | java.util.ArrayList (µs/op) | agrona.IntArrayList (µs/op) | Gain (ratio) |\n|----------|-----------------------------|-----------------------------|----------------|\n| 100      | 1.259 ± 0.043               | 1.361 ± 0.030               | ~0.93x         |\n| 1000     | 12.540 ± 0.214              | 15.924 ± 2.070              | ~0.79x         |\n| 10,000   | 127.313 ± 3.796             | 143.565 ± 8.570             | ~0.89x         |\n| 100,000  | 1678.622 ± 147.345          | 1812.784 ± 69.776           | ~0.93x         |\n\n##### Observations\n- **add**: `IntArrayList` is **3.56 to 5.19x faster**, with the maximum gain at 100,000 elements.\n- **get**: `ArrayList` is slightly faster (~0.79 to 0.93x), but the differences are small and within the margins of error.\n\n#### HashMap vs Agrona Int2IntHashMap\nThe tables below present the benchmark results comparing `java.util.HashMap\u003cInteger, Integer\u003e` and `org.agrona.collections.Int2IntHashMap` for the `put` and `get` operations.\n\n##### Operation `put`\n| Size     | java.util.HashMap (µs/op) | agrona.Int2IntHashMap (µs/op) | Gain (ratio) |\n|----------|---------------------------|-------------------------------|----------------|\n| 100      | 1.888 ± 0.145             | 1.526 ± 0.103                 | ~1.24x         |\n| 1000     | 21.514 ± 2.826            | 16.841 ± 1.456                | ~1.28x         |\n| 10,000   | 192.646 ± 19.421          | 135.527 ± 12.900              | ~1.42x         |\n| 100,000  | 3041.915 ± 1165.324       | 4571.294 ± 516.363            | ~0.67x         |\n\n##### Operation `get`\n| Size     | java.util.HashMap (µs/op) | agrona.Int2IntHashMap (µs/op) | Gain (ratio) |\n|----------|---------------------------|-------------------------------|----------------|\n| 100      | 1.581 ± 0.115             | 2.077 ± 0.070                 | ~0.76x         |\n| 1000     | 20.204 ± 1.313            | 25.864 ± 2.908                | ~0.78x         |\n| 10,000   | 306.371 ± 36.816          | 611.495 ± 869.572             | ~0.50x         |\n| 100,000  | 7610.090 ± 2464.821       | 4157.502 ± 644.385            | ~1.83x         |\n\n##### Observations\n- **put**: `Int2IntHashMap` is **1.24 to 1.42x faster** for sizes 100 to 10,000, but **~1.5x slower** at 100,000 elements, with high variability.\n- **get**: `HashMap` is faster (~1.3-2x) for 100 to 10,000 elements, but `Int2IntHashMap` is **~1,83x faster** for 100 000 elements.\n\n### eclipse-collections\n#### HashSet vs Eclipse IntHashSet\nThe tables below present the benchmark results comparing `java.util.HashSet\u003cInteger\u003e` and `org.eclipse.collections.impl.set.mutable.primitive.IntHashSet` for the `add` and `contains` operations.\n\n##### Operation `add`\n| Size     | java.util.HashSet (µs/op) | Eclipse IntHashSet (µs/op) | Gain (ratio)  |\n|----------|---------------------------|----------------------------|----------------|\n| 100      | 1,823 ± 0,190             | 0,494 ± 0,160              | ~3.69x         |\n| 1000     | 23,480 ± 6,174            | 8,385 ± 2,312              | ~2.80x         |\n| 10,000   | 231,990 ± 91,308          | 95,981 ± 4,390             | ~2.42x         |\n| 100,000  | 4253,353 ± 5232,318       | 1605,101 ± 71,214          | ~2.65x         |\n\n##### Operation `contains`\n| Size     | java.util.HashSet (µs/op) | Eclipse IntHashSet (µs/op) | Gain (ratio)  |\n|----------|---------------------------|----------------------------|----------------|\n| 100      | 0,494 ± 0,057             | 0,346 ± 0,167              | ~1.43x         |\n| 1000     | 5,620 ± 0,420             | 8,005 ± 2,673              | ~0.70x         |\n| 10,000   | 66,996 ± 21,513           | 65,854 ± 5,298             | ~1.02x         |\n| 100,000  | 1316,742 ± 230,694        | 1195,077 ± 44,682          | ~1.10x         |\n\n##### Observations\n- **add**: `IntHashSet` is **2.42 to 3.69x faster**, with the maximum gain at 100 elements. Performance remains strong at scale despite high variability for `HashSet` at 100,000.\n- **contains**: Mixed results; `IntHashSet` is faster (~1.43x) at 100 elements but slower (~0.70x) at 1,000. Performance is nearly identical at 10,000 and 100,000 (~1.02-1.10x).\n\n### ArrayList vs Eclipse IntArrayList\nThe tables below present the benchmark results comparing `java.util.ArrayList\u003cInteger\u003e` and `org.eclipse.collections.impl.list.mutable.primitive.IntArrayList` for the `add` and `get` operations.\n\n##### Operation `add`\n| Size     | java.util.ArrayList (µs/op) | Eclipse IntArrayList (µs/op) | Gain (ratio)  |\n|----------|-----------------------------|------------------------------|----------------|\n| 100      | 0,501 ± 0,030               | 0,121 ± 0,012                | ~4.14x         |\n| 1000     | 5,288 ± 0,448               | 1,325 ± 0,102                | ~3.99x         |\n| 10,000   | 57,737 ± 4,016              | 14,139 ± 0,655               | ~4.08x         |\n| 100,000  | 587,736 ± 65,289            | 181,133 ± 155,147            | ~3.24x         |\n\n##### Operation `get`\n| Size     | java.util.ArrayList (µs/op) | Eclipse IntArrayList (µs/op) | Gain (ratio)  |\n|----------|-----------------------------|------------------------------|----------------|\n| 100      | 1,326 ± 0,057               | 1,356 ± 0,116                | ~0.98x         |\n| 1000     | 12,806 ± 0,305              | 13,208 ± 0,600               | ~0.97x         |\n| 10,000   | 136,481 ± 14,400            | 139,395 ± 7,133              | ~0.98x         |\n| 100,000  | 1692,195 ± 40,706           | 2055,078 ± 1042,478          | ~0.82x         |\n\n##### Observations\n- **add**: `IntArrayList` is **3.24 to 4.14x faster**, with a consistent gain for small and medium sizes, but slightly reduced at 100,000 elements due to high variability.\n- **get**: Performance is **almost identical** (~0.97-0.98x) for sizes from 100 to 10,000, with `ArrayList` being slightly faster (~1.22x) at 100,000.\n\n#### HashMap vs MutableIntIntMap\nThe tables below present the benchmark results comparing `java.util.HashMap\u003cInteger, Integer\u003e` and `org.eclipse.collections.impl.map.mutable.primitive.IntIntHashMap` for the `put` and `get` operations.\n\n##### Operation `put`\n| Size     | java.util.HashMap (µs/op) | Eclipse IntIntHashMap (µs/op) | Gain (ratio)  |\n|----------|---------------------------|-------------------------------|----------------|\n| 100      | 2,077 ± 0,319             | 0,626 ± 0,038                 | ~3.32x         |\n| 1000     | 21,543 ± 1,554            | 6,531 ± 0,574                 | ~3.30x         |\n| 10,000   | 185,059 ± 13,089          | 65,336 ± 6,458                | ~2.83x         |\n| 100,000  | 2657,581 ± 198,832        | 649,039 ± 100,905             | ~4.09x         |\n\n##### Operation `get`\n| Size     | java.util.HashMap (µs/op) | Eclipse IntIntHashMap (µs/op) | Gain (ratio)  |\n|----------|---------------------------|-------------------------------|----------------|\n| 100      | 1,768 ± 0,360             | 1,505 ± 0,140                 | ~1.17x         |\n| 1000     | 20,620 ± 1,656            | 14,014 ± 0,273                | ~1.47x         |\n| 10,000   | 312,758 ± 58,560          | 148,900 ± 3,901               | ~2.10x         |\n| 100,000  | 7274,169 ± 1086,437       | 2318,541 ± 919,592            | ~3.14x         |\n\n##### Observations\n- **put**: `IntIntHashMap` is **2.83 to 4.09x faster**, with increasing gains at scale (~4.09x at 100,000).\n- **get**: `IntIntHashMap` is **1.17 to 3.14x faster**, with a significant advantage at 100,000 elements (~3.14x).","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhogwai%2Fbenchmark-java-collection-alternatives","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhogwai%2Fbenchmark-java-collection-alternatives","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhogwai%2Fbenchmark-java-collection-alternatives/lists"}