{"id":16782583,"url":"https://github.com/davidmoten/hilbert-curve","last_synced_at":"2025-04-08T04:13:30.576Z","repository":{"id":17861328,"uuid":"82870137","full_name":"davidmoten/hilbert-curve","owner":"davidmoten","description":"Java utilities for transforming distance along N-dimensional Hilbert Curve to a point and back. Also supports range splitting queries on the Hilbert Curve.","archived":false,"fork":false,"pushed_at":"2024-10-22T21:39:49.000Z","size":3009,"stargazers_count":111,"open_issues_count":5,"forks_count":21,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-10-23T18:01:55.236Z","etag":null,"topics":["hilbert-curve","java","space-filling-curves","spatial-index"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/davidmoten.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-02-23T01:26:21.000Z","updated_at":"2024-10-22T21:39:46.000Z","dependencies_parsed_at":"2023-01-16T20:17:19.592Z","dependency_job_id":"0ed84606-13d5-4cb6-87e3-e0f4ea21330e","html_url":"https://github.com/davidmoten/hilbert-curve","commit_stats":null,"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidmoten%2Fhilbert-curve","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidmoten%2Fhilbert-curve/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidmoten%2Fhilbert-curve/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidmoten%2Fhilbert-curve/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/davidmoten","download_url":"https://codeload.github.com/davidmoten/hilbert-curve/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247773719,"owners_count":20993639,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hilbert-curve","java","space-filling-curves","spatial-index"],"created_at":"2024-10-13T07:46:14.474Z","updated_at":"2025-04-08T04:13:30.553Z","avatar_url":"https://github.com/davidmoten.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# hilbert-curve\n\u003ca href=\"https://github.com/davidmoten/hilbert-curve/actions/workflows/ci.yml\"\u003e\u003cimg src=\"https://github.com/davidmoten/hilbert-curve/actions/workflows/ci.yml/badge.svg\"/\u003e\u003c/a\u003e\u003cbr/\u003e\n[![Maven Central](https://maven-badges.herokuapp.com/maven-central/com.github.davidmoten/hilbert-curve/badge.svg?style=flat)](https://maven-badges.herokuapp.com/maven-central/com.github.davidmoten/hilbert-curve)\u003cbr/\u003e\n[![codecov](https://codecov.io/gh/davidmoten/hilbert-curve/branch/master/graph/badge.svg)](https://codecov.io/gh/davidmoten/hilbert-curve)\u003cbr/\u003e\n\nJava utilities for \n\n* transforming distance along an N-dimensional Hilbert Curve to a point and back.\n* bounding box (N-dimensional) query support (bounding box is mapped to a number of intervals on the hilbert index for single column lookup)\n\nFeatures\n\n* supports multiple dimensions\n* method chaining\n* renders in 2-dimensions\n* benchmarked with `jmh`\n* [efficiently finds Hilbert curve index ranges for bounding box queries](#querying-n-dimensional-space)\n\nStatus: *deployed to Maven Central*\n\nMaven [reports](https://davidmoten.github.io/hilbert-curve/index.html) including [javadocs](https://davidmoten.github.io/hilbert-curve/apidocs/index.html)\n\nSee also [sparse-hilbert-index](https://github.com/davidmoten/sparse-hilbert-index).\n\nBackground\n-------------\nA [Hilbert curve](https://en.wikipedia.org/wiki/Hilbert_curve) is a continuous fractal space-filling curve first described by David Hilbert in 1891.\n\nThis library supports *approximations* to the Hilbert curve. *H\u003csub\u003en\u003c/sub\u003e* is the n-th approximation to the Hilbert curve and is a path of 2\u003csup\u003en\u003c/sup\u003e-1 straight line segments of length 1.\n\nA Hilbert curve can be used to index multiple dimensions and has useful locality properties. In short, \n\n* *Points with indexes close to an index will be close to the point corresponding to that index.*\n\n**Figure 1. 2D Hilbert curves with 1 to 6 bits (H\u003csub\u003e1\u003c/sub\u003e to H\u003csub\u003e6\u003c/sub\u003e)**\n\n| | |\n| --- | --- |\n|  \u003cimg src=\"src/docs/hilbert-2d-bits-1.png?raw=true\" /\u003e |  \u003cimg src=\"src/docs/hilbert-2d-bits-2.png?raw=true\" /\u003e\n|  \u003cimg src=\"src/docs/hilbert-2d-bits-3.png?raw=true\" /\u003e |  \u003cimg src=\"src/docs/hilbert-2d-bits-4.png?raw=true\" /\u003e\n|  \u003cimg src=\"src/docs/hilbert-2d-bits-5.png?raw=true\" /\u003e |  \u003cimg src=\"src/docs/hilbert-2d-bits-6.png?raw=true\" /\u003e\n\n```java\nHilbertCurveRenderer.renderToFile(bits, 200, filename);\n```\n\n**Figure 2. 2D Hilbert curves with 1 to 4 bits colorized and labelled**\n\n| | |\n| --- | --- |\n|  \u003cimg src=\"src/docs/hilbert-color-1.png?raw=true\" /\u003e |  \u003cimg src=\"src/docs/hilbert-color-2.png?raw=true\" /\u003e\n|  \u003cimg src=\"src/docs/hilbert-color-3.png?raw=true\" /\u003e |  \u003cimg src=\"src/docs/hilbert-color-4.png?raw=true\" /\u003e\n\n```java\nHilbertCurveRenderer.renderToFile(bits, 600, filename, Option.COLORIZE, Option.LABEL);\n```\n\n**Figure 3. 2D Hilbert curves with 5 bits colorized and labelled**\n\n\u003cimg src=\"src/docs/hilbert-color-5.png?raw=true\" /\u003e\n\n\n```java\nHilbertCurveRenderer.renderToFile(5, 1000, filename, Option.COLORIZE, Option.LABEL);\n```\n\nGetting started\n-----------------\nAdd this to your maven pom.xml:\n\n```xml\n\u003cdependency\u003e\n    \u003cgroupId\u003ecom.github.davidmoten\u003c/groupId\u003e\n    \u003cartifactId\u003ehilbert-curve\u003c/artifactId\u003e\n    \u003cversion\u003eVERSION_HERE\u003c/version\u003e\n\u003c/dependency\u003e\n``` \n\nUsage\n---------\n### Limits\nMaximum bits is 63 (index ranges from 0 to ~10\u003csup\u003e19\u003c/sup\u003e), max dimensions is not practically limited (sure you can specify 10\u003csup\u003e9\u003c/sup\u003e dimensions but that's a lot of data per point!).\n\n### Small\n\nThe maximum index on the Hilbert curve is 2\u003csup\u003ebits * dimensions\u003c/sup\u003e - 1. If your \n`bits * dimensions` is \u003c= 63 then you can increase performance and reduce allocations by using the \u003cb\u003esmall\u003c/b\u003e option which uses `long` values for indexes rather than `BigInteger` values. \nJMH benchmarks show up to 30% better throughput using `small`. \n\n### Points\nThe hilbert curve wiggles around your n-dimensional grid happily visiting each cell. The ordinates in each dimension are integers in the range 0 .. 2\u003csup\u003ebits\u003c/sup\u003e-1.\n \n### Index from point\n\nGet the index (distance along the curve in integer units) for a 2-dimensional point:\n\n```java\nHilbertCurve c = \n    HilbertCurve.bits(5).dimensions(2);\nBigInteger index = c.index(3, 4);\n```\n\nSmall option:\n\n```java\nSmallHilbertCurve c = \n    HilbertCurve.small().bits(5).dimensions(2);\n//returns long rather than BigInteger\nlong index = c.index(3, 4);\n\n```\n\n### Point from index\n\nGet the point corresponding to a particular index along the curve:\n\n```java\nHilbertCurve c = \n    HilbertCurve.bits(5).dimensions(2);\nlong[] point = c.point(22);\n//or\nlong[] point = c.point(BigInteger.valueOf(22));\n```\n\nSmall option:\n\n```java\nSmallHilbertCurve c = \n    HilbertCurve.small().bits(5).dimensions(2);\nlong[] point = c.point(22);\n```\n\nYou can save allocations if you use this method (available on `HilbertCurve` and `SmallHilbertCurve`) to calculate a point:\n\n```java\nlong[] x = new long[dimensions];\nc.point(index, x);\n```\n\nBenchmarks indicate that throughput is increased about 25% using this method with the `small()` option. \n\n### Render a curve\n\nTo render a curve (for 2 dimensions only) to a PNG of 800x800 pixels:\n\n```java\nHilbertCurveRenderer.renderToFile(bits, 800, \"target/image.png\");\n```\n\n### Querying N-dimensional space\nThis is one of the very useful applications of the Hilbert curve. By mapping n-dimensional space onto 1 dimension we enable the use of range lookups on that 1 dimension using a B-tree or binary search. A search region represented as a box in n-dimensions can be mapped to a series of ranges on the hilbert curve. \n\nSee [sparse-hilbert-index](https://github.com/davidmoten/sparse-hilbert-index) for a library that performs hilbert index queries on n-dimensional data (that can be stored in AWS S3 for instance).\n\n#### Boundary algorithm for finding ranges\nGiven an n-dimensional search region **the exact hilbert curve ranges that cover the search region can be determined just by looking at the hilbert curve values on the perimeter (boundary) of the region**. *I didn't manage to find any discussion of this very useful fact on the web and am confused as to why this isn't covered in an accessible way somewhere. Surely this is not an unpublished discovery (because it's simple to show and seems obvious in retrospect). Let me know if you find discussion of this technique elsewhere!*\n\nLet's prove this statement:\n\n##### Boundary algorithm proof\n\n**Lemma 1:** *The points corresponding to 0 on the hilbert curve and the maximum on the hilbert curve are vertices of the domain*. \n\n*Proof*: For two dimensions, note that the order 2 Hilbert curve has this property (this curve is just a U) and the known production rules for generating the Hilbert curve of order `n+1` from the curve of order `n` maintain the start and end positions of the curve on vertices. By induction the statement is true for 2 dimensions. The proof is similar for dimensions \u003e 2.\n\n**Lemma 2:** *Given the exact covering ranges of the Hilbert curve over a search region, the endpoints of those ranges must be on the boundary of the search region.*\n\n*Proof*: We proceed with a proof by contradiction. Suppose an endpoint of one of the covering ranges was not on the boundary of the search region. Remember that the hilbert curve is a single continuous line visiting every point in the region. If the range endpoint in question is inside the search region then it must be the termination of the Hilbert curve but by Lemma 1 the Hilbert curve termination points are at the vertices of the domain which cannot be wholly inside the search region. \n\n##### Boundary algorithm description\n\nHaving proved the relationship of the boundary to the ranges we can create an algorithm for extracting the exact ranges. The hilbert curve values of the perimeter (an `n-1` dimensional surface) of a search box are calculated and put in a sorted list L. Then the values in L are paired with each other into ranges (and concatenated if they are adjacent) starting with the lowest value in L and checking if the next hop along the Hilbert curve in increasing value is on the perimeter, in the box or on the outside of the box. If the next value is outside the search box then we close the current range. If the value is on the perimeter then we add that value to the range and close off the range. If the value is strictly inside the search box then the next value in L must be where the curve exits (see Lemma 2) and we can add that value to the range and close it off. We continue adding ranges using the values in L and concatenate ranges when they are adjacent.\n\nTODO amend the above description a bit to reflect how it is implemented in code (which is more efficient)\n\nNote that a simplification for 2 dimensions *may* exist in that the list L doesn't need to be sorted but rather the minimum index on the perimeter found and then travel in which ever direction along the perimeter that has the closest initial index to that minimum index.\n\n##### Boundary algorithm runtime complexity\nThe boundary of a d-dimensional region is comprised of `2d` *faces* each having `d-1` dimensions.\n\nFor a cube of side `w` in `d` dimensions the runtime complexity of obtaining the ranges is:\n\n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;O(d(d-1)w\u003csup\u003ed-1\u003c/sup\u003elog(w))\n\nTODO assess runtime complexity of range finding algorithm proposed by Lawder and King in [Querying Multi-dimensional Data Indexed Using the Hilbert Space-Filling Curve](https://sigmodrecord.org/publications/sigmodRecord/0103/3.lawder.pdf).\n\n#### Query examples\nNote that for the moment range queries have been implemented for the `SmallHilbertCurve` only. Raise an issue if you need them for the `HilbertCurve` object as well.\n\nA lot of small ranges may be inefficient due to lookup overheads and constraints so you can specify the maximum number of ranges returned (ranges are joined that have minimal gap between them). \n\n```java\nSmallHilbertCurve c = HilbertCurve.small().bits(5).dimensions(2);\nlong[] point1 = new long[] {3, 3};\nlong[] point2 = new long[] {8, 10};\n// return just one range\nint maxRanges = 1;\nRanges ranges = c.query(point1, point2, maxRanges);\nranges.stream().forEach(System.out::println);\n```\nResult:\n```\nRange [low=10, high=229]\n```\nWe can improve the ranges by increasing `maxRanges`.\n\n\n`maxRanges` is 3\n```java\nRange [low=10, high=69]\nRange [low=122, high=132]\nRange [low=210, high=229]\n```\n\n`maxRanges` is 6\n```java\nRange [low=10, high=10]\nRange [low=26, high=53]\nRange [low=69, high=69]\nRange [low=122, high=132]\nRange [low=210, high=221]\nRange [low=227, high=229]\n```\n\n`maxRanges` is 0 (unlimited)\n```java\nRange [low=10, high=10]\nRange [low=26, high=28]\nRange [low=31, high=48]\nRange [low=51, high=53]\nRange [low=69, high=69]\nRange [low=122, high=124]\nRange [low=127, high=128]\nRange [low=131, high=132]\nRange [low=210, high=221]\nRange [low=227, high=229]\n```\n\nWhen using querying do experiments with the number of bits and `maxRanges` (querying in parallel on each range) to get your ideal run time. \n\nThe number of cells on the perimeter is width\u003csup\u003edimensions-1\u003c/sup\u003e 2\u003csup\u003ebits*(dimensions-1)\u003c/sup\u003e. Once the perimeter values are collected the values are sorted so the computational complexity of the algorithm is O(Xlog(X)) where X = width\u003csup\u003edimensions-1\u003c/sup\u003e 2\u003csup\u003ebits*(dimensions-1)\u003c/sup\u003e.\n\nWhen we desire to restrict the number of ranges returned by a call to `SmallHilbertCurve.query` optimizations are applied so that missed coverage is minimal and the computational complexity is reduced.\n\n### Spatio-temporal querying\nLet's consider 3 dimensions of information being latitude, longitude and time. We'll index the full world for one day using 10 bits. When I search the Sydney (Australia) region for an hour at midday I get exact coverage with 20 ranges and those ranges are calculated in ~20ns. When we limit the number of ranges the ratio of coverage to exact coverage is below:\n\n```\n20 1.00\n19 1.05\n18 1.11\n17 1.16\n16 1.22\n15 1.49\n14 1.75\n13 2.01\n12 2.28\n11 2.54\n10 2.80\n09 3.49\n08 4.17\n07 4.86\n06 5.54\n05 7.70\n04 11.54\n03 17.07\n02 56.29\n01 370.03\n```\nThe data above in a graph (only for ranges \u003e=4):\n\n\u003cimg src=\"src/docs/coverage.png\"/\u003e\n\nSo if you use 12 ranges you will be returned points from a region that is 2.28 times bigger than required for exact coverage. If your points were uniformly distributed then you would throw away roughly half the returned points because they were outside your search region. However, the tradeoff of query overhead may mean this is worthwhile. Your own benchmarks are the only way to really check this because your datastore will have its own concurrency and overhead characteristics.\n\nNote that if we expand the search to the entire region (give me every point) then the single range to cover it is returned in about 4.4s. As search boxes approach the dimensions of the entire domain some simplifications may be useful (TODO).\n\n## Benchmarks\n\nTo run benchmarks:\n\n```bash\nmvn clean install -P benchmark\n```\n\nResult 7 May 2019,\n\nOracle Java 1.8u201, Intel® Xeon(R) CPU-E51650 v2 @ 3.50GHz:\n```\nBenchmark                                        Mode  Cnt      Score     Error  Units\nBenchmarks.pointSmallTimes512                   thrpt   10   7053.853 ± 195.516  ops/s\nBenchmarks.pointSmallTimes512LowAllocation      thrpt   10  10300.849 ± 231.470  ops/s\nBenchmarks.pointTimes512                        thrpt   10   6461.033 ± 241.782  ops/s\nBenchmarks.pointTimes512LowAllocation           thrpt   10   6382.470 ±  95.981  ops/s\nBenchmarks.querySydney                          thrpt   10  52377.208 ± 619.063  ops/s\nBenchmarks.querySydneyMaxRanges8                thrpt   10  48068.992 ± 902.004  ops/s\nBenchmarks.roundTripSmallTimes512               thrpt   10   4128.441 ±  54.041  ops/s\nBenchmarks.roundTripSmallTimes512LowAllocation  thrpt   10   5174.301 ±  70.768  ops/s\nBenchmarks.roundTripTimes512                    thrpt   10   3196.380 ±  32.564  ops/s\nBenchmarks.roundTripTimes512LowAllocation       thrpt   10   3191.937 ±  48.420  ops/s\nBenchmarks.toIndexTimes512                      thrpt   10   6410.679 ± 291.534  ops/s\nBenchmarks.toIndexTimes512Small                 thrpt   10   8375.023 ± 412.324  ops/s\n```\nCredits\n----------\nPrimary credit goes to John Skilling for his article \"Programming the Hilbert curve\" (American Institue of Physics (AIP) Conf. Proc. 707, 381 (2004)).\n\nThanks to Paul Chernoch for his [StackOverflow answer](http://stackoverflow.com/questions/499166/mapping-n-dimensional-value-to-a-point-on-hilbert-curve) which got me most of the way there.\n\nDave Moten's contribution:\n\n* translate the C# code to java (use `long` instead of `uint`)\n* write the bit manipulations between the transposed index and the `BigInteger` index\n* lots of unit tests\n* made a simple easy-to-read API (hopefully)\n* supplement with utilities for using a Hilbert Index with bounding box searches\n\nReferences\n-----------\n* Skilling, John. \"Programming the Hilbert curve.\" AIP Conference Proceedings. Vol. 707. No. 1. AIP, 2004.\n* Lawder, Jonathan K., and Peter J. H. King. \"Querying multi-dimensional data indexed using the Hilbert space-filling curve.\" ACM Sigmod Record 30.1 (2001): 19-24.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdavidmoten%2Fhilbert-curve","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdavidmoten%2Fhilbert-curve","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdavidmoten%2Fhilbert-curve/lists"}