{"id":28070137,"url":"https://github.com/geoarrow/deck.gl-layers","last_synced_at":"2025-05-12T19:38:06.099Z","repository":{"id":186819212,"uuid":"675812958","full_name":"geoarrow/deck.gl-layers","owner":"geoarrow","description":"deck.gl layers for rendering GeoArrow data","archived":false,"fork":false,"pushed_at":"2025-02-11T21:01:36.000Z","size":2715,"stargazers_count":119,"open_issues_count":26,"forks_count":8,"subscribers_count":10,"default_branch":"main","last_synced_at":"2025-05-05T13:01:25.795Z","etag":null,"topics":["apache-arrow","data-visualization","deck-gl","geoarrow","geospatial","map-visualization"],"latest_commit_sha":null,"homepage":"http://geoarrow.org/deck.gl-layers/","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/geoarrow.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-07T19:34:30.000Z","updated_at":"2025-04-22T14:18:47.000Z","dependencies_parsed_at":"2024-01-02T23:57:25.946Z","dependency_job_id":"46cc0e6b-0b97-46ec-aebf-c14c3a54765d","html_url":"https://github.com/geoarrow/deck.gl-layers","commit_stats":null,"previous_names":["kylebarron/deck.gl-geoarrow","geoarrow/deck.gl-layers"],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/geoarrow%2Fdeck.gl-layers","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/geoarrow%2Fdeck.gl-layers/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/geoarrow%2Fdeck.gl-layers/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/geoarrow%2Fdeck.gl-layers/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/geoarrow","download_url":"https://codeload.github.com/geoarrow/deck.gl-layers/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253808909,"owners_count":21967618,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache-arrow","data-visualization","deck-gl","geoarrow","geospatial","map-visualization"],"created_at":"2025-05-12T19:38:05.527Z","updated_at":"2025-05-12T19:38:06.087Z","avatar_url":"https://github.com/geoarrow.png","language":"TypeScript","readme":"# @geoarrow/deck.gl-layers\n\nThe easiest, most efficient way to render large geospatial datasets in [deck.gl](https://deck.gl), via [GeoArrow](https://geoarrow.org).\n\nThis is just a _glue library_ to deck.gl. It generates the same layer objects as upstream deck.gl does, but uses a [low-level binary interface](https://deck.gl/docs/developer-guide/performance#supply-attributes-directly) for best performance. Using the binary interface directly is really easy to mess up. Instead, the layer classes exposed by `@geoarrow/deck.gl-layers` focus on making the process easy to use and validating user input, and under the hood pass buffers to deck.gl's binary interface.\n\n![](assets/hero.jpg)\n\n\u003cp style=\"text-align:center\"\u003e3.2 million points rendered with a \u003ccode\u003eGeoArrowScatterplotLayer\u003c/code\u003e.\u003c/p\u003e\n\n## Features\n\n- **Fast**: copies binary buffers directly from an [Arrow JS](https://www.npmjs.com/package/apache-arrow) [`Table`](https://arrow.apache.org/docs/js/classes/Arrow_dom.Table.html) object to the GPU using [deck.gl's binary data interface](https://deck.gl/docs/developer-guide/performance#supply-attributes-directly).\n- **Memory-efficient**: no intermediate data representation and no garbage-collector overhead.\n- **Full layer customization**: Use the same layer properties as in the upstream deck.gl layer documentation. Any _accessor_ (layer property prefixed with `get*`) can be passed an Arrow [`Vector`](https://arrow.apache.org/docs/js/classes/Arrow_dom.Vector.html).\n- **Input validation**. Validation can be turned off via the `_validate` property on most layer types.\n- **Multi-threaded polygon triangulation**. When rendering polygon layers, a process called [polygon triangulation](https://en.wikipedia.org/wiki/Polygon_triangulation) must happen on the CPU before data can be copied to the GPU. Ordinarily, this can block the main thread for several seconds, but the `GeoArrowSolidPolygonLayer` will perform this process off the main thread, on up to 8 web workers.\n- **Progressive rendering support**. For streaming-capable data formats like Arrow IPC and Parquet, you can render a GeoArrow layer per chunk as the data loads.\n\n## Examples\n\nStandalone examples exist in the [`examples/`](examples/) directory. Create an issue if you have trouble running them.\n\nMore hosted examples on Observable are planned.\n\n## Providing accessors\n\nAll deck.gl layers have two types of properties: [\"Render Options\"](https://deck.gl/docs/api-reference/layers/scatterplot-layer#render-options) — constant properties across a layer — and \"Data Accessors\" — properties that can vary across rows. An accessor is any property prefixed with `get`, like `GeoArrowScatterplotLayer`'s `getFillColor`.\n\nWith `@geoarrow/deck.gl-layers` specifically, there are two ways to pass these data accessors, either as pre-computed columns or with function callbacks on Arrow data.\n\n### Pre-computed Arrow columns\n\nIf you have an Arrow column ([`Vector`](https://arrow.apache.org/docs/js/classes/Arrow_dom.Vector.html) in Arrow JS terminology), you can pass that directly into a layer:\n\n```ts\nimport { Table } from \"apache-arrow\";\nimport { GeoArrowScatterplotLayer } from \"@geoarrow/deck.gl-layers\";\n\nconst table = new Table(...);\nconst deckLayer = new GeoArrowScatterplotLayer({\n  id: \"scatterplot\",\n  data: table,\n  /// Geometry column\n  getPosition: table.getChild(\"geometry\")!,\n  /// Column of type FixedSizeList[3] or FixedSizeList[4], with child type Uint8\n  getFillColor: table.getChild(\"colors\")!,\n});\n```\n\nFor example, [lonboard](https://github.com/developmentseed/lonboard) computes Arrow columns on the Python side for all attributes so that end users have available the full capabilities Python. Then those columns are serialized to Python and the resulting `arrow.Vector` is passed into the relevant layer.\n\n### Function accessors\n\nGeoArrow layers accept a callback that takes an object with `index` and `data`. `data` is an `arrow.RecordBatch` object (a vertical section of the input `Table`), and `index` is the positional index of the current row of that batch. In TypeScript, you should see accurate type checking.\n\n```ts\nconst deckLayer = new GeoArrowPathLayer({\n  id: \"geoarrow-path\",\n  data: table,\n  getColor: ({ index, data, target }) =\u003e {\n    const recordBatch = data.data;\n    const row = recordBatch.get(index)!;\n    return COLORS_LOOKUP[row[\"scalerank\"]];\n  },\n}),\n```\n\nThe full example is in [`examples/multilinestring/app.tsx`](examples/multilinestring/app.tsx).\n\nYou can also use assign to the `target` prop to reduce garbage collector overhead, as described in the [deck.gl performance guide](https://deck.gl/docs/developer-guide/performance#supply-binary-blobs-to-the-data-prop).\n\n## Data Loading\n\nTo create deck.gl layers using this library, you need to first get GeoArrow-formatted data into the browser, discussed below.\n\n[OGR/GDAL](https://gdal.org/) is useful for converting among data formats on the backend, and it includes both [GeoArrow](https://gdal.org/drivers/vector/arrow.html#vector-arrow) and [GeoParquet](https://gdal.org/drivers/vector/parquet.html) drivers. Pass `-lco GEOMETRY_ENCODING=GEOARROW` when converting to Arrow or Parquet files in order to store geometries in a GeoArrow-native geometry column.\n\n### Arrow IPC\n\nIf you already have Arrow IPC files (also called Feather files) with a GeoArrow geometry column, you can use [`apache-arrow`](https://www.npmjs.com/package/apache-arrow) to load those files.\n\n```ts\nimport { tableFromIPC } from \"apache-arrow\";\nimport { GeoArrowScatterplotLayer } from \"@geoarrow/deck.gl-layers\";\n\nconst resp = await fetch(\"url/to/file.arrow\");\nconst jsTable = await tableFromIPC(resp);\nconst deckLayer = new GeoArrowScatterplotLayer({\n  id: \"scatterplot\",\n  data: jsTable,\n  /// Replace with the correct geometry column name\n  getPosition: jsTable.getChild(\"geometry\")!,\n});\n```\n\nNote those IPC files must be saved **uncompressed** (at least not internally compressed). As of v14, Arrow JS does not currently support loading IPC files with internal compression.\n\n### Parquet\n\nIf you have a Parquet file where the geometry column is stored as _GeoArrow_ encoding (i.e. not as a binary column with WKB-encoded geometries), you can use the stable `parquet-wasm` library to load those files.\n\n```ts\nimport { readParquet } from \"parquet-wasm\"\nimport { tableFromIPC } from \"apache-arrow\";\nimport { GeoArrowScatterplotLayer } from \"@geoarrow/deck.gl-layers\";\n\nconst resp = await fetch(\"url/to/file.parquet\");\nconst arrayBuffer = await resp.arrayBuffer();\nconst wasmTable = readParquet(new Uint8Array(arrayBuffer));\nconst jsTable = tableFromIPC(wasmTable.intoIPCStream());\nconst deckLayer = new GeoArrowScatterplotLayer({\n  id: \"scatterplot\",\n  data: jsTable,\n  /// Replace with the correct geometry column name\n  getPosition: jsTable.getChild(\"geometry\")!,\n});\n```\n\nSee below for instructions to load GeoParquet 1.0 files, which have WKB-encoded geometries that need to be decoded before they can be used with `@geoarrow/deck.gl-layers`.\n\n### GeoParquet\n\nAn initial version of the [`@geoarrow/geoparquet-wasm`](https://www.npmjs.com/package/@geoarrow/geoparquet-wasm) library is published, which reads a GeoParquet file to GeoArrow memory.\n\n```ts\nimport { readGeoParquet } from \"@geoarrow/geoparquet-wasm\";\nimport { tableFromIPC } from \"apache-arrow\";\nimport { GeoArrowScatterplotLayer } from \"@geoarrow/deck.gl-layers\";\n\nconst resp = await fetch(\"url/to/file.parquet\");\nconst arrayBuffer = await resp.arrayBuffer();\nconst wasmTable = readGeoParquet(new Uint8Array(arrayBuffer));\nconst jsTable = tableFromIPC(wasmTable.intoTable().intoIPCStream());\nconst deckLayer = new GeoArrowScatterplotLayer({\n  id: \"scatterplot\",\n  data: jsTable,\n  /// Replace with the correct geometry column name\n  getPosition: jsTable.getChild(\"geometry\")!,\n});\n```\n\nIf you hit a bug with `@geoarrow/geoparquet-wasm`, please create a reproducible bug report [here](https://github.com/geoarrow/geoarrow-rs/issues/new).\n\n### FlatGeobuf\n\nAn initial version of the [`@geoarrow/flatgeobuf-wasm`](https://www.npmjs.com/package/@geoarrow/flatgeobuf-wasm) library is published, which reads a FlatGeobuf file to GeoArrow memory. As of version 0.2.0-beta.1, this library does not yet support remote files, and expects the full FlatGeobuf file to exist in memory.\n\n```ts\nimport { readFlatGeobuf } from \"@geoarrow/flatgeobuf-wasm\";\nimport { tableFromIPC } from \"apache-arrow\";\nimport { GeoArrowScatterplotLayer } from \"@geoarrow/deck.gl-layers\";\n\nconst resp = await fetch(\"url/to/file.fgb\");\nconst arrayBuffer = await resp.arrayBuffer();\nconst wasmTable = readFlatGeobuf(new Uint8Array(arrayBuffer));\nconst jsTable = tableFromIPC(wasmTable.intoTable().intoIPCStream());\nconst deckLayer = new GeoArrowScatterplotLayer({\n  id: \"scatterplot\",\n  data: jsTable,\n  /// Replace with the correct geometry column name\n  getPosition: jsTable.getChild(\"geometry\")!,\n});\n```\n\nIf you hit a bug with `@geoarrow/flatgeobuf-wasm`, please create a reproducible bug report [here](https://github.com/geoarrow/geoarrow-rs/issues/new).\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgeoarrow%2Fdeck.gl-layers","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgeoarrow%2Fdeck.gl-layers","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgeoarrow%2Fdeck.gl-layers/lists"}