{"id":18722632,"url":"https://github.com/mdsumner/gdalnara","last_synced_at":"2026-03-19T06:03:51.643Z","repository":{"id":214822749,"uuid":"737419195","full_name":"mdsumner/gdalnara","owner":"mdsumner","description":"Bridge GDAL to nativeRaster","archived":false,"fork":false,"pushed_at":"2024-01-18T02:34:27.000Z","size":503,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-10-30T05:51:35.435Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://mdsumner.github.io/gdalnara/","language":"M4","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mdsumner.png","metadata":{"files":{"readme":"README.Rmd","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-12-31T00:44:50.000Z","updated_at":"2024-01-17T21:37:39.000Z","dependencies_parsed_at":"2023-12-31T05:19:14.033Z","dependency_job_id":"bedd640d-9ff2-4cdf-a023-6e487d3ab2cd","html_url":"https://github.com/mdsumner/gdalnara","commit_stats":null,"previous_names":["mdsumner/gdalnara"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/mdsumner/gdalnara","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mdsumner%2Fgdalnara","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mdsumner%2Fgdalnara/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mdsumner%2Fgdalnara/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mdsumner%2Fgdalnara/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mdsumner","download_url":"https://codeload.github.com/mdsumner/gdalnara/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mdsumner%2Fgdalnara/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29571888,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-18T06:19:27.422Z","status":"ssl_error","status_checked_at":"2026-02-18T06:18:44.348Z","response_time":162,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T13:42:16.574Z","updated_at":"2026-02-18T07:02:55.882Z","avatar_url":"https://github.com/mdsumner.png","language":"M4","funding_links":[],"categories":[],"sub_categories":[],"readme":"---\noutput: github_document\neditor_options: \n  chunk_output_type: console\n---\n\n\u003c!-- README.md is generated from README.Rmd. Please edit that file --\u003e\n\n```{r, include = FALSE}\nknitr::opts_chunk$set(\n  collapse = TRUE,\n  comment = \"#\u003e\",\n  fig.path = \"man/figures/README-\",\n  out.width = \"100%\"\n)\n\ndsn \u003c-  \"\u003cGDAL_WMS\u003e\u003cService name=\\\"VirtualEarth\\\"\u003e\u003cServerUrl\u003ehttp://a${server_num}.ortho.tiles.virtualearth.net/tiles/a${quadkey}.jpeg?g=90\u003c/ServerUrl\u003e\u003c/Service\u003e\u003cMaxConnections\u003e4\u003c/MaxConnections\u003e\u003cCache/\u003e\u003c/GDAL_WMS\u003e\"\n\nlibrary(gdalnara)\n\nnara \u003c- gdal_raster_nara(dsn, target_dim = c(1024, 0))\n\n```\n\n# gdalnara\n\n\u003c!-- badges: start --\u003e\n\u003c!-- badges: end --\u003e\n\nThe goal of gdalnara is to bridge the GDAL raster read to the nativeRaster of R. \n\nPlease consider this a temporary experiment, probably to be folded into vapour or another GDAL package. \n\n## What the heck is nativeRaster of R?\n\nWell exactly, it's an integer-encoded value of RGB(A) in a matrix with an odd ordering convention. This is different to\n\n- the xyz list of (usually degenerate) rectilinear x,y vectors and a z matrix in `graphics::image()`\n- the raster-ordered array of RGB(A) used by `graphics::rasterImage()`, either values in 0,255 or character hex values in a matrix \n\nIt's not exposed at all in R documentation, but is used internally in R, and externally by the png package, and a little bit by farver and nara (more below). \n\nFirst let's look at `rasterImage()`. \n\nIf we take a matrix of 12 numbers, and make an image using the standard spatial topleft -\u003e topright then then row-based ordering: \n\n```{r matrix}\nm \u003c- matrix(as.integer(1:12), ncol = 3L, byrow = TRUE)\n\nplot(NA, xlim = c(0, 3), ylim = c(0, 4), asp = 1)\n\nscl \u003c- function(x) {xl \u003c- range(x, na.rm = TRUE); (x - xl[1])/diff(xl)}\nrasterImage(scl(m), 0, 0, 3, 4, interpolate = FALSE)\n```\n\nwith numbers in the range 0,1 we automatically get a greyscale image, but we can do that explicitly too:\n\n```{r hex}\nmh \u003c- m\nmh[] \u003c- grey.colors(12, start = 0, end = 1, gamma = 1)[m]\n\nplot(NA, xlim = c(0, 3), ylim = c(0, 4), asp = 1)\nrasterImage(mh, 0, 0, 3, 4, interpolate = FALSE)\n\n```\n\nBut, the size of the data has been multiplied nearly 10x  and especially the size of the character matrix is quite large (let's ignore the fact that we are using full RGB for greyscale, and that we might use 3 sets of raw rather than actual character strings, or the 3 sets of integer from col2rgb ...). \n\nNote that we are using `c()` to strip attributes, because the small attributes are an significant size here compared to the data and using `c()` is forbidden by R gods so we like doing it. \n\n\n```{r large}\npryr::object_size(c(m))\npryr::object_size(c(m * 1.0))   ## when we have to rescale to 0,1 we are double\n\npryr::object_size(c(mh))\n\npryr::object_size(c(col2rgb(mh)))\n```\n\nSo what's a nativeRaster? \n\n\n```{r nativeRaster}\nrgbvals \u003c- col2rgb(t(mh))\n#mi \u003c- m  ## we making sure we are integer type, whereas m could have been double or integer\n\nmn \u003c- structure(as.integer(rgbvals[1L, ] * 256 * 256 + \n rgbvals[2L, ] * 256  +\n  rgbvals[3L, ] - (256 * 256 * 256)), dim = c(4L, 3L), \n class = \"nativeRaster\", channels = 3L)\n\n\n\nplot(NA, xlim = c(0, 3), ylim = c(0, 4), asp = 1)\nrasterImage(mn, 0, 0, 3, 4, interpolate = FALSE)\n   \n\npryr::object_size(c(mn) )\n```\n\nSo we're done!  No.  All of that above is quite expensive because values get converted to numeric and integer and it's potentially a lot of data getting thrown around when it was originally very compact in some cunning image format. \n\n\n## gdalnara\n\nThe gdalnara package exists to bridge the three functions in the GDAL package vapour that can read natively or to any output grid extent, resolution, crs - but to do as much as possible in C++. These are \n\n- gdal_raster_data (read numeric or Byte or Integer data as natively as possible in R - raw, integer, double)\n- gdal_raster_dsn (read a raster source and write it out to file)\n- gdal_raster_image (read a image data to a character string of colours in R, from 1, 3, or 4 bands)\n\nThe last image function is quite nice because 3 Byte bands are coalesced into a single character string, but as shown above that can be quite large. \n\nDoing things in C++ next to where we access the raster data with GDAL is neat, because a usual image scenario is Byte values, we can use bit shift in C++ which is very fast. In fact we copied the code from R itself because it's very simple. \n\nThis code is also used in the png package by Simon Urbanek, with the function `readPNG()` and its `native` arg, in the farver package by Thomas Pederson, and in the nara package by Mike FC (which inspired this name and the effort to figure this out). \n\n\nThere's a few packages on CRAN that know about nativeRaster: https://github.com/search?q=org%3Acran%20nativeRaster\u0026type=code\n\n## A real world example\n\nGDAL can read imagery to any extent and projection. \n\n\n```{r world}\ndsn \u003c-  \"\u003cGDAL_WMS\u003e\u003cService name=\\\"VirtualEarth\\\"\u003e\u003cServerUrl\u003ehttp://a${server_num}.ortho.tiles.virtualearth.net/tiles/a${quadkey}.jpeg?g=90\u003c/ServerUrl\u003e\u003c/Service\u003e\u003cMaxConnections\u003e4\u003c/MaxConnections\u003e\u003cCache/\u003e\u003c/GDAL_WMS\u003e\"\n\nex \u003c- c(-1, 1, -1, 1) * 2e5\ncrs \u003c- \"+proj=laea +lon_0=147 +lat_0=-42\"\nlibrary(ximage)\nlibrary(gdalnara)\nlibrary(vapour)\nsystem.time(nara \u003c- gdal_raster_nara(dsn, target_dim = c(1024, 0), target_ext = ex, target_crs = crs))\npryr::object_size(nara)\n\nsystem.time(chra \u003c- gdal_raster_image(dsn, target_dim = c(1024, 0), target_ext = ex, target_crs = crs))\npryr::object_size(chra)\n\n## it's very quick\nsystem.time(ximage(nara, asp = 1))\n```\n\n\nIt takes a lot of data to convert to RGB array and quite some time to plot. \n\n\n```{r aperm}\n\nx \u003c- aperm(nara::nr_to_array(nara[[1]])[,,-4],   ## drop alpha since we don't use it\n           c(2, 1, 3)) ## and transpose so we can prove it's actually doing something\n\nximage(nara, asp = 1)\nsystem.time(rasterImage(x, ex[1], ex[3], ex[2], ex[4]))\npryr::object_size(x)\n\n\n```\n\n\nWe can even scale data to grey from numeric and read efficiently with gdalnara. \n\n(note GDAL 3.7 required for this syntax, we might use VRT for other ways). \n\nThis is not that successful because we need to know the source range, quite complicated for subsets and reprojections etc. (but will fix later ...). \n\n```{r gdal-3.7, eval=FALSE}\ndsn \u003c- \"vrt:///vsicurl/https://gebco2023.s3.valeria.science/gebco_2023_land_cog.tif?scale=-7676,5500,0,255\u0026ot=Float32\"\n\nim \u003c- gdal_raster_nara(dsn, target_dim = c(1024, 0))\nximage(im)\n\n```\n\n![world topography scaled to grey](man/figures2/README-gdal-3.7-1.png)\n\n\n## Installation\n\nYou can install the development version of gdalnara like so:\n\n``` r\ninstall.package\npak::pak(\"mdsumner/gdalnara\")\n```\n\n\n## Code of Conduct\n  \nPlease note that the gdalnara project is released with a [Contributor Code of Conduct](https://contributor-covenant.org/version/2/1/CODE_OF_CONDUCT.html). By contributing to this project, you agree to abide by its terms.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmdsumner%2Fgdalnara","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmdsumner%2Fgdalnara","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmdsumner%2Fgdalnara/lists"}