https://github.com/mdsumner/ibcso-cog
https://github.com/mdsumner/ibcso-cog
Last synced: 2 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/mdsumner/ibcso-cog
- Owner: mdsumner
- Created: 2023-10-02T19:57:23.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2024-01-24T11:40:50.000Z (over 2 years ago)
- Last Synced: 2025-06-04T19:54:21.567Z (12 months ago)
- Size: 67.4 MB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.Rmd
Awesome Lists containing this project
README
---
output: github_document
editor_options:
chunk_output_type: console
---
## IBCSO
International Bathymetric Chart of the Southern Ocean (IBCSO) is the authoritative map of the Southern Ocean, providing a raster bathymetric chart of
depths, currently at v2 in 2023.
The .tif format is slow, it is not a Cloud-Optimized GeoTIFF so we would like to demonstrate the benefits of that format.
We download bedrock, ice surface, and chart and convert to Cloud Optimized GeoTIFF and then host it
at these urls:
```
https://github.com/mdsumner/ibcso-cog/raw/main/IBCSO_v2_ice-surface_cog.tif
```
```
https://github.com/mdsumner/ibcso-cog/raw/main/IBCSO_v2_bed_cog.tif
```
```
https://github.com/mdsumner/ibcso-cog/raw/main/IBCSO_v2_digital_chart_cog.tif
```
Prefix with `/vsicurl/` for general GDAL usage, and especially via the warper or RasterIO apis for fast access at any scale or region and in any crs.
## Citation
> Dorschel, Boris; Hehemann, Laura; Viquerat, Sacha; Warnke, Fynn; Dreutter, Simon; Schulze Tenberge, Yvonne; Accettella, Daniela; An, Lu; Barrios, Felipe; Bazhenova, Evgenia A; Black, Jenny; Bohoyo, Fernando; Davey, Craig; de Santis, Laura; Escutia Dotti, Carlota; Fremand, Alice C; Fretwell, Peter T; Gales, Jenny A; Gao, Jinyao; Gasperini, Luca; Greenbaum, Jamin S; Henderson Jencks, Jennifer; Hogan, Kelly A; Hong, Jong Kuk; Jakobsson, Martin; Jensen, Laura; Kool, Johnathan; Larin, Sergei; Larter, Robert D; Leitchenkov, German L; Loubrieu, Benoît; Mackay, Kevin; Mayer, Larry; Millan, Romain; Morlighem, Mathieu; Navidad, Francisco; Nitsche, Frank-Oliver; Nogi, Yoshifumi; Pertuisot, Cécile; Post, Alexandra L; Pritchard, Hamish D; Purser, Autun; Rebesco, Michele; Rignot, Eric; Roberts, Jason L; Rovere, Marzia; Ryzhov, Ivan; Sauli, Chiara; Schmitt, Thierry; Silvano, Alessandro; Smith, Jodie E; Snaith, Helen; Tate, Alex J; Tinto, Kirsty; Vandenbossche, Philippe; Weatherall, Pauline; Wintersteller, Paul; Yang, Chunguo; Zhang, Tao; Arndt, Jan Erik (2022): The International Bathymetric Chart of the Southern Ocean Version 2 (IBCSO v2). PANGAEA, https://doi.org/10.1594/PANGAEA.937574
## Creation (of the files above)
```
wget https://download.pangaea.de/dataset/937574/files/IBCSO_v2_ice-surface.tif
gdal_translate IBCSO_v2_ice-surface.tif IBCSO_v2_ice-surface_cog.tif -of COG -co COMPRESS=ZSTD -co PREDICTOR=2 -co SPARSE_OK=YES -co OVERVIEW_RESAMPLING=AVERAGE -co BLOCKSIZE=480 -co NUM_THREADS=ALL_CPUS
```
```
wget https://download.pangaea.de/dataset/937574/files/IBCSO_v2_bed.tif
gdal_translate IBCSO_v2_bed.tif IBCSO_v2_bed_cog.tif -of COG -co COMPRESS=ZSTD -co PREDICTOR=2 -co SPARSE_OK=YES -co OVERVIEW_RESAMPLING=AVERAGE -co BLOCKSIZE=480 -co NUM_THREADS=ALL_CPUS
```
See below for notes on these options.
Some comparisons on performance are shown below, please note that five quite different extractions from the COG url are timed for ~ 10-20 seconds, giving five quite different data configurations at different scales and in different projections. To do one of the five from the local copy of the current v2 .tif takes nearly 10 seconds on its own.
```{r cog}
dsn <- "/vsicurl/https://github.com/mdsumner/ibcso-cog/raw/main/IBCSO_v2_ice-surface_cog.tif"
library(terra)
r <- rast(dsn)
system.time({
r1 <- project(r, rast(), by_util = T)
template <- rast(r)
res(template) <- res(r) * 20
## this extent is one small part at the original resolution
(r2 <- project(r, rast(ext(c(0, 3e+05, 3400000, 3950000)), crs = crs(r), res = 500), by_util = TRUE))
## these work very quickly
r3 <- project(r, rast(ext(-180, 180, -90, -50)), by_util = T)
r4 <- project(r, rast(ext(-180, 180, -90, -50), res = 1), by_util = T)
r5 <- project(r, rast(ext(-180, 180, -90, -50), res = .25), by_util = T)
})
## even if we use the local .tif, with the older format it's much slower
rlocal <- rast("IBCSO_v2_ice-surface.tif")
system.time(project(rlocal, template, by_util = T))
```
Plot the different extractions to show that they worked.
```{r plots}
plot(r1)
plot(r2)
plot(r3)
plot(r4)
plot(r5)
plot(project(r, template, by_util = TRUE))
plot(ext(r2), add = TRUE)
```
In Python it's similar, we can do
```python
from osgeo import gdal
ds = gdal.Open("/vsicurl/https://github.com/mdsumner/ibcso-cog/raw/main/IBCSO_v2_ice-surface_cog.tif")
w = gdal.Warp("/vsimem/myfile.vrt", ds, xRes = 10000, yRes = 10000)
w.GetGeoTransform()
# (-4800000.0, 10000.0, 0.0, 4800000.0, 0.0, -10000.0)
w.ReadAsArray()
#array([[-32768, -32768, -32768, ..., -32768, -32768, -32768],
# [-32768, -32768, -32768, ..., -32768, -32768, -32768],
# [-32768, -32768, -32768, ..., -32768, -32768, -32768],
# ...,
# [-32768, -32768, -32768, ..., -32768, -32768, -32768],
# [-32768, -32768, -32768, ..., -32768, -32768, -32768],
# [-32768, -32768, -32768, ..., -32768, -32768, -32768]], dtype=int16)
```
## Chart
Digital chart version created from tif version of the PDF with
```
gdal_translate IBCSO_chart.tif IBCSO_v2_digital_chart.tif -of COG -a_ullr -4833000 4797000 4837000 -6629000 -a_srs EPSG:9354
```
## gdal translate options
We chose options blocksize 480 so that the smallest overview would be that size (at 4800 we only end up with 2 overview levels).
SPARSE_OK: ensures unused blocks are not stored (they are all 0, or nodata).
COMPRESS: could choose DEFLATE instead
OVERVIEW_RESAMPLING: AVERAGE so that the average value is stored from higher resolution levels, not just a sample
NUM_THREADS: this just makes it go faster (for the compression).