Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/smilyorg/tinygpkg-data

Small geographic datasets based on open data + tools
https://github.com/smilyorg/tinygpkg-data

data dataset geo geodata geography geojson geopackage golang naturalearth naturalearthdata

Last synced: about 2 months ago
JSON representation

Small geographic datasets based on open data + tools

Awesome Lists containing this project

README

        





Logo

tinygpkg-data


Small, TWKB compressed, and simplified GeoPackage datasets to be used with tinygpkg.




Report Bug
·
Request Feature

Table of Contents


  1. About

  2. Datasets

  3. Featured Variants

  4. Parameters

  5. Variants

  6. Contributing

  7. License

  8. Acknowledgements

## About

This repository contains a set of scripts and tools for generating [Tiny
Well-known Binary (TWKB)] compressed [GeoPackage] files from various open data
sources. Check out [Featured Variants](#featured-variants) or [releases] to
download the data or keep reading for more details and comparisons.

[releases]: https://github.com/SmilyOrg/tinygpkg-data/releases

## Datasets

| Name | Contents | Features | Source | License |
| ----------------------------- | --------------------------------------- | -------: | --------------- | ---------------------------------- |
| [ne_110m_admin_0_countries] | Country borders, 1:110m scale | 177 | [Natural Earth] | [Public Domain][ne-license] |
| [ne_10m_admin_0_countries] | Country borders, 1:10m scale | 258 | [Natural Earth] | [Public Domain][ne-license] |
| [ne_10m_urban_areas_landscan] | Big cities only, 1:10m scale | 6018 | [Natural Earth] | [Public Domain][ne-license] |
| [geoBoundariesCGAZ_ADM0] | Country-level administrative boundaries | 200 | [geoBoundaries] | [Attribution required][gb-license] |
| [geoBoundariesCGAZ_ADM2] | City-level administrative boundaries | 49689 | [geoBoundaries] | [Attribution required][gb-license] |

[ne_110m_admin_0_countries]: #ne_110m_admin_0_countries
[ne_10m_admin_0_countries]: #ne_10m_admin_0_countries
[ne_10m_urban_areas_landscan]: #ne_10m_urban_areas_landscan
[geoBoundariesCGAZ_ADM0]: #geoboundariescgaz_adm0
[geoBoundariesCGAZ_ADM2]: #geoboundariescgaz_adm2

[Natural Earth]: https://www.naturalearthdata.com/
[geoBoundaries]: https://www.geoboundaries.org
[ne-license]: https://www.naturalearthdata.com/about/terms-of-use/
[gb-license]: https://www.geoboundaries.org/index.html#citation

## Featured Variants

These are some selected useful variants of the datasets. See
[Variants](#variants) for the full list.

| File | Description | Size (vs. _source_) | Gzipped size (vs. _source_) |
| ---- | ----------- | ------------------- | --------------------------- |
| **Tiny (TWKB)** ||||
| [📥 ne_110m_admin_0_countries_s5_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_110m_admin_0_countries_s5_twkb_p3.gpkg) | 117 country borders | **393 KB**
_46.9% of 838 KB_ | 116 KB
_56.0% of 208 KB_ |
| [📥 ne_10m_admin_0_countries_s4_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_admin_0_countries_s4_twkb_p3.gpkg) | 258 country borders | **1249 KB**
_9.4% of 13287 KB_ | 782 KB
_17.0% of 4614 KB_ |
| [📥 ne_10m_urban_areas_landscan_s5_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_urban_areas_landscan_s5_twkb_p3.gpkg) | 6018 city borders | **4333 KB**
_7.5% of 57682 KB_ | 1780 KB
_22.0% of 8078 KB_ |
| [📥 geoBoundariesCGAZ_ADM2_s5_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/geoBoundariesCGAZ_ADM2_s5_twkb_p3.gpkg) | 49689 city borders | **16420 KB**
_6.0% of 272859 KB_ | 10782 KB
_6.4% of 168993 KB_ |
| **Compatible (WKB)** ||||
| [📥 ne_110m_admin_0_countries_s5_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_110m_admin_0_countries_s5_wkb.gpkg) | 117 country borders | **557 KB**
_66.4% of 838 KB_ | 215 KB
_103.8% of 208 KB_ |
| [📥 ne_10m_admin_0_countries_s4_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_admin_0_countries_s4_wkb.gpkg) | 258 country borders | **4997 KB**
_37.6% of 13287 KB_ | 2566 KB
_55.6% of 4614 KB_ |
| [📥 ne_10m_urban_areas_landscan_s5_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_urban_areas_landscan_s5_wkb.gpkg) | 6018 city borders | **21704 KB**
_37.6% of 57682 KB_ | 4663 KB
_57.7% of 8078 KB_ |
| [📥 geoBoundariesCGAZ_ADM2_s5_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/geoBoundariesCGAZ_ADM2_s5_wkb.gpkg) | 49689 city borders | **52469 KB**
_19.2% of 272859 KB_ | 34726 KB
_20.5% of 168993 KB_ |

## Parameters

The `makevalid` variant is the original source file converted to [GeoPackage]
format using [ogr2ogr] with the `-makevalid` flag. This ensures a consistent
intermediary format with valid geometry that is then further simplified and
compressed.

Source datasets are minified using two methods, simplification and [Tiny
Well-known Binary (TWKB)][TWKB] compression.

Simplification is performed using the Ramer-Douglas-Peucker [Simplify] method on
the polygons. If the simplification fails (creates an invalid polygon), less and
less simplification is used until the polygon remains valid. If the polygon has
less than "Min. Points", it is not simplified.

Precision is the maximum number of decimal places used to store the coordinates
using [TWKB]. From empirical testing, less than 3 decimal places does not save a
lot of space and more than 3 decimal places does not gain a lot in precision for
these datasets.

The `wkb` variants are only simplified and not compressed, keeping the original
Well-known Binary (WKB) format. This increases compatibility with other tools at
the cost of larger file size.

| Name | Simplify | Min. Points | Precision |
| ---------- | -------- | ----------- | --------- |
| makevalid | none | n/a | full |
| s3_twkb_p3 | 1 | 20 | 3 |
| s4_twkb_p3 | 0.1 | 20 | 3 |
| s5_twkb_p3 | 0.01 | 20 | 3 |
| s6_twkb_p3 | 0.001 | 20 | 3 |
| s7_twkb_p3 | 0.0001 | 20 | 3 |
| s8_twkb_p3 | 0.00001 | 20 | 3 |
| s3_wkb | 1 | 20 | full |
| s4_wkb | 0.1 | 20 | full |
| s5_wkb | 0.01 | 20 | full |
| s6_wkb | 0.001 | 20 | full |
| s7_wkb | 0.0001 | 20 | full |
| s8_wkb | 0.00001 | 20 | full |

[TWKB]: https://github.com/TWKB/Specification/blob/master/twkb.md
[Simplify]: https://pkg.go.dev/github.com/peterstace/simplefeatures/geom#Geometry.Simplify
[ogr2ogr]: https://gdal.org/programs/ogr2ogr.html

## Variants

These are the variants of the datasets available. Each variant is a combination of the parameters described above.

### ne_110m_admin_0_countries

Country borders, 1:110m scale sourced from [Natural Earth] ([Public Domain][ne-license]).

See [Parameters](#parameters) for what each variant means and
[Datasets](#datasets) for details on the dataset itself.

| Variant | Size | world | europe | africa | usa | japan |
| --- | --- | --- | --- | --- | --- | --- |
| [📥 makevalid](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_110m_admin_0_countries_makevalid.gpkg) | 569 KB | | | | | |
| [📥 s3_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_110m_admin_0_countries_s3_twkb_p3.gpkg) | 352 KB | | | | | |
| [📥 s4_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_110m_admin_0_countries_s4_twkb_p3.gpkg) | 393 KB | | | | | |
| [📥 s5_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_110m_admin_0_countries_s5_twkb_p3.gpkg) | 393 KB | | | | | |
| [📥 s6_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_110m_admin_0_countries_s6_twkb_p3.gpkg) | 393 KB | | | | | |
| [📥 s7_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_110m_admin_0_countries_s7_twkb_p3.gpkg) | 393 KB | | | | | |
| [📥 s8_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_110m_admin_0_countries_s8_twkb_p3.gpkg) | 393 KB | | | | | |
| [📥 s3_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_110m_admin_0_countries_s3_wkb.gpkg) | 409 KB | | | | | |
| [📥 s4_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_110m_admin_0_countries_s4_wkb.gpkg) | 520 KB | | | | | |
| [📥 s5_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_110m_admin_0_countries_s5_wkb.gpkg) | 557 KB | | | | | |
| [📥 s6_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_110m_admin_0_countries_s6_wkb.gpkg) | 557 KB | | | | | |
| [📥 s7_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_110m_admin_0_countries_s7_wkb.gpkg) | 557 KB | | | | | |
| [📥 s8_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_110m_admin_0_countries_s8_wkb.gpkg) | 557 KB | | | | | |

### ne_10m_admin_0_countries

Country borders, 1:10m scale sourced from [Natural Earth] ([Public Domain][ne-license]).

See [Parameters](#parameters) for what each variant means and
[Datasets](#datasets) for details on the dataset itself.

| Variant | Size | world | europe | africa | usa | japan |
| --- | --- | --- | --- | --- | --- | --- |
| [📥 makevalid](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_admin_0_countries_makevalid.gpkg) | 9457 KB | | | | | |
| [📥 s3_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_admin_0_countries_s3_twkb_p3.gpkg) | 921 KB | | | | | |
| [📥 s4_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_admin_0_countries_s4_twkb_p3.gpkg) | 1249 KB | | | | | |
| [📥 s5_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_admin_0_countries_s5_twkb_p3.gpkg) | 1441 KB | | | | | |
| [📥 s6_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_admin_0_countries_s6_twkb_p3.gpkg) | 1810 KB | | | | | |
| [📥 s7_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_admin_0_countries_s7_twkb_p3.gpkg) | 2146 KB | | | | | |
| [📥 s8_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_admin_0_countries_s8_twkb_p3.gpkg) | 2158 KB | | | | | |
| [📥 s3_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_admin_0_countries_s3_wkb.gpkg) | 3002 KB | | | | | |
| [📥 s4_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_admin_0_countries_s4_wkb.gpkg) | 4997 KB | | | | | |
| [📥 s5_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_admin_0_countries_s5_wkb.gpkg) | 5984 KB | | | | | |
| [📥 s6_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_admin_0_countries_s6_wkb.gpkg) | 8462 KB | | | | | |
| [📥 s7_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_admin_0_countries_s7_wkb.gpkg) | 9179 KB | | | | | |
| [📥 s8_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_admin_0_countries_s8_wkb.gpkg) | 9297 KB | | | | | |

### ne_10m_urban_areas_landscan

Big cities only, 1:10m scale sourced from [Natural Earth] ([Public Domain][ne-license]).

See [Parameters](#parameters) for what each variant means and
[Datasets](#datasets) for details on the dataset itself.

| Variant | Size | world | berlin | nyc | tokyo | ljubljana |
| --- | --- | --- | --- | --- | --- | --- |
| [📥 makevalid](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_urban_areas_landscan_makevalid.gpkg) | 48156 KB | | | | | |
| [📥 s3_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_urban_areas_landscan_s3_twkb_p3.gpkg) | 5599 KB | | | | | |
| [📥 s4_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_urban_areas_landscan_s4_twkb_p3.gpkg) | 3788 KB | | | | | |
| [📥 s5_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_urban_areas_landscan_s5_twkb_p3.gpkg) | 4333 KB | | | | | |
| [📥 s6_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_urban_areas_landscan_s6_twkb_p3.gpkg) | 6119 KB | | | | | |
| [📥 s7_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_urban_areas_landscan_s7_twkb_p3.gpkg) | 6131 KB | | | | | |
| [📥 s8_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_urban_areas_landscan_s8_twkb_p3.gpkg) | 6127 KB | | | | | |
| [📥 s3_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_urban_areas_landscan_s3_wkb.gpkg) | 30052 KB | | | | | |
| [📥 s4_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_urban_areas_landscan_s4_wkb.gpkg) | 18726 KB | | | | | |
| [📥 s5_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_urban_areas_landscan_s5_wkb.gpkg) | 21704 KB | | | | | |
| [📥 s6_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_urban_areas_landscan_s6_wkb.gpkg) | 34611 KB | | | | | |
| [📥 s7_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_urban_areas_landscan_s7_wkb.gpkg) | 34639 KB | | | | | |
| [📥 s8_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/ne_10m_urban_areas_landscan_s8_wkb.gpkg) | 34643 KB | | | | | |

### geoBoundariesCGAZ_ADM0

Country-level administrative boundaries sourced from [geoBoundaries] ([Attribution required][gb-license]).

See [Parameters](#parameters) for what each variant means and
[Datasets](#datasets) for details on the dataset itself.

| Variant | Size | world | europe | africa | usa | japan |
| --- | --- | --- | --- | --- | --- | --- |
| [📥 makevalid](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/geoBoundariesCGAZ_ADM0_makevalid.gpkg) | 157384 KB | | | | | |
| [📥 s3_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/geoBoundariesCGAZ_ADM0_s3_twkb_p3.gpkg) | 11522 KB | | | | | |
| [📥 s4_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/geoBoundariesCGAZ_ADM0_s4_twkb_p3.gpkg) | 14368 KB | | | | | |
| [📥 s5_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/geoBoundariesCGAZ_ADM0_s5_twkb_p3.gpkg) | 17473 KB | | | | | |
| [📥 s6_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/geoBoundariesCGAZ_ADM0_s6_twkb_p3.gpkg) | 18358 KB | | | | | |
| [📥 s7_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/geoBoundariesCGAZ_ADM0_s7_twkb_p3.gpkg) | 20705 KB | | | | | |
| [📥 s8_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/geoBoundariesCGAZ_ADM0_s8_twkb_p3.gpkg) | 28446 KB | | | | | |
| [📥 s3_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/geoBoundariesCGAZ_ADM0_s3_wkb.gpkg) | 29437 KB | | | | | |
| [📥 s4_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/geoBoundariesCGAZ_ADM0_s4_wkb.gpkg) | 49922 KB | | | | | |
| [📥 s5_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/geoBoundariesCGAZ_ADM0_s5_wkb.gpkg) | 67690 KB | | | | | |
| [📥 s6_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/geoBoundariesCGAZ_ADM0_s6_wkb.gpkg) | 72953 KB | | | | | |
| [📥 s7_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/geoBoundariesCGAZ_ADM0_s7_wkb.gpkg) | 89964 KB | | | | | |
| [📥 s8_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/geoBoundariesCGAZ_ADM0_s8_wkb.gpkg) | 135143 KB | | | | | |

### geoBoundariesCGAZ_ADM2

City-level administrative boundaries sourced from [geoBoundaries] ([Attribution required][gb-license]).

See [Parameters](#parameters) for what each variant means and
[Datasets](#datasets) for details on the dataset itself.

| Variant | Size | world | europe | africa | usa | japan |
| --- | --- | --- | --- | --- | --- | --- |
| [📥 makevalid](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/geoBoundariesCGAZ_ADM2_makevalid.gpkg) | 266186 KB | | | | | |
| [📥 s3_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/geoBoundariesCGAZ_ADM2_s3_twkb_p3.gpkg) | 42856 KB | | | | | |
| [📥 s4_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/geoBoundariesCGAZ_ADM2_s4_twkb_p3.gpkg) | 15691 KB | | | | | |
| [📥 s5_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/geoBoundariesCGAZ_ADM2_s5_twkb_p3.gpkg) | 16420 KB | | | | | |
| [📥 s6_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/geoBoundariesCGAZ_ADM2_s6_twkb_p3.gpkg) | 25751 KB | | | | | |
| [📥 s7_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/geoBoundariesCGAZ_ADM2_s7_twkb_p3.gpkg) | 37814 KB | | | | | |
| [📥 s8_twkb_p3](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/geoBoundariesCGAZ_ADM2_s8_twkb_p3.gpkg) | 44625 KB | | | | | |
| [📥 s3_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/geoBoundariesCGAZ_ADM2_s3_wkb.gpkg) | 232214 KB | | | | | |
| [📥 s4_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/geoBoundariesCGAZ_ADM2_s4_wkb.gpkg) | 54140 KB | | | | | |
| [📥 s5_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/geoBoundariesCGAZ_ADM2_s5_wkb.gpkg) | 52469 KB | | | | | |
| [📥 s6_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/geoBoundariesCGAZ_ADM2_s6_wkb.gpkg) | 128806 KB | | | | | |
| [📥 s7_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/geoBoundariesCGAZ_ADM2_s7_wkb.gpkg) | 208105 KB | | | | | |
| [📥 s8_wkb](https://github.com/SmilyOrg/tinygpkg-data/releases/download/v0.2.0/geoBoundariesCGAZ_ADM2_s8_wkb.gpkg) | 246050 KB | | | | | |

## Contributing

Pull requests are welcome. For major changes, please open an issue first to
discuss what you would like to change.

## License

Code and tools distributed under the MIT License. The license of the datasets follows the license of the sources used to generate them. See [LICENSE.md](LICENSE.md) for more information.

## Acknowledgements
* [Best-README-Template](https://github.com/othneildrew/Best-README-Template)
* [readme.so](https://readme.so/)

[Tiny Well-known Binary (TWKB)]: https://github.com/TWKB/Specification/blob/master/twkb.md
[GeoPackage]: http://www.geopackage.org/