{"id":22485843,"url":"https://github.com/makina-maps/makina-maps","last_synced_at":"2025-08-02T18:33:56.659Z","repository":{"id":40653483,"uuid":"216843469","full_name":"makina-maps/makina-maps","owner":"makina-maps","description":"Full Stack to Build, Serve and Update your own Vector and Raster Tiles from OpenStreetMap Data.","archived":false,"fork":false,"pushed_at":"2023-03-14T20:53:46.000Z","size":4164,"stargazers_count":94,"open_issues_count":17,"forks_count":16,"subscribers_count":16,"default_branch":"master","last_synced_at":"2024-04-14T12:47:59.491Z","etag":null,"topics":["docker","mapbox-gl","openmaptiles","openmaptiles-schema","openstreetmap","vector-tiles"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/makina-maps.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-10-22T15:10:09.000Z","updated_at":"2024-03-29T16:38:11.000Z","dependencies_parsed_at":"2023-01-29T04:30:54.641Z","dependency_job_id":null,"html_url":"https://github.com/makina-maps/makina-maps","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/makina-maps%2Fmakina-maps","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/makina-maps%2Fmakina-maps/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/makina-maps%2Fmakina-maps/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/makina-maps%2Fmakina-maps/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/makina-maps","download_url":"https://codeload.github.com/makina-maps/makina-maps/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":228500023,"owners_count":17929994,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","mapbox-gl","openmaptiles","openmaptiles-schema","openstreetmap","vector-tiles"],"created_at":"2024-12-06T17:13:20.729Z","updated_at":"2024-12-06T17:15:07.617Z","avatar_url":"https://github.com/makina-maps.png","language":"JavaScript","funding_links":[],"categories":["JavaScript"],"sub_categories":[],"readme":"# Makina Maps\n\nFull Stack to Build, Serve and Update your own Vector and Raster Tiles from OpenStreetMap Data.\n\nMakina Maps render tiles on request, no need to pre-generate all tiles on huge MBTiles archive: fast setup, fast update.\n\n* Build Vector Tiles on request from the [OpenMapTiles](https://github.com/openmaptiles/openmaptiles) database and schema\n* Served Mapbox GL Styles, with sprites and fonts with embed [TileServer GL](https://github.com/maptiler/tileserver-gl)\n* Render Raster version of Mapbox GL style (TileServer GL)\n* Cache vector and raster tiles (NGINX)\n* Update from OpenStreetMap\n\n![screen](screen.jpeg)\n\n## Install\n\nInstall as system dependencies: git, make, docker and docker-compose.\n\n### Clone Git reposittory\n\nGet the project:\n```\ngit clone --recurse-submodules https://github.com/makina-maps/makina-maps.git\ncd makina-maps\n```\n\nGet default GL Json Styles and Fonts, or use your owns:\n```\ngit clone -b gh-pages https://github.com/openmaptiles/osm-bright-gl-style.git tileserver-gl/styles/osm-bright-gl-style\ngit clone -b gh-pages https://github.com/openmaptiles/klokantech-basic-gl-style.git tileserver-gl/styles/klokantech-basic-gl-style\ngit clone -b gh-pages https://github.com/makinacorpus/cassini-gl-style.git tileserver-gl/styles/cassini-gl-style\ngit clone -b gh-pages https://github.com/openmaptiles/fonts.git tileserver-gl/fonts\n```\n\n### Setup Docker images\n\nBuild and Fetch docker images:\n```\ndocker-compose build\ncd openmaptiles\ndocker-compose pull\n```\n\n\n## Import OpenStreetMap Data\n\n### OpenMapTiles initial Load\n\nUse the management scripts from `openmaptiles` directory.\n```\ncd openmaptiles\n```\n\nTo reset from a previous install and erase database, run:\n```\ndocker-compose down -v\n```\n\nImport generic data, not from OpenStreetMap:\n```\n../scripts/10-import-generic.sh\n```\n\nPrepare import by download OpenStreetMap data and setup configuration for an area. The area names are from [Geofabrik](http://download.geofabrik.de/), store it in the `data` directory:\n```\n../scripts/20-import-prepare.sh andorra\n```\n\nImport the OpenStreetMap extract:\n```\n../scripts/30-import-extract.sh\n```\n\nThe scripts `20-import-prepare.sh` or `30-import-extract.sh` can be replayed with the same or other area.\n\n### Update OpenMapTiles data\n\nFrom the `openmaptiles` directory.\n\nRun the updater. It loops over pending updates, then wait for new update.\n```\n../scripts/40-update.sh\n```\nYou can stop with CTRL-C, it will quit at the end of the current update.\n\nImposm marks tiles to expire. Then a script in the nginx container watches and expires tiles in the nginx cache.\n\n\n## Run the tiles server\n\nTo reset the tile server from a previous install (does not touch the OpenMapTiles database), run:\n```\ndocker-compose down -v\n```\n\nFrom root directory. Start the OpenMapTiles database and the web server.\n```\ncd openmaptiles \u0026\u0026 docker-compose up -d postgres postserve \u0026\u0026 cd ..\ndocker-compose up\n```\n\nAccess to cached tiles and services at the following URLs - assuming your are hosting locally:\n\n* Demo: http://127.0.0.1:8080\n* OpenMapTiles TileJson: http://127.0.0.1:8080/data/v3.json\n* Default \"Bright\" GL JSON Style: http://127.0.0.1:8080/styles/bright/style.json\n* Default \"Bright\" raster:\n  * TileJSON: http://127.0.0.1:8080/styles/bright.json\n  * Raster tiles: http://127.0.0.1:8080/styles/bright/{z}/{x}/{y}.png\n\n## Configuration\n\nConfiguration file is Tileserver-GL configuration file `tileserver-gl/config.json`.\n\nGet more detail about this configuration at the [Tileserver-GL documentation](https://tileserver.readthedocs.io/en/latest/config.html).\n\n### Styles\n\n```js\n  \"styles\": {\n    \"bright\": {\n      // Path to the Mapbox-GL Style\n      \"style\": \"osm-bright-gl-style/style-local.json\"\n    },\n    \"basic\": {\n      // Disable the render as raster (no PNG...)\n      \"serve_rendered\": false,\n      \"style\": \"klokantech-basic-gl-style/style-local.json\"\n    }\n  },\n```\n\n### Vector tiles data sources\n\nWhere the vector tiles are come from. Can be generated on the fly from OpenMapTiles database, other external source or local MBTiles files.\n```js\n  \"data\": {\n    // Name of the source, here OpenMapTiles v3\n    \"v3\": {\n      // On demand tiles, internal URL\n      // Alternatively could be a MBTiles\n      // This option is specific to Makina Maps and not available in standard Tileserver-GL\n      \"remote_tilejson\": \"http://postserve:8090/\"\n      }\n    }\n  }\n```\n\n### Public URLs\n\nThe `domains` may be adjusted. There used only for raster tiles.\n\n```js\n    \"domains\": [\n      // Publicly available domains, for raster tiles\n      \"a.example.com\",\n      \"b.example.com\"\n    ],\n```\n\nThe vector tiles source (tilejon) should be overwritten to expose public URLs.\n```js\n  \"data\": {\n    \"v3\": {\n      \"remote_tilejson\": \"http://postserve:8090/\",\n      \"tilejson\": {\n        \"tiles\": [\n          // Public available URLs of vector tiles\n          \"http://a.example.com/data/v3/{z}/{x}/{y}.pbf\",\n          \"http://b.example.com/data/v3/{z}/{x}/{y}.pbf\"\n        ]\n      }\n    }\n  }\n```\n\nThe tiles URLs in tilejson from vector tiles producer are internal to the docker-compose and refer to internal host. Should be redefined as public URLs for outside world.\n\n### Tiles cache\n\nEnvironment variable define the size of the cache of tiles.\n```yaml\n      - CACHE_KEYS_ZONE_SIZE=50m # 8000 tile keys per 1m, in memory\n      - CACHE_MAX_SIZE=20g # File storage\n```\n\n### Download Tiles Rate Limit\n\nThere is a rate limit for typical browser usage in NGINX configuration.\n\nThe rate limit can be increased using an URL parameter `key` or by host white list. It is mainly to allow tiles download at higher speed. Should be configured in `nginx/map`\n\n```\n\"this-secret-key-allows-download-at-highspeed\" \"\";\n```\n\n\n## Overview\n\n\u003cimg src=\"architecture.svg\" alt=\"drawing\" width=\"500\"/\u003e\n\nArchitecture overview, workflow.\n\n```ascii-art\n               Nginx                  OpenMapTiles\n               Cache   TileServer-GL    postserve\n\ntilejson \u003c------ X \u003c-------- X \u003c----------- X\n                                            |\n                                   ,--------'\n                                   |\npbf tiles \u003c----- X \u003c---------------|------- X\n                 |                 |\n                  `--------\u003e X \u003c---'\n                             |\npng tiles \u003c----- X \u003c---------'\n```\n\n\n## Development\n\nThe NGINX cache can be disabled using:\n```\nNGINX_DISABLE_CACHE=1 docker-compose up\n```\n\n\n## Monitoring\n\nNGINX server status is available on localhost at `http://127.0.0.1:8082/nginx_status`.\n\n\n## Benchmark\n\n### Import size and time\n\nSpecific only on 8 CPUs (import-osm, import-sql and psql-analyze, without docker pulling time).\n\n| Area      | PBF size | Imposm cache | Postgres size | Time 8 CPUs / SSD | 1 d Update |\n| --------- | -------: | -----------: | ------------: | ----------------: | ---------: |\n| Andorra   |   1.6 MB |       3.5 MB |        167 MB |        1 min 11 s |            |\n| Alsace    |   103 MB |       297 MB |        1.2 GB |        3 min 35 s |            |\n| Aquitaine |   220 MB |       533 MB |        2.6 GB |        6 min 32 s |      1 min |\n| Austria   |   582 MB |       1.1 GB |        6.4 GB |            13 min |   2 min 50 |\n| France    |   3.6 GB |       6.5 GB |         38 GB |            99 min |      5 min |\n| Europe    |    20 Go |        35 Go |        206 Go |       11 h 50 min |        2 h |\n\nWithout SSD hardware it will be many times slower.\n\n### Database\n\nSize of Imposm cache.\n```\ndocker-compose run --rm openmaptiles-tools bash -c \"du -h /cache/\"\n```\n\nSize of the current database.\n```\ndocker-compose exec postgres psql openmaptiles openmaptiles -c \"\nSELECT\n  pg_size_pretty(sum(pg_relation_size(pg_catalog.pg_class.oid))::bigint) as table_size\nFROM\n  pg_catalog.pg_class\n  JOIN pg_catalog.pg_namespace ON\n    relnamespace = pg_catalog.pg_namespace.oid\nWHERE\n  pg_catalog.pg_namespace.nspname = 'public';\n\"\n```\n\nShow slow queries\n```sql\nALTER DATABASE openmaptiles SET log_min_duration_statement = 100;\n```\n\nAnd grep slowest queries\n```\ndocker logs openmaptiles_postgres_1 |\u0026 grep 'LOG:  duration:' | cut -d ':' -f 3- | sed 's/BOX3D([^)]*)//g' | sort -n\n```\n\n### Metrics\n\nServer metrics could be available on StatsD / Graphite on http://localhost:8899\n\nThe metrics logs are not used by default and could be enabled by adding `docker-compose-benchmark.yml` to docker-compose and uncommenting the `metrics` section from `config.yaml`.\n\n\n### Queries\n\nUse [Artillery.io](https://artillery.io) to benchmark the tiles server. Use `docker-compose-benchmark.yml`.\n\nGenerate one set of tiles coordinates to be requested. Coordinates are tile ranges at zoom level 14 (https://www.maptiler.com/google-maps-coordinates-tile-bounds-projection/)\n```\n# Continantal Europe\ndocker-compose -f docker-compose.yml -f docker-compose-benchmark.yml run --rm artillery bash -c 'ruby artillery.rb 8445-9451 5356-5891 | egrep \"^14,\" \u003e artillery.csv'\n# Aquitaine\ndocker-compose -f docker-compose.yml -f docker-compose-benchmark.yml run --rm artillery bash -c 'ruby artillery.rb 8000-8200 5800-6000 | egrep \"^14,\" \u003e artillery.csv'\n# Bordeaux area\ndocker-compose -f docker-compose.yml -f docker-compose-benchmark.yml run --rm artillery bash -c 'ruby artillery.rb 8157-8171 5895-5909 | egrep \"^14,\" \u003e artillery.csv'\n# Paris area\ndocker-compose -f docker-compose.yml -f docker-compose-benchmark.yml run --rm artillery bash -c 'ruby artillery.rb 8285-8311 5621-5645 | egrep \"^14,\" \u003e artillery.csv'\n```\n\nClear the tiles cache first. Then request the tiles server.\n```\ndocker-compose down --volumes\ndocker-compose -f docker-compose.yml -f docker-compose-benchmark.yml run --rm artillery artillery run artillery.yaml\n```\n\nRandom order tiles request on mixed urban and rural area, without concurrency. Time from server, HTTP included.\n\n| Source                |  Delay |\n| --------------------- | -----: |\n| Zoom 12 mixte Europe  |  50 ms |\n| Zoom 13, mixte Europe |  49 ms |\n| Zoom 14, mixte Europe |  60 ms |\n| Zoom 14, urban Paris  | 250 ms |\n| From cache            |   5 ms |\n****\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmakina-maps%2Fmakina-maps","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmakina-maps%2Fmakina-maps","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmakina-maps%2Fmakina-maps/lists"}