{"id":15675021,"url":"https://github.com/itzmeanjan/mapz","last_synced_at":"2025-05-06T23:41:03.995Z","repository":{"id":106438281,"uuid":"180423161","full_name":"itzmeanjan/mapZ","owner":"itzmeanjan","description":"A Geospatial Application","archived":false,"fork":false,"pushed_at":"2019-04-16T12:24:49.000Z","size":20711,"stargazers_count":16,"open_issues_count":0,"forks_count":4,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-31T04:04:43.641Z","etag":null,"topics":["express","javascript","map","mapnik","nodejs","osm","python","python3","tile-generation","tile-server","tiles","tms"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/itzmeanjan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-04-09T18:05:56.000Z","updated_at":"2024-02-15T14:28:21.000Z","dependencies_parsed_at":null,"dependency_job_id":"48b42770-5975-4ea6-9532-f980489bf96c","html_url":"https://github.com/itzmeanjan/mapZ","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/itzmeanjan%2FmapZ","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/itzmeanjan%2FmapZ/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/itzmeanjan%2FmapZ/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/itzmeanjan%2FmapZ/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/itzmeanjan","download_url":"https://codeload.github.com/itzmeanjan/mapZ/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252788376,"owners_count":21804280,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["express","javascript","map","mapnik","nodejs","osm","python","python3","tile-generation","tile-server","tiles","tms"],"created_at":"2024-10-03T15:54:55.674Z","updated_at":"2025-05-06T23:41:01.149Z","avatar_url":"https://github.com/itzmeanjan.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\u003cimg src=\"/logo/logotype-horizontal.png\"\u003e\u003c/p\u003e\n\n# mapZ\nA Geospatial Application, which I'm still working on.\n\nPutting :star: might be a bit inspirational to me.\n\n# Documentation\n\n   :eyes: : So, you interested in building a mapping application, is it ?\n   \n   :eyes: : Yes\n   \n   :eyes: : Okay, I'm gonna take you through each and every step. All you need to do is to follow me. Ready ?\n   \n   :eyes: : Definitely\n   \n   \n   \n## Platform and Tools used\n   I'm going to use [*Fedora Linux*](https://getfedora.org/) for implementing this whole process.\n   \n   For database implementation I'll use *PostgreSQL*.\n   ```shell\n    postgres (PostgreSQL) 10.7\n   ```\n   \n   GeoSpatial Data to be stored and processed using *PostGIS*.\n   ```shell\n    Name         : postgis\n    Version      : 2.4.3\n   ```\n   \n   A lot of *Python* scripts are going to be used for automating tile generation and database population procedure.\n   ```shell\n   Python 3.7.3\n   ```\n   \n   *NPM* used for installing different *JavaScript* dependencies.\n   ```shell\n    \u003e\u003e npm --version\n    6.4.1\n   ```\n   \n   *Express* app to be written for implementing Tile Map Server.\n   ```shell\n    \u003e\u003e node --version\n    v10.15.0\n    \u003e\u003e npm info express\n    express@4.16.4\n   ```\n   \n   *Mapnik* used for rendering map tiles.\n   ```shell\n   \u003e\u003e mapnik --version\n    3.0.20\n   ```\n   Well that's it :smile:.\n\n## Initial SetUp\n   First thing first, you need world map data ( here we'll be using shapefiles ) to build a map of world. We'll use [GADM](https://gadm.org) as map data source. Here is the [data](https://biogeo.ucdavis.edu/data/gadm3.6/gadm36_levels_shp.zip), which we'll require.\n\n   So I've written small bash script for you to download and unzip that data. Well you're free to do that on your own too.\n\n   ```bash\n   #!/usr/bin/bash\n   # script downloads shape files from GADM, make sure you're connected to internet\n   wget https://biogeo.ucdavis.edu/data/gadm3.6/gadm36_levels_shp.zip\n   # unzips downloaded zip into multiple layered shape files, which will be later on used for inflating features into database\n   unzip gadm36_levels_shp.zip\n   ```\n   \n   Now let's setup a [PostgreSQL](https://postgresql.org/) database with [postgis](https://postgis.net/) extension enabled.\n   Make sure you've installed PostgreSQL database properly on your system. I found [it](http://www.glom.org/wiki/index.php?title=Initial_Postgres_Configuration) helpful. \n   \n   Login to PostgreSQL.\n   \n   ```bash\n    \u003e\u003e psql --username=your-user-name-for-postgresql # which is generally postgres\n   ```\n   \n   Create a database named *world_features*.\n   \n   ```bash\n    \u003e\u003e create database world_features;\n   ```\n   \n   Now simply quit i.e. logout from *psql* prompt.\n   \n   ```bash\n    \u003e\u003e \\q\n   ```\n   \n   Relogin to use newly created database.\n   \n   ```bash\n    \u003e\u003e psql --username=your-user-name-for-postgresql --dbname=world_features\n   ```\n   \n   Let's enable *postgis extension* for this database. Make sure you've installed *postgis* first.\n   \n   ```bash\n    \u003e\u003e create extension postgis;\n   ```\n   \n   And initial setup is done. Now we gonna automate things :wink:.\n   \n\n## Database Population\n   :eyes: : That shapefile we downloaded, has 6 layers. So we'll be creating 6 different tables, where we're going to store that huge map data. \n   \n   :eyes: : :worried:\n   \n   :eyes: : Hey wait, we're going to automate that whole thing. Now happy ???\n   \n   :eyes: : :relaxed:\n   \n   Alright so let's automate. And don't forget to grab a cup of :coffee:, cause this gonna be a bit longer.\n   \n   See we're going to process almost few hundreds of thousands features ( mostly polygon / multipolygon geometry ), using *geo.Open('/path-to-gadm36_0.shp').GetLayer(0).GetFeature(j).GetGeometryRef().ExportToWkt()*, where j is feature count. So of course this gonna be time consuming.\n   \n   \n   ```python\n   \n    def app(path='/path-to-file/gadm36_{}.shp', file_id=[0, 1, 2, 3, 4, 5]):\n        # path, path to gadm shapefiles\n        # gadm has 6 layers, shape files hold corresponding layer number too\n        print('[+]Now grab a cup of coffee, cause this gonna be a little longer ...\\n')\n        for i in file_id:\n            print('[+]Working on `{}`'.format(path.format(i)))\n            datasource = geo.Open(path.format(i))  # datasource opened\n            # layer fetched, only a single layer present in a shape file\n            layer = datasource.GetLayer(0)\n            tmp = []\n            for j in range(layer.GetFeatureCount()):\n                feature = layer.GetFeature(j)  # gets feature by id\n                gid = 'NA'\n                name = 'NA'\n                # there might be some fields present in shapefile, which is None\n                if(feature.items().get('GID_{}'.format(i)) is not None):\n                    # To handle so, I'm adding these two checks, otherwise those might be causing problem during database population\n                    gid = feature.items().get('GID_{}'.format(i))\n                if(feature.items().get('NAME_{}'.format(i)) is not None):\n                    name = feature.items().get('NAME_{}'.format(i))\n                tmp.append([gid, name,\n                        feature.GetGeometryRef().ExportToWkt()])\n                # holds data in temp variable\n                # data format -- [feature_id, feature_name, outline]\n            if(inflate_into_db('world_features', 'username', 'password', {i: tmp})):\n                # finally inflate into database\n                print('[+]Success')\n    return\n   ```\n   \n   Don't forget to change username and password, required for database login, before running [this](https://github.com/itzmeanjan/mapZ/blob/master/fetch_and_push.py) script.\n   \n   ```python\n    if(inflate_into_db('world_features', 'username', 'password', {i: tmp})):\n           # finally inflate into database\n           print('[+]Success')\n   ```\n   \n   Simply run [this](https://github.com/itzmeanjan/mapZ/blob/master/fetch_and_push.py) script and it'll be done.\n   \n   ```shell\n    \u003e\u003e  python3 fetch_and_push.py \n   ```\n   \n   And it's done. Wanna check ?\n   \n   Login to *postgresql*.\n   \n   ```shell\n    \u003e\u003e psql --username=your-user-name-for-postgresql --dbname=world_features\n   ```\n   \n   Type in *psql* prompt.\n   \n   ```sql\n    \u003e\u003e select feature_id, feature_name from world_features_level_0 where feature_id = 'IND';\n    feature_id | feature_name \n    ------------+--------------\n    IND        | India\n    (1 row)\n   ```\n   \n   And this is the structure of table, which the script built for us. All *6 tables* has same structure.\n   \n   ```sql\n    \u003e\u003e \\d world_features_level_0;\n                   Table \"public.world_features_level_0\"\n        Column    |       Type        | Collation | Nullable | Default \n    --------------+-------------------+-----------+----------+---------\n    feature_id   | character varying |           | not null | \n    feature_name | character varying |           | not null | \n    outline      | geography         |           |          | \n    Indexes:\n        \"world_features_level_0_pkey\" PRIMARY KEY, btree (feature_id)\n        \"world_features_level_0_index\" gist (outline)\n   ```\n   \n  \n## Tile Generation\n   This gonna be way more time consuming than previous one. I'm still working on it. I'll be back :sunglasses:.\n   \n   \n   \n## Tile Map Server\n   The *Tile Map Server*, built using [NodeJS](https://nodejs.org/en/) i.e. [ExpressJS](http://expressjs.com/) Framework resides [here](https://github.com/itzmeanjan/mapZ/tree/master/tms).\n   \n   Get into *tms* directory and run following command, which will download all dependencies, required for running this express app.\n   \n   ```shell\n      \u003e\u003e npm install\n   ```\n   \n   \n   This *Express* app will be working in local network. Make necessary changes, so that it can be discovered from Internet.\n   ```javascript\n      app.listen(8000, '0.0.0.0', () =\u003e {\n      // tms listens at 0.0.0.0:8000, so that it can be accessed via both localhost and devices present in local network\n      console.log('[+]Tile Map Server listening at - `0.0.0.0: 8000`\\n');\n   });\n   ```\n   \n   Tile Map Server will be accepting *GET* request in  */tile/:zoom/:row/:col.png*   path, where *zoom* is Zoom Level value, *row* is Row ID( tile identifier along X-axis ) and *col* is Column ID( tile identifier along Y-axis ).\n   \n   ```javascript\n      app.get('/tile/:zoom/:row/:col.png', (req, res) =\u003e {\n      . \n      . // lots of code\n      .\n      });\n   ```\n   \n   Ready to run Tile Map Server ?\n   \n   ```shell\n   \u003e\u003e  node index.js\n   ```\n   \n   As you can see on terminal its running. Just head to [this url](http://localhost:8000/tile/0/0/0.png), and you get to see a tile, which you built during that *very long* running tile generation procedure.\n   \n   \n   ![Working Tile Map Server](https://github.com/itzmeanjan/mapZ/blob/master/screenshot_1.png)\n   \n \n## Mapping Application\n   Let's first talk about server side.\n   ### Server Side\n   We gonna write an [Express](http://expressjs.com/) App, which will be serving web pages for displaying maps, and for client, will simply reply on browser, which makes this whole venture a bit platform independent.\n   \n   Mapping application resides [here](https://github.com/itzmeanjan/mapZ/tree/master/app). As I assume, you've already successfully completed previous steps, simply get into *app* directory and run *app/index.js* for launching mapping application server.\n   \n   ```shell\n      \u003e\u003e cd app\n      \u003e\u003e node index.js\n   ```\n   \n   ![mapping application server, running](https://github.com/itzmeanjan/mapZ/blob/master/screenshot_2.png)\n   \n   Back to client.\n   \n   ### Client Side\n   You might have already found that there's a [*static*](https://github.com/itzmeanjan/mapZ/tree/master/app/static) directory inside *app*, which holds a simple web page, [*index.html*](https://github.com/itzmeanjan/mapZ/blob/master/app/static/index.html), which will be served by *app/index.js*, ( express app ) and browser will be using it for displaying map with the help of a great JavaScript library [*leaflet*](https://leafletjs.com/).\n   \n   For using *leaflet* in our app, we're going to put following two tags in *head* tag of [*index.html*](https://github.com/itzmeanjan/mapZ/blob/master/app/static/index.html).\n   ```html\n   \u003clink rel=\"stylesheet\" href=\"https://unpkg.com/leaflet@1.4.0/dist/leaflet.css\"\n        integrity=\"sha512-puBpdR0798OZvTTbP4A8Ix/l+A4dHDD0DGqYW6RQ+9jxkRFclaxxQb/SJAWZfWAkuyeQUytO7+7N4QKrDh+drA==\"\n        crossorigin=\"\" /\u003e\n    \u003cscript src=\"https://unpkg.com/leaflet@1.4.0/dist/leaflet.js\"\n        integrity=\"sha512-QVftwZFqvtRNi0ZyCtsznlKSWOStnDORoefr1enyq5mVL4tmKB3S/EnC3rRJcxCPavG10IcrVGSmPh6Qw5lwrg==\"\ncrossorigin=\"\"\u003e\u003c/script\u003e\n   ```\n   \n   For displaying a map we need a *div* element with *id* attribute set, in *body* of html. You need to specify *height* for this *div* element.\n   ```html\n      \u003cstyle\u003e\n        body {\n            margin: 0;\n            padding: 0;\n        }\n        html,\n        body,\n        #map {\n            height: 100%;\n        }\n    \u003c/style\u003e\n   ```\n   ![map on client side 1](https://github.com/itzmeanjan/mapZ/blob/master/screenshot_3.png)\n   ![map on client side 2](https://github.com/itzmeanjan/mapZ/blob/master/screenshot_4.png)\n   ![map on client side 3](https://github.com/itzmeanjan/mapZ/blob/master/screenshot_5.png)\n   \n   Put this script with in *body* of html, which will display map on browser. Of course the heavy lifting is done by *leaflet*.\n   \n   ```javascript\n   \u003cscript\u003e\n        var map = L.map('map', {\n            maxZoom: 5,\n            minZoom: 0,\n        });\n        L.tileLayer('http://localhost:8000/tile/{z}/{x}/{y}.png', {\n            attribution: '\u0026copy; 2019 Anjan Roy'\n        }).addTo(map);\n        map.setView([22, 83], 3);\n    \u003c/script\u003e\n   ```\n   \n   You see, I've created a map with *maxZoom: 5* and *minZoom: 0*, which will be displayed with in a *div* element, identified by *map*, id attribute.\n   ```javascript\n      var map = L.map('map', {\n            maxZoom: 5,\n            minZoom: 0,\n        });\n   ```\n   \n   ![map on client side 4](https://github.com/itzmeanjan/mapZ/blob/master/screenshot_6.png)\n   ![map on client side 5](https://github.com/itzmeanjan/mapZ/blob/master/screenshot_7.png)\n   ![map on client side 6](https://github.com/itzmeanjan/mapZ/blob/master/screenshot_8.png)\n   \n   Next we're going to add a *tileLayer*, used for displaying tiles. And the url for *tms* is *http://localhost:8000/tile/{z}/{x}/{y}.png*, where *z* denotes Zoom Level, *x* denotes tile-id along X-axis and *y* denotes tile-id along Y-axis.\n   \n   Well, the top-left most tile is identified as *0-0* tile. After that as you move towards right, *x* increases and moving downward increases *y* value.\n   ```javascript\n      L.tileLayer('http://localhost:8000/tile/{z}/{x}/{y}.png', {\n            attribution: '\u0026copy; 2019 Anjan Roy'\n        }).addTo(map);\n        \n   ```\n   And we add tileLayer to map. and :boom: !!!\n   \n   ![map on client side 7](https://github.com/itzmeanjan/mapZ/blob/master/screenshot_9.png)\n   ![map on client side 8](https://github.com/itzmeanjan/mapZ/blob/master/screenshot_10.png)\n   ![map on client side 9](https://github.com/itzmeanjan/mapZ/blob/master/screenshot_11.png)\n   ![map on client side 10](https://github.com/itzmeanjan/mapZ/blob/master/screenshot_12.png)\n   ![map on client side 11](https://github.com/itzmeanjan/mapZ/blob/master/screenshot_13.png)\n   ![map on client side 12](https://github.com/itzmeanjan/mapZ/blob/master/screenshot_14.png)\n   \n   \n   Was it hard ???\n   \n   \n   :eyes: -- Facing some problems ?\n   \n   :eyes: -- Yes\n   \n   :eyes: -- Alright, find me [here](https://twitter.com/meanjanry)\n   \n   \n\n## Courtesy\n   Thanking [Tobaloidee](https://github.com/Tobaloidee) :heart: -ily for designing a logo for this project.\n   \n   \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fitzmeanjan%2Fmapz","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fitzmeanjan%2Fmapz","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fitzmeanjan%2Fmapz/lists"}