{"id":13734255,"url":"https://github.com/ungarj/mapchete","last_synced_at":"2025-05-15T20:02:27.850Z","repository":{"id":35255830,"uuid":"39515694","full_name":"ungarj/mapchete","owner":"ungarj","description":"Tile-based geodata processing using rasterio \u0026 Fiona","archived":false,"fork":false,"pushed_at":"2024-10-21T18:18:29.000Z","size":11386,"stargazers_count":196,"open_issues_count":51,"forks_count":28,"subscribers_count":7,"default_branch":"main","last_synced_at":"2024-10-22T04:56:38.198Z","etag":null,"topics":["batch-processor","earth-observation","earth-sciences","fiona","geoprocessing","gis","pyramid","raster","rasterio","tile","vector","web-mercator"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ungarj.png","metadata":{"files":{"readme":"README.rst","changelog":"CHANGELOG.rst","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-07-22T15:59:22.000Z","updated_at":"2024-10-16T20:21:31.000Z","dependencies_parsed_at":"2023-11-16T11:30:10.697Z","dependency_job_id":"95bb1627-a5cc-4f3e-bf4b-e6174344b335","html_url":"https://github.com/ungarj/mapchete","commit_stats":{"total_commits":1657,"total_committers":10,"mean_commits":165.7,"dds":"0.34640917320458664","last_synced_commit":"1dab3f2f160235abb28e9df6d140361977347013"},"previous_names":[],"tags_count":112,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ungarj%2Fmapchete","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ungarj%2Fmapchete/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ungarj%2Fmapchete/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ungarj%2Fmapchete/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ungarj","download_url":"https://codeload.github.com/ungarj/mapchete/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247801155,"owners_count":20998338,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["batch-processor","earth-observation","earth-sciences","fiona","geoprocessing","gis","pyramid","raster","rasterio","tile","vector","web-mercator"],"created_at":"2024-08-03T03:00:53.919Z","updated_at":"2025-04-08T08:11:07.826Z","avatar_url":"https://github.com/ungarj.png","language":"Python","readme":".. image:: logo/mapchete.svg\n\nTile-based geodata processing.\n\n.. image:: https://badge.fury.io/py/mapchete.svg\n    :target: https://badge.fury.io/py/mapchete\n\n.. image:: https://github.com/ungarj/mapchete/actions/workflows/python-package.yml/badge.svg\n    :target: https://github.com/ungarj/mapchete/actions\n\n.. image:: https://readthedocs.org/projects/mapchete/badge/?version=latest\n    :target: http://mapchete.readthedocs.io/en/latest/?badge=latest\n    :alt: Documentation Status\n\n.. image:: https://img.shields.io/pypi/pyversions/mapchete.svg\n    :target: https://pypi.python.org/pypi/mapchete\n\nMapchete processes raster and vector geodata in digestable chunks.\n\nProcessing larger amounts of data requires chunking the input data into smaller tiles\nand process them one by one. Python provides a lot of useful packages to process geodata\nlike shapely_ or numpy_. From within your process code you will have access to the geodata\nin the form of ``NumPy`` arrays for raster data or GeoJSON-like feature dictionaries for\nvector data.\n\nInternally the processing job is split into tasks which can processed in parallel using either\n``concurrent.futures`` or build task graphs and use dask_ to intelligently process either on\nthe local machine or on a cluster.\n\nWith the help of fiona_ and rasterio_ Mapchete takes care about resampling and\nreprojecting geodata, applying your Python code to the tiles and writing the output either\ninto a single file or into a directory of files organized in a WMTS_-like tile pyramid.\nDetails on tiling scheme and available map projections are outlined in the\n`tiling documentation`_.\n\n.. _shapely: http://toblerity.org/shapely/\n.. _numpy: http://www.numpy.org/\n.. _dask: https://www.dask.org/\n.. _fiona: https://github.com/Toblerity/Fiona\n.. _rasterio: https://github.com/mapbox/rasterio/\n.. _WMTS: https://en.wikipedia.org/wiki/Web_Map_Tile_Service\n.. _`tiling documentation`: https://mapchete.readthedocs.io/en/latest/tiling.html\n\n\n.. figure:: https://mapchete.readthedocs.io/en/latest/_images/mercator_pyramid.svg\n   :align: center\n   :target: https://mapchete.readthedocs.io/en/latest/tiling.html\n\n   (standard Web Mercator pyramid used in the web)\n\n\n-----\nUsage\n-----\n\nYou need a ``.mapchete`` file for the process configuration. The configuration is based\non the ``YAML`` syntax.\n\n.. code-block:: yaml\n\n    process: my_python_process.py  # or a Python module path: mypythonpackage.myprocess\n    zoom_levels:\n        min: 0\n        max: 12\n    input:\n        dem: /path/to/dem.tif\n        land_polygons: /path/to/polygon/file.geojson\n    output:\n        format: PNG_hillshade\n        path: /output/path\n    pyramid:\n        grid: mercator\n\n    # process specific parameters\n    resampling: cubic_spline\n\n\nYou also need either a ``.py`` file or a Python module path where you specify the process\nitself.\n\n.. code-block:: python\n\n    def execute(mp, resampling=\"nearest\"):\n\n        # Open elevation model.\n        with mp.open(\"dem\") as src:\n            # Skip tile if there is no data available or read data into a NumPy array.\n            if src.is_empty(1):\n                return \"empty\"\n            else:\n                dem = src.read(1, resampling=resampling)\n\n        # Create hillshade using a built-in hillshade function.\n        hillshade = mp.hillshade(dem)\n\n        # Clip with polygons from vector file and return result.\n        with mp.open(\"land_polygons\") as land_file:\n            return mp.clip(hillshade, land_file.read())\n\n\nYou can then interactively inspect the process output directly on a map in a\nbrowser (first, install dependencies by ``pip install mapchete[serve]`` go to\n``localhost:5000``):\n\n.. code-block:: shell\n\n    $ mapchete serve hillshade.mapchete --memory\n\n\nThe ``serve`` tool recognizes changes in your process configuration or in the\nprocess file. If you edit one of these, just refresh the browser and inspect the\nchanges (note: use the ``--memory`` flag to make sure to reprocess each tile and\nturn off browser caching).\n\nOnce you are done with editing, batch process everything using the ``execute``\ntool.\n\n.. code-block:: shell\n\n    $ mapchete execute hillshade.mapchete\n\n\n-------------\nDocumentation\n-------------\n\nThere are many more options such as zoom-dependent process parameters, metatiling, tile\nbuffers or interpolating from an existing output of a higher zoom level. For deeper\ninsights, please go to the documentation_.\n\n.. _documentation: http://mapchete.readthedocs.io/en/latest/index.html\n\nMapchete is used in many preprocessing steps for the `EOX Maps`_ layers:\n\n* Merge multiple DEMs into one global DEM.\n* Create a customized relief shade for the Terrain Layer.\n* Generalize landmasks \u0026 coastline from OSM for multiple zoom levels.\n* Extract cloudless pixel for Sentinel-2 cloudless_.\n\n.. _cloudless: https://cloudless.eox.at/\n.. _`EOX Maps`: http://maps.eox.at/\n\n\n------------\nInstallation\n------------\n\nvia PyPi:\n\n.. code-block:: shell\n\n    $ pip install mapchete\n\n\nfrom source:\n\n.. code-block:: shell\n\n    $ git clone git@github.com:ungarj/mapchete.git \u0026\u0026 cd mapchete\n    $ pip install .\n\n\n\nTo make sure Rasterio, Fiona and Shapely are properly built against your local GDAL and\nGEOS installations, don't install the binaries but build them on your system:\n\n.. code-block:: shell\n\n    $ pip install --upgrade rasterio fiona shapely --no-binary :all:\n\n\nTo keep the core dependencies minimal if you install mapchete using ``pip``, some features\nare only available if you manually install additional dependencies:\n\n.. code-block:: shell\n\n    # for contour extraction:\n    $ pip install mapchete[contours]\n\n    # for dask processing:\n    $ pip install mapchete[dask]\n\n    # for S3 bucket reading and writing:\n    $ pip install mapchete[s3]\n\n    # for mapchete serve:\n    $ pip install mapchete[serve]\n\n    # for VRT generation:\n    $ pip install mapchete[vrt]\n\n\n-------\nLicense\n-------\n\nMIT License\n\nCopyright (c) 2015 - 2022 `EOX IT Services`_\n\n.. _`EOX IT Services`: https://eox.at/\n","funding_links":[],"categories":["`Python` processing of optical imagery (non deep learning)","Python"],"sub_categories":["Processing imagery - post processing"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fungarj%2Fmapchete","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fungarj%2Fmapchete","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fungarj%2Fmapchete/lists"}