{"id":28428228,"url":"https://github.com/fladi/celery-tiles","last_synced_at":"2025-07-04T15:31:47.724Z","repository":{"id":11531889,"uuid":"14016032","full_name":"fladi/celery-tiles","owner":"fladi","description":"Distributed tile renderer using celery tasks","archived":false,"fork":false,"pushed_at":"2013-11-13T08:12:17.000Z","size":148,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-06-05T12:42:02.278Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fladi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-10-31T12:19:20.000Z","updated_at":"2022-06-09T19:50:39.000Z","dependencies_parsed_at":"2022-08-29T05:20:28.690Z","dependency_job_id":null,"html_url":"https://github.com/fladi/celery-tiles","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/fladi/celery-tiles","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fladi%2Fcelery-tiles","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fladi%2Fcelery-tiles/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fladi%2Fcelery-tiles/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fladi%2Fcelery-tiles/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fladi","download_url":"https://codeload.github.com/fladi/celery-tiles/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fladi%2Fcelery-tiles/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263567807,"owners_count":23481581,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-05T12:37:45.162Z","updated_at":"2025-07-04T15:31:47.713Z","avatar_url":"https://github.com/fladi.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"celery-tiles\n============\n\ncelery-tiles is a distributed tile renderer using celery tasks. It provides a\nCLI interface and Django command integration to validate and reproject raster\ninput that can be read by GDAL (see [GDAL inputs\nformats](http://www.gdal.org/formats_list.html)).\n\nThe target [SRS](https://en.wikipedia.org/wiki/Spatial_reference_system) for the\nreprojection is [EPSG:3857](http://spatialreference.org/ref/sr-org/6864/),\nmaking the tiles useable for map viewers like GoogleMaps or OpenLayers.\n\nRequirements\n------------\n\nMandatory:\n\n* GDAL (\u003e= 1.9.2)\n* Celery (\u003e= 3.0.23)\n\nOptional:\n\n* pngnq (\u003e= 1.0)\n\nHow it works\n------------\n\nThe CLI or Django command expects the path to a file readable by GDAL. If this\nfile exists, spatial validations are applied and a\n[VRT](http://www.gdal.org/ogr/drv_vrt.html) that is reprojected to EPSG:3857 is\ncreated. The maximum and minimum zoom levels are calculated and the necessary\ntiles for each zoom level are calculated. For each tile a celery task is\ndispached that carries the parameters necessary to render a single tile.\n\nEach worker then starts picking up the tasks and renders the tile from the VRT\nfile. The inputfile is only passed as an absolut path, so there should be no\nproblem for the worker to access the VRT file that was created by the CLI or\nDjango command. If the celery workers are distributed across several nodes they\nneed a way to access the input file over shared storage.\n\nIf the option `--output=\u003cdir\u003e` is used, the tiles are stored in the given\ndirectory. If it is not specified, the tiles are stored in a directory that is\nin the same path as the input file and has the extention `.tiles`.\n\nThe tiles for the input file `foo/bar.vrt` would be stored in\n`foo/bar.vrt.tiles/`.\n\nThe directory structure beneath the output directory follows the\n[TMS](http://en.wikipedia.org/wiki/Tile_Map_Service) convention:\n\n    \u003czoom\u003e/\u003cx\u003e/\u003cy\u003e.\u003cext\u003e\n\nUsing distributed celery workers\n--------------------------------\n\nIn order to utilize celery workers that reside on distributed systems, it is\nneccessary to give them access to the inputfile and prefferably to the output\ndirectory where all tiles are to be stored.\n\nThis has been tested with [GlusterFS](http://www.gluster.org/) but it should\nalso be possible to provide access with\n[NFS](https://en.wikipedia.org/wiki/Network_File_System),\n[OCFS2](https://en.wikipedia.org/wiki/Ocfs2) or\n[Ceph](https://en.wikipedia.org/wiki/Ceph_%28storage%29).\n\nOutput\n------\n\nThe default output format is\n[PNG](https://en.wikipedia.org/wiki/Portable_Network_Graphics) but other formats\nsupported by GDAL are also possible. If the output format is PNG, optimizations\nare applied to the tile by passing it through `pngnq`, which, most of the time,\ncan decrease the size of the tile image by a significant amount. no optimization\nis done for other output formats.\n\nCLI\n---\n\ncelery_tiles \u003cinputfile\u003e\n\nDjango\n------\n\ndjango-admin tiles \u003cinputfile\u003e\n\nAcknowledgment\n--------------\n\nMost of the validation and rendering code is heavily based on `gdal2tiles.py`\nthat ships with the GDAL distribution and was written by [Klokan Petr\nPridal](http://www.klokan.cz/projects/gdal2tiles/) \u003cklokan@klokan.cz\u003e.\n\nTODO\n----\n\n* Allow different resampling algorithms\n* Provide a way to track progress\n* Improve documentation\n* Error handling for failed tasks\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffladi%2Fcelery-tiles","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffladi%2Fcelery-tiles","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffladi%2Fcelery-tiles/lists"}