{"id":28956813,"url":"https://github.com/developmentseed/tilebench","last_synced_at":"2025-06-23T21:40:55.812Z","repository":{"id":42003259,"uuid":"266191680","full_name":"developmentseed/tilebench","owner":"developmentseed","description":"Inspect Rasterio/GDAL HEAD/GET/LIST Requests ","archived":false,"fork":false,"pushed_at":"2025-04-30T08:10:42.000Z","size":142,"stargazers_count":38,"open_issues_count":2,"forks_count":4,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-06-04T07:35:25.500Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/developmentseed.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-05-22T19:28:43.000Z","updated_at":"2025-05-14T10:52:07.000Z","dependencies_parsed_at":"2023-01-28T15:30:45.738Z","dependency_job_id":"ba6191e1-7740-4a50-af02-4b875e075189","html_url":"https://github.com/developmentseed/tilebench","commit_stats":{"total_commits":79,"total_committers":3,"mean_commits":"26.333333333333332","dds":0.05063291139240511,"last_synced_commit":"3d2425cff8ef58bedc5b74577bcd271f17ac9116"},"previous_names":[],"tags_count":25,"template":false,"template_full_name":null,"purl":"pkg:github/developmentseed/tilebench","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/developmentseed%2Ftilebench","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/developmentseed%2Ftilebench/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/developmentseed%2Ftilebench/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/developmentseed%2Ftilebench/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/developmentseed","download_url":"https://codeload.github.com/developmentseed/tilebench/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/developmentseed%2Ftilebench/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261561180,"owners_count":23177543,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-23T21:40:55.543Z","updated_at":"2025-06-23T21:40:55.792Z","avatar_url":"https://github.com/developmentseed.png","language":"Python","readme":"# tilebench\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://user-images.githubusercontent.com/10407788/145784365-7ce635d1-3971-4a94-856f-cd8b081f10c1.png\" style=\"max-width: 800px;\" alt=\"tilebench\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n  \u003cem\u003eInspect HEAD/LIST/GET requests within Rasterio\u003c/em\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/developmentseed/tilebench/actions?query=workflow%3ACI\" target=\"_blank\"\u003e\n      \u003cimg src=\"https://github.com/developmentseed/tilebench/workflows/CI/badge.svg\" alt=\"Test\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://codecov.io/gh/developmentseed/tilebench\" target=\"_blank\"\u003e\n      \u003cimg src=\"https://codecov.io/gh/developmentseed/tilebench/branch/main/graph/badge.svg\" alt=\"Coverage\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://pypi.org/project/tilebench\" target=\"_blank\"\u003e\n      \u003cimg src=\"https://img.shields.io/pypi/v/tilebench?color=%2334D058\u0026label=pypi%20package\" alt=\"Package version\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://pypistats.org/packages/tilebench\" target=\"_blank\"\u003e\n      \u003cimg src=\"https://img.shields.io/pypi/dm/tilebench.svg\" alt=\"Downloads\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/developmentseed/tilebench/blob/main/LICENSE\" target=\"_blank\"\u003e\n      \u003cimg src=\"https://img.shields.io/github/license/developmentseed/tilebench.svg\" alt=\"Downloads\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n---\n\n**Source Code**: \u003ca href=\"https://github.com/developmentseed/tilebench\" target=\"_blank\"\u003ehttps://github.com/developmentseed/tilebench\u003c/a\u003e\n\n---\n\n\nInspect HEAD/GET requests withing Rasterio.\n\nNote: In GDAL 3.2, logging capabilities for /vsicurl, /vsis3 and the like was added (ref: https://github.com/OSGeo/gdal/pull/2742).\n\n## Install\n\nYou can install `tilebench` using pip\n\n```bash\n$ python -m pip install -U pip\n$ python -m pip install -U tilebench\n```\n\nor install from source:\n\n```bash\ngit clone https://github.com/developmentseed/tilebench.git\ncd tilebench\n\npython -m pip install -U pip\npython -m pip install -e .\n```\n\n## API\n\n```python\nfrom tilebench import profile\nimport rasterio\n\n@profile()\ndef info(src_path: str):\n    with rasterio.open(src_path) as src_dst:\n        return src_dst.meta\n\nmeta = info(\"https://noaa-eri-pds.s3.amazonaws.com/2022_Hurricane_Ian/20221002a_RGB/20221002aC0795145w325100n.tif\")\n\n\u003e 2023-10-18T23:00:11.184745+0200 | TILEBENCH | {\"HEAD\": {\"count\": 1}, \"GET\": {\"count\": 1, \"bytes\": 32768, \"ranges\": [\"0-32767\"]}, \"Timing\": 0.7379939556121826}\n```\n\n```python\nfrom tilebench import profile\nfrom rio_tiler.io import Reader\n\n@profile()\ndef _read_tile(src_path: str, x: int, y: int, z: int, tilesize: int = 256):\n    with Reader(src_path) as cog:\n        return cog.tile(x, y, z, tilesize=tilesize)\n\nimg = _read_tile(\n    \"https://noaa-eri-pds.s3.amazonaws.com/2022_Hurricane_Ian/20221002a_RGB/20221002aC0795145w325100n.tif\",\n    9114,\n    13216,\n    15,\n)\n\n\u003e 2023-10-18T23:01:00.572263+0200 | TILEBENCH | {\"HEAD\": {\"count\": 1}, \"GET\": {\"count\": 2, \"bytes\": 409600, \"ranges\": [\"0-32767\", \"32768-409599\"]}, \"Timing\": 1.0749869346618652}\n```\n\n## Command Line Interface (CLI)\n\n```\n$ tilebench --help\nUsage: tilebench [OPTIONS] COMMAND [ARGS]...\n\n  Command line interface for the tilebench Python package.\n\nOptions:\n  --help  Show this message and exit.\n\nCommands:\n  get-zooms  Get Mercator Zoom levels.\n  profile    Profile COGReader Mercator Tile read.\n  random     Get random tile.\n  viz        WEB UI to visualize VSI statistics for a web mercator tile request\n```\n\n#### Examples\n```\n$ tilebench get-zooms https://noaa-eri-pds.s3.amazonaws.com/2022_Hurricane_Ian/20221002a_RGB/20221002aC0795145w325100n.tif | jq\n{\n  \"minzoom\": 14,\n  \"maxzoom\": 19\n}\n\n$ tilebench random https://noaa-eri-pds.s3.amazonaws.com/2022_Hurricane_Ian/20221002a_RGB/20221002aC0795145w325100n.tif --zoom 15\n15-9114-13215\n\n$ tilebench profile https://noaa-eri-pds.s3.amazonaws.com/2022_Hurricane_Ian/20221002a_RGB/20221002aC0795145w325100n.tif --tile 15-9114-13215 --config GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR | jq\n{\n  \"HEAD\": {\n    \"count\": 1\n  },\n  \"GET\": {\n    \"count\": 2,\n    \"bytes\": 409600,\n    \"ranges\": [\n      \"0-32767\",\n      \"32768-409599\"\n    ]\n  },\n  \"Timing\": 0.9715230464935303\n}\n\n$ tilebench profile https://noaa-eri-pds.s3.amazonaws.com/2022_Hurricane_Ian/20221002a_RGB/20221002aC0795145w325100n.tif --tile 15-9114-13215 --config GDAL_DISABLE_READDIR_ON_OPEN=FALSE | jq\n{\n  \"HEAD\": {\n    \"count\": 8\n  },\n  \"GET\": {\n    \"count\": 3,\n    \"bytes\": 409600,\n    \"ranges\": [\n      \"0-32767\",\n      \"32768-409599\"\n    ]\n  },\n  \"Timing\": 2.1837549209594727\n}\n```\n\n\n## Starlette Middleware\n\n**Warning**: This is highly experimental and should not be used in production (https://github.com/developmentseed/tilebench/issues/6)\n\nIn addition of the `viz` CLI we added a starlette middleware to easily integrate VSI statistics in your web services.\n\n```python\nfrom fastapi import FastAPI\n\nfrom tilebench.middleware import VSIStatsMiddleware\n\napp = FastAPI()\napp.add_middleware(VSIStatsMiddleware)\n```\n\nThe middleware will add a `vsi-stats` entry in the response `headers` in form of:\n\n```\nvsi-stats: list;count=1, head;count=1, get;count=2;size=196608, ranges; values=0-65535|65536-196607\n```\n\nSome paths may be excluded from being handeld by the middleware by the `exclude_paths` argument:\n\n```python\napp.add_middleware(VSIStatsMiddleware, exclude_paths=[\"/foo\", \"/bar\"])\n```\n\n## GDAL config options\n\n- **CPL_TIMESTAMP**: Add timings on GDAL Logs\n- **GDAL_DISABLE_READDIR_ON_OPEN**: Allow or Disable listing of files in the directory (e.g external overview)\n- **GDAL_INGESTED_BYTES_AT_OPEN**: Control how many bytes GDAL will ingest when opening a dataset (useful when a file has a big header)\n- **CPL_VSIL_CURL_ALLOWED_EXTENSIONS**: Limit valid external files\n- **GDAL_CACHEMAX**: Cache size\n- **GDAL_HTTP_MERGE_CONSECUTIVE_RANGES**\n- **VSI_CACHE**\n- **VSI_CACHE_SIZE**\n\nSee the full list at https://gdal.org/user/configoptions.html\n\n## Internal tiles Vs Mercator grid\n\n```\n$ tilebench viz https://noaa-eri-pds.s3.amazonaws.com/2022_Hurricane_Ian/20221002a_RGB/20221002aC0795145w325100n.tif --config GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR\n```\n\n![](https://user-images.githubusercontent.com/10407788/103528918-17180880-4e85-11eb-91b3-d60659b15e80.png)\n\nBlue lines represent the mercator grid for a specific zoom level and the red lines represent the internal tiles bounds\n\nWe can then click on a mercator tile and see how much requests GDAL/RASTERIO does.\n\n![](https://user-images.githubusercontent.com/10407788/103529132-65c5a280-4e85-11eb-96e2-f59e915c8ed8.png)\n\n## Docker\n\nReady to use docker image can be found on Github registry.\n\n- https://github.com/developmentseed/tilebench/pkgs/container/tilebench\n\n```bash\ndocker run \\\n  --volume \"$PWD\":/data \\\n  --platform linux/amd64 \\\n  --rm -it -p 8080:8080 ghcr.io/developmentseed/tilebench:latest \\\n  tilebench viz --host 0.0.0.0 https://noaa-eri-pds.s3.us-east-1.amazonaws.com/2020_Nashville_Tornado/20200307a_RGB/20200307aC0865700w360900n.tif\n```\n\n## Contribution \u0026 Development\n\nSee [CONTRIBUTING.md](https://github.com/developmentseed/tilebench/blob/main/CONTRIBUTING.md)\n\n## License\n\nSee [LICENSE](https://github.com//developmentseed/tilebench/blob/main/LICENSE)\n\n## Authors\n\nSee [contributors](https://github.com/developmentseed/tilebench/graphs/contributors) for a listing of individual contributors.\n\n## Changes\n\nSee [CHANGES.md](https://github.com/developmentseed/tilebench/blob/main/CHANGES.md).\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevelopmentseed%2Ftilebench","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdevelopmentseed%2Ftilebench","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevelopmentseed%2Ftilebench/lists"}