{"id":16272141,"url":"https://github.com/progamergov/blended-tiling","last_synced_at":"2025-06-30T23:03:16.732Z","repository":{"id":36999432,"uuid":"503953108","full_name":"ProGamerGov/blended-tiling","owner":"ProGamerGov","description":"A seamless / blended tiling module for PyTorch, capable of blending any 4D NCHW tensors together","archived":false,"fork":false,"pushed_at":"2024-12-18T22:34:57.000Z","size":641,"stargazers_count":28,"open_issues_count":0,"forks_count":4,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-30T23:03:11.498Z","etag":null,"topics":["image-processing","pytorch","seamless-tiling","tiler","tiling"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ProGamerGov.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-06-15T23:42:09.000Z","updated_at":"2025-06-12T09:52:54.000Z","dependencies_parsed_at":"2024-10-27T21:39:20.689Z","dependency_job_id":"41d9da06-2716-4f69-adb5-c3eee66e9cf4","html_url":"https://github.com/ProGamerGov/blended-tiling","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/ProGamerGov/blended-tiling","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ProGamerGov%2Fblended-tiling","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ProGamerGov%2Fblended-tiling/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ProGamerGov%2Fblended-tiling/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ProGamerGov%2Fblended-tiling/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ProGamerGov","download_url":"https://codeload.github.com/ProGamerGov/blended-tiling/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ProGamerGov%2Fblended-tiling/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262863687,"owners_count":23376452,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["image-processing","pytorch","seamless-tiling","tiler","tiling"],"created_at":"2024-10-10T18:16:25.359Z","updated_at":"2025-06-30T23:03:16.628Z","avatar_url":"https://github.com/ProGamerGov.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# blended-tiling\n\n[![GitHub - License](https://img.shields.io/github/license/progamergov/blended-tiling?logo=github\u0026style=flat\u0026color=green)][#github-license]\n[![PyPI](https://img.shields.io/pypi/v/blended-tiling.svg)][#pypi-package]\n[![DOI](https://zenodo.org/badge/503953108.svg)](https://zenodo.org/badge/latestdoi/503953108)\n\n[#github-license]: https://github.com/progamergov/blended-tiling/blob/main/LICENSE\n[#pypi-package]: https://pypi.org/project/blended-tiling/\n\n\nThis module adds support for splitting NCHW tensor inputs like images \u0026 activations into overlapping tiles of equal size, and then blending those overlapping tiles together after they have been altered. This module is also fully Autograd \u0026 JIT / TorchScript compatible.\n\nThis tiling solution is intended for situations where one wishes to render / generate outputs that are larger than what their computing device can support. Tiles can be separately rendered and periodically blended together to maintain tile feature coherence.\n\n## Setup:\n\n**Installation Requirements**\n- Python \u003e= 3.6\n- PyTorch \u003e= 1.6\n\n**Installation via `pip`:**\n\n```\npip install blended-tiling\n```\n\n**Dev / Manual install**:\n\n```\ngit clone https://github.com/progamergov/blended-tiling.git\ncd blended-tiling\npip install -e .\n\n# Notebook installs also require appending to environment variables\n# import sys\n# sys.path.append('/content/blended-tiling')\n```\n\n\n## Documentation\n\n### `TilingModule`\n\nThe base blended tiling module.\n\n```\nblended_tiling.TilingModule(tile_size=(224, 224), tile_overlap=(0.25, 0.25), base_size=(512, 512))\n```\n\n**Initialization Variables**\n\n* `tile_size` (int or tuple of int): The size of tiles to use. A single integer to use for both the height and width dimensions, or a list / tuple of dimensions with a shape of: `[height, width]`. The chosen tile sizes should be less than or equal to the sizes of the full NCHW tensor (`base_size`).\n* `tile_overlap` (int or tuple of int): The amount of overlap to use when creating tiles. A single integer to use for both the height and width dimensions, or a list / tuple of dimensions with a shape of: `[height, width]`. The chosen overlap percentages should be in the range [0.0, 0.50] (0% - 50%).\n* `base_size` (int or tuple of int): The size of the NCHW tensor being split into tiles. A single integer to use for both the height and width dimensions, or a list / tuple of dimensions with a shape of: `[height, width]`.\n\n\n#### Methods\n\n\n**`num_tiles()`**\n\n  * Returns\n    * `num_tiles` (int): The number of tiles that the full image shape is divided into based on specified parameters.\n\n**`tiling_pattern()`**\n\n  * Returns:\n    * `pattern` (list of int): The number of tiles per column and number of tiles per row, in the format of: `[n_tiles_per_column, n_tiles_per_row]`.\n\n**`split_into_tiles(x)`**: Splits an NCHW image input into overlapping tiles, and then returns the tiles. The `base_size` parameter is automatically readjusted to match the input.\n  * Returns:\n    * `tiles` (torch.Tensor): A set of tiles created from the input image.\n\n**`get_tile_masks(channels=3, device=torch.device(\"cpu\"), dtype=torch.float)`**: Return a stack of NCHW masks corresponding to the tiles outputted by `.split_into_tiles(x)`.\n\n  * Variables:\n    * `channels` (int, optional): The number of channels to use for the masks. Default: 3\n    * `device` (torch.device, optional): The desired device to create the masks on. Default: torch.device(\"cpu\")\n    * `dtype` (torch.dtype, optional): The desired dtype to create the masks with. Default: torch.float\n  * Returns:\n    * `masks` (torch.Tensor): A set of tile masks stacked across the batch dimension.\n\n\n**`rebuild(tiles, border=None, colors=None)`**: Creates and returns the full image from a stack of NCHW tiles stacked across the batch dimension.\n  * Variables:\n    * `tiles` (torch.Tensor): A set of tiles that may or not be masked, stacked across the batch dimension.\n    * `border` (int, optional): Optionally add a border of a specified size to the edges of tiles in the full image for debugging and explainability. Set to None for no border.\n    * `colors` (list of float, optional): A set of floats to use for the border color, if using borders. Default is set to red unless specified.\n  * Returns:\n    * `full_image` (torch.Tensor): The full image made up of tiles merged together without any blending.\n\n**`rebuild_with_masks(tiles, border=None, colors=None)`:** Creates and returns the full image from a stack of NCHW tiles stacked across the batch dimension, using tile blend masks.\n  * Variables:\n    * `tiles` (torch.Tensor): A set of tiles that may or not be masked, stacked across the batch dimension.\n    * `border` (int, optional): Optionally add a border of a specified size to the edges of tiles in the full image for debugging and explainability. Set to None for no border.\n    * `colors` (list of float, optional): A set of floats to use for the border color, if using borders. Default is set to red unless specified.\n  * Returns:\n    * `full_image` (torch.Tensor): The full image made up of tiles blended together using masks.\n\n**`forward(x)`:** Takes a stack of tiles, combines them into the full image with blending masks, then splits the image back into tiles.\n  * Variables:\n    * `x` (torch.Tensor): A set of tiles to blend the overlapping regions together of.\n  * Returns:\n    * `x` (torch.Tensor): A set of tiles with overlapping regions blended together.\n\n\n### Supported Tensor Types\n\nThe `TilingModule` class has been tested with and is confirmed to work with the following PyTorch [Tensor Data types / dtypes](https://pytorch.org/docs/stable/tensors.html): `torch.float32` / `torch.float`, `torch.float64` / `torch.double`, `torch.float16` / `torch.half`, \u0026 `torch.bfloat16`.\n\n\n## Usage\n\nThe `TilingModule` class is pretty easy to use.\n\n```\nfrom blended_tiling import TilingModule\n\n\nfull_size = [512, 512]\ntile_size = [224, 224]\ntile_overlap = [0.25, 0.25]  # 25% overlap on both H \u0026 W\n\ntiling_module = TilingModule(\n    tile_size=tile_size,\n    tile_overlap=tile_overlap,\n    base_size=full_size,\n)\n\n# Shape of tiles expected in forward pass\ninput_shape = [tiling_module.num_tiles(), 3] + tile_size\n\n# Tiles are blended together and then split apart by default\nblended_tiles = tiling_module(torch.ones(input_shape))\n```\n\n\nTiles can be created and then merged back into the original tensor like this:\n\n```\nfull_tensor = torch.ones(1, 3, 512, 512)\n\ntiles = tiling_module.split_into_tiles(full_tensor)\n\nfull_tensor = tiling_module.rebuild_with_masks(tiles)\n```\n\nThe tile boundaries can be viewed on the full tensor like this:\n\n```\ntiles = torch.ones(9, 3, 224, 224)\nfull_tensor = tiling_module.rebuild_with_masks(tiles, border=2)\n```\n\nAnd the number of tiles and tiling pattern can be obtained like this:\n\n```\nnum_tiles = tiling_module.num_tiles()\n\ntiling_pattern = tiling_module.tiling_pattern()\nprint(\"{}x{}\".format(tiling_pattern[0], tiling_pattern[1]))\n```\n\n\n### Custom Classes\n\nIt's also easy to modify the forward function of the tiling module:\n\n```\nfrom typing import Union, List, Tuple\nfrom blended_tiling import TilingModule\n\nclass CustomTilingModule(TilingModule):\n    def __init__(\n        self,\n        tile_size: Union[int, List[int], Tuple[int, int]] = [224, 224],\n        tile_overlap: Union[float, List[float], Tuple[float, float]] = [0.25, 0.25],\n        base_size: Union[int, List[int], Tuple[int, int]] = [512, 512],\n    ) -\u003e None:\n        TilingModule.__init__(self, tile_size, tile_overlap, base_size)\n        self.custom_module = torch.nn.Identity()\n\n    def forward(self, x: torch.Tensor) -\u003e torch.Tensor:\n        x = self.rebuild_with_masks(x)\n        x = self.custom_module(x) + 4.0\n        return self._get_tiles_and_coords(x)[0]\n```\n\n\n## Examples\n\n\nTo demonstrate the tile blending abilities of the `TilingModule` class, an example has been created below.\n\n\n\nFirst we'll create a set of tiles \u0026 give them all unique colors for this example:\n\n```\n# Setup TilingModule instance\nfull_size = [768, 1014]\ntile_size = [256, 448]\ntile_overlap = [0.25, 0.25]\ntiling_module = TilingModule(\n    tile_size=tile_size,\n    tile_overlap=tile_overlap,\n    base_size=full_size,\n)\n\n# Create unique colors for tiles\ntile_colors = [\n    [0.5334, 0.0, 0.8459],\n    [0.0, 1.0, 0.0],\n    [0.0, 0.7071, 0.7071],\n    [0.7071, 0.7071, 0.0],\n    [1.0, 0.0, 0.0],\n    [0.8459, 0.0, 0.5334],\n    [0.7071, 0.0, 0.7071],\n    [0.0, 0.8459, 0.5334],\n    [0.5334, 0.8459, 0.0],\n    [0.0, 0.5334, 0.8459],\n    [0.0, 0.0, 1.0],\n    [0.8459, 0.5334, 0.0],\n]\ntile_colors = torch.as_tensor(tile_colors).view(12, 3, 1, 1)\n\n# Create tiles\ntiles = torch.ones([tiling_module.num_tiles(), 3] + tile_size)\n\n# Color tiles\ntiles = tiles * tile_colors\n```\n\n\u003cimg src=\"https://github.com/ProGamerGov/blended-tiling/raw/main/examples/without_masks_separate_tiles.jpg\" width=\"500\"\u003e\n\nNext we apply the blend masks to the tiles:\n\n```\ntiles = tiles * tiling_module.get_tile_masks()\n```\n\n\u003cimg src=\"https://github.com/ProGamerGov/blended-tiling/raw/main/examples/with_masks_separate_tiles.jpg\" width=\"500\"\u003e\n\n\nWe can now combine the masked tiles into the full image:\n\n```\n# Build full tiled image\noutput = tiling_module.rebuild(tiles)\n```\n\n\u003cimg src=\"https://github.com/ProGamerGov/blended-tiling/raw/main/examples/with_masks.jpg\" width=\"500\"\u003e\n\nWe can also view the tile boundaries like so:\n\n```\n# Build full tiled image\noutput = tiling_module.rebuild(tiles, border=2, colors=[0,0,0])\n```\n\n\u003cimg src=\"https://github.com/ProGamerGov/blended-tiling/raw/main/examples/with_masks_and_borders.jpg\" width=\"500\"\u003e\n\nWe can view an animation of the tiles being added like this:\n\n```\nfrom torchvision.transforms import ToPILImage\n\ntile_steps = [\n    tiling_module.rebuild(tiles[: i + 1]) for i in range(tiles.shape[0])\n]\ntile_frames = [\n    ToPILImage()(x[0])\n    for x in [torch.zeros_like(tile_steps[0])] + tile_steps + [tile_steps[-1]]\n]\ntile_frames[0].save(\n    \"tiles.gif\",\n    format=\"GIF\",\n    append_images=tile_frames[1:],\n    save_all=True,\n    duration=700,\n    loop=0,\n)\n```\n\n\u003cimg src=\"https://github.com/ProGamerGov/blended-tiling/raw/main/examples/with_masks.gif\" width=\"500\"\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprogamergov%2Fblended-tiling","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprogamergov%2Fblended-tiling","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprogamergov%2Fblended-tiling/lists"}