{"id":26131281,"url":"https://github.com/alexanderlutsenko/nobuco","last_synced_at":"2025-05-15T07:05:41.365Z","repository":{"id":151527577,"uuid":"623971438","full_name":"AlexanderLutsenko/nobuco","owner":"AlexanderLutsenko","description":"Pytorch to Keras/Tensorflow/TFLite conversion made intuitive","archived":false,"fork":false,"pushed_at":"2025-03-10T09:40:17.000Z","size":5099,"stargazers_count":308,"open_issues_count":15,"forks_count":19,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-05-15T07:05:33.415Z","etag":null,"topics":["conversion","converter","deep-learning","keras","machine-learning","model-conversion","model-converter","pytorch","tensorflow","tensorflow-js","tensorflow-lite","tfjs","tflite"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AlexanderLutsenko.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-05T13:26:18.000Z","updated_at":"2025-05-05T08:51:39.000Z","dependencies_parsed_at":null,"dependency_job_id":"f3a3aade-e977-463d-85fa-52f64c0ed9a7","html_url":"https://github.com/AlexanderLutsenko/nobuco","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlexanderLutsenko%2Fnobuco","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlexanderLutsenko%2Fnobuco/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlexanderLutsenko%2Fnobuco/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlexanderLutsenko%2Fnobuco/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AlexanderLutsenko","download_url":"https://codeload.github.com/AlexanderLutsenko/nobuco/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254292041,"owners_count":22046426,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["conversion","converter","deep-learning","keras","machine-learning","model-conversion","model-converter","pytorch","tensorflow","tensorflow-js","tensorflow-lite","tfjs","tflite"],"created_at":"2025-03-10T21:55:28.979Z","updated_at":"2025-05-15T07:05:36.334Z","avatar_url":"https://github.com/AlexanderLutsenko.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/nobuco.png\"\u003e\n\u003csup\u003e\u003ca href=\"https://www.behance.net/diliajl\"\u003ediliajl\u003c/a\u003e\u003c/sup\u003e\n\u003c/p\u003e\n\n**No** **Bu**llshit **Co**nverter is a tool that helps you translate Pytorch models into Keras/Tensorflow/TFLite graphs without losing your mind.\n\n- Supports a wide range of architectures\n  - [x] Control flow ops (If, For, While)\n  - [x] Recurrent layers (LSTM, GRU)\n  - [x] Stateful modules\n  - [x] Arbitrary torch functions\n- Simple\n- Flexible\n- Efficient\n- Sanity-preserving, with clear mistake messaging\n\n\u003e [!IMPORTANT]  \n\u003e Nobuco only supports Keras 2 at the moment. If you'd like to use the new multi-backend Keras 3, please bump up the related issue: https://github.com/keras-team/keras/issues/19314\n\n## Installation \u003cimg src=\"https://img.shields.io/pypi/v/nobuco?color=blue\u0026style=flat-square\"\u003e\n\u003cimg src=\"https://img.shields.io/badge/PyTorch-2.1-EE4C2C.svg?style=flat\u0026logo=pytorch\"\u003e \u003cimg src=\"https://img.shields.io/badge/TensorFlow-2.15-FF6F00.svg?style=flat\u0026logo=tensorflow\"\u003e\n\n```bash\npip install -U nobuco\n```\n\n\u003c!-- toc --\u003e\n\n## Table of Contents\n- [Essentials](#essentials)\n- [Channel order wizardry](#channel-order-wizardry)\n- [Implementation mismatch: pick your poison](#implementation-mismatch-pick-your-poison)\n- [Going dynamic](#going-dynamic)\n  - [Control flows](#control-flows)\n  - [Dynamic shapes](#dynamic-shapes)\n- [No, that's too dynamic!](#no-thats-too-dynamic)\n  - [Forcing static crops](#forcing-static-crops)\n- [In-place operations](#in-place-operations)\n- [A little white lie: tracing mode](#a-little-white-lie-tracing-mode)\n- [Ad hoc modifications](#ad-hoc-modifications) \n- [So we put a converter inside your converter](#so-we-put-a-converter-inside-your-converter)\n- [But was it worth it?](#but-was-it-worth-it)\n- [Nobuco knowledge base](#nobuco-knowledge-base)\n\n\u003c!-- tocstop --\u003e\n\n\u003c!-- toc --\u003e\n\n- [Deep dive](#deep-dive)\n  - [Aggressive transposition removal: fighting fire with fire](#aggressive-transposition-removal-fighting-fire-with-fire)\n\n\u003c!-- tocstop --\u003e\n\n## Essentials\n\nSuppose we want to convert a Pytorch module similar to this one:\n\n````python\nclass MyModule(nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.conv = nn.Conv2d(3, 16, kernel_size=(3, 3), padding=(1, 1), stride=(2, 2))\n\n    def forward(self, x):\n        x = self.conv(x)\n        x = nn.Hardsigmoid()(x)\n        x = 1 - x[:, ::2] * x[:, 1::2]\n        return x\n````\n\nThe process is exactly what you would expect. Instantiate the module, create dummy inputs and call the magic function:\n\n```python\nimport nobuco\nfrom nobuco import ChannelOrder, ChannelOrderingStrategy\nfrom nobuco.layers.weight import WeightLayer\n```\n\n````python\ndummy_image = torch.rand(size=(1, 3, 256, 256))\npytorch_module = MyModule().eval()\n\nkeras_model = nobuco.pytorch_to_keras(\n    pytorch_module,\n    args=[dummy_image], kwargs=None,\n    inputs_channel_order=ChannelOrder.TENSORFLOW,\n    outputs_channel_order=ChannelOrder.TENSORFLOW\n)\n````\n\nAaaand done! That's all it takes to... hold on, what's that?\n\n\u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/essentials1.svg\" width=\"100%\"\u003e\n\nNobuco says it doesn't know how to handle hard sigmoid.\nApparently, it's our job to provide a node converter for either `F.hardsigmoid` or the enclosing `Hardsigmoid` module (or the entire `MyModule`, but that makes little sense). Here, we'll go for the former.\n\nConversion is done directly. No layers upon layers of abstraction, no obscure intermediate representation. \nA node converter is just a `Callable` that takes the same arguments as the corresponding node in Pytorch and outputs an equivalent node in Tensorflow. \nThe converted node preserves the original node's signature, but Pytorch tensors replaced with Tensorflow counterparts (be that `tf.Tensor`, `KerasTensor`, `tf.Variable`, or `ResourceVariable`).\n\nThis should do the trick:\n\n````python\n@nobuco.converter(F.hardsigmoid, channel_ordering_strategy=ChannelOrderingStrategy.MINIMUM_TRANSPOSITIONS)\ndef hardsigmoid(input: torch.Tensor, inplace: bool = False):\n    return lambda input, inplace=False: tf.keras.activations.hard_sigmoid(input)\n````\n\n\u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/essentials2.svg\" width=\"100%\"\u003e\n\nIt works, but the outputs don't quite match. Perhaps we should check how [Pytorch](https://pytorch.org/docs/stable/generated/torch.nn.functional.hardsigmoid.html) and [Tensorflow](https://www.tensorflow.org/api_docs/python/tf/keras/activations/hard_sigmoid) define hard sigmoid. \nAnd sure enough, their implementations differ. Have to type in the formula manually, I guess...\n\n````python\n@nobuco.converter(F.hardsigmoid, channel_ordering_strategy=ChannelOrderingStrategy.MINIMUM_TRANSPOSITIONS)\ndef hardsigmoid(input: torch.Tensor, inplace: bool = False):\n    return lambda input, inplace=False: tf.clip_by_value(input/6 + 1/2, clip_value_min=0, clip_value_max=1)\n````\n\nAnd the happy result:\n\n\u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/essentials3.svg\" width=\"100%\"\u003e\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/tutorial.png\" width=\"30%\"\u003e\n\u003c/p\u003e\n\nThe example above is artificial, but it illustrates the point.\nIt's not feasible to provide a node converter for every existing Pytorch op. There are literally [thousands](https://dev-discuss.pytorch.org/t/where-do-the-2000-pytorch-operators-come-from-more-than-you-wanted-to-know/) of them! \nBest we can do without the converter constantly lacking essential functionality, being riddled with bugs, doing weird stuff and breaking apart with every other PT/TF release \nis to keep the tool simple and customizable, make it clear where a problem comes from and let the _user_ sort things out.\nUsually it's easy for a human to translate an isolated operation from one framework to another.\nReproducing the graph structure is a different matter entirely. For that, Nobuco has you covered!\n\nNobuco lets you intervene in conversion at each step, asks for help where needed and doesn't bother you with routine stuff.\n\nhttps://user-images.githubusercontent.com/2457934/233740603-cc11acc5-cd6b-48c8-b089-ff3ead772dd0.mp4\n\n\u003cp align=\"center\"\u003e\u003cem\u003e\nWith an IDE, you can jump right where the node was [I]nvoked, [D]efined and [C]onverted\n\u003c/em\u003e\u003c/p\u003e\n\n## Channel order wizardry\n\nSome operations assume its input tensors have a channel dimension. \nAnd as you probably know, Pytorch and Tensorflow do not agree on the layout of such tensors.\nPytorch adopts channel-first layout (_B**C**H_, _B**C**HW_, etc.) \nwhile Tensorflow works efficiently with channel-last tensors (_BH**C**_, _BHW**C**_, ...).\nTransposing tensors between the two layouts incurs non-trivial overhead as generally, tensor data must be physically rearranged.\nIn an effort to keep that overhead to the minimum, Nobuco does layout coercions _lazily_. \nA couple of things are needed to make it possible:\n\n- Tensorflow tensors are augmented with an additional property which stores their channel order, either Pytorch (channel first) or Tensorflow (channel last) style.\n- Node converters have requirements on what channel order their inputs must have. Said requirements are expressed with `channel_ordering_strategy` argument. \n\nChannel ordering strategies are\n- `FORCE_TENSORFLOW_ORDER`\n  - Input tensors will be coerced to Tensorflow channel order.\n  - Convenient for converting channel-aware operations (convolution, batchnorm).\n- `FORCE_PYTORCH_ORDER`\n  - Input tensors entering the node will look exactly as they do in the original Pytorch graph. \n  - Use it when the node does not interpret its input tensors as having a channel dimension (linear, matmul). \n- `MINIMUM_TRANSPOSITIONS`\n  - The channel order is decided by a majority vote (whichever prevails among the inputs). This way the number of coercions (i.e. tensor transpositions) is kept to the minimum.\n  It also means whenever there's only one input, it will be left untouched.\n  - Best choice for element-wise ops (most activations).\n- `MANUAL`\n  - You are on your own. In exchange for unrestricted freedom, you take responsibility to coerce input tensors to suitable channel order and to also annotate output tensors with their order.\n\nThe simple lazy approach makes wonders in most situations, but sometimes it produces suboptimal graphs.\nConsider the code below. Imagine this is some sort of text processing network. \nIt first applies a GRU layer which assumes the inputs do not have a channel dimension, so its input/output layouts are the same in both Pytorch and Tensorflow.\nBut then, the outputs are passed to a couple of 1D convolutions which are channel-aware. \nBecause of that, a transpose op must be put in the converted graph.\n\n```python\nclass MyModule(nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.gru = nn.GRU(32, 128, num_layers=1, batch_first=True, bidirectional=False)\n        self.conv1 = nn.Conv1d(12, 40, kernel_size=3, padding=1)\n        self.conv2 = nn.Conv1d(12, 60, kernel_size=1, padding=0)\n\n    def forward(self, x):\n        x, hx = self.gru(x)\n        x1 = self.conv1(x)\n        x2 = self.conv2(x)\n        return x1, x2\n\npytorch_module = MyModule().eval()\n\ninputs = [\n    torch.normal(0, 1, size=(1, 12, 32)),\n]\nkeras_model = nobuco.pytorch_to_keras(\n    pytorch_module, inputs,\n    inputs_channel_order=ChannelOrder.PYTORCH,\n)\n```\n\nThe laziness shoots us in the foot here, and we get not one transpose but two:\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/channel_ordering.png\" width=\"30%\"\u003e\n\u003c/p\u003e\n\nFor such occasions, there's two brethren functions: `force_tensorflow_order` and `force_pytorch_order`.\n\n```python\nx, hx = self.gru(x)\nx = nobuco.force_tensorflow_order(x)\nx1 = self.conv1(x)\nx2 = self.conv2(x)\n```\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/channel_ordering_forced.png\" width=\"30%\"\u003e\n\u003c/p\u003e\n\nIn case you are curious, the implementation is trivial:\n\n```python\n@nobuco.traceable\ndef force_tensorflow_order(inputs):\n    return inputs\n\n\n@nobuco.converter(force_tensorflow_order, channel_ordering_strategy=ChannelOrderingStrategy.FORCE_TENSORFLOW_ORDER)\ndef converter_force_tensorflow_order(inputs):\n    return lambda inputs: inputs\n```\n\n`force_pytorch_order` is defined analogously.\n\n## Implementation mismatch: pick your poison\n\nSometimes, Pytorch and Tensorflow just don't go along. \nIn case of `hardsigmoid`, it's a mere inconvenience, but it can be much more sinister.\n\nTake the model below, for example.\n\n```python\nclass MyModule(nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.factor = 4\n        self.conv = nn.Conv2d(3*self.factor**2, 3*self.factor**2, kernel_size=1)\n\n    def forward(self, x):\n        x = nn.PixelUnshuffle(self.factor)(x)\n        x = self.conv(x)\n        x = nn.PixelShuffle(self.factor)(x)\n        return x\n```\n\nIdeally, there would only be three nodes in the converted graph. That's not what we get, though.\n\nTensorflow does not have `pixel_unshuffle`/`pixel_shuffle`.\nTheir closest counterparts, `tf.nn.space_to_depth`/`tf.nn.depth_to_space`,\ndo almost the same thing but not quite: output channels are in a different order.\nThe order must be fixed with a pricey `transpose`, no way around that. Or is there?\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/pixelshuffle1.png\" width=\"20%\"\u003e\n\u003c/p\u003e\n\nInstead of emulating an absent Pytorch op in Tensorflow, \nwe might do the procedure in reverse: provide a Pytorch implementation for the Tensorflow node we want to convert to.\nThe overhead would be carried by the original Pytorch model leaving the converted graph nice and clean.\n\n```python\nfrom nobuco.addons.torch.space_to_depth import SpaceToDepth\nfrom nobuco.addons.torch.depth_to_space import DepthToSpace\n\n\nclass MyModuleTFOptimized(nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.factor = 4\n        self.conv = nn.Conv2d(3*self.factor**2, 3*self.factor**2, kernel_size=1)\n\n    def forward(self, x):\n        x = SpaceToDepth(self.factor)(x)\n        x = self.conv(x)\n        x = DepthToSpace(self.factor)(x)\n        return x\n```\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/pixelshuffle2.png\" width=\"20%\"\u003e\n\u003c/p\u003e\n\n\u003ctable align=\"center\"\u003e\n\u003ctr\u003e\n\u003cth\u003eTorch-optimized\u003c/th\u003e\n\u003cth\u003eTensorflow-optimized\u003c/th\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\n\n**Torch implementation**\n\n```python\nF.pixel_unshuffle\n```\n\n**Tensorflow converter**\n```python\n@nobuco.converter(F.pixel_unshuffle, \n                  channel_ordering_strategy=ChannelOrderingStrategy.FORCE_TENSORFLOW_ORDER)\ndef converter_pixel_unshuffle(input: Tensor, downscale_factor: _int):\n    def func(input, downscale_factor):\n        x = tf.nn.space_to_depth(input, downscale_factor)\n        x = channel_interleave2d(x, downscale_factor, reverse=True)\n        return x\n    return func\n\n\ndef channel_interleave2d(x, block_size: int, reverse: bool):\n    b, h, w, c = x.shape\n    n_blocks = block_size ** 2\n\n    if reverse:\n        x = tf.reshape(x, (b, h, w, n_blocks, c // n_blocks))\n    else:\n        x = tf.reshape(x, (b, h, w, c // n_blocks, n_blocks))\n\n    x = tf.transpose(x, (0, 1, 2, 4, 3))\n    x = tf.reshape(x, (b, h, w, c))\n    return x\n```\n\n\u003c/td\u003e\n\u003ctd\u003e\n\n**Torch implementation**\n```python\nclass SpaceToDepth(nn.Module):\n    def __init__(self, block_size):\n        super().__init__()\n        self.block_size = block_size\n\n    def forward(self, input):\n        x = F.pixel_unshuffle(input, self.block_size)\n        x = channel_interleave2d(x, self.block_size, reverse=False)\n        return x\n\n    \ndef channel_interleave2d(x: torch.Tensor, block_size: int, reverse: bool) -\u003e torch.Tensor:\n    b, c, h, w = x.shape\n    n_blocks = block_size ** 2\n\n    if reverse:\n        x = x.view(b, n_blocks, c // n_blocks, h, w)\n    else:\n        x = x.view(b, c // n_blocks, n_blocks, h, w)\n\n    x = x.transpose(1, 2).reshape(b, c, h, w)\n    return x\n```\n\n**Tensorflow converter**\n```python\n@nobuco.converter(SpaceToDepth, \n                  channel_ordering_strategy=ChannelOrderingStrategy.FORCE_TENSORFLOW_ORDER)\ndef converter_space_to_depth(self, input: torch.Tensor):\n    return lambda input: tf.nn.space_to_depth(input, self.block_size)\n```\n\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\n## Going dynamic\n\n### Control flows\nIntroducing python control flow statements into the compute graph is no easy feat.\nTensorflow can do so via `tf.autograph`, but at a cost of [system's complexity](https://www.youtube.com/watch?v=NIEgzljyDyI) and with some notable [limitations](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/autograph/g3doc/reference/control_flow.md).\nStuff like that is way above Nobuco's paygrade, so the following module cannot be properly handled without human intervention.\n\n```python\nclass ControlIf(nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.conv_pre = nn.Conv2d(3, 16, kernel_size=(1, 1))\n        self.conv_true = nn.Conv2d(16, 32, kernel_size=(1, 1))\n        self.conv_false = nn.Conv2d(16, 32, kernel_size=(1, 1))\n        self.conv_shared = nn.Conv2d(32, 32, kernel_size=(1, 1))\n\n    def forward(self, x):\n        x = self.conv_pre(x)\n        if x.mean() \u003e 0:\n            x = self.conv_true(x)\n            x = torch.tanh(x)\n            x = self.conv_shared(x)\n            x = x + 1\n        else:\n            x = self.conv_false(x)\n            x = torch.sigmoid(x)\n            x = self.conv_shared(x)\n            x = x - 1\n        x = self.conv_shared(x)\n        return x\n```\n\nOf course, it's possible to translate the dynamic module into a Tensorflow layer\n(don't forget to decorate it with `@tf.function` for autograph to kick in).\nBut what if it contains inner modules, do you replicate them in Tensorflow all by hand?\nNot unless you want to! \nJust convert them separately and use the resulting graphs inside the parent layer.\n\n```python\nclass ControlIfKeras(tf.keras.layers.Layer):\n    def __init__(self, conv_pre, conv_true, conv_false, conv_shared, *args, **kwargs):\n        super().__init__(*args, **kwargs)\n        self.conv_pre = conv_pre\n        self.conv_true = conv_true\n        self.conv_false = conv_false\n        self.conv_shared = conv_shared\n\n    def get_config(self):\n        config = super().get_config()\n        config.update({\n            \"conv_pre\": self.conv_pre,\n            \"conv_true\": self.conv_true,\n            \"conv_false\": self.conv_false,\n            \"conv_shared\": self.conv_shared,\n        })\n        return config\n\n    @tf.function\n    def call(self, x):\n        x = self.conv_pre(x)\n        if tf.reduce_mean(x) \u003e 0:\n            x = self.conv_true(x)\n            x = tf.tanh(x)\n            x = self.conv_shared(x)\n            x = x + 1\n        else:\n            x = self.conv_false(x)\n            x = tf.sigmoid(x)\n            x = self.conv_shared(x)\n            x = x - 1\n        x = self.conv_shared(x)\n        return x\n\n\n@nobuco.converter(ControlIf, channel_ordering_strategy=ChannelOrderingStrategy.FORCE_TENSORFLOW_ORDER)\ndef converter_ControlIf(self, x):\n    order = ChannelOrder.TENSORFLOW\n    kwargs = {'inputs_channel_order': order, 'outputs_channel_order': order, 'return_outputs_pt': True}\n    \n    conv_pre, out_pre = nobuco.pytorch_to_keras(self.conv_pre, [x], **kwargs)\n    conv_true, out_true = nobuco.pytorch_to_keras(self.conv_true, [out_pre], **kwargs)\n    conv_false, out_false = nobuco.pytorch_to_keras(self.conv_false, [out_pre], **kwargs)\n    conv_shared, _ = nobuco.pytorch_to_keras(self.conv_shared, [out_true], **kwargs)\n    layer = ControlIfKeras(conv_pre, conv_true, conv_false, conv_shared)\n    return layer\n```\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/control_if.png\" width=\"25%\"\u003e\n\u003c/p\u003e\n\nSee [examples](/examples) for other ways to convert control flow ops.\n\n### Dynamic shapes\n\nWhat if we wanted our module to accept images of arbitrary height and width?\nCan we have that? Let's try:\n\n```python\nclass DynamicShape(nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.conv = nn.Conv2d(3, 16, kernel_size=(1, 1))\n\n    def forward(self, x):\n        x = self.conv(x)\n\n        # Produces static shape\n        b, c, h, w = x.shape\n\n        x = x[:, :, h//3:, w//3:]\n        return x\n\n\ninput = torch.normal(0, 1, size=(1, 3, 128, 128))\npytorch_module = DynamicShape().eval()\n\nkeras_model = nobuco.pytorch_to_keras(\n    pytorch_module,\n    args=[input],\n    input_shapes={input: (None, 3, None, None)}, # Annotate dynamic axes with None\n    inputs_channel_order=ChannelOrder.TENSORFLOW,\n    outputs_channel_order=ChannelOrder.TENSORFLOW,\n)\n```\n\nSomething's not right. We don't see shape extraction ops in the debug output or the graph:\n\n\u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/dynamic_shape1.svg\" width=\"100%\"\u003e\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/dynamic_shape1.png\" width=\"15%\"\u003e\n\u003c/p\u003e\n\nThat's not surprising, actually. \nIn Pytorch, tensor shape is a tuple of regular integers, not tensors, so it's quite difficult to track them.\n`nobuco.shape` solves this problem.\nThis function returns tensors, much like [`tf.shape`](https://www.tensorflow.org/api_https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/python/tf/shape) does:\n\n```python\n# Allows for dynamic shape\nb, c, h, w = nobuco.shape(x)\n```\n\n\u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/dynamic_shape2.svg\" width=\"100%\"\u003e\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/dynamic_shape2.png\" width=\"30%\"\u003e\n\u003c/p\u003e\n\nIt's also possible to automatically substitute every `.shape`/`.size` call with `nobuco.shape` during the tracing phase by setting `trace_shape` flag:\n\n```python\nkeras_model = nobuco.pytorch_to_keras(\n  # ...\n  trace_shape=True\n)\n```\n\n## No, that's too dynamic!\n\n### Forcing static crops\n\nSometimes, dynamic tensors appear against our will. They can also be very detrimental, especially when it comes to efficient inference. Let me show you.\n\nOur Pytorch module here involves extracting a crop of a fixed size, something along these lines:\n\n```python\nclass CroppingModule(nn.Module):\n    def __init__(self, crop_height, crop_width):\n        super().__init__()\n        self.crop_height = crop_height\n        self.crop_width = crop_width\n\n    def forward(self, x, crop_x, crop_y):\n        _, _, h, w = x.shape\n        crop_x = (crop_x * (w - self.crop_width)).int()\n        crop_y = (crop_y * (h - self.crop_height)).int()\n        crop = x[:, :, crop_y: crop_y + self.crop_height, crop_x: crop_x + self.crop_width]  # Poor way to crop\n        return crop\n\n\nx = torch.normal(0, 1, size=(1, 3, 128, 128))\ncrop_y = torch.rand(())\ncrop_x = torch.rand(())\npytorch_module = CroppingModule(64, 32).eval()\n```\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/static_crop1.png\" width=\"50%\"\u003e\n\u003c/p\u003e\n\n```console\nINFO: Created TensorFlow Lite XNNPACK delegate for CPU.\nWARNING: Attempting to use a delegate that only supports static-sized tensors with a graph that has dynamic-sized tensors (tensor#17 is a dynamic-sized tensor).\nERROR: Failed to apply XNNPACK delegate.\n```\n\nTFLite fails to recognize the crop shape as static. Why? Consider this:\n\n```python\ny0 = torch.tensor(())\ny1 = y0 + h\ncrop = image[:, y0: y1]\n```\n\nWhen we invoke `image[:, y0: y1]`, the indexing operator (`[]`) has no way of knowing that `y0` and `y1` are related and `y1 - y0 == const`. \nIt wouldn't be a problem if instead of `(y0, y1)`, the op accepted `(y0, h)` as cropping parameters, since `h` is a constant. \n\nIs there such op? Meet `torch.narrow`:\n\n```python\ncrop = x.narrow(2, crop_y, self.crop_height).narrow(3, crop_x, self.crop_width)  # Better way to crop\n```\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/static_crop2.png\" width=\"35%\"\u003e\n\u003c/p\u003e\n\nThat's it, problem solved, the crop is statically-shaped now. \nAlas, `narrow` only operates on one dimension at a time, so we have to apply it repeatedly.\n`tf.slice`, however, has no such limitation. See where I'm going? It's custom converter time!\n\n```python\n@nobuco.traceable\ndef get_crop(x, crop_y, crop_x, h, w):\n    return x[:, :, crop_y: crop_y + h, crop_x: crop_x + w]\n\n\n@nobuco.converter(get_crop, channel_ordering_strategy=nobuco.ChannelOrderingStrategy.FORCE_TENSORFLOW_ORDER)\ndef converter_get_crop(x, crop_y, crop_x, h, w):\n    def func(x, crop_y, crop_x, h, w):\n        return tf.image.crop_to_bounding_box(x, crop_y, crop_x, h, w)  # Calls tf.slice under the hood\n    return func\n```\n\n```python\ncrop = get_crop(x, crop_y, crop_x, self.crop_height, self.crop_width)  # Best way to crop\n```\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/static_crop3.png\" width=\"35%\"\u003e\n\u003c/p\u003e\n\n## In-place operations\n\nNobuco can handle most situations where tensors are modified in-place. For instance, these will work just fine:\n\n```python\nclass MyModule(nn.Module):\n    def forward(self, x):\n        x[:, 1:2, 16:25, 8::2] *= 2\n        torch.relu_(x)\n        return x\n```\n\n\u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/inplace1.svg\" width=\"100%\"\u003e\n\nHowever, applying in-place operation to a slice yields incorrect result. What gives?\n\n```python\nclass MyModule(nn.Module):\n    def forward(self, x):\n        torch.relu_(x[:, 1:2, 16:25, 8::2])\n        return x\n```\n\n\u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/inplace2.svg\" width=\"100%\"\u003e\n\nYou see, Tensorflow graphs (and many other formats like ONNX) do not support in-place ops.\nSo when we take slice (`x[:, 1:2, 16:25, 8::2]`) in TF/ONNX, the result is not a view of the original tensor but a copy. \nThis copy is then passed to `relu` (which is not in-place either), and its result is not used anywhere. \nAs you can see above, the output tensors of `__getitem__` and `relu_` are \u003cspan style=\"color:gray\"\u003egrayed out\u003c/span\u003e, and these operations are excluded from the graph.\nIn fact, it's empty:\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/inplace_empty.png\" width=\"30%\"\u003e\n\u003c/p\u003e\n\nThe easiest way of fixing this is to explicitly assign the result to the slice.\nConveniently enough, most standard in-place operations in Pytorch do return their modified arguments as outputs.\n\n```python\nclass MyModule(nn.Module):\n    def forward(self, x):\n        x[:, 1:2, 16:25, 8::2] = torch.relu_(x[:, 1:2, 16:25, 8::2])\n        return x\n```\n\n\u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/inplace3.svg\" width=\"100%\"\u003e\n\n## A little white lie: tracing mode\n\nThe flexibility and dynamic nature of Pytorch graphs can make it quite challenging to directly translate them not only to Tensorflow but ONNX as well.\nHow are these types of problems solved for real-world models?\nOnce again, we'll learn by example:\n\n```python\npytorch_module = torchvision.models.detection.ssdlite320_mobilenet_v3_large(weights=SSDLite320_MobileNet_V3_Large_Weights.DEFAULT).eval()\n\nx = torch.rand(size=(1, 3, 320, 320))\n\nkeras_model = nobuco.pytorch_to_keras(\n    pytorch_module, \n    args=[x],\n)\n```\n\n\u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/tracing1.svg\" width=\"100%\"\u003e\n\nIs that Nobuco's failure to handle in-place `copy_`? Yes, but there's more to the story.\nLet's peek into the model's source code (set `debug_traces=nobuco.TraceLevel.ALWAYS` for easier navigation). Here's the culprit:\n\n```python\ndef batch_images(self, images: List[Tensor], size_divisible: int = 32) -\u003e Tensor:\n    if torchvision._is_tracing():\n        # batch_images() does not export well to ONNX\n        # call _onnx_batch_images() instead\n        return self._onnx_batch_images(images, size_divisible) # \u003c- Alternative ONNX-friendly implementation\n\n    max_size = self.max_by_axis([list(img.shape) for img in images])\n    stride = float(size_divisible)\n    max_size = list(max_size)\n    max_size[1] = int(math.ceil(float(max_size[1]) / stride) * stride)\n    max_size[2] = int(math.ceil(float(max_size[2]) / stride) * stride)\n\n    batch_shape = [len(images)] + max_size\n    batched_imgs = images[0].new_full(batch_shape, 0)\n    for i in range(batched_imgs.shape[0]):\n        img = images[i]\n        batched_imgs[i, : img.shape[0], : img.shape[1], : img.shape[2]].copy_(img) # \u003c- In-place copy\n\n    return batched_imgs\n```\n\nThis method is certainly not fit for tracing. \nThe model's authors knew it and provided an alternative implementation in case we'd want to export it to ONNX (works for Keras, too!).\n`torchvision._is_tracing()` returns True whenever the model is being traced (e.g. invoked inside `torch.onnx.export(...)`).\nThis state is not directly controllable by the user, yet Nobuco can gaslight the model into thinking it's being traced by Pytorch itself:\n\n```python\nkeras_model = nobuco.pytorch_to_keras(\n    # ...\n    enable_torch_tracing=True\n)\n```\n\n\u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/tracing2.svg\" width=\"100%\"\u003e\n\nWhy is it not enabled by default, then? \nYou see, the other effect of tracing mode is equivalent to that of `trace_shape=True`: `shape` calls return tensors instead of ints.\nMany Pytorch models were never meant to be converted to anything, and tracing mode may break them.\nNobuco tries to minimize silent/obscure errors and user's confusion, and not being too smart is a reasonable tradeoff.\n\n## Ad hoc modifications\n\nLet's say, for illustrative purposes, that we prefer putting batchnorm _before_ convolution.\nWe were sure `TFLiteConverter` would fuse these two linear operations into one.\nAlas, it failed to meet our expectations. \nCan we still get the fusion to work without re-training or messing around with the model checkpoint?\n\n```python\nclass FusibleModule(nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.bn = nn.BatchNorm2d(3)\n        self.conv = nn.Conv2d(3, 16, kernel_size=(3, 3), padding=(0, 0))\n        self.act = nn.ReLU()\n\n    def forward(self, x):\n        x = self.bn(x)\n        x = self.conv(x)\n        x = self.act(x)\n        return x\n```\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/fusion1.png\" width=\"20%\"\u003e\n\u003c/p\u003e\n\nHere's one way to do it:\n- Wrap the two ops in a `Callable`. Decorate it with `@nobuco.traceable`. \n- Make a custom converter for it which does the desired optimization. \n\n```python\nclass FusibleModule(nn.Module):\n    # ...\n\n    @nobuco.traceable\n    def bn_conv(self, x):\n        x = self.bn(x)\n        x = self.conv(x)\n        return x\n\n    def forward(self, x):\n        x = self.bn_conv(x)\n        x = self.act(x)\n        return x\n```\n\n```python\n@nobuco.converter(FusibleModule.bn_conv, channel_ordering_strategy=ChannelOrderingStrategy.FORCE_TENSORFLOW_ORDER)\ndef converter_bn_conv(self, x):\n    order = ChannelOrder.TENSORFLOW\n    bn, out_bn = nobuco.pytorch_to_keras(self.bn, [x], inputs_channel_order=order, outputs_channel_order=order, return_outputs_pt=True)\n    conv = nobuco.pytorch_to_keras(self.conv, [out_bn], inputs_channel_order=order, outputs_channel_order=order)\n\n    gamma, beta, moving_mean, moving_variance = bn.get_weights()\n    kernel, bias = conv.get_weights()\n    eps = self.bn.eps\n\n    '''\n    y = gamma * (x - moving_mean) / sqrt(moving_variance + eps) + beta\n    z = kernel * y + bias\n    =\u003e\n    z = kernel_fused * x + bias_fused WHERE\n    kernel_fused = kernel * gamma / sqrt(moving_variance + eps)\n    bias_fused = -kernel_fused * moving_mean + kernel * beta + bias\n    '''\n    kernel_fused = kernel * (gamma / np.sqrt(moving_variance + eps))[None, None, :, None]\n    bias_fused = (-kernel_fused * moving_mean[None, None, :, None] + kernel * beta[None, None, :, None]).sum(axis=(0, 1, 2)).flatten() + bias\n    conv.set_weights([kernel_fused, bias_fused])\n    return lambda self, x: conv(x)\n```\n\n\u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/fusion2.svg\" width=\"100%\"\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/fusion2.png\" width=\"20%\"\u003e\n\u003c/p\u003e\n\n## So we put a converter inside your converter\n\nAs we've learned, Nobuco gets confused when in-place operation is applied to a slice.\nThere's a way to fix that, but let's not do it now.\nInstead, we'll use it as an excuse to explain the concept of nested converters.\nSo, for this module, conversion will give us incorrect result:\n\n```python\nclass SliceReLU(nn.Module):\n    def forward(self, x):\n        # Gives incorrect result after conversion\n        torch.relu_(x[:, 1:2, 16:25, 8::2])\n        # That's the recommended approach, but we're not going for it now\n        # x[:, 1:2, 16:25, 8::2] = torch.relu_(x[:, 1:2, 16:25, 8::2])\n        return x\n\n\nclass MyModule(nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.conv = nn.Conv2d(3, 3, kernel_size=(3, 3), padding=(1, 1))\n\n    def forward(self, x):\n        x = self.conv(x)\n        SliceReLU()(x)\n        return x\n```\n\n\u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/converter_inside_converter1.svg\" width=\"100%\"\u003e\n\nWe've seen it's possible to invoke a Nobuco converter inside another Nobuco converter.\nCan we embed some third-party converter? You bet! Why? Because it might just do what we need.\nLet's consider the standard route: Pytorch -\u003e ONNX -\u003e Tensorflow, with the latter step done with [onnx-tf](https://github.com/onnx/onnx-tensorflow).\nThis library likes transposing stuff so much, converting the whole graph with it may introduce intolerable inference overhead. Nonetheless, it does the job.\nA sensible tradeoff would be to wrap the problematic operation into its own `nn.Module` and give it a special treat, while handling everything else with Nobuco.\n\n```python\nimport onnx\nfrom onnx_tf.backend import prepare\n\n\n@nobuco.converter(SliceReLU, channel_ordering_strategy=ChannelOrderingStrategy.FORCE_PYTORCH_ORDER, reusable=False)\ndef converter_SliceReLU(self, x):\n    model_path = 'slice_relu'\n    onnx_path = model_path + '.onnx'\n\n    # NB: onnx.export in implemented via tracing i.e. it may modify the inputs!\n    torch.onnx.export(self, (x,), onnx_path, opset_version=12, input_names=['input'],\n                      dynamic_axes={'input': [0, 1, 2, 3]}\n                      )\n    onnx_model = onnx.load(onnx_path)\n    tf_rep = prepare(onnx_model)\n    tf_rep.export_graph(model_path)\n    model = tf.keras.models.load_model(model_path)\n    return keras.layers.Lambda(lambda x: model(input=x))\n```\n\n\u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/converter_inside_converter2.svg\" width=\"100%\"\u003e\n\n## But was it worth it?\n\nLet's cut to the chase, here's the numbers.\n\n**mobilenet_v3_large** (26.8 Mb)\n\n|                   | nobuco  | onnx_tf  | speedup |\n|-------------------|---------|----------|---------|\n| x86 (XNNPACK)     | 11.1 ms | 14.7 ms  | 1.3x    |\n| Arm CPU (XNNPACK) | 24.3 ms | 40.3 ms  | 1.6x    |\n| Arm GPU (OpenCL)  | 21.3 ms | 192.6 ms | 9x      |\n\n**deeplabv3_resnet50** (158.5 Mb)\n\n|                   | nobuco | onnx_tf | speedup |\n|-------------------|--------|---------|---------|\n| x86 (XNNPACK)     | 1.25 s | 1.34 s  | 1.07x   |\n| Arm CPU (XNNPACK) | 2.0 s  | 2.7 s   | 1.35x   |\n| Arm GPU (OpenCL)  | 1.6 s  | 2.6 s   | 1.62x   |\n\nAs we can see, redundant transpositions may completely ruin the performance, especially on a GPU.\nBut that's not the only issue.\nLet's test this:\n\n```python\nclass SliceReLU(nn.Module):\n    def forward(self, x):\n        x[:, 1:2, 16:25, 8::2] = torch.relu_(x[:, 1:2, 16:25, 8::2])\n        return x\n```\n\n|                   | nobuco     | onnx_tf    | speedup |\n|-------------------|------------|------------|---------|\n| x86 (XNNPACK)     | 0.40 ms    | 1.57 ms    | 3.9x    |\n| Arm CPU           | 4.6 ms     | **2.9** ms | 0.6x    |\n| Arm CPU (XNNPACK) | **2.1** ms | FAIL       | —       |\n| Arm GPU (OpenCL)  | 21.8 ms    | FAIL       | —       |\n\nAgain, the graph obtained with `onnx_tf` is much slower on x86 CPU.\nWorse yet, on mobile processor, optimized TFLite delegates for both GPU and CPU failed.\nNo transpose ops were added this time, so who's to blame?\nJust look what `torch.onnx.export` gives us:\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/slice_relu_onnx.png\" width=\"100%\"\u003e\n  \u003cb\u003eslice_relu.onnx\u003c/b\u003e\n\u003c/p\u003e\n\n`onnx_tf` does a fair job optimizing the monstrosity of a graph it's given,\nbut combining consecutive `slice` ops seems too much to ask.\nIt also leaves out garbage nodes sometimes (note the free-floating `While` in this example).\n\nNobuco evades these types of problems by simply not dealing with `onnx`.\n\n\u003ctable align=\"center\"\u003e\n  \u003ctr\u003e\n    \u003cth\u003eslice_relu_nobuco\u003c/th\u003e\n    \u003cth\u003eslice_relu_onnxtf\u003c/th\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\n      \u003cp align=\"center\"\u003e\n        \u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/slice_relu_nobuco.png\" width=\"60%\"\u003e\n      \u003c/p\u003e\n    \u003c/td\u003e\n    \u003ctd\u003e\n      \u003cp align=\"center\"\u003e\n        \u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/slice_relu_onnxtf.png\" width=\"60%\"\u003e\n      \u003c/p\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n## Nobuco knowledge base\n\nDon't want to convert anything but looking for a Tensorflow equivalent of a certain Pytorch node (operation or module)?\nNobuco already implements quite a few node converters, most written in a concise and, hopefully, understandable way.\nThese are located in [nobuco/node_converters](https://github.com/AlexanderLutsenko/nobuco/tree/master/nobuco/node_converters),\nand there's a utility function to help you find what you need:\n\n```python\nnode = torch.Tensor.repeat\n# node = F.relu_\n# node = nn.LSTM\n\nlocation_link, source_code = nobuco.locate_converter(node)\nprint('Converter location:')\nprint(location_link)\nprint('Converter source code:')\nprint(source_code)\n```\n\n```console\nConverter location:\nFile \"/home/user/anaconda3/envs/nb/lib/python3.9/site-packages/nobuco/node_converters/tensor_manipulation.py\", line 141\n\nConverter source code:\n@converter(torch.Tensor.repeat, channel_ordering_strategy=ChannelOrderingStrategy.MINIMUM_TRANSPOSITIONS)\ndef converter_repeat(self, *sizes):\n    def func(self, *sizes):\n        if get_channel_order(self) == ChannelOrder.TENSORFLOW:\n            sizes = permute_pytorch2keras(sizes)\n        return tf.tile(self, sizes)\n    return func\n```\n\n---\n\n## Deep dive\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003ch3\u003eAggressive transposition removal: fighting fire with fire\u003c/h3\u003e\u003c/summary\u003e\n\nDespite trying its best, Nobuco may produce outrageously inefficient graphs:\n\n```python\nclass MyModule(nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.conv1 = nn.Conv1d(3, 3, kernel_size=1)\n        self.conv2 = nn.Conv1d(6, 6, kernel_size=1)\n    \n                                    ################################\n                                    # How it's translated to Keras #\n                                    ################################\n    def forward(self, x):           #\n        x = self.conv1(x)           # \u003c- Output is in TENSORFLOW order\n                                    #\n        x = x.reshape(-1, 6, 2)     # \u003c- Expects input in PYTORCH order, transposition needed\n                                    #    Output is in PYTORCH order\n                                    #\n        x = self.conv2(x)           # \u003c- Expects input in TENSORFLOW order, transposition needed \n        return x                    #\n```\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/reshape1.png\" width=\"15%\"\u003e\n\u003c/p\u003e\n\nTwo transpositions out of nowhere?! How could Nobuco fail so miserably?\n\nLet's try it ourselves, then. First, the original operation. (implemented in Numpy, just not to be partial to either of the two frameworks)\n\n```python\nimport numpy as np\n\nx_torch = np.asarray([\n    [1, 2, 3, 4],\n    [5, 6, 7, 8],\n    [9, 10, 11, 12]\n])\nprint('x_torch:\\n', x_torch)\n\ny_torch = x_torch.reshape((6, 2))\nprint('y_torch:\\n', y_torch)\n```\n\n```console\nx_torch:\n [[ 1  2  3  4]\n [ 5  6  7  8]\n [ 9 10 11 12]]\ny_torch:\n [[ 1  2]\n [ 3  4]\n [ 5  6]\n [ 7  8]\n [ 9 10]\n [11 12]]\n```\n\nBut remember the catch: in Keras, the inputs to `reshape` are transposed relative to the original Pytorch implementation, because that's how `conv1` returns them.\n\n```python\nx_keras = x_torch.transpose()\nprint('x_keras:\\n', x_torch)\n```\n\n```console\nx_keras:\n [[ 1  5  9]\n [ 2  6 10]\n [ 3  7 11]\n [ 4  8 12]]\n```\n\nSo, if we just permute the `shape` parameter accordingly, will that work? No, the result is scrambled beyond recognition!\n\n\n```python\ndef reshape_keras_incorrect(x_keras, shape_torch):\n    shape_keras = list(reversed(shape_torch))\n    return x_keras.reshape(shape_keras)\n\ny_keras = reshape_keras_incorrect(x_keras, (6, 2))\nprint('y_keras:\\n', y_keras)\nprint('Is correct:', np.array_equal(y_keras.transpose(), y_torch))\n```\n\n```console\ny_keras:\n [[ 1  5  9  2  6 10]\n [ 3  7 11  4  8 12]]\nIs correct: False\n```\n\nTo get it work correctly for all shapes, we have to perform `reshape` on the original non-transposed tensor, i.e. prepare the input beforehand.\nWe also transpose the output to later pass it to Keras convolution. \n\n\n```python\ndef reshape_keras(x_keras, shape_torch):\n    x_torch = x_keras.transpose()\n    y_torch = x_torch.reshape(shape_torch)\n    y_keras = y_torch.transpose()\n    return y_keras\n\ny_keras = reshape_keras(x_keras, (6, 2))\nprint('y_keras:\\n', y_keras)\nprint('Is correct:', np.array_equal(y_keras.transpose(), y_torch))\n```\n\n```console\ny_keras:\n [[ 1  3  5  7  9 11]\n [ 2  4  6  8 10 12]]\nIs correct: True\n```\n\nIs there a better way? Yes, if we can afford to modify the Pytorch model. Remember the [pick-your-poison](#implementation-mismatch-pick-your-poison) section? Same thing.\n\n\u003e :dart:\n\u003e Here's the general recipe to get rid of redundant transpositions: \n\u003e 1) permute inputs to Tensorflow channel order\n\u003e 2) define the subgraph as you want to see it in the converted model\n\u003e 3) permute outputs back to Pytorch channel order\n\nSolving the transposition problem with more transpositions, huh?\nNo mystery here, two adjacent permutations are easily fused into one. Being opposites, `pytorch-\u003etensorflow` and `tensorflow-\u003epytorch` permutations just cancel each other out.\n\nBut wait, is Nobuco sophisticated enough to perform global optimization? It's not, and it doesn't.\nInstead, when it sees a `permute` op, it checks whether the op can be construed as transposition from Pytorch to Keras or vice versa. \nIf so, no work is done on the input tensor, only its metadata (`channel_order` field) is changed.\n\n```python\nclass MyModule(nn.Module):          ################################\n    # ...                           # How it's translated to Keras #\n                                    ################################\n    def forward(self, x):           #\n        x = self.conv1(x)           # \u003c- Output is in TENSORFLOW order\n                                    #\n        # BCH -\u003e BHC                #\n        x = x.permute(0, 2, 1)      # \u003c- No actual transposition done, just order marked as PYTORCH\n        # Reshape transposed input  #\n        x = x.reshape(-1, 2, 6)     # \u003c- Expects input in PYTORCH order, no transposition needed\n                                    #    Output is in PYTORCH order\n        # BHC -\u003e BCH                #\n        x = x.permute(0, 2, 1)      # \u003c- No actual transposition done, just order marked as TENSORFLOW\n                                    #\n        x = self.conv2(x)           # \u003c- Expects input in TENSORFLOW order, no transposition needed \n        return x                    #\n```\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/AlexanderLutsenko/nobuco/master/docs/reshape2.png\" width=\"15%\"\u003e\n\u003c/p\u003e\n\n\u003c/details\u003e\n\n---\n\n### Acknowledgements\n\nSlice assign converter is based on [Zaccharie Ramzi's tf-slice-assign script](https://github.com/zaccharieramzi/tf-slice-assign).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falexanderlutsenko%2Fnobuco","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falexanderlutsenko%2Fnobuco","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falexanderlutsenko%2Fnobuco/lists"}