{"id":22600882,"url":"https://github.com/mlrichter/receptive_field_analysis_toolbox","last_synced_at":"2025-04-05T21:09:02.603Z","repository":{"id":37023076,"uuid":"436921258","full_name":"MLRichter/receptive_field_analysis_toolbox","owner":"MLRichter","description":"A toolbox for receptive field analysis and visualizing neural network architectures","archived":false,"fork":false,"pushed_at":"2025-03-11T17:26:57.000Z","size":1415,"stargazers_count":113,"open_issues_count":13,"forks_count":5,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-29T20:07:22.291Z","etag":null,"topics":["deep-learning","hacktoberfest","machine-learning","neural-architecture-optimization","neural-architecture-search","neural-networks","pytorch","receptive-field","receptive-field-analysis","tensorflow","visualization"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MLRichter.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["MLRichter"]}},"created_at":"2021-12-10T09:26:28.000Z","updated_at":"2024-11-22T07:32:36.000Z","dependencies_parsed_at":"2023-01-24T12:45:55.397Z","dependency_job_id":"37dcef8a-f199-4ff3-a18b-16659323007d","html_url":"https://github.com/MLRichter/receptive_field_analysis_toolbox","commit_stats":{"total_commits":214,"total_committers":5,"mean_commits":42.8,"dds":0.3691588785046729,"last_synced_commit":"03f95bfe7acc8f34f1a5845915134cbdd9f7b623"},"previous_names":[],"tags_count":27,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MLRichter%2Freceptive_field_analysis_toolbox","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MLRichter%2Freceptive_field_analysis_toolbox/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MLRichter%2Freceptive_field_analysis_toolbox/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MLRichter%2Freceptive_field_analysis_toolbox/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MLRichter","download_url":"https://codeload.github.com/MLRichter/receptive_field_analysis_toolbox/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247399878,"owners_count":20932880,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","hacktoberfest","machine-learning","neural-architecture-optimization","neural-architecture-search","neural-networks","pytorch","receptive-field","receptive-field-analysis","tensorflow","visualization"],"created_at":"2024-12-08T12:12:49.626Z","updated_at":"2025-04-05T21:09:02.586Z","avatar_url":"https://github.com/MLRichter.png","language":"Python","funding_links":["https://github.com/sponsors/MLRichter"],"categories":[],"sub_categories":[],"readme":"# ReceptiveFieldAnalysisToolbox\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/MLRichter/receptive_field_analysis_toolbox/actions?query=workflow%3ACI\"\u003e\n    \u003cimg src=\"https://img.shields.io/github/workflow/status/MLRichter/receptive_field_analysis_toolbox/CI/main?label=CI\u0026logo=github\u0026style=flat-square\" alt=\"CI Status\" \u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://receptive-field-analysis-toolbox.readthedocs.io/en/latest/index.html\"\u003e\n    \u003cimg src=\"https://img.shields.io/readthedocs/receptive-field-analysis-toolbox.svg?logo=read-the-docs\u0026logoColor=fff\u0026style=flat-square\" alt=\"Documentation Status\"\u003e\n  \u003c/a\u003e\n    \u003ca href=\"https://codecov.io/gh/MLRichter/receptive_field_analysis_toolbox\"\u003e\n        \u003cimg src=\"https://codecov.io/gh/MLRichter/receptive_field_analysis_toolbox/branch/main/graph/badge.svg?token=3K52NPEAEU\"/\u003e\n    \u003c/a\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://python-poetry.org/\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/packaging-poetry-299bd7?style=flat-square\u0026logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAA4AAAASCAYAAABrXO8xAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAJJSURBVHgBfZLPa1NBEMe/s7tNXoxW1KJQKaUHkXhQvHgW6UHQQ09CBS/6V3hKc/AP8CqCrUcpmop3Cx48eDB4yEECjVQrlZb80CRN8t6OM/teagVxYZi38+Yz853dJbzoMV3MM8cJUcLMSUKIE8AzQ2PieZzFxEJOHMOgMQQ+dUgSAckNXhapU/NMhDSWLs1B24A8sO1xrN4NECkcAC9ASkiIJc6k5TRiUDPhnyMMdhKc+Zx19l6SgyeW76BEONY9exVQMzKExGKwwPsCzza7KGSSWRWEQhyEaDXp6ZHEr416ygbiKYOd7TEWvvcQIeusHYMJGhTwF9y7sGnSwaWyFAiyoxzqW0PM/RjghPxF2pWReAowTEXnDh0xgcLs8l2YQmOrj3N7ByiqEoH0cARs4u78WgAVkoEDIDoOi3AkcLOHU60RIg5wC4ZuTC7FaHKQm8Hq1fQuSOBvX/sodmNJSB5geaF5CPIkUeecdMxieoRO5jz9bheL6/tXjrwCyX/UYBUcjCaWHljx1xiX6z9xEjkYAzbGVnB8pvLmyXm9ep+W8CmsSHQQY77Zx1zboxAV0w7ybMhQmfqdmmw3nEp1I0Z+FGO6M8LZdoyZnuzzBdjISicKRnpxzI9fPb+0oYXsNdyi+d3h9bm9MWYHFtPeIZfLwzmFDKy1ai3p+PDls1Llz4yyFpferxjnyjJDSEy9CaCx5m2cJPerq6Xm34eTrZt3PqxYO1XOwDYZrFlH1fWnpU38Y9HRze3lj0vOujZcXKuuXm3jP+s3KbZVra7y2EAAAAAASUVORK5CYII=\" alt=\"Poetry\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/ambv/black\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/code%20style-black-000000.svg?style=flat-square\" alt=\"black\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/pre-commit/pre-commit\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit\u0026logoColor=white\u0026style=flat-square\" alt=\"pre-commit\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://pypi.org/project/rfa_toolbox/\"\u003e\n    \u003cimg src=\"https://img.shields.io/pypi/v/rfa_toolbox.svg?logo=python\u0026logoColor=fff\u0026style=flat-square\" alt=\"PyPI Version\"\u003e\n  \u003c/a\u003e\n  \u003cimg src=\"https://img.shields.io/pypi/pyversions/rfa_toolbox.svg?style=flat-square\u0026logo=python\u0026amp;logoColor=fff\" alt=\"Supported Python versions\"\u003e\n  \u003cimg src=\"https://img.shields.io/pypi/l/rfa_toolbox.svg?style=flat-square\" alt=\"License\"\u003e\n\u003c/p\u003e\n\nThis is RFA-Toolbox, a simple and easy-to-use library that allows you to optimize your neural network architectures\nusing receptive field analysis (RFA) and create graph visualizations of your architecture.\n\n## Installation\n\nInstall this via pip:\n\n`pip install rfa_toolbox`\n\n## What is Receptive Field Analysis?\n\nReceptive Field Analysis (RFA) is a simple yet effective way to optimize the efficiency of any neural architecture without\ntraining it.\n\n## Usage\n\nThis library allows you to look for certain inefficiencies withing your convolutional neural network setup without\never training the model.\nYou can do this simply by importing your architecture into the format of RFA-Toolbox and then use the in-build functions\nto visualize your architecture using GraphViz.\nThe visualization will automatically mark layers predicted to be unproductive red and critical layers, that are potentially unproductive orange.\nIn edge case scenarios, where the receptive field expands of the boundaries of the image on some but not all tensor-axis, the layer will be marked yellow,\nsince such a layer is probably not operating and maximum efficiency.\nBeing able to detect these types of inefficiencies is especially useful if you plan to train your model on resolutions that are substantially lower than the\ndesign-resolution of most models.\nAs an alternative, you can also use the graph from RFA-Toolbox to hook RFA-toolbox more directly into your program.\n\n### Examples\n\nThere are multiple ways to import your model into RFA-Toolbox for analysis, with additional ways being added in future\nreleases.\n\n#### PyTorch\n\nThe simplest way of importing a model is by directly extracting the compute-graph from the PyTorch-implementation of your model.\nHere is a simple example:\n\n```python\nimport torchvision\nfrom rfa_toolbox import create_graph_from_pytorch_model, visualize_architecture\nmodel = torchvision.models.alexnet()\ngraph = create_graph_from_pytorch_model(model)\nvisualize_architecture(\n    graph, f\"alexnet_32_pixel\", input_res=32\n).view()\n```\n\nThis will create a graph of your model and visualize it using GraphViz and color all layers that are predicted to be\nunproductive for an input resolution of 32x32 pixels:\n![rf_stides.PNG](https://github.com/MLRichter/receptive_field_analysis_toolbox/blob/main/images/alexnet.PNG?raw=true)\n\nKeep in mind that the Graph is reverse-engineerd from the PyTorch JIT-compiler, therefore no looping-logic is\nallowed within the forward pass of the model.\nAlso, the use of the functional-library for stateful-operations like pooling or convolutional layers is discouraged,\nsince it can cause malformed graphs during the reverse engineering of the compute graph.\n\n#### Tensorflow / Keras\n\nSimilar to the PyTorch-import, models can be imported from TensorFlow as well.\n\nThe code below will create a graph of VGG16 and visualize it using GraphViz and color all layers that are predicted to be\nunproductive for an input resolution of 32x32 pixels. This is analog to the PyTorch-variant depicted above:\n\n```python\n\nfrom keras.applications.vgg19 import VGG19\nfrom rfa_toolbox import create_graph_from_tensorflow_model\n\nmodel = VGG19(include_top=True, weights=None)\ngraph: EnrichedNetworkNode = create_graph_from_tensorflow_model(model)\nvisualize_architecture(graph, \"VGG16\", input_res=32).view()\n\n```\n\nThis will create the following visualization:\n![vgg16.PNG](https://github.com/MLRichter/receptive_field_analysis_toolbox/blob/main/images/vgg16.PNG?raw=true)\n\nCurrently, only the use of the keras.Model object is supported.\n\n#### Custom\n\nIf you are not able to automatically import your model from PyTorch or you just want some visualization, you can\nalso directly implement the model with the propriatary-Graph-format of RFA-Toolbox.\nThis is similar to coding a compute-graph in a declarative style like in TensorFlow 1.x.\n\n```python\nfrom rfa_toolbox import visualize_architecture\nfrom rfa_toolbox.graphs import EnrichedNetworkNode, LayerDefinition\n\n\nconv1 = EnrichedNetworkNode(\n    name=\"Conv1\",\n    layer_info=LayerDefinition(\n        name=\"Conv3x3\",\n        kernel_size=3, stride_size=1,\n        filters=64\n    ),\n    predecessors=[]\n)\nconv2 = EnrichedNetworkNode(\n    name=\"Conv2\",\n    layer_info=LayerDefinition(\n        name=\"Conv3x3\",\n        kernel_size=3, stride_size=1,\n        filters=128\n    ),\n    predecessors=[conv1]\n)\n\nconv3 = EnrichedNetworkNode(\n    name=\"Conv3\",\n    layer_info=LayerDefinition(\n        name=\"Conv3x3\",\n        kernel_size=3, stride_size=1,\n        filters=256\n    ),\n    predecessors=[conv1]\n)\n\nconv4 = EnrichedNetworkNode(\n    name=\"Conv4\",\n    layer_info=LayerDefinition(\n        name=\"Conv3x3\",\n        kernel_size=3, stride_size=1,\n        filters=256\n    ),\n    predecessors=[conv2, conv3]\n)\n\nout = EnrichedNetworkNode(\n    name=\"Softmax\",\n    layer_info=LayerDefinition(\n        name=\"Fully Connected\",\n        units=1000\n    ),\n    predecessors=[conv4]\n)\nvisualize_architecture(\n    out, f\"example_model\", input_res=32\n).view()\n\n```\n\nThis will produce the following graph:\n\n![simple_conv.png](https://github.com/MLRichter/receptive_field_analysis_toolbox/blob/main/images/simple_conv.png?raw=true)\n\n#### Finding the feasible input range of a neural network\n\nWith RFA-Toolbox you can also easily determine in which range your input resolution should be in order to allow\nthe network to use all parameters to solve the problem, which is the most efficient way to operate the model.\n\n```python\n\nimport torchvision\nfrom rfa_toolbox import create_graph_from_pytorch_model, input_resolution_range\nmodel = torchvision.models.resnet50()\ngraph = create_graph_from_pytorch_model(model)\n# set filter_all_inf_rf to True, if your model contains Squeeze-And-Excitation-Modules\nmin_res, max_res = input_resolution_range(graph, filter_all_inf_rf=False) # (75, 75), (427, 427)\nprint(\"Mininum Resolution:\", min_res, \"Maximum Resolution:\", max_res)\n```\n\nEvery resolution below the minimum resolution will result in unproductive layers in the network, due to oversized\nreceptive field sizes (see next section for details).\nEvery resolution above the maximum resolution exceeds the maximum receptive field size of any non-fully-connected\nlayer in the model, which means that potentially patterns exist in the input that the network cannot extract with\nits feature-extractor.\n\n### A quick primer on the Receptive Field\n\nTo understand how RFA works, we first need to understand what a receptive field is, and it's effect on what the network is learning to detect.\nEvery layer in a (convolutional) neural network has a receptive field. It can be considered the \"field of view\" of this layer.\nIn more precise terms, we define a receptive field as the area influencing the output of a single position of the\nconvolutional kernel.\nHere is a simple, 1-dimensional example:\n![rf.PNG](https://github.com/MLRichter/receptive_field_analysis_toolbox/blob/main/images/rf.PNG?raw=true)\nThe first layer of this simple architecture can only ever \"see\" the information the input pixels directly\nunder its kernel, in this scenario 3 pixels.\nAnother observation we can make from this example is that the receptive field size is expanding from layer to layer.\nThis is happening, because the consecutive layers also have kernel sizes greater than 1 pixel, which means that\nthey combine multiple adjacent positions on the feature map into a single position in their output.\nIn other words, every consecutive layer adds additional context to each feature map position by expanding the receptive field.\nThis ultimately allows networks to go from detecting small and simple patterns to big and very complicated ones.\n\nThe effective size of the kernel is not the only factor influence the growth of the receptive field size.\nAnother important factor is the stride size:\n![rf_stides.PNG](https://github.com/MLRichter/receptive_field_analysis_toolbox/blob/main/images/rf_strides.PNG?raw=true)\nThe stride size is the size of the step between the individual kernel positions. Commonly, every possible\nposition is evaluated, which is not affecting the receptive field size in any way.\nWhen the stride size is greater than one however valid positions of the kernel are skipped, which reduces\nthe size of the feature map. Since now information on the feature map is now condensed on fewer feature map positions,\nthe growth of the receptive field is multiplied for future layers.\nIn real-world architectures, this is typically the case when downsampling layers like convolutions with a stride size of 2\nare used.\n\n### Why does the Receptive Field Matter?\n\nAt this point you may be wondering why the receptive field of all things is useful for optimizing an\narchitecture.\nThe short answer to this is: because it influences where the network can process patterns of a certain size.\nSimply speaking each convolutional layer is only able to detect patterns of a certain size because of its receptive field.\nInterestingly this also means that there is an upper limit to the usefulness of expanding the receptive field.\nAt the latest, this is the case when the receptive field of a layer is BIGGER than the input image, since no novel\ncontext can be added at this point.\nFor convolutional layers this is a problem, because layers past this \"Border Layer\" now lack the primary mechenism convolutional layers\nuse to improve the intermediate representation of the data, making these layers unproductive.\nIf you are interested in the details of this phenomenon I recommend that you read these:\n\n- [(Input) Size Matters for Convolutional Neural Network Classifier](https://link.springer.com/chapter/10.1007/978-3-030-86340-1_11)\n- [Should You Go Deeper? Optimizing Convolutional Neural Network Architectures without Training](https://arxiv.org/abs/2106.12307)\n  (published at the 20th IEEE International Conference for Machine Learning Application - ICMLA)\n\n### Optimizing Architectures using Receptive Field Analysis\n\nSo far, we learned that the expansion of the receptive field is the primary mechanism for improving\nthe intermediate solution utilized by convolutional layers.\nAt the point where this is no longer possible, layers are not able to contribute to the quality of the output of the model\nand become unproductive.\nWe refer to these layers as unproductive layers. Layers who advance the receptive field sizes beyond the input resolution\nare referred to as critical layers.\nCritical layers are not necessarily unproductive, since they are still able to incorporate some novel context into the data,\ndepending on how large the receptive field size of the input is.\n\nOf course, being able to predict why and which layer will become dead weight during training is highly useful, since\nwe can now adjust the design of the architecture to fit our input resolution better without spending time on training models.\nDepending on the requirements, we may choose to emphasize efficiency by primarily removing unproductive layers. Another\noption is to focus on predictive performance by making the unproductive layers productive again.\n\nWe now illustrate how you may choose to optimize an architecture on a simple example:\n\nLet's take the ResNet architecture, which is a very popular CNN-model.\nWe want to train ResNet18 on ResizedImageNet16, which has a 16 pixel input resolution.\nWhen we apply Receptive Field Analysis, we can see that most convolutional layers will in fact not contribute\nto the inference process (unproductive layers marked red, probable unproductive layers marked orange):\n\n![resnet18.PNG](https://github.com/MLRichter/receptive_field_analysis_toolbox/blob/main/images/resnet18.png?raw=true)\n\nWe can clearly see that most of the network's layers will not contribute anything useful to the quality of the output, since\ntheir receptive field sizes are too large.\n\nFrom here on we have multiple ways of optimizing the setup.\nOf course, we can simply increase the resolution, to involve more layers in the inference process, but that is usually\nvery expensive from a computational point of view.\nIn the first scenario, we are not interested in increasing the predictive performance of the model, we simply want to save\ncomputational resources. We reduce the kernel size of the\nfirst layer to 3x3 from 7x7.\nThis change allows the first three building blocks to contribute more to\nthe quality of the prediction, since no layer is predicted to be unproductive.\nWe then simply replace the remaining building blocks with a simple output head.\nThis new architecture then looks like this:\n\n![resnet18eff.PNG](https://github.com/MLRichter/receptive_field_analysis_toolbox/blob/main/images/resnet18eff.png?raw=true)\n\nNote that all previously unproductive layers are now either removed or only marked as \"critical\", which\nis generally not a big problem, since the receptive field size is \"reset\" by the receptive field size\nafter each building block.\nAlso note that fully connected layers are always marked as critical or unproductive, since they\ntechnically have an infinite receptive field size.\n\nThe resulting architecture achieves slightly better predictive performance as\nthe original architecture, but with substantially lower computational cost.\nIn this case we save approx. 80% of the computational cost and improve the predictive performance slightly\nfrom 17% to 18%.\n\nIn another scenario we may not be satisfied with the predictive performance.\nIn other words, we want to make use of the underutilized parameters of the network\nby turning all unproductive layers into productive layers.\nWe achieve this by changing their receptive field sizes.\nThe biggest lever when it comes to changing the receptive field size is always the quantity of downsampling layers.\nDownsampling layers have a multiplicative effect on the growth of the receptive field for all consecutive layers.\nWe can exploit this by simply removing the MaxPooling layer, which is the second layer of the original\narchitecture.\nWe also reduce the kernel size of the first layer to 3x3 from 7x7, and it's stride size to 1.\nThis drastically reduces the receptive field sizes of the entire architecture, making most layers productive again.\nWe address the remaining unproductive layers to by removing the final downsampling layer and distributing the building\nblocks as evenly as possible among the three stages between the remaining downsampling layers.\n\nThe resulting architecture now looks like this:\n\n![resnet18perf.PNG](https://github.com/MLRichter/receptive_field_analysis_toolbox/blob/main/images/resnet18perf.png?raw=true)\n\nThe architecture now no longer has unproductive layers in their building blocks and only 2 critical layers.\nThis improved architecture also achieves 34% Top1-Accuracy in ResizedImageNet16 instead of the 17% of the original architecture.\nHowever, this improvement comes at a price, since the removed downsampling layers have a negative impact on the computations\nrequired to process an image, which increases by roughly a factor of 8.\n\nIn any way, RFAToolbox allows you to optimize your convolutional neural network architectures\nfor efficiency, performance or a sweetspot between the two without the need for long-running trial-and-error sessions.\n\n## Citation\n\nIf you use ReceptiveFieldAnalysisToolbox for your publication, please cite [1] and [2].\n\n[1] M.L. Richter, J. Schöning, A. Wiedenroth \u0026 U. Krumnack. Should You Go Deeper? Optimizing Convolutional Neural Network Architectures without Training. In International Conference On Machine Learning And Applications (ICMLA) 2021. IEEE.\n\n[2] M.L. Richter, J. Schöning, A. Wiedenroth \u0026 U. Krumnack. Receptive Field Analysis for Optimizing Convolutional Neural Network Architectures Without Training. In Deep Learning Applications 2022. Springer (InPress).\n\n## Credits\n\nThis package was created with\n[Cookiecutter](https://github.com/audreyr/cookiecutter) and the\n[browniebroke/cookiecutter-pypackage](https://github.com/browniebroke/cookiecutter-pypackage)\nproject template.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmlrichter%2Freceptive_field_analysis_toolbox","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmlrichter%2Freceptive_field_analysis_toolbox","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmlrichter%2Freceptive_field_analysis_toolbox/lists"}