{"id":22558602,"url":"https://github.com/vduchauffour/transformers-visualizer","last_synced_at":"2026-03-19T23:08:11.775Z","repository":{"id":64988340,"uuid":"560868350","full_name":"VDuchauffour/transformers-visualizer","owner":"VDuchauffour","description":"Explain your 🤗 transformers without effort! Plot the internal behavior of your model.","archived":false,"fork":false,"pushed_at":"2022-12-29T16:55:03.000Z","size":2424,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-08T04:36:49.065Z","etag":null,"topics":["ai","explainability","explainable-ai","huggingface","huggingface-transformers","nlp","transformer","transformers"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/VDuchauffour.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-11-02T12:57:06.000Z","updated_at":"2023-03-23T03:36:34.000Z","dependencies_parsed_at":"2023-01-13T15:05:54.193Z","dependency_job_id":null,"html_url":"https://github.com/VDuchauffour/transformers-visualizer","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VDuchauffour%2Ftransformers-visualizer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VDuchauffour%2Ftransformers-visualizer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VDuchauffour%2Ftransformers-visualizer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VDuchauffour%2Ftransformers-visualizer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/VDuchauffour","download_url":"https://codeload.github.com/VDuchauffour/transformers-visualizer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246025918,"owners_count":20711575,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","explainability","explainable-ai","huggingface","huggingface-transformers","nlp","transformer","transformers"],"created_at":"2024-12-07T20:16:18.222Z","updated_at":"2026-03-19T23:08:11.740Z","avatar_url":"https://github.com/VDuchauffour.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003eTransformers visualizer\u003c/h1\u003e\n\u003cp align=\"center\"\u003eExplain your 🤗 transformers without effort!\u003c/p\u003e\n\u003ch1 align=\"center\"\u003e\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://opensource.org/licenses/Apache-2.0\"\u003e\n        \u003cimg alt=\"Apache\" src=\"https://img.shields.io/badge/License-Apache%202.0-blue.svg\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://github.com/VDuchauffour/transformers-visualizer/blob/main/.github/workflows/unit_tests.yml\"\u003e\n        \u003cimg alg=\"Unit tests\" src=\"https://github.com/VDuchauffour/transformers-visualizer/actions/workflows/unit_tests.yml/badge.svg\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"\"\u003e\n        \u003cimg alt=\"PyPI - Python Version\" src=\"https://img.shields.io/pypi/pyversions/transformers-visualizer?color=red\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://github.com/VDuchauffour/transformers-visualizer\"\u003e\n        \u003cimg alt=\"PyPI - Package Version\" src=\"https://img.shields.io/pypi/v/transformers-visualizer?label=version\"\u003e\n    \u003c/a\u003e\n\u003c/p\u003e\n\nTransformers visualizer is a python package designed to work with the [🤗 transformers](https://huggingface.co/docs/transformers/index) package. Given a `model` and a `tokenizer`, this package supports multiple ways to explain your model by plotting its internal behavior.\n\nThis package is mostly based on the [Captum][Captum] tutorials [[1]][captum_part1] [[2]][Captum_part2].\n\n## Installation\n\n```shell\npip install transformers-visualizer\n```\n\n## Quickstart\n\nLet's define a model, a tokenizer and a text input for the following examples.\n\n```python\nfrom transformers import AutoModel, AutoTokenizer\n\nmodel_name = \"bert-base-uncased\"\nmodel = AutoModel.from_pretrained(model_name)\ntokenizer = AutoTokenizer.from_pretrained(model_name)\ntext = \"The dominant sequence transduction models are based on complex recurrent or convolutional neural networks that include an encoder and a decoder.\"\n```\n\n### Visualizers\n\n\u003cdetails\u003e\u003csummary\u003eAttention matrices of a specific layer\u003c/summary\u003e\n\n\u003cp\u003e\n\n```python\nfrom transformers_visualizer import TokenToTokenAttentions\n\nvisualizer = TokenToTokenAttentions(model, tokenizer)\nvisualizer(text)\n```\n\nInstead of using `__call__` function, you can use the `compute` method. Both work in place, `compute` method allows chaining method.\n\n`plot` method accept a layer index as parameter to specify which part of your model you want to plot. By default, the last layer is plotted.\n\n```python\nimport matplotlib.pyplot as plt\n\nvisualizer.plot(layer_index = 6)\nplt.savefig(\"token_to_token.jpg\")\n```\n\n\u003cp align=\"center\"\u003e\n    \u003cimg alt=\"token to token\" src=\"https://raw.githubusercontent.com/VDuchauffour/transformers-visualizer/main/images/token_to_token.jpg\" /\u003e\n\u003c/p\u003e\n\n\u003c/p\u003e\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eAttention matrices normalized across head axis\u003c/summary\u003e\n\n\u003cp\u003e\n\nYou can specify the `order` used in `torch.linalg.norm` in `__call__` and `compute` methods. By default, an L2 norm is applied.\n\n```python\nfrom transformers_visualizer import TokenToTokenNormalizedAttentions\n\nvisualizer = TokenToTokenNormalizedAttentions(model, tokenizer)\nvisualizer.compute(text).plot()\n```\n\n\u003cp align=\"center\"\u003e\n    \u003cimg alt=\"normalized token to token\"src=\"https://raw.githubusercontent.com/VDuchauffour/transformers-visualizer/main/images/token_to_token_normalized.jpg\" /\u003e\n\u003c/p\u003e\n\n\u003c/p\u003e\n\n\u003c/details\u003e\n\n## Plotting\n\n`plot` method accept to skip special tokens with the parameter `skip_special_tokens`, by default it's set to `False`.\n\nYou can use the following imports to use plotting functions directly.\n\n```python\nfrom transformers_visualizer.plotting import plot_token_to_token, plot_token_to_token_specific_dimension\n```\n\nThese functions or the `plot` method of a visualizer can use the following parameters.\n\n- `figsize (Tuple[int, int])`: Figsize of the plot. Defaults to (20, 20).\n- `ticks_fontsize (int)`: Ticks fontsize. Defaults to 7.\n- `title_fontsize (int)`: Title fontsize. Defaults to 9.\n- `cmap (str)`: Colormap. Defaults to \"viridis\".\n- `colorbar (bool)`: Display colorbars. Defaults to True.\n\n## Upcoming features\n\n- [x] Add an option to mask special tokens.\n- [ ] Add an option to specify head/layer indices to plot.\n- [ ] Add other plotting backends such as Plotly, Bokeh, Altair.\n- [ ] Implement other visualizers such as [vector norm](https://arxiv.org/pdf/2004.10102.pdf).\n\n## References\n\n- [[1]][captum_part1] Captum's BERT tutorial (part 1)\n- [[2]][captum_part2] Captum's BERT tutorial (part 2)\n\n## Acknowledgements\n\n- [Transformers Interpret](https://github.com/cdpierse/transformers-interpret) for the idea of this project.\n\n[Captum]: https://captum.ai/\n[captum_part1]: https://captum.ai/tutorials/Bert_SQUAD_Interpret\n[Captum_part2]: https://captum.ai/tutorials/Bert_SQUAD_Interpret2","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvduchauffour%2Ftransformers-visualizer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvduchauffour%2Ftransformers-visualizer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvduchauffour%2Ftransformers-visualizer/lists"}