{"id":13535268,"url":"https://github.com/jessevig/bertviz","last_synced_at":"2025-05-13T20:09:15.071Z","repository":{"id":37251411,"uuid":"162021652","full_name":"jessevig/bertviz","owner":"jessevig","description":"BertViz: Visualize Attention in NLP Models (BERT, GPT2, BART, etc.) ","archived":false,"fork":false,"pushed_at":"2023-08-24T13:40:48.000Z","size":203130,"stargazers_count":7356,"open_issues_count":22,"forks_count":816,"subscribers_count":73,"default_branch":"master","last_synced_at":"2025-04-28T10:55:16.016Z","etag":null,"topics":["bert","gpt2","machine-learning","natural-language-processing","neural-network","nlp","pytorch","roberta","transformer","transformers","visualization"],"latest_commit_sha":null,"homepage":"https://towardsdatascience.com/deconstructing-bert-part-2-visualizing-the-inner-workings-of-attention-60a16d86b5c1","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jessevig.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2018-12-16T16:50:42.000Z","updated_at":"2025-04-28T04:18:48.000Z","dependencies_parsed_at":"2022-07-14T11:30:44.786Z","dependency_job_id":"68a1cc67-4e4f-48d7-a7cb-4d6f93f091b8","html_url":"https://github.com/jessevig/bertviz","commit_stats":{"total_commits":301,"total_committers":7,"mean_commits":43.0,"dds":0.2691029900332226,"last_synced_commit":"b42b61ec58f6fc72a55e987b24e55d141e80c4bb"},"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jessevig%2Fbertviz","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jessevig%2Fbertviz/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jessevig%2Fbertviz/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jessevig%2Fbertviz/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jessevig","download_url":"https://codeload.github.com/jessevig/bertviz/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254020605,"owners_count":22000752,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bert","gpt2","machine-learning","natural-language-processing","neural-network","nlp","pytorch","roberta","transformer","transformers","visualization"],"created_at":"2024-08-01T08:00:52.367Z","updated_at":"2025-05-13T20:09:15.027Z","avatar_url":"https://github.com/jessevig.png","language":"Python","funding_links":[],"categories":["Pretrained Language Model","BERT  visualization toolkit:","Python","XAI Libraries for NLP","Explainability","Tools","其他_NLP自然语言处理","Technologies","BertViz"],"sub_categories":["Repository","Other","其他_文本生成、文本对话"],"readme":"\u003ch1 align=\"center\"\u003e\n    BertViz\n\u003c/h1\u003e\n\u003ch3 align=\"center\"\u003e\n Visualize Attention in NLP Models\n\u003c/h3\u003e\n\u003ch3 align=\"center\"\u003e\n    \u003ca href=\"#-quick-tour\"\u003eQuick Tour\u003c/a\u003e \u0026bull;\n    \u003ca href=\"#%EF%B8%8F-getting-started\"\u003eGetting Started\u003c/a\u003e \u0026bull;\n    \u003ca href=\"https://colab.research.google.com/drive/1hXIQ77A4TYS4y3UthWF-Ci7V7vVUoxmQ?usp=sharing\"\u003eColab Tutorial\u003c/a\u003e \u0026bull;\n    \u003ca href=\"#-paper\"\u003ePaper\u003c/a\u003e\n\u003c/h3\u003e\n\nBertViz is an interactive tool for visualizing attention in [Transformer](https://jalammar.github.io/illustrated-transformer/) language models such as BERT, GPT2, or T5. It can be run inside a Jupyter or Colab\n notebook through a simple Python API that supports most [Huggingface models](https://huggingface.co/models). BertViz extends the\n   [Tensor2Tensor visualization tool](https://github.com/tensorflow/tensor2tensor/tree/master/tensor2tensor/visualization)\n    by [Llion Jones](https://medium.com/@llionj), providing multiple views that each offer a unique lens into the attention mechanism.\n\nGet updates for this and related projects on [Twitter\n![Twitter logo](images/twitter.svg)](https://twitter.com/jesse_vig).\n\n## 🚀 Quick Tour\n\n### Head View\nThe *head view* visualizes attention for one or more attention heads in the same \n layer. It is based on the excellent [Tensor2Tensor visualization tool](https://github.com/tensorflow/tensor2tensor/tree/master/tensor2tensor/visualization) by [Llion Jones](https://medium.com/@llionj). \n\n🕹 Try out the head view in the [\u003cb\u003e\u003cu\u003eInteractive Colab Tutorial\u003c/u\u003e\u003c/b\u003e](https://colab.research.google.com/drive/1hXIQ77A4TYS4y3UthWF-Ci7V7vVUoxmQ?usp=sharing) (all visualizations pre-loaded).\n\u003cp\u003e\n    \u003cimg src=\"https://raw.githubusercontent.com/jessevig/bertviz/master/images/head-view.gif\" width=\"425\"/\u003e\n\u003c/p\u003e\n\n\n### Model View \n\nThe *model view* shows a bird's-eye view of attention across all layers and heads.\n\n🕹 Try out the model view in the [\u003cb\u003e\u003cu\u003eInteractive Colab Tutorial\u003c/u\u003e\u003c/b\u003e](https://colab.research.google.com/drive/1hXIQ77A4TYS4y3UthWF-Ci7V7vVUoxmQ?usp=sharing) (all visualizations pre-loaded).\n\n![model view](images/model-view-noscroll.gif)\n\n### Neuron View \nThe *neuron view* visualizes individual neurons in the query and key vectors and shows how they are used to compute attention.\n\n🕹 Try out the neuron view in the [\u003cb\u003e\u003cu\u003eInteractive Colab Tutorial\u003c/u\u003e\u003c/b\u003e](https://colab.research.google.com/drive/1hXIQ77A4TYS4y3UthWF-Ci7V7vVUoxmQ?usp=sharing) (all visualizations pre-loaded).\n\n\n![neuron view](images/neuron-view-dark.gif)\n\n## ⚡️ Getting Started\n\n### Running BertViz in a Jupyter Notebook\n\nFrom the command line:\n\n```bash\npip install bertviz\n```\nYou must also have Jupyter Notebook and ipywidgets installed:\n\n```bash\npip install jupyterlab\npip install ipywidgets\n```\n(If you run into any issues installing Jupyter or ipywidgets, consult the documentation [here](https://jupyter.org/install) and [here](https://ipywidgets.readthedocs.io/en/stable/user_install.html).)\n\nTo create a new Jupyter notebook, simply run:\n\n```bash\njupyter notebook\n```\n\nThen click `New` and select `Python 3 (ipykernel)` if prompted.\n\n\n### Running BertViz in Colab\n\nTo run in [Colab](https://colab.research.google.com/), simply add the following cell at the beginning of your Colab notebook:\n\n```\n!pip install bertviz\n```\n\n### Sample code\nRun the following code to load the `xtremedistil-l12-h384-uncased` model and display it in the model view: \n\n```python\nfrom transformers import AutoTokenizer, AutoModel, utils\nfrom bertviz import model_view\nutils.logging.set_verbosity_error()  # Suppress standard warnings\n\nmodel_name = \"microsoft/xtremedistil-l12-h384-uncased\"  # Find popular HuggingFace models here: https://huggingface.co/models\ninput_text = \"The cat sat on the mat\"  \nmodel = AutoModel.from_pretrained(model_name, output_attentions=True)  # Configure model to return attention values\ntokenizer = AutoTokenizer.from_pretrained(model_name)\ninputs = tokenizer.encode(input_text, return_tensors='pt')  # Tokenize input text\noutputs = model(inputs)  # Run model\nattention = outputs[-1]  # Retrieve attention from model outputs\ntokens = tokenizer.convert_ids_to_tokens(inputs[0])  # Convert input ids to token strings\nmodel_view(attention, tokens)  # Display model view\n```\n\nThe visualization may take a few seconds to load. Feel free to experiment with different input texts and\n [models](https://huggingface.co/models). \nSee [Documentation](#-documentation) for additional use cases and examples, e.g., encoder-decoder models.\n\n#### Running sample notebooks\n\nYou may also run any of the sample [notebooks](notebooks/) included with BertViz:\n\n```bash\ngit clone --depth 1 git@github.com:jessevig/bertviz.git\ncd bertviz/notebooks\njupyter notebook\n```\n## 🕹 Interactive Tutorial\n\nCheck out the [\u003cb\u003e\u003cu\u003eInteractive Colab Tutorial\u003c/u\u003e\u003c/b\u003e](https://colab.research.google.com/drive/1hXIQ77A4TYS4y3UthWF-Ci7V7vVUoxmQ?usp=sharing) \nto learn more about BertViz and try out the tool. \u003cb\u003eNote\u003c/b\u003e: all visualizations are pre-loaded, so there is no need to execute any cells.\n\n[![Tutorial](images/tutorial-screenshots.jpg)](https://colab.research.google.com/drive/1hXIQ77A4TYS4y3UthWF-Ci7V7vVUoxmQ?usp=sharing)\n\n\n## 📖 Documentation\n\n### Table of contents\n\n- [Self-attention models (BERT, GPT-2, etc.)](#self-attention-models-bert-gpt-2-etc)\n  * [Head and Model Views](#head-and-model-views)\n  * [Neuron View](#neuron-view-1)\n- [Encoder-decoder models (BART, T5, etc.)](#encoder-decoder-models-bart-t5-etc)\n- [Installing from source](#installing-from-source)\n- [Additional options](#additional-options)\n  * [Dark / light mode](#dark--light-mode)\n  * [Filtering layers](#filtering-layers)\n  * [Setting default layer/head(s)](#setting-default-layerheads)\n  * [Visualizing sentence pairs](#visualizing-sentence-pairs)\n  * [Obtain HTML representations](#obtain-HTML-representations)\n  * [Non-Huggingface models](#non-huggingface-models)\n - [Limitations](#%EF%B8%8F-limitations)\n\n### Self-attention models (BERT, GPT-2, etc.)\n\n#### Head and Model Views\nFirst load a Huggingface model, either a pre-trained model as shown below, or your own fine-tuned model.\n Be sure to set `output_attentions=True`.\n```python\nfrom transformers import AutoTokenizer, AutoModel, utils\nutils.logging.set_verbosity_error()  # Suppress standard warnings\ntokenizer = AutoTokenizer.from_pretrained(\"bert-base-uncased\")\nmodel = AutoModel.from_pretrained(\"bert-base-uncased\", output_attentions=True)\n```\n\nThen prepare inputs and compute attention:\n\n```python\ninputs = tokenizer.encode(\"The cat sat on the mat\", return_tensors='pt')\noutputs = model(inputs)\nattention = outputs[-1]  # Output includes attention weights when output_attentions=True\ntokens = tokenizer.convert_ids_to_tokens(inputs[0]) \n```\n\nFinally, display the attention weights using the [`head_view`](bertviz/head_view.py) or [`model_view`](bertviz/model_view.py)\n functions:\n\n```python\nfrom bertviz import head_view\nhead_view(attention, tokens)\n```\n\n\u003cb\u003eExamples\u003c/b\u003e: DistilBERT ([Model View Notebook](notebooks/model_view_distilbert.ipynb), [Head View Notebook](notebooks/head_view_distilbert.ipynb))\n\nFor full API, please refer to the source code for the [head view](bertviz/head_view.py) or [model view](bertviz/model_view.py).\n\n\n#### Neuron View\n\nThe neuron view is invoked differently than the head view or model view, due to requiring access to the model's\nquery/key vectors, which are not returned through the Huggingface API. It is currently limited to custom versions of BERT, GPT-2, and\nRoBERTa included with BertViz.\n\n```python\n# Import specialized versions of models (that return query/key vectors)\nfrom bertviz.transformers_neuron_view import BertModel, BertTokenizer\nfrom bertviz.neuron_view import show\n\nmodel_type = 'bert'\nmodel_version = 'bert-base-uncased'\ndo_lower_case = True\nsentence_a = \"The cat sat on the mat\"\nsentence_b = \"The cat lay on the rug\"\nmodel = BertModel.from_pretrained(model_version, output_attentions=True)\ntokenizer = BertTokenizer.from_pretrained(model_version, do_lower_case=do_lower_case)\nshow(model, model_type, tokenizer, sentence_a, sentence_b, layer=2, head=0)\n```\n\n\u003cb\u003eExamples\u003c/b\u003e:\nBERT ([Notebook](notebooks/neuron_view_bert.ipynb),\n[Colab](https://colab.research.google.com/drive/1m37iotFeubMrp9qIf9yscXEL1zhxTN2b)) •\nGPT-2 ([Notebook](notebooks/neuron_view_gpt2.ipynb),\n[Colab](https://colab.research.google.com/drive/1s8XCCyxsKvNRWNzjWi5Nl8ZAYZ5YkLm_)) •\nRoBERTa\n([Notebook](notebooks/neuron_view_roberta.ipynb))  \n\nFor full API, please refer to the [source](bertviz/neuron_view.py).\n\n### Encoder-decoder models (BART, T5, etc.)\n\nThe head view and model view both support encoder-decoder models.\n\nFirst, load an encoder-decoder model:\n\n```python\nfrom transformers import AutoTokenizer, AutoModel\n\ntokenizer = AutoTokenizer.from_pretrained(\"Helsinki-NLP/opus-mt-en-de\")\nmodel = AutoModel.from_pretrained(\"Helsinki-NLP/opus-mt-en-de\", output_attentions=True)\n```\n\nThen prepare the inputs and compute attention:\n```python\nencoder_input_ids = tokenizer(\"She sees the small elephant.\", return_tensors=\"pt\", add_special_tokens=True).input_ids\nwith tokenizer.as_target_tokenizer():\n    decoder_input_ids = tokenizer(\"Sie sieht den kleinen Elefanten.\", return_tensors=\"pt\", add_special_tokens=True).input_ids\n\noutputs = model(input_ids=encoder_input_ids, decoder_input_ids=decoder_input_ids)\n\nencoder_text = tokenizer.convert_ids_to_tokens(encoder_input_ids[0])\ndecoder_text = tokenizer.convert_ids_to_tokens(decoder_input_ids[0])\n```\n\nFinally, display the visualization using either [`head_view`](bertviz/head_view.py) or [`model_view`](bertviz/model_view.py).\n```python\nfrom bertviz import model_view\nmodel_view(\n    encoder_attention=outputs.encoder_attentions,\n    decoder_attention=outputs.decoder_attentions,\n    cross_attention=outputs.cross_attentions,\n    encoder_tokens= encoder_text,\n    decoder_tokens = decoder_text\n)\n```\n\nYou may select `Encoder`, `Decoder`, or `Cross` attention from the drop-down in the upper left corner of the visualization.\n\n\u003cb\u003eExamples\u003c/b\u003e: MarianMT ([Notebook](notebooks/model_view_encoder_decoder.ipynb)) • BART ([Notebook](notebooks/model_view_bart.ipynb))\n\nFor full API, please refer to the source code for the [head view](bertviz/head_view.py) or [model view](bertviz/model_view.py).\n\n### Installing from source\n```bash\ngit clone https://github.com/jessevig/bertviz.git\ncd bertviz\npython setup.py develop\n```\n\n### Additional options\n\n#### Dark / light mode\n\nThe model view and neuron view support dark (default) and light modes. You may set the mode using\nthe `display_mode` parameter:\n```python\nmodel_view(attention, tokens, display_mode=\"light\")\n```\n\n\n#### Filtering layers\n\nTo improve the responsiveness of the tool when visualizing larger models or inputs, you may set the `include_layers`\n parameter to restrict the visualization to a subset of layers (zero-indexed). This option is available in the head view and model\nview.\n\n**Example:** Render model view with only layers 5 and 6 displayed\n```python\nmodel_view(attention, tokens, include_layers=[5, 6])\n```\n\nFor the model view, you may also restrict the visualization to a subset of attention heads (zero-indexed) by setting the \n`include_heads` parameter. \n\n\n#### Setting default layer/head(s)\n\nIn the head view, you may choose a specific `layer` and collection of `heads` as the default selection when the\n visualization first renders. Note: this is different from the `include_heads`/`include_layers` parameter (above), which \n removes layers and heads from the visualization completely.\n\n**Example:** Render head view with layer 2 and heads 3 and 5 pre-selected\n```python\nhead_view(attention, tokens, layer=2, heads=[3,5])\n```\n\nYou may also pre-select a specific `layer` and single `head` for the neuron view. \n\n#### Visualizing sentence pairs\n\nSome models, e.g. BERT, accept a pair of sentences as input. BertViz optionally supports a drop-down menu that allows \nuser to filter attention based on which sentence the tokens are in, e.g. only show attention between tokens in first\n sentence and tokens in second sentence.\n \n \n##### Head and model views\nTo enable this feature when invoking the [`head_view`](bertviz/head_view.py) or [`model_view`](bertviz/model_view.py) functions, set\n the `sentence_b_start` parameter to the start index of the second sentence. Note that the method for computing this\n index will depend on the model.\n \nExample (BERT):\n\n```python\nfrom bertviz import head_view\nfrom transformers import AutoTokenizer, AutoModel, utils\nutils.logging.set_verbosity_error()  # Suppress standard warnings\n\n# NOTE: This code is model-specific\nmodel_version = 'bert-base-uncased'\nmodel = AutoModel.from_pretrained(model_version, output_attentions=True)\ntokenizer = AutoTokenizer.from_pretrained(model_version)\nsentence_a = \"the rabbit quickly hopped\"\nsentence_b = \"The turtle slowly crawled\"\ninputs = tokenizer.encode_plus(sentence_a, sentence_b, return_tensors='pt')\ninput_ids = inputs['input_ids']\ntoken_type_ids = inputs['token_type_ids'] # token type id is 0 for Sentence A and 1 for Sentence B\nattention = model(input_ids, token_type_ids=token_type_ids)[-1]\nsentence_b_start = token_type_ids[0].tolist().index(1) # Sentence B starts at first index of token type id 1\ntoken_ids = input_ids[0].tolist() # Batch index 0\ntokens = tokenizer.convert_ids_to_tokens(token_ids)    \nhead_view(attention, tokens, sentence_b_start)\n``` \n\n\n##### Neuron view\n\nTo enable this option in the neuron view, simply set the `sentence_a` and `sentence_b` parameters in [`neuron_view.show()`](bertviz/neuron_view.py).\n\n#### Obtain HTML representations\n\nSupport to retrieve the generated HTML representations has been added to head_view, model_view and neuron_view.\n\nSetting the 'html_action' parameter to 'return' will make the function call return a single HTML Python object that can be further processed. Remember you can access the HTML source using the data attribute of a Python HTML object.\n\nThe default behavior for 'html_action' is 'view', which will display the visualization but won't return the HTML object.\n\nThis functionality is useful if you need to:\n- Save the representation as an independent HTML file that can be accessed via web browser\n- Use custom display methods as the ones needed in Databricks to visualize HTML objects\n\nExample (head and model views):\n\n```python\nfrom transformers import AutoTokenizer, AutoModel, utils\nfrom bertviz import head_view\n\nutils.logging.set_verbosity_error()  # Suppress standard warnings\ntokenizer = AutoTokenizer.from_pretrained(\"bert-base-uncased\")\nmodel = AutoModel.from_pretrained(\"bert-base-uncased\", output_attentions=True)\n\ninputs = tokenizer.encode(\"The cat sat on the mat\", return_tensors='pt')\noutputs = model(inputs)\nattention = outputs[-1]  # Output includes attention weights when output_attentions=True\ntokens = tokenizer.convert_ids_to_tokens(inputs[0]) \n\nhtml_head_view = head_view(attention, tokens, html_action='return')\n\nwith open(\"PATH_TO_YOUR_FILE/head_view.html\", 'w') as file:\n    file.write(html_head_view.data)\n\n```\n\nExample (neuron view):\n\n```python\n# Import specialized versions of models (that return query/key vectors)\nfrom bertviz.transformers_neuron_view import BertModel, BertTokenizer\nfrom bertviz.neuron_view import show\n\nmodel_type = 'bert'\nmodel_version = 'bert-base-uncased'\ndo_lower_case = True\nsentence_a = \"The cat sat on the mat\"\nsentence_b = \"The cat lay on the rug\"\nmodel = BertModel.from_pretrained(model_version, output_attentions=True)\ntokenizer = BertTokenizer.from_pretrained(model_version, do_lower_case=do_lower_case)\nhtml_neuron_view = show(model, model_type, tokenizer, sentence_a, sentence_b, layer=2, head=0, html_action='return')\n\nwith open(\"PATH_TO_YOUR_FILE/neuron_view.html\", 'w') as file:\n    file.write(html_neuron_view.data)\n```\n\n#### Non-Huggingface models\n\nThe head view and model view may be used to\n visualize self-attention for any standard Transformer model,\nas long as the attention weights are available and follow the format specified in [`head_view`](bertviz/head_view.py) and\n [`model_view`](bertviz/model_view.py) (which is the format \nreturned from Huggingface models). In some case, Tensorflow checkpoints may be loaded as Huggingface models as described\n in the\n [Huggingface docs](https://huggingface.co/transformers/). \n \n \n### ⚠️ Limitations\n\n#### Tool\n* This tool is designed for shorter inputs and may run slowly if the input text is very long and/or the model is very large.\n To mitigate this, you may wish to filter the layers displayed by setting the **`include_layers`** parameter, as described [above](#filtering-layers).\n* When running on Colab, some of the visualizations will fail (runtime disconnection) when the input text is long.  To mitigate this, you may wish to filter the layers displayed by setting the **`include_layers`** parameter, as described [above](#filtering-layers).\n* The *neuron view* only supports the custom BERT, GPT-2, and RoBERTa models included with the tool. This view needs access to the query and key vectors, \nwhich required modifying the model code (see `transformers_neuron_view` directory), which has only been done for these three models.\n\n\n#### Attention as \"explanation\"?\n* Visualizing attention weights illuminates one type of architecture within the model but does not\nnecessarily provide a direct *explanation* for predictions [[1](https://arxiv.org/pdf/1909.11218.pdf), [2](https://arxiv.org/abs/1902.10186), [3](https://arxiv.org/pdf/1908.04626.pdf)].\n* If you wish to understand how the input text influences output predictions more directly, consider [saliency methods](https://arxiv.org/pdf/2010.05607.pdf) provided \nby tools such as the [Language Interpretability Toolkit](https://github.com/PAIR-code/lit) or [Ecco](https://github.com/jalammar/ecco).\n\n\n## 🔬 Paper\n\n[\u003cb\u003eA Multiscale Visualization of Attention in the Transformer Model\u003c/b\u003e](https://www.aclweb.org/anthology/P19-3007.pdf) (ACL System Demonstrations 2019).\n\n\n### Citation\n```bibtex\n@inproceedings{vig-2019-multiscale,\n    title = \"A Multiscale Visualization of Attention in the Transformer Model\",\n    author = \"Vig, Jesse\",\n    booktitle = \"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations\",\n    month = jul,\n    year = \"2019\",\n    address = \"Florence, Italy\",\n    publisher = \"Association for Computational Linguistics\",\n    url = \"https://www.aclweb.org/anthology/P19-3007\",\n    doi = \"10.18653/v1/P19-3007\",\n    pages = \"37--42\",\n}\n```\n## Authors\n[Jesse Vig](https://twitter.com/jesse_vig)\n\n## 🙏 Acknowledgments\nWe are grateful to the authors of the following projects, which are incorporated into this repo:\n* https://github.com/tensorflow/tensor2tensor\n* https://github.com/huggingface/pytorch-pretrained-BERT\n\n## License\n\nThis project is licensed under the Apache 2.0 License - see the [LICENSE](LICENSE) file for details\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjessevig%2Fbertviz","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjessevig%2Fbertviz","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjessevig%2Fbertviz/lists"}