https://github.com/jonathanbratt/RBERTviz

Visualization tools to use with RBERT
https://github.com/jonathanbratt/RBERTviz

bert htmlwidgets natural-language-processing nlp rstats rstudio tensorflow

Last synced: 3 months ago
JSON representation

Visualization tools to use with RBERT

• Host: GitHub
• URL: https://github.com/jonathanbratt/RBERTviz
• Owner: jonathanbratt
• License: apache-2.0
• Created: 2019-09-18T17:49:25.000Z (about 5 years ago)
• Default Branch: master
• Last Pushed: 2021-06-14T21:41:42.000Z (over 3 years ago)
• Last Synced: 2024-05-21T02:10:29.116Z (6 months ago)
• Topics: bert, htmlwidgets, natural-language-processing, nlp, rstats, rstudio, tensorflow
• Language: R
• Homepage:
• Size: 5.84 MB
• Stars: 44
• Watchers: 4
• Forks: 6
• Open Issues: 3
• Metadata Files:
  • Readme: README.Rmd
  • Contributing: CONTRIBUTING.md
  • License: LICENSE

Awesome Lists containing this project

• jimsghstars - jonathanbratt/RBERTviz - Visualization tools to use with RBERT (R)

README

        
---

output: github_document

---

```{r, include = FALSE}

knitr::opts_chunk$set(

  collapse = TRUE,

  comment = "#>",

  fig.path = "man/figures/README-",

  out.width = "100%"

)

```

[![Lifecycle: maturing](https://img.shields.io/badge/lifecycle-maturing-blue.svg)](https://www.tidyverse.org/lifecycle/#maturing)

[![Travis build status](https://travis-ci.org/jonathanbratt/RBERTviz.svg?branch=master)](https://travis-ci.org/jonathanbratt/RBERTviz)

[![AppVeyor build status](https://ci.appveyor.com/api/projects/status/github/jonathanbratt/RBERTviz?branch=master&svg=true)](https://ci.appveyor.com/project/jonathanbratt/RBERTviz)

[![Codecov test coverage](https://codecov.io/gh/jonathanbratt/RBERTviz/branch/master/graph/badge.svg)](https://codecov.io/gh/jonathanbratt/RBERTviz?branch=master)

RBERTviz provides tools for convenient visualization of BERT models in the RBERT package.

## Installation

You can install RBERTviz from [GitHub](https://github.com/) with:

```{r installation, eval = FALSE}

# install.packages("devtools")

devtools::install_github(

  "jonathanbratt/RBERTviz", 

  build_vignettes = TRUE

)

```

### RBERT Installation

RBERTviz is intended to be used alongside the RBERT package. See the installation instructions

in that [repository](https://github.com/jonathanbratt/RBERT).

## Basic usage

RBERTviz currently enables visualization of

1. the attention matrices for each attention head

2. the output vectors at each transformer layer

The attention visualizer is basically a wrapper around an earlier

version of the [transformer visualization tools](https://github.com/tensorflow/tensor2tensor/tree/master/tensor2tensor/visualization),

as adapted by [Jesse Vig](https://github.com/jessevig/bertviz/).

![Chicken](man/attention1.png) ![Road](man/attention2.png)

The output vector visualizer is a collection of routines for generating 2D PCA

plots of the layer outputs, which can be interpreted as context-dependent

embedding vectors.

![Train](man/embeddings_layers_animation.gif)

See the "Introduction to RBERTviz" vignette included with the package for more

detailed examples of usage.

## Disclaimer

This is not an officially supported Macmillan Learning product.

## Contact information

Questions or comments should be directed to Jonathan Bratt

([email protected]) and Jon Harmon ([email protected]).