Other Resources

Other

• Microsoft/AzureML-BERT - End-to-end walk through for fine-tuning BERT using Azure Machine Learning.

Tasks

Classification

• brightmart/sentiment_analysis_fine_grain - Multi-label classification with BERT; Fine Grained Sentiment Analysis from AI challenger.
• zhpmatrix/Kaggle-Quora-Insincere-Questions-Classification - Kaggle baseline—fine-tuning BERT and tensor2tensor based Transformer encoder solution.

Knowledge Graph

• sakuranew/BERT-AttributeExtraction - Using BERT for attribute extraction in knowledge graph. Fine-tuning and feature extraction. The BERT-based fine-tuning and feature extraction methods are used to extract knowledge attributes of Baidu Encyclopedia characters.

Named-Entity Recognition (NER)

• kyzhouhzau/BERT-NER - Use google BERT to do CoNLL-2003 NER.
• zhpmatrix/bert-sequence-tagging - Chinese sequence labeling.
• ProHiryu/bert-chinese-ner - Use the pre-trained language model BERT to do Chinese NER.

Text Generation

• asyml/texar - Toolkit for Text Generation and Beyond. Texar is a general-purpose text generation toolkit, has also implemented BERT here for classification, and text generation applications by combining with Texar's other modules.

Tools

Other

• jessevig/bertviz - Tool for visualizing attention in the Transformer model.

Transformer Implementations By Communities

Chainer

• soskek/bert-chainer - Chainer implementation of "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding".

Keras

• CyberZHG/keras-bert - Implementation of BERT that could load official pre-trained models for feature extraction and prediction.

PyTorch

• codertimo/BERT-pytorch - Google AI 2018 BERT pytorch implementation.
• deepset-ai/FARM - Simple & flexible transfer learning for the industry.
• dbiir/UER-py - Open Source Pre-training Model Framework in PyTorch & Pre-trained Model Zoo (with more focus on Chinese).

PyTorch and TensorFlow

• spacy-transformers - a library that wrap Hugging Face's Transformers, in order to extract features to power NLP pipelines. It also calculates an alignment so the Transformer features can be related back to actual words instead of just wordpieces.
• FasterTransformer - Transformer related optimization, including BERT and GPT. This repo provides a script and recipe to run the highly optimized transformer-based encoder and decoder component, and it is tested and maintained by NVIDIA.