https://github.com/sleepingcat4/bert-textgeneration

A complete guide in generating text using bert and fine-tune
https://github.com/sleepingcat4/bert-textgeneration

bert-fine-tuning bert-model text-generation tranformers

Last synced: 8 months ago
JSON representation

A complete guide in generating text using bert and fine-tune

• Host: GitHub
• URL: https://github.com/sleepingcat4/bert-textgeneration
• Owner: sleepingcat4
• License: mit
• Created: 2023-03-21T16:15:16.000Z (about 3 years ago)
• Default Branch: master
• Last Pushed: 2023-03-22T07:45:33.000Z (about 3 years ago)
• Last Synced: 2025-07-19T05:23:09.029Z (11 months ago)
• Topics: bert-fine-tuning, bert-model, text-generation, tranformers
• Language: Jupyter Notebook
• Homepage:
• Size: 687 KB
• Stars: 11
• Watchers: 1
• Forks: 2
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
BERT for Text Generation

Recently, text generation models like GPT and T5 have been the talk of the town. However, I wanted to explore the capabilities of text generation using BERT - one of the first large language models that has been instrumental in the progress of NLP. While BERT is primarily known for sentimental and classification tasks, I was curious to see if it could generate a few words or sentences as well.

What did I find?

BERT is indeed capable of generating words within a sentence by predicting masked words in a reasonable manner. However, when it comes to generating full sentences, it faces certain obstacles such as tokenizer limitations and the issue of hallucination.

What's the repository for?

This repository aims to compile all known techniques that can be employed to generate text of any length using the original BERT core architecture.

Why did I create this?

I created this small project to experiment with well-known NLP models. There's a lot to learn from attempting tasks with a model that isn't specifically designed for it. Even a little success can be rewarding!

Author's Note

I welcome contributions to this project! Please feel free to add any additional techniques that I may have missed.