https://github.com/santhoshkolloju/Abstractive-Summarization-With-Transfer-Learning

Abstractive summarisation using Bert as encoder and Transformer Decoder
https://github.com/santhoshkolloju/Abstractive-Summarization-With-Transfer-Learning

abstractive-summarization abstractive-text-summarization bert bert-model nlg nlp summarization transfer-learning transformer

Last synced: 2 months ago
JSON representation

Abstractive summarisation using Bert as encoder and Transformer Decoder

• Host: GitHub
• URL: https://github.com/santhoshkolloju/Abstractive-Summarization-With-Transfer-Learning
• Owner: santhoshkolloju
• Created: 2019-01-31T02:42:25.000Z (almost 6 years ago)
• Default Branch: master
• Last Pushed: 2023-05-30T05:41:22.000Z (over 1 year ago)
• Last Synced: 2024-08-02T08:10:01.794Z (5 months ago)
• Topics: abstractive-summarization, abstractive-text-summarization, bert, bert-model, nlg, nlp, summarization, transfer-learning, transformer
• Language: Python
• Size: 788 KB
• Stars: 404
• Watchers: 21
• Forks: 100
• Open Issues: 30
• Metadata Files:
  • Readme: Readme.md

Awesome Lists containing this project

• awesome-bert - santhoshkolloju/Abstractive-Summarization-With-Transfer-Learning

README

        
Abstractive summarization using bert as encoder and transformer decoder


I have used a text generation library called Texar , Its a beautiful library with a lot of abstractions, i would say it to be 

scikit learn for text generation problems.

The main idea behind this architecture is to use the transfer learning from pretrained BERT a masked language model ,

I have replaced the Encoder part with BERT Encoder and the deocder is trained from the scratch.

One of the advantages of using Transfomer Networks is training is much faster than LSTM based models as we elimanate sequential behaviour in Transformer models.

Transformer based models generate more gramatically correct  and coherent sentences.

To run the model


wget https://storage.googleapis.com/bert_models/2018_10_18/uncased_L-12_H-768_A-12.zip 

unzip uncased_L-12_H-768_A-12.zip


Place the story and summary files under data folder with the following names.

-train_story.txt

-train_summ.txt

-eval_story.txt

-eval_summ.txt

each story and summary must be in a single line (see sample text given.)

Step1:

Run Preprocessing

python preprocess.py

This creates two tfrecord files under the data folder.

Step 2:

python main.py

Configurations for the model can be changes from config.py file

Step 3:

Inference 

Run the command python inference.py

This code runs a flask server 

Use postman to send the POST request @http://your_ip_address:1118/results

with two form parameters story,summary