https://github.com/sayamalt/next-word-prediction-using-bidirectional-lstms
Successfully developed a text generation model to predict the next word for a given sequence using LSTMs.
https://github.com/sayamalt/next-word-prediction-using-bidirectional-lstms
deep-learning long-short-term-memory-network lstm-networks natural-language-processing next-word-prediction recurrent-neural-networks text-generation
Last synced: 3 months ago
JSON representation
Successfully developed a text generation model to predict the next word for a given sequence using LSTMs.
- Host: GitHub
- URL: https://github.com/sayamalt/next-word-prediction-using-bidirectional-lstms
- Owner: SayamAlt
- License: apache-2.0
- Created: 2025-02-23T06:22:44.000Z (4 months ago)
- Default Branch: main
- Last Pushed: 2025-02-23T07:05:49.000Z (4 months ago)
- Last Synced: 2025-02-23T07:26:06.321Z (4 months ago)
- Topics: deep-learning, long-short-term-memory-network, lstm-networks, natural-language-processing, next-word-prediction, recurrent-neural-networks, text-generation
- Language: Jupyter Notebook
- Homepage:
- Size: 4.95 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# 📖 Next-Word Prediction using Bidirectional LSTMs
## 🚀 Project Overview
This repository contains a **Next-Word Prediction Model** built using **Bidirectional Long Short-Term Memory (BiLSTM)** networks. The model is trained to predict the next word in a sentence given a sequence of preceding words. It leverages **PyTorch** for deep learning implementation and **NLTK** for text preprocessing.
## 🛠 Features
✅ **Deep Learning Model**: Uses **Bidirectional LSTMs** to improve context awareness.
✅ **Trained on a Large Text Corpus**: Enhances generalization and accuracy.
✅ **PyTorch Implementation**: Efficient training and inference pipeline.
✅ **Tokenization with NLTK**: Ensures structured preprocessing of text data.
✅ **Pre-Trained Model Included**: Easily load the trained `.pt` file for inference.
## 📂 Project Structure
The repository consists of:
- `next_word_prediction_using_bidirectional_lstms.ipynb` → Jupyter Notebook with full model implementation, training, and evaluation.
- `next_word_predictor.pt` → Pre-trained PyTorch model for inference.
## 🔧 Installation & Setup
To set up the project, follow these steps:
### 1️⃣ Clone the Repository
```bash
git clone https://github.com/SayamAlt/Next-Word-Prediction-using-Bidirectional-LSTMs.git
cd Next-Word-Prediction-using-Bidirectional-LSTMs
```
### 2️⃣ Install Dependencies
Ensure you have Python 3.8+ installed, then install the required libraries:
```bash
pip install nltk torch torchinfo numpy pandas matplotlib
```
### 🏗 Bidirectional LSTM (BiLSTM) Model
The **Bidirectional LSTM (BiLSTM)** model consists of:
- **Embedding Layer**: Converts words into dense vector representations.
- **Bidirectional LSTM**: Captures both past and future context.
- **Fully Connected Layer**: Predicts the next word.
### LSTM Updates
Mathematically, the LSTM updates are given by:
$$
f_t = \sigma(W_f [h_{t-1}, x_t] + b_f)
$$
$$
i_t = \sigma(W_i [h_{t-1}, x_t] + b_i)
$$
$$
C_t~ = \tanh(W_C [h_{t-1}, x_t] + b_C)
$$
$$
C_t = f_t \odot C_{t-1} + i_t \odot \tilde{C}_t
$$
$$
o_t = \sigma(W_o [h_{t-1}, x_t] + b_o)
$$
$$
h_t = o_t \odot \tanh(C_t)
$$
where:
- \( f_t, i_t, o_t \) are the **forget, input, and output gates**.
- \( C_t \) is the **cell state**.
- \( h_t \) is the **hidden state**.