{"id":25486612,"url":"https://github.com/prince2004patel/next_word_prediction","last_synced_at":"2026-04-11T03:12:49.263Z","repository":{"id":278147949,"uuid":"934663448","full_name":"prince2004patel/next_word_prediction","owner":"prince2004patel","description":"This project implements a Next Word Prediction model using LSTM with an Attention mechanism.","archived":false,"fork":false,"pushed_at":"2025-02-18T08:10:16.000Z","size":13116,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-18T16:50:15.610Z","etag":null,"topics":["keras","next-word-prediction","python","streamlit","tensorflow"],"latest_commit_sha":null,"homepage":"https://next-word-prediction-by-prince.streamlit.app/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/prince2004patel.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-02-18T07:52:42.000Z","updated_at":"2025-02-18T08:12:20.000Z","dependencies_parsed_at":"2025-02-18T09:32:56.083Z","dependency_job_id":null,"html_url":"https://github.com/prince2004patel/next_word_prediction","commit_stats":null,"previous_names":["prince2004patel/next_word_prediction"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/prince2004patel/next_word_prediction","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prince2004patel%2Fnext_word_prediction","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prince2004patel%2Fnext_word_prediction/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prince2004patel%2Fnext_word_prediction/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prince2004patel%2Fnext_word_prediction/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/prince2004patel","download_url":"https://codeload.github.com/prince2004patel/next_word_prediction/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prince2004patel%2Fnext_word_prediction/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":272936432,"owners_count":25018163,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-31T02:00:09.071Z","response_time":79,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["keras","next-word-prediction","python","streamlit","tensorflow"],"created_at":"2025-02-18T19:21:20.098Z","updated_at":"2025-12-30T21:34:29.537Z","avatar_url":"https://github.com/prince2004patel.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Next Word Prediction Using LSTM + Attention\n\n## Overview\nThis project implements a **Next Word Prediction** model using **LSTM (Long Short-Term Memory) with an Attention mechanism**. The model is trained on **Shakespeare's Hamlet dataset** and achieves **70% accuracy** in predicting the next word in a sequence.\n\n## Live Demo :-\n\n[![Streamlit App](https://img.shields.io/badge/Streamlit-App-blue)](https://next-word-prediction-by-prince.streamlit.app/)\n\n## Project Workflow\n\n### 1. Data Collection\n- Collected **Shakespeare's Hamlet** dataset as raw text.\n\n### 2. Data Preprocessing\n- **Tokenization:** Used `Tokenizer` from Keras to convert text into sequences.\n- **Lowercasing:** Converted all text to lowercase for consistency.\n- **Vocabulary Size Check:** Identified a total of **4,818 unique words**.\n- **Input Sequence Creation:**\n  - Generated input sequences by taking a sliding window of words.\n  - Example: \"to be or not to\" → \"be or not to predict_next_word\".\n- **Finding Maximum Sequence Length:** Computed the longest input sequence length.\n- **Padding Sequences:**\n  - Used `pad_sequences()` to apply **pre-padding** (padding at the beginning) to standardize input size.\n\n### 3. Feature Engineering\n- **Divided dataset into X (input sequences) and y (target words).**\n- **Performed train-test split** to prepare the dataset for model training.\n\n### 4. Model Development\n- Built an **LSTM-based neural network with an Attention mechanism**.\n- Model architecture:\n  - **Embedding Layer** to convert words into vector representations.\n  - **LSTM Layer** to capture sequential dependencies.\n  - **Attention Layer** to focus on important words in the sequence.\n  - **Dense Output Layer** with a softmax activation for predicting the next word.\n- **Compiled the model using categorical cross-entropy loss** and Adam optimizer.\n\n### 5. Model Training\n- Trained the model on the dataset.\n- Achieved **70% accuracy** on test data.\n\n### 6. Prediction Function\n- Created a **helper function** for predicting the next word:\n  - Takes input text.\n  - Tokenizes and pads it to match the model's input shape.\n  - Predicts the most likely next word.\n- Successfully tested predictions on different inputs.\n\n### 7. Streamlit Web Application\n- Built a **Streamlit-based UI** to interact with the model.\n- Users can enter a sentence, and the model predicts the next word.\n\n## How to Run the Project\n1. Install dependencies:\n   ```bash\n   pip install -r requirements.txt\n   ```\n2. Run the Streamlit app:\n   ```bash\n   streamlit run app.py\n   ```\n3. Open the **localhost URL** to interact with the app.\n\n## Technologies Used\n- **Python**\n- **TensorFlow \u0026 Keras** (Deep Learning)\n- **Streamlit** (Web Interface)\n- **Numpy ,Pandas \u0026 Pickle** (Data Handling)\n\n## Future Improvements\n- Train on a **larger dataset** for improved generalization.\n- Experiment with **transformer-based models (e.g., GPT, BERT)** for better predictions.\n- Optimize the **attention mechanism** to enhance word predictions.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprince2004patel%2Fnext_word_prediction","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprince2004patel%2Fnext_word_prediction","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprince2004patel%2Fnext_word_prediction/lists"}