{"id":28688946,"url":"https://github.com/pointer2alvee/paper-hbert-sarcasm-detection","last_synced_at":"2026-01-31T22:06:20.230Z","repository":{"id":288867366,"uuid":"969398933","full_name":"pointer2Alvee/paper-hbert-sarcasm-detection","owner":"pointer2Alvee","description":"Implementation of the paper \"A Novel Hierarchical BERT Architecture for Sarcasm Detection\" using Python \u0026 TensorFlow","archived":false,"fork":false,"pushed_at":"2025-04-20T04:53:03.000Z","size":8,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-20T05:19:22.913Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pointer2Alvee.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-20T03:57:02.000Z","updated_at":"2025-04-20T04:53:06.000Z","dependencies_parsed_at":"2025-04-20T05:29:32.524Z","dependency_job_id":null,"html_url":"https://github.com/pointer2Alvee/paper-hbert-sarcasm-detection","commit_stats":null,"previous_names":["pointer2alvee/paper-hbert-sarcasm-detection"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/pointer2Alvee/paper-hbert-sarcasm-detection","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pointer2Alvee%2Fpaper-hbert-sarcasm-detection","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pointer2Alvee%2Fpaper-hbert-sarcasm-detection/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pointer2Alvee%2Fpaper-hbert-sarcasm-detection/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pointer2Alvee%2Fpaper-hbert-sarcasm-detection/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pointer2Alvee","download_url":"https://codeload.github.com/pointer2Alvee/paper-hbert-sarcasm-detection/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pointer2Alvee%2Fpaper-hbert-sarcasm-detection/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259763117,"owners_count":22907414,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-14T05:11:16.339Z","updated_at":"2026-01-31T22:06:20.224Z","avatar_url":"https://github.com/pointer2Alvee.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv style=\"display: flex; justify-content: space-around; align-items: center;\"\u003e\n  \u003cimg src=\"assets/images/hbert_arch.JPG\" alt=\"Image 1\" style=\"width: 100%; margin: 10px;\"\u003e\n\u003c/div\u003e\n\n## 📜 paper-hbert-sarcasm-detection\n#### 📌 Summary \nImplementation of the paper : \n**[\"A Novel Hierarchical BERT Architecture for Sarcasm Detection\"](https://aclanthology.org/2020.figlang-1.14.pdf)** using **Python**, **Pandas**, **NumPy**, **Kaggle** and **TensorFlow**.\n\n#### 🧠 Overview\nThis project implements a Hierarchical BERT (H-BERT) architecture for sarcasm detection in text, inspired by the paper **\"A Novel Hierarchical BERT Architecture for Sarcasm Detection\".** This model is a combination of LSTM \u0026 CNN with the BERT embeddings. Unlike traditional sentiment analysis tasks, sarcasm detection often requires understanding multi-sentence context and subtle cues. This hierarchical model addresses that by processing text at both the sentence and context level using a 5 layered architecture that includes (1) sentence encoding, (2) context summarization with Conv1D, (3) a bi-directional LSTM for capturing temporal dependencies, (4) CNN layers for local feature extraction, and a (5) final fully connected layer for classification.\n\nThe model is trained on the **\"Sarcasm on Reddit\"** dataset from Kaggle rather than the dataset used in the original paper. Preprocessing includes data cleaning and dimensionality reduction. Tokenization is done using the **'bert-base-uncased tokenizer'** from Hugging Face. Evaluation is performed using accuracy as the primary metric, and the entire model is implemented in Python using TensorFlow.\n\n#### 🎯 Use Cases \n- Detecting Sarcastic comments or opinions on social media platforms\n\n#### 🟢 Project Status\n- Current Version: V1.0\n- Completed\n\n#### 📂 Repository Structure\n```\npaper-hbert-sarcasm-detection/\n├── README.md\n├── LICENSE\n├── .gitignore                  \n├── assets/                      \n│   └── images/\n├── datasets/                      \n│   └── sarcasm-reddit/\n|   │   └── sarcasm_trained_balanced.csv\n└── notebooks/               \n    └── sarcasm-analysis.ipynb            \n```\n\n### ✨ Features\n- ✅ Custom `Hierarchical-BERT` model class\n- ✅ Preprocessed Data\n- ✅ Sentence tokenization and padding\n- ✅ Evaluation metrics: Accuracy \n\n🛠️ In progress:\n- Modification of H-bert Architecutre \n- Introducing Precision, Recall, F1-score\n\n\u003c!--\n### 🎥 Demo\n \u003ca href=\"https://www.youtube.com/shorts/wexIv6X45eE?feature=share\" target=\"_blank\"\u003e\n  \u003cimg src=\"assets/images/2_2.JPG\" alt=\"YouTube Video\" width=\"390\" height=\"270\"\u003e\n\u003c/a\u003e \n--\u003e\n\n### 🚀 Getting Started\n#### 📚 Knowledge \u0026 Skills Required \n- Python programming\n- ML/DL fundamentals, Transformers, Hugging Face Hub\n- Neural Network Arhitecutres (CNN, LSTM, RNN, BERT)\n\n#### 💻 Software Requirements\n- IDE (VS Code) or jupyter notebook or google colab\n- **Best run on Kaggle using GPU P100**\n  \n#### 🛡️ Tech Stack\n- Language: python\n- NLP/ML: huggingface_hub, sklearn, pandas, numpy\n- Deep Learning: pytorch, CNN, LSTM, transformers-models (bert-base-uncased) \n- Visualization: matplotlib\n\n#### 🔍 Modules Breakdown\n\u003cb\u003e 📥 (1) Data-Preprocessing :\u003c/b\u003e wh \n- Using a different Dataset than what was used on paper\n- Loading [Sarcasm on Reddit](https://www.kaggle.com/datasets/danofer/sarcasm?select=train-balanced-sarcasm.csv) dataset from kaggle \n- Dimention Reduced, Data Cleaned\n\n\u003cb\u003e 🎭 (2) Tokenization :\u003c/b\u003e basi\n- Used **'bert-base-uncased'** model from hugging face for tokenization\n- Tokenized \"comment\" column\n\n\u003cb\u003e 🤖 (3) Heirarchial-BERT :\u003c/b\u003e \n- The proposed H-BERT Architecture proposed in the paper and implemented in code is explained beiefly. The model consists of **5 layers** :-\n  1. **(L-1) Sentence Encoding Layer**  \n     - Encode input data into fixed sized vectors\n  2. **(L-2) Context Summarization Layer**  \n     - Convert the indvidual sequence-embeddings from layer-1 to common single-vect\n     - **Conv1d sentence-summarizer layer** : Paper used Conv2d as their data 1.3B but we use only 30k so we use Conv1d\n  3. **(L-3) LSTM - Context Encoder Layer**  \n     - Implement a bi-direc LSTM to capture temporal dependencies from summ-sent layer-2\n     - Bi-direc LSTM process data in both forward \u0026 backward direc, makes capturing v. easy\n  4. **(L-4) CNN Layer**  \n     - Extracts local features from encoded context vectors of layer-3\n     - Try to emphasis significant features relavant to model and \n     - Give less attn to irrelavant features\n     - **Kernel-layer**  : We use Conv1d kernels instead of Conv2d, also called cnn-filters\n     - **Pooling-layer** : Use max pooling to extract imp features\n  5. **(L-5) FFN / Fully Connected dense-net Layer**  \n     - Proecess the model to give a final output \n     - Maps features to final predictions.\n\n- The DataFlow:- \n(L-1) output --\u003e (L-2) output --\u003e (L-3) output --\u003e (L-4) output --\u003e (L-5) output --\u003e predictions (0 or 1)\n\n\n##### 📊 Evaluation\n- Using 'Accuracy' Meterics to evaluate model performance\n- Future work : precision , recall , f1\n\n#### ⚙️ Installation\n```\ngit clone https://github.com/pointer2Alvee/paper-hbert-sarcasm-detection.git\ncd hbert-sarcasm-detection\n\n# Recommended: Use a virtual environment\npip install -r requirements.txt\n```\n\n##### 🖇️ requirements.txt (core packages):\n```\ntransformers==4.44.0\nhuggingface_hub==0.24.5\npandas\nnumpy\ntensorflow\nmatplotlib\n```\n\n##### 💻 Running the App Locally\n1. Open Repo in VS code / Kaggle (recommended)\n2. Run Command\n3. See accuracy\n\n#### 📖 Usage\n- Open VS Code / kaggle\n\n### 🧪 Sample Topics Implemented\n- ✅ BERT model \n- ✅ LSTM, CNN, CONVOLUTION, POOLING\n  \n- ⏳ Upcoming  : Modification of Hbert\n\n### 🧭 Roadmap\n- [x] Paper Implementation of Hbert \n\n\n### 🤝 Contributing\nContributions are welcomed!\n1. Fork the repo. \n2. Create a branch: ```git checkout -b feature/YourFeature```\n3. Commit changes: ```git commit -m 'Add some feature'```\n4. Push to branch: ```git push origin feature/YourFeature```\n5. Open a Pull Request.\n\n### 📜License\nDistributed under the MIT License. See LICENSE.txt for more information.\n\n### 🙏Acknowledgements\n- Special thanks to the open-source community / youtube for tools and resources.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpointer2alvee%2Fpaper-hbert-sarcasm-detection","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpointer2alvee%2Fpaper-hbert-sarcasm-detection","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpointer2alvee%2Fpaper-hbert-sarcasm-detection/lists"}