https://github.com/rahul-vinay/showattendtell

https://github.com/rahul-vinay/showattendtell

Last synced: 25 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/rahul-vinay/showattendtell
• Owner: rahul-vinay
• Created: 2025-01-01T21:19:31.000Z (about 1 month ago)
• Default Branch: main
• Last Pushed: 2025-01-01T21:28:38.000Z (about 1 month ago)
• Last Synced: 2025-01-01T22:27:09.615Z (about 1 month ago)
• Language: Jupyter Notebook
• Size: 2.31 MB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# ShowAttendAndTell: Image Caption Generation with Adaptive Attention

This project implements an adaptive attention mechanism for image captioning, inspired by the **"Show, Attend and Tell"** paper. It dynamically balances focus between visual features and language context, achieving a baseline **BLEU score of ~18.5** on the Flickr8k dataset.

---

## 📂 **Project Overview**

- **Objective**: Generate captions by focusing on relevant image regions while dynamically incorporating language context.

- **Model**: Combines a ResNet50-based encoder, an LSTM decoder, and adaptive attention with a sentinel gate.

- **Dataset**: Flickr8k, with preprocessing for tokenization, padding, and vocabulary creation.

- **Evaluation**: BLEU scores to measure caption quality.

---

## 📊 **Results**

- **BLEU Score**: ~18.5 (baseline).

- Demonstrated ability to generate grammatically correct captions, with room for improvement on complex scenes.

---

## 🔍 **Future Directions**

- Scale to larger datasets (e.g., Flickr30k, MS COCO).

- Explore metrics like METEOR or CIDEr for contextual evaluation.

- Integrate advanced spatial encodings for improved scene understanding.

---

## 🛠 **Technologies**

- Python, PyTorch, Google Colab

- ResNet50, LSTM, Adaptive Attention

- BLEU Scoring, NLTK

---

## 📄 **For More Details**

Refer to the detailed project report: `SAT Report.pdf`.

---

## 📬 **Contact**

Developed by Rahul Vinay  

Reach out: [email protected]