https://github.com/mountchicken/imagecaptioning-attention-pyqt5

ImageCaptioning improved with an attention mechanism. Also a PyQt5 application
https://github.com/mountchicken/imagecaptioning-attention-pyqt5

attention imagecaptioning pyqt5 pytorch

Last synced: 18 days ago
JSON representation

ImageCaptioning improved with an attention mechanism. Also a PyQt5 application

• Host: GitHub
• URL: https://github.com/mountchicken/imagecaptioning-attention-pyqt5
• Owner: Mountchicken
• Created: 2021-04-22T01:09:21.000Z (over 3 years ago)
• Default Branch: main
• Last Pushed: 2021-10-02T03:48:44.000Z (about 3 years ago)
• Last Synced: 2024-10-06T11:20:51.858Z (about 1 month ago)
• Topics: attention, imagecaptioning, pyqt5, pytorch
• Language: Python
• Homepage:
• Size: 5.3 MB
• Stars: 8
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# ImageCaptioning-Attention-PyQt5

ImageCaptioning improved with attention. Also a PyQt5 applications 

# Welcome !

- Hello guys, hope you are doing awesome these days !😄

- In my previous ImageCaption repository, I implemented a ImageCaption algorithm and I promised to upload an attention based version latter. And here it is ！😄

- Using the `ResNet50` pretrained on ImageNet as the backbone(no finetune) and also some attention, the model can describe image like human(most of the time).

- Moreover,`Beam Search` are also used during the inferrence part and this give another great improvment on the model's performence 

- Now, let's enjoy some funny stuff😎

# 1.Examples👍

## ①.doggy doggy, juicy doggy

- 


## ③.You don't want to mess up with No. 1 shooter in the west

- 


## ④.🌶④💉💧🐮🍺

- 


## ⑤.Portland Timbers, Assemble!

- 


## ⑥.mountchicken must has something to do with mountain

- 


 

# 2.Requirements

- skimage

- spacy

- PyQt5

- Pip install them

# 3.Train😣

## download flickr30k

- Download the flickr30k dataset, unpack all the images into the folder `flickr30k/flickr30k-images`. I have already preprocessed the captions.txt, and you don't need to download that 

- [flickr(提取码:hrf3)](https://pan.baidu.com/s/1r0RVUwctJsI0iNuVXHQ6kA)

## download my checkpoint(if you don't want to train it with 14h on GeForce2080ti)

- Put the downloaded checkpoint into the folder `checkpoint`

- [checkpoint(提取码:qny4)](https://pan.baidu.com/s/189u5i5vZbzIp9r4XoEYn6A)

## change some parameters

- `train.py` line20 - line26, set the dataset path

- `train.py` line31 - line34, `load_model`:load my checkpoint or not.

- Ok, you can train now

# 4.Inferrence😀

- `inferrence.py` line245, choose your predict image path

# 5.APP

## Run main_gui.py

### if you run the .py file succesfully, it should look like this

- 


### Then, you need to push the initialize button to load the model, after that, just wait the `Finished` sign appers in the right.

- 


### Finally, load the image with `Load Image` button and press `Detect`

- 


# For more issue, contact me

- `Email Address` [email protected]