https://github.com/kahsolt/emotionroi-ex

Unofficial implementation of the EmotionROI essay and extended applications
https://github.com/kahsolt/emotionroi-ex

emotion-analysis emotion-classification region-of-interest

Last synced: about 4 hours ago
JSON representation

Unofficial implementation of the EmotionROI essay and extended applications

• Host: GitHub
• URL: https://github.com/kahsolt/emotionroi-ex
• Owner: Kahsolt
• License: mit
• Created: 2023-12-27T03:44:26.000Z (about 1 year ago)
• Default Branch: master
• Last Pushed: 2024-04-20T06:41:37.000Z (9 months ago)
• Last Synced: 2024-11-09T07:47:15.970Z (about 2 months ago)
• Topics: emotion-analysis, emotion-classification, region-of-interest
• Language: Python
• Homepage:
• Size: 2.07 MB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# EmotionROI-ex

    Unofficial implementation of the EmotionROI essay and extended applications

----

ℹ The pretrained weights can be found [here](https://pan.quark.cn/s/13e59e043b1c) :)

Reproduction the **FCNEL** model in essay [WHERE DO EMOTIONS COME FROM? PREDICTING THE EMOTION STIMULI MAP](http://chenlab.ece.cornell.edu/Publication/Kuan-Chuan/ICIP16_EmotionROI.pdf), and extended applications.

The results reported in the EmotionROI/FCNEL essay:

![FCNEL](img/FCNEL.png)

### Reproduce FCNEL for ROI prediction

⚠ Can NOT reproduce essay results exactly, here're the implementation differences:

- The essay uses `FCN-32s + AlexNet`, we use `FCN-8s + ResNet50`

- The essay uses `lr=1e-8` to refine `20` epochs, which is two small and doesn't have any effect in our setting, we instead use `lr=1e-2` to train `100` epochs with `CosineAnnealingLR`

- The essay might use the default `Bilinear` for output upsacling, we instead use `Nearest-Exact` to simulate `FCN-32s` resolution

![FCNEL-infer](img/FCNEL-infer.png)  

![FCNEL-loss](img/FCNEL-loss.png)

### Modifyied EmoFCNEL for ROI prediction with emotion condition

ℹ Predict ROI with any emotion label (except for `sadness`) results into nearly the same output...  

ℹ The ROI annotations are probably NOT related to certain emotion, but only indicates **human attention focus**  

![EmoFCNEL-infer](img/EmoFCNEL-infer.png)

![EmoFCNEL-loss](img/EmoFCNEL-loss.png)

![EmoFCNEL-embed](img/EmoFCNEL-embed.png)

### Predict emotion directly from ROI annotation

⚠ Mission impossible :(

| Model | Train acc. | Test acc. |

| :-: | :-: | :-: |

| Resnet50 | 24.098% | 17.172% |

![RIO_clf](img/RIO_clf-acc.png)

### references

- essay

  - info page: [http://chenlab.ece.cornell.edu/publications.html](http://chenlab.ece.cornell.edu/publications.html)

  - download link: [http://chenlab.ece.cornell.edu/Publication/Kuan-Chuan/ICIP16_EmotionROI.pdf](http://chenlab.ece.cornell.edu/Publication/Kuan-Chuan/ICIP16_EmotionROI.pdf)

- EmotionROI dataset

  - info page: [http://chenlab.ece.cornell.edu/downloads.html](http://chenlab.ece.cornell.edu/downloads.html)

  - download link: [http://chenlab.ece.cornell.edu/people/kuanchuan/publications/EmotionROI.zip](http://chenlab.ece.cornell.edu/people/kuanchuan/publications/EmotionROI.zip)

----

by Armit

2023/12/27