Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/SiddhantBikram/MemeCLIP
Official Repository for the MemeCLIP framework and PrideMM Dataset @ EMNLP 2024
https://github.com/SiddhantBikram/MemeCLIP
computational-social-science deep-learning machine-learning multimodal
Last synced: about 1 month ago
JSON representation
Official Repository for the MemeCLIP framework and PrideMM Dataset @ EMNLP 2024
- Host: GitHub
- URL: https://github.com/SiddhantBikram/MemeCLIP
- Owner: SiddhantBikram
- Created: 2024-09-23T02:29:11.000Z (5 months ago)
- Default Branch: main
- Last Pushed: 2025-01-14T18:38:35.000Z (about 1 month ago)
- Last Synced: 2025-01-14T20:03:38.121Z (about 1 month ago)
- Topics: computational-social-science, deep-learning, machine-learning, multimodal
- Language: Python
- Homepage:
- Size: 237 KB
- Stars: 8
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
MemeCLIP: Leveraging CLIP Representations for Multimodal Meme Classification
Siddhant Bikram Shah, Shuvam Shiwakoti, Maheep Chaudhary, Haohan Wang
This is the code repository for our paper **MemeCLIP: Leveraging CLIP Representations for Multimodal Meme Classification** published in EMNLP 2024.
## PrideMM Dataset
The images and labels for the PrideMM dataset are available here (Warning: Insensitive content).
## Annotation Terminology
### Hate
| Class | Terminology |
| :--------: | :--------: |
| No Hate | 0 |
| Hate | 1 |
### Targets of Hate
| Class | Terminology |
| :--------: | :--------: |
| Undirected | 0 |
| Individual | 1 |
| Community | 2 |
| Organization | 3 |
### Stance
| Class | Terminology |
| :--------: | :--------: |
| Neutral | 0 |
| Support | 1 |
| Oppose | 2 |
### Humor
| Class | Terminology |
| :--------: | :--------: |
| No Humor | 0 |
| Humor | 1 |
## MemeCLIP Code
All experimental changes can be made through a single file: configs.py.
Directory names can be set in the following variables:
+ cfg.root_dir
+ cfg.img_folder
+ cfg.info_file
+ cfg.checkpoint_path
+ cfg.checkpoint_file
To train, validate, and test MemeCLIP, set cfg.test_only = False and run main.py.
To test MemeCLIP, set cfg.test_only = True and run main.py.
CSV files are expected to contain image path, text, and label in no particular order.
## Pre-trained Weights
Pre-trained weights for MemeCLIP (Hate Classification Task) are available here.
## Citation
```
@article{shah2024memeclip,
title={MemeCLIP: Leveraging CLIP Representations for Multimodal Meme Classification},
author={Shah, Siddhant Bikram and Shiwakoti, Shuvam and Chaudhary, Maheep and Wang, Haohan},
journal={arXiv preprint arXiv:2409.14703},
year={2024}
}
```
OR
Siddhant Bikram Shah, Shuvam Shiwakoti, Maheep Chaudhary, and Haohan Wang. 2024. MemeCLIP: Leveraging CLIP Representations for Multimodal Meme Classification. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17320–17332, Miami, Florida, USA. Association for Computational Linguistics.