https://github.com/zer0int/clip-attention-entropy

A small script for CLIP attn entropy plots
https://github.com/zer0int/clip-attention-entropy

attention clip contrastive-learning entropy plot text-image transformer visualization vit

Last synced: 3 months ago
JSON representation

A small script for CLIP attn entropy plots

• Host: GitHub
• URL: https://github.com/zer0int/clip-attention-entropy
• Owner: zer0int
• Created: 2024-10-21T16:55:02.000Z (7 months ago)
• Default Branch: CLIP-vision
• Last Pushed: 2024-10-21T17:00:38.000Z (7 months ago)
• Last Synced: 2025-01-02T17:33:06.874Z (5 months ago)
• Topics: attention, clip, contrastive-learning, entropy, plot, text-image, transformer, visualization, vit
• Language: Python
• Homepage:
• Size: 42 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# CLIP-attention-entropy

A small script for CLIP attn entropy plots. Add your own finetune & dataset!

Example dataset: 

- Texts -> attn-test-coco-spright-val.json included.

- Images: Center-Crop of [huggingface.co/datasets/SPRIGHT-T2I/spright_coco](https://huggingface.co/datasets/SPRIGHT-T2I/spright_coco)

Results (with [my GmP-smooth finetune](https://huggingface.co/zer0int/CLIP-GmP-ViT-L-14)):

![clip-vs-clip-attn-entropy](https://github.com/user-attachments/assets/5eb45563-e013-4dad-814f-360bd64dceb0)