https://github.com/samhaswon/cross-attention-fusion
https://github.com/samhaswon/cross-attention-fusion
Last synced: 8 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/samhaswon/cross-attention-fusion
- Owner: samhaswon
- Created: 2024-11-20T13:06:24.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-12-24T18:05:27.000Z (over 1 year ago)
- Last Synced: 2024-12-24T19:19:37.488Z (over 1 year ago)
- Language: Python
- Size: 19.5 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# cross-attention-fusion
models:
mhsa_vit: Multi-Head Self-Attention, concatenation
mhsa_2: Multi-Head Self-Attention, addition
early_concat: early concatenation
## Accuracies
| | Train | Eval |
|:------------:|:--------------------------:|:--------------------------:|
| mhsa_vit | 74.9665% | 63.2500% |
| mhsa_2 | dnf/nan | dnf/nan |
| early_concat | 99.7389% (max of 99.7570%) | 80.2500% (max of 82.0000%) |