https://github.com/laion-ai/temporal-embedding-aggregation
Aggregating embeddings over time
https://github.com/laion-ai/temporal-embedding-aggregation
Last synced: about 1 month ago
JSON representation
Aggregating embeddings over time
- Host: GitHub
- URL: https://github.com/laion-ai/temporal-embedding-aggregation
- Owner: LAION-AI
- Created: 2022-06-27T10:35:48.000Z (almost 3 years ago)
- Default Branch: master
- Last Pushed: 2023-01-19T17:26:08.000Z (over 2 years ago)
- Last Synced: 2025-05-07T18:13:31.682Z (about 1 month ago)
- Language: Python
- Size: 122 KB
- Stars: 31
- Watchers: 3
- Forks: 4
- Open Issues: 21
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# temporal-embedding-aggregation
Experimental repo for testing various ways of aggregating information across video frame embeddings
## Kinetics700 Baseline results:
### Zero-shot Classification:
| | Accuracy |
| ---------------- | -------- |
| Top-1 | 0.31 |
| Top-5 | 0.56 |
| mean(Top1, Top5) | 0.44 |
### Linear-probe Classification:
| | Accuracy |
| ---------------- | -------- |
| Top-1 | 0.41 |
| Top-5 | 0.65 |
| mean(Top1, Top5) | 0.53 |
# Notes:
EmbeddingDatasetReader isn't in the current pip install version of clip_video_encode. Please install with ```python setup.py install``` from the cloned clip_video_encode repo.
Huggingface load_dataset is not currently implemented for the CLIP-Kinetics700 dataset (https://github.com/iejMac/clip-video-encode/issues/14). Please use:
```bash
git lfs install
git clone https://huggingface.co/datasets/iejMac/CLIP-Kinetics700
cd CLIP-Kinetics700
git lfs pull
```