https://github.com/zer0int/clip-tokenizer
A simple CLIP tokenizer encode / decode script. For when you need to know CLIP's tokenization and / or token IDs for some reason.
https://github.com/zer0int/clip-tokenizer
Last synced: 5 months ago
JSON representation
A simple CLIP tokenizer encode / decode script. For when you need to know CLIP's tokenization and / or token IDs for some reason.
- Host: GitHub
- URL: https://github.com/zer0int/clip-tokenizer
- Owner: zer0int
- Created: 2024-10-17T15:14:40.000Z (7 months ago)
- Default Branch: CLIP-vision
- Last Pushed: 2024-10-18T09:57:31.000Z (7 months ago)
- Last Synced: 2024-11-09T01:49:25.728Z (6 months ago)
- Language: Python
- Size: 1.27 MB
- Stars: 2
- Watchers: 2
- Forks: 0
- Open Issues: 0