https://github.com/butanium/chat_templates_tools
https://github.com/butanium/chat_templates_tools
Last synced: 4 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/butanium/chat_templates_tools
- Owner: Butanium
- Created: 2025-07-22T12:44:15.000Z (7 months ago)
- Default Branch: main
- Last Pushed: 2025-09-19T19:57:20.000Z (5 months ago)
- Last Synced: 2025-09-19T22:04:20.349Z (5 months ago)
- Language: Python
- Size: 51.8 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
## Features
todo, but `tokenization_utils.py` was initially created for gemma 2 only, so there are some functions with very generic docs but that will actually only work for gemma 2. I need to fix this :)
## Templates
The `templates` folder contains the following templates:
- `{model_name}_chat_template.jinja`: Chat template for `model_name` with assistant mask implemented.
- `{model_name}_chat_template_ctrl_tokens.jinja`: Chat template for `model_name` where assistant mask returns 1 for the template tokens.
- `{model_name}_force_thinking.jinja`: Chat template for `model_name`, where `enable_thinking=True` in `tokenizer.apply_chat_template` force the model to think.