https://github.com/chrishayuk/embeddings
https://github.com/chrishayuk/embeddings
Last synced: about 1 month ago
JSON representation
- Host: GitHub
- URL: https://github.com/chrishayuk/embeddings
- Owner: chrishayuk
- Created: 2024-03-10T08:54:47.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-04-21T17:35:56.000Z (over 1 year ago)
- Last Synced: 2024-04-21T23:17:14.604Z (over 1 year ago)
- Language: Python
- Size: 7.81 KB
- Stars: 12
- Watchers: 3
- Forks: 5
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- jimsghstars - chrishayuk/embeddings - (Python)
README
# Introduction
These scripts are really about exploring embeddings, specifically input embeddings.
## Print Layers
If we need to print out the layers of an LLM, just call (by default it'll choose Gemma 2B)
```bash
python print_layers.py
```
and for other models such as mistral
```bash
python print_layers.py --model "mistralai/Mistral-7B-v0.1"
```
or llama 7b
```bash
python print_layers.py --model "meta-llama/Llama-2-7b-hf"
```
larger models such as llama-2-70b chat
```bash
python print_layers.py --model "meta-llama/Llama-2-70b-chat-hf"
```
## Print Tokens
If we need to print out the tokens of an LLM, just call (by default it'll choose Gemma 2B) and the phrase "Who is Ada Lovelace?"
```bash
python print_layers.py
```
and for other models such as mistral
```bash
python print_tokens.py --tokenizer "mistralai/Mistral-7B-v0.1" --prompt "Who is Kitty Purry?"
```
```bash
python extract_embeddings.py --tokenizer "meta-llama/Meta-Llama-3-8b-Instruct" --model "meta-llama/Meta-Llama-3-8b-Instruct" --embeddings_file "./output/llama3_8b_embeddings_layer.pth" --dimensions 4096
```
```bash
python visualize_cosine_similarity.py --tokenizer "meta-llama/Meta-Llama-3-8b-Instruct" --model "meta-llama/Meta-Llama-3-8b-Instruct" --embeddings_file "./output/llama3_8b_embeddings_layer.pth" --dimensions 4096 --prompt "Sit Sat Mat Bat Hat Cat Nap Kit Kat Dog Fish Tree Math London Paris Rio Berlin Sydney Moscow Red Blue Green Black White for while print loop"
```