https://github.com/lun-4/smollama

fucked script to create a tiny llm for phone use because i dont know how to use pytorch
https://github.com/lun-4/smollama

Last synced: about 1 month ago
JSON representation

fucked script to create a tiny llm for phone use because i dont know how to use pytorch

• Host: GitHub
• URL: https://github.com/lun-4/smollama
• Owner: lun-4
• License: unlicense
• Created: 2024-05-11T03:34:12.000Z (8 months ago)
• Default Branch: mistress
• Last Pushed: 2024-05-11T04:19:40.000Z (8 months ago)
• Last Synced: 2024-12-01T10:54:20.650Z (about 1 month ago)
• Language: Python
• Size: 10.7 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# smollama

fucked script to create a tiny llm for phone use because i dont know how to use pytorch

option 1:

 - i learn pytorch

 - i tweak hparams to get a llama model class that is tiny w/ random weights

 - i then use llama.cpp conversion script

option 2:

 - i cook a model file directly with gguf-py

 - hopefully it works?

this project is _option 2_

```sh

git clone ...

cd ...

wget 'https://huggingface.co/TheBloke/TinyLlama-1.1B-1T-OpenOrca-GPTQ/raw/main/tokenizer.json'

# outputs to /tmp/example.gguf because its faster (since tmpfs is ram)

LLAMACPP_PATH=path/to/llama.cpp python3 ./random_llama.py

path/to/llama.cpp/main -m /tmp/example.gguf

```