https://github.com/tpaviot/llama.cpp-azure

Build, test and run llama.cpp on Azure
https://github.com/tpaviot/llama.cpp-azure

Last synced: over 1 year ago
JSON representation

Build, test and run llama.cpp on Azure

• Host: GitHub
• URL: https://github.com/tpaviot/llama.cpp-azure
• Owner: tpaviot
• License: mit
• Created: 2023-07-11T05:11:36.000Z (about 3 years ago)
• Default Branch: main
• Last Pushed: 2023-09-01T14:57:31.000Z (almost 3 years ago)
• Last Synced: 2025-01-29T13:29:25.942Z (over 1 year ago)
• Language: Python
• Homepage:
• Size: 56.6 KB
• Stars: 1
• Watchers: 4
• Forks: 1
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# llama.cpp-azure

[![Azure Build Status](https://dev.azure.com/tpaviot/llama.cpp-azure/_apis/build/status/tpaviot.llama.cpp-azure?branchName=main)](https://dev.azure.com/tpaviot/llama.cpp-azure/_build?definitionId=13)

Use azure to build, test and run [llama.cpp](https://github.com/ggerganov/llama.cpp) master branch.

**About:**

- Build llama.cpp using Blas

- Download openlm-research/open_llama_3b_v2 hf model

- Test Export model to ggml F16

- Test q5_k_ quantization

- Download CodeLlama-7B-Python-GGUF-q4_k_m from hf/TheBloke

- Test inference of Q4_k_m quantized model

- Compute perplexity a small subset of wikitext-2