https://github.com/abichinger/typechat-llama-example
Sentiment example using TypeChat with a self hosted LLM
https://github.com/abichinger/typechat-llama-example
ai llama llm
Last synced: 8 months ago
JSON representation
Sentiment example using TypeChat with a self hosted LLM
- Host: GitHub
- URL: https://github.com/abichinger/typechat-llama-example
- Owner: abichinger
- License: mit
- Created: 2023-08-01T19:11:13.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2023-08-01T20:56:45.000Z (almost 3 years ago)
- Last Synced: 2025-09-23T02:51:30.453Z (9 months ago)
- Topics: ai, llama, llm
- Language: TypeScript
- Homepage:
- Size: 233 KB
- Stars: 11
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# TypeChat-llama-example - TypeChat with Llama
Sentiment example using [TypeChat](https://github.com/microsoft/TypeChat) with a self hosted LLM. No API keys required.
## ✅ Requirements
- Node.js version 18 or above
- [llama-cpp-python](https://github.com/abetlen/llama-cpp-python#web-server) web server
- Model supported by [llama.cpp](https://github.com/ggerganov/llama.cpp#description)
## ⚡️ Quick start
**Download a model**: e.g. llama-2-13b-chat.ggmlv3.q4_0
```bash
curl -L "https://huggingface.co/TheBloke/Llama-2-13B-chat-GGML/resolve/main/llama-2-13b-chat.ggmlv3.q4_0.bin" --create-dirs -o models/llama-2-13b-chat.ggmlv3.q4_0.bin
```
**Start llama-cpp-python web server:**
```bash
docker-compose up -d
```
**Run the example:**
```bash
yarn install
yarn start
```
## Notes
- [llama-node](https://llama-node.vercel.app/docs/start) could be used instead of the web server
## Used software components
- This example is a modified version of the TypeChat [sentiment example](https://github.com/microsoft/TypeChat/tree/main/examples/sentiment)
Copyright (c) Microsoft Corporation - [MIT License](https://github.com/microsoft/TypeChat/blob/main/LICENSE)
- [llama-cpp-python](https://github.com/abetlen/llama-cpp-python)
Copyright (c) 2023 Andrei Betlen - [MIT License](https://github.com/abetlen/llama-cpp-python/blob/main/LICENSE.md)