https://github.com/ariya/tinker-chat

https://github.com/ariya/tinker-chat

chatbot generative-ai gpt llama llama2 llm mistral nlp openai

Last synced: 2 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/ariya/tinker-chat
• Owner: ariya
• License: mit
• Created: 2023-08-11T05:45:26.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2024-07-20T18:01:50.000Z (6 months ago)
• Last Synced: 2025-01-01T09:32:29.058Z (10 days ago)
• Topics: chatbot, generative-ai, gpt, llama, llama2, llm, mistral, nlp, openai
• Language: Clojure
• Homepage:
• Size: 286 KB
• Stars: 28
• Watchers: 3
• Forks: 6
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# tinker-chat

![Screenshot](public/screenshot.png)

This chat app supports GPT from OpenAI or your own local LLM.

### GPT from OpenAI

To use GPT from OpenAI, set the environment variable `OPENAI_API_KEY` to your [API key](https://platform.openai.com/account/api-keys).

### Local LLM

To utilize [llama.cpp](https://github.com/ggerganov/llama.cpp) locally with its inference engine, first load a quantized model such as [Phi-3 Mini](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf), e.g.:

```bash

/path/to/llama.cpp/server -m Phi-3-mini-4k-instruct-q4.gguf

```

Before launching the demo, set the environment variable `OPENAI_API_BASE`:

```bash

export OPENAI_API_BASE=http://127.0.0.1:8080

```

## Demo

With [Node.js](https://nodejs.org) >= v18:

```

npm install

npm start

```

and open `localhost:5000` with a web browser.