https://github.com/robertanto/local-llm-ui

This repository contains the code to deploy a Mistral-based chatbot using Docker Compose and Huggingface Inference API.
https://github.com/robertanto/local-llm-ui

Last synced: about 4 hours ago
JSON representation

This repository contains the code to deploy a Mistral-based chatbot using Docker Compose and Huggingface Inference API.

• Host: GitHub
• URL: https://github.com/robertanto/local-llm-ui
• Owner: robertanto
• Created: 2024-02-04T21:33:43.000Z (11 months ago)
• Default Branch: main
• Last Pushed: 2024-10-13T16:18:49.000Z (3 months ago)
• Last Synced: 2024-11-11T06:05:44.732Z (about 2 months ago)
• Language: Python
• Homepage:
• Size: 245 KB
• Stars: 1
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Deploy a chatbot with Huggingface Inference API

This repository contains the code to deploy a Mistral-based chatbot using Docker Compose and Huggingface Inference API.



## Technological stack

As shown in the figure below the following frameworks have been used in this project:

- Langchain

- Huggingface API

- FastAPI

- Gradio



## How to use

1. Clone the repository.

```bash

git clone https://github.com/robertanto/local-chatbot-ui.git

cd local-chatbot-ui

```

2. Create a Huggingface API token as shown [here](https://huggingface.co/docs/hub/security-tokens) and insert it in the `docker-compose.yaml` file.

3. Run the containers

```bash

docker compose up -d

```

You can interact with the chatbot at [http://localhost:7860/](http://localhost:7860/).