https://github.com/synw/infergui

Graphical user inferface for local language models
https://github.com/synw/infergui

Last synced: about 2 months ago
JSON representation

Graphical user inferface for local language models

• Host: GitHub
• URL: https://github.com/synw/infergui
• Owner: synw
• License: mit
• Created: 2023-08-05T11:23:41.000Z (almost 2 years ago)
• Default Branch: main
• Last Pushed: 2024-05-24T15:26:34.000Z (about 1 year ago)
• Last Synced: 2025-03-23T21:47:11.287Z (2 months ago)
• Language: Vue
• Size: 27.8 MB
• Stars: 9
• Watchers: 6
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Infer gui

[![pub package](https://img.shields.io/npm/v/infergui)](https://www.npmjs.com/package/infergui)

A graphical user interface for local language models. It support different local backends:

- [Llama.cpp server](https://github.com/ggerganov/llama.cpp/tree/master/examples/server#llamacppexampleserver)

- [Koboldcpp](https://github.com/LostRuins/koboldcpp)

- [Ollama](https://github.com/jmorganca/ollama)

This interface uses no hidden magic: everything is explicit. What you see is what you prompt.

## Features

- Template editor, model configuration, inference params editor

- Gbnf grammars, multimodal

- Multiple local and remote servers config

## Install

### With npm

Install:

```bash

npm i -g infergui

```

Run:

```bash

infergui

```

Run a local backend and open `localhost:5183` in a browser

### With Python

```

git clone https://github.com/synw/infergui

cd infergui/dist

python3 -m http.server

```

Run a local backend and open `localhost:8000` in a browser

## Screenshots

### Templates editor

![Screenshot](/docs/template_editor.png)

### Gbnf grammar editor

![Screenshot](/docs/grammar_editor.png)

### Multimodal

![Screenshot](/docs/multimodal.png)

## Model picker (Ollama only)

![Screenshot](/docs/model_picker.png)

## Dev mode

Clone and install the dependencies:

```

npm i

```

To run in dev mode with watch autoreload:

```

npm run dev

```

Run a local backend and open `localhost:5173` in a browser

To build and run:

```

npm run build

npm run local

```