https://github.com/jabberjabberjabber/llmtester

Upload text to an LLM using the Koboldcpp API. Change prompts. See what you get with different settings.
https://github.com/jabberjabberjabber/llmtester

Last synced: 3 months ago
JSON representation

Upload text to an LLM using the Koboldcpp API. Change prompts. See what you get with different settings.

• Host: GitHub
• URL: https://github.com/jabberjabberjabber/llmtester
• Owner: jabberjabberjabber
• License: mit
• Created: 2024-08-15T12:18:54.000Z (9 months ago)
• Default Branch: main
• Last Pushed: 2024-08-15T13:33:58.000Z (9 months ago)
• Last Synced: 2024-08-15T14:59:47.480Z (9 months ago)
• Language: Python
• Size: 110 KB
• Stars: 1
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# LLMTester

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

LLMTester lets you test new models and figure out prompting styles and sampler settings.

![LLMTester Screenshot](screenshot.PNG)

## Features

- You use it to make consistent prompts and send content to a Koboldcpp API

## Table of Contents

- [Installation](#installation)

- [Usage](#usage)

- [Configuration](#configuration)

- [FAQ](#FAQ)

- [Contributing](#contributing)

- [License](#license)

## Installation

### Windows

1. Clone the repository or download the zip file and extract it.

2. Install [Python for Windows](https://www.python.org/downloads/windows/).

4. Download the appropriate [KoboldCPP](https://github.com/LostRuins/koboldcpp/releases) and place it in the LLMTester folder. Make sure that you name it 'koboldcpp.exe'

5. Run `llmTester-run.bat` and wait for the model weights to download and Koboldcpp to start. 

The script will set up the Python environment, install dependencies, and download necessary model weights. This initial setup is performed only once and will take a few minutes depending on your download speed.

## Usage

1. Ensure KoboldCPP is running. Do not use the GUI windows until you see:

   ```

   Please connect to custom endpoint at http://localhost:5001

   ```

## Configuration

## FAQ

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.