https://github.com/roialfassi/promptqueuerunner
automate the batching and execution of prompts.
https://github.com/roialfassi/promptqueuerunner
ai ai-automation llama llm lm-studio local-llm openai openai-api-python
Last synced: 3 months ago
JSON representation
automate the batching and execution of prompts.
- Host: GitHub
- URL: https://github.com/roialfassi/promptqueuerunner
- Owner: Roialfassi
- License: mit
- Created: 2024-04-08T11:02:30.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2024-06-24T07:25:40.000Z (11 months ago)
- Last Synced: 2025-02-24T09:39:59.257Z (3 months ago)
- Topics: ai, ai-automation, llama, llm, lm-studio, local-llm, openai, openai-api-python
- Language: Python
- Homepage:
- Size: 9.16 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: readme.md
- License: LICENSE
Awesome Lists containing this project
README
## PromptQueueRunner
This Python script allows you to query a locally hosted Large Language Model (LLM) with multiple prompts and save the responses to text files. It provides a command-line interface for easy usage and configuration.
### Prerequisites
* Python 3.8+
* openai library (install with pip install openai)
### Usage
* Clone or download the repository to your local machine.
* Change the credentials to the api you are willing to use
if you are using LM studio then it is:
```json
{
"base_url": "http://localhost:1234/v1",
"api_key": "lm-studio"
}
```
Replace the values for "base_url" and "api_key" with the appropriate values
Create a JSON file (e.g., prompts.json) containing an array of prompt and title dictionaries. Example:
```json
[
{
"prompt": "Introduce yourself.",
"title": "llm_introduction"
},
{
"prompt": "What is the meaning of life?",
"title": "meaning_of_life"
},
{
"prompt": "Describe your favorite food.",
"title": "favorite_food"
}
]
```
Open a terminal or command prompt and navigate to the project directory.
Open venv install requirements and then
Run the script with the following command:
```bash
python PromptQueueRunner.py -p prompts.json
```
This will process the prompts from the prompts.json file, query the local LLM with each prompt, and save the responses in the output/general folder with the corresponding titles as filenames.
## Command-line Arguments
The script supports the following command-line arguments:
* -p or --prompts: Path to the JSON file containing prompts and titles (required).
* -c or --credentials: Path to the JSON file containing LLM credentials (default: credentials.json).
* -o or --output: Path to the output folder where the response files will be saved (default: output).
* -t or --topic: Topic subfolder within the output folder where the response files will be saved (default: general).
* -s or --system: System role (instructions for the LLM) (default: "Always answer in rhymes.").
You can use the --help argument to display the help message and the available options:
```bash
python PromptQueueRunner.py --help
```
## Example
To run the script with custom options, use the following format:
```bash
python PromptQueueRunner.py -p prompts.json -c my_credentials.json -o my_output --topic science -s "Please respond in a scientific tone."
```
This will process the prompts from the prompts.json file, load the LLM credentials from the my_credentials.json file, save the responses in the my_output/science folder, and use the system role "Please respond in a scientific tone." when querying the LLM.
## Notes
* The script sanitizes the titles to ensure they are valid filenames. Characters that are not alphanumeric, underscore, hyphen, period, or parentheses are removed, and spaces are replaced with underscores.
* The responses from the LLM are saved as plain text files with the sanitized title as the filename.
* Make sure to update the credentials.json file with the correct credentials for your local LLM server.
* You can modify the prompts.json file to include different prompts and titles as needed.
## Creating prompts.json
Go to GPT or Claude and enter the following prompt:
```text
Can you write me a json array with 30 prompts for
{Your Subject}
each of the elements should have "title" and "prompt"
title explains the objective of the prompt in 1 line
and prompt is the prompt itself
```
Then copy the prompts to the json and run it.