https://github.com/ingydotnet/yamllm
https://github.com/ingydotnet/yamllm
Last synced: 8 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/ingydotnet/yamllm
- Owner: yaml
- Created: 2024-06-16T14:00:03.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2024-11-11T01:29:59.000Z (over 1 year ago)
- Last Synced: 2025-02-04T04:01:46.514Z (over 1 year ago)
- Language: Makefile
- Size: 44.9 KB
- Stars: 7
- Watchers: 5
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: ReadMe.md
Awesome Lists containing this project
README
yamllm
======
A YAMLScript tool to query LLMs
## Synopsis
```
$ yamllm 'Why pink?'
Q: Why pink?
llama3-8b-8192:
The question of pink!
The color pink has a long and complex history, and its associations and
meanings have evolved over time. Here are some possible reasons why pink might
be associated with certain qualities or themes:
...
$
```
## Overview
This is a project to play with various LLMs from the command line.
At the moment it works with [Groq](https://console.groq.com/) which you can use
for free with generous limits.
It's also pretty fast.
It also works with [OpenAI's API](https://platform.openai.com/docs/overview),
which you have to pay for but costs pennies for single user interactive usage.
The tool is written in YAMLScript and requires a recent `ys`.
You can install it here (in `$PWD/bin/`) with `make install-ys`.
## Installation
* Get a Groq API key [here](https://console.groq.com/keys).
* Or get an OpenAI API key [here](https://platform.openai.com/api-keys).
* Or get an Anthropic API key [here](
https://console.anthropic.com/settings/keys).
Run these commands:
```
export YAMLLM_GROQ_API_KEY=
# or
export YAMLLM_OPENAI_API_KEY=
# or
export YAMLLM_ANTHROPIC_API_KEY=
git clone https://github.com/yaml/yamllm
source .rc # Adds `$PWD/bin` to your PATH (for bash or zsh)
yamllm --help # Ask it for some help
yamllm 'Why pink?' # Get to the important stuff
```
### Optional
These programs make it nicer to use but are not required:
* Install `rlwrap` for readline support and query history.
* Install `prettier` for 80 column output reformatting.
```
npm install -g prettier
```
### Try in a Browser with Github Codespaces
Use the [Codespace Quickstart Link](
https://codespaces.new/yaml/yamllm?quickstart=1) to try out yamllm in a browser
environment.
## CLI Usage
To start a query loop:
```
$ yamllm
```
Use `ctl-d`, `exit` or `q` to exit.
To run a single query:
```
$ yamllm 'Tell me a story'
```
## Environment Variables
To use these variables, be sure to `export` them after setting them.
One of the first 2 here is required.
* `YAMLLM_GROQ_API_KEY=`
Required to access the Groq API (free).
Get one [here](https://console.groq.com/keys).
* `YAMLLM_OPENAI_API_KEY=`
Required to access the OpenAI API (not free, but not expensive).
Get one [here](https://platform.openai.com/api-keys).
* `YAMLLM_MODEL=`
Choose one of the Groq or OpenAI models:
```
* llama3 (llama3-70b-8192 Meta)
* llama38b (llama3-8b-8192 Meta)
* gemma (gemma-7b-it Google)
* gemma2 (gemma2-9b-it Google)
* mixtral (mixtral-8x7b-32768 Mistral)
* whisper (whisper-large-v3 OpenAI)
* gpt4o- (gpt-4o-mini OpenAI)
* gpt4o (gpt-4o OpenAI)
* gpt4t (gpt-4-turbo OpenAI)
* gpt4 (gpt-4 OpenAI)
* gpt3 (gpt-3.5-turbo OpenAI)
* dalle2 (dall-e-2 OpenAI)
* dalle3 (dall-e-3 OpenAI)
```
Default is `llama3` if `YAMLLM_GROQ_API_KEY` is set, `gpt4o-` if
`YAMLLM_OPENAI_API_KEY` is set else error.
* `YAMLLM_LOG_DIR=`
A directory to store query results.
Directory can be relative or absolute but must exist.
* `YAMLLM_PRE=`
File containing text to prepend to every prompt sent to API.
* `YAMLLM_POST=`
File containing text to append to every prompt sent to API.
* `YAMLLM_PROMPT=`
File containing the prompt text to send to API.
`YAMLLM_PRE` and `YAMLLM_POST` are added if specified.
* `YAMLLM_IMAGE_SIZE=`
One of `1024x1024`, `1024x1792` or `1792x1024`.
* `YAMLLM_TEMP=`
Higher is more random.
Lower is more deterministic.
Default is 0.8.
* `YAMLLM_TOPP=`
Alternative way to do something like `YAMLLM_TEMP`.
Default is 1.0.
* `YAMLLM_TEST_FILE=$PWD/test/test1/yaml`
Set this to run the YAML interaction test suite.
* `YAMLLM_DEBUG=1`
Dump API payloads.