Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/codito/arey
Simple large language model playground app
https://github.com/codito/arey
ai cli cpu-inference gguf large-language-models llama2 llamacpp llm local-model mistral ollama
Last synced: 3 months ago
JSON representation
Simple large language model playground app
- Host: GitHub
- URL: https://github.com/codito/arey
- Owner: codito
- License: mit
- Created: 2023-07-22T10:21:30.000Z (over 1 year ago)
- Default Branch: master
- Last Pushed: 2024-10-07T16:41:33.000Z (3 months ago)
- Last Synced: 2024-10-10T05:38:43.615Z (3 months ago)
- Topics: ai, cli, cpu-inference, gguf, large-language-models, llama2, llamacpp, llm, local-model, mistral, ollama
- Language: Python
- Homepage: https://apps.codito.in/arey/
- Size: 8.25 MB
- Stars: 4
- Watchers: 4
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# arey
> Arey (अरे, sanskrit) — ind. Interjection of calling.
Arey is a simple large language model playground in your terminal.
- ✨ Command line interface, works everywhere python is available.
- 🤖 Use any llama.cpp, ollama model or an openai compatible endpoint.
- 💬 Chat with your favorite local models. CPU friendly 🍀
- 🙋 Ask anything to AI models with a single command.
- 📋 Supercharged prompt fine-tuning workflow ❤️ Edit your prompt in _any_ editor
and `arey` will generate a completion on save.
- 🔓 No telemetry, no internet, nothing to sell.
See [Get Started](https://apps.codito.in/arey) or notes below for a quick guide.
https://github.com/codito/arey/assets/28766/6b886e49-6124-4256-84d9-20449c783a34
🚧 See **Roadmap** below.
## Installation
```sh
# Install pipx if needed: `pip install pipx`
# Ensure ~/.local/bin is available in system PATH
pipx install arey
```
Windows troubleshooting notes are [here](docs/windows.md).
[WSL]: https://learn.microsoft.com/en-us/windows/wsl/install
## Usage
```sh
❯ arey --help
Usage: arey [OPTIONS] COMMAND [ARGS]...
Arey - a simple large language model app.
Options:
-v, --verbose BOOLEAN Show verbose logs.
--help Show this message and exit.
Commands:
ask Run an instruction and generate response.
chat Chat with an AI model.
play Watch FILE for model, prompt and generate response on edit.
```
On the first run, `arey` will create a configuration file in following location:
- `~/.config/arey/arey.yml` for Linux or Mac systems.
- `~/.arey/arey.yml` for Windows.
Please update the `models` section in the config yml to your local model path.
### 1. Ask Arey something!
`arey ask "Who is Seneca? Tell me one of his teachings"`
### 2. Chat with Arey
`arey chat`
### 3. Run Arey in play mode
Use to fine-tune a prompt in your editor while `arey` keeps completing your prompt on every save.
```sh
❯ arey play /tmp/arey_playzl9igj3d.md
Welcome to arey play! Edit the play file below in your favorite editor and I'll generate a
response for you. Use `Ctrl+C` to abort play session.
Watching `/tmp/arey_playzl9igj3d.md` for changes...
───────────────────────────────────── 2024-01-21 17:20:01 ──────────────────────────────────────
✓ Model loaded. 0.57s.
Life is short because it passes by quickly and can end at any moment. We should make the most of
our time here on earth and live a virtuous life according to stoicism.
◼ Canceled.
Watching `/tmp/arey_playzl9igj3d.md` for changes...
```
## Development
See .
## Roadmap
- [x] Chat and task modes for interactive or batch queries
- [x] Define new tasks with only a prompt, no code. See docs/samples directory
for examples.
- [x] Markdown formatting for chat mode.
- [x] Ollama support.
- [ ] OpenAI server support.
- [ ] Command support in chat. E.g., logs, change model, copy, clear, etc.
- [ ] Discover prompts from user directory
- [ ] Manage prompts and create new interactively
- [ ] Download models and manage them
- [ ] Release v0.1
- [ ] Add [textfx](https://github.com/google/generative-ai-docs/tree/main/demos/palm/web/textfx)
- [ ] Add offline knowledge bases and RAG. See
## License
MIT