Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/simonw/llm
Access large language models from the command-line
https://github.com/simonw/llm
ai llms openai
Last synced: about 2 months ago
JSON representation
Access large language models from the command-line
- Host: GitHub
- URL: https://github.com/simonw/llm
- Owner: simonw
- License: apache-2.0
- Created: 2023-04-01T21:16:57.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-07-18T19:33:15.000Z (2 months ago)
- Last Synced: 2024-07-20T00:18:32.797Z (2 months ago)
- Topics: ai, llms, openai
- Language: Python
- Homepage: https://llm.datasette.io
- Size: 584 KB
- Stars: 3,517
- Watchers: 34
- Forks: 186
- Open Issues: 206
-
Metadata Files:
- Readme: README.md
- Contributing: docs/contributing.md
- License: LICENSE
Awesome Lists containing this project
- awesome-local-llms - llm - line | 3,272 | 163 | 188 | 19 | 25 | Apache License 2.0 | 9 days, 23 hrs, 32 mins | (Open-Source Local LLM Projects)
- awesome-ChatGPT-repositories - llm - Access large language models from the command-line (NLP)
- jimsghstars - simonw/llm - Access large language models from the command-line (Python)
README
# LLM
[](https://pypi.org/project/llm/)
[](https://llm.datasette.io/)
[](https://llm.datasette.io/en/stable/changelog.html)
[](https://github.com/simonw/llm/actions?query=workflow%3ATest)
[](https://github.com/simonw/llm/blob/main/LICENSE)
[](https://datasette.io/discord-llm)
[](https://formulae.brew.sh/formula/llm)
A CLI utility and Python library for interacting with Large Language Models, both via remote APIs and models that can be installed and run on your own machine.
[Run prompts from the command-line](https://llm.datasette.io/en/stable/usage.html#executing-a-prompt), [store the results in SQLite](https://llm.datasette.io/en/stable/logging.html), [generate embeddings](https://llm.datasette.io/en/stable/embeddings/index.html) and more.
Consult the **[LLM plugins directory](https://llm.datasette.io/en/stable/plugins/directory.html)** for plugins that provide access to remote and local models.
Full documentation: **[llm.datasette.io](https://llm.datasette.io/)**
Background on this project:
- [llm, ttok and strip-tags—CLI tools for working with ChatGPT and other LLMs](https://simonwillison.net/2023/May/18/cli-tools-for-llms/)
- [The LLM CLI tool now supports self-hosted language models via plugins](https://simonwillison.net/2023/Jul/12/llm/)
- [Accessing Llama 2 from the command-line with the llm-replicate plugin](https://simonwillison.net/2023/Jul/18/accessing-llama-2/)
- [Run Llama 2 on your own Mac using LLM and Homebrew](https://simonwillison.net/2023/Aug/1/llama-2-mac/)
- [Catching up on the weird world of LLMs](https://simonwillison.net/2023/Aug/3/weird-world-of-llms/)
- [LLM now provides tools for working with embeddings](https://simonwillison.net/2023/Sep/4/llm-embeddings/)
- [Build an image search engine with llm-clip, chat with models with llm chat](https://simonwillison.net/2023/Sep/12/llm-clip-and-chat/)
- [Many options for running Mistral models in your terminal using LLM](https://simonwillison.net/2023/Dec/18/mistral/)
## Installation
Install this tool using `pip`:
```bash
pip install llm
```
Or using [Homebrew](https://brew.sh/):
```bash
brew install llm
```
[Detailed installation instructions](https://llm.datasette.io/en/stable/setup.html).
## Getting started
If you have an [OpenAI API key](https://platform.openai.com/api-keys) you can get started using the OpenAI models right away.
As an alternative to OpenAI, you can [install plugins](https://llm.datasette.io/en/stable/plugins/installing-plugins.html) to access models by other providers, including models that can be installed and run on your own device.
Save your OpenAI API key like this:
```bash
llm keys set openai
```
This will prompt you for your key like so:
```
Enter key:
```
Now that you've saved a key you can run a prompt like this:
```bash
llm "Five cute names for a pet penguin"
```
```
1. Waddles
2. Pebbles
3. Bubbles
4. Flappy
5. Chilly
```
Read the [usage instructions](https://llm.datasette.io/en/stable/usage.html) for more.
## Installing a model that runs on your own machine
[LLM plugins](https://llm.datasette.io/en/stable/plugins/index.html) can add support for alternative models, including models that run on your own machine.
To download and run Mistral 7B Instruct locally, you can install the [llm-gpt4all](https://github.com/simonw/llm-gpt4all) plugin:
```bash
llm install llm-gpt4all
```
Then run this command to see which models it makes available:
```bash
llm models
```
```
gpt4all: all-MiniLM-L6-v2-f16 - SBert, 43.76MB download, needs 1GB RAM
gpt4all: orca-mini-3b-gguf2-q4_0 - Mini Orca (Small), 1.84GB download, needs 4GB RAM
gpt4all: mistral-7b-instruct-v0 - Mistral Instruct, 3.83GB download, needs 8GB RAM
...
```
Each model file will be downloaded once the first time you use it. Try Mistral out like this:
```bash
llm -m mistral-7b-instruct-v0 'difference between a pelican and a walrus'
```
You can also start a chat session with the model using the `llm chat` command:
```bash
llm chat -m mistral-7b-instruct-v0
```
```
Chatting with mistral-7b-instruct-v0
Type 'exit' or 'quit' to exit
Type '!multi' to enter multiple lines, then '!end' to finish
>
```
## Using a system prompt
You can use the `-s/--system` option to set a system prompt, providing instructions for processing other input to the tool.
To describe how the code a file works, try this:
```bash
cat mycode.py | llm -s "Explain this code"
```
## Help
For help, run:
llm --help
You can also use:
python -m llm --help