https://github.com/simonw/llm-llamafile

Access llamafile localhost models via LLM
https://github.com/simonw/llm-llamafile

Last synced: about 1 year ago
JSON representation

Access llamafile localhost models via LLM

• Host: GitHub
• URL: https://github.com/simonw/llm-llamafile
• Owner: simonw
• License: apache-2.0
• Created: 2024-04-22T04:06:18.000Z (about 2 years ago)
• Default Branch: main
• Last Pushed: 2024-04-22T04:10:19.000Z (about 2 years ago)
• Last Synced: 2024-10-18T07:53:18.518Z (over 1 year ago)
• Language: Python
• Size: 5.86 KB
• Stars: 14
• Watchers: 1
• Forks: 0
• Open Issues: 2
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-ai-devtools - llm-llamafile - llamafile#readme) / [Repo](https://github.com/simonw/llm-llamafile) | (Browse The Shelves / Local LLM developer tools)

README

          
# llm-llamafile

[![PyPI](https://img.shields.io/pypi/v/llm-llamafile.svg)](https://pypi.org/project/llm-llamafile/)

[![Changelog](https://img.shields.io/github/v/release/simonw/llm-llamafile?include_prereleases&label=changelog)](https://github.com/simonw/llm-llamafile/releases)

[![Tests](https://github.com/simonw/llm-llamafile/actions/workflows/test.yml/badge.svg)](https://github.com/simonw/llm-llamafile/actions/workflows/test.yml)

[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/llm-llamafile/blob/main/LICENSE)

Access llamafile localhost models via LLM

## Installation

Install this plugin in the same environment as [LLM](https://llm.datasette.io/).

```bash

llm install llm-llamafile

```

## Usage

Make sure you have a `llamafile` running on `localhost`, serving an OpenAI compatible API endpoint on port 8080.

You can then use `llm` to interact with that model like so:

```bash

llm -m llamafile "3 neat characteristics of a pelican"

```

## Development

To set up this plugin locally, first checkout the code. Then create a new virtual environment:

```bash

cd llm-llamafile

python3 -m venv venv

source venv/bin/activate

```

Now install the dependencies and test dependencies:

```bash

llm install -e '.[test]'

```

To run the tests:

```bash

pytest

```