https://github.com/laurent2916/nio-llm

You own little LLM in your matrix chatroom
https://github.com/laurent2916/nio-llm

bot llamacpp matrix-nio python3

Last synced: 9 months ago
JSON representation

You own little LLM in your matrix chatroom

• Host: GitHub
• URL: https://github.com/laurent2916/nio-llm
• Owner: Laurent2916
• License: mit
• Archived: true
• Created: 2023-05-24T20:42:22.000Z (about 3 years ago)
• Default Branch: master
• Last Pushed: 2023-10-21T10:26:23.000Z (over 2 years ago)
• Last Synced: 2025-03-13T14:24:45.390Z (over 1 year ago)
• Topics: bot, llamacpp, matrix-nio, python3
• Language: Python
• Homepage:
• Size: 179 KB
• Stars: 2
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# Nio LLM

[![GitHub](https://img.shields.io/github/license/Laurent2916/nio-llm)](https://github.com/Laurent2916/nio-llm/blob/master/LICENSE)

[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)

[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/badge/v1.json)](https://github.com/charliermarsh/ruff)

You own little LLM in your matrix chatroom.

## Usage

This project is split in two parts: the client and the server.

The server simply downloads an LLM and starts a llama-cpp-python server (which mimics an openai server).

The client connects to the matrix server and queries the llama-cpp-python server to create matrix messages.

## Special thanks

- https://github.com/abetlen/llama-cpp-python

- https://github.com/ggerganov/llama.cpp/