https://github.com/kspviswa/pyomlx

A wannabe Ollama equivalent for Apple MlX models
https://github.com/kspviswa/pyomlx

chatbot mlx model-serving

Last synced: 22 days ago
JSON representation

A wannabe Ollama equivalent for Apple MlX models

• Host: GitHub
• URL: https://github.com/kspviswa/pyomlx
• Owner: kspviswa
• License: mit
• Created: 2024-01-31T22:43:52.000Z (10 months ago)
• Default Branch: main
• Last Pushed: 2024-03-02T05:11:41.000Z (9 months ago)
• Last Synced: 2024-04-21T06:17:46.661Z (7 months ago)
• Topics: chatbot, mlx, model-serving
• Language: Python
• Homepage:
• Size: 2.67 MB
• Stars: 8
• Watchers: 2
• Forks: 1
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE
  • Roadmap: roadmap.md

Awesome Lists containing this project

README

        
![](logo_readme.png)

# PyOMlx

### Serve MlX models locally!

## Motivation

Inspired by [Ollama](https://github.com/ollama/ollama) project, I wanted to have a similar experience for serving [MLX models](https://github.com/ml-explore/mlx-examples). [Mlx from ml-explore](https://github.com/ml-explore/mlx) is a new framework for running ML models in Apple Silicon. This app is intended to be used along with [PyOllaMx](https://github.com/kspviswa/pyOllaMx)

I'm using these in my day to day workflow and I intend to keep develop these for my use and benifit.

If you find this valuable, feel free to use it and contribute to this project as well. Please ⭐️ this repo to show your support and make my day!

I'm planning on work on next items on this [roadmap.md](roadmap.md). Feel free to comment your thoughts (if any) and influence my work (if interested)

MacOS DMGs are available in [Releases](https://github.com/kspviswa/PyOMlx/releases) page

## How to use

1) [Download](https://github.com/kspviswa/PyOMlx/releases) & Install the PyOMlx MacOS App

2) Run the app

3) You will now see the application running in the system tray. Use [PyOllaMx](https://github.com/kspviswa/pyOllaMx) to chat with MLX models seamlessly

## Demo

https://github.com/kspviswa/pyOllaMx/assets/7476271/dc686d60-182d-4f90-a771-9c1df1c70b5c

## Features

### [v0.0.3](https://github.com/kspviswa/PyOMlx/releases/tag/0.0.1)

- Updated `mlx-lm` to support Gemma models

### [v0.0.1](https://github.com/kspviswa/PyOMlx/releases/tag/0.0.1)

- Automatically discover & serve MLX models that are downloaded from [MLX Huggingface community](https://huggingface.co/mlx-community).

- Easy start-up / shutdown via MacOS App

- System tray indication