{"id":18608993,"url":"https://github.com/kspviswa/PyOMlx","last_synced_at":"2025-04-10T22:31:06.474Z","repository":{"id":220730221,"uuid":"751088229","full_name":"kspviswa/PyOMlx","owner":"kspviswa","description":"A wannabe Ollama equivalent for Apple MlX models","archived":false,"fork":false,"pushed_at":"2025-03-02T00:15:44.000Z","size":2822,"stargazers_count":62,"open_issues_count":0,"forks_count":4,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-03T08:43:00.093Z","etag":null,"topics":["chatbot","mlx","model-serving"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kspviswa.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":"roadmap.md","authors":null,"dei":null}},"created_at":"2024-01-31T22:43:52.000Z","updated_at":"2025-03-28T05:19:25.000Z","dependencies_parsed_at":"2024-02-03T23:21:23.085Z","dependency_job_id":"57c8d9ec-8607-4fce-bdff-5796d4febb57","html_url":"https://github.com/kspviswa/PyOMlx","commit_stats":null,"previous_names":["kspviswa/pyomlx"],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kspviswa%2FPyOMlx","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kspviswa%2FPyOMlx/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kspviswa%2FPyOMlx/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kspviswa%2FPyOMlx/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kspviswa","download_url":"https://codeload.github.com/kspviswa/PyOMlx/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248309467,"owners_count":21082216,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chatbot","mlx","model-serving"],"created_at":"2024-11-07T03:04:47.646Z","updated_at":"2025-04-10T22:31:06.466Z","avatar_url":"https://github.com/kspviswa.png","language":"Shell","funding_links":[],"categories":["Rising projects","Demo","Shell"],"sub_categories":["2024"],"readme":"![](logo_readme.png)\n# PyOMlx\n### Serve MlX models locally!\n\n### ![Downloads](https://img.shields.io/github/downloads/kspviswa/PyOMlx/total.svg)\n\n## Motivation\nInspired by [Ollama](https://github.com/ollama/ollama) project, I wanted to have a similar experience for serving [MLX models](https://github.com/ml-explore/mlx-examples). [Mlx from ml-explore](https://github.com/ml-explore/mlx) is a new framework for running ML models in Apple Silicon. This app is intended to be used along with [PyOllaMx](https://github.com/kspviswa/pyOllaMx)\n\nI'm using these in my day to day workflow and I intend to keep develop these for my use and benifit.\n\nIf you find this valuable, feel free to use it and contribute to this project as well. Please ⭐️ this repo to show your support and make my day!\n\nI'm planning on work on next items on this [roadmap.md](roadmap.md). Feel free to comment your thoughts (if any) and influence my work (if interested)\n\nMacOS DMGs are available in [Releases](https://github.com/kspviswa/PyOMlx/releases) page\n\n## How to use\n\n1) [Download](https://github.com/kspviswa/PyOMlx/releases) \u0026 Install the PyOMlx MacOS App\n\n2) Run the app\n\n3) You will now see the application running in the system tray. Use [PyOllaMx](https://github.com/kspviswa/pyOllaMx) to chat with MLX models seamlessly\n\n## Features\n\n### [v0.1.1](https://github.com/kspviswa/PyOMlx/releases/tag/0.1.1)\n- Revamped the http server portion to use the `mlx_lm.server` module. As of the latest version (`v0.20.5`) the module accepts dynamic model information from the incoming request. Hence this can be better utilized by PyOMlx. Also the `load()` function supports automatic model download from HF if not available in local `~/.cache `directory. This replaces the `/download` endpoint.\n- Finally, since `mlx_lm.server` runs a `httpd`, there is no need for external `flask`. So I got rid of that too. Resulting PyOMlx binary is very slim (~100 MB) and much much faster.\n- Rest everything is same as [v0.1.0](https://github.com/kspviswa/PyOMlx/releases/tag/0.1.0)\n\n\n### [v0.1.0](https://github.com/kspviswa/PyOMlx/releases/tag/0.1.0)\n- Added OpenAI API Compatible [chat completions](https://platform.openai.com/docs/api-reference/chat/create) and [list models](https://platform.openai.com/docs/api-reference/models/list) endpoint.\n- Added `/download` endpoint to download MLX models directly from HuggingFace Hub. All models will be downloaded from [MLX Community](https://huggingface.co/mlx-community) in HF Hub.\n- Added `/swagger.json` endpoint to serve OpenAPI Spec of all endpoints available with PyOMlx.\n\nNow you simply use any standard OpenAI Client to interact with your MLX models easily. More info on the [v0.1.0 release](https://github.com/kspviswa/PyOMlx/releases/tag/0.1.0) page.\n\n### [v0.0.3](https://github.com/kspviswa/PyOMlx/releases/tag/0.0.3)\n- Updated `mlx-lm` to support Gemma models\n\n### [v0.0.1](https://github.com/kspviswa/PyOMlx/releases/tag/0.0.1)\n- Automatically discover \u0026 serve MLX models that are downloaded from [MLX Huggingface community](https://huggingface.co/mlx-community).\n- Easy start-up / shutdown via MacOS App\n- System tray indication\n\n## Demo\n\nhttps://github.com/kspviswa/pyOllaMx/assets/7476271/dc686d60-182d-4f90-a771-9c1df1c70b5c","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkspviswa%2FPyOMlx","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkspviswa%2FPyOMlx","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkspviswa%2FPyOMlx/lists"}