{"id":16622601,"url":"https://github.com/lmstudio-ai/mlx-engine","last_synced_at":"2026-01-15T01:19:59.724Z","repository":{"id":258635162,"uuid":"869139913","full_name":"lmstudio-ai/mlx-engine","owner":"lmstudio-ai","description":"LM Studio Apple MLX engine ","archived":false,"fork":false,"pushed_at":"2025-12-17T15:32:26.000Z","size":476,"stargazers_count":842,"open_issues_count":36,"forks_count":76,"subscribers_count":11,"default_branch":"main","last_synced_at":"2025-12-18T23:31:25.643Z","etag":null,"topics":["mlx","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lmstudio-ai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-10-07T19:32:25.000Z","updated_at":"2025-12-18T14:48:45.000Z","dependencies_parsed_at":null,"dependency_job_id":"f7630466-cc9a-48f9-a2fc-1d058f9f27f2","html_url":"https://github.com/lmstudio-ai/mlx-engine","commit_stats":null,"previous_names":["lmstudio-ai/mlx-engine"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/lmstudio-ai/mlx-engine","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lmstudio-ai%2Fmlx-engine","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lmstudio-ai%2Fmlx-engine/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lmstudio-ai%2Fmlx-engine/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lmstudio-ai%2Fmlx-engine/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lmstudio-ai","download_url":"https://codeload.github.com/lmstudio-ai/mlx-engine/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lmstudio-ai%2Fmlx-engine/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28441031,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-15T00:55:22.719Z","status":"ssl_error","status_checked_at":"2026-01-15T00:55:20.945Z","response_time":107,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["mlx","python"],"created_at":"2024-10-12T03:01:22.752Z","updated_at":"2026-01-15T01:19:59.711Z","avatar_url":"https://github.com/lmstudio-ai.png","language":"Python","readme":"\u003cp align=\"center\"\u003e\n  \u003cpicture\u003e \n    \u003cimg alt=\"lmstudio + MLX\" src=\"https://github.com/user-attachments/assets/128bf3ba-d8d6-4fc8-85c9-4d0113ba5499\"\u003e\n  \u003c/picture\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\u003cbold\u003e\u003ccode\u003emlx-engine\u003c/code\u003e - \u003ca href=\"https://github.com/ml-explore/mlx\"\u003eApple MLX\u003c/a\u003e LLM Engine for \u003ca href=\"https://lmstudio.ai/\"\u003eLM Studio\u003c/a\u003e\u003c/bold\u003e\u003c/p\u003e\n\u003cbr/\u003e\n\u003cp align=\"center\"\u003e\u003ca href=\"https://discord.gg/aPQfnNkxGC\"\u003e\u003cimg alt=\"Discord\" src=\"https://img.shields.io/discord/1110598183144399058?logo=discord\u0026style=flat\u0026logoColor=white\"\u003e\u003c/a\u003e\u003c/p\u003e\n\n# mlx-engine\nMLX engine for LM Studio\n\n\u003cbr/\u003e\n\n## Built with\n- [mlx-lm](https://github.com/ml-explore/mlx-lm) - Apple MLX inference engine (MIT)\n- [Outlines](https://github.com/dottxt-ai/outlines) - Structured output for LLMs (Apache 2.0)\n- [mlx-vlm](https://github.com/Blaizzy/mlx-vlm) - Vision model inferencing for MLX (MIT)\n\n\u003cbr/\u003e\n\n## How to use in LM Studio\nLM Studio 0.3.4 and newer for Mac ships pre-bundled with mlx-engine.\nDownload LM Studio from [here](https://lmstudio.ai/download?os=mac)\n\n\u003cbr/\u003e\n\n## Standalone Demo\n\n### Prerequisites\n\n- macOS 14.0 (Sonoma) or greater.\n- python3.11\n  - The requirements.txt file is compiled specifically for python3.11. python3.11 is the python version bundled within the LM Studio MLX runtime\n  - `brew install python@3.11` is a quick way to add python3.11 to your path that doesn't break your default python setup\n\n### Install Steps\nTo run a demo of model load and inference:\n1. Clone the repository\n```\ngit clone https://github.com/lmstudio-ai/mlx-engine.git\ncd mlx-engine\n```\n2. Create a virtual environment (optional)\n```\n python3.11 -m venv .venv\n source .venv/bin/activate\n```\n3. Install the required dependency packages\n```\npip install -U -r requirements.txt\n```\n\n### Text Model Demo\nDownload models with the `lms` CLI tool. The `lms` CLI documentation can be found here: https://lmstudio.ai/docs/cli\nRun the `demo.py` script with an MLX text generation model:\n```bash\nlms get mlx-community/Meta-Llama-3.1-8B-Instruct-4bit\npython demo.py --model mlx-community/Meta-Llama-3.1-8B-Instruct-4bit \n```\n[mlx-community/Meta-Llama-3.1-8B-Instruct-4bit](https://model.lmstudio.ai/download/mlx-community/Meta-Llama-3.1-8B-Instruct-4bit) - 4.53 GB\n\nThis command will use a default prompt. For a different prompt, add a custom `--prompt` argument like:\n```bash\nlms get mlx-community/Mistral-Small-Instruct-2409-4bit\npython demo.py --model mlx-community/Mistral-Small-Instruct-2409-4bit --prompt \"How long will it take for an apple to fall from a 10m tree?\"\n```\n[mlx-community/Mistral-Small-Instruct-2409-4bit](https://model.lmstudio.ai/download/mlx-community/Mistral-Small-Instruct-2409-4bit) - 12.52 GB\n\n### Vision Model Demo\nRun the `demo.py` script with an MLX vision model:\n```bash\nlms get mlx-community/pixtral-12b-4bit\npython demo.py --model mlx-community/pixtral-12b-4bit --prompt \"Compare these images\" --images demo-data/chameleon.webp demo-data/toucan.jpeg\n```\nCurrently supported vision models include:\n - [Llama-3.2-Vision](https://model.lmstudio.ai/download/mlx-community/Llama-3.2-11B-Vision-Instruct-4bit)\n   - `lms get mlx-community/Llama-3.2-11B-Vision-Instruct-4bit`\n - [Pixtral](https://model.lmstudio.ai/download/mlx-community/pixtral-12b-4bit)\n   - `lms get mlx-community/pixtral-12b-4bit`\n - [Qwen2-VL](https://model.lmstudio.ai/download/mlx-community/Qwen2-VL-7B-Instruct-4bit)\n   - `lms get mlx-community/Qwen2-VL-7B-Instruct-4bit`\n - [Llava-v1.6](https://model.lmstudio.ai/download/mlx-community/llava-v1.6-mistral-7b-4bit)\n   - `lms get mlx-community/llava-v1.6-mistral-7b-4bit`\n\n### Speculative Decoding Demo\nRun the `demo.py` script with an MLX text generation model and a compatible `--draft-model`\n```bash\nlms get mlx-community/Qwen2.5-7B-Instruct-4bit\nlms get lmstudio-community/Qwen2.5-0.5B-Instruct-MLX-8bit\npython demo.py \\\n    --model mlx-community/Qwen2.5-7B-Instruct-4bit \\\n    --draft-model lmstudio-community/Qwen2.5-0.5B-Instruct-MLX-8bit \\\n    --prompt \"\u003c|im_start|\u003esystem\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.\u003c|im_end|\u003e\n\u003c|im_start|\u003euser\nWrite a quick sort algorithm in C++\u003c|im_end|\u003e\n\u003c|im_start|\u003eassistant\n\"\n```\n\n## Development Setup\n\n### Pre-commit Hooks\n\nWe use pre-commit hooks to maintain code quality. Before contributing, please:\n\n1. Install pre-commit:\n   ```bash\n   pip install pre-commit \u0026\u0026 pre-commit install\n    ```\n2. Run pre-commit:\n   ```bash\n   pre-commit run --all-files\n   ```\n3. Fix any issues before submitting your PR\n\n## Testing\n\nTo run tests, run the following from the root of this repo:\n```bash\npython -m pip install pytest\npython -m pytest tests/\n```\n\nTo test specific vision models:\n```bash\npython -m pytest tests/test_vision_models.py -k pixtral\n```\n\n## Attribution\n\nErnie 4.5 modeling code is sourced from [Baidu](https://huggingface.co/baidu/ERNIE-4.5-0.3B-PT/tree/da6f3b1158d5d0d2bbf552bfc3364c9ec64e8aa5)\n","funding_links":[],"categories":["HarmonyOS","Python","Apps, UI \u0026 Tooling"],"sub_categories":["Windows Manager"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flmstudio-ai%2Fmlx-engine","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flmstudio-ai%2Fmlx-engine","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flmstudio-ai%2Fmlx-engine/lists"}