{"id":13678814,"url":"https://github.com/PicoMLX/PicoMLXServer","last_synced_at":"2025-04-29T15:33:24.223Z","repository":{"id":228854352,"uuid":"767257109","full_name":"PicoMLX/PicoMLXServer","owner":"PicoMLX","description":"The easiest way to run the fastest MLX-based LLMs locally","archived":false,"fork":false,"pushed_at":"2024-10-30T17:47:15.000Z","size":2392,"stargazers_count":209,"open_issues_count":2,"forks_count":15,"subscribers_count":7,"default_branch":"main","last_synced_at":"2024-11-08T20:47:57.863Z","etag":null,"topics":["ai","llama3","mlx","ollama","openai","openai-api"],"latest_commit_sha":null,"homepage":"","language":"Swift","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PicoMLX.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-05T00:58:50.000Z","updated_at":"2024-11-08T11:08:28.000Z","dependencies_parsed_at":"2024-07-27T19:17:04.716Z","dependency_job_id":"d66c3ed9-666a-4dd0-8801-9bab91e51c02","html_url":"https://github.com/PicoMLX/PicoMLXServer","commit_stats":{"total_commits":10,"total_committers":1,"mean_commits":10.0,"dds":0.0,"last_synced_commit":"92e05e4a2b0f72b089391e72ceb16dc14486a73a"},"previous_names":["ronaldmannak/picomlxserver","picomlx/picomlxserver"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PicoMLX%2FPicoMLXServer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PicoMLX%2FPicoMLXServer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PicoMLX%2FPicoMLXServer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PicoMLX%2FPicoMLXServer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PicoMLX","download_url":"https://codeload.github.com/PicoMLX/PicoMLXServer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224179135,"owners_count":17269009,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","llama3","mlx","ollama","openai","openai-api"],"created_at":"2024-08-02T13:00:58.643Z","updated_at":"2024-11-11T21:31:34.638Z","avatar_url":"https://github.com/PicoMLX.png","language":"Swift","funding_links":[],"categories":["Swift","HarmonyOS","LLM \u0026 Inference"],"sub_categories":["Windows Manager"],"readme":"# Pico MLX Server\n\nPico MLX Server is the easiest way to get started with Apple's [MLX AI framework](https://github.com/ml-explore/mlx)\n\nPico MLX Server provides a GUI for [MLX Server](https://github.com/mustafaaljadery/mlxserver). MLX server provides an API for local MLX models conform the [OpenAI API](https://platform.openai.com/docs/api-reference/completions/create). This allows you to use most existing OpenAI chat clients with Pico MLX Server.\n\n![Menu extra screenshot](Images/menuExtra.png)\n\n## Highlights\n\n- Start and stop servers quickly via menu bar extra\n- Download MLX models from the [MLX community on HuggingFace](https://huggingface.co/mlx-community)\n- Install and setup Python environment, MLX, and MLX Server from within Pico MLX Server.\n- Run multiple servers on different ports\n- View logs of the servers in separte windows\n- Custom link to open your favorite chat client (defaults to [Pico AI Assistant](https://apps.apple.com/us/app/pico-ai-copilot/id1668205047))\n\n### Supported MLX models\n\nSee [MLX Community on HuggingFace](https://huggingface.co/mlx-community)\n\n## Getting Started\n\nTo install Pico MLX Server, build the source using Xcode, or download the [notarized executable directly from GitHub](https://github.com/ronaldmannak/PicoMLXServer/tags).\n\nTo set up Pico MLX Server, open the app and \n- Install and set up Python, pip, [MLX](https://github.com/ml-explore/mlx) and [MLX Server](https://github.com/mustafaaljadery/mlxserver) and optionally Conda manually\n- Use Pico MLX Server's automated Setup (`MLX -\u003e Setup...`)\n- Pico MLX Server uses [Conda](https://docs.conda.io/en/latest/) by default to create a virtual environment for the servers called pico. This (hopefully) will avoid any Python version issues. Conda can be disabled in `Settings`\n\n![Setup window screenshot](Images/setup.png)\n\n### Requirements\n- MacOS 14.0 (Sonoma) or later\n\n### Create a New Server\n\n![Menu extra screenshot](Images/serverManager.png)\n\n- Select `MLX -\u003e Servers -\u003e New Server...`\n- Press `Create` to create the default server `mlx-community/Nous-Hermes-2-Mistral-7B-DPO-4bit-MLX` on port `8080`\n- To use a different model, click on the `v` button or type in a model manually from the [MLX Community](https://huggingface.co/mlx-community) on HuggingFace (make sure to use the `mlx-community/` prefix)\n- Press the `View Logs` buttons to open a window with the server's real-time logs\n\n\n![Menu extra screenshot](Images/log.png)\n\n### Use Pico MLX Server with an AI client\n\n- Point any OpenAI API compatible AI assistant to `http://127.0.0.1:8080` (or any other port you used in Pico MLX Server). (Instructions for Pico AI Assistant coming soon)\n- Curl:\n\n  ```\n  curl -X GET 'http://127.0.0.1:8080/generate?prompt=write%20me%20a%20poem%20about%the%20ocean\u0026stream=true'\n  ```\n\n### API Endpoints\n\n- Pico MLX Server uses OpenAI's `POST /v1/completions` API. See for more information https://platform.openai.com/docs/api-reference/completions/create.\n\n## Known Issues\n\n- Pico MLX Server doesn't detect if a port is already in use (use `lsof -i:8080` in the terminal to find the PID of the running server)\n- There is a SwiftUI issue in `New Server` window and the `Servers` menu where the state of servers isn't updated\n\n## Roadmap\n\n- Switch from Python to [MLX Swift](https://github.com/ml-explore/mlx-swift)\n- Swift-based HTTP server\n\n## Related projects\n\n- [MLX](https://github.com/ml-explore/mlx)\n- [MLX Swift](https://github.com/ml-explore/mlx-swift)\n- [MLX Server](https://github.com/mustafaaljadery/mlxserver)\n- [MLX Community](https://huggingface.co/mlx-community)\n\nPico MLX Server is part of a bundle of open source Swift tools for AI engineers.\nLooking for a server-side Swift OpenAI proxy to protect your OpenAI keys? Check out [Swift OpenAI Proxy](https://github.com/ronaldmannak/SwiftOpenAIProxy).\n\n## Authors and Acknowledgements\n\nPico MLX Server, [Swift OpenAI Proxy](https://github.com/ronaldmannak/SwiftOpenAIProxy), and [Pico AI Assistant](https://apps.apple.com/us/app/pico-ai-copilot/id1668205047) were created by [Ronald Mannak](https://twitter.com/ronaldmannak) with help from [Ray Fernando](https://twitter.com/rayfernando1337/)\n\nMLX Server was created by [Mustafa Aljadery](https://www.maxaljadery.com/) \u0026 [Siddharth Sharma](https://stanford.edu/~sidshr/)\n\nCode used from [MLX Swift Chat](https://github.com/PreternaturalAI/mlx-swift-chat) and [Swift Chat](https://github.com/huggingface/swift-chat)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FPicoMLX%2FPicoMLXServer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FPicoMLX%2FPicoMLXServer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FPicoMLX%2FPicoMLXServer/lists"}