{"id":31452676,"url":"https://github.com/managedcode/mlxsharp","last_synced_at":"2026-01-20T16:56:26.300Z","repository":{"id":317307539,"uuid":"1065811569","full_name":"managedcode/MLXSharp","owner":"managedcode","description":"MLXSharp is the first .NET wrapper around Apple MLX that plugs straight into Microsoft.Extensions.AI. It is designed and tested on macOS (Apple silicon) but ships prebuilt native binaries for both macOS and Linux so that NuGet consumers can run out-of-the-box.","archived":false,"fork":false,"pushed_at":"2025-10-07T10:18:35.000Z","size":126,"stargazers_count":7,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-20T09:53:08.716Z","etag":null,"topics":["ai","apple","dotnet","managedcode","microsoft-extensions-ai","mlx","native"],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/managedcode.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-28T13:31:18.000Z","updated_at":"2025-10-14T10:15:51.000Z","dependencies_parsed_at":"2025-09-30T05:48:26.797Z","dependency_job_id":"e9a98162-a55f-4c20-b3b4-73ac9f8eec6a","html_url":"https://github.com/managedcode/MLXSharp","commit_stats":null,"previous_names":["managedcode/mlxsharp"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/managedcode/MLXSharp","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/managedcode%2FMLXSharp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/managedcode%2FMLXSharp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/managedcode%2FMLXSharp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/managedcode%2FMLXSharp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/managedcode","download_url":"https://codeload.github.com/managedcode/MLXSharp/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/managedcode%2FMLXSharp/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":281862825,"owners_count":26574710,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-30T02:00:06.501Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","apple","dotnet","managedcode","microsoft-extensions-ai","mlx","native"],"created_at":"2025-10-01T07:43:26.018Z","updated_at":"2025-10-30T18:55:20.422Z","avatar_url":"https://github.com/managedcode.png","language":"C#","readme":"# MLXSharp\n\n[![NuGet](https://img.shields.io/nuget/v/ManagedCode.MLXSharp.svg)](https://www.nuget.org/packages/ManagedCode.MLXSharp)\n[![NuGet](https://img.shields.io/nuget/v/ManagedCode.MLXSharp.SemanticKernel.svg)](https://www.nuget.org/packages/ManagedCode.MLXSharp.SemanticKernel)\n\nMLXSharp is the first .NET wrapper around [Apple MLX](https://github.com/ml-explore/mlx) that plugs straight into [`Microsoft.Extensions.AI`](https://learn.microsoft.com/dotnet/ai/microsoft-extensions-ai). It is designed and tested on macOS (Apple silicon) but ships prebuilt native binaries for both macOS and Linux so that NuGet consumers can run out-of-the-box.\n\n## Highlights\n- .NET 9 / C# 13 preview friendly APIs that implement `IChatClient`, `IEmbeddingGenerator\u003cstring, Embedding\u003cfloat\u003e\u003e`, and image helpers.\n- Dependency Injection extensions (`AddMlx`) and Semantic Kernel integration (`AddMlxChatCompletion`).\n- Native runtime loader that understands `MLXSHARP_LIBRARY`, custom paths, and packaged runtimes.\n- CI pipeline that builds the managed code once, produces macOS and Linux native libraries in parallel, and packs everything into distributable NuGet packages.\n\n## Quick Start\nAdd the package and wire it into the service collection:\n\n```bash\ndotnet add package ManagedCode.MLXSharp\n```\n\n```csharp\nusing Microsoft.Extensions.DependencyInjection;\nusing MLXSharp;\n\nvar services = new ServiceCollection();\nservices.AddMlx(builder =\u003e\n{\n    builder.Configure(options =\u003e\n    {\n        options.ChatModelId = \"mlx-chat\";\n        options.EmbeddingModelId = \"mlx-embedding\";\n    });\n\n    builder.UseManagedBackend(new MlxManagedBackend());\n    // Switch to the native backend when libmlxsharp.{dylib|so} is available.\n    // builder.UseNativeBackend();\n});\n\nvar provider = services.BuildServiceProvider();\nvar chat = provider.GetRequiredService\u003cIChatClient\u003e();\nvar reply = await chat.GetResponseAsync(\"hello MLX\", CancellationToken.None);\n```\n\n### Semantic Kernel\n\n```bash\ndotnet add package ManagedCode.MLXSharp.SemanticKernel\n```\n\n```csharp\nusing Microsoft.SemanticKernel;\nusing MLXSharp.SemanticKernel;\n\nvar kernelBuilder = Kernel.CreateBuilder();\nkernelBuilder.AddMlxChatCompletion(b =\u003e b.UseManagedBackend(new MlxManagedBackend()));\nvar kernel = kernelBuilder.Build();\n\nvar history = new ChatHistory();\nhistory.AddUserMessage(\"Summarise MLX in one sentence\");\nvar response = await kernel\n    .GetRequiredService\u003cIChatCompletionService\u003e()\n    .GetChatMessageContentsAsync(history, new PromptExecutionSettings(), kernel, CancellationToken.None);\n```\n\n## Repository Layout\n```\nextern/mlx/                  # MLX sources (git submodule)\nnative/                      # CMake project that builds libmlxsharp\nsrc/MLXSharp/                # Managed MLXSharp library + runtime assets\nsrc/MLXSharp.SemanticKernel/ # Semantic Kernel integration\nsrc/MLXSharp.Tests/          # Integration tests\n```\n\n## Building the native wrapper locally\n1. Install .NET 9, CMake, and ensure Xcode command-line tools (macOS) or build-essential + OpenBLAS/LAPACK headers (Linux) are available.\n2. Sync submodules:\n   ```bash\n   git submodule update --init --recursive\n   ```\n3. Build for your platform:\n   ```bash\n   # macOS (Apple silicon)\n   cmake -S native -B native/build/macos -DCMAKE_BUILD_TYPE=Release\n   cmake --build native/build/macos\n   export MLXSHARP_LIBRARY=$(pwd)/native/build/macos/libmlxsharp.dylib\n\n   # Linux\n   cmake -S native -B native/build/linux -DCMAKE_BUILD_TYPE=Release\n   cmake --build native/build/linux\n   export MLXSHARP_LIBRARY=$(pwd)/native/build/linux/libmlxsharp.so\n   ```\n4. Run your application or `dotnet pack` with explicit paths:\n   ```bash\n   dotnet pack src/MLXSharp/MLXSharp.csproj \\\n       -p:MLXSharpMacNativeBinary=$PWD/native/build/macos/libmlxsharp.dylib \\\n       -p:MLXSharpMacMetallibBinary=$PWD/native/build/macos/extern/mlx/mlx/backend/metal/kernels/mlx.metallib \\\n       -p:MLXSharpLinuxNativeBinary=$PWD/native/build/linux/libmlxsharp.so\n   ```\n\nThe CMake project vendored from MLX builds MLX and the shim in one go. macOS builds enable Metal automatically; disable or tweak MLX options by passing flags such as `-DMLX_BUILD_METAL=OFF` or `-DMLX_BUILD_BLAS_FROM_SOURCE=ON`.\n\n## CI overview\n1. `dotnet-build` (Ubuntu): restores the solution and compiles managed projects.\n2. `native-assets` (Ubuntu): downloads the signed native binaries published with the latest MLXSharp release and uploads them as workflow artifacts.\n3. `package-test` (macOS): pulls down the staged native artifacts, copies them into `src/MLXSharp/runtimes/{rid}/native`, rebuilds, runs the integration tests, and produces NuGet packages.\n\n## Testing\nThe managed integration tests still piggy-back on `mlx_lm` until the native runner is feature-complete. Bring your own HuggingFace bundle (any MLX-compatible repo) and point `MLXSHARP_MODEL_PATH` to it before running:\n\n```bash\nexport MLXSHARP_HF_MODEL_ID=\u003cyour-mlx-model\u003e\nexport MLXSHARP_MODEL_PATH=$PWD/models/\u003cyour-mlx-model\u003e\nhuggingface-cli download \"$MLXSHARP_HF_MODEL_ID\" --local-dir \"$MLXSHARP_MODEL_PATH\"\npython -m pip install mlx-lm\ndotnet test\n```\n\n`MLXSHARP_HF_MODEL_ID` is picked up by the Python smoke test; omit it to fall back to `mlx-community/Qwen1.5-0.5B-Chat-4bit`.\n\nWhen running locally you can place prebuilt binaries under `libs/native-osx-arm64` (and/or `libs/native-libs`) and a corresponding model bundle under `model/`. The test harness auto-discovers these folders and configures `MLXSHARP_LIBRARY`, `MLXSHARP_MODEL_PATH`, and `MLXSHARP_TOKENIZER_PATH` so you can iterate completely offline.\n\nThe integration suite invokes `python -m mlx_lm.generate` with deterministic settings (temperature `0`, seed `42`) and asserts that the generated response for prompts like “Скільки буде 2+2?” contains the correct answer. Test output includes the raw generation transcript so you can verify the model behaviour directly from the CI logs.\n\n### Native pipeline (experimental)\n\nWork is in progress to move inference fully into the native MLX backend. The current build exposes new configuration knobs via `MlxClientOptions`:\n\n| Option | Description |\n| --- | --- |\n| `EnableNativeModelRunner` | Turns on the experimental native transformer pipeline. Still returns “not implemented” until the native side is completed. |\n| `NativeModelDirectory` | Directory containing `config.json`, `*.safetensors`, etc. |\n| `TokenizerPath` | Path to the HuggingFace `tokenizer.json` (loaded with `Microsoft.ML.Tokenizers`). |\n| `MaxGeneratedTokens`, `Temperature`, `TopP`, `TopK` | Generation parameters that will flow into the native pipeline. |\n\nWhen the C++ implementation catches up you’ll be able to set the environment variables below and exercise the path end-to-end:\n\n```bash\nexport MLXSHARP_TOKENIZER_PATH=$PWD/models/\u003cmodel-name\u003e/tokenizer.json\nexport MLXSHARP_MODEL_PATH=$PWD/models/\u003cmodel-name\u003e\n```\n\nUntil then, `EnableNativeModelRunner` should stay `false` to avoid runtime errors from the stub implementation.\n\n### MSBuild properties\n\n| Property | Purpose |\n| --- | --- |\n| `MLXSharpMacNativeBinary` | Path to `libmlxsharp.dylib` that gets packaged into the NuGet runtime folder. |\n| `MLXSharpMacMetallibBinary` | Path to the matching `mlx.metallib` that ships next to the dylib. |\n| `MLXSharpLinuxNativeBinary` | Path to the Linux shared object (`libmlxsharp.so`). |\n| `MLXSharpSkipMacNativeValidation` / `MLXSharpSkipLinuxNativeValidation` | Opt-out flags for validation logic when you intentionally omit platform binaries. |\n\n## Versioning \u0026 platform support\nThis initial release is focused on macOS developers who want MLX inside .NET applications. Linux binaries are produced to keep NuGet packages complete, and Windows support is not yet available.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmanagedcode%2Fmlxsharp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmanagedcode%2Fmlxsharp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmanagedcode%2Fmlxsharp/lists"}