{"id":17061336,"url":"https://github.com/filipw/strathweb-phi-engine","last_synced_at":"2025-04-12T18:24:06.526Z","repository":{"id":233536255,"uuid":"785638419","full_name":"filipw/strathweb-phi-engine","owner":"filipw","description":"A cross platform library with C#/Swift/Kotlin/Python bindings for running Phi inference","archived":false,"fork":false,"pushed_at":"2025-04-11T16:34:26.000Z","size":5737,"stargazers_count":20,"open_issues_count":2,"forks_count":1,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-11T17:43:17.865Z","etag":null,"topics":["ai","phi"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/filipw.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-04-12T09:52:33.000Z","updated_at":"2025-03-27T09:00:52.000Z","dependencies_parsed_at":"2024-04-16T16:25:42.781Z","dependency_job_id":"9cd2b32e-388b-4cce-aa21-d4aa48427673","html_url":"https://github.com/filipw/strathweb-phi-engine","commit_stats":null,"previous_names":["filipw/strathweb-swift-phi-engine","filipw/strathweb-phi-engine"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/filipw%2Fstrathweb-phi-engine","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/filipw%2Fstrathweb-phi-engine/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/filipw%2Fstrathweb-phi-engine/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/filipw%2Fstrathweb-phi-engine/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/filipw","download_url":"https://codeload.github.com/filipw/strathweb-phi-engine/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248611107,"owners_count":21133048,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","phi"],"created_at":"2024-10-14T10:46:48.808Z","updated_at":"2025-04-12T18:24:06.520Z","avatar_url":"https://github.com/filipw.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Strathweb Phi Engine\n\nA cross-platform library for running Microsoft's [Phi-3](https://azure.microsoft.com/en-us/blog/introducing-phi-3-redefining-whats-possible-with-slms/) locally using [candle](https://github.com/huggingface/candle) in GGUF and safe tensors format.\n\n## Supported platforms\n\n|                        | **Swift**                                        | **.NET**                                                                                    | **Kotlin**                        | **Python**                                                  |\n|------------------------|--------------------------------------------------|---------------------------------------------------------------------------------------------|-----------------------------------|-------------------------------------------------------------|\n| **Package**            | Swift Package                                    | Nuget                                                                                       | N/A                               | N/A                                                         |\n| **Manual Integration** | Bindings + XCFramework Bindings + native library | Bindings + native library                                                                   | Bindings + native library         | Bindings + native library                                   |\n| **Platforms**          | macOS arm64\u003cbr/\u003eiOS                                  | Windows x64\u003cbr/\u003eWindows arm64 (not via Nuget)\u003cbr/\u003eLinux x64\u003cbr/\u003eLinux arm64 (not via Nuget)\u003cbr/\u003emacOS arm64 | Windows x64\u003cbr/\u003eLinux x64\u003cbr/\u003emacOS arm64 | Windows x64\u003cbr/\u003eWindows arm64\u003cbr/\u003eLinux x64\u003cbr/\u003eLinux arm64\u003cbr/\u003emacOS arm64 |\n\n![](content/strathweb-phi-engine.excalidraw.png)\n\n## Building instructions\n\n### Swift\n\nBuild the Swift Package (arm64 Mac required).\n\n```shell\n./build-swift.sh\n```\n\nThis builds:\n - the Swift Package under `packages/swift/Strathweb.Phi.Engine`\n - XCFramework under `artifacts/swift/strathweb_phi_engine_framework.xcframework`\n\nNow open `samples/io/phi.engine.sample/phi.engine.sample.xcodeproj` and build the SwiftUI app (iOS), or go to `samples/swift` and run `./run.sh` (macOS) to launch the Swift console app.\n\n### C#\n\nInstall UniFFI C# bindings generator\n\n```shell\ncargo install uniffi-bindgen-cs --git https://github.com/NordSecurity/uniffi-bindgen-cs --tag v0.8.0+v0.25.0\n```\n\nBuild the Nuget package for your platform:\n\n```shell\n./build-dotnet.sh\n```\n\nor (on Windows)\n\n```cmd\nbuild-dotnet.bat\n ```\n\nor\n\n```shell\ncargo build --release --manifest-path strathweb-phi-engine/Cargo.toml\ndotnet build packages/csharp -c Release\ndotnet pack packages/csharp -c Release -o artifacts/csharp\n```\n\nNuget package will be in `artifacts/csharp/Strathweb.Phi.Engine.0.1.0.nupkg`.\n(Optional) Run the sample console app:\n\n```shell\ncd samples/csharp/console\ndotnet run -c Release\n```\n\n### Kotlin\n\nRun the sample console app:\n\n```shell\ncd samples/kotlin\n./run.sh\n```\n\n### Python\n\nRun the sample console app:\n\n```shell\ncd samples/python/console\n./run.sh # or run.bat on Windows\n```\n\nor use the Jupyter Notebooks\n\n```shell\ncd samples/python/jupyter\n./init.sh # or init.bat on Windows\n```\n\nNow open the Notebook and run the cells.\n\n## Compatibility notes\n\n### .NET\n\n✅ Tested on Windows arm64\n\n✅ Tested on Windows x64\n\n✅ Tested on Linux arm64\n\n✅ Tested on Linux x64\n\n✅ Tested on macOS arm64.\n\n### Swift\n\n✅ Tested on macOS arm64.\n\n✅ Tested on iPad Air M1 8GB RAM\n\n✅ Should work on 6GB RAM iPhones too\n\n❌ Will not work on 4GB RAM iPhones\n\nHowever, for 4GB RAM iPhones, it's possible to use the (very) low fidelity Q2_K quantized model. Such model is not included in the official Phi-3 release, but I tested [this one from HuggingFace](https://huggingface.co/SanctumAI/Phi-3-mini-4k-instruct-GGUF) on an iPhone 12 mini successfully.\n\n### Kotlin\n\n✅ Tested on macOS arm64.\n\n### Python\n\n✅ Tested on Windows arm64\n\n✅ Tested on macOS arm64.\n\n## GPU Support\n\nCurrently the library supports Metal on MacOS. On other platforms only CPU is supported.\n\n## AutoGen\n\nThe repository also contains a C# integration library for [AutoGen](https://github.com/microsoft/autogen/tree/dotnet/dotnet), called `Strathweb.Phi.Engine.AutoGen`. There is an example in the `samples/csharp/autogen` folder. It allows creating a local Phi-3 agent and integrating it into the other typical AutoGen workflows.\n\nThe NuGet package is built when running `./build-dotnet.sh` (`build-dotnet.bat` on Windows).\n\n## Blog posts\n\n📝 an initial [announcement post](https://strathweb.com/2024/07/announcing-strathweb-phi-engine-a-cross-platform-library-for-running-phi-3-anywhere/)\n\n📝 safe tensors [announcement post](https://www.strathweb.com/2024/11/strathweb-phi-engine-now-with-safe-tensors-support/)\n\n📝 [AutoGen library](https://www.strathweb.com/2024/09/using-local-phi-3-models-in-autogen-with-strathweb-phi-engine/)\n\n📝 [.NET examples and Microso.Extensions.AI support](https://www.strathweb.com/2024/12/running-phi-inference-in-net-applications-with-strathweb-phi-engine/)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffilipw%2Fstrathweb-phi-engine","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffilipw%2Fstrathweb-phi-engine","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffilipw%2Fstrathweb-phi-engine/lists"}