{"id":34243859,"url":"https://github.com/mattt/swift-xet","last_synced_at":"2026-02-22T18:12:26.856Z","repository":{"id":332426831,"uuid":"1133709226","full_name":"mattt/swift-xet","owner":"mattt","description":"A Swift implementation of the Xet protocol","archived":false,"fork":false,"pushed_at":"2026-01-19T13:28:53.000Z","size":347,"stargazers_count":17,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-01-26T21:34:17.694Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Swift","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mattt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-01-13T18:02:11.000Z","updated_at":"2026-01-21T17:33:29.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/mattt/swift-xet","commit_stats":null,"previous_names":["mattt/swift-xet"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/mattt/swift-xet","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mattt%2Fswift-xet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mattt%2Fswift-xet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mattt%2Fswift-xet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mattt%2Fswift-xet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mattt","download_url":"https://codeload.github.com/mattt/swift-xet/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mattt%2Fswift-xet/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29721388,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-22T15:10:41.462Z","status":"ssl_error","status_checked_at":"2026-02-22T15:10:04.636Z","response_time":110,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["uniffi","xet"],"created_at":"2025-12-16T05:13:01.736Z","updated_at":"2026-02-22T18:12:26.822Z","avatar_url":"https://github.com/mattt.png","language":"Swift","funding_links":[],"categories":[],"sub_categories":[],"readme":"# swift-xet\n\n[Xet](https://huggingface.co/docs/hub/en/xet/index)\nis a storage system for large binary files that uses chunk-level deduplication.\nHugging Face uses Xet so users can download only the files they need \nwithout cloning the entire repository history.\n\n\u003cpicture\u003e\n  \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"Assets/xet-speed-dark.gif\"\u003e\n  \u003csource media=\"(prefers-color-scheme: light)\" srcset=\"Assets/xet-speed.gif\"\u003e\n  \u003cimg alt=\"XET vs LFS\"\u003e\n\u003c/picture\u003e\n\nThis project provides Swift bindings to\nthe [xet-core](https://github.com/huggingface/xet-core) Rust crate\nusing [UniFFI](https://mozilla.github.io/uniffi-rs/).\n\n\u003e [!WARNING]  \n\u003e This project is under active development, and not ready for production use.\n\n\n## Requirements\n\n- Swift 6.0+ / Xcode 16+\n- iOS 13+ / macOS 10.15+\n\n## Installation\n\n### Swift Package Manager\n\nAdd the following dependency to your `Package.swift` file:\n\n```swift\ndependencies: [\n    .package(url: \"https://github.com/mattt/swift-xet.git\", branch: \"main\")\n],\ntargets: [\n    .target(\n        name: \"YourTarget\",\n        dependencies: [\n            .product(name: \"Xet\", package: \"swift-xet\")\n        ]\n    )\n]\n```\n\n### Xcode\n\n1. In Xcode, select **File** → **Add Package Dependencies...**\n2. Enter the repository URL: `https://github.com/mattt/swift-xet.git`\n3. Select the version you want to use\n4. Add the `Xet` library to your target\n\n## Usage\n\n### Basic Example\n\n```swift\nimport Xet\n\nlet xet = try XetClient()\n\nguard let fileInfo = try xet.getFileInfo(\n    repo: \"Qwen/Qwen3-0.6B\",\n    path: \"tokenizer.json\",\n    revision: \"main\"\n) else {\n    fatalError(\"Pointer file missing Xet metadata\")\n}\n\nlet jwt = try xet.getCasJwt(\n    repo: \"Qwen/Qwen3-0.6B\",\n    revision: \"main\",\n    isUpload: false\n)\n\nlet downloads = try xet.downloadFiles(\n    fileInfos: [fileInfo],\n    destinationDir: FileManager.default.temporaryDirectory.path,\n    jwtInfo: jwt\n)\nprint(\"Downloaded blobs: \\(downloads)\")\n```\n\n### Authentication\n\nTo use authenticated requests, create a client with a token:\n\n```swift\nlet xet = try XetClient.withToken(token: \"your-hf-token\")\n```\n\n### High-performance downloads\n\n`XetClient` intentionally exposes only the primitives needed for high-performance CAS transfers:\n\n- `getFileInfo` reads pointer files to extract the Xet content hash + size.\n- `getCasJwt` obtains the short-lived CAS JWT required to talk to the storage backend.\n- `downloadFiles` performs the actual chunked, parallel download using the same data client that powers `hf_transfer`.\n\nIf the Hub response doesn’t advertise Xet metadata the call fails.\nThere is no transparent HTTP fallback,\nso you always know you’re getting the fast path.\n\n## Development\n\n### Prerequisites\n\n- **Rust toolchain**:\n  Install via [rustup](https://rustup.rs/)\n  ```bash\n  curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh\n  ```\n- **Swift toolchain**: \n  Download [Xcode](https://developer.apple.com/xcode/) or install with [swiftly](https://www.swift.org/install/)\n  ```bash\n  curl -O https://download.swift.org/swiftly/darwin/swiftly.pkg \u0026\u0026 \\\n  installer -pkg swiftly.pkg -target CurrentUserHomeDirectory \u0026\u0026 \\\n  ~/.swiftly/bin/swiftly init --quiet-shell-followup \u0026\u0026 \\\n  . \"${SWIFTLY_HOME_DIR:-$HOME/.swiftly}/env.sh\" \u0026\u0026 \\\n  hash -r\n  ```\n\n### FFI Interface Definition\n\nThe FFI interface is defined using UniFFI's \n[Interface Definition Language (UDL)](https://mozilla.github.io/uniffi-rs/0.27/udl_file_spec.html):\n\n- **Interface Definition**: `Rust/src/swift_xet_rust.udl` - Declares the types, functions, and errors exposed to Swift\n- **Implementation**: `Rust/src/lib.rs` - Implements the Rust side of the FFI bindings\n\n### Generating Bindings\n\nTo generate the Swift bindings from the Rust code:\n\n```bash\n./Scripts/generate-bindings.sh\n```\n\n\u003e [!NOTE]\n\u003e The first build will compile the Rust library,\n\u003e which may take a few minutes. \n\u003e Subsequent builds are incremental and much faster.\n\nThis script performs the following steps:\n\n1. **Builds the Rust library** (`libswift_xet_rust.a`) for the host platform\n2. **Runs the custom UniFFI generator** located in `Rust/uniffi-gen/` to process the UDL file\n3. **Generates Swift files** and places them in `Sources/Xet/`\n4. **Generates C FFI headers** and places them in `Sources/XetFFI/`\n\nThe custom UniFFI generator in `Rust/uniffi-gen/` is based on the `uniffi_bindgen` crate and tailored for this project's specific needs.\n\n### Swift Package Structure\n\nThe Swift package is defined in `Package.swift` and consists of two main targets:\n\n**1. `Xet` Target (Public Module)**\n\nThe public-facing Swift module containing the generated UniFFI bindings and convenience wrappers:\n\n```swift\n.target(\n    name: \"Xet\",\n    dependencies: [\"XetFFI\"],\n    path: \"Sources/Xet\"\n)\n```\n\nThis is the module that Swift projects import and use.\n\n**2. `XetFFI` Target (Internal System Library)**\n\nAn internal system library target that exposes the C FFI headers:\n\n```swift\n.systemLibrary(\n    name: \"XetFFI\",\n    path: \"Sources/XetFFI\",\n    pkgConfig: \"swift-xet-rust\"\n)\n```\n\nThis target bridges the generated Swift code to the underlying Rust library via C FFI. \nIt is an internal dependency and not directly used by consumers.\n\n### Building the Package\n\nOnce bindings are generated, build the Swift package:\n\n```bash\nswift build\n```\n\nRun tests:\n\n```bash\nswift test\n```\n\n### Adding New Xet Functionality\n\nThe current implementation provides a complete foundation ready for actual Xet API integration:\n\n1. **Expand the UDL Interface** (`Rust/src/swift_xet_rust.udl`):\n   - Add new types, functions, and errors following UniFFI conventions\n   - Ensure types are UniFFI-compatible (primitives, Vec, HashMap, Option, Result, custom structs/enums)\n\n2. **Implement in Rust** (`Rust/src/lib.rs`):\n   - Import and wrap `hub_client` crate functionality\n   - Add proper error handling with `Result\u003cT, XetError\u003e`\n   - Handle async operations (UniFFI supports async with proper setup)\n\n3. **Regenerate Bindings**:\n   ```bash\n   ./Scripts/generate-bindings.sh\n   ```\n\n4. **Add Swift Convenience APIs** (optional):\n   - Create idiomatic Swift wrappers in `Sources/Xet/` if needed\n   - Follow Swift naming conventions\n\n5. **Test**:\n   ```bash\n   swift build\n   swift test\n   ```\n\n## Troubleshooting\n\n### Build Errors\n\n**Rust not found or targets missing:**\n```bash\n# Install Rust\ncurl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh\nsource $HOME/.cargo/env\n\n# Add required targets for cross-compilation\nrustup target add aarch64-apple-ios\nrustup target add x86_64-apple-ios\nrustup target add aarch64-apple-darwin\nrustup target add x86_64-apple-darwin\n```\n\n**UniFFI generation errors:**\n- Ensure `uniffi` and `uniffi_build` are in `Rust/Cargo.toml` dependencies\n- Check that the UDL file syntax is correct\n- Verify the custom generator in `Rust/uniffi-gen/` builds successfully\n\n**Dependency resolution fails:**\n- Verify the xet-core repository path in `Rust/Cargo.toml` is correct\n- Check that the `hub_client` crate exists in the xet-core workspace\n- Try updating dependencies with `cargo update`\n\n### Runtime Errors\n\n**\"Cannot find type 'XetClient' in scope\":**\n\nBindings haven't been generated yet. Run:\n```bash\n./Scripts/generate-bindings.sh\n```\n\n**Linking errors:**\n- Ensure the Rust library was built for the correct architecture\n- Check that `Sources/XetFFI/module.modulemap` correctly points to headers\n- Verify library search paths in Swift package configuration\n\n### Testing Without Full Integration\n\nTo test the Rust side independently:\n\n```bash\ncd Rust\ncargo build\ncargo test\n```\n\n## License\n\nThis project is available under the MIT license.\nSee the LICENSE file for more info.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmattt%2Fswift-xet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmattt%2Fswift-xet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmattt%2Fswift-xet/lists"}