{"id":48871900,"url":"https://github.com/kylehowells/demucs-mlx-swift","last_synced_at":"2026-04-15T22:30:43.219Z","repository":{"id":344316065,"uuid":"1177404087","full_name":"kylehowells/demucs-mlx-swift","owner":"kylehowells","description":null,"archived":false,"fork":false,"pushed_at":"2026-03-16T20:45:27.000Z","size":7763,"stargazers_count":4,"open_issues_count":1,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-03-29T16:07:31.913Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Swift","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kylehowells.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-10T01:54:43.000Z","updated_at":"2026-03-25T13:52:22.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/kylehowells/demucs-mlx-swift","commit_stats":null,"previous_names":["kylehowells/demucs-mlx-swift"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/kylehowells/demucs-mlx-swift","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kylehowells%2Fdemucs-mlx-swift","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kylehowells%2Fdemucs-mlx-swift/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kylehowells%2Fdemucs-mlx-swift/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kylehowells%2Fdemucs-mlx-swift/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kylehowells","download_url":"https://codeload.github.com/kylehowells/demucs-mlx-swift/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kylehowells%2Fdemucs-mlx-swift/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31863321,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-15T15:24:51.572Z","status":"ssl_error","status_checked_at":"2026-04-15T15:24:39.138Z","response_time":63,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-04-15T22:30:42.498Z","updated_at":"2026-04-15T22:30:43.214Z","avatar_url":"https://github.com/kylehowells.png","language":"Swift","funding_links":[],"categories":[],"sub_categories":[],"readme":"# demucs-mlx-swift\n\nSwift package for Demucs music source separation on Apple Silicon, using the [MLX](https://github.com/ml-explore/mlx-swift) GPU framework.\n\nSeparates audio into stems: drums, bass, other, vocals (and guitar + piano for the 6-stem model).\n\n## Features\n\n- `DemucsMLX` library target (importable from macOS/iOS apps)\n- `demucs-mlx-swift` CLI demo\n- All 8 pretrained Demucs models supported (HTDemucs, HDemucs, Demucs v3)\n- Chunked overlap-add inference with configurable segment/overlap/batch/shifts\n- Async separation API with progress reporting, ETA estimation, and cancellation\n- Multi-format output: WAV (16/24/32-bit), FLAC, ALAC, AAC\n- Two-stem mode (e.g. vocals + no_vocals)\n- Automatic model download from Hugging Face\n\n## Supported Models\n\nAll 8 pretrained Demucs models are supported. Benchmarks on a 3:19 track (M1 Pro, batch_size=8):\n\n| Model | Type | Stems | Time |\n|-------|------|-------|------|\n| `htdemucs` | HTDemucs (hybrid transformer) | 4 | 14.8s |\n| `htdemucs_ft` | HTDemucs (fine-tuned, bag of 4) | 4 | 95.8s |\n| `htdemucs_6s` | HTDemucs (6-stem) | 6 | 17.7s |\n| `hdemucs_mmi` | HDemucs (hybrid, bag of 4) | 4 | 44.5s |\n| `mdx` | Demucs v3 + HDemucs (bag of 4) | 4 | 140.7s |\n| `mdx_extra` | HTDemucs (bag of 4) | 4 | 163.7s |\n| `mdx_q` | Demucs v3 + HDemucs (bag of 4) | 4 | 120.6s |\n| `mdx_extra_q` | HTDemucs (bag of 4) | 4 | 153.3s |\n\nModel files are downloaded automatically from [Hugging Face](https://huggingface.co/iky1e/demucs-mlx) on first use.\n\n## Requirements\n\n- Swift 6.2+\n- macOS 14+ or iOS 17+\n- Xcode 15+\n- Apple Silicon\n\n## Installation (SPM)\n\n```swift\ndependencies: [\n    .package(url: \"https://github.com/kylehowells/demucs-mlx-swift\", branch: \"master\")\n]\n```\n\nThen add product dependency and import:\n\n```swift\nimport DemucsMLX\n```\n\n## Library Usage\n\n### Synchronous\n\n```swift\nimport DemucsMLX\n\nlet separator = try DemucsSeparator(modelName: \"htdemucs\")\n\nlet result = try separator.separate(fileAt: URL(fileURLWithPath: \"song.mp3\"))\n\nfor (source, audio) in result.stems {\n    let url = URL(fileURLWithPath: \"\\(source).wav\")\n    try AudioIO.writeAudio(audio, to: url, format: .wav(bitDepth: .int16))\n}\n```\n\n### Async with Progress and Cancellation\n\n```swift\nlet separator = try DemucsSeparator(modelName: \"htdemucs\")\nlet cancelToken = DemucsCancelToken()\n\nseparator.separate(\n    fileAt: inputURL,\n    cancelToken: cancelToken,\n    progress: { progress in\n        // Called on main queue\n        print(\"\\(Int(progress.fraction * 100))% - \\(progress.stage)\")\n        if let eta = progress.estimatedTimeRemaining {\n            print(\"ETA: \\(Int(eta))s\")\n        }\n    },\n    completion: { result in\n        // Called on main queue\n        switch result {\n        case .success(let separation):\n            for (source, audio) in separation.stems {\n                try? AudioIO.writeAudio(audio, to: outputDir.appendingPathComponent(\"\\(source).wav\"))\n            }\n        case .failure(let error):\n            print(\"Error: \\(error)\")\n        }\n    }\n)\n\n// To cancel:\ncancelToken.cancel()\n```\n\n### Custom Parameters\n\n```swift\nlet separator = try DemucsSeparator(\n    modelName: \"htdemucs_ft\",\n    parameters: DemucsSeparationParameters(\n        shifts: 2,         // shift augmentations (improves quality, multiplies time)\n        overlap: 0.25,     // overlap ratio between segments\n        split: true,       // chunked overlap-add inference\n        segmentSeconds: nil, // nil = use model default\n        batchSize: 1,      // chunks processed in parallel (1 = lowest memory)\n        seed: 42           // deterministic shifts\n    ),\n    modelDirectory: URL(fileURLWithPath: \"/path/to/models\")\n)\n```\n\n## CLI Demo\n\nBuild:\n\n```bash\nswift build -c release\n```\n\nRun:\n\n```bash\n.build/release/demucs-mlx-swift track.mp3 -o separated\n```\n\nOptions:\n\n| Option | Description |\n|--------|-------------|\n| `-n, --name` | Model name (default: `htdemucs`) |\n| `-o, --out` | Output directory (default: `separated`) |\n| `--model-dir` | Local model directory |\n| `--segment` | Segment length in seconds |\n| `--overlap` | Overlap ratio [0, 1) (default: 0.25) |\n| `--shifts` | Shift augmentations (default: 1) |\n| `--seed` | Random seed for deterministic shifts |\n| `-b, --batch-size` | Chunk batch size (default: 1) |\n| `--no-split` | Disable chunked overlap-add |\n| `--two-stems` | Output one stem + complement (e.g. `vocals`) |\n| `--async` | Use async API with progress reporting |\n| `--list-models` | List available models |\n| `--mp3` | Output as AAC in .m4a |\n| `--flac` | Output as FLAC lossless |\n| `--alac` | Output as Apple Lossless in .m4a |\n| `--int24` | Output 24-bit integer WAV |\n| `--float32` | Output 32-bit float WAV |\n\nExamples:\n\n```bash\n# Separate vocals only\n.build/release/demucs-mlx-swift song.mp3 --two-stems vocals -o out\n\n# Use fine-tuned model with FLAC output\n.build/release/demucs-mlx-swift song.mp3 -n htdemucs_ft --flac -o out\n\n# 6-stem separation (drums, bass, other, vocals, guitar, piano)\n.build/release/demucs-mlx-swift song.mp3 -n htdemucs_6s -o out\n\n# Async with progress bar\n.build/release/demucs-mlx-swift song.mp3 --async -o out\n```\n\n## Performance Tuning\n\nThe default `batchSize=1` provides the best balance of speed and memory for most models. Larger batch sizes increase memory usage and are often slower due to exceeding GPU cache limits.\n\nBenchmarks on a 3:13 track (M1 Pro):\n\n| Model | bs=1 | bs=4 | bs=8 |\n|-------|------|------|------|\n| `htdemucs` | 9.3s / 1.2GB | **8.5s** / 1.8GB | 10.3s / 2.1GB |\n| `htdemucs_6s` | **9.3s / 1.5GB** | 10.3s / 2.5GB | 45.1s / 2.1GB |\n| `hdemucs_mmi` | **9.2s / 2.1GB** | 19.6s / 3.2GB | 32.2s / 3.6GB |\n\nFor memory-constrained environments (iOS apps), also consider limiting MLX's memory cache:\n\n```swift\nimport MLX\n\n// Reduce MLX cache to prevent holding onto large GPU allocations after use.\n// Default is unlimited; 2 MB keeps peak RSS under ~1.3 GB with no speed penalty\n// for single-model inference. Larger caches (unlimited) can give ~25% speedup\n// on some workloads at the cost of ~5x memory retention.\nMemory.cacheLimit = 2 * 1024 * 1024\n```\n\n## Model Resolution\n\nModels are resolved in this order:\n\n1. Explicit `--model-dir` (or library `modelDirectory` parameter)\n2. `DEMUCS_MLX_SWIFT_MODEL_DIR` environment variable\n3. `~/.cache/demucs-mlx-swift-models/\u003cmodel\u003e`\n4. Local paths: `.scratch/models/\u003cmodel\u003e`, `Models/\u003cmodel\u003e`, `./\u003cmodel\u003e`\n5. Hugging Face download (default repo: `iky1e/demucs-mlx`)\n\nEnvironment overrides:\n\n- `DEMUCS_MLX_SWIFT_MODEL_REPO` — set to a Hub repo ID (`org/repo`) or URL.\n\n## Metal Shader Library (Required for MLX Inference)\n\nMLX inference requires `mlx.metallib`.\n\nAfter `swift build`, generate it with:\n\n```bash\n./scripts/build_mlx_metallib.sh release\n```\n\nIf you run an `xcodebuild`/DerivedData binary, place `mlx.metallib` next to that executable.\n\nIf you see `missing Metal Toolchain`, run:\n\n```bash\nxcodebuild -downloadComponent MetalToolchain\n```\n\n## Make Targets\n\n```bash\nmake build   # release + mlx.metallib\nmake debug   # debug + mlx.metallib\nmake test\nmake clean\n```\n\n## Exporting Models from PyTorch\n\nA script is included to export all 8 pretrained models directly from the original PyTorch Demucs package:\n\n```bash\npip install demucs safetensors numpy\npython scripts/export_from_pytorch.py --out-dir ~/.cache/demucs-mlx-swift-models\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkylehowells%2Fdemucs-mlx-swift","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkylehowells%2Fdemucs-mlx-swift","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkylehowells%2Fdemucs-mlx-swift/lists"}