{"id":34549474,"url":"https://github.com/rsyncosx/rsyncprocessstreaming","last_synced_at":"2026-05-24T23:01:20.770Z","repository":{"id":329146220,"uuid":"1118318833","full_name":"rsyncOSX/RsyncProcessStreaming","owner":"rsyncOSX","description":null,"archived":false,"fork":false,"pushed_at":"2026-05-02T15:57:51.000Z","size":126,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-02T17:28:09.802Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Swift","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rsyncOSX.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-17T15:20:53.000Z","updated_at":"2026-05-02T15:57:54.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/rsyncOSX/RsyncProcessStreaming","commit_stats":null,"previous_names":["rsyncosx/rsyncprocessstreaming"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/rsyncOSX/RsyncProcessStreaming","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rsyncOSX%2FRsyncProcessStreaming","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rsyncOSX%2FRsyncProcessStreaming/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rsyncOSX%2FRsyncProcessStreaming/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rsyncOSX%2FRsyncProcessStreaming/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rsyncOSX","download_url":"https://codeload.github.com/rsyncOSX/RsyncProcessStreaming/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rsyncOSX%2FRsyncProcessStreaming/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33453557,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-24T19:21:36.376Z","status":"ssl_error","status_checked_at":"2026-05-24T19:21:10.562Z","response_time":57,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-12-24T07:45:43.690Z","updated_at":"2026-05-24T23:01:20.757Z","avatar_url":"https://github.com/rsyncOSX.png","language":"Swift","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# RsyncProcessStreaming\nRsyncProcessStreaming is a Swift package for executing `rsync` while streaming stdout and stderr in real time. It keeps memory usage low, surfaces errors quickly, and mirrors the handler-oriented API used in RsyncUI.\n\n## Purpose\nProvide a robust, handler-driven interface to run `rsync` with live output streaming, error-aware processing, cancellation, and optional timeout control—ideal for responsive CLIs and UIs.\n\n## Core Capabilities\n- **Live line-by-line streaming**: Streams stdout incrementally and preserves partial lines until complete, enabling responsive UIs and progress indicators.\n- **Error-aware processing**: Captures stderr separately and lets clients enforce custom error detection per line before the process completes.\n- **Rolling accumulation**: Maintains an in-memory rolling buffer of all output without requiring full buffering before callbacks fire.\n- **Resource-safe lifecycle**: Starts, monitors, cancels, and cleans up `Process` instances without leaking pipe handlers.\n- **Configurable environment**: Supports custom `rsync` paths and environment variables to match user installations.\n\n## Installation\n### Swift Package Manager (remote)\nAdd the package to your `Package.swift` dependencies:\n\n```swift\ndependencies: [\n  .package(url: \"https://github.com/\u003cyour-org\u003e/RsyncProcessStreaming.git\", from: \"0.1.0\")\n]\n```\n\nThen add `RsyncProcessStreaming` to your target:\n\n```swift\ntargets: [\n  .target(\n    name: \"YourApp\",\n    dependencies: [\n      .product(name: \"RsyncProcessStreaming\", package: \"RsyncProcessStreaming\")\n    ]\n  )\n]\n```\n\n### Local checkout\nIf you have the repo locally, you can add it as a local package in Xcode or reference it via a relative path in a workspace.\n\n## Supported Platforms\n- macOS 14+\n\n## Quick Start\nHere's a minimal example to sync two directories with live progress:\n\n```swift\nimport RsyncProcessStreaming\n\n@MainActor\nfunc syncFolders() async throws {\n    let handlers = ProcessHandlers(\n        processTermination: { output, _ in\n            if let lines = output {\n                print(\"✓ Sync completed with \\(lines.count) output lines\")\n            }\n        },\n        fileHandler: { count in\n            print(\"📦 Processed \\(count) files...\")\n        },\n        rsyncPath: \"/usr/bin/rsync\",\n        checkLineForError: { line in\n            // Detect rsync errors in real time\n            if line.contains(\"rsync error:\") || line.contains(\"failed:\") {\n                throw RsyncProcessError.processFailed(exitCode: 1, errors: [line])\n            }\n        },\n        updateProcess: { process in\n            if let pid = process?.processIdentifier {\n                print(\"🚀 Process started with PID: \\(pid)\")\n            }\n        },\n        propagateError: { error in\n            print(\"❌ Error: \\(error.localizedDescription)\")\n        },\n        checkForErrorInRsyncOutput: true,\n        environment: nil\n    )\n    \n    let process = RsyncProcess(\n        arguments: [\"-av\", \"--progress\", \"~/source/\", \"~/destination/\"],\n        handlers: handlers,\n        useFileHandler: true,\n        timeout: 60\n    )\n    \n    try process.executeProcess()\n}\n\n// Usage\nTask { @MainActor in\n    try await syncFolders()\n}\n```\n\n**Key Points:**\n- Set `useFileHandler: true` to get per-line progress callbacks.\n- Use `checkLineForError` to abort on custom error patterns.\n- `timeout` ensures runaway processes terminate automatically.\n- All output is accumulated and delivered to `processTermination`.\n\n## Public API Highlights\n- **`RsyncProcess`** ([Sources/RsyncProcessStreaming/RsyncProcessStreaming.swift](Sources/RsyncProcessStreaming/RsyncProcessStreaming.swift))\n  - `init(arguments:hiddenID:handlers:useFileHandler:timeout:)` wires rsync arguments with a user-provided handler set and optional timeout.\n  - `executeProcess()` validates the executable, spawns the process, wires streaming handlers, and begins collection.\n  - `cancel()` terminates a running process, marking it as cancelled for downstream handling.\n  - State surfaces via `isRunning` and `isCancelled` to simplify UI binding.\n  - Convenience surfaces: `currentState`, `commandDescription`, `processIdentifier`, `terminationStatus`.\n\n- **`ProcessHandlers`** ([Sources/RsyncProcessStreaming/ProcessHandlers.swift](Sources/RsyncProcessStreaming/ProcessHandlers.swift))\n  - Callbacks for termination, per-file counting, process updates, error propagation, and per-line error checks.\n  - Configuration knobs for rsync path, rsync v3 compatibility, stderr checking, and environment.\n  - `withOutputCapture(...)` convenience builder for common setups.\n\n- **`StreamAccumulator`** (internal) ([Sources/RsyncProcessStreaming/Internal/StreamAccumulator.swift](Sources/RsyncProcessStreaming/Internal/StreamAccumulator.swift))\n  - Actor that splits incoming text into lines, preserves trailing partials, and counts lines for progress callbacks.\n  - Retains both stdout and stderr snapshots for post-run inspection.\n\n- **Logging utilities** ([Sources/RsyncProcessStreaming/Internal/PackageLogger.swift](Sources/RsyncProcessStreaming/Internal/PackageLogger.swift))\n  - `Logger.process` category with debug-only helpers for command and thread diagnostics.\n\n## Execution Flow\n1. **Setup**: Build `ProcessHandlers` to define callbacks and configure rsync path/version/environment.\n2. **Start**: Instantiate `RsyncProcess` and call `executeProcess()`; validation guards against missing executables.\n3. **Streaming**: `AsyncStream` readers feed stdout/stderr into `StreamAccumulator`, emitting lines to callbacks immediately.\n4. **Error detection**: Custom `checkLineForError` can abort early; stderr content is recorded for termination review.\n5. **Termination**: Final buffers flush, `processTermination` fires with accumulated stdout and optional hidden ID, and `updateProcess(nil)` clears state.\n6. **Cancellation**: `cancel()` stops the underlying process and propagates `processCancelled` to consumers.\n\n## Process Termination and Output Draining\n\nThe package employs a multi-stage termination strategy to ensure all process output is captured before cleanup:\n\n### Thread Architecture\nWhen the `Process` terminates, the `terminationHandler` is called on an arbitrary system thread. Rather than processing directly on this thread, the handler immediately dispatches to a dedicated background queue:\n\n```swift\nlet queue = DispatchQueue(label: \"com.rsync.process.termination\", qos: .userInitiated)\n```\n\nThis custom queue with `.userInitiated` QoS priority:\n- Provides a controlled, serial execution context for cleanup operations\n- Keeps blocking I/O operations off the main thread\n- Ensures prompt processing of termination without interfering with UI responsiveness\n- Makes debugging easier with a labeled queue visible in Instruments\n\n### Output Draining Sequence\nThe termination handler follows this precise sequence to guarantee complete output capture:\n\n1. **Brief delay (50ms)**: Allows OS-level pipe buffers to flush any remaining data after process termination\n   ```swift\n   Thread.sleep(forTimeInterval: 0.05)\n   ```\n\n2. **Synchronous pipe draining**: Exhaustively reads all remaining data from both stdout and stderr pipes\n   ```swift\n   let outputData = Self.drainPipe(outputPipe.fileHandleForReading)\n   let errorData = Self.drainPipe(errorPipe.fileHandleForReading)\n   ```\n   The `drainPipe` method loops until no data remains:\n   ```swift\n   while true {\n       let data = fileHandle.availableData\n       if data.isEmpty { break }\n       allData.append(data)\n   }\n   ```\n\n3. **Handler cleanup**: Only after draining completes are the readability handlers cleared to prevent concurrent access\n   ```swift\n   outputPipe.fileHandleForReading.readabilityHandler = nil\n   errorPipe.fileHandleForReading.readabilityHandler = nil\n   ```\n\n4. **MainActor transition**: All captured data is then passed to `@MainActor`-isolated methods for final processing\n   ```swift\n   Task { @MainActor in\n       await self.processFinalOutput(...)\n   }\n   ```\n\n5. **Final processing**: The accumulated output is processed, trailing partial lines are flushed, and termination callbacks fire\n\n6. **Deferred cleanup**: A `defer` block ensures resources (timers, process references) are released only after all termination logic completes\n\n### Why This Approach?\n- **No data loss**: The sleep + exhaustive draining pattern ensures even late-arriving pipe data is captured\n- **Thread safety**: Background draining prevents blocking the main thread, then safely transitions to MainActor for state updates\n- **Race prevention**: Clearing handlers only after draining prevents concurrent read attempts\n- **Predictable ordering**: Serial queue execution ensures cleanup steps happen in the correct sequence\n- **Resource safety**: Deferred cleanup guarantees proper resource release even if early returns or errors occur\n\n## Error Model\n- **`executableNotFound`**: rsync path fails executability check.\n- **`processFailed`**: Non-zero exit combined with collected stderr when `checkForErrorInRsyncOutput` is enabled.\n- **`processCancelled`**: User-requested cancellation path.\n- **`timeout`**: Process terminated after exceeding configured `timeout` interval.\n- Errors propagate through `propagateError` for centralized handling.\n\n## Configuration and Extensibility\n- **Rsync location**: Override via `rsyncPath` to support custom installs or sandboxed environments.\n- **Environment**: Supply `environment` to mirror user shells or inject required variables.\n- **Version flags**: `rsyncVersion3` allows downstream callers to tailor behavior for legacy rsync output patterns.\n- **File progress**: Enable `useFileHandler` to receive per-line counts for UI progress or telemetry.\n\n## Testing and Quality\n- Unit and integration tests live in [Tests/RsyncProcessStreamingTests](Tests/RsyncProcessStreamingTests) using Swift Testing to cover line splitting, termination callbacks, cancellation, and error detection hooks.\n- Code quality practices are tracked in [CODE_QUALITY.md](CODE_QUALITY.md) with concurrency, error handling, and testing notes.\n\n## Example Usage\n```swift\nimport RsyncProcessStreaming\n\nlet handlers = ProcessHandlers.withOutputCapture(\n    processTermination: { output, hiddenID in\n        print(\"Completed\", hiddenID ?? -1)\n        print(output ?? [])\n    },\n    fileHandler: { count in print(\"Lines processed: \\(count)\") },\n    rsyncPath: \"/usr/bin/rsync\",\n    checkLineForError: { line in\n        if line.contains(\"rsync error:\") {\n            throw RsyncProcessError.processFailed(exitCode: 1, errors: [line])\n        }\n    },\n    updateProcess: { _ in },\n    propagateError: { error in print(\"Error: \\(error)\") },\n    logger: { _, _ in },\n    checkForErrorInRsyncOutput: true,\n    rsyncVersion3: true,\n    environment: nil\n)\n\nlet process = RsyncProcess(arguments: [\"--version\"], handlers: handlers, useFileHandler: false, timeout: 5)\ntry process.executeProcess()\n```\n\n## When to Use This Package\n- Building Swift clients that need responsive, real-time rsync output (CLI or UI).\n- Integrating with RsyncUI-compatible handler signatures.\n- Minimizing memory footprint while still retaining full stdout/stderr history.\n- Implementing cancellable, error-aware rsync workflows with Swift concurrency.\n\n## Build \u0026 Test\nUse the provided VS Code tasks or `xcrun` directly:\n\n```bash\n# Build\nxcrun swift build\n\n# Run tests (verbose)\nxcrun swift test -v\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frsyncosx%2Frsyncprocessstreaming","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frsyncosx%2Frsyncprocessstreaming","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frsyncosx%2Frsyncprocessstreaming/lists"}