{"id":15056419,"url":"https://github.com/akash-akya/exile","last_synced_at":"2026-02-10T05:10:18.121Z","repository":{"id":43395369,"uuid":"262542542","full_name":"akash-akya/exile","owner":"akash-akya","description":"Alternative to ports for running external programs. It provides back-pressure, non-blocking io, and solves port related issues","archived":false,"fork":false,"pushed_at":"2025-06-08T11:34:22.000Z","size":1057,"stargazers_count":176,"open_issues_count":3,"forks_count":12,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-11-21T20:11:50.869Z","etag":null,"topics":["back-pressure","elixir","erlang","non-blocking","port","stream"],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/akash-akya.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-05-09T10:07:11.000Z","updated_at":"2025-11-12T14:12:44.000Z","dependencies_parsed_at":"2024-01-24T07:30:01.677Z","dependency_job_id":"8f6b8c41-8ae1-46d2-be20-48bae498db63","html_url":"https://github.com/akash-akya/exile","commit_stats":{"total_commits":99,"total_committers":2,"mean_commits":49.5,"dds":0.02020202020202022,"last_synced_commit":"4260f6ada3e6061757e066301cf025888facc4f1"},"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"purl":"pkg:github/akash-akya/exile","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akash-akya%2Fexile","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akash-akya%2Fexile/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akash-akya%2Fexile/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akash-akya%2Fexile/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/akash-akya","download_url":"https://codeload.github.com/akash-akya/exile/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akash-akya%2Fexile/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29290997,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-10T03:42:42.660Z","status":"ssl_error","status_checked_at":"2026-02-10T03:42:41.897Z","response_time":65,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["back-pressure","elixir","erlang","non-blocking","port","stream"],"created_at":"2024-09-24T21:51:08.661Z","updated_at":"2026-02-10T05:10:18.105Z","avatar_url":"https://github.com/akash-akya.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Exile\n\n[![CI](https://github.com/akash-akya/exile/actions/workflows/ci.yaml/badge.svg)](https://github.com/akash-akya/exile/actions/workflows/ci.yaml)\n[![Hex.pm](https://img.shields.io/hexpm/v/exile.svg)](https://hex.pm/packages/exile)\n[![docs](https://img.shields.io/badge/docs-hexpm-blue.svg)](https://hexdocs.pm/exile/)\n\nExile is a powerful Elixir library for running external programs with proper back-pressure and streaming capabilities. Think of it as Ports done right - giving you fine-grained control over input/output streams while preventing memory issues.\n\n## Key Features\n\n- Stream-based API with proper back-pressure\n- Memory-efficient handling of large outputs\n- Selective control over stdin/stdout/stderr\n- Clean process management (no zombies!)\n- No external dependencies or middleware programs\n- Built on battle-tested POSIX system calls\n\n## Quick Installation\n\n```elixir\ndef deps do\n  [\n    {:exile, \"~\u003e x.x.x\"} # Replace with latest version\n  ]\nend\n```\n\n\n## Quick Start Examples\n\n### Basic Command Execution\n```elixir\n# Simple output\nExile.stream!(~w(echo Hello World!))\n|\u003e Enum.into(\"\") # \"Hello World!\\n\"\n\n# With input to stdin\nExile.stream!(~w(cat), input: [\"Hello \", \"Back!\"])\n|\u003e Enum.into(\"\") # \"Hello Back!\"\n\n# Stream as input\ninput_stream = Stream.map(1..10, fn num -\u003e \"#{num}\\n\" end)\nExile.stream!(~w(cat), input: input_stream)\n|\u003e Enum.into(\"\")  # \"1\\n2\\n3\\n4\\n5\\n6\\n7\\n8\\n9\\n10\\n\"\n\n# With stderr\nExile.stream!([\"sh\", \"-c\", \"echo stdout \u0026\u0026 echo stderr \u003e\u00262\"], stderr: :consume)\n|\u003e Enum.to_list()  # [{:stdout, \"stdout\\n\"}, {:stderr, \"stderr\\n\"}]\n\n# With stderr redirected to stdout\nExile.stream!([\"sh\", \"-c\", \"echo stdout \u0026\u0026 echo stderr \u003e\u00262\"], stderr: :redirect_to_stdout)\n|\u003e Enum.to_list()  # [\"stdout\\nstderr\\n\"]\n```\n\n## Why Exile?\n\n### The Problem with Ports\nTraditional Elixir ports have several limitations:\n- No back-pressure → memory exhaustion with large outputs\n- Can't selectively close stdin\n- Potential zombie processes\n- Message-box flooding\n\n### Exile's Solution\nExile solves these by:\n- Using non-blocking async I/O\n- Implementing proper back-pressure\n- Providing fine-grained stream control\n- Handling process cleanup\n- Zero external dependencies\n\n## Performance Comparison\n\nMemory usage when reading from `/dev/random`:\n\n**With Port:**\n```elixir\n# Memory grows unbounded!\nPort.open({:spawn_executable, \"/bin/cat\"}, [{:args, [\"/dev/random\"]}, {:line, 10}, :binary, :use_stdio])\n```\n\n![Port memory consumption](./images/port.png)\n\n**With Exile:**\n```elixir\n# Stable memory usage\nExile.stream!(~w(cat /dev/random))\n|\u003e Enum.each(fn data -\u003e\n  IO.puts(IO.iodata_length(data))\nend)\n```\n\n![Exile memory consumption](./images/exile.png)\n\n## Advanced Examples\n\n### Working with Large Data Streams\n```elixir\n# Process large data with controlled chunk size\nExile.stream!(~w(cat huge_file.log), max_chunk_size: 65536)\n|\u003e Stream.map(\u0026process_batch/1)\n|\u003e Stream.run()\n\n# Handle infinite streams\ninfinite_stream = Stream.repeatedly(fn -\u003e \"data\\n\" end)\nExile.stream!(~w(cat), input: infinite_stream, ignore_epipe: true) # we need to ignore epipe since we are terminating the program before the input completes\n|\u003e Stream.take(3)\n|\u003e Enum.into(\"\") # \"data\\ndata\\n...\"\n```\n\n### Media Processing\n```elixir\n# Extract audio from video\nExile.stream!(~w(ffmpeg -i pipe:0 -f mp3 pipe:1), input: File.stream!(\"video.mkv\", [], 65_535))\n|\u003e Stream.into(File.stream!(\"audio.mp3\"))\n|\u003e Stream.run()\n\n# Convert video format with progress on stderr\nExile.stream!([\"ffmpeg\", \"-i\", \"input.mp4\", \"-c:v\", \"libx264\", \"pipe:1\"], stderr: :consume)\n|\u003e Enum.each(fn\n  {:stdout, data} -\u003e File.write!(\"output.mp4\", data, [:append])\n  {:stderr, data} -\u003e IO.write(:stderr, data) # Show progress\nend)\n```\n\n\n### Exit Status Handling\n\n```elixir\n# Using stream!/2 (raises on error non-zero exit status)\ntry do\n  Exile.stream!([\"sh\", \"-c\", \"exit 10\"])\n  |\u003e Enum.to_list()\nrescue\n  e in Exile.Stream.AbnormalExit -\u003e\n    IO.puts(\"Failed with status: #{e.exit_status}\")\nend # \"Failed with status: 10\"\n\n# Using stream/2 (returns exit status)\nExile.stream([\"sh\", \"-c\", \"exit 10\"])\n|\u003e Enum.to_list()  # [..., {:exit, {:status, 10}}]\n```\n\n### Wait for Input Close\n```elixir\n# base64 waits for input EOF before producing output\nExile.stream!(~w(base64), input: [\"abcdef\"])\n|\u003e Enum.into(\"\") # \"YWJjZGVm\\n\"\n```\n\n### Collectable as input (Callback)\n\n```elixir\n# Using input callback\nExile.stream!(~w(cat), input: fn sink -\u003e\n  Stream.repeatedly(fn -\u003e \"data\\n\" end)\n  |\u003e Stream.take(2)\n  |\u003e Stream.into(sink)\n  |\u003e Stream.run()\nend)\n|\u003e Enum.into(\"\") # \"data\\ndata\\n\"\n```\n\n## Alternative Libraries\n\nFor simpler use cases where NIF-based solutions aren't preferred, consider using [ExCmd](https://github.com/akash-akya/ex_cmd), which provides similar functionality using pure Elixir.\n\n## Support \u0026 Contribution\n\n- [Documentation](https://hexdocs.pm/exile/)\n- [GitHub Issues](https://github.com/akash-akya/exile/issues)\n\n## License\n\nCopyright (c) 2020-2025 Akash Hiremath.\nReleased under Apache License 2.0. See [LICENSE](LICENSE.md) for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fakash-akya%2Fexile","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fakash-akya%2Fexile","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fakash-akya%2Fexile/lists"}