{"id":13747346,"url":"https://github.com/LendingHome/pipe_operator","last_synced_at":"2025-05-09T08:32:28.457Z","repository":{"id":56888127,"uuid":"161083550","full_name":"LendingHome/pipe_operator","owner":"LendingHome","description":"Elixir/Unix style pipe operations in Ruby - PROOF OF CONCEPT","archived":true,"fork":false,"pushed_at":"2019-01-10T18:31:14.000Z","size":60,"stargazers_count":144,"open_issues_count":0,"forks_count":4,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-05-03T09:18:31.401Z","etag":null,"topics":["elixir","pipe-operator","proposal","ruby","unix"],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/LendingHome.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-12-09T21:34:46.000Z","updated_at":"2024-10-16T10:04:02.000Z","dependencies_parsed_at":"2022-08-21T00:20:48.314Z","dependency_job_id":null,"html_url":"https://github.com/LendingHome/pipe_operator","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LendingHome%2Fpipe_operator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LendingHome%2Fpipe_operator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LendingHome%2Fpipe_operator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LendingHome%2Fpipe_operator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/LendingHome","download_url":"https://codeload.github.com/LendingHome/pipe_operator/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252166990,"owners_count":21705018,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elixir","pipe-operator","proposal","ruby","unix"],"created_at":"2024-08-03T06:01:25.704Z","updated_at":"2025-05-09T08:32:28.188Z","avatar_url":"https://github.com/LendingHome.png","language":"Ruby","readme":"# ![LendingHome](https://avatars0.githubusercontent.com/u/5448482?s=24\u0026v=4) pipe_operator\n\n\u003e Elixir/Unix style pipe operations in Ruby - **PROOF OF CONCEPT**\n\n```ruby\n\"https://api.github.com/repos/ruby/ruby\".pipe do\n  URI.parse\n  Net::HTTP.get\n  JSON.parse.fetch(\"stargazers_count\")\n  yield_self { |n| \"Ruby has #{n} stars\" }\n  Kernel.puts\nend\n#=\u003e Ruby has 15120 stars\n```\n\n```ruby\n-9.pipe { abs; Math.sqrt; to_i } #=\u003e 3\n\n# Method chaining is supported:\n-9.pipe { abs; Math.sqrt.to_i.to_s } #=\u003e \"3\"\n```\n\n```ruby\nsqrt = Math.pipe.sqrt #=\u003e #\u003cPipeOperator::Closure:0x00007fc1172ed558@pipe_operator/closure.rb:18\u003e\nsqrt.call(9)          #=\u003e 3.0\nsqrt.call(64)         #=\u003e 8.0\n\n[9, 64].map(\u0026Math.pipe.sqrt)           #=\u003e [3.0, 8.0]\n[9, 64].map(\u0026Math.pipe.sqrt.to_i.to_s) #=\u003e [\"3\", \"8\"]\n```\n\n## Why?\n\nThere's been some recent activity related to `Method` and `Proc` composition in Ruby:\n\n* [#6284 - Add composition for procs](https://bugs.ruby-lang.org/issues/6284)\n* [#13581 - Syntax sugar for method reference](https://bugs.ruby-lang.org/issues/13581)\n* [#12125 - Shorthand operator for Object#method](https://bugs.ruby-lang.org/issues/12125)\n\nThis gem was created to **propose an alternative syntax** for this kind of behavior.\n\n## Matz on Ruby\n\nSource: [ruby-lang.org/en/about](https://www.ruby-lang.org/en/about)\n\nRuby is a language of careful **balance of both functional and imperative programming**.\n\nMatz has often said that he is **trying to make Ruby natural, not simple**, in a way that mirrors life.\n \nBuilding on this, he adds: Ruby is **simple in appearance, but is very complex inside**, just like our human body.\n\n## Concept\n\nThe general idea is to **pass the result of one expression as an argument to another expression** - similar to [Unix pipelines](https://en.wikipedia.org/wiki/Pipeline_(Unix)):\n\n```ruby\necho \"testing\" | sed \"s/ing//\" | rev\n#=\u003e tset\n```\n\nThe [Elixir pipe operator documentation](https://elixirschool.com/en/lessons/basics/pipe-operator/) has some other examples but basically it allows expressions like:\n\n```ruby\nJSON.parse(Net::HTTP.get(URI.parse(url)))\n```\n\nTo be **inverted** and rewritten as **left to right** or **top to bottom** which is more **natural to read** in English:\n\n```ruby\n# left to right\nurl.pipe { URI.parse; Net::HTTP.get; JSON.parse }\n\n# or top to bottom for clarity\nurl.pipe do\n  URI.parse\n  Net::HTTP.get\n  JSON.parse\nend\n```\n\nThe differences become a bit **clearer when other arguments are involved**:\n\n```ruby\nloans = Loan.preapproved.submitted(Date.current).where(broker: Current.user)\ndata = loans.map { |loan| LoanPresenter.new(loan).as_json }\njson = JSON.pretty_generate(data, allow_nan: false)\n```\n\nUsing pipes **removes the verbosity of maps and temporary variables**:\n\n```ruby\njson = Loan.pipe do\n  preapproved\n  submitted(Date.current)\n  where(broker: Current.user)\n  map(\u0026LoanPresenter.new.as_json)\n  JSON.pretty_generate(allow_nan: false)\nend\n```\n\nWhile the ability to perform a job correctly and efficiently is certainly important - the **true beauty of a program lies in its clarity and conciseness**:\n\n```ruby\n\"https://api.github.com/repos/ruby/ruby\".pipe do\n  URI.parse\n  Net::HTTP.get\n  JSON.parse.fetch(\"stargazers_count\")\n  yield_self { |n| \"Ruby has #{n} stars\" }\n  Kernel.puts\nend\n#=\u003e Ruby has 15115 stars\n```\n\nThere's nothing really special here - it's just a **block of expressions like any other Ruby DSL** and pipe operations have been [around for decades](https://en.wikipedia.org/wiki/Pipeline_(Unix))!\n\n```ruby\nRuby.is.so(elegant, \u0026:expressive).that(you can) do\n  pretty_much ANYTHING if it.compiles!\nend\n```\n\nThis concept of **pipe operations could be a great fit** like it has been for many other languages:\n\n* [Caml composition operators](http://caml.inria.fr/pub/docs/manual-ocaml/libref/Pervasives.html#1_Compositionoperators)\n* [Closure threading macros](https://clojure.org/guides/threading_macros)\n* [Elixir pipe operator](https://elixirschool.com/en/lessons/basics/pipe-operator/)\n* [Elm operators](https://elm-lang.org/docs/syntax#operators)\n* [F# function composition and pipelining](https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/functions/index#function-composition-and-pipelining)\n* [Hack pipe operator](https://docs.hhvm.com/hack/operators/pipe-operator)\n* [Haskell pipes](http://hackage.haskell.org/package/pipes-4.3.9/docs/Pipes-Tutorial.html)\n* [JavaScript pipeline operator proposals](https://github.com/tc39/proposal-pipeline-operator/wiki)\n* [LiveScript piping](http://livescript.net/#piping)\n* [Unix pipelines](https://en.wikipedia.org/wiki/Pipeline_(Unix))\n\n## Usage\n\n**WARNING - EXPERIMENTAL PROOF OF CONCEPT**\n\nThis has only been **tested in isolation with RSpec and Ruby 2.5.3**!\n\n```ruby\n# First `gem install pipe_operator`\nrequire \"pipe_operator\"\n```\n\n## Implementation\n\nThe [PipeOperator](https://github.com/lendinghome/pipe_operator/blob/master/lib/pipe_operator.rb) module has a method named `__pipe__` which is aliased as `pipe` for convenience:\n\n```ruby\nmodule PipeOperator\n  def __pipe__(*args, \u0026block)\n    Pipe.new(self, *args, \u0026block)\n  end\nend\n\nBasicObject.send(:include, PipeOperator)\nKernel.alias_method(:pipe, :__pipe__)\n```\n\nWhen no arguments are passed to `__pipe__` then a [PipeOperator::Pipe](https://github.com/lendinghome/pipe_operator/blob/master/lib/pipe_operator/pipe.rb) object is returned:\n\n```ruby\nMath.pipe #=\u003e #\u003cPipeOperator::Pipe:Math\u003e\n```\n\nAny methods invoked on this object returns a [PipeOperator::Closure](https://github.com/lendinghome/pipe_operator/blob/master/lib/pipe_operator/closure.rb) which **calls the method on the object later**:\n\n```ruby\nsqrt = Math.pipe.sqrt       #=\u003e #\u003cPipeOperator::Closure:0x00007fc1172ed558@pipe_operator/closure.rb:18\u003e\nsqrt.call(16)               #=\u003e 4.0\n\nmissing = Math.pipe.missing #=\u003e #\u003cPipeOperator::Closure:0x00007fc11726f0e0@pipe_operator/closure.rb:18\u003e\nmissing.call                #=\u003e NoMethodError: undefined method 'missing' for Math:Module\n\nMath.method(:missing)       #=\u003e NameError: undefined method 'missing' for class '#\u003cClass:Math\u003e'\n```\n\nWhen `__pipe__` is called **with arguments but without a block** then it behaves similar to `__send__`:\n\n```ruby\nsqrt = Math.pipe(:sqrt) #=\u003e #\u003cPipeOperator::Closure:0x00007fe52e0cdf80@pipe_operator/closure.rb:18\u003e\nsqrt.call(16)           #=\u003e 4.0\n\nsqrt = Math.pipe(:sqrt, 16) #=\u003e #\u003cPipeOperator::Closure:0x00007fe52fa18fd0@pipe_operator/closure.rb:18\u003e\nsqrt.call                   #=\u003e 4.0\nsqrt.call(16)               #=\u003e ArgumentError: wrong number of arguments (given 2, expected 1)\n```\n\nThese [PipeOperator::Closure](https://github.com/lendinghome/pipe_operator/blob/master/lib/pipe_operator/closure.rb) objects can be [bound as block arguments](https://github.com/lendinghome/pipe_operator/blob/master/lib/pipe_operator/proxy.rb#L10-L13) just like any other [Proc](https://ruby-doc.org/core-2.5.3/Proc.html):\n\n```ruby\n[16, 256].map(\u0026Math.pipe.sqrt) #=\u003e [4.0, 16.0]\n```\n\nSimple **closure composition is supported** via [method chaining](https://github.com/lendinghome/pipe_operator/blob/master/lib/pipe_operator/closure.rb#L56):\n\n```ruby\n[16, 256].map(\u0026Math.pipe.sqrt.to_i.to_s) #=\u003e [\"4\", \"16\"]\n```\n\nThe **block** form of `__pipe__` behaves **similar to instance_exec** but can also [call methods on other objects](https://github.com/lendinghome/pipe_operator/blob/master/lib/pipe_operator/pipe.rb#L81):\n\n```ruby\n\"abc\".pipe { reverse }        #=\u003e \"cba\"\n\"abc\".pipe { reverse.upcase } #=\u003e \"CBA\"\n\n\"abc\".pipe { Marshal.dump }                  #=\u003e \"\\x04\\bI\\\"\\babc\\x06:\\x06ET\"\n\"abc\".pipe { Marshal.dump; Base64.encode64 } #=\u003e \"BAhJIghhYmMGOgZFVA==\\n\"\n```\n\nOutside the context of a `__pipe__` block things behave like normal:\n\n```ruby\nMath.sqrt     #=\u003e ArgumentError: wrong number of arguments (given 0, expected 1)\nMath.sqrt(16) #=\u003e 4.0\n```\n\nBut within a `__pipe__` block the `Math.sqrt` expression returns a [PipeOperator::Closure](https://github.com/lendinghome/pipe_operator/blob/master/lib/pipe_operator/closure.rb) instead:\n\n```ruby\n16.pipe { Math.sqrt }     #=\u003e 4.0\n16.pipe { Math.sqrt(16) } #=\u003e ArgumentError: wrong number of arguments (given 2, expected 1)\n```\n\nThe **piped object is passed as the first argument by default** but can be customized by specifying `self`:\n\n```ruby\nclass String\n  def self.join(*args, with: \"\")\n    args.map(\u0026:to_s).join(with)\n  end\nend\n\n\"test\".pipe { String.join(\"123\", with: \"-\") }       #=\u003e \"test-123\"\n\n\"test\".pipe { String.join(\"123\", self, with: \"-\") } #=\u003e \"123-test\"\n```\n\nInstance methods like `reverse` below [do not receive the piped object](https://github.com/lendinghome/pipe_operator/blob/master/lib/pipe_operator/pipe.rb#L79) as [an argument](https://github.com/lendinghome/pipe_operator/blob/master/lib/pipe_operator/closure.rb#L47) since it's available as `self`:\n\n```ruby\nBase64.encode64(Marshal.dump(\"abc\").reverse)          #=\u003e \"VEUGOgZjYmEIIkkIBA==\\n\"\n\n\"abc\".pipe { Marshal.dump; reverse; Base64.encode64 } #=\u003e \"VEUGOgZjYmEIIkkIBA==\\n\"\n\n\"abc\".pipe { Marshal.dump.reverse; Base64.encode64 }  #=\u003e \"VEUGOgZjYmEIIkkIBA==\\n\"\n```\n\nPipes also support **multi-line blocks for clarity**:\n\n```ruby\n\"abc\".pipe do\n  Marshal.dump.reverse\n  Base64.encode64\nend\n```\n\nThe closures created by these **pipe expressions are evaluated via reduce**:\n\n```ruby\npipeline = [\n  -\u003e object { Marshal.dump(object) },\n  -\u003e object { object.reverse },\n  -\u003e object { Base64.encode64(object) },\n]\n\npipeline.reduce(\"abc\") do |object, pipe|\n  pipe.call(object)\nend\n```\n\n[Intercepting methods](https://github.com/lendinghome/pipe_operator/blob/master/lib/pipe_operator/proxy.rb#L19-L25) within pipes requires [prepending](https://github.com/lendinghome/pipe_operator/blob/master/lib/pipe_operator/pipe.rb#L38) a [PipeOperator::Proxy](https://github.com/lendinghome/pipe_operator/blob/master/lib/pipe_operator/proxy.rb) module infront of `::Object` and all [nested constants](https://github.com/lendinghome/pipe_operator/blob/master/lib/pipe_operator/proxy_resolver.rb#L46):\n\n```ruby\ndefine_method(method) do |*args, \u0026block|\n  if Pipe.open\n    Pipe.new(self).__send__(method, *args, \u0026block)\n  else\n    super(*args, \u0026block)\n  end\nend\n```\n\nThese **proxy modules are prepended everywhere**!\n\nIt's certainly something that **could be way more efficient as a core part of Ruby**.\n\nMaybe somewhere **lower level where methods are dispatched**? Possibly somewhere in this [vm_eval.c switch](https://github.com/ruby/ruby/blob/trunk/vm_eval.c#L111)?\n\n```c\nagain:\n  switch (cc-\u003eme-\u003edef-\u003etype) {\n    case VM_METHOD_TYPE_ISEQ\n    case VM_METHOD_TYPE_NOTIMPLEMENTED\n    case VM_METHOD_TYPE_CFUNC\n    case VM_METHOD_TYPE_ATTRSET\n    case VM_METHOD_TYPE_IVAR\n    case VM_METHOD_TYPE_BMETHOD\n    case VM_METHOD_TYPE_ZSUPER\n    case VM_METHOD_TYPE_REFINED\n    case VM_METHOD_TYPE_ALIAS\n    case VM_METHOD_TYPE_MISSING\n    case VM_METHOD_TYPE_OPTIMIZED\n    case OPTIMIZED_METHOD_TYPE_SEND\n    case OPTIMIZED_METHOD_TYPE_CALL\n    case VM_METHOD_TYPE_UNDEF\n  }\n```\n\nThen we'd **only need Ruby C API ports** for [PipeOperator::Pipe](https://github.com/LendingHome/pipe_operator/blob/master/lib/pipe_operator/pipe.rb) and [PipeOperator::Closure](https://github.com/LendingHome/pipe_operator/blob/master/lib/pipe_operator/closure.rb)!\n\nAll other objects in this proof of concept are related to **method interception** and would no longer be necessary.\n\n## Bugs\n\nThis test case doesn't work yet - seems like the [object is not proxied](https://github.com/lendinghome/pipe_operator/blob/master/lib/pipe_operator/pipe.rb#L39) for some reason:\n\n```ruby\nclass Markdown\n  def format(string)\n    string.upcase\n  end\nend\n\n\"test\".pipe(Markdown.new, \u0026:format) # expected \"TEST\"\n#=\u003e ArgumentError: wrong number of arguments (given 0, expected 1)\n```\n\n## Caveats\n\n* `PIPE_OPERATOR_AUTOLOAD`\n    * Constants flagged for autoload are NOT proxied by default (for performance)\n    * Set `ENV[\"PIPE_OPERATOR_AUTOLOAD\"] = 1` to enable this behavior\n* `PIPE_OPERATOR_FROZEN`\n    * Objects flagged as frozen are NOT proxied by default\n    * Set `ENV[\"PIPE_OPERATOR_FROZEN\"] = 1` to enable this behavior (via [Fiddle](http://ruby-doc.org/stdlib-2.5.3/libdoc/fiddle/rdoc/Fiddle.html))\n* `PIPE_OPERATOR_REBIND`\n    * `Object` and its recursively nested `constants` are only proxied ONCE by default (for performance)\n    * Constants defined after `__pipe__` is called for the first time are NOT proxied\n    * Set `ENV[\"PIPE_OPERATOR_REBIND\"] = 1` to enable this behavior\n* `PIPE_OPERATOR_RESERVED`\n    * The following methods are reserved on `PipeOperator::Closure` objects:\n        * `==`\n        * `[]`\n        * `__chain__`\n        * `__send__`\n        * `__shift__`\n        * `call`\n        * `class`\n        * `kind_of?`\n    * The following methods are reserved on `PipeOperator::Pipe` objects:\n        * `!`\n        * `!=`\n        * `==`\n        * `__call__`\n        * `__id__`\n        * `__pop__`\n        * `__push__`\n        * `__send__`\n        * `instance_exec`\n        * `method_missing`\n    * These methods can be piped via `send` as a workaround:\n        * `9.pipe { Math.sqrt.to_s.send(:[], 0) }`\n        * `example.pipe { send(:__call__, 1, 2, 3) }`\n        * `example.pipe { send(:instance_exec) { } }`\n\n## Testing\n\n```bash\nbundle exec rspec\n```\n\n## Inspiration\n\n* https://github.com/hopsoft/pipe_envy\n* https://github.com/akitaonrails/chainable_methods\n* https://github.com/kek/pipelining\n* https://github.com/k-motoyan/shelike-pipe\n* https://github.com/nwtgck/ruby_pipe_chain\n* https://github.com/teamsnap/pipe-ruby\n* https://github.com/danielpclark/elixirize\n* https://github.com/tiagopog/piped_ruby\n* https://github.com/jicksta/methodphitamine\n* https://github.com/jicksta/superators\n* https://github.com/baweaver/xf\n\n## Contributing\n\n* Fork the project.\n* Make your feature addition or bug fix.\n* Add tests for it. This is important so we don't break it in a future version unintentionally.\n* Commit, do not mess with the version or history.\n* Open a pull request. Bonus points for topic branches.\n\n## Authors\n\n* [Sean Huber](https://github.com/shuber)\n\n## License\n\n[MIT](https://github.com/lendinghome/pipe_operator/blob/master/LICENSE) - Copyright © 2018 LendingHome\n","funding_links":[],"categories":["Ruby"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FLendingHome%2Fpipe_operator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FLendingHome%2Fpipe_operator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FLendingHome%2Fpipe_operator/lists"}