{"id":17956045,"url":"https://github.com/sleipnir/grpc-stream","last_synced_at":"2025-09-12T04:46:40.609Z","repository":{"id":242435754,"uuid":"809535476","full_name":"sleipnir/grpc-stream","owner":"sleipnir","description":"Elixir Stream basead gRPC Server","archived":false,"fork":false,"pushed_at":"2024-06-03T00:44:28.000Z","size":7,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-09T05:44:38.468Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sleipnir.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-03T00:28:48.000Z","updated_at":"2024-06-03T00:44:31.000Z","dependencies_parsed_at":"2024-06-03T01:51:03.184Z","dependency_job_id":null,"html_url":"https://github.com/sleipnir/grpc-stream","commit_stats":null,"previous_names":["sleipnir/grpc-stream"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sleipnir%2Fgrpc-stream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sleipnir%2Fgrpc-stream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sleipnir%2Fgrpc-stream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sleipnir%2Fgrpc-stream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sleipnir","download_url":"https://codeload.github.com/sleipnir/grpc-stream/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247043352,"owners_count":20874087,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-29T10:34:22.180Z","updated_at":"2025-08-01T00:32:27.572Z","avatar_url":"https://github.com/sleipnir.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# gRPC Stream\n\n**Backpressure-enabled gRPC streaming adapter for Elixir using GenStage and GRPCStream**\n\n`GRPCStream` is an Elixir module designed to simplify gRPC server-side streaming by transforming incoming gRPC streams into `GRPCStream` pipelines, offering backpressure and integration with additional unbounded producers (e.g., RabbitMQ, Kafka, or other `GenStage` producer).\n\n## ✨ Features\n\n- Convert gRPC streaming requests into `GRPCStream` pipelines.\n- Full support for GenStage backpressure.\n- Plug in additional unbounded `GenStage` producers for infinite/event-driven streaming.\n- Send processed messages back to clients via gRPC streams.\n\n---\n\n\n## 🚀 Installation\n\nAdd the dependencies to your `mix.exs` file:\n\n```elixir\ndef deps do\n  [\n    {:grpc_stream, github: \"sleipnir/grpc_stream\"},\n  ]\nend\n```\n\n## ⚙️ Basic Usage\n\n```elixir\ndefmodule MyGRPCService do\n  use GRPC.Server, service: MyService.Service\n  alias GRPCStream\n\n  def route_chat(request, materializer) do\n    GRPCStream.from(request, max_demand: 10)\n    |\u003e GRPCStream.map(fn note -\u003e\n      # Process incoming gRPC message\n      %MyProto.Note{message: \"[echo] #{note.message}\"}\n    end)\n    |\u003e GRPCStream.run_with(materializer)\n  end\nend\n``` \n\n## 🔁 Using an External Unbounded Producer\n\nYou can enhance the stream by passing an unbounded GenStage producer (like RabbitMQ, Kafka consumer or any else GenStage producer):\n\n```elixir\ndefmodule MyGRPCService do\n  use GRPC.Server, service: MyService.Service\n  alias GRPCStream\n\n  def stream_events(request, materializer) do\n    {:ok, rabbit_producer} = MyApp.RabbitMQ.Producer.start_link([])\n\n    GRPCStream.from(request, join_with: rabbit_producer, max_demand: 10)\n    |\u003e GRPCStream.map(\u0026transform_event/1)\n    |\u003e GRPCStream.run_with(materializer)\n  end\n\n  defp transform_event({_, grpc_msg}), do: grpc_msg\n  defp transform_event(event), do: %MyProto.Event{data: inspect(event)}\nend\n```\n\n## 📡 Synchronous Request-Response with Processes\nUse ask/3 to implement request-response patterns with arbitrary processes:\n\n```elixir\ndefmodule ChatHandler do\n  def start do\n    spawn(fn -\u003e \n      receive do\n        {:request, msg, from} -\u003e \n          processed = \"[ECHO] #{msg}\"\n          send(from, {:response, processed})\n      end\n    end)\n  end\nend\n\ndefmodule MyGRPCService do\n  use GRPC.Server, service: Chat.Service\n  \n  def chat_stream(req_enum, materializer) do\n    handler_pid = ChatHandler.start()\n    \n    GRPCStream.from(req_enum)\n    |\u003e GRPCStream.ask(handler_pid)\n    |\u003e GRPCStream.map(fn\n      {:error, :timeout} -\u003e %ChatMsg{text: \"Server timeout!\"}\n      response -\u003e %ChatMsg{text: response}\n    end)\n    |\u003e GRPCStream.run_with(materializer)\n  end\nend\n```\n\n## 🏗️ Using GenServer for Backend Processing\nFor more robust interactions, use the GenServer version with registered modules:\n\n```elixir\ndefmodule AnalyticsServer do\n  use GenServer\n  \n  def start_link(), do: GenServer.start_link(__MODULE__, [], name: __MODULE__)\n  \n  # GenServer implementation\n  def handle_call({:request, event}, _from, state) do\n    processed_event = process_analytics(event)\n    {:reply, {:response, processed_event}, state}\n  end\n  \n  defp process_analytics(event), do: # ... analytics logic ...\nend\n\ndefmodule MyGRPCService do\n  use GRPC.Server, service: Analytics.Service\n  \n  @spec event_stream(any(), GRPC.Server.Stream.t()) :: any()\n  def event_stream(request, materializer) do\n    AnalyticsServer.start_link()\n    \n    GRPCStream.from(request)\n    |\u003e GRPCStream.ask(AnalyticsServer, 10_000)\n    |\u003e GRPCStream.map(fn\n      {:error, :timeout} -\u003e %AnalyticEvent{status: :TIMEOUT}\n      result -\u003e %AnalyticEvent{data: result}\n    end)\n    |\u003e GRPCStream.run_with(materializer)\n  end\nend\n```\n\n## 🛠️ Hybrid Example with External Producer\nCombine with external systems while maintaining request-response semantics:\n\n```elixir\ndefmodule TransactionService do\n  use GenServer\n  \n  def handle_call({:request, tx}, _from, state) do\n    {:reply, {:response, validate_transaction(tx)}, state}\n  end\n  \n  defp validate_transaction(tx) do\n    :timer.sleep(500)\n    %TransactionResult{valid: true}\n  end\nend\n\ndefmodule MyGRPCService do\n  use GRPC.Server, service: Transaction.Service\n  \n  def process_transactions(request, materializer) do\n    {:ok, kafka_producer} = MyApp.KafkaProducer.start_link()\n    TransactionService.start_link() # or start in another place\n    \n    GRPCStream.from(req_enum, \n      join_with: kafka_producer,\n      max_demand: 20\n    )\n    |\u003e GRPCStream.ask(TransactionService) # Validate via GenServer\n    |\u003e GRPCStream.filter(fn\n      %TransactionResult{valid: true} -\u003e true\n      _ -\u003e false\n    end)\n    |\u003e GRPCStream.run_with(materializer)\n  end\nend\n``` \n\nSee more in [tests](./test/grpc_stream_test.exs)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsleipnir%2Fgrpc-stream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsleipnir%2Fgrpc-stream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsleipnir%2Fgrpc-stream/lists"}