{"id":13741212,"url":"https://github.com/tsloughter/grpcbox","last_synced_at":"2025-05-16T07:05:33.360Z","repository":{"id":38309147,"uuid":"112283778","full_name":"tsloughter/grpcbox","owner":"tsloughter","description":"Erlang grpc on chatterbox","archived":false,"fork":false,"pushed_at":"2024-04-11T16:55:55.000Z","size":510,"stargazers_count":142,"open_issues_count":37,"forks_count":61,"subscribers_count":12,"default_branch":"main","last_synced_at":"2025-05-13T18:13:42.802Z","etag":null,"topics":["erlang","grpc","grpc-server"],"latest_commit_sha":null,"homepage":null,"language":"Erlang","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tsloughter.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-11-28T04:01:48.000Z","updated_at":"2025-04-16T04:26:34.000Z","dependencies_parsed_at":"2024-06-18T15:36:35.017Z","dependency_job_id":"b3fd9800-6f2e-4c65-b354-b0e18d8a160b","html_url":"https://github.com/tsloughter/grpcbox","commit_stats":{"total_commits":144,"total_committers":14,"mean_commits":"10.285714285714286","dds":"0.23611111111111116","last_synced_commit":"a5aa6e918a2ddbff40e8ebb3c12b1453fe953e03"},"previous_names":[],"tags_count":18,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tsloughter%2Fgrpcbox","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tsloughter%2Fgrpcbox/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tsloughter%2Fgrpcbox/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tsloughter%2Fgrpcbox/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tsloughter","download_url":"https://codeload.github.com/tsloughter/grpcbox/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254485060,"owners_count":22078767,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["erlang","grpc","grpc-server"],"created_at":"2024-08-03T04:00:56.867Z","updated_at":"2025-05-16T07:05:28.350Z","avatar_url":"https://github.com/tsloughter.png","language":"Erlang","funding_links":[],"categories":["Language-Specific"],"sub_categories":["Erlang"],"readme":"grpcbox\n=====\n\n\n![Tests](https://github.com/tsloughter/grpcbox/workflows/Common%20Test/badge.svg)\n[![codecov](https://codecov.io/gh/tsloughter/grpcbox/branch/main/graph/badge.svg)](https://codecov.io/gh/tsloughter/grpcbox)\n[![Hex.pm](https://img.shields.io/hexpm/v/grpcbox.svg?maxAge=2592000)](https://hex.pm/packages/grpcbox)\n[![Hex.pm](https://img.shields.io/hexpm/dt/grpcbox.svg?maxAge=2592000)](https://hex.pm/packages/grpcbox)\n\nLibrary for creating [grpc](https://grpc.io) services (client and server) in Erlang, based on the [chatterbox](https://github.com/joedevivo/chatterbox) http2 library.\n\nFeatures\n---\n\n* Unary, client stream, server stream and bidirectional rpcs\n* Client load balancing\n* Interceptors\n* Health check service\n* Reflection service\n* [OpenCensus](https://opencensus.io) interceptors for stats and tracing\n* [Plugin](https://github.com/tsloughter/grpcbox_plugin) for generating clients and behaviour type specs for service server implementation\n\nImplementing a Service Server\n----\n\nThe quickest way to play around is with the test service and client that is used by `grpcbox`. Simply pull up a shell with, `rebar3 as test shell` and the route guide service will start on port 8080 and you'll have the client, `routeguide_route_guide_client`, in the path.\n\nThe easiest way to get started on your own project is using the plugin, [grpcbox_plugin](https://github.com/tsloughter/grpcbox_plugin):\n\n```erlang\n{deps, [grpcbox]}.\n\n{grpc, [{protos, \"protos\"},\n        {gpb_opts, [{module_name_suffix, \"_pb\"}]}]}.\n\n{plugins, [grpcbox_plugin]}.\n```\n\nCurrently `grpcbox` and the plugin are a bit picky and the `gpb` options will always include `[use_packages, maps, {i, \".\"}, {o, \"src\"}]`.\n\nAssuming the `protos` directory of your application has the `route_guide.proto` found in this repo, `protos/route_guide.proto`, the output from running the plugin will be:\n\n```shell\n$ rebar3 grpc gen\n===\u003e Writing src/route_guide_pb.erl\n===\u003e Writing src/grpcbox_route_guide_bhvr.erl\n```\n\nA behaviour is used because it provides a way to generate the interface and types without being where the actual implementation is also done. This way if a change happens to the proto you can regenerate the interface without any issues with the implementation of the service, simply then update the implementation callbacks to match the changed interface.\n\nRuntime configuration for `grpcbox` can be done in `sys.config`, specifying the compiled proto modules to use for finding the services available, which services to actually enable for requests and what module implements them, acceptor pool and http server settings. See `interop/config/sys.config` for a working example.\n\nIn the interop config the portion for defining services to handle requests for is:\n\n``` erlrang\n{grpcbox, [{servers, [#{grpc_opts =\u003e #{service_protos =\u003e [test_pb],\n                                       services =\u003e #{'grpc.testing.TestService' =\u003e grpc_testing_test_service}}}]},\n...\n```\n\n`test_pb` is the `gpb` generated module that exports `get_service_names/0`. The results of that function are used to construct the metadata needed for handling requests. The `services` map gives the module to call for handling methods of a service. If a service is not defined in that map it will result in the grpc error code 12, `Unimplemented`.\n\nThe services will be started when the application starts assuming the services are all configured in the `sys.config` and it is loaded. To manually start a service use either `grpcbox:start_server/1` which will start a `grpcbox_service_sup` supervisor under the `grpcbox_services_simple_sup` simple one for one supervisor, or get a child spec `grpcbox:server_child_spec(ServerOpts, GrpcOpts, ListenOpts, PoolOpts, TransportOpts)` to include the service supervisor in your own supervision tree.\n\n#### Unary RPC\n\nUnary RPCs receive a single request and return a single response. The RPC `GetFeature` takes a single `Point` and returns the `Feature` at that point:\n\n```protobuf\nrpc GetFeature(Point) returns (Feature) {}\n```\n\nThe callback generated by the `grpcbox_plugin` will look like:\n\n```erlang\n-callback get_feature(ctx:ctx(), route_guide_pb:point()) -\u003e\n    {ok, route_guide_pb:feature(), ctx:ctx(} | grpcbox_stream:grpc_error_response().\n```\n\nAnd the implementation is as simple as an Erlang function that takes the arguments `Ctx`, the context of this current request, and a `Point` map, returning a `Feature` map:\n\n```erlang\nget_feature(Ctx, Point) -\u003e\n    Feature = #{name =\u003e find_point(Point, data()),\n                location =\u003e Point},\n    {ok, Feature, Ctx}.\n```\n\n#### Streaming Output\n\nInstead of returning a single feature the server can stream a response of multiple features by defining the RPC to have a `stream Feature` return:\n\n```protobuf\nrpc ListFeatures(Rectangle) returns (stream Feature) {}\n```\n\nIn this case the callback still receives a map argument but also a `grpcbox_stream` argument:\n\n```erlang\n-callback list_features(route_guide_pb:rectangle(), grpcbox_stream:t()) -\u003e\n    ok | {error, term()}.\n```\n\nThe `GrpcStream` variable is passed to `grpcbox_stream:send/2` for returning an individual feature over the stream to the client. The stream is ended by the server when the function completes.\n\n```erlang\nlist_features(_Message, GrpcStream) -\u003e\n    grpcbox_stream:send(#{name =\u003e \u003c\u003c\"Tour Eiffel\"\u003e\u003e,\n                                        location =\u003e #{latitude =\u003e 3,\n                                                      longitude =\u003e 5}}, GrpcStream),\n    grpcbox_stream:send(#{name =\u003e \u003c\u003c\"Louvre\"\u003e\u003e,\n                          location =\u003e #{latitude =\u003e 4,\n                                        longitude =\u003e 5}}, GrpcStream),\n    ok.\n```\n\n#### Streaming Input\n\nThe client can also stream a sequence of messages:\n\n```protobuf\nrpc RecordRoute(stream Point) returns (RouteSummary) {}\n```\n\nIn this case the callback receives a `reference()` instead of a direct value from the client:\n\n```erlang\n-callback record_route(reference(), grpcbox_stream:t()) -\u003e\n    {ok, route_guide_pb:route_summary()} | {error, term()}.\n```\n\nThe process the callback is running in will receive the individual messages on the stream as tuples `{reference(), route_guide_pb:point()}`. The end of the stream is sent as the message `{reference(), eos}` at which point the function can return the response:\n\n```erlang\nrecord_route(Ref, GrpcStream) -\u003e\n    record_route(Ref, #{t_start =\u003e erlang:system_time(1),\n                            acc =\u003e []}, GrpcStream).\n\nrecord_route(Ref, Data=#{t_start := T0, acc := Points}, GrpcStream) -\u003e\n    receive\n        {Ref, eos} -\u003e\n            {ok, #{elapsed_time =\u003e erlang:system_time(1) - T0,\n                   point_count =\u003e length(Points),\n                   feature_count =\u003e count_features(Points),\n                   distance =\u003e distance(Points)}, GrpcStream};\n        {Ref, Point} -\u003e\n            record_route(Ref, Data#{acc =\u003e [Point | Points]}, GrpcStream)\n    end.\n```\n\n#### Streaming In and Out\n\nA bidrectional streaming RPC is defined when both input and output are streams:\n \n```protobuf\nrpc RouteChat(stream RouteNote) returns (stream RouteNote) {}\n```\n\n```erlang\n-callback route_chat(reference(), grpcbox_stream:t()) -\u003e\n    ok | {error, term()}.\n```\n\nThe sequence of input messages will again be sent to the callback's process as Erlang messages and any output messages are sent to the client with `grpcbox_stream`:\n\n```erlang\nroute_chat(Ref, GrpcStream) -\u003e\n    route_chat(Ref, [], GrpcStream).\n\nroute_chat(Ref, Data, GrpcStream) -\u003e\n    receive\n        {Ref, eos} -\u003e\n            ok;\n        {Ref, #{location := Location} = P} -\u003e\n            Messages = proplists:get_all_values(Location, Data),\n            [grpcbox_stream:send(Message, GrpcStream) || Message \u003c- Messages],\n            route_chat(Ref, [{Location, P} | Data], GrpcStream)\n    end.\n```\n\n#### Interceptors\n\n##### Unary Interceptor\n\nA unary interceptor can be any function that accepts a context, decoded request body, server info map and the method function:\n\n```erlang\nsome_unary_interceptor(Ctx, Request, ServerInfo, Fun) -\u003e\n    %% do some interception stuff\n    Fun(Ctx, Request).\n```\n\nThe interceptor is configured in the `grpc_opts` set in the environment or passed to the supervisor `start_child` function. An example from the test suite sets `grpc_opts` in the application environment:\n\n```erlang\n#{service_protos =\u003e [route_guide_pb],\n  unary_interceptor =\u003e fun(Ctx, Req, _, Method) -\u003e\n                         Method(Ctx, #{latitude =\u003e 30,\n                                       longitude =\u003e 90})\n                       end}\n```\n\n##### Streaming Interceptor\n\n##### Middleware\n\nThere is a provided interceptor `grpcbox_chain_interceptor` which accepts a list of interceptors to apply in order, with the final interceptor calling the method handler. An example from the test suite adds a trailer in each interceptor to show the chain working:\n\n```erlang\n#{service_protos =\u003e [route_guide_pb],\n  unary_interceptor =\u003e\n    grpcbox_chain_interceptor:unary([fun ?MODULE:one/4,\n                                     fun ?MODULE:two/4,\n                                     fun ?MODULE:three/4])}\n```\n\n#### Tracing\n\nThe provided interceptor `grpcbox_trace` supports the [OpenCensus](http://opencensus.io/) wire protocol using [opencensus-erlang](https://github.com/census-instrumentation/opencensus-erlang). It will use the `trace_id`, `span_id` and any options or tags from the trace context.\n\nConfigure as an interceptor:\n\n```erlang\n#{service_protos =\u003e [route_guide_pb],\n  unary_interceptor =\u003e {grpcbox_trace, unary}}\n```\n\nOr as a middleware in the chain interceptor:\n\n```erlang\n#{service_protos =\u003e [route_guide_pb],\n  unary_interceptor =\u003e\n    grpcbox_chain_interceptor:unary([..., \n                                     fun grpcbox_trace:unary/4, \n                                     ...])}\n```\n\nSee [opencensus-erlang](https://github.com/census-instrumentation/opencensus-erlang) for details on configuring reporters.\n\n#### Statistics\n\nStatistics are collected by implementing a stats handler module. A handler for OpenCensus stats (be sure to include [OpenCensus](https://hex.pm/packages/opencensus) as a dependency and make sure it starts on boot) is provided and can be enabled for the server with a config option:\n\n``` erlang\n{grpcbox, [{servers, [#{grpc_opts =\u003e #{stats_handler =\u003e grpcbox_oc_stats_handler\n                                       ...}}]}]}\n```\n\nFor the client the stats handler is a per-channel configuration, see the Defining Channels section below.\n\nYou can verify it is working by enabling the stdout exporter:\n\n``` erlang\n {opencensus, [{stat, [{exporters, [{oc_stat_exporter_stdout, []}]}]}]}\n```\n\nFor actual use, an [exporter for Prometheus](https://github.com/opencensus-beam/prometheus) is available.\n\nDetails on all the metrics that are collected can be found in the [OpenCensus gRPC Stats specification](https://github.com/census-instrumentation/opencensus-specs/blob/master/stats/gRPC.md).\n\n#### Metadata\n\nMetadata is sent in headers and trailers.\n\nUsing a Service Client\n----\n\nFor each service in the protos passed to `rebar3 gprc gen` it will generate a `\u003cservice\u003e_client` module containing a function for each method in the service.\n\n#### Defining Channels\n\nChannels maintain connections to grpc servers and offer client side load balancing between servers with various methods, round robin, random, hash.\n\nIf no channel is specified in the options to a rpc call the `default_channel` is used. Setting the default to connect to localhost on port 8080 in your `sys.config` would look like:\n\n```\n{client, #{channels =\u003e [{default_channel, [{http, \"localhost\", 8080, []}], #{}}]}}\n```\n\nUnix sockets (UDS) may also be used with the same notation that is defined in `gen_tcp`. Considerations:\n* for UDS, only the `http` scheme is permitted\n* the port must strictly be `0`\n* only available on POSIX operating systems\n* abstract UDS are only available on Linux, and such sockets' names must start with a zero byte\n```\n{client, #{channels =\u003e [{default_channel, [{http, {local, \"/path/to/unix/socket_name\"}, 0, []}], #{}}]}}\n%% or to use an abstract Unix socket:\n%% {client, #{channels =\u003e [{default_channel, [{http, {local,  [0 | \"socket_name\"]}, 0, []}], #{}}]}}\n```\n\nThe empty map at the end can contain configuration for the load balancing algorithm, interceptors, statistics handling and compression:\n\n```\n#{balancer =\u003e round_robin | random | hash | direct | claim,\n  encoding =\u003e identity | gzip | deflate | snappy | atom(),\n  stats_handler =\u003e grpcbox_oc_stats_handler,\n  unary_interceptor =\u003e term(),\n  stream_interceptor =\u003e term()} \n```\n\nThe default balancer is round robin and encoding is identity (no compression). Encoding can also be passed in the options map to individual requests.\n\n#### Calling Unary Client RPC\n\nThe `RouteGuide` service has a single unary method, `GetFeature`, in the client we have a function `get_feature/2`:\n\n```erlang\nPoint = #{latitude =\u003e 409146138, longitude =\u003e -746188906},\n{ok, Feature, HeadersAndTrailers} = routeguide_route_guide_client:get_feature(Point).\n```\n\n#### Client Streaming RPC\n\n```erlang\n{ok, S} = routeguide_route_guide_client:record_route(),\nok = grpcbox_client:send(S, #{latitude =\u003e 409146138, longitude =\u003e -746188906}),\nok = grpcbox_client:send(S, #{latitude =\u003e 234818903, longitude =\u003e -823423910}),\nok = grpcbox_client:close_send(S),\n{ok, #{point_count := 2} = grpcbox_client:recv_data(S)).\n```\n\n#### Client with Server Streaming RPC\n\n```erlang\nRectangle = #{hi =\u003e #{latitude =\u003e 1, longitude =\u003e 2},\n              lo =\u003e #{latitude =\u003e 3, longitude =\u003e 5}},\n{ok, S} = routeguide_route_guide_client:list_features(Rectangle),\n{ok, #{\u003c\u003c\":status\"\u003e\u003e := \u003c\u003c\"200\"\u003e\u003e}} = grpcbox_client:recv_headers(S),\n{ok, #{name := _} = grpcbox_client:recv_data(S),\n{ok, #{name := _}} = grpcbox_client:recv_data(S),\n{ok, _} = grpcbox_client:recv_trailers(S).\n```\n\n#### Bidirectional RPC\n\n```erlrang\n{ok, S} = routeguide_route_guide_client:route_chat(),\nok = grpcbox_client:send(S, #{location =\u003e #{latitude =\u003e 1, longitude =\u003e 1}, message =\u003e \u003c\u003c\"hello there\"\u003e\u003e}),\nok = grpcbox_client:send(S, #{location =\u003e #{latitude =\u003e 1, longitude =\u003e 1}, message =\u003e \u003c\u003c\"hello there\"\u003e\u003e}),\n{ok, #{message := \u003c\u003c\"hello there\"\u003e\u003e}} = grpcbox_client:recv_data(S)),\nok = grpcbox_client:send(S, #{location =\u003e #{latitude =\u003e 1, longitude =\u003e 1}, message =\u003e \u003c\u003c\"hello there\"\u003e\u003e}),\n{ok, #{message := \u003c\u003c\"hello there\"\u003e\u003e}}, grpcbox_client:close_and_recv(S)).\n```\n\n#### Context\n\nClient calls optionally accept a [context](https://hex.pm/packages/ctx) as the first argument. Contexts are used to set and propagate deadlines and [OpenCensus](https://hex.pm/packages/opencensus) tags.\n\n```erlang\nCtx = ctx:with_deadline_after(300, seconds),\nPoint = #{latitude =\u003e 409146138, longitude =\u003e -746188906},\n{ok, Feature, HeadersAndTrailers} = routeguide_route_guide_client:get_feature(Ctx, Point).\n```\n\n\nCT Tests\n---\n\nTo run the Common Test suite:\n\n```\n$ rebar3 ct\n```\n\nInterop Tests\n---\n\nThe `interop` rebar3 profile builds with an implementation of the `test.proto` for grpc interop testing:\n\n\nFor testing grpcbox's server:\n\n```\n$ rebar3 as interop shell\n```\n\nWith the shell running the tests can then be run from a script:\n\n```\n$ interop/run_server_tests.sh\n```\n\nThe script by default uses the Go test client that can be installed with the following:\n\n```\n$ go get -u github.com/grpc/grpc-go/interop\n$ go build -o $GOPATH/bin/go-grpc-interop-client github.com/grpc/grpc-go/interop/client\n```\n\nFor testing the grpcbox client you can use the Go test server. But first, add `_ \"google.golang.org/grpc/encoding/gzip\"` to `server.go` imports or else the gzip tests will fail. Then simply build and run it:\n\n```\n$ go build -o $GOPATH/bin/go-grpc-interop-server github.com/grpc/grpc-go/interop/server\n$ $GOPATH/bin/go-grpc-interop-server -port 8080\n```\n\nAnd run the interop client test suite:\n\n```\nrebar3 as interop ct\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftsloughter%2Fgrpcbox","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftsloughter%2Fgrpcbox","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftsloughter%2Fgrpcbox/lists"}