{"id":21354871,"url":"https://github.com/salrashid123/envoy_grpc_decode","last_synced_at":"2026-02-25T08:35:27.622Z","repository":{"id":61819655,"uuid":"554237086","full_name":"salrashid123/envoy_grpc_decode","owner":"salrashid123","description":"Filtering gRPC Messages using Envoy","archived":false,"fork":false,"pushed_at":"2024-11-15T17:37:35.000Z","size":76,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-11-15T18:33:19.441Z","etag":null,"topics":["envoy","envoyproxy","grpc","wireshark"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/salrashid123.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-10-19T13:30:01.000Z","updated_at":"2024-11-15T17:37:39.000Z","dependencies_parsed_at":"2022-10-21T20:00:30.087Z","dependency_job_id":null,"html_url":"https://github.com/salrashid123/envoy_grpc_decode","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salrashid123%2Fenvoy_grpc_decode","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salrashid123%2Fenvoy_grpc_decode/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salrashid123%2Fenvoy_grpc_decode/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salrashid123%2Fenvoy_grpc_decode/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/salrashid123","download_url":"https://codeload.github.com/salrashid123/envoy_grpc_decode/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225839601,"owners_count":17532308,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["envoy","envoyproxy","grpc","wireshark"],"created_at":"2024-11-22T04:14:47.359Z","updated_at":"2026-02-25T08:35:27.580Z","avatar_url":"https://github.com/salrashid123.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n## Decoding gRPC Messages using Envoy\n\nEnvoy configuration set which decodes gRPC messages in several ways:\n\n1. `envoy.filters.http.proto_message_extraction`\n2. `envoy.filters.http.grpc_field_extraction`\n3. `envoy.filters.http.ext_proc`\n\nThe first two will extract _basic_ (text/timestamp/numeric) fields only and make them available as envoy metadata to use in other filters.\n\nThe external processing filter will allow for full mutation.\n\n### External Processing filter\n\nWith the [External Processing Filter](https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/ext_proc/v3/processing_mode.proto) will decode it externally and alters gRPC messages.\n\nIn this flow, the envoy filter will recieve gRPC messages from clients over TLS, then decode and send an altered message to the gRPC Server.\n\n\n![images/ext_grpc.png](images/ext_grpc.png)\n\n\nThis sample builds ontop of these articles:\n\n* [Envoy External Processing Filter](https://blog.salrashid.dev/articles/2021/envoy_ext_proc/)\n* [gRPC Unary requests the hard way: using protorefelect, dynamicpb and wire-encoding to send messages](https://blog.salrashid.dev/articles/2022/grpc_wireformat/)\n\n\nBasically, the external filter decode the grpc wireformat message into byte messages using (`\"github.com/psanford/lencode\"`), then `proto.Unmarshal` that into an actual gRPC message we can inspect.\n\n\n---\n\nIn this demo, given the proto\n\n```proto\nsyntax = \"proto3\";\n\npackage echo;\n\nservice EchoServer {\n  rpc SayHelloUnary (EchoRequest) returns (EchoReply) {}\n  rpc SayHelloServerStream(EchoRequest) returns (stream EchoReply) {}\n}\n\nmessage EchoRequest {\n  string name = 1;\n}\n\nmessage EchoReply {\n  string message = 1;\n}\n```\n\nif the client sends `SayHelloUnary` using  `EchoRequest` with `name=alice`, the filter will alter the payload and send `name=bob` to the grpcServer\n\nif the client sends`SayHelloServerStream` with `name=carol`, the gRPC server will stream two responses back with `message=\"hi carol\"`.  However the filter will alter the final grpc message to the client as `message=\"hi sally\"`\n\n\n```bash\ncd ext_proc/\n\n# start external processing server\ngo run filter.go\n\n## Start envoy\n### docker cp `docker create envoyproxy/envoy-dev:latest`:/usr/local/bin/envoy .\nenvoy -c envoy_ext_proc.yaml -l debug\n```\n\n\nTHen the grpc client and server.\n\n```bash\ncd grpc_server/\n\n# run server\ngo run greeter_server/grpc_server.go --grpcport :50051 \n\n# test client directly to server\ngo run greeter_client/grpc_client.go --host localhost:50051\n    2022/10/19 17:37:46 hi alice\n    2022/10/19 17:37:46 hi carol\n    2022/10/19 17:37:46 hi carol\n\n# test client via envoy\ngo run greeter_client/grpc_client.go --host localhost:8081\n    2022/10/19 17:37:56 hi bob\n    2022/10/19 17:37:57 hi sally\n    2022/10/19 17:37:57 hi sally\n```\n\n\n### Proto message and gRPC Field extraction\n\nThe example proxy has an additional filter which extracts values from the grpc request itself:\n[envoy.filters.http.grpc_field_extraction](https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/grpc_field_extraction/v3/config.proto#grpc-field-extraction-proto) and [envoy.filters.http.grpc_field_extraction](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/proto_message_extraction_filter)\n\n\nFor the first two, you will need to specify the proto descriptor as a file/datasource:\n\n```yaml\n          http_filters:\n          - name: envoy.filters.http.proto_message_extraction\n            typed_config:\n              \"@type\": type.googleapis.com/envoy.extensions.filters.http.proto_message_extraction.v3.ProtoMessageExtractionConfig\n              data_source: \n                filename: \"../grpc_server/echo/echo.proto.pb\"\n              extraction_by_method:\n                echo.EchoServer.SayHelloUnary:\n                  request_extraction_by_field:\n                    name: \"EXTRACT\"\n          - name: envoy.filters.http.grpc_field_extraction\n            typed_config:\n              \"@type\": type.googleapis.com/envoy.extensions.filters.http.grpc_field_extraction.v3.GrpcFieldExtractionConfig\n              descriptor_set: \n                filename: \"../grpc_server/echo/echo.proto.pb\"\n              extractions_by_method:\n                echo.EchoServer.SayHelloUnary:\n                  request_field_extractions:\n                    name: {}\n```\n\n\nThe specific configuration below reads in the descriptor and sets envoy metadata for the `echoRequest` method\n\n```yaml\n          - name: envoy.filters.http.grpc_field_extraction\n            typed_config:\n              \"@type\": type.googleapis.com/envoy.extensions.filters.http.grpc_field_extraction.v3.GrpcFieldExtractionConfig\n              descriptor_set: \n                filename: \"../grpc_server/echo/echo.proto.pb\"\n              extractions_by_method:\n                echo.EchoServer.SayHelloUnary:\n                  request_field_extractions:\n                    name: {}\n```\n\nenvoy logs would show this even before the external processor is called\n\n```log\n[2025-01-18 09:14:33.131][326513][debug][http] [source/common/http/conn_manager_impl.cc:1160] [Tags: \"ConnectionId\":\"1\",\"StreamId\":\"2199321609371850066\"] request headers complete (end_stream=false):\n':method', 'POST'\n':scheme', 'https'\n':path', '/echo.EchoServer/SayHelloUnary'\n':authority', 'grpc.domain.com'\n'content-type', 'application/grpc'\n'user-agent', 'grpc-go/1.33.2'\n'te', 'trailers'\n'grpc-timeout', '997171u'\n\n\n\n[2025-01-18 09:14:33.132][326513][info][misc] [source/extensions/filters/http/grpc_field_extraction/message_converter/message_converter.cc:154] 12 + 0\n[2025-01-18 09:14:33.132][326513][info][misc] [source/extensions/filters/http/grpc_field_extraction/message_converter/message_converter.cc:32] Checking buffer limits: actual 12 \u003e limit 268435456?\n\n[2025-01-18 09:14:33.132][326513][info][misc] [source/extensions/filters/http/grpc_field_extraction/message_converter/message_converter.cc:154] 12 + 0\n[2025-01-18 09:14:33.132][326513][debug][misc] [./source/extensions/filters/http/grpc_field_extraction/message_converter/stream_message.h:22] owned len(owned_bytes_)=7\n[2025-01-18 09:14:33.132][326513][info][misc] [source/extensions/filters/http/grpc_field_extraction/message_converter/message_converter.cc:62] len(parsing_buffer_)=0\n\n[2025-01-18 09:14:33.132][326513][debug][misc] [source/extensions/filters/http/proto_message_extraction/extractor_impl.cc:46] Extracted fields: fields {\n  key: \"@type\"\n  value {\n    string_value: \"type.googleapis.com/echo.EchoRequest\"\n  }\n}\nfields {\n  key: \"name\"\n  value {\n    string_value: \"alice\"\n  }\n}\n\n[2025-01-18 09:14:33.132][326513][debug][filter] [source/extensions/filters/http/proto_message_extraction/filter.cc:389] [Tags: \"ConnectionId\":\"1\",\"StreamId\":\"2199321609371850066\"] Injected request dynamic metadata `envoy.filters.http.proto_message_extraction` with `fields {\n  key: \"requests\"\n  value {\n    struct_value {\n      fields {\n        key: \"first\"\n        value {\n          struct_value {\n            fields {\n              key: \"@type\"\n              value {\n                string_value: \"type.googleapis.com/echo.EchoRequest\"\n              }\n            }\n            fields {\n              key: \"name\"\n              value {\n                string_value: \"alice\"\n              }\n            }\n          }\n        }\n      }\n    }\n  }\n}\n`\n[2025-01-18 09:14:33.132][326513][trace][http] [source/common/http/filter_manager.cc:572] [Tags: \"ConnectionId\":\"1\",\"StreamId\":\"2199321609371850066\"] decode headers called: filter=envoy.filters.http.grpc_field_extraction status=1\n\n\n[2025-01-18 09:14:33.132][326513][debug][misc] [source/extensions/filters/http/grpc_field_extraction/extractor_impl.cc:47] extracted the following resource values from the name field: list_value {\n  values {\n    string_value: \"alice\"\n  }\n}\n\n[2025-01-18 09:14:33.132][326513][debug][filter] [source/extensions/filters/http/grpc_field_extraction/filter.cc:221] [Tags: \"ConnectionId\":\"1\",\"StreamId\":\"2199321609371850066\"] injected dynamic metadata `envoy.filters.http.grpc_field_extraction` with `fields {\n  key: \"name\"\n  value {\n    list_value {\n      values {\n        string_value: \"alice\"\n      }\n    }\n  }\n}\n`\n\n[2025-01-18 09:14:33.132][326513][debug][ext_proc] [source/extensions/filters/http/ext_proc/ext_proc.cc:340] Opening gRPC stream to external processor\n[2025-01-18 09:14:33.132][326513][debug][router] [source/common/router/router.cc:527] [Tags: \"ConnectionId\":\"0\",\"StreamId\":\"541208930907820837\"] cluster 'ext_proc_cluster' match for URL '/envoy.service.ext_proc.v3.ExternalProcessor/Process'\n[2025-01-18 09:14:33.132][326513][debug][router] [source/common/router/router.cc:756] [Tags: \"ConnectionId\":\"0\",\"StreamId\":\"541208930907820837\"] router decoding headers:\n':method', 'POST'\n':path', '/envoy.service.ext_proc.v3.ExternalProcessor/Process'\n':authority', 'ext_proc_cluster'\n':scheme', 'http'\n'te', 'trailers'\n'content-type', 'application/grpc'\n'x-envoy-internal', 'true'\n'x-forwarded-for', '192.168.1.160'\n\n\n[2025-01-18 09:14:33.136][326513][debug][router] [source/common/router/router.cc:527] [Tags: \"ConnectionId\":\"1\",\"StreamId\":\"2199321609371850066\"] cluster 'grpc_upstream_svc' match for URL '/echo.EchoServer/SayHelloUnary'\n[2025-01-18 09:14:33.136][326513][debug][router] [source/common/router/router.cc:756] [Tags: \"ConnectionId\":\"1\",\"StreamId\":\"2199321609371850066\"] router decoding headers:\n':method', 'POST'\n':scheme', 'https'\n':path', '/echo.EchoServer/SayHelloUnary'\n':authority', 'grpc.domain.com'\n'content-type', 'application/grpc'\n'user-agent', 'grpc-go/1.33.2'\n'te', 'trailers'\n'grpc-timeout', '997171u'\n'x-forwarded-proto', 'https'\n'x-request-id', '84b2e8f3-fd0c-4530-b167-39565bfff77d'\n'x-envoy-expected-rq-timeout-ms', '15000'\n\n\n\n[2025-01-18 09:14:33.142][326513][trace][router] [source/common/router/upstream_request.cc:269] [Tags: \"ConnectionId\":\"1\",\"StreamId\":\"2199321609371850066\"] end_stream: false, upstream response headers:\n':status', '200'\n'content-type', 'application/grpc'\n\n\n[2025-01-18 09:14:33.142][326513][debug][misc] [source/extensions/filters/http/proto_message_extraction/extractor_impl.cc:46] Extracted fields: fields {\n  key: \"@type\"\n  value {\n    string_value: \"type.googleapis.com/echo.EchoReply\"\n  }\n}\n\n[2025-01-18 09:14:33.142][326513][debug][filter] [source/extensions/filters/http/proto_message_extraction/filter.cc:423] [Tags: \"ConnectionId\":\"1\",\"StreamId\":\"2199321609371850066\"] Injected response dynamic metadata `envoy.filters.http.proto_message_extraction` with `fields {\n  key: \"responses\"\n  value {\n    struct_value {\n      fields {\n        key: \"first\"\n        value {\n          struct_value {\n            fields {\n              key: \"@type\"\n              value {\n                string_value: \"type.googleapis.com/echo.EchoReply\"\n              }\n            }\n          }\n        }\n      }\n    }\n  }\n}\n`\n\n[2025-01-18 09:14:33.143][326513][debug][http] [source/common/http/conn_manager_impl.cc:1879] [Tags: \"ConnectionId\":\"1\",\"StreamId\":\"2199321609371850066\"] encoding trailers via codec:\n'grpc-status', '0'\n'grpc-message', ''\n\n```\n\n### Appendix\n\nTODO: see if we can create a wasm filter to do the same (which isn't that easy since we need to decode the wireformat)\n- [Envoy WASM Filter](https://github.com/salrashid123/envoy_wasm)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsalrashid123%2Fenvoy_grpc_decode","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsalrashid123%2Fenvoy_grpc_decode","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsalrashid123%2Fenvoy_grpc_decode/lists"}