{"id":13741737,"url":"https://github.com/vladimirvivien/go-grpc","last_synced_at":"2025-06-24T21:31:41.025Z","repository":{"id":43115612,"uuid":"100124671","full_name":"vladimirvivien/go-grpc","owner":"vladimirvivien","description":"A collection of gRPC and Go examples showcasing features of the framework","archived":false,"fork":false,"pushed_at":"2022-02-28T19:57:03.000Z","size":71,"stargazers_count":249,"open_issues_count":0,"forks_count":40,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-06-12T01:43:57.217Z","etag":null,"topics":["go","golang","grpc","rpc"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vladimirvivien.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-08-12T16:33:53.000Z","updated_at":"2025-05-29T07:33:50.000Z","dependencies_parsed_at":"2022-09-22T06:11:13.760Z","dependency_job_id":null,"html_url":"https://github.com/vladimirvivien/go-grpc","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/vladimirvivien/go-grpc","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vladimirvivien%2Fgo-grpc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vladimirvivien%2Fgo-grpc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vladimirvivien%2Fgo-grpc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vladimirvivien%2Fgo-grpc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vladimirvivien","download_url":"https://codeload.github.com/vladimirvivien/go-grpc/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vladimirvivien%2Fgo-grpc/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261759105,"owners_count":23205496,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","golang","grpc","rpc"],"created_at":"2024-08-03T04:01:02.223Z","updated_at":"2025-06-24T21:31:40.962Z","avatar_url":"https://github.com/vladimirvivien.png","language":"Go","funding_links":[],"categories":["Resources"],"sub_categories":["Examples"],"readme":"# Go and the gRPC Framework\nIf you are reading this, chances are you have some familiarity with Go and are looking to start working with gRPC.  Or, maybe you are have been working with gRPC in a different language and are looking to start using it with Go.  Either way, welcome!\n\nThis repository is a collection of code samples that showcases several features of the gRPC-go framework.  Before we jump in too deep, let us start from the beginning with an overview of gRPC, exploring the nuts and bolts of its components.\n\n## Why gRPC\nTo explain gRPC, let us establish a scenario where we have a financial provider that wants to create a *Currency Service* that allows lookup and validation of currency info (i.e. name, code, country, and ISO number).  The service is spec'd to have the following non-functional properties:\n- Be accessible from mobile front-ends (Java, Objective-C)\n- Accessible from backend other backends (Python, Java)\n- Accessible from Node.js backend to suppor JS front-end\n- Provides desktop access (#C) for a reporting tool\n\nYour first reaction, to implement such service, may be to reach for JSON+HTTP (REST) to implement an HTTP-based API server.  And, that would be a good choice as these technologies are mature and well-understood by the developer community.  JSON was a response to the shortcomings of SOAP-era technologies and its soup of acronyms (let's not even mentioned the stuff SOAP replaced).  Similarly, now gRPC is an evolution of this service-based approach that uses RPC to fill the technological gaps left by the REST including:\n- JSON offers weak data types\n- Lack of standardized machine-enforced interface contracts in JSON\n- JSON is a flexible but inefficient text-based encoding \n- Services are exposed as requests/responses of JSON docs causing a lost of semantics (HTTP only has GET,PUT,POST,DELETE, and PATCH)\n- Services code and clients are generally manually generated \n- Versioning, update, backward compatibility can be problems\n\nIf you are about to say SOAP, stop.  While SOAP-based technologies introduced the machin-readable and enforceable contracts, it had its own shortmings:\n- Cumbersome contract definitions that were meant for machine, but often created by human\n- Encoding and decoding SOAP was so resource intensive, that it became its own market with vendors selling hardware to do it\n- XML over the wire is inefficient for low-bandwidth devices (mobile)\n- Tooling varied by platform and language\n\ngRPC attempts to incorporate best practices from these technlogies by leveraging the efficiencies of protocol boffers and HTTP/2.  It also introduce features not found in prior statck like bi-directional streaming allowing the creation of data intensive applications.\n\n## gRPC Overview\ngRPC is a \"high performance, open-source universal RPC framework\".  Specifically gRPC provides all the tools necessary to write services and clients, in a variety of languages, that can communicate by expressing remote services as transparent native methods on the client.\n\ngRPC is designed to work efficiently with usage ranging from datacenter computing to small IoT devices.  It continues where REST and SOAP left off and provides the following features:\n- Uses Protocol Buffers as a typed and efficient binary wire format\n- Machine-to-machine contracts are defined using a simple interface definition language (IDL)\n- Tools to generate code, from IDL, used to implement service methods and client code to invoke those methods remotely\n- Uses HTTP/2 which multiplexes long-lived connections for fast and efficient communication\n- Support for bi-directional streaming between client and servers\n- Extensive middleware API to control structural concerns such as security, logging, and service policy\n\n## Exploring Protocol Buffers\ngRPC's efficiency is partly due to its use of protocol buffers (or protobuf).  It is a language-neutral and platform-neutral technology to efficiently serialize data.  Protocol buffers can be used independently of gRPC as a binary wire or storage format.\n\nThe first step to using protocol buffers is to define `messages` which are structures that consist of strongly-typed fields representing the data to be encoded.  Protocol buffer supports fields of diverse tyes including numeric, string, boolean, enums, or other messages. \n\nThe following is a simple protobuf definition with two messages: `Currency` and `CurrencyList`:\n\n```protobuf\nsyntax = \"proto3\";\npackage curproto;\n\nmessage Currency {\n    string code = 1; \n    string name = 2;\n    int32 number = 3;\n    string country = 4;\n}\n\nmessage CurrencyList {\n    repeated Currency items = 1;\n}\n```\n*Protocol buffers file [pb-examples/curproto/currency.proto](https://github.com/vladimirvivien/go-grpc/blob/master/pb-examples/curproto/currency.proto)*\n\nMessages are defined in a file with a `.proto` extension (convention) where they can be arranged as complex and nested data structures.\n\n### Compile the .proto file\nThe protobuf file by itself is not much use.  The next step is to compile the file using the protocol buffers compiler (`protoc`) located at https://developers.google.com/protocol-buffers/.  The compiler does the followings:\n- Create source code containing data structres based on the protobuf messages\n- Create code that can serialize and deserialize data into the generated structures\n\nThe compiler can generate code into several languages using a pluggable architecture. To generate Go code, download the Go generator for protoc using:\n\n```shell\n$\u003e go get github.com/golang/protobuf/protoc-gen-go\n```\nTo generate the Go code from the protobuf file, we can use the following command:\n```sh\n$ protoc --go_out=./curproto ./curproto/currency.proto\n```\nThe previous command uses parameter `--go_out` to specify Go code generation.  It will compile file `currency.proto` in directory `curproto` and place the generated Go code there as well.  The compilation step will generate Go source file `currency.pb.go` which contains code for serialization, deserialization, and struct types matching the messages defined in the .proto file:\n```go\ntype Currency struct {\n\tCode    string `protobuf:\"bytes,1,opt,name=code\" json:\"code,omitempty\"`\n\tName    string `protobuf:\"bytes,2,opt,name=name\" json:\"name,omitempty\"`\n\tNumber  int32  `protobuf:\"varint,3,opt,name=number\" json:\"number,omitempty\"`\n\tCountry string `protobuf:\"bytes,4,opt,name=country\" json:\"country,omitempty\"`\n}\n...\ntype CurrencyList struct {\n\tItems []*Currency `protobuf:\"bytes,1,rep,name=items\" json:\"items,omitempty\"`\n}\n```\n*Generated Go file [pb-examples/curproto/currency.pb.go](https://github.com/vladimirvivien/go-grpc/blob/master/pb-examples/curproto/currency.pb.go)*\n\n### Using Protobuf directly\nOnce we have the ability to serialize and deserialize our data, we can use it for any purpose where binary encoding can be applied.  For instance, the following example shows how to use protocol buffer as an effeicient format for storage.  The source snippet below loads data from a CSV file and saves it as a protocol buffer encoded file.\n\n```go\nimport (\n    ...\n    \"github.com/golang/protobuf/proto\"\n    \"github.com/vladimirvivien/go-grpc/pg-example/curproto\"\n)\n\nconst fileName = \"data.pb\"\n\nfunc main() {\n\tcurrencyItems, err := createPbFromCsv(\"../curdata.csv\")\n\tif err != nil {\n\t\tlog.Fatalf(\"failed to load csv: %v\\n\", err)\n\t}\n\n\t// encode data as protobuf binary\n\tdata, err := proto.Marshal(currencyItems)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\t// save the encoded binary to file\n\tif err := ioutil.WriteFile(fileName, data, 0644); err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tlog.Println(\"data file saved as\", fileName)\n}\n\n// read csv content and return rows as *curproto.CurrencyList\n// a type generated from protobuf\nfunc createPbFromCsv(path string) (*curproto.CurrencyList, error) {\n\titems := make([]*curproto.Currency, 0)\n\tfile, err := os.Open(path)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tdefer file.Close()\n\n\t// create CSV reader from file\n\treader := csv.NewReader(file)\n\tfor {\n\t\trow, err := reader.Read()\n\t\tif err != nil {\n\t\t\tif err == io.EOF {\n\t\t\t\tbreak\n\t\t\t} else {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t}\n\t\tvar num int32\n\t\tif i, err := strconv.Atoi(row[3]); err == nil {\n\t\t\tnum = int32(i)\n\t\t}\n\t\t// copy row data into protobuf-generated type\n\t\tc := \u0026curproto.Currency{\n\t\t\tCountry: row[0],\n\t\t\tName:    row[1],\n\t\t\tCode:    row[2],\n\t\t\tNumber:  num,\n\t\t}\n\t\titems = append(items, c)\n\t}\n\treturn \u0026curproto.CurrencyList{Items: items}, err\n}\n```\n*Protobuf example file [pb-examples/encode_pb.go](https://github.com/vladimirvivien/go-grpc/blob/master/pb-examples/encode_pb.go)*\n\nWhen the code (above) is executed, it will produce file `data.pb` with the data encoded using the protocol buffer binary format.  Doing something similar using JSON ([source code](https://github.com/vladimirvivien/go-grpc/blob/master/pb-examples/encode_json.go)), we can compare the resulting data file sizes for a rough comparison in efficiency as shown below:\n```shell\n-rw-r--r--  1 vvivien  staff    20K  data.js\n-rw-r--r--  1 vvivien  staff    10K  data.pb\n```\nThis simple test reveals that the protobuf-encoded file is half the size of the JSON-encoded file. This saving can be even more pronounced when the data is composed of mostly numeric data.\n\n## Creating Services with Protobuf and gRPC\nEarlier we have seen how protocol buffers work.  Now, let us see how to use protobuf and the gPRC framework to build efficient and fast RPC services.  When creating services with gRPC, there are three general steps that must be followed:\n\n\u003e 1. Create a protobuf file (IDL) to define messages and service methods\n\u003e 2. Compile the protobuf IDL into code to generate types and service interfaces\n\u003e 3. Implement the code for the service remote methods\n\n\n### 1. Define Protocol Buffers IDL\nUsing the currency service scenario, presented earlier, let us define the protobuf file that contains the messages and the service definition:\n\n```protobuf\nsyntax = \"proto3\";\n  \nmessage Currency {\n    string code = 1; \n    string name = 2;\n    int32 number = 3;\n    string country = 4;\n}\n\nmessage CurrencyList {\n    repeated Currency items = 1;\n}\n\nmessage CurrencyRequest {\n    string code = 1;\n    int32 number = 2;\n}\n\n// CurrencyService exposes methods to call\nservice CurrencyService {\n    rpc GetCurrencyList(CurrencyRequest) returns (CurrencyList){}\n}\n```\n*Protocol Buffers IDL [protobuf/currency.proto](https://github.com/vladimirvivien/go-grpc/blob/master/protobuf/currency.proto)*\n\nThe protocol buffer defines the messages that we saw earlier. However, it now also contains a `service` block which defines one or more `rpc` methods.  In our example, service `CurrencyService` :\n  - Offers method `GetCurrencyList` \n  - The method takes `CurrencyRequest` as input paremeter \n  - And, returns `CurrencyList` to the client\n\n### 2. Compile the IDL File\nAs before, the protobuf IDL file needs to be compiled into source code. To generate code for gPRC, however, we need to specify additional protoc parameters to trigger the gRPC plugin which will generate the code necessary to bootstrap the RPC services and remote methods. \n\nAssuming the IDL file above is located in a folder called `./protobuf`, the following will generate, in addition to the message types, the gRPC code needed to implement remote server methods and the client stubs to call them:\n```sh\n $ protoc -I=./protobuf --go_out=plugins=grpc:./protobuf ./protobuf/currency.proto\n```\n\u003e Notice the additional parameter value in `--go_out`\n\nThe compiler will genearate file `currency.pb.go` in the `./protobuf` directory.  The generated source contains the message struct types (as before), but also includes the service methods as a Go interface to be implemented:\n```go\ntype Currency struct {\n\tCode    string `protobuf:\"bytes,1,opt,name=code\" json:\"code,omitempty\"`\n\tName    string `protobuf:\"bytes,2,opt,name=name\" json:\"name,omitempty\"`\n\tNumber  int32  `protobuf:\"varint,3,opt,name=number\" json:\"number,omitempty\"`\n\tCountry string `protobuf:\"bytes,4,opt,name=country\" json:\"country,omitempty\"`\n}\n\ntype CurrencyList struct {\n\tItems []*Currency `protobuf:\"bytes,1,rep,name=items\" json:\"items,omitempty\"`\n}\n\ntype CurrencyRequest struct {\n\tCode   string `protobuf:\"bytes,1,opt,name=code\" json:\"code,omitempty\"`\n\tNumber int32  `protobuf:\"varint,2,opt,name=number\" json:\"number,omitempty\"`\n}\n\n// service interface for CurrencyService\ntype CurrencyServiceServer interface {\n\tGetCurrencyList(context.Context, *CurrencyRequest) (*CurrencyList, error)\n}\n```\n### 3. Implement the service\nThe last general step is to implement remote methods for the service.  For our this example, we will implement method `GetCurrencyList()` which return a value of type `CurrencyList` as defined in the IDL.\n\n```go\nimport (\n\t\"golang.org/x/net/context\"\n\t\"google.golang.org/grpc\"\n\n\tpb \"github.com/vladimirvivien/go-grpc/protobuf\"\n\t\"github.com/vladimirvivien/go-grpc/util\"\n)\n\ntype CurrencyService struct {\n\tdata []*pb.Currency\n}\n\nfunc newCurrencyService(data []*pb.Currency) *CurrencyService {\n\treturn \u0026CurrencyService{data: data}\n}\n\n// GetCurrencyList searches (by Code or Number) and return CurrencyList\nfunc (c *CurrencyService) GetCurrencyList(\n\tctx context.Context,\n\treq *pb.CurrencyRequest,\n) (*pb.CurrencyList, error) {\n\n\tvar items []*pb.Currency\n\tfor _, cur := range c.data {\n\t\tif cur.GetNumber() == req.GetNumber() || cur.GetCode() == req.GetCode() {\n\t\t\titems = append(items, cur)\n\t\t}\n\t}\n\n\treturn \u0026pb.CurrencyList{Items: items}, nil\n}\n\nfunc main() {\n\n\t// load data into protobuf structures\n\tdata, err := util.LoadPbFromCsv(\"./../curdata.csv\")\n\tif err != nil {\n\t\tlog.Fatal(err) // dont start\n\t}\n\n    lstnr, err := net.Listen(\"tcp\", \":50050\")\n\tif err != nil {\n\t\tlog.Fatal(\"failed to start server:\", err)\n\t}\n\n\t// setup and register currency service\n\tcurService := newCurrencyService(data)\n\tgrpcServer := grpc.NewServer()\n\tpb.RegisterCurrencyServiceServer(grpcServer, curService)\n\n\t// start service's server\n\tlog.Println(\"starting currency rpc service on\", port)\n\tif err := grpcServer.Serve(lstnr); err != nil {\n\t\tlog.Fatal(err)\n\t}\n}\n```\n*gRPC server file [grpc/server.go](https://github.com/vladimirvivien/go-grpc/blob/master/grpc/server.go)*\n\nIn function `main`, the code uses package `grpc` to register the service implementation and bootstrap the server to expose remote method `GetCurrencyList()`.  When the code is executed, an HTTP/2 server will start listening for requests on `TCP` port `50050`.  This is the general pattern that is often used to get a gRPC service up and running.  \n\n### Using the service\nAt this point, we are ready to write a simple client which can invoke the remote method defined earlier.  The snippet below shows a Go client that calls the service.\n\n\u003eIt should be noted that the client stubs used to call the remote service are (can be) generated when the protocol buffer definition file is compiled (see above).\n\n```go\n// printUSD demonstrates simple binary call from client\nfunc printUSD(client pb.CurrencyServiceClient) {\n\tcurReq := \u0026pb.CurrencyRequest{Code: \"USD\"}\n\tcurList, err := client.GetCurrencyList(context.Background(), curReq)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tfmt.Println(\"\\nUSD Countries\")\n\tfmt.Println(\"-------------\")\n\tfor _, cur := range curList.Items {\n\t\tfmt.Printf(\"%-50s%-10s\\n\", cur.GetCountry(), cur.GetCode())\n\t}\n}\n\nfunc main() {\n\tserverAddr := net.JoinHostPort(\"localhost\", \"50050\")\n\n\t// setup insecure connection\n\tconn, err := grpc.Dial(serverAddr, grpc.WithInsecure())\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\tclient := pb.NewCurrencyServiceClient(conn)\n\n\tprintUSD(client)\n}\n```\nGo client file [grpc/client.go](https://github.com/vladimirvivien/go-grpc/blob/master/grpc/client.go)\n\nIn function `main()` the code uses package `grpc` to setup connection to the RPC server.  The client stub is generated during protoc compilation and provides an extensive API to communicate with the server.  Note in function `printUSD` the call to `client.GetCurrencyList()` looks like it is a local call.  However, its an abstraction that hides the complicated dance of serialization and deserialization of protocol buffers to communicat with the server.\n\n## Other gRPC Examples\nThis repository contains an extensive list of gRPC examples and Go.  You may find some of the followings useful:\n- [grpc_auth](https://github.com/vladimirvivien/go-grpc/tree/master/grpc_auth): example of implementation of JWT token-based authorization.\n- [grpc_err](https://github.com/vladimirvivien/go-grpc/tree/master/grpc_err): shows how to do error handling in gRPC including the use of complex error objects.\n- [grpc_intrcpt](https://github.com/vladimirvivien/go-grpc/tree/master/grpc_intrcpt): introduction to intercept for logging.\n- [grpc_limits](https://github.com/vladimirvivien/go-grpc/tree/master/grpc_limits): shows how add preventive measures to guard your gRPC service and clients from failures by specifying limits. \n- [grpc_rate](https://github.com/vladimirvivien/go-grpc/tree/master/grpc_rate): shows how to setup limits on the rate at which a service can be called for a given period of times.\n- [grpc_retry](https://github.com/vladimirvivien/go-grpc/tree/master/grpc_retry): shows how to use intecerptors to implement a simple retry logic.\n- [grpc_tls](https://github.com/vladimirvivien/go-grpc/tree/master/grpc_tls):shows how to setup TLS-based auth on both client and the server.\n- [grpc_to](https://github.com/vladimirvivien/go-grpc/tree/master/grpc_to): shows how to use context timeout to indicate to the framework how long a request should take.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvladimirvivien%2Fgo-grpc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvladimirvivien%2Fgo-grpc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvladimirvivien%2Fgo-grpc/lists"}