{"id":13724080,"url":"https://github.com/ankur-anand/simple-go-rpc","last_synced_at":"2026-02-03T20:16:23.511Z","repository":{"id":48309761,"uuid":"198426748","full_name":"ankur-anand/simple-go-rpc","owner":"ankur-anand","description":"RPC explained by writing simple RPC framework in 300 lines of pure Golang.","archived":false,"fork":false,"pushed_at":"2021-10-19T17:47:50.000Z","size":22,"stargazers_count":556,"open_issues_count":0,"forks_count":66,"subscribers_count":13,"default_branch":"master","last_synced_at":"2024-08-04T01:23:46.931Z","etag":null,"topics":["go","golang","rpc","rpc-framework"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ankur-anand.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-07-23T12:32:28.000Z","updated_at":"2024-07-09T09:50:38.000Z","dependencies_parsed_at":"2022-09-01T23:02:27.520Z","dependency_job_id":null,"html_url":"https://github.com/ankur-anand/simple-go-rpc","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ankur-anand%2Fsimple-go-rpc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ankur-anand%2Fsimple-go-rpc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ankur-anand%2Fsimple-go-rpc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ankur-anand%2Fsimple-go-rpc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ankur-anand","download_url":"https://codeload.github.com/ankur-anand/simple-go-rpc/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224628361,"owners_count":17343327,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","golang","rpc","rpc-framework"],"created_at":"2024-08-03T01:01:49.765Z","updated_at":"2026-02-03T20:16:18.484Z","avatar_url":"https://github.com/ankur-anand.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"## Simple GoRPC\n\nLearning RPC basic building blocks by building a simple RPC framework in Golang from scratch.\n\n## RPC\n\nIn Simple Term **Service A** wants to call **Service B** functions. But those two services are not in the same memory space. So it cannot be called directly.\n\nSo, in order to make this call happen, we need to express the semantics of how to call and also how to pass the communication through the network.\n\n### Let's think what we do when we call function in the same memory space (local call)\n\n```go\ntype User struct {\n\tName string\n\tAge int\n}\n\nvar userDB = map[int]User{\n\t1: User{\"Ankur\", 85},\n\t9: User{\"Anand\", 25},\n\t8: User{\"Ankur Anand\", 27},\n}\n\n\nfunc QueryUser(id int) (User, error) {\n\tif u, ok := userDB[id]; ok {\n\t\treturn u, nil\n\t}\n\n\treturn User{}, fmt.Errorf(\"id %d not in user db\", id)\n}\n\n\nfunc main() {\n\tu , err := QueryUser(8)\n\tif err != nil {\n\t\tfmt.Println(err)\n\t\treturn\n\t}\n\n\tfmt.Printf(\"name: %s, age: %d \\n\", u.Name, u.Age)\n}\n```\n\nNow, how do we do the same function call over the network?\n\n**Client** will call _QueryUser(id int)_ function over the network and there will be one server which will Serve the Call to this function and return the Response _User{\"Name\", id}, nil_.\n\n## Network Transmission Data format.\n\nSimple-gorpc will do TLV (fixed-length header + variable-length message body) encoding scheme to regulate the transmission of data, over the tcp.\n**More on this later**\n\n### Before we send our data over the network we need to define the structure how we are going to send the data over the network.\n\nThis helps us to define a common protocol that, the client and server both can understand. (protobuf IDL define what both server and client understand).\n\n### So data received by the server needs to have:\n\n- the name of the function to be called\n- list of parameters to be passed to that function\n\nAlso let's agree that the second return value is of type error, indicating the RPC call result.\n\n```go\n// RPCdata transmission format\ntype RPCdata struct {\n\tName string        // name of the function\n\tArgs []interface{} // request's or response's body expect error.\n\tErr  string        // Error any executing remote server\n}\n```\n\nSo now that we have a format, we need to serialize this so that we can send it over the network.\nIn our case we will use the `go` default binary serialization protocol for encoding and decoding.\n\n```go\n// be sent over the network.\nfunc Encode(data RPCdata) ([]byte, error) {\n\tvar buf bytes.Buffer\n\tencoder := gob.NewEncoder(\u0026buf)\n\tif err := encoder.Encode(data); err != nil {\n\t\treturn nil, err\n\t}\n\treturn buf.Bytes(), nil\n}\n\n// Decode the binary data into the Go struct\nfunc Decode(b []byte) (RPCdata, error) {\n\tbuf := bytes.NewBuffer(b)\n\tdecoder := gob.NewDecoder(buf)\n\tvar data RPCdata\n\tif err := decoder.Decode(\u0026data); err != nil {\n\t\treturn Data{}, err\n\t}\n\treturn data, nil\n}\n```\n\n### Network Transmission\n\nThe reason for choosing the TLV protocol is due to the fact that it's very simple to implement, and it also fullfills our need over identification of the length of data to read, as we need to identify the number of bytes to read for this request over the stream of incoming request. `Send and Receive does the same`\n\n```go\n// Transport will use TLV protocol\ntype Transport struct {\n\tconn net.Conn // Conn is a generic stream-oriented network connection.\n}\n\n// NewTransport creates a Transport\nfunc NewTransport(conn net.Conn) *Transport {\n\treturn \u0026Transport{conn}\n}\n\n// Send TLV data over the network\nfunc (t *Transport) Send(data []byte) error {\n\t// we will need 4 more byte then the len of data\n\t// as TLV header is 4bytes and in this header\n\t// we will encode how much byte of data\n\t// we are sending for this request.\n\tbuf := make([]byte, 4+len(data))\n\tbinary.BigEndian.PutUint32(buf[:4], uint32(len(data)))\n\tcopy(buf[4:], data)\n\t_, err := t.conn.Write(buf)\n\tif err != nil {\n\t\treturn err\n\t}\n\treturn nil\n}\n\n// Read TLV sent over the wire\nfunc (t *Transport) Read() ([]byte, error) {\n\theader := make([]byte, 4)\n\t_, err := io.ReadFull(t.conn, header)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tdataLen := binary.BigEndian.Uint32(header)\n\tdata := make([]byte, dataLen)\n\t_, err = io.ReadFull(t.conn, data)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn data, nil\n}\n```\n\nNow that we have the DataFormat and Transport protocol defined. We need and **RPC Server** and **RPC CLient**\n\n## RPC SERVER\n\nRPC Server will receive the `RPCData` which will have an function Name.\n**So we need to maintain and map that contains an function name to actual function mapping**\n\n```go\n// RPCServer ...\ntype RPCServer struct {\n\taddr string\n\tfuncs map[string] reflect.Value\n}\n\n// Register the name of the function and its entries\nfunc (s *RPCServer) Register(fnName string, fFunc interface{}) {\n\tif _,ok := s.funcs[fnName]; ok {\n\t\treturn\n\t}\n\n\ts.funcs[fnName] = reflect.ValueOf(fFunc)\n}\n```\n\nNow that we have the func registered, when we receive the request we will check if the name of func passed during the execution of the function is present or not. and then will execute it accordingly\n\n```go\n// Execute the given function if present\nfunc (s *RPCServer) Execute(req RPCdata) RPCdata {\n\t// get method by name\n\tf, ok := s.funcs[req.Name]\n\tif !ok {\n\t\t// since method is not present\n\t\te := fmt.Sprintf(\"func %s not Registered\", req.Name)\n\t\tlog.Println(e)\n\t\treturn RPCdata{Name: req.Name, Args: nil, Err: e}\n\t}\n\n\tlog.Printf(\"func %s is called\\n\", req.Name)\n\t// unpackage request arguments\n\tinArgs := make([]reflect.Value, len(req.Args))\n\tfor i := range req.Args {\n\t\tinArgs[i] = reflect.ValueOf(req.Args[i])\n\t}\n\n\t// invoke requested method\n\tout := f.Call(inArgs)\n\t// now since we have followed the function signature style where last argument will be an error\n\t// so we will pack the response arguments expect error.\n\tresArgs := make([]interface{}, len(out) - 1)\n\tfor i := 0; i \u003c len(out) - 1; i ++ {\n\t\t// Interface returns the constant value stored in v as an interface{}.\n\t\tresArgs[i] = out[i].Interface()\n\t}\n\n\t// pack error argument\n\tvar er string\n\tif e, ok := out[len(out) - 1].Interface().(error); ok {\n\t\t// convert the error into error string value\n\t\ter = e.Error()\n\t}\n\treturn RPCdata{Name: req.Name, Args: resArgs, Err: er}\n}\n```\n\n## RPC CLIENT\n\nSince the concrete implementation of the function is on the server side, the client only has the prototype of the function, so we need complete prototype of the calling function, so that we can call it.\n\n```go\nfunc (c *Client) callRPC(rpcName string, fPtr interface{}) {\n\tcontainer := reflect.ValueOf(fPtr).Elem()\n\tf := func(req []reflect.Value) []reflect.Value {\n\t\tcReqTransport := NewTransport(c.conn)\n\t\terrorHandler := func(err error) []reflect.Value {\n\t\t\toutArgs := make([]reflect.Value, container.Type().NumOut())\n\t\t\tfor i := 0; i \u003c len(outArgs)-1; i++ {\n\t\t\t\toutArgs[i] = reflect.Zero(container.Type().Out(i))\n\t\t\t}\n\t\t\toutArgs[len(outArgs)-1] = reflect.ValueOf(\u0026err).Elem()\n\t\t\treturn outArgs\n\t\t}\n\n\t\t// Process input parameters\n\t\tinArgs := make([]interface{}, 0, len(req))\n\t\tfor _, arg := range req {\n\t\t\tinArgs = append(inArgs, arg.Interface())\n\t\t}\n\n\t\t// ReqRPC\n\t\treqRPC := RPCdata{Name: rpcName, Args: inArgs}\n\t\tb, err := Encode(reqRPC)\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t\terr = cReqTransport.Send(b)\n\t\tif err != nil {\n\t\t\treturn errorHandler(err)\n\t\t}\n\t\t// receive response from server\n\t\trsp, err := cReqTransport.Read()\n\t\tif err != nil { // local network error or decode error\n\t\t\treturn errorHandler(err)\n\t\t}\n\t\trspDecode, _ := Decode(rsp)\n\t\tif rspDecode.Err != \"\" { // remote server error\n\t\t\treturn errorHandler(errors.New(rspDecode.Err))\n\t\t}\n\n\t\tif len(rspDecode.Args) == 0 {\n\t\t\trspDecode.Args = make([]interface{}, container.Type().NumOut())\n\t\t}\n\t\t// unpackage response arguments\n\t\tnumOut := container.Type().NumOut()\n\t\toutArgs := make([]reflect.Value, numOut)\n\t\tfor i := 0; i \u003c numOut; i++ {\n\t\t\tif i != numOut-1 { // unpackage arguments (except error)\n\t\t\t\tif rspDecode.Args[i] == nil { // if argument is nil (gob will ignore \"Zero\" in transmission), set \"Zero\" value\n\t\t\t\t\toutArgs[i] = reflect.Zero(container.Type().Out(i))\n\t\t\t\t} else {\n\t\t\t\t\toutArgs[i] = reflect.ValueOf(rspDecode.Args[i])\n\t\t\t\t}\n\t\t\t} else { // unpackage error argument\n\t\t\t\toutArgs[i] = reflect.Zero(container.Type().Out(i))\n\t\t\t}\n\t\t}\n\n\t\treturn outArgs\n\t}\n\tcontainer.Set(reflect.MakeFunc(container.Type(), f))\n}\n```\n\n### Testing our framework\n\n```go\npackage main\n\nimport (\n\t\"encoding/gob\"\n\t\"fmt\"\n\t\"net\"\n)\n\ntype User struct {\n\tName string\n\tAge  int\n}\n\nvar userDB = map[int]User{\n\t1: User{\"Ankur\", 85},\n\t9: User{\"Anand\", 25},\n\t8: User{\"Ankur Anand\", 27},\n}\n\nfunc QueryUser(id int) (User, error) {\n\tif u, ok := userDB[id]; ok {\n\t\treturn u, nil\n\t}\n\n\treturn User{}, fmt.Errorf(\"id %d not in user db\", id)\n}\n\nfunc main() {\n\t// new Type needs to be registered\n\tgob.Register(User{})\n\taddr := \"localhost:3212\"\n\tsrv := NewServer(addr)\n\n\t// start server\n\tsrv.Register(\"QueryUser\", QueryUser)\n\tgo srv.Run()\n\n\t// wait for server to start.\n\ttime.Sleep(1 * time.Second)\n\n\t// start client\n\tconn, err := net.Dial(\"tcp\", addr)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tcli := NewClient(conn)\n\n\tvar Query func(int) (User, error)\n\tcli.callRPC(\"QueryUser\", \u0026Query)\n\n\tu, err := Query(1)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tfmt.Println(u)\n\n\tu2, err := Query(8)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tfmt.Println(u2)\n}\n```\n\nOutput\n\n```\n2019/07/23 20:26:18 func QueryUser is called\n{Ankur 85}\n2019/07/23 20:26:18 func QueryUser is called\n{Ankur Anand 27}\n```\n\n`go run main.go`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fankur-anand%2Fsimple-go-rpc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fankur-anand%2Fsimple-go-rpc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fankur-anand%2Fsimple-go-rpc/lists"}