{"id":21436761,"url":"https://github.com/modfin/yarf","last_synced_at":"2025-03-16T23:23:51.039Z","repository":{"id":57550401,"uuid":"308282670","full_name":"modfin/yarf","owner":"modfin","description":null,"archived":false,"fork":false,"pushed_at":"2023-02-18T03:31:40.000Z","size":245,"stargazers_count":1,"open_issues_count":1,"forks_count":0,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-01-23T09:33:58.686Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/modfin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-10-29T09:45:27.000Z","updated_at":"2022-12-06T11:23:10.000Z","dependencies_parsed_at":"2024-06-20T12:59:24.941Z","dependency_job_id":"4dfba232-17b8-45e0-83c9-69c1ec7017ce","html_url":"https://github.com/modfin/yarf","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modfin%2Fyarf","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modfin%2Fyarf/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modfin%2Fyarf/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modfin%2Fyarf/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/modfin","download_url":"https://codeload.github.com/modfin/yarf/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243945787,"owners_count":20372932,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-23T00:15:13.671Z","updated_at":"2025-03-16T23:23:51.008Z","avatar_url":"https://github.com/modfin.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![GoDoc](http://img.shields.io/badge/go-documentation-blue.svg?style=flat-square)](https://godoc.org/github.com/modfin/yarf)\n[![Go Report Card](https://goreportcard.com/badge/github.com/modfin/yarf)](https://goreportcard.com/report/github.com/modfin/yarf)\n[![License](http://img.shields.io/badge/license-mit-blue.svg?style=flat-square)](https://raw.githubusercontent.com/labstack/echo/master/LICENSE)\n\n\n[comment]: ( https://codecov.io )\n[comment]: ( Something ci )\n\n# yarf - Yet Another RPC Framework \nFor simple comunication between services\n\n## Motivation \nThere are a few rpc frameworks out there, so why one more. The simple\nanswer is that may of them out there, such as gRPC, Twirp and so on did\nnot fit our need. They are often very opinionated, overly complex and\nare in some cases much of a black box trying to solve every problem in\nmany languages.\n\nWhat we found was that we were writing models i protobuf to be used in\nframeworks such as gRPC and then on both client and server side had to\nmap them into local structs since protobuf was not expresive enough.\nThis instead of just having a versioned struct in a common kit repo\nor such. In essens fighting with the rpc libs we tired in order for it\nto work with our use cases.\n\n## Overview\nYarf is a rpc framework focusing on ease of use and clear options of\nhow to use.\n\n\n## Features\n* Separation between protocol and transport\n* Support for synchronise calls, channals and callbacks\n* Support for middleware on both client and server side\n* Support for context passing between client and server\n* Support for Custom serializing, both of protocol and content level.\n* Expressiv builder pattern fron client calls\n* Client and server Middleware\n\n\n## Supported transport layers\n* http\n* nats\n\n## Quickstart\nSee examples for more examples/simple\n\n### Intallation\n```\ngo get github.com/modfin/yarf\ngo get github.com/modfin/yarf/...\n```\n\n### Server\n```go\npackage main\nimport (\n    \"github.com/modfin/yarf\"\n    \"github.com/modfin/yarf/transport/thttp\"\n    \"log\"\n)\n\n\nfunc SHA256(req *yarf.Msg, resp *yarf.Msg) (err error) {\n\n    hash := hashing.Sum256(req.Content)\n    resp.SetParam(\"hash\",\n        base64.StdEncoding.EncodeToString(hash[:]))\n    return\n}\n\nfunc join(req *yarf.Msg, resp *yarf.Msg) (err error) {\n\n    slice, ok := req.Param(\"slice\").StringSlice()\n    if !ok{\n        return errors.New(\"param arr was not a string slice\")\n    }\n\n    joined := strings.Join(slice, \"\")\n\n    resp.SetContent(joined)\n\n    return nil\n}\n\nfunc main(){\n\n\ttransport, err := thttp.NewHttpTransporter(thttp.Options{})\n\tif err != nil {\n        log.Fatal(err)\n    }\n    server := yarf.NewServer(transport, \"a\", \"namespace\")\n\n    server.HandleFunc(SHA256)\n    server.HandleFunc(join)\n    server.Handle(\"add\", func(req *yarf.Msg, resp *yarf.Msg) (err error){\n        res := req.Param(\"val1\").IntOr(0)+req.Param(\"val2\").IntOr(0)\n        resp.SetParam(\"res\", res)\n        return nil\n    })\n\n\n\n    \n    log.Fatal(transport.Start())\n}\n```\n\n### Client\n```go\npackage main\nimport (\n    \"github.com/modfin/yarf\"\n    \"github.com/modfin/yarf/transport/thttp\"\n    \"log\"\n    \"fmt\"\n)\nfunc main(){\n\n    transport, err := thttp.NewHttpTransporter(thttp.Options{Discovery: \u0026thttp.DiscoveryDefault{Host:\"localhost\"}})\n    if err != nil {\n        log.Fatal(err)\n    }\n    client := yarf.NewClient(transport)\n\n\n    // Sending and reciving params\n    msg, err := client.Request(\"a.namespace.add\").\n        WithParam(\"val1\", 5).\n        WithParam(\"val2\", 7).\n        Get()\n    \n    if err != nil{\n        log.Fatal(err)\n    }\n    fmt.Println(\" Result of 5 + 7 =\", res.Param(\"res\").IntOr(-1))\n\n\n    //Sending binary content\n    msg, err = client.Request(\"a.namespace.SHA256\").\n        WithBinaryContent([]byte(\"Hello Yarf\")).\n        Get()\n\n    if err != nil {\n        return \"\", err\n    }\n\n    hash, ok := msg.Param(\"hash\").String()\n    fmt.Println(\"ok\", ok, \"hash\", hash)\n\n\n    var joined string\n\n    // Binding response content to string (works with structs, slices and so on)\n    err = client.Request(\"a.namespace.join\").\n        WithParam(\"slice\", []string{\"jo\", \"in\", \"ed\"}).\n        BindResponseContent(\u0026s).\n        Done()\n\n    fmt.Println(\"joined\", joined, \"err\", err)\n\n    // or\n    err = client.Call(\"a.namespace.join\", nil, \u0026joined, yarf.NewParam(\"slice\", []string{\"joi\", \"ned\"}))\n\n    fmt.Println(\"joined\", joined, \"err\", err)\n}\n\n```\n\n### Test\n`go test -v ./...`\n`./test.sh`, docker is requierd to run integration tests\n\n\n\n## Design\n\nYarf consist in large of 5 things, API, Middleware, Protocol, Serlization\nand Transport. We try to provide good defaults that both preform and are esay to use.\nBut, as most of us know, its not rare to end up in a corner case where things\nhave to be flexible. Therefore we try to seperate things in a clear way\nthat is easy to extend and change.\n\n\n### API\nOur main api that we provide for users of yarf is made up by a client, server\nand the messages that pass between the two.\n\n\n### Middleware\nYarf has support for middleware on both the client and server side of request\nand both of them expect the same function type. Middleware can be use\nwhen the need to decorate a message arise, do logging, caching, authentication or\nanything else reqiering interception of messages\n\n\ne.g. simple time logging\n```go\n func(request *yarf.Msg, response *yarf.Msg, next yarf.NextMiddleware) error {\n\n\t\t// Runs before client requst\n        start = time.Now()\n\n\t\terr := next() // Running other middleware and request to server\n\n\t\t// Runs after client request\n\t\telapsed := time.Now().Sub(start)\n        fmt.Println(\"Request to function\", request.Function(), \"took\", elapsed)\n\n\t\treturn err\n\t}\n```\n\n\ne.g. a simple server side caching\n```go\n func(request *yarf.Msg, response *yarf.Msg, next yarf.NextMiddleware) error {\n\n\t\t// Runs before handler function\n\n\t\tkey, ok := request.Param(\"cachekey\").String()\n\t\tif !ok {\n\t\t   // could not find cachekey in message, runs anyway\n\t\t   return next()\n\t\t}\n\n\t\tb, found := getCachedItem(key)\n\n\t\tif found{\n\t\t    response.SetBinaryContent(b)\n\t\t    return nil\n\t\t}\n\n\t\terr := next() // Running other middleware and handler\n\n\t\t// Runs after handler function\n\n        if err != nil{\n            return err\n        }\n\n        setCachedItem(\"key\", response.Content)\n\n\t\treturn err\n\t}\n```\n\nOn a client, middleware can be set on per request bases (`local`) or for all request going through\nthe client (`global`) and are run as follows\n\n```\nClient\n   |\n   V\n   Global0 --\u003e Global1 --\u003e Local0 --\u003e Local1 --\u003e Call to transport layer\n                                                        |\n                                                        V\n   Global0 \u003c-- Global1 \u003c-- Local0 \u003c-- Local1 \u003c-- Response from transport layer\n   |\n   V\nClient\n\n```\n\nOn the server, middleware can be set on per handler bases (`local`) or for all reguest going through\nthe server (`global`) and are run as follow\n\n```\nIncomming from transport layer\n   |\n   V\n   Global0 --\u003e Global1 --\u003e Local0 --\u003e Local1 --\u003e Running handler function\n                                                      |\n                                                      V\n   Global0 \u003c-- Global1 \u003c-- Local0 \u003c-- Local1 \u003c-- Response from handler function\n   |\n   V\nOutgoing to transport layer\n```\n\n\n## Protocol\nThe yarf protocol is pretty straight forward but has a few layers to it.\nIt is not really that interesting unless you for some reason whant your\nown serilization or need to change it\n\nA message sent between client and server always start with a string prepended\nby a newline, \"\\n\", followed by bytes.\n\nThe first line describes the content type of the followig bytes, this\nin order to deserlize the message into a yarf.Msg. This exist in order\nto allow for different ways of (de)serlizing a message\n\nThe following bytes are then deserlized into the follwoing struct\n```go\ntype Msg struct {\n\tHeaders map[string]interface{}\n\tContent []byte\n}\n```\n\nThe message struct header field contain mutiple keys that is usefull for the\nyarf, but can also be used to pass paremeters. I shall howerver always contain\na content-type which helps yarf in dezerlization of Content, if needed.\n\n\n\n## Serialization\n\nThis might be a some what confusing topic since there is a few layers to it.\nBut in general there are only two that has to be considerd. The serialization\nof the protocol and the serialization of the content.\n\nFor both the the protocol and the content a content type shall aways be provided.\nThis helps the reciver of the message to deserialize the message and the content.\n\nSerlizations can be done in different combinations and independet of each other.\nThe default is **msgpack** for both but can be changed per client, server or message basis.\nSince it is might be hard to track what server has what and so on, serializers can\nbe regiserd in yarf by `yarf.RegisterSerializer(serializer)`. Yarf also provde some\nextras ones, msgpack and json\n\n\n\n## Transport\nThe transport layer works independetly of everything else and is responsibel\nfor service discovery, transport of data and provide to a context that can be\ncanceld from the client to the server.\n\nA implmentation using Nats and one using HTTP is provided with yarf.\n\nA transporter shall implment a rather simple api in order to work with yarf.\nBut since different transporters have different properties, some thinsgs may\nvary. e.g. the function namespacing using Nats is a global and has no real need\nfor service discover, while HTTP has local namespace for each specific serivece.\n\n\n\n## TODO\n* Unit testing\n* More documentation\n* Add support for reader and writers, for streaming requests/responses\n* Http Transport\n    * Improving service discover on HTTP transport\n        * Consul\n        * etcd\n        * DNS A\n        * DNS SRV\n    * Improving loadbalancing on HTTP transport\n    * Support for http2 and tls transport\n* Middlewares\n    * Proper Logging\n    * Statistics and latency collection\n    * Circuit breakers\n    * Caching\n    * Authentictaion, JWT","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmodfin%2Fyarf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmodfin%2Fyarf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmodfin%2Fyarf/lists"}