https://github.com/cloudwego/frugal
A very fast dynamic Thrift serializer & deserializer.
https://github.com/cloudwego/frugal
Last synced: about 1 month ago
JSON representation
A very fast dynamic Thrift serializer & deserializer.
- Host: GitHub
- URL: https://github.com/cloudwego/frugal
- Owner: cloudwego
- License: apache-2.0
- Created: 2021-07-28T13:08:43.000Z (almost 4 years ago)
- Default Branch: main
- Last Pushed: 2025-04-01T04:04:14.000Z (3 months ago)
- Last Synced: 2025-04-12T04:49:39.660Z (2 months ago)
- Language: Go
- Size: 1.23 MB
- Stars: 124
- Watchers: 15
- Forks: 21
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project
README
# Frugal
English | [中文](README_cn.md)
A very fast dynamic Thrift serializer & deserializer without generating code.
## Features
### Code Generation Free
Traditional Thrift serializer and deserializer are based on generated code which is no longer needed since we can make use of struct field tags.
### High Performance
Based on the test cases in `frugal/tests`, Frugal's performance is 3 to 4 times better than Apache Thrift (TBinaryProtocol).
There may be variations between different test cases. Feel free to share your test cases with us.
```text
go version go1.23.6 linux/amd64
goos: linux
goarch: amd64
pkg: github.com/cloudwego/frugal/tests
cpu: Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz
Marshal_ApacheThrift/small-4 3468714 346.1 ns/op 1684.32 MB/s 0 B/op 0 allocs/op
Marshal_ApacheThrift/medium-4 128386 9343 ns/op 1875.07 MB/s 0 B/op 0 allocs/op
Marshal_ApacheThrift/large-4 7208 164521 ns/op 1845.68 MB/s 109 B/op 0 allocs/op
Marshal_Frugal/small-4 13032746 92.45 ns/op 6306.09 MB/s 0 B/op 0 allocs/op
Marshal_Frugal/medium-4 327564 3669 ns/op 4774.38 MB/s 0 B/op 0 allocs/op
Marshal_Frugal/large-4 18751 64212 ns/op 4728.90 MB/s 0 B/op 0 allocs/op
Unmarshal_ApacheThrift/small-4 1548732 774.1 ns/op 753.15 MB/s 1120 B/op 4 allocs/op
Unmarshal_ApacheThrift/medium-4 42676 30665 ns/op 571.27 MB/s 44704 B/op 175 allocs/op
Unmarshal_ApacheThrift/large-4 2106 515642 ns/op 588.88 MB/s 775936 B/op 3030 allocs/op
Unmarshal_Frugal/small-4 4963635 266.2 ns/op 2189.92 MB/s 544 B/op 1 allocs/op
Unmarshal_Frugal/medium-4 99786 11321 ns/op 1547.45 MB/s 19908 B/op 57 allocs/op
Unmarshal_Frugal/large-4 5838 197987 ns/op 1533.69 MB/s 349252 B/op 997 allocs/op
```
## What can you do with Frugal ?
### Use Frugal as [Kitex](https://github.com/cloudwego/kitex) serializer and deserializer
No more massive serialization and deserialization code, leads to a more tidy project. No more meaningless diff of generated code in code review.
### Serialized and Deserialize struct generated by [Thriftgo](https://github.com/cloudwego/thriftgo)
If you have a Thrift file, and all you need is using Frugal to do serialization and deserialization. You can use thriftgo to generate Go struct, then you can use Frugal.
### Serialization and deserialization on a customized Go struct
If you don't want any Thrift files, and you want serialize or deserialize a customized Go struct. You can add some struct field tag to the Go struct, then you can use Frugal.
## Usage
### Using with Kitex
#### 1. Update Kitex to v0.4.2 or higher version
```shell
go get github.com/cloudwego/kitex@latest
```
#### 2. Generate code with `-thrift frugal_tag` option
Example:
```shell
kitex -thrift frugal_tag -service a.b.c my.thrift
```
If you don't need codec code, you can use `-thrift template=slim` option.
```shell
kitex -thrift frugal_tag,template=slim -service a.b.c my.thrift
```
#### 3. Init clients and servers with `WithPayloadCodec(thrift.NewThriftFrugalCodec())` option
Client example:
```go
package client
import (
"context"
"example.com/kitex_test/client/kitex_gen/a/b/c/echo"
"github.com/cloudwego/kitex/client"
"github.com/cloudwego/kitex/pkg/remote/codec/thrift"
)
func Echo() {
code := thrift.NewThriftCodecWithConfig(thrift.FastRead | thrift.FastWrite | thrift.FrugalRead | thrift.FrugalWrite)
cli := echo.MustNewClient("a.b.c", client.WithPayloadCodec(codec))
...
}
```
Server example:
```go
package main
import (
"log"
"github.com/cloudwego/kitex/server"
c "example.com/kitex_test/kitex_gen/a/b/c/echo"
"github.com/cloudwego/kitex/pkg/remote/codec/thrift"
)
func main() {
code := thrift.NewThriftCodecWithConfig(thrift.FastRead | thrift.FastWrite | thrift.FrugalRead | thrift.FrugalWrite)
svr := c.NewServer(new(EchoImpl), server.WithPayloadCodec(code))
err := svr.Run()
if err != nil {
log.Println(err.Error())
}
}
```
### Using with Thrift IDL
#### Prepare Thrift file
We can define a struct in Thrift file like below:
my.thrift:
```thrift
struct MyStruct {
1: string msg
2: i64 code
}
```
#### Use Thriftgo to generate code
Now we have thrift file, we can use Thriftgo with `frugal_tag` option to generate Go code.
Example:
```shell
thriftgo -r -o thrift -g go:frugal_tag,package_prefix=example.com/kitex_test/thrift my.thrift
```
If you don't need codec code, you can use `template=slim` option
```shell
thriftgo -r -o thrift -g go:frugal_tag,template=slim,package_prefix=example.com/kitex_test/thrift my.thrift
```
#### Use Frugal to serialize or deserialize
Now we can use Frugal to serialize or deserialize the struct defined in thrift file.
Example:
```go
package main
import (
"github.com/cloudwego/frugal"
"example.com/kitex_test/thrift"
)
func main() {
ms := &thrift.MyStruct{
Msg: "my message",
Code: 1024,
}
...
buf := make([]byte, frugal.EncodedSize(ms))
frugal.EncodeObject(buf, nil, ms)
...
got := &thrift.MyStruct{}
frugal.DecodeObject(buf, got)
...
}
```
### Serialization and deserialization on a customized Go struct
#### Define a Go struct
We can define a struct like this:
```go
type MyStruct struct {
Msg string
Code int64
Numbers []int64
}
```
#### Add Frugal tag to struct fields
Frugal tag is like `frugal:"1,default,string"`, `1` is field ID, `default` is field requiredness, `string` is field type. Field ID and requiredness is always required, but field type is only required for `list`, `set` and `enum`.
You can add Frugal tag to `MyStruct` like below:
```go
type MyStruct struct {
Msg string `frugal:"1,default"`
Code int64 `frugal:"2,default"`
Numbers []int64 `frugal:"3,default,list"`
}
```
All types example:
```go
type MyEnum int64
type Example struct {
MyOptBool *bool `frugal:"1,optional"`
MyReqBool bool `frugal:"2,required"`
MyOptByte *int8 `frugal:"3,optional"`
MyReqByte int8 `frugal:"4,required"`
MyOptI16 *int16 `frugal:"5,optional"`
MyReqI16 int16 `frugal:"6,required"`
MyOptI32 *int32 `frugal:"7,optional"`
MyReqI32 int32 `frugal:"8,required"`
MyOptI64 *int64 `frugal:"9,optional"`
MyReqI64 int64 `frugal:"10,required"`
MyOptString *string `frugal:"11,optional"`
MyReqString string `frugal:"12,required"`
MyOptBinary []byte `frugal:"13,optional"`
MyReqBinary []byte `frugal:"14,required"`
MyOptI64Set []int64 `frugal:"15,optional,set"`
MyReqI64Set []int64 `frugal:"16,required,set"`
MyOptI64List []int64 `frugal:"17,optional,list"`
MyReqI64List []int64 `frugal:"18,required,list"`
MyOptI64StringMap map[int64]string `frugal:"19,optional"`
MyReqI64StringMap map[int64]string `frugal:"20,required"`
MyOptEnum *MyEnum `frugal:"21,optional,i64"`
MyReqEnum *MyEnum `frugal:"22,optional,i64"`
}
```
#### Use Frugal to serialize or deserialize
Example:
```go
package main
import (
"github.com/cloudwego/frugal"
)
func main() {
ms := &thrift.MyStruct{
Msg: "my message",
Code: 1024,
Numbers: []int64{0, 1, 2, 3, 4},
}
...
buf := make([]byte, frugal.EncodedSize(ms))
frugal.EncodeObject(buf, nil, ms)
...
got := &thrift.MyStruct{}
frugal.DecodeObject(buf, got)
...
}
```