{"id":19870595,"url":"https://github.com/yycoder/thrifter","last_synced_at":"2025-03-01T00:35:42.922Z","repository":{"id":57636415,"uuid":"401300322","full_name":"YYCoder/thrifter","owner":"YYCoder","description":"Non-destructive thrift parser with zero third-party dependency.","archived":false,"fork":false,"pushed_at":"2023-11-09T14:43:01.000Z","size":118,"stargazers_count":1,"open_issues_count":1,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-11T16:29:38.372Z","etag":null,"topics":["ast","formatter","golang","golang-package","parser","thrift","thrift-parser"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/YYCoder.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-08-30T10:16:48.000Z","updated_at":"2024-08-13T16:56:37.000Z","dependencies_parsed_at":"2024-06-20T17:20:01.199Z","dependency_job_id":"a713d51c-4826-4497-af57-ca72cc8e18a3","html_url":"https://github.com/YYCoder/thrifter","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YYCoder%2Fthrifter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YYCoder%2Fthrifter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YYCoder%2Fthrifter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YYCoder%2Fthrifter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/YYCoder","download_url":"https://codeload.github.com/YYCoder/thrifter/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241292162,"owners_count":19939561,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ast","formatter","golang","golang-package","parser","thrift","thrift-parser"],"created_at":"2024-11-12T16:09:30.153Z","updated_at":"2025-03-01T00:35:42.896Z","avatar_url":"https://github.com/YYCoder.png","language":"Go","readme":"# thrifter\n**Non-destructive** parser/printer for [thrift](https://thrift.apache.org/docs/types.html) with zero third-party dependency.\n\n[![YYCoder](https://circleci.com/gh/YYCoder/thrifter.svg?style=svg)](https://app.circleci.com/pipelines/github/YYCoder/thrifter)\n[![goreportcard](https://goreportcard.com/badge/github.com/yycoder/thrifter)](https://goreportcard.com/report/github.com/yycoder/thrifter)\n[![GoDoc](https://pkg.go.dev/badge/github.com/YYCoder/thrifter)](https://pkg.go.dev/github.com/YYCoder/thrifter)\n[![Codecov](https://codecov.io/gh/YYCoder/thrifter/branch/master/graph/badge.svg)](https://codecov.io/gh/YYCoder/thrifter)\n[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](http://makeapullrequest.com)\n\n[中文文档](./docs/cn.md)\n\n## Inspiration\nThere are several thrift parsers on github, but each of them have issues on preserve original format, since mostly, they are used to generate rpc code. But, this project serves different purpose, which is focus on helping you write thrift code more efficiently, thanks to its non-destructive code transformation. Since it's a non-destructive parser, we can do a lot of stuff on top of it, such as **code formatting**, **code transforming**, etc.\n\nCurrently, it's mainly used by my other little project called [protobuf-thrift](https://github.com/YYCoder/protobuf-thrift), which is a code transformer between protobuf and thrift.\n\nHere are some other thrift parsers on github I discovered before start protobuf-thrift, none of them is 100 percent suitable for it.\n\n### Similar Packages\n1. [go-thrift](https://github.com/samuel/go-thrift): mainly used to generate rpc code, ignore white space, comments and lose statements order\n\n2. [thriftrw-go](https://github.com/thriftrw/thriftrw-go): thrift parser and code generator which open sourced by Uber, same issue as above\n\n3. [thriftgo](https://github.com/cloudwego/thriftgo): another thrift parser and code generator, same issue as above\n\n4. [thrift-parser](https://github.com/creditkarma/thrift-parser): thrift parser written in typescript, same issue\n\nSo, that's why I started thinking of writing a new thrift parser which preserves all the format.\n\nThanks to [rocambole](https://github.com/millermedeiros/rocambole), behind which idea is perfect for this project.\n\n### Core Concept\nThe main idea behind thrifter on achieve non-destructive is that, we use a linked-list to chain all the tokens.\n\nLets think of the essence of source code, it's just a chain of token, different token combination declare different syntax, so if we want to preserve original format, we must preserve all tokens from the source code.\n\nThe best data structure for this chain of tokens is linked-list, since it's easier to modify than array, we only need to change some pointer, and we can patch start token and end token to each ast node, so that we are able to easily iterate over tokens within a node.\n\nWhen iterate over token linked-list, we also provide a Map structure for each ContainerType, such as enum/struct/service, in order to find the field node started by the token.\n\n## Usage\nInitialize Parser first, and specify the io.Reader which consume source code:\n\n```go\nparser := thrifter.NewParser(strings.NewReader(XXX), false)\n// or\nfile, err := os.Open(XXX)\nif err != nil {\n   return nil, err\n}\ndefer file.Close()\nparser := thrifter.NewParser(file, false)\n```\n\nthen, simply use `parser.Parse` to start parsing:\n\n```go\ndefinition, err := parser.Parse(YOUR_FILE_NAME)\n```\n\nand that's it, now we have the root node for the source code, which structure like this:\n\n```go\ntype Thrift struct {\n\tNodeCommonField\n\t// thrift file name, if it exists\n\tFileName string\n\t// since Thrift is the root node, we need a property to access its children\n\tNodes []Node\n}\n```\n\nYou might wonder what the hell is `NodeCommonField` nested into Thrift node, that's the magic of thrifter, we will discuss it in the **AST Node** section.\n\n### Code Print\nThe most amazing thing about thrifter is that, it is also a non-destructive code printer.\n\nThink about this case, When you want to write a code generator to optimize your workflow, normally you would use a code parser to get the code ast, and then manipulate it. Under some circumstances, you merely want to add some new code to it and leave the rest intact, normal code parser could not able to do that, since they will ignore whitespace like line-breaks/indents.\n\nWith thrifter, you can just initialize your new code ast node, and then patch the `StartToken` to the original ast's token linked-list, all other code is unchanged, like this:\n\n```go\n// 1. initialize new node, enum, for instance\n// for simplicity, you can just initialize a parser to parse the code you want to generate, in order to get the code tokens linked-list\np := thrifter.NewParser(`enum a {\n    A = 1\n    B = 2;\n    C\n    D;\n}`, false)\nstartTok := parser.next() // consume enum token\nenumNode := NewEnum(startTok, nil)\nif err := enumNode.parse(p); err != nil {\n   t.Errorf(\"unexpected error: %v\", err)\n   return\n}\n\n// 2. patch the generated code StartToken to any where you want to put it\npreNext := someNodeFromOriginalCode.EndToken.Next\nsomeNodeFromOriginalCode.EndToken.Next = enumNode.StartToken\nenumNode.EndToken.Next = preNext\n\n// 3. last, use node.String to print the code\nfmt.Println(thriftNodeFromOriginalCode.String())\n```\n\nEach `thrifter.Node` have their own `String` function, so, you can also print the node standalone not the whole thrift file.\n\nThe principle of `String` is pretty simple, it just traverse the token and write them one by one:\n\n```go\nfunc toString(start *Token, end *Token) string {\n\tvar res bytes.Buffer\n\tcurr := start\n\tfor curr != end {\n\t\tres.WriteString(curr.Raw)\n\t\tcurr = curr.Next\n\t}\n\tres.WriteString(end.Raw)\n\treturn res.String()\n}\n```\n\nNote that, when you manipulate original ast like above, the original `Thrift.Nodes` fields is unchanged, but it doesn't affect code print, since it only iterate over tokens, not nodes. However, you can manually add the node to `Thrift.Nodes` by yourself for consistency.\n\n### AST Node\nTo understand the idea behind thrifter, there are two struct and one interface you must know:\n\n```go\ntype NodeCommonField struct {\n\tParent     Node\n\tNext       Node\n\tPrev       Node\n\tStartToken *Token\n\tEndToken   *Token\n}\n\ntype Token struct {\n\tType  token\n\tRaw   string // tokens raw value, e.g. comments contain prefix, like // or /* or #; strings contain ' or \"\n\tValue string // tokens transformed value\n\tNext  *Token\n\tPrev  *Token\n\tPos   scanner.Position\n}\n\ntype Node interface {\n\t// recursively output current node and its children\n\tString() string\n\t// recursively parse current node and its children\n\tparse(p *Parser) error\n\t// get node value\n\tNodeValue() interface{}\n\t// get node type, value specified from each node\n\tNodeType() string\n}\n```\n\nFirstly, `NodeCommonField` is the basic of achieving non-destructive, it will be nested into each ast node, whatever `node.Type` is. These two fields are essential:\n\n* **StartToken**: the start token of the node, which means you can easily iterate over tokens within the node\n\n* **EndToken**: the end token of the node, when iteration within node reaches it, means iteration is done\n\nSecond struct `Token` represents a basic token of thrifter, a token can be a symbol, e.g. `-` or `+`, or string literal `\"abc\"` or `'abc'`, and also a identifier.\n\n\u003e Note that, thrifter considers comment as a token, not a node, currently. I'm not entirely sure it is a good idea, so if some one have questions about it, please open an issue.\n\nAnd the last interface `Node` represents a thrifter node. Since it's a interface, if you want to access the node fields, you can use `NodeType` to get the type of node, and then do a type assertion of the node:\n\n```go\nfor _, node := range thrift.Nodes {\n    switch node.NodeType() {\n    case \"Namespace\":\n        n := node.(*thrifter.Namespace)\n        fmt.Printf(\"Namespace: %+v\", n)\n    case \"Enum\":\n        n := node.(*thrifter.Enum)\n        fmt.Printf(\"Enum: %+v\", n)\n    case \"Struct\":\n        n := node.(*thrifter.Struct)\n        fmt.Printf(\"Struct: %+v\", n)\n    case \"Service\":\n        n := node.(*thrifter.Service)\n        fmt.Printf(\"Service: %+v\", n)\n    case \"Include\":\n        n := node.(*thrifter.Include)\n        fmt.Printf(\"Include: %+v\", n)\n    }\n}\n```\n\n\n## Notice\n1. `senum` not supported: since thrift officially don't recommend to use it, thrifter will not handle it, too.\n\n2. current parser implementation is not completely validating `.thrift` definitions, since we think validation feature is better to leave to specific linter.\n\n## Related Packages\nSome packages build on top of thrifter:\n\n* [protobuf-thrift](https://github.com/YYCoder/protobuf-thrift): transforming protobuf idl to thrift, and vice versa.\n\n## Contribution\n**Working on your first Pull Request?** You can learn how from this *free* series [How to Contribute to an Open Source Project on GitHub](https://kcd.im/pull-request).\n\n### TODO\n- [] support comment node\n- [] Thrift node support `ElemsMap` to map start token to each element node","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyycoder%2Fthrifter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyycoder%2Fthrifter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyycoder%2Fthrifter/lists"}