{"id":13709528,"url":"https://github.com/src-d/proteus","last_synced_at":"2025-04-04T13:13:51.670Z","repository":{"id":49361840,"uuid":"76378791","full_name":"src-d/proteus","owner":"src-d","description":"Generate .proto files from Go source code.","archived":false,"fork":false,"pushed_at":"2020-10-19T17:17:47.000Z","size":286,"stargazers_count":738,"open_issues_count":24,"forks_count":69,"subscribers_count":12,"default_branch":"master","last_synced_at":"2025-03-28T12:01:36.824Z","etag":null,"topics":["go","grpc","protobuf","protobuf3"],"latest_commit_sha":null,"homepage":"https://blog.sourced.tech/post/proteus/","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/src-d.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-12-13T16:43:42.000Z","updated_at":"2025-03-23T00:47:34.000Z","dependencies_parsed_at":"2022-08-12T20:10:51.843Z","dependency_job_id":null,"html_url":"https://github.com/src-d/proteus","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/src-d%2Fproteus","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/src-d%2Fproteus/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/src-d%2Fproteus/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/src-d%2Fproteus/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/src-d","download_url":"https://codeload.github.com/src-d/proteus/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247174415,"owners_count":20896078,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","grpc","protobuf","protobuf3"],"created_at":"2024-08-02T23:00:40.908Z","updated_at":"2025-04-04T13:13:51.647Z","avatar_url":"https://github.com/src-d.png","language":"Go","readme":"# ![proteus](https://rawgit.com/src-d/proteus/master/proteus.svg)\n\n[![GoDoc](https://godoc.org/github.com/src-d/proteus?status.svg)](https://godoc.org/github.com/src-d/proteus) [![Build Status](https://travis-ci.org/src-d/proteus.svg?branch=master)](https://travis-ci.org/src-d/proteus) [![codecov](https://codecov.io/gh/src-d/proteus/branch/master/graph/badge.svg)](https://codecov.io/gh/src-d/proteus) [![License](http://img.shields.io/:license-mit-blue.svg)](http://doge.mit-license.org) [![Go Report Card](https://goreportcard.com/badge/github.com/src-d/proteus)](https://goreportcard.com/report/github.com/src-d/proteus) [![codebeat badge](https://codebeat.co/badges/976ff535-c79b-429d-b35c-888a048a3201)](https://codebeat.co/projects/github-com-src-d-proteus)\n\n**Notice: This repository is abandoned, and no further updates will be done on the code base, nor issues/PRs will be answered or attended.**\n\n[Proteus](https://en.wikipedia.org/wiki/Proteus) /proʊtiəs/ is a tool to generate protocol buffers version 3 compatible `.proto` files from your Go structs, types and functions.\n\nThe motivation behind this library is to use Go as a source of truth for your models instead of the other way around and then generating Go code from a `.proto` file, which does not generate idiomatic code.\n\nProteus scans all the code in the selected packages and generates protobuf messages for every exported struct (and all the ones that are referenced in any other struct, even though they are not exported). The types that semantically are used as enumerations in Go are transformed into proper protobuf enumerations.\nAll the exported functions and methods will be turned into protobuf RPC services.\n\nWe want to build proteus in a very extensible way, so every step of the generation can be hackable via plugins and everyone can adapt proteus to their needs without actually having to integrate functionality that does not play well with the core library. We are releasing the plugin feature after Go 1.8 is released, which includes the `plugin` package of the standard library.\n\nFor an overall overview of the code architecture take a look at [the architecture documentation](/ARCHITECTURE.md).\n\nYou can read more about the motivations behind building proteus in [this blog post](https://blog.sourced.tech/post/proteus/).\n\n### Install\n\n```\ngo get -v gopkg.in/src-d/proteus.v1/...\n```\n\n### Requirements\n\nThere are two requirements for the full process.\n\n* [`protoc`](https://github.com/google/protobuf) binary installed on your path\n* `go get -u github.com/gogo/protobuf/...`\n\n### Usage\n\nYou can generate the proto files, the marshal/unmarshal and the rest of protobuf stuff for your Go types, the RPC client and server interface and the RPC server implementation for your packages. That is, the whole process.\n\n```bash\nproteus -f /path/to/protos/folder \\\n        -p my/go/package \\\n        -p my/other/go/package\n```\n\nYou can generate proto files only using the command line tool provided with proteus.\n\n```bash\nproteus proto -f /path/to/output/folder \\\n        -p my/go/package \\\n        -p my/other/go/package\n        --verbose\n```\n\nYou can also only generate gRPC server implementations for your packages.\n\n```bash\nproteus rpc -p my/go/package \\\n        -p my/other/go/package\n```\n\n**NOTE:** Of course, if the defaults don't suit your needs, until proteus is extensible via plugins, you can hack together your own generator command using the provided components. Check out the [godoc documentation of the package](http://godoc.org/github.com/src-d/proteus).\n\n### Generate protobuf messages\n\nProteus will generate protobuf messages with the structure of structs with the comment `//proteus:generate`. Obviously, the structs have to be exported in Go (first letter must be in upper case).\n\n```go\n//proteus:generate\ntype Exported struct {\n        Field string\n}\n\ntype NotExported struct {\n        Field string\n}\n```\n\n**Generated by requirement**\n\nNote that, even if the struct is not explicitly exported, if another struct has a field of that type, it will be generated as well.\n\n```go\n//proteus:generate\ntype Preference struct {\n        Name string\n        Value string\n        Options Options\n}\n\ntype Options struct {\n        Enabled bool\n}\n```\n\nIn that example, even if `Options` is not explicitly generated, it will be because it is required to generate `Preference`.\n\nSo far, this does not happen if the field is an enum. It is a known problem and we are working on fixing it. Until said fix lands, please, explicitly mark enums to be generated.\n\n**Struct embedding**\n\nYou can embed structs as usual and they will be generated as if the struct had the fields of the embedded struct.\n\n```go\n//proteus:generate\ntype User struct {\n        Model\n        Username string\n}\n\ntype Model struct {\n        ID int\n        CreatedAt time.Time\n}\n```\n\nThis example will generate the following protobuf message.\n\n```\nmessage User {\n        int32 id = 1;\n        google.protobuf.Timestamp created_at = 2;\n        string username = 3;\n}\n```\n\n**CAUTION:** if you redefine a field, it will be ignored and the one from the embedded will be taken. Same thing happens if you embed several structs and they have repeated fields. This may change in the future, for now this is the intended behaviour and a warning is printed.\n\n**Ignore specific fields**\n\nYou can ignore specific fields using the struct tag `proteus:\"-\"`.\n\n```go\n//proteus:generate\ntype Foo struct {\n        Bar int\n        Baz int `proteus:\"-\"`\n}\n```\n\nThis becomes:\n\n```\nmessage Foo {\n        int32 bar = 1;\n}\n```\n\n### Generating enumerations\n\nYou can make a type declaration (not a struct type declaration) be exported as an enumeration, instead of just an alias with the comment `//proteus:generate`.\n\n```go\n//proteus:generate\ntype Status int\n\nconst (\n        Pending Status = iota\n        Active\n        Closed\n)\n```\n\nThis will generate:\n\n```\nenum Status {\n        PENDING = 0;\n        ACTIVE = 1;\n        CLOSED = 2;\n}\n```\n\n**NOTE:** protobuf enumerations require you to start the enumeration with 0 and do not have gaps between the numbers. So keep that in mind when setting the values of your consts.\n\nFor example, if you have the following code:\n\n```go\ntype PageSize int\n\nconst (\n        Mobile PageSize = 320\n        Tablet PageSize = 768\n        Desktop PageSize = 1024\n)\n```\n\nInstead of doing an enumeration, consider not exporting the type and instead it will be treated as an alias of `int32` in protobuf, which is the default behaviour for not exported types.\n\n### Generate services\n\nFor every package, a single service is generated with all the methods or functions having `//proteus:generate`.\n\nFor example, if you have the following package:\n\n```go\npackage users\n\n//proteus:generate\nfunc GetUser(id uint64) (*User, error) {\n        // impl\n}\n\n//proteus:generate\nfunc (s *UserStore) UpdateUser(u *User) error {\n        // impl\n}\n```\n\nThe following protobuf service would be generated:\n\n```proto\nmessage GetUserRequest {\n        uint64 arg1 = 1;\n}\n\nmessage UserStore_UpdateUserResponse {\n}\n\nservice UsersService {\n        rpc GetUser(users.GetUserRequest) returns (users.User);\n        rpc UserStore_UpdateUser(users.User) returns (users.UserStore_UpdateUserResponse);\n}\n```\n\nNote that protobuf does not support input or output types that are not messages or empty input/output, so instead of returning nothing in `UserStore_UpdateUser` it returns a message with no fields, and instead of receiving an integer in `GetUser`, receives a message with only one integer field.\nThe last `error` type is ignored.\n\n### Generate RPC server implementation\n\n`gogo/protobuf` generates the interface you need to implement based on your `.proto` file. The problem with that is that you actually have to implement that and maintain it. Instead, you can just generate it automatically with proteus.\n\nConsider the Go code of the previous section, we could generate the implementation of that service.\n\nSomething like this would be generated:\n\n```\ntype usersServiceServer struct {\n}\n\nfunc NewUsersServiceServer() *usersServiceServer {\n        return \u0026usersServiceServer{}\n}\n\nfunc (s *userServiceServer) GetUser(ctx context.Context, in *GetUserRequest) (result *User, err error) {\n        result = GetUser(in.Arg1)\n        return\n}\n\nfunc (s *userServiceServer) UserStore_UpdateUser(ctx context.Context, in *User) (result *UserStore_UpdateUser, err error) {\n        s.UserStore.UpdateUser(in)\n        return\n}\n```\n\nThere are 3 interesting things in the generated code that, of course, would not work:\n- `usersServiceServer` is a generated empty struct.\n- `NewUsersServiceServer` is a generated constructor for `usersServiceServer`.\n- `UserStore_UpdateUser` uses the field `UserStore` of `userServiceServer` that, indeed, does not exist.\n\nThe server struct and its constructor are always generated empty **but only if they don't exist already**. That means that you can, and should, implement them yourself to make this code work.\n\nFor every method you are using, you are supposed to implement a receiver in the server type and initialize it however you want in the constructor. How would we fix this?\n\n```go\ntype userServiceServer struct {\n        UserStore *UserStore\n}\n\nfunc NewUserServiceServer() *userServiceServer {\n        return \u0026userServiceServer{\n                UserStore: NewUserStore(),\n        }\n}\n```\n\nNow if we generate the code again, the server struct and the constructor are implemented and the defaults will not be added again. Also, `UserStore_UpdateUser` would be able to find the field `UserStore` in `userServiceServer` and the code would work.\n\n### Not scanned types\n\nWhat happens if you have a type in your struct that is not in the list of scanned packages? It is completely ignored. The only exception to this are `time.Time` and `time.Duration`, which are allowed by default even though you are not adding `time` package to the list.\n\nIn the future, this will be extensible via plugins.\n\n### Examples\n\nYou can find an example of a *real* use case on the [example](example) folder.\nFor checking how the server and client works, see\n[server.go](example/server/server.go) and\n[client.go](example/client/client.go). The orchestration of server and client\nis done in [example_test.go](example_test.go). The example creates a server and\na new client and tests the output.\n\n\n### Features to come\n\n- Extensible mapping and options via plugins (waiting for Go 1.8 release).\n\n### Known Limitations\n\nThe following is a list of known limitations and the current state of them.\n\n* Only `{,u}int32` and `{,u}int64` is supported in protobuf. Therefore, all\n  `{,u}int` types are upgraded to the next size and a warning is printed. Same\n  thing happens with `float`s and `double`s.\n* If a struct contains a field of type `time.Time`, then that struct can only\n  be serialized and deserialized using the `Marshal` and `Unmarshal` methods.\n  Other marshallers use reflection and need a few struct tags generated by\n  protobuf that your struct won't have. This also happens with fields whose\n  type is a declaration to a slice of another type (`type Alias []base`).\n\n### Contribute\n\nIf you are interested on contributing to **proteus**, open an [issue](https://github.com/src-d/proteus/issues) explaining which missing functionality you want to work in, and we will guide you through the implementation, and tell you beforehand if that is a functionality we might consider merging in the first place.\n\n### License\n\nMIT, see [LICENSE](/LICENSE)\n\n","funding_links":[],"categories":["Repositories","Language-Specific"],"sub_categories":["Go"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsrc-d%2Fproteus","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsrc-d%2Fproteus","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsrc-d%2Fproteus/lists"}