{"id":29210654,"url":"https://github.com/embarkstudios/proto-gen","last_synced_at":"2025-07-02T21:07:26.496Z","repository":{"id":150778636,"uuid":"574430609","full_name":"EmbarkStudios/proto-gen","owner":"EmbarkStudios","description":"A protobuf generation runner/cli using tonic build","archived":false,"fork":false,"pushed_at":"2025-02-21T13:51:55.000Z","size":240,"stargazers_count":20,"open_issues_count":0,"forks_count":2,"subscribers_count":12,"default_branch":"main","last_synced_at":"2025-06-03T22:01:55.452Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/EmbarkStudios.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE-APACHE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2022-12-05T09:47:19.000Z","updated_at":"2025-05-17T11:09:07.000Z","dependencies_parsed_at":"2023-12-11T20:26:03.967Z","dependency_job_id":"aab9b53a-2767-4ec6-9ace-a4a9e53be765","html_url":"https://github.com/EmbarkStudios/proto-gen","commit_stats":null,"previous_names":[],"tags_count":11,"template":false,"template_full_name":"EmbarkStudios/opensource-template","purl":"pkg:github/EmbarkStudios/proto-gen","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EmbarkStudios%2Fproto-gen","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EmbarkStudios%2Fproto-gen/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EmbarkStudios%2Fproto-gen/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EmbarkStudios%2Fproto-gen/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/EmbarkStudios","download_url":"https://codeload.github.com/EmbarkStudios/proto-gen/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EmbarkStudios%2Fproto-gen/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263215298,"owners_count":23431895,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-07-02T21:07:25.643Z","updated_at":"2025-07-02T21:07:26.475Z","avatar_url":"https://github.com/EmbarkStudios.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003c!-- Allow this file to not have a first line heading --\u003e\n\u003c!-- markdownlint-disable-file MD041 no-emphasis-as-heading --\u003e\n\n\u003c!-- inline html --\u003e\n\u003c!-- markdownlint-disable-file MD033 --\u003e\n\n\u003cdiv align=\"center\"\u003e\n\n# `💠 proto-gen`\n\n**[Protobuf](https://protobuf.dev/) to [Rust](https://www.rust-lang.org/) code generation using [tonic-build](https://docs.rs/tonic-build/latest/tonic_build/)**\n\n[![Embark](https://img.shields.io/badge/embark-open%20source-blueviolet.svg)](https://embark.dev)\n[![Embark](https://img.shields.io/badge/discord-ark-%237289da.svg?logo=discord)](https://discord.gg/dAuKfZS)\n[![Crates.io](https://img.shields.io/crates/v/proto-gen.svg)](https://crates.io/crates/proto-gen)\n[![dependency status](https://deps.rs/repo/github/EmbarkStudios/proto-gen/status.svg)](https://deps.rs/repo/github/EmbarkStudios/proto-gen)\n[![Build status](https://github.com/EmbarkStudios/proto-gen/workflows/CI/badge.svg)](https://github.com/EmbarkStudios/proto-gen/actions)\n\u003c/div\u003e\n\n## What\nThe repo contains a cli tool that uses [tonic-build](https://docs.rs/tonic-build/latest/tonic_build/) to generate rust-code from protobuf-files.  \n[tonic-build](https://docs.rs/tonic-build/latest/tonic_build/) already does this, the cli is a front-end to \nthat with some added code to make sure that the generated files are placed in a valid path, and takes care of the \nmodule structuring.\n\n## Why\n[prost-build](https://docs.rs/prost-build/latest/prost_build/) used to ship with `cmake` which we would like to avoid.  \n`cmake` was used to build `protoc` which was then used for the proto-to-rust codegen.  \nThe final decision from the prost maintainers side is that the user should provide their own `protoc` and check in the code\ninstead of building it in a `build.rs`, to make that process simpler, therefore this cli was created.  \n\n## How\nUses [a local version of protoc on path](https://grpc.io/docs/protoc-installation/) to generate rust code from a \nworkspace specification passed to the CLI. The CLI can also be used to validate the generated Rust code against source \nprotobuf files, for example, in CI.  \n\n## Considerations\nThere are quite a few problems with this tool, for now it remains the best alternative for our protobuf pipeline, \nsome things to take into account follows.  \n\n### Requires manual installation on the user's machine\nRunning this tool requires at minimum `protoc` to be installed on the user's machine, \nmany protocol buffers found in the wild have dependencies in the form of other `.proto` files. Which also \nhave to be sourced.  \nMany `.proto` files that you might want to build have dependencies on the `google.protobuf` \n[types](https://protobuf.dev/reference/protobuf/google.protobuf/), \nmany `.proto` authors assume that these are available to the user, which may or may not be true.  \n\nThere has been discussions about vendoring them here since they are so common but for now no vendoring will be done, \nsince they are not mandatory in many cases. \n\nA middle-ground is to vendor the dependency protos yourself, and run this tool with the directory where you placed those \n`.proto` files specified (`-d my-dep-dir`).  \n\n### Version incompatibility if tonic version has changed (even minor versions)\nOn tonic minor versions sometimes comments or other non-functional changes are made, these will cause the \n`validate` command to reject old protos generated by a lower version of this tool, requiring regeneration.  \nIt could be argued that therefore a minor version change in `tonic` becomes a major version change \nfor this tool, although that may be a bit philosophical since the generated code runs the same.  \n\n\n### Doc comments are executed\nTonic converts protobuf comments to doc comments, if these include code examples, cargo will attempt to execute them\nwhen running `cargo test` (`doc-test` stage).  \nThis behaviour is different from using [tonic-build](https://docs.rs/tonic-build/latest/tonic_build/), since those doc-comments are generated where cargo doesn't try \nto run them.  \nRunning them is almost always guaranteed to fail in the best case (because they will be non-rust code or not code at all), \nand be a lurking security vulnerability in the worst case where someone have inserted malicious Rust-code in a doc-comment.  \n\nIdeally it'd be easier to disable running doc-test on a per-module basis. Placing generated protos in a separate lib \nwhich doesn't run doc-comments is a safer choice, although doc-tests being opt-in would be significantly safer.  \n\n## Usage \n\n### Install\nTo install `proto-gen` from source do the following. This assumed you have installed a recent [rust toolchain](https://www.rust-lang.org/tools/install).\n```\ncargo install --locked proto-gen\n```\n\n### Command-line options\n\nUsage:\n\n```\nproto-gen [OPTIONS] \u003cCOMMAND\u003e\n```\n\n#### Commands:\n\nThe top-level commands are:\n\n- `validate` Generate new Rust code for proto files, checking current files for differences. Returns error code 1 on any found differences.\n- `generate` Generate new Rust code for proto files, overwriting old files if present.\n- `help` Print this message or the help of the given subcommand(s).\n\n`validate` \u0026 `generate` share the following sub-options:\n\n-  `-d`, `--proto-dirs` `\u003cPROTO_DIRS\u003e` Directories containing proto files to source (Ex. Dependencies), needs to include any directory containing files to be included in generation.\n- `-f`, `--proto-files` `\u003cPROTO_FILES\u003e` The files to be included in generation.\n- `-t`, `--tmp-dir` `\u003cTMP_DIR\u003e` Temporary working directory, if left blank, `tempfile` is used to create a temporary directory.\n- `-o`, `--output-dir` `\u003cOUTPUT_DIR\u003e` Where to place output files. Will get cleaned up (all contents deleted). A module file will be placed in the parent of this directory.\n- `-h`, `--help` Print help.\n\n#### Options:\nThe top-level options are:\n\n- `-s`, `--build-server` Whether to build server code.\n- `-c`, `--build-client` Whether to build client code.\n- `--generate-transport` Whether to generate the `::connect` and similar functions for tonic.\n- `-d`, `--disable-comments` `\u003cDISABLE_COMMENTS\u003e` Disable comments based on proto path. Passing `'.'` disables all comments.\n- `-b`, `--btree-map` `\u003cBTREE_MAPS\u003e` Output maps as `BTreeMap` instead of `HashMap`. Passing `'.'` makes all maps `BTreeMap`.\n- `--type-attribute` `\u003cTYPE_ATTRIBUTES\u003e` Type attributes to add.\n- `--enum-attribute` `\u003cENUM_ATTRIBUTES\u003e` Enum attributes to add.\n- `--client-attribute` `\u003cCLIENT_ATTRIBUTES\u003e` Client mod attributes to add.\n- `--server-attribute` `\u003cSERVER_ATTRIBUTES\u003e` Server mod attributes to add.\n- `-f`, `--format` Use `rustfmt` on the code after generation, `rustfmt` needs to be on the path.\n- `-p`, `--prepend-header` Prepend header indicating tool version in generated source files.\n- `--prepend-header-file` `\u003cPREPEND_HEADER_FILE\u003e` Prepend header file in generated source files.\n- `--toplevel-attribute` `\u003cTOPLEVEL_ATTRIBUTE\u003e` Toplevel mod attribute to add.\n- `-h`, `--help` Print help.\n- `-V`, `--version` Print version.\n\n### Examples in this project\nThis will generate Rust code from the proto specified in `examples/example-project/proto/my-proto.proto` and place it \nin `examples/example-project/src/proto_types`.\n\n```bash\nproto-gen generate \\\n  -d examples/example-project/proto \\\n  -f examples/example-project/proto/my-proto.proto \\\n  -o examples/example-project/src/proto_types\n```\n\nThis will also generate Rust code (to a temporary directory) and the run a diff against the code contained in `examples/example-project/src/proto_types`. \nIf it finds any diffs it will exit with error code 1 and a message.\n\n```bash\nproto-gen validate \\\n  -d examples/example-project/proto \\\n  -f examples/example-project/proto/my-proto.proto \\\n  -o examples/example-project/src/proto_types\n```\n\nIf we want to use includes, the directory to include needs to be specified. Here we're passing `-d` twice, once to include the dependency protos, and one to include the protos we want to generate.  \n\n```bash \nproto-gen generate \\\n  -d examples/example-project/dep_protos \\\n  -d examples/example-project/proto_with_deps \\\n  -f examples/example-project/proto_with_deps/my-proto.proto \\\n  -o examples/example-project/src/proto_types\n```\n\nThe same applies for validate.\n\n```bash \nproto-gen validate \\\n  -d examples/example-project/dep_protos \\\n  -d examples/example-project/proto_with_deps \\\n  -f examples/example-project/proto_with_deps/my-proto.proto \\\n  -o examples/example-project/src/proto_types\n```\n\n## Contributing\n\n[![Contributor Covenant](https://img.shields.io/badge/contributor%20covenant-v1.4-ff69b4.svg)](CODE_OF_CONDUCT.md)\n\nWe welcome community contributions to this project.\n\nPlease read our [Contributor Guide](CONTRIBUTING.md) for more information on how to get started.\nPlease also read our [Contributor Terms](CONTRIBUTING.md#contributor-terms) before you make any contributions.\n\nAny contribution intentionally submitted for inclusion in an Embark Studios project, shall comply with the Rust standard licensing model (MIT OR Apache 2.0) and therefore be dual licensed as described below, without any additional terms or conditions:\n\n### License\n\nThis contribution is dual licensed under EITHER OF\n\n- Apache License, Version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or \u003chttp://www.apache.org/licenses/LICENSE-2.0\u003e)\n- MIT license ([LICENSE-MIT](LICENSE-MIT) or \u003chttp://opensource.org/licenses/MIT\u003e)\n\nat your option.\n\nFor clarity, \"your\" refers to Embark or any other licensee/user of the contribution.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fembarkstudios%2Fproto-gen","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fembarkstudios%2Fproto-gen","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fembarkstudios%2Fproto-gen/lists"}