{"id":13726282,"url":"https://github.com/mransan/ocaml-protoc","last_synced_at":"2025-10-04T06:04:59.240Z","repository":{"id":2469119,"uuid":"43656914","full_name":"mransan/ocaml-protoc","owner":"mransan","description":"A Protobuf Compiler for OCaml","archived":false,"fork":false,"pushed_at":"2024-09-17T17:01:01.000Z","size":2544,"stargazers_count":181,"open_issues_count":41,"forks_count":33,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-03-30T03:08:49.181Z","etag":null,"topics":["compiler","ocaml","protobuf","protocol","serialization"],"latest_commit_sha":null,"homepage":"https://mransan.github.io/ocaml-protoc/","language":"OCaml","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mransan.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-10-04T23:37:37.000Z","updated_at":"2025-03-06T10:04:17.000Z","dependencies_parsed_at":"2023-10-16T16:26:04.074Z","dependency_job_id":"fa933a8b-ac8e-450b-bc8e-52edfc845866","html_url":"https://github.com/mransan/ocaml-protoc","commit_stats":{"total_commits":370,"total_committers":17,"mean_commits":"21.764705882352942","dds":0.2945945945945946,"last_synced_commit":"736af401afba6ad9e041db6f2e4cbae1c00e8044"},"previous_names":[],"tags_count":30,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mransan%2Focaml-protoc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mransan%2Focaml-protoc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mransan%2Focaml-protoc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mransan%2Focaml-protoc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mransan","download_url":"https://codeload.github.com/mransan/ocaml-protoc/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247451667,"owners_count":20940944,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["compiler","ocaml","protobuf","protocol","serialization"],"created_at":"2024-08-03T01:02:57.972Z","updated_at":"2025-10-04T06:04:59.145Z","avatar_url":"https://github.com/mransan.png","language":"OCaml","readme":"# ocaml-protoc [![build](https://github.com/mransan/ocaml-protoc/actions/workflows/main.yml/badge.svg)](https://github.com/mransan/ocaml-protoc/actions/workflows/main.yml)\n\n\u003e :dromedary_camel: **A [protobuf](https://protobuf.dev/) compiler for OCaml :dromedary_camel:.** \n\n* [Introduction](#introduction)\n* [Simple Example](#a-simple-example)\n* [Install and Build](#install-and-build)\n* [Runtime library](#runtime-library)\n* [All Generated Files and Encodings](#all-generated-files-and-encodings)\n* [Protobuf \u003c-\u003e OCaml mapping](doc/protobuf_ocaml_mapping.md)\n* [Services](#services)\n* [Compiler Internals](doc/compiler_internals.md)\n* [Protobuf extensions](doc/ocaml_extensions.md)\n* [Benchmarking](doc/benchmarking.md)\n\n### Introduction \n\n⇨ `ocaml-protoc` compiles [protobuf message files](https://goo.gl/YqNT7Q) into \n**OCaml types** along with **serialization functions** for a variety of encodings.\n\n⇨ `ocaml-protoc` supports **both** proto syntax **2 and 3** as well as **binary** and **JSON** encodings. \n\n⇨ `ocaml-protoc` supports **JavaScript** object encoding through the  **BuckleScript \ncompiler**. See [here](https://github.com/mransan/bs-protobuf-demo) for complete example.\n\n\n\n### A simple example\n\n\u003e This example generates the binary encoding, if you are more interested in a **JavaScript** \n\u003e example, go [here](https://github.com/mransan/bs-protobuf-demo)\n\n*  **Write** in `example.proto`\n\n```Protobuf\nmessage Person {\n  required string name = 1;\n  required int32 id = 2;\n  optional string email = 3;\n  repeated string phone = 4;\n}\n```\n\n* **Run:**\n\n```bash\n$ ocaml-protoc --binary --ml_out ./ example.proto\n.. Generating example.mli\n.. Generating example.ml\n```\n\n* **example.mli**:\n\n```OCaml\n(** example.proto Generated Types *)\n\n(** {2 Types} *)\n\ntype person = {\n  name : string;\n  id : int32;\n  email : string;\n  phone : string list;\n}\n\n(** {2 Default values} *)\n\nval default_person : \n  ?name:string -\u003e\n  ?id:int32 -\u003e\n  ?email:string -\u003e\n  ?phone:string list -\u003e\n  unit -\u003e\n  person\n(** [default_person ()] is the default value for type [person] *)\n\n(** {2 Protobuf Encoding} *)\n\nval encode_pb_person : person -\u003e Pbrt.Encoder.t -\u003e unit\n(** [encode_pb_person v encoder] encodes [v] with the given [encoder] *)\n\n\n(** {2 Protobuf Decoding} *)\n\nval decode_pb_person : Pbrt.Decoder.t -\u003e person\n(** [decode_pb_person decoder] decodes a [person] binary value from [decoder] *)\n```\n\n* in `main.ml`, write the following to encode a person value and save it to a file: \n\n```OCaml\nlet () =\n\n  (* Create OCaml value of generated type *) \n  let person = Example.({ \n    name = \"John Doe\"; \n    id = 1234l;\n    email = Some \"jdoe@example.com\"; \n    phone = [\"123-456-7890\"];\n  }) in \n  \n  (* Create a Protobuf encoder and encode value *)\n  let encoder = Pbrt.Encoder.create () in \n  Example.encode_pb_person person encoder; \n\n  (* Output the protobuf message to a file *) \n  let oc = open_out \"myfile\" in \n  output_bytes oc (Pbrt.Encoder.to_bytes encoder);\n  close_out oc\n```\n\n* then in the same `main.ml` append the following to read from the same file:\n\n```OCaml\nlet () = \n  (* Read bytes from the file *) \n  let bytes = \n    let ic = open_in \"myfile\" in \n    let len = in_channel_length ic in \n    let bytes = Bytes.create len in \n    really_input ic bytes 0 len; \n    close_in ic; \n    bytes \n  in \n  \n  (* Decode the person and Pretty-print it *)\n  Example.decode_pb_person (Pbrt.Decoder.of_bytes bytes)\n```\n\n* :heavy_exclamation_mark: **Int32** vs **int**\n\n*OCaml users will immediately point to the use of `int32` type in the generated code which might not be the most convenient choice.\nOne can modify this behavior using [custom extensions](doc/ocaml_extensions.md).* \n\n### Install \u0026 Build\n\n**Prerequesite**\n\n`ocaml-protoc` only depends on\n* the OCaml compiler distribution (byte code/native compiler).\n* dune\n* stdlib-shims for the compiler itself\n* a C99 compiler for the runtime library's stubs\n\n**Intall from OPAM**\n\n```bash\n$ opam install ocaml-protoc\n```\n\n**Or from source**\n\n```bash\n$ mkdir -p tmp/bin\n$ export PREFIX=`pwd`/tmp\n$ make install\n```\n\n**Build your program** \n\nUsing dune, the program can be compiled with:\n\n```\n(executable\n  (name main)\n  (modules main example)\n  (libraries pbrt))\n```\n\nMore manually, the program can be built directly using [ocamlfind](http://projects.camlcity.org/projects/findlib.html):\n\n```Bash\n$ ocamlfind ocamlopt -linkpkg -package pbrt \\\n    -o example \\\n    example.mli example.ml \\\n    main.ml\n```\n\n🏁 You can now run the example\n```Bash\n$ ./example\n```\n\n### Runtime library\n\nThe generated code depends on the opam package \"pbrt\", defining\na module `Pbrt`.\n\nOnline documentation [here](https://mransan.github.io/ocaml-protoc/dev/pbrt/Pbrt/index.html)\n\n### All Generated Files and Encodings:\n\n| Command line switch | Description | Runtime |\n| ------------- | ------------- | ----------|\n| | Type definition along with a `default` constructor function to conveniently create values of that type | |\n| --make | `make` constructor functions |  |\n| --binary  | Binary encodings | `pbrt` |\n| --yojson | JSON encoding using the widely popular [yojson](https://github.com/mjambon/yojson) library | `pbrt_yojson` |\n| --bs | BuckleScript encoding using the BuckleScript core binding to JS json library | [bs-ocaml-protoc-json][3] |\n| --pp | pretty printing functions based on the Format module. | `pbrt` |\n| --services | RPC definitions. | `pbrt_services` |\n\n[3]:https://www.npmjs.com/package/bs-ocaml-protoc-json\n\n### Protobuf \u003c-\u003e OCaml mapping\nsee [here](doc/protobuf_ocaml_mapping.md).\n\n### Services\n\nWith the `--services` option, ocaml-protoc now generates stubs for service\ndeclarations.\n\nFor example with the given `calculator.proto` file:\n\n```proto\nsyntax = \"proto3\";\n\nmessage I32 {\n  int32 value = 0;\n}\n\nmessage AddReq {\n  int32 a = 1;\n  int32 b = 2;\n}\n\nservice Calculator {\n  rpc add(AddReq) returns (I32);\n\n  rpc add_stream(stream I32) returns (I32);\n}\n```\n\nUsing `ocaml-protoc --binary --services --ml_out=. calculator.proto`, we get the normal\ntype definitions, but also this service definition:\n\n```ocaml\n(** Calculator service *)\nmodule Calculator : sig\n  open Pbrt_services\n  open Pbrt_services.Value_mode\n\n  module Client : sig\n    val add : (add_req, unary, i32, unary) Client.rpc\n    val add_stream : (i32, stream, i32, unary) Client.rpc\n  end\n  \n  module Server : sig\n    (** Produce a server implementation from handlers *)\n    val make : \n      add:((add_req, unary, i32, unary) Server.rpc -\u003e 'handler) -\u003e\n      add_stream:((add_req, stream, i32, unary) Server.rpc -\u003e 'handler) -\u003e\n      unit -\u003e 'handler Pbrt_services.Server.t\n  end\nend\n```\n\nThis can then potentially be used with libraries that implement specific protobuf-based\nnetwork protocols, such as [ocaml-grpc](https://github.com/dialohq/ocaml-grpc)\nor [ocaml-twirp](https://github.com/c-cube/ocaml-twirp), or other custom protocols.\n\nProtobuf service endpoints take a single type and return a single type, but they have the ability\nto stream either side. We represent this ability with the `Pbrt_services.Value_mode` types:\n\n```ocaml\n(** Whether there's a single value or a stream of them *)\nmodule Value_mode = struct\n  type unary\n  type stream\nend\n```\n\n#### Client-side\n\nA `(req, req_kind, res, res_kind) Client.rpc` is a bundle describing a single RPC endpoint,\nfrom the client perspective. It contains the RPC name, service, etc. alongside encoders for\nthe request type `req`, and decoders for the response type `res`.\n\nThe phantom types `req_kind` and `res_kind` represent the value mode for request,\nrespectively response. Here we see that `Calculator.Client.add` is unary for both\n(it takes a single argument and returns a single value)\nbut `Calculator.Client.add_stream` takes a string of `i32` as parameters before\nreturning a single result.\n\nWith transports such as grpc, all [4 combinations](https://grpc.io/docs/what-is-grpc/core-concepts/#rpc-life-cycle)\nare possible. With twirp over HTTP 1.1, only unary mode is supported.\n\n#### Server-side\n\nOn the server side, ocaml-protoc generates individual stubs,\nlike on the client side; but it also generates _services_ as bundles\nof endpoints. One service corresponds to a `service` declaration\nin the `.proto` file.\n\n\u003cdetails open\u003e\n\u003csummary\u003e\nDetailed explanation of how server-side services work\n\u003c/summary\u003e\n\nIn practice, in something like twirp, a service could be added to a web server\nby adding each endpoint to a single HTTP route; or a twirp-aware router could\ndirectly map incoming HTTP queries to services.\n\nThe trickiest part here is that the type `'handler Pbrt_services.Server.t` is\nparametric. Indeed it'd be hard for the generated code to cater to every possible\ncombination of network transport and concurrency library (eio, lwt, async, etc.).\n\nInstead, the code is generic over `'handler` (the type of a query handler for a _single_\nendpoint; e.g. a HTTP endpoint for a single route). The function\n```ocaml\n  module Server : sig\n    val make : \n      add:((add_req, unary, i32, unary) Server.rpc -\u003e 'handler) -\u003e\n      add_stream:((add_req, stream, i32, unary) Server.rpc -\u003e 'handler) -\u003e\n      unit -\u003e 'handler Pbrt_services.Server.t\n  end\n```\nseen previously is used to build the `'handler service` by asking the user\nto provide a handler for each method. The builder for `add` is given a\ndescription of the `add` endpoint (with decoders for requests; and encoders\nfor responses), and must return a handler that knows how to decode the request,\nadd numbers, and turn that back into a response.\n\nLibraries will provide facilities to build such handlers, so that the user\nonly has to provide the actual logic (here, adding numbers). For example\nin `twirp_tiny_httpd` (part of `ocaml-twirp`), implementing a\nserver looks like this[^1]:\n\n[^1]: we use a different `.proto` because twirp doesn't handle streams.\n\n```proto\nsyntax = \"proto3\";\n\nmessage I32 {\n  int32 value = 0;\n}\n\nmessage AddReq {\n  int32 a = 1;\n  int32 b = 2;\n}\n\nmessage AddAllReq {\n  repeated int32 ints = 1;\n}\n\nservice Calculator {\n  rpc add(AddReq) returns (I32);\n\n  rpc add_all(AddAllReq) returns (I32);\n}\n```\n\n```ocaml\nlet add (a : add_req) : i32 = default_i32 ~value:Int32.(add a.a a.b) ()\n\nlet add_all (a : add_all_req) : i32 =\n  let l = ref 0l in\n  List.iter (fun x -\u003e l := Int32.add !l x) a.ints;\n  default_i32 ~value:!l ()\n\nlet calc_service : Twirp_tiny_httpd.handler Pbrt_services.Server.t =\n  Calculator.Server.make\n    ~add:(fun rpc -\u003e Twirp_tiny_httpd.mk_handler rpc add)\n    ~add_all:(fun rpc -\u003e Twirp_tiny_httpd.mk_handler rpc add_all)\n    ()\n\nlet() =\n  let server = Tiny_httpd.create ~port:1234 () in\n  Twirp_tiny_httpd.add_service ~prefix:(Some \"twirp\") server calc_service;\n  Tiny_httpd.run_exn server\n```\n\nHere we see that all the logic is in `add` and `add_all`, which know nothing\nabout protobuf or serialization. A `calc_service` bundle, using the `Twirp_tiny_httpd.handler`\ntype for each handler, is built from them. Finally, a HTTP server is created,\nthe service is added to it (binding some routes), and we enter the\nserver's main loop.\n\n\u003c/details\u003e\n\n\n### Compiler Internals\n\nsee [here](doc/compiler_internals.md)\n\n### Protobuf Extensions\n \nsee [here](doc/ocaml_extensions.md)\n\n### Benchmarking\n \nsee [here](doc/benchmarking.md)\n","funding_links":[],"categories":["OCaml","\u003ca name=\"OCaml\"\u003e\u003c/a\u003eOCaml"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmransan%2Focaml-protoc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmransan%2Focaml-protoc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmransan%2Focaml-protoc/lists"}