{"id":51015148,"url":"https://github.com/streamnative/lightproto","last_synced_at":"2026-06-21T09:02:55.037Z","repository":{"id":234582217,"uuid":"758326242","full_name":"streamnative/lightproto","owner":"streamnative","description":"Protobuf compatible code generator","archived":false,"fork":false,"pushed_at":"2026-05-07T19:44:08.000Z","size":542,"stargazers_count":4,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-05-07T21:34:29.908Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"splunk/lightproto","license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/streamnative.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":"NOTICE","maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-02-16T04:23:56.000Z","updated_at":"2026-05-07T19:44:12.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/streamnative/lightproto","commit_stats":null,"previous_names":["streamnative/lightproto"],"tags_count":14,"template":false,"template_full_name":null,"purl":"pkg:github/streamnative/lightproto","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/streamnative%2Flightproto","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/streamnative%2Flightproto/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/streamnative%2Flightproto/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/streamnative%2Flightproto/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/streamnative","download_url":"https://codeload.github.com/streamnative/lightproto/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/streamnative%2Flightproto/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34603663,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-21T02:00:05.568Z","response_time":54,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-21T09:02:54.529Z","updated_at":"2026-06-21T09:02:55.028Z","avatar_url":"https://github.com/streamnative.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LightProto\n\nHigh-performance Protocol Buffers code generator for Java, optimized for serialization and deserialization speed.\n\n## Features\n\n- **Fastest Java Protobuf SerDe** — Unsafe-based serialization bypasses Netty ByteBuf boundary checks\n- **100% wire-compatible** with proto2 and proto3 definitions\n- **Zero-copy deserialization** using Netty `ByteBuf` (direct and heap memory)\n- **Zero heap allocations** — reusable mutable objects, no Builder pattern overhead\n- **Lazy string/bytes deserialization** — decoded only on access\n- **Optimized string handling** — single-copy ASCII fast path via `sun.misc.Unsafe`\n- **Protobuf-compatible JSON serialization and deserialization** — `toJson()` / `parseFromJson()` methods\n- **Protobuf TextFormat (de)serialization** — opt-in `toTextFormat()` / `parseFromTextFormat()` for compatibility with `com.google.protobuf.TextFormat`\n- **No runtime dependencies** — generated code is self-contained\n- **Maven and Gradle plugins** for seamless build integration\n\n## Usage\n\n### Maven\n\nAdd the Maven plugin to your `pom.xml`:\n\n```xml\n\u003cplugin\u003e\n    \u003cgroupId\u003eio.streamnative.lightproto\u003c/groupId\u003e\n    \u003cartifactId\u003elightproto-maven-plugin\u003c/artifactId\u003e\n    \u003cversion\u003e0.7.3\u003c/version\u003e\n    \u003cexecutions\u003e\n        \u003cexecution\u003e\n            \u003cgoals\u003e\n                \u003cgoal\u003egenerate\u003c/goal\u003e\n            \u003c/goals\u003e\n        \u003c/execution\u003e\n    \u003c/executions\u003e\n\u003c/plugin\u003e\n```\n\nPlace `.proto` files in `src/main/proto/` and LightProto will generate Java classes during the\n`generate-sources` phase.\n\n### Gradle\n\nAdd the plugin to your `build.gradle`:\n\n```groovy\nplugins {\n    id 'io.streamnative.lightproto' version '0.7.3'\n}\n```\n\nOr using the `buildscript` block:\n\n```groovy\nbuildscript {\n    repositories {\n        mavenCentral()\n    }\n    dependencies {\n        classpath 'io.streamnative.lightproto:lightproto-gradle-plugin:0.7.3'\n    }\n}\n\napply plugin: 'io.streamnative.lightproto'\n```\n\nPlace `.proto` files in `src/main/proto/` and LightProto will generate Java classes automatically\nbefore compilation. Optional configuration:\n\n```groovy\nlightproto {\n    classPrefix = ''           // prefix for generated class names\n    singleOuterClass = false   // wrap all messages in a single outer class\n    generateTextFormat = false // also generate protobuf TextFormat (de)serialization methods\n    protocVersion = '4.34.0'   // protoc compiler version\n    // protocPath = '/usr/local/bin/protoc'  // use a local protoc binary\n}\n```\n\nFor Maven, the same options are configured under `\u003cconfiguration\u003e` on the plugin:\n\n```xml\n\u003cplugin\u003e\n    \u003cgroupId\u003eio.streamnative.lightproto\u003c/groupId\u003e\n    \u003cartifactId\u003elightproto-maven-plugin\u003c/artifactId\u003e\n    \u003cconfiguration\u003e\n        \u003cgenerateTextFormat\u003etrue\u003c/generateTextFormat\u003e\n    \u003c/configuration\u003e\n    \u003c!-- ... --\u003e\n\u003c/plugin\u003e\n```\n\n### API Example\n\nLightProto generates mutable, reusable objects instead of the Builder pattern used by Google Protobuf:\n\n```java\n// Create and populate\nMessageMetadata md = new MessageMetadata();\nmd.setProducerName(\"producer-1\")\n  .setSequenceId(12345)\n  .setPublishTime(System.currentTimeMillis());\nmd.addProperty().setKey(\"key1\").setValue(\"value1\");\n\n// Serialize to ByteBuf\nByteBuf buf = PooledByteBufAllocator.DEFAULT.buffer();\nmd.writeTo(buf);\n\n// Deserialize from ByteBuf (zero-copy, lazy strings)\nMessageMetadata parsed = new MessageMetadata();\nparsed.parseFrom(buf, buf.readableBytes());\n\n// Reuse the object for the next message\nmd.clear();\nmd.setProducerName(\"producer-2\")...\n```\n\n### JSON Serialization and Deserialization\n\nEvery generated message has built-in JSON support compatible with\nprotobuf's [`JsonFormat`](https://protobuf.dev/programming-guides/json/):\n\n```java\n// Serialize to JSON string\nString json = md.toJson();\n// {\"producerName\":\"producer-1\",\"sequenceId\":\"12345\",\"publishTime\":\"1711234567890\",...}\n\n// Or write directly to a ByteBuf for zero-copy networking\nByteBuf jsonBuf = PooledByteBufAllocator.DEFAULT.buffer();\nmd.writeJsonTo(jsonBuf);\n\n// Deserialize from JSON string\nMessageMetadata parsed = new MessageMetadata();\nparsed.parseFromJson(json);\n\n// Or from a ByteBuf (avoids String allocation)\nparsed.parseFromJson(jsonBuf);\n\n// Or from a byte array\nparsed.parseFromJson(jsonBytes);\n```\n\nThe JSON encoding follows protobuf conventions: lowerCamelCase field names, int64 values quoted\nas strings, enum values as names, and bytes fields as base64. Unknown fields are silently ignored\nduring parsing, ensuring forward compatibility.\n\n### Protobuf TextFormat (opt-in)\n\nSet `generateTextFormat = true` (Gradle) or `\u003cgenerateTextFormat\u003etrue\u003c/generateTextFormat\u003e`\n(Maven) to also emit TextFormat (de)serialization on every message. The output is compatible\nwith `com.google.protobuf.TextFormat.printer()` and `TextFormat.merge()` for backward\ncompatibility with existing TextFormat data:\n\n```java\n// Serialize to a multi-line, indented TextFormat string\nString text = md.toTextFormat();\n// producer_name: \"producer-1\"\n// sequence_id: 12345\n// publish_time: 1711234567890\n// property {\n//   key: \"key1\"\n//   value: \"value1\"\n// }\n\n// Or write to a StringBuilder\nStringBuilder sb = new StringBuilder();\nmd.writeTextFormatTo(sb);\n\n// Deserialize from a String, byte[], or ByteBuf\nMessageMetadata parsed = new MessageMetadata();\nparsed.parseFromTextFormat(text);\n```\n\nDifferences from JSON: field names are the original proto snake_case, `int64` values are\n**not** quoted, enum values are emitted as bare identifiers, and bytes are written as\nquoted strings with C-style escapes. The parser tolerates `# … ` comments, single- or\ndouble-quoted strings, angle-bracket sub-messages (`field \u003c…\u003e`), `[v1, v2]` array syntax for\nrepeated fields, and ignores unknown fields.\n\n### gRPC Integration\n\nLightProto generates `*Grpc.java` service stubs directly from `service` definitions in `.proto`\nfiles — no extra protoc plugin needed. The generated code follows the same structure as\n`protoc-gen-grpc-java` but uses LightProto messages, so you get gRPC with zero-copy\nserialization and no Google Protobuf runtime dependency.\n\nAdd the gRPC dependency to your project:\n\n```xml\n\u003cdependency\u003e\n    \u003cgroupId\u003eio.grpc\u003c/groupId\u003e\n    \u003cartifactId\u003egrpc-stub\u003c/artifactId\u003e\n    \u003cversion\u003e1.68.0\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\nDefine a service in your `.proto` file:\n\n```protobuf\nsyntax = \"proto3\";\npackage myapp;\n\nmessage HelloRequest  { string name = 1; }\nmessage HelloResponse { string greeting = 1; }\n\nservice Greeter {\n    rpc SayHello (HelloRequest) returns (HelloResponse);\n    rpc SayHelloStream (HelloRequest) returns (stream HelloResponse);\n}\n```\n\nImplement the service by extending the generated `ImplBase`:\n\n```java\nclass GreeterImpl extends GreeterGrpc.GreeterImplBase {\n    @Override\n    public void sayHello(HelloRequest request, StreamObserver\u003cHelloResponse\u003e responseObserver) {\n        HelloResponse response = new HelloResponse();\n        response.setGreeting(\"Hello, \" + request.getName() + \"!\");\n        responseObserver.onNext(response);\n        responseObserver.onCompleted();\n    }\n}\n```\n\nCreate a client using the generated stubs:\n\n```java\n// Blocking stub (unary and server-streaming)\nGreeterGrpc.GreeterBlockingStub blocking = GreeterGrpc.newBlockingStub(channel);\nHelloRequest request = new HelloRequest();\nrequest.setName(\"World\");\nHelloResponse response = blocking.sayHello(request);\n\n// Async stub (all method types including client/bidi streaming)\nGreeterGrpc.GreeterStub async = GreeterGrpc.newStub(channel);\nasync.sayHelloStream(request, new StreamObserver\u003cHelloResponse\u003e() { ... });\n```\n\nThe generated stubs support all four gRPC method types: unary, server streaming,\nclient streaming, and bidirectional streaming.\n\n## Supported Features\n\n| Feature | proto2 | proto3 |\n|:---|:---:|:---:|\n| Scalar fields (int32, int64, uint32, uint64, sint32, sint64, fixed32, fixed64, sfixed32, sfixed64, float, double, bool) | ✅ | ✅ |\n| String fields | ✅ | ✅ |\n| Bytes fields | ✅ | ✅ |\n| Enum fields | ✅ | ✅ |\n| Nested messages | ✅ | ✅ |\n| `optional` fields (explicit presence) | ✅ | ✅ |\n| `required` fields | ✅ | — |\n| Implicit presence (no `has*()`, skip defaults) | — | ✅ |\n| `oneof` | ✅ | ✅ |\n| `repeated` fields | ✅ | ✅ |\n| `repeated` packed encoding | ✅ | ✅ (default) |\n| `map\u003cK, V\u003e` fields | ✅ | ✅ |\n| Nested enum / message definitions | ✅ | ✅ |\n| Default values | ✅ | — |\n| Multiple `.proto` files / `import` | ✅ | ✅ |\n| `service` / RPC definitions (gRPC stubs) | ✅ | ✅ |\n| JSON serialization and deserialization | ✅ | ✅ |\n| TextFormat serialization and deserialization (opt-in) | ✅ | ✅ |\n| Extensions | ❌ | — |\n| `Any`, `Timestamp`, well-known types | ❌ | ❌ |\n| `group` (deprecated) | ❌ | — |\n\n## Performance\n\n### Throughput Comparison (ops/\u0026mu;s, higher is better)\n\n| Benchmark | Google Protobuf | LightProto | Speedup |\n|:---|---:|---:|---:|\n| **AddressBook** (nested messages, strings) | | | |\n| \u0026emsp;Serialize | 6.4 | 22.6 | **3.5x** |\n| \u0026emsp;Fill + Serialize | 2.2 | 12.8 | **5.9x** |\n| \u0026emsp;Deserialize | 3.8 | 19.2 | **5.1x** |\n| **Simple** (small numeric messages) | | | |\n| \u0026emsp;Serialize | 30.8 | 245.6 | **8.0x** |\n| \u0026emsp;Deserialize | 20.0 | 100.9 | **5.0x** |\n| **Pulsar MessageMetadata** (strings, properties, batch fields) | | | |\n| \u0026emsp;Serialize | 2.8 | 10.9 | **3.8x** |\n| \u0026emsp;Deserialize | 4.2 | 14.1 | **3.4x** |\n| **Pulsar BaseCommand+Send** (nested message, numerics) | | | |\n| \u0026emsp;Serialize | 15.8 | 26.8 | **1.7x** |\n| \u0026emsp;Deserialize | 12.8 | 40.3 | **3.2x** |\n\n### Speedup Chart\n\n```\n                        Speedup over Google Protobuf (x times faster)\n                        1x    2x    3x    4x    5x    6x    7x    8x\n                        |     |     |     |     |     |     |     |\nAddressBook Ser         |████████████████████                       3.5x\nAddressBook Fill+Ser    |██████████████████████████████████████████  5.9x\nAddressBook Deser       |█████████████████████████████████          5.1x\n                        |     |     |     |     |     |     |     |\nSimple Ser              |█████████████████████████████████████████████████ 8.0x\nSimple Deser            |█████████████████████████████████          5.0x\n                        |     |     |     |     |     |     |     |\nPulsar MD Ser           |██████████████████████                     3.8x\nPulsar MD Deser         |███████████████████                        3.4x\n                        |     |     |     |     |     |     |     |\nPulsar Cmd Ser          |████████                                   1.7x\nPulsar Cmd Deser        |█████████████████                          3.2x\n                        |     |     |     |     |     |     |     |\n```\n\n### Raw JMH Output\n\n\u003cdetails\u003e\n\u003csummary\u003eClick to expand full benchmark output\u003c/summary\u003e\n\n```\nBenchmark                                                    Mode  Cnt    Score    Error   Units\n\n-- AddressBook (nested messages with strings) --\nProtoBenchmark.protobufSerialize                            thrpt    3    6.422 ±  0.242  ops/us\nProtoBenchmark.protobufFillAndSerialize                     thrpt    3    2.190 ±  0.365  ops/us\nProtoBenchmark.protobufDeserialize                          thrpt    3    3.777 ±  0.253  ops/us\nProtoBenchmark.lightProtoSerialize                          thrpt    3   22.551 ±  0.355  ops/us\nProtoBenchmark.lightProtoFillAndSerialize                   thrpt    3   12.827 ±  2.816  ops/us\nProtoBenchmark.lightProtoDeserialize                        thrpt    3   19.166 ±  1.022  ops/us\n\n-- Simple (small numeric messages) --\nSimpleBenchmark.protobufSerialize                           thrpt    3   30.768 ± 14.107  ops/us\nSimpleBenchmark.protobufDeserialize                         thrpt    3   19.988 ±  0.499  ops/us\nSimpleBenchmark.lightProtoSerialize                         thrpt    3  245.575 ± 87.004  ops/us\nSimpleBenchmark.lightProtoDeserialize                       thrpt    3  100.857 ±  2.221  ops/us\nSimpleBenchmark.lightProtoDeserializeReadString             thrpt    3   54.111 ± 14.056  ops/us\n\n-- Pulsar MessageMetadata (strings, repeated properties, batch fields) --\nPulsarApiBenchmark.protobufSerializeMessageMetadata         thrpt    3    2.838 ±  0.580  ops/us\nPulsarApiBenchmark.protobufDeserializeMessageMetadata       thrpt    3    4.176 ±  0.241  ops/us\nPulsarApiBenchmark.lightProtoSerializeMessageMetadata       thrpt    3   10.852 ±  0.677  ops/us\nPulsarApiBenchmark.lightProtoDeserializeMessageMetadata     thrpt    3   14.141 ±  8.862  ops/us\n\n-- Pulsar BaseCommand + CommandSend (nested message, mostly numerics) --\nPulsarApiBenchmark.protobufSerializeBaseCommand             thrpt    3   15.818 ±  4.591  ops/us\nPulsarApiBenchmark.protobufDeserializeBaseCommand           thrpt    3   12.771 ±  3.192  ops/us\nPulsarApiBenchmark.lightProtoSerializeBaseCommand           thrpt    3   26.814 ±  1.107  ops/us\nPulsarApiBenchmark.lightProtoDeserializeBaseCommand         thrpt    3   40.296 ±  2.816  ops/us\n```\n\n\u003c/details\u003e\n\n### Running Benchmarks\n\n```bash\nmvn -B install\njava -jar benchmark/target/benchmarks.jar\n```\n\n## License\n\nLicensed under the Apache License, Version 2.0.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstreamnative%2Flightproto","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstreamnative%2Flightproto","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstreamnative%2Flightproto/lists"}