Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/sbt/sbt-protobuf

sbt plugin for compiling protobuf files
https://github.com/sbt/sbt-protobuf

protobuf protocol-buffers sbt sbt-plugin scala

Last synced: 4 days ago
JSON representation

sbt plugin for compiling protobuf files

Awesome Lists containing this project

README 

        
# sbt-protobuf

A plugin for sbt that transforms `*.proto` files into gazillion-loc Java source files, and potentially to other languages too.



## Usage



### Adding the plugin dependency

In your project, create a file for plugin library dependencies `project/plugins.sbt` and add the following line:



```scala

addSbtPlugin("com.github.sbt" % "sbt-protobuf" % "0.8.1")

```



The dependency to `"com.google.protobuf" % "protobuf-java"` is automatically added to the `Compile` scope.

The version for `protobuf-java` can be controlled by the setting `ProtobufConfig / version` (set to `3.9.0` by default).



### Importing sbt-protobuf settings

To actually "activate" the plugin, its settings need to be included in the build.



##### build.sbt



```scala

enablePlugins(ProtobufPlugin)

```



### Declaring dependencies

#### Artifacts containing both `*.proto` files and java binaries

Assuming an artifact contains both `*.proto` files as well as the binaries of the generated `*.java` files, you can specify the dependency like so:



```scala

libraryDependencies += "some.groupID" % "some.artifactID" % "1.0" % ProtobufConfig.name // #1



libraryDependencies += "some.groupID" % "some.artifactID" % "1.0" // #2

```



Line #1 tells `sbt-protobuf` that the specified artifact contains *.proto files which it needs to extract and add to the `--proto_path` for `protoc`.

Internally the setting `protobufExternalIncludePath` is used to track 3rd party proto files.



Line #2 adds the artifact to the regular compile classpath.



The `*.proto` files of dependencies are extracted and added to the `--proto_path` parameter for `protoc`, but are not compiled.



#### Artifacts in the `protobuf` configuration containing only `*.proto` files

You can specify a dependency on an artifact that contains only `.proto` files in the `protobuf` configuration with a `proto` classifier like so:

```

libraryDependencies += ("some.groupID" % "some.artifactID" % "1.0" classifier protoClassifier) % s"${ProtobufConfig.name}->${ProtobufConfig.name}"

```



### Compiling external proto files

Sometimes it's desirable to compile external proto files (eg. because the library is compiled with an older version of `protoc`).

This can be achieved by adding the following setting:



```scala

ProtobufConfig / sourceDirectories += (ProtobufConfig / protobufExternalIncludePath).value

```



### Compiling gRPC proto files

The following enables gRPC support.



```scala

ProtobufConfig / protobufGrpcEnabled := true

```



### Packaging proto files

`*.proto` files can be included in the jar file by adding the following setting to your build definition:



```scala

Compile / unmanagedResourceDirectories += (ProtobufConfig / sourceDirectory).value

```



Alternatively, `*.proto` files can be packaged in a separate jar file in the `protobuf` configuration with a `proto` classifier:



```scala

addArtifact(ProtobufConfig / protobufPackage / artifact, ProtobufConfig / protobufPackage)

```



### Changing the location of the generated java files

By default, the compiled proto files are created in `/target//src_managed/main/compiled_protobuf`. Changing the location to `/src/generated` can be done by adding the following setting to your build definition:



```scala

ProtobufConfig / javaSource := ((Compile / sourceDirectory).value / "generated")

```



**WARNING:** The content of this directory is **removed** by the `clean` task. Don't set it to a directory containing files you hold dear to your heart.



### Note



1. If you encounter compile errors, as ```[...] is already defined as object [...]``` you could change the compile order

as ```compileOrder := CompileOrder.JavaThenScala```,the default is ```mixed```.



2. The inner message's name could not be the ```.proto```'s file name.that will cause problem too, you could change the inner message's name or the ```.proto``` file name or add the ```option java_outer_classname = "NewNameNotAsTheFileName";``` to you ```.proto``` file.



### Additional options to protoc

All options passed to `protoc` are configured via the `protobufProtocOptions`. To add options, for example to run a custom plugin, add them to this setting key. For example:



```scala

ProtobufConfig / protobufProtocOptions ++= Seq("--custom-option")

```



### Additional target directories

The source directories where the files are generated, and the globs used to identify the generated files, are configured by `ProtobufConfig / protobufGeneratedTargets`.

In case only Java files are generated, this setting doesn't need to change, since it automatically inherits the value of `ProtobufConfig / javaSource`, paired with the glob `*.java`.

In case other types of source files are generated, for example by using a custom plugin (see previous section), the corresponding target directories and source file globs must be configured by adding them to this setting. For example:



```scala

ProtobufConfig / protobufGeneratedTargets ++= {

  Seq(((Compile / sourceDirectory).value / "generated" / "scala", "*.scala"))

}

```



This plugin uses the `protobufGeneratedTargets` setting to:

- add the generated source directories to `cleanFiles` and `managedSourceDirectories`

- collect the generated files after running `protoc` and return them to SBT for the compilation phase



## Scope

All settings and tasks are in the `protobuf` scope. If you want to execute the `protobufGenerate` task directly, just run `protobuf:protobufGenerate`.



## Settings



namebuilt-in keydefaultdescription



    sourceDirectory

    x

    "src/main/protobuf"

    Path containing *.proto files.



    sourceDirectories

    x

    sourceDirectory

    This setting is used to collect all directories containing *.proto files to compile



    javaSource

    x

    "$sourceManaged/compiled_protobuf"

    Path for the generated *.java files.



    version

    x

    "3.25.3"

    Which version of the protobuf library should be used. A dependency to "com.google.protobuf" % "protobuf-java" % "$version" is automatically added to libraryDependencies



    protobufGrpcVersion

    

    "1.62.2"

    Which version of the gRPC plugin should be used.



    protobufGrpcEnabled

    

    false

    The setting to enable gRPC plugin.



    protobufProtoc

    

    "protoc"

    The path to the 'protoc' executable.



    protobufIncludeFilters

    

    Glob(d.toPath()) / "**" / "*.proto"

    The list of *.proto glob expressions to include.



    protobufExcludeFilters

    

    Glob(d.toPath()) / "google" / "protobuf" / "*.proto"

    The list of *.proto glob expressions to exclude.



    protobufIncludePaths

    

    Seq($generated-source, protobufExternalIncludePath)

    The path for additional *.proto files.



    protobufExternalIncludePath

    

    target/protobuf_external

    The path to which protobuf:libraryDependencies are extracted and which is used as protobuf:protobufIncludePath for protoc



    protobufGeneratedTargets

    

    (file(java source directory based on ProtobufConfig / javaSource), "*.java")

    The list of target directories and source file globs for the generated files



## Tasks



  name

  description



  protobufGenerate

  Performs the hardcore compiling action and is automatically executed as a "source generator" in the Compile scope.



  protobufUnpackDependencies

  Extracts proto files from libraryDependencies into protobufExternalInludePatch



  protobufRunProtoc

  A function that executes the protobuf compiler with the given arguments,

    returning the exit code. The default implementation runs the executable referenced by the protoc setting.



  protobufPackage

  Produces a jar artifact containing only *.proto files, with a proto classifier



    protobufProtocOptions

    

    --java_out=[java generated source directory from protobufGeneratedTargets]

    The list of options passed to the protoc binary



## IntelliJ IDEA BSP bug

IntelliJ has a [bug](https://youtrack.jetbrains.com/issue/SCL-19517) where it only recognizes generated sources if there is at least one Scala class in the same package - otherwise you'll see red squiggles. As a workaround, you can configure your project to add a private empty class, e.g. like this:

```scala

Compile / sourceGenerators += Def.task {

  // adapt this for your build:

  val protoPackage = "org.example.proto.foo"



  val scalaFile = (Compile/sourceManaged).value / "_ONLY_FOR_INTELLIJ.scala"



  IO.write(scalaFile,

    s"""package $protoPackage

      |

      |private class _ONLY_FOR_INTELLIJ

      |""".stripMargin)



  Seq(scalaFile)

}.taskValue

```



## Credits

`sbt-protobuf` is based on [softprops/coffeescripted-sbt](https://github.com/softprops/coffeescripted-sbt) for the sbt-0.10 specific parts and [codahale/protobuf-sbt](https://github.com/codahale/protobuf-sbt) for the protobuf specifics.