https://github.com/iadvize/sbt-avro
Plugin SBT to Generate Scala classes from Apache Avro schemas hosted on a remote Confluent Schema Registry.
https://github.com/iadvize/sbt-avro
avro generating plugin sbt sbt-avro scala schema schema-registry
Last synced: 10 months ago
JSON representation
Plugin SBT to Generate Scala classes from Apache Avro schemas hosted on a remote Confluent Schema Registry.
- Host: GitHub
- URL: https://github.com/iadvize/sbt-avro
- Owner: iadvize
- License: mit
- Created: 2017-09-08T08:45:50.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2018-03-14T10:36:16.000Z (about 8 years ago)
- Last Synced: 2025-04-01T16:52:01.335Z (about 1 year ago)
- Topics: avro, generating, plugin, sbt, sbt-avro, scala, schema, schema-registry
- Language: Scala
- Homepage:
- Size: 29.3 KB
- Stars: 15
- Watchers: 39
- Forks: 6
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Codeowners: CODEOWNERS
Awesome Lists containing this project
README
# sbt-avro
sbt-avro is a sbt 1.x plugin for generating Scala classes from Apache Avro schemas hosted on a remote Confluent Schema Registry.
## Examples
Example of build.sbt file:
```scala
schemaRegistryEndpoint in Avro := "http://0.0.0.0:8081",
schemas in Avro ++= Seq(
Schema("cc_payments-value", Version(1)),
Schema("telecom_italia_data-value", Version(3)),
Schema("nyc_yellow_taxi_trip_data-value", Version(1))
)
```
## Install
Add the plugin to your build, eg in `./project/plugins.sbt` add this line:
```scala
addSbtPlugin("com.iadvize" % "sbt-avro" % "1.0.0-SNAPSHOT")
```
### Download Avro schema files from Schema Registry
By default sbt-avro will download all Avro schema files from local schema registry to your default resources_managed directory (ie: `target/scala-2.12/resources_managed/main/avro/`).
Please check settings section for more information about available settings.
Example:
```scala
sbt avro:download
```
### Upload Avro schema files to Schema Registry
sbt-avro can upload all Avro Schema files from your resources directory (ie: `src/main/resources/avro`) to a local schema registry.
Please check settings section for more information about available settings.
Example:
```scala
sbt avro:upload
```
### Test compatibility of schemas with their latest versions in Schema Registry
sbt-avro can test schemas for compatibility against the latest version of a subject's schema. This makes use of the
Schema Registry [compatibility](https://docs.confluent.io/current/schema-registry/docs/api.html#id1) resource.
The plugin will look for `*.avsc` files in :
- Resources folder (ie: `src/main/resources/avro/`)
- Resources Managed folder (ie: `target/scala-2.12/resources_managed/main/avro/`)
Example:
```scala
sbt avro:compatibility
```
### Generate scala classes from Avro schema files
The plugin sbt-avro will look for `*.avsc` files in :
- Resources folder (ie: `src/main/resources/avro/`)
- Resources Managed folder (ie: `target/scala-2.12/resources_managed/main/avro/`)
Important : If a duplicate avro schema (same name ) is detected in your resources folder and in your resources_managed folder, the compiler will compile only the schema in resources folder. So, for example, in dev if you want to try some local changes, you can duplicate your schema from your resources managed folder into your resources folder and compile your changes.
So put your schema files there and run:
```scala
sbt avro:generate
```
The case classes will get generated in your default src_managed directory (ie:`target/scala-2.12/src_managed/main/avro/`).
## Documentation
### Settings
Name | Default | Description
-----|---------|------------
schemaRegistryEndpoint | ``localhost:8081`` | Schema Registry endpoint
schemas | All Avro schemas (with last version) | A list of (subject, version)
resourceManagedDirectory | ``$resourceManaged/main/avro`` | Path containing *.avsc files from schema registry.
sourceManagedDirectory | ``$sourceManaged/main/avro`` | Path for the generated *.scala files
resourceDirectory | ``$resource/main/avro`` | Path containing your *.avs files.
templateDirectory | ``internal`` | Path containing the templates.
#### Tasks
Task | Description
----------------------------------------|----------------------------------------
download | Download Avro schemas from a Schema Registry
generate | Generate case classes from avro files
upload | Upload Avro schemas to a Schema Registry
compatibility | Test compatibility of Avro schemas against their latest versions in a Schema Registry
Important: `sbt:compile` task will only call avro:generate and not call download task.
#### Custom Templates
You can use your custom templates to generate scala source from Avro schema.
If you want to use custom templates, please add your custom templates into `$templateDirectory` folder.
Template example:
```velocity
#if ($schema.getNamespace())
package $schema.getNamespace()
#end
#if ($schema.getDoc())
/** $schema.getDoc() */
#end
object ${this.mangle($schema.getName())} extends Enumeration {
val Schema = "${this.javaEscape($schema.toString())}"
val #foreach ($symbol in ${schema.getEnumSymbols()})${this.mangle($symbol)}#if ($velocityHasNext), #end#end = Value
}
```
## Contribute
Look at contribution guidelines here : [CONTRIBUTING.md](CONTRIBUTING.md)
### Contributors
- [Victor Pirat](https://github.com/atvictor)
- [Jocelyn Dréan](https://github.com/jocelyndrean)
- [Ian Duffy](https://github.com/imduffy15)