julianpeeters / sbt-avrohugger

sbt plugin for generating Scala sources for Apache Avro schemas and protocols.

GitHub

sbt-avrohugger

Travis CI Join the chat at https://gitter.im/julianpeeters/avrohugger Maven Central

sbt plugin for generating Scala case classes and ADTs from Apache Avro schemas, datafiles, and protocols.

Install the plugin (compatible with sbt 1.x)

Add the following lines to the file myproject/project/plugins.sbt in your project directory:

addSbtPlugin("com.julianpeeters" % "sbt-avrohugger" % "2.0.0-RC22")

NOTE: On Nexus, please set nexus proxy layout to permissive in to resolve artifacts with a sbt-version suffixes

Usage

The following tasks and settings are automatically imported to your build:

Tasks:

Name Description
avroScalaGenerate Compiles the Avro files into Scala case classes.
avroScalaGenerateScavro Compiles the Avro files into Scala case class Scavro wrapper classes.
avroScalaGenerateSpecific Compiles the Avro files into Scala case classes implementing SpecificRecord.

Compile

Wire the tasks into compile in your build.sbt:

e.g.: sourceGenerators in Compile += (avroScalaGenerate in Compile).taskValue

By default, the plugin looks Avro files in src/main/avro and generates Scala files in $sourceManaged, e.g., target/scala-2.12/src_managed/main/compiled_avro/ (to choose different locations, please see Changing Settings).

Test

And/Or wire the tasks into the Test config, putting Avro files in src/test/avro:

e.g. sourceGenerators in Test += (avroScalaGenerate in Test).taskValue

Manually

To run the tasks manually, please see Changing Settings or the sbt docs in order to ensure the compiler will be able to find the generated files.

Watch Avro Files

To enable file-watching for avro files, e.g. in ~compile, use:

e.g.: watchSources ++= ((avroSourceDirectory in Compile).value ** "*.avdl").get

Settings:

Standard Settings

Name Default Description
avroSourceDirectories Seq("src/main/avro") List of paths containing *.avsc, *.avdl, and/or *.avro files.
avroScalaSource $sourceManaged/main/compiled_avro Path for the generated *.scala or *.java files.
avroScalaCustomTypes Standard.defaultTypes Customizable Type Mapping.
avroScalaCustomNamespace Map.empty[String, String] Map for reassigning namespaces.

Scavro Settings

Name Default Description
avroScavroSourceDirectories Seq("src/main/avro") List of paths containing *.avsc, *.avdl, and/or *.avro files.
avroScavroScalaSource $sourceManaged/main/compiled_avro Path for the generated *.scala or *.java files.
avroScalaScavroCustomTypes Scavro.defaultTypes Customizable Type Mapping.
avroScalaScavroCustomNamespace Map.empty[String, String] Map for reassigning namespaces.

SpecificRecord Settings

Name Default Description
avroSpecificSourceDirectories Seq("src/main/avro") Path containing *.avsc, *.avdl, and/or *.avro files.
avroSpecificScalaSource $sourceManaged/main/compiled_avro Path for the generated *.scala or *.java files.
avroScalaSpecificCustomTypes SpecificRecord.defaultTypes Customizable Type Mapping.
avroScalaSpecificCustomNamespace Map.empty[String, String] Map for reassigning namespaces.

Changing Settings

Settings for each format's task can be extended/overridden by adding lines to your build.sbt file.

E.g., to change how classes of SpecificRecords format are generated, use:

avroSpecificSourceDirectories in Compile += (sourceDirectory in Compile).value / "myavro"

avroSpecificScalaSource in Compile := new java.io.File("myScalaSource")

avroScalaSpecificCustomNamespace in Compile := Map("example"->"overridden")

avroScalaSpecificCustomTypes in Compile := {
  avrohugger.format.SpecificRecord.defaultTypes.copy(
    array = avrohugger.types.ScalaVector)
}
  • record can be assigned to ScalaCaseClass and ScalaCaseClassWithSchema (with schema in a companion object)
  • array can be assigned to ScalaSeq, ScalaArray, ScalaList, and ScalaVector
  • enum can be assigned to JavaEnum, ScalaCaseObjectEnum, EnumAsScalaString, and ScalaEnumeration
  • union can be assigned to OptionEitherShapelessCoproduct and OptionalShapelessCoproduct
  • int, long, float, double can be assigned to ScalaInt, ScalaLong, ScalaFloat, ScalaDouble
  • date logical type can be assigned to JavaTimeLocalDate and JavaSqlDate
  • timestamp-millis logical type can be assigned to JavaTimeInstant and JavaSqlTimestamp
  • uuid logical type can be assigned to UUID
  • decimal can be assigned to e.g. ScalaBigDecimal(Some(BigDecimal.RoundingMode.HALF_EVEN)) and ScalaBigDecimalWithPrecision(None) (via Shapeless Tagged Types)
  • protocol can be assigned to ScalaADT and NoTypeGenerated (see Protocol Support)

Datatypes

Supports generating case classes with arbitrary fields of the following datatypes: see avrohugger docs - supported datatypes

Testing

Please run unit tests in src/test/scala with ^ test, and integration tests in src/sbt-test with ^ scripted sbt-avrohugger/*, or, e.g. scripted sbt-avrohugger/specific.

Credits

sbt-avrohugger is based on sbt-avro by Juan Manuel Caicedo, and depends on avrohugger.

Contributors

Fork away, just make sure the tests pass before you send a pull request.

Criticism is appreciated.