SBT plugin for downloading Avro schemas from a Confluent Schema Registry. Adds a schemaRegistryDownload task and configuration settings to declare which schemas to fetch.

Add the following to your project/plugins.sbt:

resolvers ++= Seq(
  "Confluent" at "https://packages.confluent.io/maven/",
)

addSbtPlugin("org.galaxio" % "sbt-schema-registry-plugin" % "<plugin-version>")

The plugin is built as a standard sbt 1.x autoplugin and requires Scala 2.12 (the Scala version used by sbt itself). It does not depend on the Scala version of your project — you can use it in a Scala 2.13 or Scala 3 project without any changes.

In your build.sbt:

import org.galaxio.avro.{RegistrySubject, SchemaRegistryAuth}

val schemas = Seq(
  RegistrySubject("hello.world.schema", 2),      // specific version
  RegistrySubject("schema1-name", 12),            // specific version
  RegistrySubject.latest("schema2-name"),          // latest version — resolved at task run time
)

lazy val root = (project in file("."))
  .settings(
    schemaRegistryUrl      := "http://schema-registry-host:8081",
    schemaRegistrySubjects ++= schemas,
  )

schemaRegistryAuth := Some(SchemaRegistryAuth.BasicAuth("username", "password"))

Pass any Confluent client properties (SSL, timeouts, etc.):

schemaRegistryProperties := Map(
  "schema.registry.ssl.truststore.location" -> "/path/to/truststore.jks",
  "schema.registry.ssl.truststore.password" -> "changeit",
)

Download all schemas listed in schemaRegistrySubjects:

sbt "Compile / schemaRegistryDownload"

Schema files are saved as <subject>-<version>.avsc in schemaRegistryTargetFolder (default: src/main/avro). The build will fail if any schema download fails.

Output file behaviour: by default incremental download is enabled (schemaRegistryIncremental := true). The task records downloaded versions in a manifest (.schema-versions.json under the project's sbt cache directory) and skips any subject whose version is already current, so unchanged schemas are not re-fetched or re-written. Set schemaRegistryIncremental := false to overwrite every file on every run. For RegistrySubject.latest(...) subjects the version number is resolved at task-run time, so the output filename (e.g. my-subject-7.avsc) may change between runs when the latest version advances.

Schemas can reference other schemas (e.g. an Order that embeds a Customer type). Confluent Schema Registry tracks these references; downloading the referencing schema alone is not enough — consumers also need the referenced schemas.

When schemaRegistryResolveReferences is true (the default), downloading a subject also downloads every schema it references, transitively. Each referenced schema is fetched at the exact version recorded in the reference (pinned, not latest) and written using the normal <subject>-<version>.<ext> layout, so a single download run produces a complete, self-contained set of files.

schemaRegistrySubjects += RegistrySubject.latest("order-value")
// If order-value references customer-value, both files are downloaded:
//   src/main/avro/order-value-3.avsc
//   src/main/avro/customer-value-1.avsc

Resolution is cycle-safe (a reference graph with cycles terminates) and de-duplicates by subject+version, so a shared dependency is written once while two genuinely different pinned versions of the same subject are written as separate files. The first failed reference fetch fails the whole task with an error naming the failing subject.

To download only the explicitly requested subjects (the pre-1.7 behaviour), disable it:

schemaRegistryResolveReferences := false

Known limitation: the incremental-download manifest keys by subject name (one version per subject). If a single run resolves two different versions of the same subject, only the last-written version is recorded, so the other may be re-downloaded (identical bytes) on a later incremental run. This is wasteful but never incorrect, and only affects the rare divergent-version case.

Select subjects by regex instead of (or in addition to) listing them explicitly. Patterns are matched against every subject reported by the registry; each match is downloaded at its latest version.

schemaRegistrySubjectPatterns := Seq("""order-.*-value""", """.*\.events""")

Enabled by default. Downloaded versions are tracked in a .schema-versions.json manifest under the project's sbt cache directory, and subjects already at the current version are skipped. Disable to always re-fetch and overwrite:

schemaRegistryIncremental := false

Schemas are fetched concurrently with a bounded thread pool. Tune the degree of concurrency (1 = sequential) and the retry budget for transient failures:

schemaRegistryParallelism := 8   // 1..32
schemaRegistryRetries     := 3   // 0..10, 0 = no retry

Discover what exists in the registry without reaching for curl or a separate UI. The schemaRegistryListSubjects task fetches every subject — sorted, with its version range and compatibility level — and prints it to the sbt log. It reuses the same connection settings (schemaRegistryUrl, schemaRegistryAuth, schemaRegistryProperties) as the other tasks.

sbt schemaRegistryListSubjects
# [info] Found 42 subject(s):
# [info]   user-value                                (versions: 1..5, compat: BACKWARD)
# [info]   order-value                               (versions: 1..3, compat: (default))

A subject with no subject-level compatibility override is shown as (default) (the global default applies). Compatibility is advisory and best-effort: any failed compatibility lookup (including a permissions error) is also shown as (default) and never fails the task — only the subject list and per-subject versions are hard requirements. Narrow the listing with schemaRegistrySubjectFilter — a case-insensitive substring match on the subject name (an unset or empty filter lists everything):

sbt 'set schemaRegistrySubjectFilter := Some("order")' schemaRegistryListSubjects
# [info] Found 1 subject(s):
# [info]   order-value                               (versions: 1..3, compat: FULL)

Per-subject metadata (versions + compatibility) is fetched concurrently using the same schemaRegistryParallelism budget as downloads (1 = sequential, up to 32), so large registries list quickly. The task is read-only and never registers, deletes, or mutates anything. It fails with a clear message if the registry is unreachable, or if a listed subject's versions cannot be fetched.

Register (push) local schema files to Schema Registry:

import org.galaxio.avro.{RegistryRegistration, SchemaType}

schemaRegistryRegistrations := Seq(
  RegistryRegistration("user-value", baseDirectory.value / "src/main/avro/User.avsc"),
  RegistryRegistration("order-value", baseDirectory.value / "src/main/avro/Order.avsc"),
)

sbt "Compile / schemaRegistryRegister"

The default schema type is Avro. For Protobuf or JSON Schema, specify the type explicitly:

RegistryRegistration("user-value", file("src/main/avro/User.proto"), SchemaType.Protobuf)
RegistryRegistration("user-value", file("src/main/avro/User.json"), SchemaType.Json)

Note: Protobuf and JSON Schema require the corresponding Confluent provider dependencies (kafka-protobuf-provider or kafka-json-schema-provider) on the sbt classpath.

All connection settings (schemaRegistryUrl, schemaRegistryAuth, schemaRegistryProperties) are shared between download, registration, and compatibility tasks.

Verify that local schemas are compatible with versions already registered in Schema Registry before deploying:

sbt "Compile / schemaRegistryTestCompatibility"

The task uses the same schemaRegistryRegistrations setting as schemaRegistryRegister. For each subject it calls testCompatibilityVerbose against the registry and reports:

• Compatible — the new schema can safely be registered
• Incompatible — the registry would reject registration; verbose messages explain why
• Failed — an error occurred (file not found, parse error, network issue)

The build fails if any subject is incompatible or fails. Use this in CI before registration to catch breaking changes early:

Compile / compile := (Compile / compile)
  .dependsOn(Compile / schemaRegistryTestCompatibility)
  .value

sbt "Compile / schemaRegistryDownload"

To ensure schemas are always downloaded before compilation, make compile depend on the download task:

Compile / compile := (Compile / compile)
  .dependsOn(Compile / schemaRegistryDownload)
  .value

With this in place a plain sbt compile (or sbt test) will pull fresh schemas first.

If you use a code-generation tool (e.g. sbt-avro) that reads .avsc files and produces Scala sources, register the download task as a source generator so sbt's task graph runs it before code generation:

Compile / sourceGenerators += (Compile / schemaRegistryDownload).map(_ => Seq.empty[File])

This wires schemaRegistryDownload into the standard sourceGenerators chain without declaring any generated source files itself (the actual source generation is handled by the Avro plugin downstream).

import org.galaxio.avro.{RegistrySubject, SchemaRegistryAuth}

lazy val root = (project in file("."))
  .settings(
    // Schema registry connection
    schemaRegistryUrl  := sys.env.getOrElse("SCHEMA_REGISTRY_URL", "http://localhost:8081"),
    schemaRegistryAuth := sys.env.get("SCHEMA_REGISTRY_USER").map { user =>
      SchemaRegistryAuth.BasicAuth(user, sys.env("SCHEMA_REGISTRY_PASSWORD"))
    },

    // Schemas to download
    schemaRegistrySubjects ++= Seq(
      RegistrySubject("com.example.OrderCreated", 3),
      RegistrySubject.latest("com.example.OrderUpdated"),
    ),

    // Output directory (must match your Avro code-gen plugin's source directory)
    schemaRegistryTargetFolder := file("src/main/avro"),

    // Download schemas before compiling
    Compile / compile := (Compile / compile)
      .dependsOn(Compile / schemaRegistryDownload)
      .value,
  )

Symptom: the task fails with an HTTP 401 or a message like Unauthorized.

Checks:

• Confirm the credentials are correct. Test directly:

  curl -u "$USER:$PASS" "$SCHEMA_REGISTRY_URL/subjects"

• Make sure schemaRegistryAuth is set before the task runs. If credentials come from environment variables, verify they are exported in the shell that runs sbt.
• Basic auth must be enabled on the registry side; some deployments use mTLS or token-based auth instead — those require custom schemaRegistryProperties rather than BasicAuth.

Symptom: the task fails with SSLHandshakeException, PKIX path building failed, or similar.

Solution: supply the truststore (and optionally keystore) via schemaRegistryProperties:

schemaRegistryProperties := Map(
  // Trust store — the CA that signed the registry's certificate
  "schema.registry.ssl.truststore.location" -> "/etc/ssl/certs/registry-ca.jks",
  "schema.registry.ssl.truststore.password" -> "changeit",

  // Key store — only needed for mutual TLS (mTLS)
  "schema.registry.ssl.keystore.location"   -> "/etc/ssl/certs/client.jks",
  "schema.registry.ssl.keystore.password"   -> "changeit",
)

The keys map directly to Confluent Schema Registry client configuration properties.

If the registry certificate is signed by a well-known CA that is already in the JVM's default truststore, no additional configuration is required.

Symptom: build fails with HTTP 404 or Subject not found.

Checks:

• Verify the subject name is spelled exactly as registered (subject names are case-sensitive).
• Confirm the requested version exists:

  curl "$SCHEMA_REGISTRY_URL/subjects/<subject-name>/versions"

• For RegistrySubject.latest(...), ensure at least one version has been registered under that subject.

RegistrySubject.latest("subject") resolves the current latest version at task-run time. Each time a new schema version is published to the registry the downloaded file name will change (e.g. from subject-4.avsc to subject-5.avsc). If you need a stable, reproducible build, pin to a specific version number instead:

RegistrySubject("subject", 4)   // always downloads version 4

Built with sbt 1.12.11 on Scala 2.12.21 (the Scala version sbt runs on). The build is split into two modules: the plugin itself (root) and an it subproject that holds the Testcontainers-based integration tests.

sbt scalafmtAll scalafmtSbt   # format
sbt compile test              # compile + unit tests (no external services)
sbt it/test                   # integration tests — spins up Schema Registry + Kafka, requires Docker
sbt scripted                  # plugin e2e tests (download-success needs Docker)

ci.yml runs formatting, unit tests, integration tests, and scripted tests on every PR and on main / release/*.

Trunk-based: main is the trunk; cut release/* branches from it for stabilization. Releases are tag-driven — push a vX.Y.Z tag on main or a release/* branch and release.yml will:

1. verify the tag sits on main / release/*,
2. run sbt compile test,
3. publish to Maven Central via sbt-ci-release (version derived from the tag by dynver),
4. generate release notes from Conventional Commits with git-cliff (cliff.toml) and create a GitHub Release.

Apache 2.0