univalence / spark-tools

GitHub
Version Matrix

Spark-tools

Spark-tools is a set of tools dedicated to Spark and aims to make the life of data engineers easier.

The tools

  • Centrifuge, a couple of techniques for data quality on Spark
  • Fenek, a DSL for semi-typed transformation in Scala for Spark
  • Plumbus, light misc things for Spark
  • Spark-ZIO, Spark in ZIO environment

Documentation

libraryDependencies += "io.univalence" %% "spark-tools" % "0.2+134-05f4f7c5"
Ammonite REPL 
import $ivy.`io.univalence::spark-tools:0.2+134-05f4f7c5`
<dependency>
  <groupId>io.univalence</groupId>
  <artifactId>spark-tools_2.11</artifactId>
  <version>0.2+134-05f4f7c5</version>
</dependency>
compile group: 'io.univalence', name: 'spark-tools_2.11', version: '0.2+134-05f4f7c5'
Mill build tool 
ivy"io.univalence::spark-tools:0.2+134-05f4f7c5"
Try online with Scastie
Try on Scastie

Statistics

Contributors

License

6 Dependencies

Java Dependencies