Terasort-like benchmark for spark 2.x that uses dataframes, saves files in parquet etc for a more realistic testing.

sparkbench 1.0.14


Group ID:
com.bigstep
Artifact ID:
sparkbench_2.11
Version:
1.0.14
Release Date:
Jul 13, 2018
Licenses:
Files:

libraryDependencies += "com.bigstep" %% "sparkbench" % "1.0.14"

Mill build tool

ivy"com.bigstep::sparkbench:1.0.14"

Scala CLI

//> using lib "com.bigstep::sparkbench:1.0.14"

Ammonite REPL

import $ivy.`com.bigstep::sparkbench:1.0.14`

<dependency>
  <groupId>com.bigstep</groupId>
  <artifactId>sparkbench_2.11</artifactId>
  <version>1.0.14</version>
</dependency>

compile group: 'com.bigstep', name: 'sparkbench_2.11', version: '1.0.14'