pheymann / artie

Scala test-framework for REST refactorings


Build Status Maven Central

[WIP] artie {from rrt := rest-refactoring-test-framework}

You want to change a (legacy) REST service which has no tests and it is impossible to write some tests without rebuilding the whole thing? If so this tool may help you. It is a small framework to generate REST request from different data sets, run them against two instances of your service (old and new) and compare the responses.

The only thing you have to do is:

import artie._
import artie.implicits._

// write a refactoring spec
object MyServiceRefactoring extends RefactoringSpec("my-service") {

  import DatabaseGenerator.DatabaseConfig

  // give some informations
  val conf = Config("old-host", 8080, "new-host", 8080)
  val db   = mysql("db-host", "user", "pwd")

  // add some data
  val providers = Providers ~
    // random ages between 10 and 100
    ('ages, provide[Int].random(10, 100) ~

    // some user ids from a db
    ('userIds, provide[Long].database.random("users_table", "user_id", limit = 100, db)

  // you have to provide `read` (see below)
  check("get-user", providers, conf, read[User]) { implicit r => p =>
    val userId = select('userIds, p).next
    val ageO   = select('ages, p).nextOpt

    // request builder
    get(s"/user/$userId", Params <&> ("age", ageO))

And run it:

# if both instances behave the same
sbt "it:runMain MyServiceRefactoring"

testing refactorings for my-service:
  + check get-user

Success: Total: 1; Succeeded: 1, Invalid: 0; Failed: 0

# in presence of differences
sbt "it:runMain MyServiceRefactoring"

testing refactorings for my-service:
  + check get-user
    processed: 1 / 1
    Get /user/0?age=20
      city: "Hamburg" != "New York"

Failed: Total: 1; Succeeded: 0, Invalid: 0; Failed: 1

Here Invalid indicates requests which lead to 3.x.x, 4.x.x or 5.x.x responses from both service instances and Failed requests which produced different responses were at least one succeeded.

For some examples take a look here.

Table of Contents

Get This Framework

You can add it as dependency for Scala 2.11 and 2.12:

// take a look at the maven batch to find the latest version
libraryDependencies += "com.github.pheymann" %% "artie" % <version> % Test

or build it locally:

git clone
cd artie
sbt "publishLocal"

In Master you will find the build for Scala 2.12.x. If you need 2.11.x checkout branch 2.11.x.


I tried to keep the dependencies to external libraries as small as possible. Currently this framework uses:

Read responses

You have to provide functions mapping raw json strings to your case class instances. They are called Reads and implemented like this:

// by hand
val userRead = new Read[User] {
  def apply(json: String): Either[String, User] = ???

// or by reusing your mappings from some frameworks
object PlayJsonToRead {

  def read[U](implicit reads: play.json.Reads[U]): Read[U] = new Read[U] {
    def apply(json: String): Either[String, U] = 
      Json.fromJson[U](Json.parse(json)) match {
        case JsSuccess(u, _) => Right(u)
        case JsError(errors) => Left(errors.mkString("\n"))


Providers select a single element randomly on every next from an underlying data set.

To select the next element you have to determine the provide by its id:

// for a scalar value
val id = select('userIds, providers).next

// for an Option; maybe a Some maybe a None
val idO = select('userIds, providers).nextOpt

This id based select is typesafe thanks to shapeless. This means your compiler will tell you if you try to select an non-existing provider.


Provides data from a static sequence:

provide[User].static(User("foo"), User("bar"))


Provides data from a random generator in a range of min / max:

provide[Long].random(0, 100)


Provides data from a Database query:

// provide a query (add a LIMIT as all data is eagerly loaded)!
provide[Long].database("select id from users limit 100", db)

// or randomly select elements
provide[Long].database.random("users", "id", 100, db)

Currently provided are mysql and h2 and have to be used like this:

val db0 = mysql(<host>, <user>, <password>)
val db1 = h2(<host>, <user>, <password>)


Mandatory informations:

  • baseHost: host address of the old service
  • basePort: port of the old service
  • refactoredHost: host address of the new (refactored) service
  • refactoredPort: port of the new (refactored) service

Additional settings (function calls):

  • repetitions: how many requests will be created (repeat this check)
  • parallelism: how many requests can be ran in parallel
  • stopOnFailure: default is true, if set to false the test will continue in the presence of a difference
  • shownDiffsLimit: default is 1, how many diffs are shown

Data Selector

You can select data for your requests by:

select('id, p).next // single element
select('id, p).nextOpt // single element which can be `Some` or `None`
select('id, p).nextSeq(10) // sequence of elements of length 10
select('id, p).nextSet(10) // set of elements of maximum size 10

Request Builder

You can create:

  • get
  • put
  • post
  • delete

requests by calling:

get(<url>, <query-params>, <headers>)

post((<url>, <query-params>, <headers>, Some(<json-string>))

Query Parameter and Headers

If you need query params or headers use:

val p = Params <&> ("a", 1) <&> ("b", Some(0)) <&> ("c", Seq(1, 2, 3))
val h = Headers <&> ("Content-Type", "application/json")

post(???, p, h, Some("""{"id": 0}"""))

Test Suite

You don't want to execute all your specs by hand? Then you artie.suite.RefactoringSpec (as replacement for artie.RefactoringSpec) and artie.suite.RefactoringSuite to build and collect specs you execute together.

Ignore Response Fields

Sometimes it is necessary to ignore some response fields (eg. timestamp). If you don't want to rewrite your json mapping you can provide a IgnoreFields instance:

final case class Log(msg: String, time: Long)

implicit val logIgnore = IgnoreFields[Log].ignore('time)
check("log-endpoint", providers, conf, read[Log]) { ...}

The Symbol has to be equal to the field name. If you write something which doesn't exists in your case class the compiler will tell you.

Response Comparison

Response comparison is done by creating a list of field-value pairs (LabelledGenerics) from your responses of class R and comparing each field:

(old: R, refact: R) => (old: FieldValues, refact: FieldValues) => Seq[Diff]

The result is a sequence of Diff with each diff providing the field name and:

  • the two original values,
  • a set of diffs of the two values.

Currently the framework is able to provide detailed compare-results (fields with values) for:

  • simple case classes (case class User(id: Long, name: String)
  • nested case classes (case class Friendship(base: User, friend: User))
  • sequences, sets and arrays of case classes (case class Group(members: Set[User]))
  • maps of case classes (case class UserTopic(topics: Map[Topic, User]))
  • combination of these

Everythings else will be compare by != and completely reported on failure.

If you need something else take a look here to get an idea how to implement it.

Add your Database

You can add your Database as easy as this:

trait Mysql extends Database {

  val driver = "com.mysql.jdbc.Driver"

  def qualifiedHost = "jdbc:mysql://" + host

  def randomQuery(table: String, column: String, limit: Int): String =
    s"""SELECT DISTINCT t.$column
       |FROM $table AS t
       |ORDER BY RAND()
       |LIMIT $limit

object Mysql {

  def apply(_host: String, _user: String, _password: String) = new Mysql {
    val host = _host
    val user = _user
    val password = _password