Build Status CircleCI codecov

Quality Gate Status Maintainability Rating Commitizen friendly

Javadocs Maven

Gitter

Introduction

Let me go straight to the point. I'd argue that this is the most declarative, concise, composable, and beautiful Json generator in the whole world! As a functional developer, tired of spending much time manipulating mutable Jsons, I developed json-values, which is the first immutable Json in the JVM ecosystem implemented with persistent data structures. To test that library, I used property-based-testing with ScalaCheck, and I developed several Json generators that I decided to publish in this repo.

What to use json-scala-values-generator for

If you practice property-based testing and use ScalaCheck, you'll be able to design composable Json generators very quickly and naturally, as if you were writing out a Json.

Defining custom Json generators

Using json-scala-values, you can create Jsons in different ways. One of them turns out to be very natural because it's close to the Json representation itself. It may ring a bell if you have taken the Scala courses from Martin Odersky on Coursera.

import jsonvalues.{JsObj,JsArray}
import jsonvalues.Preamble._

JsObj("@type" -> "person",
      "name" -> "Rafael Merino García",
      "birth_date" -> "13-03-1982",
      "email" -> "imrafaelmerino@gmail.com",
      "gender" -> "Male",
      "address" -> JsObj("country" -> "ES",
                         "location" -> JsArray(40.1693500,
                                               -4.2154900
                                                 )
                         )
      )

Let's create a person generator using the same philosophy:

import jsonvalues.JsObj
import jsonvalues.Preamble._
import jsonvaluesgen.Preamble._
import jsonvaluesgen.{JsObjGen,JsArrayGen}
import org.scalacheck.Gen

def nameGen: Gen[String] = ???
def birthDateGen: Gen[String] = ???
def latitudeGen: Gen[Double] = ???
def longitudeGen: Gen[Double] = ???
def emailGen: Gen[String] = ???
def countryGen: Gen[String] = ???

def gen:Gen[JsObj] = JsObjGen("@type" -> "person",
                              "name" -> nameGen,
                              "birth_date" -> birthDateGen,
                              "email" -> emailGen,
                              "gender" -> Gen.oneOf("Male",
                                                    "Female"
                                                    ),
                              "address" -> JsObjGen("country" -> countryGen,
                                                    "location" -> JsArrayGen(latitudeGen,
                                                                             longitudeGen
                                                                             )
                                                    )
                              )

If you are using other Json library, you can still use this generator mapping the generated json into its string representation, and then creating your object from that string:

import x.y.z.MyJson

def gen:Gen[MyJson] =  personGen.map(MyJson(_.toString))

Another way of creating Jsons in json-scala-values is from pairs of paths and values:

import jsonvalues.JsObj
import jsonvalues.JsPath._
import jsonvalues.Preamble._

JsObj( ("@type" -> "person"), 
       ("name" -> "Rafael Merino García"),
       ("birth_date" -> "13-03-1982"),
       ("email" -> "imrafaelmerino@gmail.com"),
       ("gender" -> "Male"),
       ("address" / "country" -> "ES"),
       ("address" / "location" / 0 -> 40.1693500),
       ("address" / "location" / 1 -> -4.2154900)
     )

And again, we can create Json generators following the same approach:

import jsonvalues._
import jsonvalues.JsPath._
import jsonvalues.Preamble._
import jsonvaluesgen._
import jsonvaluesgen.Preamble._
import org.scalacheck.Gen

def nameGen: Gen[String] = ???
def birthDateGen: Gen[String] = ???
def latitudeGen: Gen[Double] = ???
def longitudeGen: Gen[Double] = ???
def emailGen: Gen[String] = ???
def countryGen: Gen[String] = ???

JsObjGen.fromPairs(("@type" -> "person"), 
                  ("name" -> nameGen),
                  ("birth_date" -> birthDateGen),
                  ("email" -> emailGen),
                  ("gender" -> Gen.oneOf("Male",
                                         "Female"
                                        )
                  ),
                  ("address" / "country" -> countryGen),
                  ("address" / "location" / 0 -> latitudeGen),
                  ("address" / "location" / 1 -> longitudeGen)
                 )

A typical scenario is when we want some elements not to be always generated, which can be easily achieved using the special value jsonvalues.JsNothing. Inserting JsNothing in a Json at a path is like removing the element. Taking that into account, let's create a generator that produces Jsons without the key name with a probability of 50 percent:

def nameGen: Gen[JsStr] = ???

def optNameGen: Gen[JsValue] = Gen.oneOf(JsNothing,nameGen)

JsObjGen("@type" -> "person",
         "name" -> optNameGen,
         ...
         )

//syntactic sugar to do the same thing but typing less!
JsObjGen("@type" -> "person",
         "name" ->  ?(nameGen),
         ...
         )

And we can change that probability using the ScalaCheck function Gen.frequencies:

def nameGen: Gen[JsStr] = ???

def optNameGen: Gen[JsValue] = Gen.frequencies((10,JsNothing),
                                               (90,nameGen)
                                            )

JsObjGen("@type" -> "person",
         "name" ->  optNameGen,
         ...
         )

//syntactic sugar to do the same thing but typing less!
JsObjGen("@type" -> "person",
         "name" ->  ?(90,nameGen),
         ...
        )

Defining random Json generators

There are times when you are only interested in generating random Jsons; after all, every function of a Json API has to work, no matter the Json it's tested with.

import jsonvaluesgen.{RandomJsObjGen,RandomJsArrayGen}

//produces any imaginable Json object
def randomObjGen: Gen[JsObj] = RandomJsObjGen()

//produces any imaginable Json array
def randomArrayGen: Gen[JsArray] = RandomJsArrayGen()

These random generators are also customizable to some extent. The following named parameters can be passed in:

  • arrLengthGen: Gen[Int]: to control the length of arrays.
  • objSizeGen: Gen[Int]: to control the size of objects. Take into account that if JsNothing is generated, no element is inserted and the final size of the object may be lower than the returned by this generator:
val gen = RandomJsObjGen(objSizeGen= Gen.const(5),
                         objPrimitiveGen = PrimitiveGen(strGen=Gen.oneOf(JsNothing,JsStr("a")))
                        )

In the previous example, for those cases where JsNothing is generated, the size of the Json object will be four and not five.

  • keyGen: Gen[String]: to control the name of the keys in objects.
  • arrValueFreq: ValueFreq: to control the type of the elements generated in arrays. For primitive types, values are generated by the corresponding generator defined in the param arrPrimitiveGen. Nested objects can be generated, i.e. array of objects or array of arrays and so on. Nested objects have to be configured carefully to not blow up the stack due to recursion.
  • arrPrimitiveGen: PrimitiveGen: to control the value of the primitive types generated in arrays.
  • objValueFreq: ValueFreq: to control the type of the elements generated in objects. For primitive types, the value are generated by the generator defined in the param objPrimitiveGen.
  • objPrimitiveGen: PrimitiveGen: to control the value of the primitive types generated in objects. Find below the definition of the classes PrimitiveGen and ValueFreq:
val ALPHABET: Seq[String] = "abcdefghijklmnopqrstuvwzyz".split("").toIndexedSeq

case class PrimitiveGen(strGen: Gen[String] = Gen.oneOf(ALPHABET),
                        intGen: Gen[Int] = Arbitrary.arbitrary[Int],
                        longGen: Gen[Long] = Arbitrary.arbitrary[Long],
                        doubleGen: Gen[Double] = Arbitrary.arbitrary[Double],
                        floatGen: Gen[Float] = Arbitrary.arbitrary[Float],
                        boolGen: Gen[Boolean] = Arbitrary.arbitrary[Boolean],
                        bigIntGen: Gen[BigInt] = Arbitrary.arbitrary[BigInt],
                        bigDecGen: Gen[BigDecimal] = Arbitrary.arbitrary[BigDecimal]
                        ) 
           
case class ValueFreq(obj: Int = 1,
                     arr: Int = 1,
                     str: Int = 5,
                     int: Int = 5,
                     long: Int = 5,
                     double: Int = 5,
                     bigInt: Int = 5,
                     bigDec: Int = 5,
                     bool: Int = 5,
                     `null`: Int = 5
                     )

As you may notice, the class ValueFreq has two params obj and arr that allows you to generate nested Jsons. The default frequency assigned to them is lower than the rest, otherwise the process can diverge and a StackOverFlowException would be thrown.

To make it clearer, let's define a JsObj generator with the following specifications:

  • max size of 10
  • keys of three letters from the alphabet
  • values are String or Int or JsArray with the same probability, where:
    • String values are colors
    • Int values are numbers between -100 y 100
    • JsArrays are never empty, max length of 5, values are either arbitrary booleans or null with a probability of 90% and 10% respectively
def arrLengthGen:Gen[Int] = Gen.choose(1,5)
def objSizeGen:Gen[Int] = Gen.choose(0,10)
def letterGen:Gen[String] = Gen.oneOf(ALPHABET)
def keyGen: Gen[String] = for {
                                a <- letterGen
                                b <- letterGen
                                c <- letterGen
                              } yield s"$a$b$c"

def objectValueFreq:ValueFreq = ValueFreq(arr = 1,
                                          str = 1,
                                          obj = 0,
                                          int = 1,
                                          long = 0,
                                          double = 0,
                                          bigInt = 0,
                                          bigDec = 0,
                                          bool = 0,
                                          `null` = 0
                                          )

def objectPrimitiveGen:PrimitiveGen = PrimitiveGen(strGen = Gen.oneOf("blue","red","brown"),
                                                   intGen = Gen.choose(-100,100)
                                                  )

def arrayValueFreq:ValueFreq = ValueFreq(arr = 0,
                                         str = 0,
                                         obj = 0,
                                         int = 0,
                                         long = 0,
                                         double = 0,
                                         bigInt = 0,
                                         bigDec = 0,
                                         bool = 9,
                                         `null` = 1
                                        )


def jsonGen:Gen[JsObj] = RandomJsObjGen(objValueFreq = objectValueFreq,
                                        objPrimitiveGen = objectPrimitiveGen,
                                        keyGen = keyGen,
                                        objSizeGen = objSizeGen,
                                        arrValueFreq = arrayValueFreq,
                                        arrLengthGen = arrLengthGen
                                        )

Composing Json generators

Composing Json generators is key in order to handle complexity and reuse code avoiding repetition. There are two ways, inserting pairs into generators and joining generators:

def addressGen:Gen[JsObj] = JsObjGen("street" -> streetGen, 
                                     "city" -> cityGen, 
                                     "zip_code" -> zipCodeGen
                                    )

//let's insert location generators (JsPath,Gen[JsValue]) into our addressGen
def addressWithLocationGen:Gen[JsObj] = JsObjGen.inserted(addressGenerator,
                                                          ("location" / 0, latitudeGen),
                                                          ("location" / 1, longitudeGen)
                                                          )

def namesGen = JsObjGen("family_name" -> familyNameGen,
                        "given_name" -> givenNameGen)

def contactGen = JsObjGen("email" -> emailGen,
                          "phone" -> phoneGen,
                          "twitter_handle" -> handleGen
                         )

def clientGen = JsObjGen.concat(namesGen,
                                contactGen,
                                addressWithLocationGen
                               )

Installation

libraryDependencies += "com.github.imrafaelmerino" %% "json-scala-values-generator" % "1.2.1" % "test"

// you need a version of json-scala-values in your classpath

libraryDependencies += "com.github.imrafaelmerino" %% "json-scala-values" % "X.Y.Z"