Version Matrix

鸕鶿lu ci

Extensible Free Monad Effects

Do one thing and do it well micro birds library series

libraryDependencies += "us.oyanglul" %% "luci" % <version>"

Table of Contents

The Problem

When you want to mix in some native effects into your Free Monad DSL, some effects won't work

for instance, we have two effects IO and StateT, and we would like to do some math using StateT

Here is the Program

import Free.{liftInject => free}
type Program[A] = EitherK[IO, StateT[IO, Int, ?], A]
type ProgramF[A] = Free[Program, A]

def program : Program[Int] = for {
  initState <- free[Program](StateT.get[IO, Int])
  _ <- free[Program](IO(println(s"init state is $initState")))
  _ <- free[Program](StateT.modify[IO, Int](_ + 1))
  res <- free[Program](StateT.modify[IO, Int](_ + 1))
} yield res

and Interpreters

def ioInterp =[IO]
def stateTInterp(initState: Int) = Lambda[StateT[IO, Int, ?] ~> IO[(Int, ?)]] {}

If we run the program

program foldMap (ioInterp or stateTInterp(0))

Guess what, it doesn't work, the result will be 1 not 2

because we run the state for each effect separately in the stateTInterp

One of the option is to use FreeT

But with FreeT:

  • you can only mixin 1 effect, what if I have multiple effects that I want them to be stateful across the whole program.
  • all other effects need to be lift to FreeT as well. It might have huge impact to our existing code base that is Free already.

The Ultimate Solution

is using both meow-mtl and ReaderT/Kleisli, then we can easily integrate mtl into Free Monad Effects

  1. instead of using interpreter Program ~> IO, we can use Program ~> Kleisli[IO, ProgramContext, ?], and we have a better name for it - Compiler
  2. init state of stateful effects can then be injected into program via ProgramContext when actually running Kleisli[IO, ProgramContext, ?]

Some Effects out of the box

  • Id
  • WriterT
  • ReaderT/Kleisli
  • StateT
  • EitherT
  • Http4sClient
  • Doobie ConnectionIO
  • fs2

It's very similar but just one more step to run the Kleisli

  1. create Program dsl
  2. compile Program into a Kleisli
  3. run Kleisli in a context

Step 1: Create DSL

e.g. our Program has lot of effects... WriterT, Http4sClient, ReaderT, IO, StateT and Doobie's ConnectionIO

few of them need to be stateful across all over the program like WriterT, StateT

 type Program[A] = Eff7[
      Http4sClient[IO, ?],
      WriterT[IO, Chain[String], ?],
      ReaderT[IO, Config, ?],
      StateT[IO, Int, ?],
      Either[Throwable, ?],
type ProgramF[A] = Free[Program, A]

EffX is predefined alias of type to construct multiple kind in EitherK

Now lets start using these effects to do our work

val program = for {
    config <- free[Program](Kleisli.ask[IO, Config])
    _ <- free[Program](
    _ <- free[Program](StateT.modify[IO, Int](1 + _))
    _ <- free[Program](StateT.modify[IO, Int](1 + _))
    state <- free[Program](StateT.get[IO, Int])
    _ <- free[Program](
    WriterT.tell[IO, Chain[String]]("config: " + config.token)))
    resOrError <- free[Program](sql"""select true""".query[Boolean].unique)
    _ <- free[Program](
    resOrError.handleError(e => println(s"handle db error $e")))
    _ <- free[Program](IO(println(s"im IO...state: $state")))
} yield ()

Step 2: Compile the Program

if we compile our program, we should get a binary ProgramBin

val binary = compile(program)

imagine that you have a binary of command line tool, when you run it you would probably need to provide some --args

same here, if you want to run ProgramBin, which is basically just a Kleisli, we need to provide args with is ProgramContext

Step 3: Run Program

run the program with real --args

val args = (httpclient ::
    logRef.tellInstance ::
    config ::
    Unit ::
    transactor ::
    stateRef.stateInstance ::
    Unit ::

for stateful WriterT and StateT here, we can get FunctorTell and MonadState instances from Ref[IO, ?] and inject them into program via ProgramContext

each one corespond to program's effect's context

  1. binary for Http4sClient[IO, ?] needs Client[IO] to run
  2. binary for WriterT[IO, Chain[String], ?] needs FuntorTell[IO, Chain[String]], presented by meow-mtl .tellInstance
  3. binary for ReaderT[IO, Config, ?] needs Config to run
  4. binary for IO needs nothing so Unit
  5. binary for ConnectionIO needs Transactor[IO]
  6. binary for StateT[IO, Int, ?] needs MonadState[IO, Int] to run, which presented here by meow-mtl from .stateInstance
  7. binary for Either[Throwable, ?] needs nothing so Unit

Create Your Own Effect

creating an new compilable effect is pretty simple in 2 steps

Step 1: Create Data Type

This is nothing different from creating a effect data type for Free Monad

For instance we need a s3 putObject Effect


sealed trait S3[A]

case class PutObject(bucketName: String, fileName: String, content: String)
    extends S3[PutObjectResult]

Step 2: Create Compiler

To create a compiler for new data type s3, we'll need to create instance for type class Compiler

trait Compiler[F[_], E[_]] {
  type Env <: HList
  val compile: F ~> Kleisli[E, Env, ?]

We need a type of Env where the program needs to be compile. e.g. S3 need a AWS S3 Client

trait S3Compiler[E[_]] {
  implicit def s3Compiler(implicit F: Applicative[E]) = new Compiler[S3, E] {
    type Env = AmazonS3 :: HNil
    val compile = Lambda[S3 ~> Kleisli[E, Env, ?]] (_ match {
      case PutObject(bucketName, fileName, content) =>
        Kleisli(env => F.pure(env.head.putObject(bucketName, fileName, content)))

There were few point to be noted here:

  1. be mindful that the Env needs to be a shapeless HList, thus, here it's AmazonS3 :: HNil not AmazonS3
  2. the compiler is generic on E[_], but with restriction that E has to have instance for Applicative so we can use .pure
  3. Kleisli will get env of type AmazonS3 :: HNil, it's just like list but more acurate on it's content, you can get head safely since we know that there must be an item of type AmazonS3 in head.

Step 3 Use the effect

to be honest you don't need to make S3Compiler so generic since you may be the only person who using it. But it's a good practic to make every thing as genric as possible.

any way to use the generic effect, you can create a specific object just for IO(or Task of your choice)

object s3IoCompiler extends S3Compiler[IO]

and then import it to where you need to compile

import s3IoCompiler._

Or, simply extends it on the object or class you intent to compile your program

object Main extends S3Compiler[IO] with All{
  val binary = compile(program)