Hxl is a pure applicative batching library for Scala.
Hxl is based on the ideas presented in Haxl, but diverges in in a few ways. Notably, Hxl does not perform side effects, but is instead based on a free applicative structure.
Hxl is very small (only a couple hundred lines of code) and only depends on cats.
Hxl is written in tagless final, which allows for molding the library to your needs.
Hxl is available on Maven Central for Scala 2.13 and 3.2.
libraryDependencies += "com.github.casehubdk" %% "hxl" % "0.3.0"There are two primitive structures in Hxl: DataSource[F, K, V] and DSKey[K, V].
A DataSource[F, K, V] abstracts over a function NonEmptyList[K] => F[Map[K, V]].
A DataSource is uniquely (as in Scala universal equals) identified by its DSKey.
import cats._
import cats.implicits._
final case class MyDSKey(id: String)
case object MyDSKey extends DSKey[MyDSKey, String]
val database = Map(
"foo" -> "bar",
"baz" -> "qux"
)
def dataSource[F[_]: Applicative] = DataSource.from[F, MyDSKey, String](MyDSKey) { keys =>
keys.toList.flatMap(key => database.get(key.id).toList.tupleLeft(key)).toMap.pure[F]
}
val fa: Hxl[Id, Option[String]] = Hxl(MyDSKey("foo"), dataSource[Id])
val fb: Hxl[Id, Option[String]] = Hxl(MyDSKey("baz"), dataSource[Id])
Hxl.runSequential((fa, fb).mapN(_.mkString + " " + _.mkString)) // "bar qux"Hxl forms an applicative, but sometimes you need a monad.
Hxl is like Validated from cats, in that it can escape it's applicative nature via a method andThen.
However, if you need Hxl to become a monad (like Either is to Validated), you can use request a monadic view of your effect:
val fa: Hxl[F, String] = ???
val m: HxlM[F, String] = fa.monadic.flatMap{ x =>
???
}
val back: Hxl[F, String] = m.hxlSince Hxl is written in tagless final, you can add various behaviors to your data sources. For instance, you can add (pure) caching.
import cats.data._
import cats.implicits._
final case class MyDSKey(id: String)
case object MyDSKey extends DSKey[MyDSKey, String]
val database = Map(
"foo" -> "bar",
"baz" -> "qux"
)
type Cache = Map[String, String]
type Effect[A] = State[Cache, A]
def dataSource: DataSource[Effect, MyDSKey, String] = DataSource.from(MyDSKey) { keys =>
State[Cache, Map[MyDSKey, String]] { cache =>
val (misses, hits) = keys.toList.partitionEither(k => cache.get(k.id).tupleLeft(k).toRight(k))
val fetched = misses.flatMap(key => database.get(key.id).toList.map(key -> _)).toMap
(cache ++ fetched.map{ case (k, v) => k.id -> v }, hits.toMap ++ fetched)
}
}Hxl's interface is public and small, so extension is very possible.
Under the hood, Hxl compiles your structure into F[A] via a natural transformation:
type Target[F[_], G[_], A] = G[Either[Hxl[F, A], A]]
type Compiler[F[_], G[_]] = Hxl[F, *] ~> Target[F, G, *]Hxl repeats your natural transformation until the result becomes Right, like tailRecM.
def fa: Hxl[F, A] = ???
fa.foldMap(Hxl.parallelRunner[F]): F[A]Compilers can be composed like ordinary functions such that the core of Hxl is exposed for extension.
As an example, let's add tracing (from natchez) to Hxl:
import _root_.natchez._
import cats.data._
import cats.implicits._
import cats._
import Hxl._
def traceRequests[F[_]: Trace: Applicative, A](req: Requests[F, A]): Requests[F, A] = {
def traceSource[K, V](source: DataSource[F, K, V]): DataSource[F, K, V] =
DataSource.full[F, K, V](source.key) { ks =>
Trace[F].span(s"datasource.${source.key}") {
Trace[F].put("keys" -> ks.size.toString) *> source.batch(ks)
}
}(source.optimization)
req.visit {
new Requests.DataSourceVisitor[F] {
def visit[K, V](source: DataSource[F, K, V], k: K): (DataSource[F, K, V], K) =
(traceSource(source), k)
}
}
}
def composeTracing[F[_]: Trace: Applicative, G[_]: Trace: Applicative](
compiler: Compiler[F, G]
): Compiler[F, StateT[G, Int, *]] = {
type Effect[A] = StateT[G, Int, A]
new Compiler[F, Effect] {
def apply[A](fa: Hxl[F, A]): Hxl.Target[F, Effect, A] =
fa match {
case Hxl.LiftF(unFetch) =>
StateT.liftF {
Trace[G].span("hxl.fetch") {
compiler(Hxl.LiftF(unFetch))
}
}
case bind: Hxl.Bind[F, a, b] =>
StateT { round: Int =>
Trace[G]
.span("hxl.bind") {
Trace[G].put("round" -> round.toString) *> compiler {
Hxl.Bind(traceRequests(bind.requests), bind.f)
}
}
.map(round + 1 -> _)
}
case other => StateT.liftF(compiler(other))
}
}
}
def fa: Hxl[F, String] = ???
val result: F[String] = fa.foldMap(composeTracing[F, F](Hxl.parallelRunner)).runA(0)