Async Utilities for the Twitter/Finagle ecosystem
This library implements the AsyncFunctorK
typeclass from our Async Utils library for Twitter Futures. It also contains scalafix rules to rewrite the code output by Scrooge to re-introduce higher-kinded types into the generated code (see twitter/scrooge#352 for more background), and convenience wrappers for starting Finagle Thrift clients and servers, either with or without Natchez tracing integration.
See the async-utils README for more background on why AsyncFunctorK
exists and how it works.
This repository was forked from async-utils due to Twitter's {YEAR}-{MONTH}
version scheme, where the combination of the two forms a major version. Twitter does not typically maintain binary compatibility between versions, so when new versions are released, we expect to have to release a new major version of this project to support the new version.
Twitter Futures
import cats.data.ReaderT
import cats.tagless.{Derive, FunctorK}
import com.twitter.util.Closable
// generated by twitter-scrooge
trait FooScroogeService[F[_]] {
def foo(i: Int): F[Unit]
def asClosable: Closable = Closable.nop
}
object FooScroogeService {
// Let Scalafix generate these instances for you!
// Follow the instructions in the `Scalafix Rule` section below.
implicit def FooScroogeServiceReaderT[F[_]]: FooScroogeService[ReaderT[F, FooScroogeService[F], *]] =
Derive.readerT[FooScroogeService, F]
implicit val FooScroogeServiceFunctorK: FunctorK[FooScroogeService] = Derive.functorK[FooScroogeService]
}
Finagle Clients
Safely create a Finagle client in IO
from an implementation in Twitter Future:
import com.dwolla.util.async.finagle.ThriftClient
import com.dwolla.util.async.twitter._
val fooClient: Resource[IO, FooScroogeService[IO]] = ThriftClient[FooScroogeService]("destination")
Finagle Servers
Safely create a Finagle server in Twitter Future from an implementation in IO
:
import com.dwolla.util.async.finagle.ThriftServer
import com.dwolla.util.async.twitter._
val fooImpl: FooScroogeService[IO] = new FooScroogeService[IO] {
def foo(i: Int): IO[Unit] = IO(println(i))
}
val thriftServer: IO[Nothing] = ThriftServer("address", fooImpl)
Scalafix Rule
Add Scalafix to your project's build by following the instructions:
-
Add the Scalafix plugin to the project by adding this to
project/plugins.sbt
:addSbtPlugin("ch.epfl.scala" % "sbt-scalafix" % "0.9.34")
-
Enable SemanticDB by adding this to
build.sbt
:ThisBuild / semanticdbEnabled := true ThisBuild / semanticdbVersion := scalafixSemanticdb.revision ThisBuild / scalafixScalaBinaryVersion := CrossVersion.binaryScalaVersion(scalaVersion.value) ThisBuild / scalafixDependencies += "com.dwolla" %% "finagle-tagless-scalafix" % "0.0.7"
-
Run the Scalafix rule automatically after generating the Thrift sources by adding this to
build.sbt
:Compile / scalafix / unmanagedSources := (Compile / sources).value Compile / compile := Def.taskDyn { val compileOutput = (Compile / compile).value Def.task { (Compile / scalafix).toTask(" AddCatsTaglessInstances").value compileOutput } }.value libraryDependencies ++= { val catsTaglessV = "0.14.0" Seq( "org.typelevel" %% "cats-tagless-core" % catsTaglessV, "org.typelevel" %% "cats-tagless-macros" % catsTaglessV, ) }
AddCatsTaglessInstances
The AddCatsTaglessInstances
rule finds generated Thrift service traits and adds implicit instances of
ThriftService[Kleisli[F, ThriftService[Future], *]]
and FunctorK[ThriftService]
to each service's
companion object.
Twitter's Scrooge project changed the way it generates code for Thrift services, removing the
higher-kinded service trait used by this library, leaving only the MethodPerEndpoint
trait
that used to extend the higher-kinded service trait, setting the type parameter to com.twitter.util.Future
.
The AddCatsTaglessInstances
rule now addresses this as well, rewriting MethodPerEndpoint
to
{Name}Service
and reintroducing the type parameter. (A new MethodPerEndpoint
is also added,
going back to how it used to extend {Name}Service[Future]
.)
This Scalafix rule should be idempotent, so it can be rerun many times.
AdaptHigherKindedThriftCode
Because the AddCatsTaglessInstances
rewrite rule couldn't easily move the new {Name}Service
trait up
to the same level as the {Name}Service
object, the new traits must be addressed differently. In other
words, instead of finding the trait at com.example.ThriftService
, it will now be
at com.example.ThriftService.ThriftService
.
The AdaptHigherKindedThriftCode
rule exists to adapt existing code to the new location. It will
find references to traits that extend com.twitter.finagle.thrift.ThriftService
and have a type
parameter of the correct shape, and add the object name before the trait name (i.e., rewriting
ThriftService
to ThriftService.ThriftService
or com.example.ThriftService
to
com.example.ThriftService.ThriftService
).
This rule is not idempotent, but it will typically only be executed once per codebase.
The order in which the rule is executed matters. Follow these steps:
-
Add Scalafix to your project by following steps 1 and 2 under "Scalafix Rule" above.
-
Look at your project's sbt project graph. Because the rule is a semantic rule, it depends on the compiler being able to compile the code it will modify. This means the leaves of the project graph need to be updated before the nodes that depend on each leaf.
For example, run
Test/scalafix AdaptHigherKindedThriftCode
before runningCompile/scalafix AdaptHigherKindedThriftCode
. -
Only after running the
AdaptHigherKindedThriftCode
rule should you update the Scrooge and Finagle version being used in the project. Once this is updated, you can run theAddCatsTaglessInstances
rule on the updated generated code.
Artifacts
The Group ID for each artifact is "com.dwolla"
. All artifacts are published to Maven Central.
Artifact | Description | Scala 2.13 | Scala 2.12 |
---|---|---|---|
"async-utils-twitter" |
Implementation for Twitter Future |
||
"async-utils-finagle" |
Safely create Thrift clients and servers using cats-effect as the effect type | ||
"async-utils-finagle-natchez" |
Bridge between Natchez tracing and Finagle's built-in Zipkin support | ||
"finagle-tagless-scalafix" |
Automatically adds implicit instances needed by `asyncMapK` to the companion objects of Finagle services generated by Scrooge |
Credits
Thanks to Georgi Krastev and the cats-tagless project for the idea to use ReaderT
in this way.