kitlangton / parallel-for   0.1.0

Apache License 2.0 GitHub

Automatically parallelize your for-comprehensions at compile time.

Scala versions: 2.13
Scala.js versions: 1.x

Parallel For

Release Artifacts Snapshot Artifacts

Automatically parallelize your for comprehensions at compile time.

// build.sbt
libraryDependencies ++= Seq(
  "io.github.kitlangton" %% "parallel-for" % "0.0.6",
  "io.github.kitlangton" %% "parallel-for-zio" % "0.0.6", // for ZIO 2
  "io.github.kitlangton" %% "parallel-for-zio1" % "0.0.6", // for ZIO 1
  "io.github.kitlangton" %% "parallel-for-zio-query" % "0.0.6", // for ZIO QUERY (for ZIO 2)
  "io.github.kitlangton" %% "parallel-for-cats-effect-3" % "0.0.6", // for Cats Effect 3
)

Example

Wrapping a for-comprehension in par will automatically parallelize it at compile time.

import parallelfor._
import parallelfor.interop.zio._ // Change this line depending on your effect system
import zio._

val program =
  par {
    for {
      users  <- loadUsers
      files  <- loadFiles
      config <- loadConfig
      result <- process(users, files, config)
      _      <- fireTheMissiles
    } yield result
  }

The par macro will rewrite the above program to—essentially—the following:

val program =
  for {
    (users, files, config, _) <- loadUsers zipPar loadFiles zipPar loadConfig zipPar fireTheMissiles
    result                    <- process(users, files, config)
  } yield result

Neato!

Similar project: VirtusLab/avocADO (only for Scala 3)

TODO

  • Work with ZIO and ZManaged
  • Maximize parallelization with topological sorting
  • Get it to work with val assignments inside of for-comprehensions
  • Set up publishing
  • Generalize to work with any zippable structure (implement against Parallelizable type-class)
  • Fix withFilter interaction
  • Cross-build for JS and Native
  • Cross-build against 2.11/2.12
  • Get it to work with Scala 3