Snappy flows

Maven Central Build Status

Snappy compression and decompression Akka Steams Flows.

Uses Snappy's framing format.

Getting started

In your build.sbt:

libraryDependencies += "me.maciejb.snappyflows" %% "snappy-flows" % "0.2.0"

Snappy flows are available for both Scala 2.11 and 2.12. We use a recent Akka 2.4 branch version (2.4.14 as of snappy-flows 0.2.0).

Previous releases:

Snappy flows Akka Scala
0.1.3 - 0.1.5 Akka >2.4.2 2.11
0.1.2 Akka 2.4.2-RC3 2.11
0.1.1 Akka Streams 2.0 2.11
<= 0.1.0 Akka Streams 2.0-M2 2.11

Usage

Sync and async versions of the flows are available:

import me.maciejb.snappyflows.SnappyFlows

val Parallelism = 4

// Let's take a source we want to decompress
val sourceWithCompressedData: Source[ByteString] = Source(...)

sourceWithCompressedData.via(SnappyFlows.decompress())
// or
sourceWithCompressedData.via(SnappyFlows.decompressAsync(Parallelism))

// Now, one we want to compress
val sourceWithRawBytes: Source[ByteString] = Source(...)

sourceWithRawBytes.via(SnappyFlows.compress())
// or
sourceWithRawBytes.via(SnappyFlows.compressAsync(Parallelism))

Performance

Initial benchmarks show that the non-async Snappy Flows achieve ~90% of performance of SnappyFramedInputStream and SnappyFramedOutputStream. async versions provide very good speedup giving 550-600MiB/s throughput on the test data on a a late 2013 15’ MBP with Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz, 16GiB. For details refer to benchmarks/README.md.

Resources