dacr / zio-lmdb   1.5.0


Lightning Memory Database (LMDB) for scala ZIO

Scala versions: 3.x 2.13

Lightning Memory Database (LMDB) for ZIO


Why ZIO-lmdb ? Because I wanted a very simple embedded (in the same process) ACID database for small applications while keeping deployment, maintenance, upgrades as simple as possible.

ZIO-lmdb is based on the powerful lmdb-java library and bring a higher level API in order to enhance the developer experience.

So ZIO-lmdb is an embedded key/value database, with an easy to use opinionated API, choices have been made to make the developer experience as simple as possible :

  • JSON based storage using zio-json,
  • safe update by using a lambda which will be called with the previous value if it exists and returns the new value,
  • identifiers are managed by the developer, just use UUID or ULID.

API is designed to not lie, all functions signatures describe precisely what you must expect from them, thanks to ZIO and Scala3.


For a better understanding, this library use a slightly different vocabulary from LMDB original one :

  • Database : (LMDB talk about Environment)
    • The place where the database file is stored on your file system
    • A set of configuration for this database (expected maximum size, expected collection number)
  • Collection : (LMDB talk about Database)
    • A sorted map where to store your data
    • One database contains multiple collection
  • Transaction : (the same for LMDB)
    • for global coherency within the same database
    • only one simultaneous write access is possible within the same database


Configuration is based on the standard ZIO config mechanism, the default configuration provider uses environnment variables or java properties to resolve this library configuration parameters.

Configuration key Environment variable Description Default value
lmdb.name LMDB_NAME Database name, which will be also used as the directory name default
lmdb.home LMDB_HOME Where to store the database directory $HOME/.lmdb
lmdb.sync LMDB_SYNC Synchronize the file system with all database write operations false
lmdb.maxReaders LMDB_MAXREADERS The maximum number of readers 100
lmdb.maxCollections LMDB_MAXCOLLECTIONS The maximum number of collections which can be created 10_000
lmdb.mapSize LMDB_MAPSIZE The maximum size of the whole database including metadata 100_000_000_000L

Usages example

//> using scala  "3.3.0"
//> using dep "fr.janalyse::zio-lmdb:1.4.3"
//> using javaOpt "--add-opens", "java.base/java.nio=ALL-UNNAMED", "--add-opens", "java.base/sun.nio.ch=ALL-UNNAMED"

import zio.*, zio.lmdb.*, zio.json.*
import java.io.File, java.util.UUID, java.time.OffsetDateTime

case class Record(uuid: UUID,name: String,age: Int,addedOn: OffsetDateTime) derives JsonCodec

object SimpleExample extends ZIOAppDefault {
  override def run = example.provide(LMDB.liveWithDatabaseName("lmdb-data-simple-example"), zio.Scope.default)

  val collectionName = "examples"
  val example        = for {
    _         <- LMDB.collectionCreate[Record](collectionName).ignore
    examples  <- LMDB.collectionGet[Record](collectionName)
    recordId  <- Random.nextUUID
    dateTime  <- Clock.currentDateTime
    record     = Record(recordId, "John Doe", 42, dateTime)
    _         <- examples.upsertOverwrite(recordId.toString, record)
    gotten    <- examples.fetch(recordId.toString).some
    collected <- examples.collect()
    _         <- Console.printLine(s"collection $collectionName contains ${collected.size} records")
    _         <- ZIO.foreach(collected)(record => Console.printLine(record))
    lmdb      <- ZIO.service[LMDB]
    _         <- Console.printLine("""LMDB standard tools can be used to manage the database content : sudo apt-get install lmdb-utils""")
    _         <- Console.printLine(s"""To get some statistics     : mdb_stat -s $collectionName ${lmdb.databasePath}/""")
    _         <- Console.printLine(s"""To dump collection content : mdb_dump -p -s $collectionName ${lmdb.databasePath}/""")
  } yield ()


To run the previous logic, you'll have to provide the LMDB layer, two layers are available :

  • LMDB.live : Fully configurable using standard zio-config
  • LMDB.liveWithDatabaseName("chosen-database-name") : to override/force the database name (quite useful when writing scala scripts)

ZIO-LMDB based Applications

Code snippets using ZIO-LMDB, runnable with scala-cli

Operating lmdb databases

LMDB standard tools can be used to manage the databases content : sudo apt-get install lmdb-utils

  • to get some database statistics : mdb_stat -a database_directory_path/
  • to dump the content of a database : mdb_dump -a -p database_directory_path/
  • to dump the content of a database collection : mdb_dump -s collectionName -p database_directory_path/
  • to restore some collection or the entire database use the command named mdb_load which uses the same format as for mdb_dump

As zio-lmdb is using json format, dumps are just text, which can be edited and then loaded back. So simple data migration is straightforward.


When LVMDB is used as persistence store with recent JVM it requires some JVM options :

--add-opens java.base/java.nio=ALL-UNNAMED
--add-opens java.base/sun.nio.ch=ALL-UNNAMED

Contributors :)