simerplaha / swaydb   0.2.1

Apache License 2.0 Website GitHub

Persistent and in-memory key-value storage engine for JVM that scales on a single machine.

Scala versions: 2.13 2.12 2.11

SwayDB Gitter Chat Build status Maven central

Persistent and in-memory key-value storage engine for the JVM aimed at high performance & resource efficiency.

Small footprint: around 7.1 MB jar size. No external core dependency (#307).

Scalable on a single machine: Distribute data on multiple local SSDs. Allocate single or multiple Threads for reads, caching & compaction.

Branches: This is the master branch. See develop for latest (unstable).

Status: Under testing & performance optimisations. See project status.

Documentation

Sponsors

Thank you JetBrains & JProfiler for full-featured open-source licences to their awesome development tools!

Jetbrains support JProfiler support [Become a sponsor]

Overview

  • Simple data types - Map, Set, Queue, SetMap & MultiMap with native Java and Scala collections support.
  • Conditional updates using any pure JVM function - No query language.
  • Atomic updates and inserts with Transaction API.
  • Non-blocking core with configurable APIs for blocking, non-blocking and/or reactive use-cases.
  • Single or multiple disks persistent, in-memory or eventually persistent.
  • Streams - Async & sync forward and reverse data streaming/iteration.
  • TTL - non-blocking, non-polling asynchronous auto expiring key-values.
  • Range operations to update, remove & expire a range of key-values.
  • Key only iterations (Lazily fetched values).
  • Configurable compression with LZ4 & Snappy
  • Configurable core internals to support custom workloads.
  • Duplicate values elimination with compressDuplicateValues.

Use cases

Highly configurable to suit different workloads. Some known use-cases are:

  • General key-value storage
  • Message queues
  • Time-series or Events data
  • Caching
  • Application logs
  • Archiving data or cold storage with high file level compression

Quick start

Contributing

Contributions are encouraged and welcomed. We are here to help and answer any questions.

Code of conduct - Be nice, welcoming, friendly & supportive of each other. Follow the Apache foundation's COC.

  • Contributing to data management API

    • Build new data structures extending existing data structures. See MultiMap for example which is an extension of Map.
      • Graph
      • List
      • Geospatial
      • Logs
      • Observables
      • Full-text search
      • etc
    • Add support for external serialising, streaming & compression libraries.
    • Test and improve existing data structures.
  • Contributing to core API

  • Contributing to core internals

    • See code marked TODO.
    • Pick any small section to improve. You will find that everything is a function and can be unit-tested independently, so you can easily pick anything to work on.

Project status

Under testing & performance optimisations.

Your feedback and support is very important to get to production. Please get involved via chat, discussion, issues or by becoming a sponsor.

Future releases might not be backward compatible until we are production ready.

See tasks labelled Production release that are required before becoming production ready.

Related GitHub projects