mrpowers/spark-stringmetric

Spark functions to run popular phonetic and string matching algorithms

hablapps/sparkoptics

Optics for Spark DataFrames

eto-ai/rikai

Parquet-based ML data format optimized for working with unstructured data

benfradet/struct-type-encoder

Deriving Spark DataFrame schemas from case classes

galliaproject/gallia-core

A schema-aware Scala library for data transformation

tharwaninitin/etlflow

EtlFlow is an ecosystem of functional libraries in Scala based on ZIO for writing various different tasks, jobs, scheduling those jobs and monitoring those jobs using UI.

flipkart-incubator/spark-transformers

Spark-Transformers: Library for exporting Apache Spark MLLIB models to use them in any Java application with no other dependencies.

bizreach/aws-kinesis-scala

Scala client for Amazon Kinesis. Also provides write to Kinesis capability for Apache Spark or Spark Streaming.

arangodb/arangodb-spark-connector

zuinnote/spark-hadoopoffice-ds

A Spark datasource for the HadoopOffice library

lewuathe/dllib

dllib is a distributed deep learning library running on Apache Spark

master/spark-stemming

Spark MLlib wrapper for the Snowball framework

g-research/spark-dgraph-connector

A connector for Apache Spark and PySpark to Dgraph databases.

sansa-stack/archived-sansa-query

SANSA Query Layer

yaooqinn/itachi

A library that brings useful functions from various modern database management systems to Apache Spark

xskipper-io/xskipper

An Extensible Data Skipping Framework

intenthq/pucket

Bucketing and partitioning system for Parquet

sansa-stack/archived-sansa-inference

A general Inference API based on two of the most popular Big Data processing engines: Apache Spark and Apache Flink

diana-hep/spark-root

Apache Spark Data Source for ROOT File Format

absaoss/hyperdrive

Extensible streaming ingestion pipeline on top of Apache Spark