florentf9/sparkml-som

:sparkles: Spark ML implementation of SOM algorithm (Kohonen self-organizing map)

piotr-kalanski/data-quality-monitoring

Data Quality Monitoring Tool

hammerlab/spark-util

low-level helpers for Apache Spark libraries and tests

kotlin/kotlin-spark-api

This projects gives Kotlin bindings and several extensions for Apache Spark. We are looking to have this as a part of Apache Spark 3.x

qubole/s3-sqs-connector

A library for reading data from Amzon S3 with optimised listing using Amazon SQS using Spark SQL Streaming ( or Structured streaming).

arcizon/spark-filetransfer

API for reading and writing data via various file transfer protocols from Apache Spark.

absaoss/spark-hofs

Scala API for Apache Spark SQL high-order functions

getsentry/sentry-spark

Apache Spark Sentry Integration

absaoss/pramen

Resilient data pipeline framework running on Apache Spark

qubole/spark-state-store

Rocksdb state storage implementation for Structured Streaming.

fqaiser94/mse

Make Structs Easy (MSE)

zuinnote/spark-hadoopcryptoledger-ds

A Spark datasource for the HadoopCryptoLedger library

hunters-ai/spark-adaptive-file-connector

Adaptive File Source Connector for Spark, optimised for reading from object stores

exasol/spark-connector

A connector for Apache Spark to access Exasol

jtnystrom/discount

Very large scale k-mer counting and analysis on Apache Spark.

woltapp/spark-osm-datasource

Native Spark OSM PBF data source

tupol/spark-tools

Executable Apache Spark Tools: Format Converter & SQL Processor

phymbert/spark-search

Spark Search - high performance advanced search features based on Apache Lucene

data-tools/big-data-types

A library to transform Scala product types and Schemas from different databases into other Schemas. Any implemented type gets automatically methods to convert it into the rest of the types and vice versa. For example, an Spark Schema can be transformed into a BigQuery table.