apache/spark

Apache Spark - A unified analytics engine for large-scale data processing

delta-io/delta

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

spark-jobserver/spark-jobserver

REST job server for Apache Spark

microsoft/synapseml

Simple and Distributed Machine Learning

salesforce/transmogrifai

TransmogrifAI (pronounced trăns-mŏgˈrə-fī) is an AutoML library for building modular, reusable, strongly typed machine learning workflows on Apache Spark with minimal hand-tuning

zio/zio-quill

Compile-time Language Integrated Queries for Scala

johnsnowlabs/spark-nlp

State of the Art Natural Language Processing

datastax/spark-cassandra-connector

DataStax Spark Cassandra Connector

almond-sh/almond

A Scala kernel for Jupyter

combust/mleap

MLeap: Deploy ML Pipelines to Production

apache/incubator-kyuubi

Apache Kyuubi is a distributed multi-tenant JDBC server for large-scale data processing and analytics, built on top of Apache Spark

redislabs/spark-redis

A connector for Spark that allows reading and writing to/from Redis cluster

typelevel/frameless

Expressive types for Spark.

bigdatagenomics/adam

ADAM is a genomics analysis platform with specialized file formats built using Apache Avro, Apache Spark, and Apache Parquet. Apache 2 licensed.

tibcosoftware/snappydata

Project SnappyData - memory optimized analytics database, based on Apache Spark™ and Apache Geode™. Stream, Transact, Analyze, Predict in one cluster

mrpowers/spark-daria

Essential Spark extensions and helper methods ✨😲

h2oai/sparkling-water

Sparkling Water provides H2O functionality inside Spark cluster

frees-io/freestyle

A cohesive & pragmatic framework of FP centric Scala libraries

pingcap/tispark

TiSpark is built for running Apache Spark on top of TiDB/TiKV

lucacanali/sparkmeasure

This is the development repository for sparkMeasure, a tool for performance troubleshooting of Apache Spark workloads. It simplifies the collection and analysis of Spark task and stage metrics data.