Search *

microsoft/mobius

C# and F# language binding and extensions to Apache Spark

mapreduce mobius bigdata kafka-streaming dataset dstream eventhubs apache-spark dataframe spark csharp fsharp near-real-time spark-streaming streaming rdd

737 173 32

lucacanali/sparkmeasure

This is the development repository of SparkMeasure, a tool for performance troubleshooting of Apache Spark workloads. It simplifies the collection and analysis of Spark task metrics data.

Scala (JVM): 2.11 2.12

performance-metrics apache-spark performance-troubleshooting spark-troubleshooting spark

323 74 7

hydrospheredata/mist

Serverless proxy for Spark cluster

Scala (JVM): 2.10 2.11 2.12

apache-spark api big-data serverless

247 59 24

azure/azure-cosmosdb-spark

Apache Spark Connector for Azure Cosmos DB

Scala (JVM): 2.10 2.11

changefeed cosmos-db lambda-architecture azure-databricks pyspark apache-spark databricks-notebooks spark connector azure-cosmos-db databricks jupyter-notebook

171 113 22

whylabs/whylogs-java

Profile and monitor your ML data pipeline end-to-end

Scala (JVM): 2.11 2.12

data-quality mlops java statistical-properties dataset apache-spark aiops calculate-statistics approximate-statistics spark ai-pipelines statistics whylogs

166 6 4

azure/azure-event-hubs-spark

Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs

Scala (JVM): 2.11 2.12

stream kafka event-hubs apache real-time ingestion bigdata continuous eventhubs scala connector spark-streaming azure streaming spark databricks microsoft structured-streaming apache-spark

131 117 31

kakao/cuesheet

A framework for writing Spark 2.x applications in a pretty way

Scala (JVM): 2.10 2.11

magic apache-spark scala spark mango yarn

78 21 2

swoop-inc/spark-records

Bulletproof Apache Spark jobs with fast root cause analysis of failures.

Scala (JVM): 2.12

big-data sparksql spark-records scala spark swoop apache-spark

58 14 4

chermenin/spark-states

Custom state store providers for Apache Spark

Scala (JVM): 2.11 2.12

apache stateful spark-structured-streaming state-store spark-streaming state spark structured-streaming apache-spark

53 17 3

uosdmlab/spark-nkp

Natural Korean Processor for Apache Spark

Scala (JVM): 2.11

spark-mllib korean-nlp text-mining apache-spark nlp natural-language-processing spark

41 13 2

bizreach/aws-kinesis-scala

Scala client for Amazon Kinesis. Also provides write to Kinesis capability for Apache Spark or Spark Streaming.

Scala (JVM): 2.11 2.12 2.13

aws-kinesis apache-spark aws spark scala kinesis

26 11 4

flipkart-incubator/spark-transformers

Spark-Transformers: Library for exporting Apache Spark MLLIB models in to use them in any Java application with no other dependencies.

Scala (JVM): 2.10 2.11

mllib machine-learning-library java export scala transformers data-pipelines machine-learning apache-spark machine-learning-algorithms spark

25 20 8

isarn/isarn-sketches-spark

Routines and data structures for using isarn-sketches idiomatically in Apache Spark

Scala (JVM): 2.10 2.11 2.12

datasets spark-ml t-digest feature-importance dataset aggregator pyspark apache-spark scala python data-sketches data-sketching dataframe variable-importance sketching-algorithm dataframes udaf spark

23 6 2

heartsavior/spark-state-tools

Spark Structured Streaming State Tools

Scala (JVM): 2.11 2.12

apache-spark structured-streaming

22 4 2

absaoss/hyperdrive

Extensible streaming ingestion pipeline on top of Apache Spark

Scala (JVM): 2.11 2.12

kafka ingestion streaming-etl pipeline spark-structured-streaming apache-spark streaming framework spark

22 9 9

itspawanbhardwaj/spark-fuzzy-matching

Fuzzy matching function in spark (https://spark-packages.org/package/itspawanbhardwaj/spark-fuzzy-matching)

Scala (JVM): 2.10 2.11

fuzzy-matching apache-spark scala

16 5 1

tupol/spark-utils

Basic framework utilities to quickly start writing production ready Apache Spark applications

Scala (JVM): 2.11 2.12

spark-applications convenience data-sink data-source apache-spark scala spark-streaming framework spark

15 4 1

liquidsvm/liquidsvm

Support vector machines (SVMs) and related kernel-based learning algorithms are a well-known class of machine learning algorithms, for non-parametric classification and regression. liquidSVM is an implementation of SVMs whose key features are: fully integrated hyper-parameter selection, extreme speed on both small and large data sets, full flexibility for experts, and inclusion of a variety of different learning scenarios: multi-class classification, ROC, and Neyman-Pearson learning, and least-squares, quantile, and expectile regression.

Scala (JVM): 2.11

c-plus-plus rstats regression matlab quantile-regression r-package expectile-regression octave python svm ml classification r machine-learning apache-spark

14 2 3

astrolabsoftware/spark-fits

FITS data source for Spark SQL and DataFrames

Scala (JVM): 2.10 2.11 2.12

fitsio pyspark apache-spark scala hdfs fits spark-sql

11 4 3

exasol/spark-exasol-connector

A connector for Apache Spark to access Exasol

Scala (JVM): 2.11 2.12

exasol exasol-integration apache-spark connector spark

10 6 5

Sorting

Filters

microsoft/mobius

lucacanali/sparkmeasure

hydrospheredata/mist

azure/azure-cosmosdb-spark

whylabs/whylogs-java

azure/azure-event-hubs-spark

kakao/cuesheet

swoop-inc/spark-records

chermenin/spark-states

uosdmlab/spark-nkp

bizreach/aws-kinesis-scala

flipkart-incubator/spark-transformers

isarn/isarn-sketches-spark

heartsavior/spark-state-tools

absaoss/hyperdrive

itspawanbhardwaj/spark-fuzzy-matching

tupol/spark-utils

liquidsvm/liquidsvm

astrolabsoftware/spark-fits

exasol/spark-exasol-connector