Search *

microsoft/mobius

C# and F# language binding and extensions to Apache Spark

streaming bigdata fsharp rdd mapreduce spark csharp dstream apache-spark eventhubs dataframe mobius near-real-time spark-streaming kafka-streaming dataset

737 173 32

lucacanali/sparkmeasure

This is the development repository of SparkMeasure, a tool for performance troubleshooting of Apache Spark workloads. It simplifies the collection and analysis of Spark task metrics data.

Scala (JVM): 2.11 2.12

spark-troubleshooting performance-troubleshooting spark performance-metrics apache-spark

323 74 7

hydrospheredata/mist

Serverless proxy for Spark cluster

Scala (JVM): 2.10 2.11 2.12

apache-spark api big-data serverless

247 59 24

azure/azure-cosmosdb-spark

Apache Spark Connector for Azure Cosmos DB

Scala (JVM): 2.10 2.11

connector databricks pyspark jupyter-notebook spark databricks-notebooks apache-spark azure-databricks azure-cosmos-db lambda-architecture changefeed cosmos-db

154 108 22

azure/azure-event-hubs-spark

Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs

Scala (JVM): 2.11 2.12

streaming continuous databricks azure real-time apache-spark eventhubs stream scala microsoft event-hubs structured-streaming connector spark-streaming apache spark

128 113 29

whylabs/whylogs-java

Profile and monitor your ML data pipeline end-to-end

Scala (JVM): 2.11 2.12

aiops calculate-statistics spark statistical-properties apache-spark ai-pipelines mlops data-quality whylogs statistics java approximate-statistics dataset

80 4 2

kakao/cuesheet

A framework for writing Spark 2.x applications in a pretty way

Scala (JVM): 2.10 2.11

mango yarn spark apache-spark scala magic

78 21 2

seznam/euphoria

Euphoria is an open source Java API for creating unified big-data processing flows. It provides an engine independent programming model which can express both batch and stream transformations.

Scala (JVM): dotty

kafka java-api batch-processing hdfs apache-flink apache-spark streaming-data hadoop big-data unified-bigdata-processing

59 9 10

chermenin/spark-states

Custom state store providers for Apache Spark

Scala (JVM): 2.11 2.12

spark-streaming state apache-spark structured-streaming state-store stateful spark-structured-streaming apache spark

53 17 3

uosdmlab/spark-nkp

Natural Korean Processor for Apache Spark

Scala (JVM): 2.11

nlp spark-mllib spark korean-nlp apache-spark natural-language-processing text-mining

41 13 2

bizreach/aws-kinesis-scala

Scala client for Amazon Kinesis. Also provides write to Kinesis capability for Apache Spark or Spark Streaming.

Scala (JVM): 2.11 2.12 2.13

aws-kinesis kinesis spark aws apache-spark scala

26 11 4

heartsavior/spark-state-tools

Spark Structured Streaming State Tools

Scala (JVM): 2.11 2.12

apache-spark structured-streaming

22 4 2

flipkart-incubator/spark-transformers

Spark-Transformers: Library for exporting Apache Spark MLLIB models in to use them in any Java application with no other dependencies.

Scala (JVM): 2.10 2.11

transformers export spark machine-learning machine-learning-algorithms scala java mllib machine-learning-library data-pipelines apache-spark

25 20 8

isarn/isarn-sketches-spark

Routines and data structures for using isarn-sketches idiomatically in Apache Spark

Scala (JVM): 2.10 2.11 2.12

datasets data-sketches feature-importance apache-spark aggregator data-sketching dataset sketching-algorithm udaf dataframes variable-importance python pyspark spark-ml t-digest spark dataframe scala

20 6 2

itspawanbhardwaj/spark-fuzzy-matching

Fuzzy matching function in spark (https://spark-packages.org/package/itspawanbhardwaj/spark-fuzzy-matching)

Scala (JVM): 2.10 2.11

fuzzy-matching apache-spark scala

16 5 1

tupol/spark-utils

Basic framework utilities to quickly start writing production ready Apache Spark applications

Scala (JVM): 2.11 2.12

spark-streaming convenience spark framework apache-spark data-source scala data-sink spark-applications

15 4 1

liquidsvm/liquidsvm

Support vector machines (SVMs) and related kernel-based learning algorithms are a well-known class of machine learning algorithms, for non-parametric classification and regression. liquidSVM is an implementation of SVMs whose key features are: fully integrated hyper-parameter selection, extreme speed on both small and large data sets, full flexibility for experts, and inclusion of a variety of different learning scenarios: multi-class classification, ROC, and Neyman-Pearson learning, and least-squares, quantile, and expectile regression.

Scala (JVM): 2.11

regression python classification octave machine-learning apache-spark matlab expectile-regression ml svm quantile-regression r rstats c-plus-plus r-package

14 2 3

astrolabsoftware/spark-fits

FITS data source for Spark SQL and DataFrames

Scala (JVM): 2.10 2.11 2.12

fitsio pyspark hdfs apache-spark scala spark-sql fits

11 4 3

astrolabsoftware/spark3d

Spark extension for processing large-scale 3D data sets: Astrophysics, High Energy Physics, Meteorology, …

Scala (JVM): 2.11

pyspark scientific-computing apache-spark spatial-data scala 3d

9 3 3

tupol/spark-tools

Executable Apache Spark Tools: Format Converter & SQL Processor

Scala (JVM): 2.11 2.12

spark tools apache-spark scala format-converter sql converts

9 0 1

Sorting

Filters

microsoft/mobius

lucacanali/sparkmeasure

hydrospheredata/mist

azure/azure-cosmosdb-spark

azure/azure-event-hubs-spark

whylabs/whylogs-java

kakao/cuesheet

seznam/euphoria

chermenin/spark-states

uosdmlab/spark-nkp

bizreach/aws-kinesis-scala

heartsavior/spark-state-tools

flipkart-incubator/spark-transformers

isarn/isarn-sketches-spark

itspawanbhardwaj/spark-fuzzy-matching

tupol/spark-utils

liquidsvm/liquidsvm

astrolabsoftware/spark-fits

astrolabsoftware/spark3d

tupol/spark-tools