Search *

microsoft/mobius

C# and F# language binding and extensions to Apache Spark

kafka-streaming dstream dataframe apache-spark mapreduce near-real-time spark-streaming bigdata dataset spark mobius eventhubs streaming csharp rdd fsharp

737 173 32

lucacanali/sparkmeasure

This is the development repository of SparkMeasure, a tool for performance troubleshooting of Apache Spark workloads. It simplifies the collection and analysis of Spark task metrics data.

Scala (JVM): 2.11 2.12

apache-spark performance-metrics performance-troubleshooting spark spark-troubleshooting

323 74 7

hydrospheredata/mist

Serverless proxy for Spark cluster

Scala (JVM): 2.10 2.11 2.12

apache-spark api big-data serverless

247 59 24

azure/azure-cosmosdb-spark

Apache Spark Connector for Azure Cosmos DB

Scala (JVM): 2.10 2.11

pyspark changefeed apache-spark azure-cosmos-db databricks-notebooks databricks connector spark jupyter-notebook lambda-architecture azure-databricks cosmos-db

160 109 23

azure/azure-event-hubs-spark

Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs

Scala (JVM): 2.11 2.12

real-time event-hubs structured-streaming spark-streaming apache stream scala spark microsoft eventhubs streaming ingestion apache-spark databricks azure connector bigdata continuous kafka

131 117 31

whylabs/whylogs-java

Profile and monitor your ML data pipeline end-to-end

Scala (JVM): 2.11 2.12

mlops calculate-statistics aiops apache-spark java whylogs statistics ai-pipelines dataset spark approximate-statistics data-quality statistical-properties

80 4 2

kakao/cuesheet

A framework for writing Spark 2.x applications in a pretty way

Scala (JVM): 2.10 2.11

magic yarn apache-spark scala spark mango

78 21 2

chermenin/spark-states

Custom state store providers for Apache Spark

Scala (JVM): 2.11 2.12

state stateful apache-spark structured-streaming spark-streaming spark-structured-streaming apache spark state-store

53 17 3

uosdmlab/spark-nkp

Natural Korean Processor for Apache Spark

Scala (JVM): 2.11

apache-spark text-mining nlp spark spark-mllib natural-language-processing korean-nlp

41 13 2

bizreach/aws-kinesis-scala

Scala client for Amazon Kinesis. Also provides write to Kinesis capability for Apache Spark or Spark Streaming.

Scala (JVM): 2.11 2.12 2.13

kinesis scala spark aws aws-kinesis apache-spark

26 11 4

heartsavior/spark-state-tools

Spark Structured Streaming State Tools

Scala (JVM): 2.11 2.12

apache-spark structured-streaming

22 4 2

flipkart-incubator/spark-transformers

Spark-Transformers: Library for exporting Apache Spark MLLIB models in to use them in any Java application with no other dependencies.

Scala (JVM): 2.10 2.11

data-pipelines apache-spark java export scala machine-learning machine-learning-library spark machine-learning-algorithms mllib transformers

25 20 8

isarn/isarn-sketches-spark

Routines and data structures for using isarn-sketches idiomatically in Apache Spark

Scala (JVM): 2.10 2.11 2.12

spark-ml dataframe apache-spark t-digest sketching-algorithm datasets variable-importance pyspark python udaf dataframes dataset data-sketching scala aggregator spark feature-importance data-sketches

20 6 2

itspawanbhardwaj/spark-fuzzy-matching

Fuzzy matching function in spark (https://spark-packages.org/package/itspawanbhardwaj/spark-fuzzy-matching)

Scala (JVM): 2.10 2.11

fuzzy-matching apache-spark scala

16 5 1

tupol/spark-utils

Basic framework utilities to quickly start writing production ready Apache Spark applications

Scala (JVM): 2.11 2.12

framework apache-spark spark-streaming scala spark data-source spark-applications convenience data-sink

15 4 1

liquidsvm/liquidsvm

Support vector machines (SVMs) and related kernel-based learning algorithms are a well-known class of machine learning algorithms, for non-parametric classification and regression. liquidSVM is an implementation of SVMs whose key features are: fully integrated hyper-parameter selection, extreme speed on both small and large data sets, full flexibility for experts, and inclusion of a variety of different learning scenarios: multi-class classification, ROC, and Neyman-Pearson learning, and least-squares, quantile, and expectile regression.

Scala (JVM): 2.11