Search *

Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs

stream kafka event-hubs apache real-time ingestion bigdata continuous eventhubs scala connector spark-streaming azure streaming spark databricks microsoft structured-streaming apache-spark

131 117 31

Custom state store providers for Apache Spark

apache stateful spark-structured-streaming state-store spark-streaming state spark structured-streaming apache-spark

53 17 3

Spark Structured Streaming State Tools

22 4 2

Rocksdb state storage implementation for Structured Streaming.

real-time-processing qubole performance structured-streaming state-management rocksdb streaming spark scalability

10 3 3

Kafka offset committer for structured streaming query

9 3 2

A library for reading data from Amzon S3 with optimised listing using Amazon SQS using Spark SQL Streaming ( or Structured streaming).

sqs structured-streaming scala s3 spark-streaming streaming spark

1 0 2

Qubole Streaminglens tool for tuning Spark Structured Streaming Pipelines

sparklens structured-streaming scala micro-batches cluster-management spark-streaming streaming-pipeline streaming spark sla

0 0 2