Search *

apache/parquet-mr

Apache Parquet

Scala (JVM): 2.10 2.12

parquet java big-data

1282 1071 139

spotify/ratatool

A tool for data sampling, data generation, and data diffing

Scala (JVM): 2.10 2.11 2.12 2.13

avro scalacheck scala protobuf bigquery parquet

279 52 24

mjakubowski84/parquet4s

Read and write Parquet in Scala. Use Scala classes as schema. No need to start a cluster.

Scala (JVM): 2.11 2.12 2.13

aws google-storage streams scala parquet reader fs2 writer akka-streams parquet-files hadoop akka

145 39 12

51zero/eel-sdk

Big Data Toolkit for the JVM

Scala (JVM): 2.10 2.11 2.12

etl scala orc big-data kafka hadoop hive kudu parquet

117 25 9

monix/monix-connect

A set of connectors for Monix. 🔛

Scala (JVM): 2.12 2.13

workflow aws connectors reactive-streams s3 parquet monix google-cloud-storage dynamodb hdfs mongodb sqs elasticsearch redis scala

51 18 11

saurfang/sparksql-protobuf

Read SparkSQL parquet file as RDD[Protobuf]

Scala (JVM): 2.10 2.11

protobuf sparksql parquet

47 18 1

indix/schemer

Schema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.

Scala (JVM): 2.11

avro tsv spark graphql-api schema-registry json schema-inference parquet

41 2 2

nevillelyh/parquet-extra

A collection of Apache Parquet add-on modules

Scala (JVM): 2.11 2.12 2.13

avro scala-macros scala magnolia tensorflow parquet

30 7 3

nevillelyh/parquet-avro-extra

Scala macros for generating Parquet schema projections and filter predicates

Scala (JVM): 2.10 2.11 2.12 2.13

scala-macros scala avro parquet

26 4 2

agile-lab-dev/wasp

WASP is a framework to build complex real time big data applications. It relies on a kind of Kappa/Lambda architecture mainly leveraging Kafka and Spark. If you need to ingest huge amount of heterogeneous data and analyze them through complex pipelines, this is the framework for you.