Search *

apache/parquet-mr

Apache Parquet

Scala (JVM): 2.10 2.12

parquet java big-data

1282 1071 139

spotify/ratatool

A tool for data sampling, data generation, and data diffing

Scala (JVM): 2.10 2.11 2.12 2.13

parquet scala protobuf bigquery avro scalacheck

279 52 24

mjakubowski84/parquet4s

Read and write Parquet in Scala. Use Scala classes as schema. No need to start a cluster.

Scala (JVM): 2.11 2.12 2.13

parquet akka scala akka-streams fs2 streams reader aws parquet-files google-storage hadoop writer

131 37 11

51zero/eel-sdk

Big Data Toolkit for the JVM

Scala (JVM): 2.10 2.11 2.12

hive scala etl orc hadoop big-data parquet kafka kudu

117 25 9

saurfang/sparksql-protobuf

Read SparkSQL parquet file as RDD[Protobuf]

Scala (JVM): 2.10 2.11

protobuf sparksql parquet

47 18 1

indix/schemer

Schema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.

Scala (JVM): 2.11

spark parquet schema-inference tsv graphql-api avro schema-registry json

41 2 2

monix/monix-connect

A set of connectors for Monix. 🔛

Scala (JVM): 2.12 2.13

redis scala aws monix google-cloud-storage mongodb hdfs dynamodb s3 parquet connectors akka reactive-streams

40 13 9

nevillelyh/parquet-extra

A collection of Apache Parquet add-on modules

Scala (JVM): 2.11 2.12 2.13

magnolia parquet scala tensorflow avro scala-macros

30 7 3

nevillelyh/parquet-avro-extra

Scala macros for generating Parquet schema projections and filter predicates

Scala (JVM): 2.10 2.11 2.12 2.13

scala-macros scala avro parquet

26 4 2

agile-lab-dev/wasp

WASP is a framework to build complex real time big data applications. It relies on a kind of Kappa/Lambda architecture mainly leveraging Kafka and Spark. If you need to ingest huge amount of heterogeneous data and analyze them through complex pipelines, this is the framework for you.