Search *

spotify/scio

A Scala API for Apache Beam and Google Cloud Dataflow.

Scala (JVM): 2.10 2.11 2.12 2.13

dataflow bigquery beam google-cloud ml scio scala streaming batch data

2116 421 120

spotify/ratatool

A tool for data sampling, data generation, and data diffing

Scala (JVM): 2.10 2.11 2.12 2.13

parquet scala protobuf bigquery avro scalacheck

279 52 24

wix/quix

Quix Notebook Manager

Scala (JVM): 2.12 2.13

notebook-manager presto athena bigquery

186 24 20

googleclouddataproc/spark-bigquery-connector

BigQuery data source for Apache Spark: Read data from BigQuery into DataFrames, write DataFrames into BigQuery tables.

Scala (JVM): 2.11 2.12

spark bigquery-storage-api google-cloud-dataproc google-bigquery bigquery google-cloud

126 84 27

spotify/magnolify

A collection of Magnolia add-on modules

Scala (JVM): 2.11 2.12 2.13

magnolia scala tensorflow avro google-cloud scalacheck datastore bigquery cats

77 17 15

samelamin/spark-bigquery

Google BigQuery support for Spark, Structured Streaming, SQL, and DataFrames with easy Databricks integration.

Scala (JVM): 2.11

bigquery data-frame schema spark

63 29 5

nevillelyh/shapeless-datatype

Shapeless utilities for common data types

Scala (JVM): 2.10 2.11 2.12 2.13

shapeless scala tensorflow bigquery google-cloud scalacheck datastore

48 7 5

tharwaninitin/etlflow

Functional, Composable library in Scala based on ZIO for writing ETL jobs in AWS and GCP

Scala (JVM): 2.11 2.12

spark zio scala aws etl-framework gcp etl-pipeline etl bigquery

26 5 4

ocadaruma/scalikejdbc-bigquery

ScalikeJDBC extension for Google BigQuery

Scala (JVM): 2.11 2.12 2.13

scalikejdbc scala bigquery

16 9 6

data-tools/big-data-types

A library to transform basic Scala types and case classes into SQL types and Schemas from different database. Any implemented type gets automatically methods to convert it into the rest of the types and vice versa. For example, an Spark Schema can be transformed into a BigQuery table.

Scala (JVM): 2.12 2.13 scala3