Search *

spotify/scio

A Scala API for Apache Beam and Google Cloud Dataflow.

Scala (JVM): 2.10 2.11 2.12 2.13

beam ml bigquery batch streaming google-cloud data dataflow scio scala

2186 441 129

spotify/ratatool

A tool for data sampling, data generation, and data diffing

Scala (JVM): 2.10 2.11 2.12 2.13

scala parquet protobuf scalacheck avro bigquery

293 56 25

wix/quix

Quix Notebook Manager

Scala (JVM): 2.12 2.13

notebook-manager presto athena bigquery

226 28 23

googleclouddataproc/spark-bigquery-connector

BigQuery data source for Apache Spark: Read data from BigQuery into DataFrames, write DataFrames into BigQuery tables.

Scala (JVM): 2.11 2.12

google-cloud-dataproc bigquery-storage-api google-cloud spark google-bigquery bigquery

160 100 30

spotify/magnolify

A collection of Magnolia add-on modules

Scala (JVM): 2.11 2.12 2.13

datastore magnolia cats scala google-cloud avro bigquery scalacheck tensorflow

110 19 15

samelamin/spark-bigquery

Google BigQuery support for Spark, Structured Streaming, SQL, and DataFrames with easy Databricks integration.

Scala (JVM): 2.11

bigquery data-frame schema spark

63 29 5

nevillelyh/shapeless-datatype

Shapeless utilities for common data types

Scala (JVM): 2.10 2.11 2.12 2.13

datastore bigquery scala shapeless google-cloud scalacheck tensorflow

48 7 5

tharwaninitin/etlflow

Functional, Composable library in Scala based on ZIO for writing ETL jobs in AWS and GCP

Scala (JVM): 2.11 2.12

etl aws gcp etl-framework scala etl-pipeline spark bigquery zio

34 6 4

ocadaruma/scalikejdbc-bigquery

ScalikeJDBC extension for Google BigQuery

Scala (JVM): 2.11 2.12 2.13

scalikejdbc scala bigquery

16 9 6

data-tools/big-data-types

A library to transform Scala basic and product types into SQL types and Schemas from different database. Any implemented type gets automatically methods to convert it into the rest of the types and vice versa. For example, an Spark Schema can be transformed into a BigQuery table.

Scala (JVM): 2.12 2.13 scala3