Search *

spotify/scio

A Scala API for Apache Beam and Google Cloud Dataflow.

Scala (JVM): 2.10 2.11 2.12 2.13

google-cloud scala ml bigquery data batch dataflow scio beam streaming

2147 438 125

spotify/ratatool

A tool for data sampling, data generation, and data diffing

Scala (JVM): 2.10 2.11 2.12 2.13

avro scalacheck scala protobuf bigquery parquet

279 52 24

wix/quix

Quix Notebook Manager

Scala (JVM): 2.12 2.13

notebook-manager presto athena bigquery

211 25 20

googleclouddataproc/spark-bigquery-connector

BigQuery data source for Apache Spark: Read data from BigQuery into DataFrames, write DataFrames into BigQuery tables.

Scala (JVM): 2.11 2.12

google-cloud bigquery spark google-cloud-dataproc bigquery-storage-api google-bigquery

143 96 30

spotify/magnolify

A collection of Magnolia add-on modules

Scala (JVM): 2.11 2.12 2.13

scalacheck scala datastore cats magnolia avro google-cloud tensorflow bigquery

77 17 15

samelamin/spark-bigquery

Google BigQuery support for Spark, Structured Streaming, SQL, and DataFrames with easy Databricks integration.

Scala (JVM): 2.11

bigquery data-frame schema spark

63 29 5

nevillelyh/shapeless-datatype

Shapeless utilities for common data types

Scala (JVM): 2.10 2.11 2.12 2.13

google-cloud scalacheck scala datastore shapeless tensorflow bigquery

48 7 5

tharwaninitin/etlflow

Functional, Composable library in Scala based on ZIO for writing ETL jobs in AWS and GCP

Scala (JVM): 2.11 2.12

scala etl-pipeline bigquery spark gcp zio etl-framework etl aws

26 5 4

ocadaruma/scalikejdbc-bigquery

ScalikeJDBC extension for Google BigQuery

Scala (JVM): 2.11 2.12 2.13

scalikejdbc scala bigquery

16 9 6

data-tools/big-data-types

A library to transform Scala basic and product types into SQL types and Schemas from different database. Any implemented type gets automatically methods to convert it into the rest of the types and vice versa. For example, an Spark Schema can be transformed into a BigQuery table.

Scala (JVM): 2.12 2.13 scala3