The BigQuery table which is present in a dataset in the format [[project:]dataset.]table
options
Description
-------------
-------------
parentProject
The Google Cloud resource hierarchy resembles the file system which manages entities hierarchically . The Google Cloud Project ID of the table.
project
The Google Cloud Project ID of the table. A project organizes all your Google Cloud resources .For example, all of your Cloud Storage buckets and objects, along with user permissions for accessing them, reside in a project.
dataset
A dataset is contained within a specific project. Datasets are top-level containers that are used to organize and control access to your tables and views
The BigQuery table which is present in a dataset in the format [[project:]dataset.]table
options
Description
-------------
-------------
parentProject
The Google Cloud resource hierarchy resembles the file system which manages entities hierarchically . The Google Cloud Project ID of the table.
project
The Google Cloud Project ID of the table. A project organizes all your Google Cloud resources .For example, all of your Cloud Storage buckets and objects, along with user permissions for accessing them, reside in a project.
dataset
A dataset is contained within a specific project. Datasets are top-level containers that are used to organize and control access to your tables and views
temporaryGcsBucket
The GCS bucket that temporarily holds the data before it is loaded to BigQuery. Required unless set in the Spark configuration (spark.conf.set(...)).
Example
importcom.github.music.of.the.ainur.almaren.Almarenimportcom.github.music.of.the.ainur.almaren.bigquery.BigQuery.BigQueryImplicitimportcom.github.music.of.the.ainur.almaren.builder.Core.Implicitimportorg.apache.spark.sql.SaveModevalalmaren=Almaren("App Name")
spark.conf.set("gcpAccessToken","token")
almaren.builder
.sourceSql("""SELECT sha2(concat_ws("",array(*)),256) as id,*,current_timestamp from deputies""")
.coalesce(30)
.targetBigQuery("dataset.table",Map("parentProject"->"project_name","project"->"project_name","temporaryGcsBucket"->"bucket"),SaveMode.Overwrite)
.batch