saint1991 / sbt-gcs

sbt plugin to manipulate objects on Google Cloud Storage.

GitHub

sbt-gcs

Build Status Download License

sbt-gcs is a simple sbt plugin to manipulate objects on Google Cloud Storage.

Install

As a prerequisite, sbt 1.0.0 or newer is required.
In order to add the sbt-gcs plugin to your build, add the following to project/plugins.sbt.

addSbtPlugin("com.github.saint1991" % "sbt-gcs" % "0.2.0")

This plugin automatically add some tasks to manipulate Google Cloud Storage.

Usage

Uploading

To upload files to Google Cloud Storage, you need to specify source/destination mapping at the mappings key.

// upload a text file and build artifact created by sbt-assembly
mappings in gcsUpload := Seq(
  Path.userHome / "foo.txt" -> "gs://bucket_name/path/to/upload/foo.txt",
  assembly.value -> "gs://bucket_name/path/to/artifact.jar" 
)

Then execute it as follows:

$ sbt gcsUpload

Downloading

Similarly, to download objects from Google Cloud Storage, you also need to specify mappings. Please note that you need to write it in the destination -> source order. For example, by the following settings, image.png will be downloaded from gs://image_bucket/path/to/image.png to the user home.

// download an image file from  
mappings in gcsUpload := Seq(
  (Path.userHome / "image.png" , "gs://image_bucket/path/to/image.png") 
)

Executing it by:

$ sbt gcsDownload

Deleting

To delete objects from Google Cloud Storage, you need to specify a list of object URLs to delete via the gcsUrls key.

gcsUrls in gcsDelete := Seq(
  "gs://bucket_name/path/to/object/to/delete.json"
)

Executing it by:

$ sbt gcsDelete

Credentials

By default, sbt-gcs uses default credentials stated here. You can also explicitly set it via gcsCredential key.

e.g.

import java.io.FileInputStream
import com.google.auth.oauth2.UserCredentials

gcsCredential := Some(
    UserCredentials.fromStream(new FileInputStream(new File("credential.json")))
)

Tasks

Name Description
gcsDelete Deletes objects from a bucket on Google Cloud Storage
gcsUpload Uploads files to a bucket on Google Cloud Storage
gcsDownload Downloads objects from Google Cloud Storage

Settings

Name Default Description
gcsCredential None Credential to authenticate Google Cloud Storage
gcsUrls Seq() URLs to delete
gcsOperationParallelism 8 The parallelism of operations
gcsOperationTimeout 10 minutes The timeout for each operation
gcsChunkSize 8192 bytes The chunk size for data transfer
gcsProgress false The flag whether showing progress bar on uploading/downloading

License

This code is open source software licensed under the Apache 2.0 License