sbt-conductr

Note that this project is EOL as of 2018-01-31.

sbt-conductr is an sbt plugin that provides commands in sbt to:

• Produce a ConductR bundle (a bundle contains everything required to run your service, including your service itself)
• Start and stop a local ConductR cluster
• Manage a ConductR cluster within a sbt session

Table of contents

• Prerequisite
• Setup
• Plugin Overview
• Command Overview
• ConductR Plugin
• Bundle Plugins
• Developers

Prerequisite

• Docker (when using Docker based bundles)
• conductr-cli (used to manage the ConductR cluster)

Setup

Add sbt-conductr to your project/plugins.sbt (all versions of Akka, Play and Lagom 1.3 onward):

addSbtPlugin("com.lightbend.conductr" % "sbt-conductr" % "2.7.2")

If your project is using Lagom 1.2.x or prior please use:

addSbtPlugin("com.lightbend.conductr" % "sbt-conductr" % "2.2.9")

In the context of Play or Lagom you should ensure that plugins are enabled as follows:

For anything else:

lazy val root = (project in file(".")).enablePlugins(JavaAppPackaging)

Command Overview

The following sbt-conductr tasks are available:

Type sandbox -h and conduct -h in order to explore the full functionality offered by these commands. In summary, these commands invoke the ConductR CLI directly from sbt and avoid you having to exit any interactive sbt session.

The following sections illustrate how some of these commands can be used.

Starting a ConductR cluster

To start a ConductR cluster on your local machine use:

sandbox run <CONDUCTR_VERSION>

Visit the ConductR Developer page to pick up the latest ConductR version from the section Quick Configuration.

Features

The sandbox contains handy features which can be optionally enabled during startup by specifying the --feature option, e.g.:

sandbox run <CONDUCTR_VERSION> --feature visualization

The following features are available:

Stopping a cluster

To stop the ConductR sandbox use:

sandbox stop

Installing your project

The install command will introspect your project and its sub-projects and then load and run everything in ConductR at once. The local sandbox is expected to be running and it will first be restarted to ensure that it is in a clean state.

Generating an installation script

Just like the install command, the generateInstallationScript command also introspects your project but then writes what is required to load and run everything to a script. The command will output the location of the generated script once done. You are encouraged to copy this script and use it as the basis of installing your project for production deployments.

You can run the script as many times as you need; ConductR load and run commands are idempotent thus allowing you to conveniently load and run any individual components that have stopped for some reason.

Retrieving bundle state

To check the status of your bundles use:

conduct info

To inspect a bundle's full configuration use:

conduct info <my-bundle>

...where <my-bundle> is the identity of your bundle (a bundle id or name).

Generating and loading bundles

To load a bundle to the ConductR cluster, first generate one if you have not done so already (install will generate one). Ensure that you're in the sub-project that you want to generate a bundle for:

project my-sub-project

SBT TIP: type projects to list your sub-projects, or type project <HIT THE TAB KEY>.

...then generate your sub-project's bundle:

bundle:dist

You are now ready to load your bundle:

conduct load <HIT THE TAB KEY AND THEN RETURN>

Using the tab completion feature of sbt will produce a URI representing the location of the last produced bundle within the sbt project. In the context of a multi sbt project it is possible that multiple bundles, each per sub project, has been created. In order to use the tab completion it is then necessary to first switch to the sub project and then use conduct load.

Running bundles

To start a bundle in the cluster use:

conduct run <my-bundle>

This will start the bundle on one instance. To scale it to several instances use the --scale option:

conduct run --scale 3 <my-bundle>

As the bundle name you can either use the bundle id or bundle name. Also the name specified doesn't need to exactly match to the name of the bundle. The specified name only needs to be unqiue within the ConductR cluster. So in case you want to run the bundle this-is-a-very-long-bundle-name you can just type:

conduct run t

If multiple bundles with a starting t exist then the command is aborted and an error message is displayed.

Tags

Tags are an array of strings that can be used to further qualify a bundle name. Tags are often represented as versions e.g. "1.0.0-beta.1", "version1"" etc. By default we use your project's version. Taking the above run example further, we can express a tag by using a : to delimit it with a name:

conduct run <my-bundle>:<my-tag>

More info

To get an overview of all available conduct run options use:

sandbox run --help

Note that all sub-commands offer help.

Stopping bundles

Use the stop command to stop a bundle:

conduct stop <my-bundle>

Retrieve log messages of a bundle

To retrieve log messages of a bundle use:

conduct logs <my-bundle>

Bundle Plugins

By definition, a bundle is a package of one or more components and has metadata describing them all. Bundle metadata includes the name of your bundle, tags, resource requirements, proxying requirements and more.

A bundle typically has just one component which is your service, although it may have more than one component. The general idea of a bundle is that it is entirely self-contained i.e. ConductR does not need to solicit any more information or data regarding your service to invoke it at runtime. This approach permits ConductR to start your services quickly and reliably given that very few network interactions are required.

The bundle plugin produces ConductR bundles and bundle configurations. sbt-conductr contains several bundle plugin. One of the bundle plugin gets used for your project. Check out the Plugin Overview section for more information.

Producing a bundle

To recap, produce a bundle by using the following command within your sub-project:

bundle:dist

Scheduling parameters

A service needs to provide ConductR scheduling parameters to produce a bundle successfully. These parameters effectively describe what resources are used by your service and are used to determine which machine they will run on.

Play and Lagom bundle plugins provide default scheduling parameters. For any other service it is mandatory to specify them, otherwise the bundle:dist command will fail.

Defaults

Play

• Heap Memory: 128 MiB
• Resident Memory: 384 MiB
• Cpus: 0.1
• Disk space: 200 MB

Lagom

• Heap Memory: 128 MiB
• Resident Memory: 384 MiB
• Cpus: 0.1
• Disk space: 200 MB

Set custom scheduling parameters

If the above parameters are inappropriate or you are not using Play or Lagom then we recommend starting with the following custom scheduling parameters in your project's build.sbt e.g.:

import ByteConversions._

javaOptions in Universal := Seq(
  "-J-Xmx128m",
  "-J-Xms128m"
)

BundleKeys.nrOfCpus := 0.1
BundleKeys.memory := 384.MiB
BundleKeys.diskSpace := 200.MB

The javaOptions values declare the maximum and minimum heap size for your application respectively. Profiling your application under load will help you determine an optimal heap size. We recommend declaring the BundleKeys.memory value to be at least 384 MiB noting that this represents the resident memory size of your service. Resident memory includes the heap, thread stacks, code caches, the code itself and so forth. On Unix, use the top command and observe the resident memory column (RES) with your application under load.

BundleKeys.memory is used for locating machines with enough resources to run your application, and so it is particularly important to size it before you go to production. Not setting this value correctly can lead to your bundle being killed by the operating system.

Bundle configuration

It is possible to produce additional configuration bundles that contain an optional bundle.conf to override the main bundle, as well as arbitrary shell scripts. These additional configuration files must be placed in your project's src/bundle-configuration/default folder. Note that configuration bundles can also be declared using settings (explained later).

The bundle-configuration folder may contain many configurations in order to support development style scenarios, the desired configuration can be specified with the setting ("default" is the default folder name) in the build.sbt:

BundleKeys.configurationName := "default"

Then, to produce this additional bundle execute:

configuration:dist

Note that bundle configuration that is generally performed from within sbt is therefore part of the project to support developer use-cases. Operational use-cases where sensitive data is held in configuration is intended to be performed outside of sbt, and in conjunction with the ConductR CLI (specifically the shazar command).

Advanced bundles and configuration

sbt-conductr is capable of producing many bundles and bundle configurations for a given sbt module.

Adding Start command options

Use the BundleKeys.startCommand to add additional options to the start command. Let's say you are using a Play application and want to specify a custom application secret then the option to the BundleKeys.startCommand:

BundleKeys.startCommand += "-Dplay.crypto.secret=dontsharethiskey"

Note that memory heap is controlled by the memory BundleKey and heap flags should not be passed here.

Renaming an executable

Sometimes you need to invoke something other than the script that the native packager assumes. For example, if you have a script in the bin folder named start.sh, and it isn't expecting any Java options:

BundleKeys.executableScriptPath in Bundle := (file((normalizedName in Bundle).value) / "bin" / "start.sh").getPath
javaOptions in Bundle := Seq.empty

Extending bundles

Suppose that you have an sbt module where there are multiple ways in which it can be produced. ReactiveMaps is one such example where the one application can be deployed in three ways:

• frontend
• backend-region
• backend-summary

Its frontend configuration is expressed in the regular way i.e. within the global scope:

// Main bundle configuration

normalizedName := "reactive-maps-frontend"
BundleKeys.nrOfCpus := 0.1
...

Thus a regular bundle:dist will produce the frontend bundle.

We can then extend the bundle configuration and overlay some new values for a different target. Here's a sample of what the backend-region target looks like:

lazy val BackendRegion = config("backend-region").extend(Bundle)
BundlePlugin.bundleSettings(BackendRegion)
inConfig(BackendRegion)(Seq(
  normalizedName := "reactive-maps-backend-region",
  BundleKeys.configurationName := (normalizedName in BackendRegion).value,
  ...
))

A new configuration is created that extends the regular Bundle one for the purposes of delegating sbt settings. Therefore anything declared within the inConfig function will have precedence over that which is declared in the Bundle sbt configuration. The bundleSettings function defines a few important settings that you need.

To produce the above bundle then becomes a matter of just backend-region:dist.

Extending bundle configurations

The optional bundle.conf file can either be provided directly, or be generated via sbt settings. The following shows how to create an sbt configuration and then define bundle.conf settings. The settings are for a fictitious backend configuration that overrides the bundle name and the roles:

lazy val Backend = config("backend").extend(BundleConfiguration)
BundlePlugin.configurationSettings(Backend)
inConfig(Backend)(Seq(
  normalizedName := "reactive-maps-backend",
  roles := Set("big-backend-server")
))

Note the distinction between the configurationSettings and bundleSettings for bundle configurations and bundles respectively.

You must also associate the configuration with your project:

lazy val root = (project in file("."))
  .enablePlugins(JavaAppPackaging)
  .configs(Backend)

A configuration for the above can then be generated:

backend:dist

Bundle settings

The following bundle settings are provided under the BundleKeys object:

HOCON

HOCON is an alternative to JSON that should provides some nice extensions. To illustrate, here are some annotations declared:

BundleKeys.annotations := Some(
  """
    |{
    |  com.mycompany {
    |    reference-id = "124f534as"
    |    zones = ["us-east", "us-west"]
    |  }
    |}
  """.stripMargin)

Plugin Developers

Running unit tests

# Run all tests
sbt test

# Run a single test
sbt test-only com.lightbend.conductr.sbt.BundlePluginSpec

Running scripted tests

# bash - Run all tests
sbt scripted

# bash - Run a single test
sbt "scripted sbt-conductr/conduct-common-args"

# sbt - Interactive test output
set scriptedBufferLog := false
scripted sbt-conductr/conduct-common-args

© Lightbend Inc., 2014-2017