A framework for testing Pentaho Kettle Transformations.

• sbt >= 1.4.3
• Scala >= 2.13.6
• Java JDK >= 1.8
• Git

1. Clone the Git repository:

git clone https://github.com/nationalarchives/kettle-test-framework.git

2. Compile and install to your local Ivy or Maven repository (or use the version from Maven Central):

sbt clean compile publishLocal

After this other builds on the same machine can depend on it:

libraryDependencies += "uk.gov.nationalarchives" %% "kettle-test-framework" % "0.4.0-SNAPSHOT"

The test framework is designed to allow you to test Pentaho Kettle transformations without needing to have the Pentaho Data Integration tool or an RDBMS installed on the local machine. It achieves by running Pentaho and the H2 database in embedded mode during tests.

Testing generally involves three phases:

1. Populating the database with test data by writing a SQL script which can be executed via the DatabaseManager

2. Executing the transformation via the WorkflowManager, having specifying any parameters or plugins that are needed by the transformation.

3. Querying the resulting output to check for expected results via the QueryManager. Currently the only RDF output data and SPARQL queries are supported, but this may be extended to other formats in future.

An example test is provided at src/test/scala/ExampleWorkflowSpec.scala

To ensure that your Kettle transformations are suitable for testing it is important that they have no dependencies on specific databases or file paths. Such elements should be parameterised in the transformations so that they can be specified at execution time by the test framework.

1. Run sbt clean release
2. Answer the questions
3. Login to https://oss.sonatype.org/ then Close, and Release the Staging Repository

It is recommended for developers to make use of the 'formatAndCommit' alias command as part of a pre-push git hook in order to make sure the code is properly formatted before pushing it to remote. CTD projects are now configured to fail git actions if code is not properly formatted, so in order to avoid this a git hook is handly.

In order to create the hook you need to go to the following path '.git/hooks' and create a new script file called 'pre-push' containing just a single line 'sbt formatAndCommit'. This file should have execution permissions.