lucidsoftware / piezo

Piezo is a set of tools for operating a quartz scheduling cluster.

GitHub

Piezo

Piezo is a system for operating and managing a Quartz Scheduler cluster. The first component is the Worker, which is a driver or main class for running a Quartz instance. The second is the Admin, which is a web interface for administrating a Quartz cluster, including managing which jobs run, and viewing a history of what processing the cluster has completed. The third project in the diagram below is your library containing the actual jobs to run.

Piezo project architecture

Worker

Worker is a process that runs a Quartz Scheduler instance.

Worker provides a java main() function for running a quartz scheduler as a daemon. It writes a PID file for help with start and stop (e.g. init.d) scripts. It handles the runtime shutdown event and graceful exits when it receives a shutdown signal (ctrl-c/SIGINT).

Worker also expands the set of tables that quartz uses with additional tables to track historical job execution data.

Setup

  1. Create a database. Piezo includes a sample database creation script
  2. Create the standard job store using ONE of the following methods:
    1. Use the sample script included as worker/src/main/resources/tables_mysql.sql (easiest method).
    2. See the quartz job store documentation for the complete set of options.
    3. From the documentation: "JDBCJobStore works with nearly any database, it has been used widely with Oracle, PostgreSQL, MySQL, MS SQLServer, HSQLDB, and DB2. To use JDBCJobStore, you must first create a set of database tables for Quartz to use. You can find table-creation SQL scripts in the 'docs/dbTables' directory of the Quartz distribution. If there is not already a script for your database type, just look at one of the existing ones, and modify it in any way necessary for your DB."
  3. Create the Piezo job history tables.
  4. Modify the included sample quartz.properties to point to your database (see Quartz scheduler library config file).
  5. Run Piezo as specified in Running.

Building

You must have sbt 0.13.

sbt worker/compile compiles sources.

sbt worker/packageBin creates a JAR.

Configuration

JVM properties

Running

When developing

sbt worker/run

This uses the quartz.properties file at the repo root and runs a heatbeat job.

To use to run jobs add com.lucidchart:piezo-worker:<version> as a dependency to your project, and then run the com.lucidchart.piezo.Worker class. For example,

java  -Dorg.quartz.properties=<path to quartz properties> -Dpidfile.path=<path to pid file> -cp <classpath> com.lucidchart.piezo.Worker

Stats

Worker reports statistics to a StatsD server if available.

It also stores historical job execution data in a pair of database tables defined in create_history_tables.sql. These tables should be added to the same datasource as the standard quartz tables.

Admin

Admin is a web interface for viewing and managing the scheduled jobs.

Setup

  1. Follow the steps for the Worker Setup above.

Building

You must have sbt 0.13.

sbt admin/debian:packageBin creates a .deb that includes all library dependencies, and installs piezo-admin as an Upstart service running as piezo-admin.

Configuration

JVM properties

org.quartz.properties

The properties file must have org.quartz.scheduler.classLoadHelper.class: com.lucidchart.piezo.GeneratorClassLoader.

Running

When developing,

sbt admin/run

Then go to http://localhost:8001/ to view the admin.

Debian install

Piezo admin can be installed as a service from a .deb (see Building).

apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 379CE192D401AB61
echo 'deb http://dl.bintray.com/lucidsoftware/apt/ piezo main' > /etc/apt/sources.list.d/piezo.list

apt-get update
apt-get install piezo-admin

By default, it will use /etc/piezo-admin/quartz.properties. Adjust runtime options using /etc/piezo-admin/application.ini:

# -J for for Java options
-J-Xmx1g
# -D for system propertis
-Dorg.quartz.properties=path/to/quartz.properties
# -jvm-debug to enable JVM debugging
-jvm-debug 5005

For deploying to non-Debian platforms, run sbt admin/universal:packageBin. This produces a zip of JARs and scripts in admin/target/universal