JobScheduler Installation and Configuration

Posted on Posted in JobScheduler, linux

JobScheduler

Welcome to another Sysadmin  & DBA tips, in this post I’ll explain how install, configure and create your first Job with the OpenSource JobScheduler tool.

In the University of Navarra, where I work a new challenge arose. We needed a tool to manage all the scheduled jobs with a central infrastructure, like a centralized Cron. The idea was a Web Console that will permit control all scheduled jobs, his states and a history  log information. This web interface would facilitate, for example, stop all Mysql Backups or launch it before a programmed system Upgrade.

For this and other jobs, We decided used JobScheduler. Like the official documentation said:

JobScheduler is used for the automation of, for example, processes and can use the proprietary SQL Server, Oracle or DB2 database systems or the free MySQL, PostgreSQL or Firebird.

You can use JobScheduler to launch executable files and shell scripts and to run database procedures automatically. Jobs are configurable as Web Services thereby providing interoperability with enterprise applications.

Installation

This manual collected a new JobScheduler Standalone installation with Mysql database integration. The tool can be used without a database but it is strongly recommended that the JobScheduler use a database. The JobScheduler needs the database to archive logs and to remember Job and Order states.

Requirements

Java Runtime Environment

Regardless of the architecture of you machine, the Oracle™ Java Runtime Environment (JRE) 32-Bit at least version 1.6.x is necessary. Download and install

[root@scheduler]# chmod u+x jre-6u34-linux-i586-rpm.bin
[root@scheduler]#./jre-6u34-linux-i586-rpm.bin

Libraries

The JobScheduler requires some 32-Bit libraries

• linux-gate.so.1
• libz.so.1
• libpthread.so.0
• libdl.so.2
• libm.so.6
• ld-linux.so.2
• libgtk-x11-2.0.so.0
• libXtst.so.6

[root@scheduler]# yum install gtk2-2.18.9-10.el6.i686
[root@scheduler]# yum install libXtst-1.0.99.2-3.el6.i686

Mysql JDBC

Because of licensing restrictions when used with MySQL, a JDBC driver appropriate to the database version used must be provided by the end users themselves. Download and save it, during the installation the installer will ask us.

[root@scheduler]# wget http://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.22.tar.gz/from/http://cdn.mysql.com/

Mysql Database

Before running the installer, we will create the database

mysql> create database jobScheduler;
Query OK, 1 row affected (0.03 sec)

mysql> grant all privileges on jobScheduler.* to 'scheduler'@'scheduler.albertolarripa.com' identified by '****';
Query OK, 0 rows affected (0.00 sec)

 JobScheduler

Ok, it’s the moment to start with JobScheduler installation, but before that, you must create a new user that will run the daemon service.

useradd jobscheduler
passwd jobscheduler

Now granted all privileges for this user, use visudo to do that

scheduler ALL=(ALL) ALL

Be sure that the requireTTY must be commented

#Defaults requiretty

After that, download and install the software

[root@scheduler]# wget http://downloads.sourceforge.net/project/jobscheduler/scheduler_linux.1.3.12.2347.tar.gz
[root@scheduler]# tar xvfz scheduler_linux.1.3.12.2347.tar.gz

Wizard Installation

Execute the installer with the X environment and the scheduler user

[root@scheduler]# su - scheduler /usr/local/etc/JobScheduler/jobscheduler.1.3.12.2347/setup.sh

Follow the installation catch

Select GNU GPL 2.0 License

Accept the Licensing Agreements

Select the installation binaries and libraries directory

Used the same directory for the configuration and log files

Now select the packs you want to install

Define your JobScheduler ID, the host and port where the service will run and finally allow the Web connection for all hosts.

Different installations are possible but this is the primary JobScheduler server, select standalone configuration

Configure your mail recipients and SMTP server

Choose the levels of notifications

Jobscheduler Update Service, won’t allow automatic updates

Configure the Mysql database server

Select the JDBC that We have previously downloaded.

In my case this part of the installation failed, but I can setup the database later and after call the script <INSTALLATION>/install/scheduler_install_tables.sh

Crontab configuration Job

The installation will be started

Database installation

Like I said, in my case the Database installation wizard failed. To solved this error edit <PAH_INSTALLATION>/config/sos_settings.ini 

[configuration]
class = SOSMySQLConnection
driver = com.mysql.jdbc.Driver
url = jdbc:mysql://mysql.albertolarripa.com:3309/jobScheduler
user = scheduler
password = <PASSWORD>
compatibility = normal

Install the database

[root@scheduler]# ./install/scheduler_install_tables.sh 

...the database management system is mysql
...installing tables in SQL database
20:11:02.296---------- sábado enero 26 20:11:02 2013
20:11:02.296	[info]   file successfully processed: /usr/local/etc2/jobScheduler/scheduler/db/mysql/scheduler_user_jobs.sql
20:11:02.453	[info]   file successfully processed: /usr/local/etc2/jobScheduler/scheduler/db/mysql/sosdailyschedule.sql
20:11:02.667	[info]   file successfully processed: /usr/local/etc2/jobScheduler/scheduler/db/mysql/sosftphistory.sql
20:11:02.724	[info]   file successfully processed: /usr/local/etc2/jobScheduler/scheduler/db/mysql/scheduler_loganalyser.sql
20:11:02.853	[info]   file successfully processed: /usr/local/etc2/jobScheduler/scheduler/db/mysql/scheduler_sanity.sql
20:11:03.548	[warn]   an error occurred processing file [/usr/local/etc2/jobScheduler/scheduler/db/mysql/scheduler.sql]: sos.connection.SOSConnection.executeStatements(3036) : java.lang.Exception: sos.connection.SOSConnection.executeUpdate(2034):java.sql.SQLException: Cannot execute statement: impossible to write to binary log since BINLOG_FORMAT = STATEMENT and at least one table uses a storage engine limited to row-based logging. InnoDB is limited to row-logging when transaction isolation level is READ COMMITTED or READ UNCOMMITTED.
20:11:03.603	[info]   file successfully processed: /usr/local/etc2/jobScheduler/scheduler/db/mysql/scheduler_events.sql
...insert initial rows into tables in SQL database
20:11:05.480---------- sábado enero 26 20:11:05 2013
20:11:05.480	[warn]   an error occurred processing file [/usr/local/etc2/jobScheduler/scheduler/db/mysql/scheduler_sanity_insert.sql]: sos.connection.SOSConnection.executeStatements(3036) : java.lang.Exception: sos.connection.SOSConnection.executeUpdate(2034):java.sql.SQLException: Cannot execute statement: impossible to write to binary log since BINLOG_FORMAT = STATEMENT and at least one table uses a storage engine limited to row-based logging. InnoDB is limited to row-logging when transaction isolation level is READ COMMITTED or READ UNCOMMITTED.
...done

Ohh!!!, some errors during the installation

20:11:03.548 [warn] an error occurred processing file [/usr/local/etc2/jobScheduler/scheduler/db/mysql/scheduler.sql]: sos.connection.SOSConnection.executeStatements(3036) : java.lang.Exception: sos.connection.SOSConnection.executeUpdate(2034):java.sql.SQLException: Cannot execute statement: impossible to write to binary log since BINLOG_FORMAT = STATEMENT and at least one table uses a storage engine limited to row-based logging. InnoDB is limited to row-logging when transaction isolation level is READ COMMITTED or READ UNCOMMITTED.

You will need to change the binlog mode to either ROW or MIXED in order to run the load of the data into the database
Edit your my.cnf and add:

------------------
binlog_format=mixed
------------------

Recreate the database

mysql> drop database jobScheduler;
Query OK, 22 rows affected (0.39 sec)

mysql> create database jobScheduler;
Query OK, 1 row affected (0.00 sec)

Relaunches the installation

[root@scheduler]# ./scheduler_install_tables.sh 
...the database management system is mysql
...installing tables in SQL database
20:18:05.994---------- sábado enero 26 20:18:05 2013
20:18:05.994	[info]   file successfully processed: /usr/local/etc2/jobScheduler/scheduler/db/mysql/scheduler_user_jobs.sql
20:18:06.112	[info]   file successfully processed: /usr/local/etc2/jobScheduler/scheduler/db/mysql/sosdailyschedule.sql
20:18:06.306	[info]   file successfully processed: /usr/local/etc2/jobScheduler/scheduler/db/mysql/sosftphistory.sql
20:18:06.373	[info]   file successfully processed: /usr/local/etc2/jobScheduler/scheduler/db/mysql/scheduler_loganalyser.sql
20:18:06.512	[info]   file successfully processed: /usr/local/etc2/jobScheduler/scheduler/db/mysql/scheduler_sanity.sql
20:18:07.172	[info]   file successfully processed: /usr/local/etc2/jobScheduler/scheduler/db/mysql/scheduler.sql
20:18:07.252	[info]   file successfully processed: /usr/local/etc2/jobScheduler/scheduler/db/mysql/scheduler_events.sql
...insert initial rows into tables in SQL database
20:18:09.172---------- sábado enero 26 20:18:09 2013
20:18:09.172	[info]   file successfully processed: /usr/local/etc2/jobScheduler/scheduler/db/mysql/scheduler_sanity_insert.sql
...done

Java Enviroment variables

Now edit jobscheduler_environment_variables.sh to define your JAVA_HOME & LD_LIBRARY_PATH

+++++++++++++++++++++++++++++++++++++++
 JAVA_HOME="/usr/java/jre1.6.0_34"
 LD_LIBRARY_PATH="$SCHEDULER_HOME/lib:/usr/local/lib:/usr/java/jre1.6.0_34/lib:/usr/java/jre1.6.0_34/lib/i386/server:$LD_LIBRARY_PATH"
 +++++++++++++++++++++++++++++++++++++++

Start JobScheduler Service

$SCHEDULER_HOME/bin/jobscheduler.sh start

And enter via WebBrowser jobserver_ip:4444

If you want the JobScheduler to be started automatically at server startup, copy this script and the $SCHEDULER_HOME/bin/jobscheduler_environment_variables.sh to the /etc/init.d directory

[root@scheduler bin]# cp -pr jobscheduler.sh /etc/init.d/
[root@scheduler bin]# cp -pr jobscheduler_environment_variables.sh /etc/init.d/

Configuration files

  • sos.ini –> License key, Java options.
  • factory.ini –> Email, database connection and Java archives classpath.
  • scheduler.xml –> The JobScheduler port configuration.
  • custom.js –> Used by JOC (JobScheduler Operations Center). Beside other settings you can configure the language and filters in particular.
  • hibernate.cfg.xml –> Used by y JID (JobScheduler Information Dashboard) to get the database connection
  • jobscheduler_environment_variables.sh –> In this file the start parameter and the ID of the JobScheduler are set
  • jobeditor_environment_variables.sh –> In this file the environment of JOE (JobScheduler Object Editor) can be set.
  • dashboard_environment_variables.sh –> In this file the environment of JID (JobScheduler Information Dashboard) can be set.

 Working with JobScheduler

JobScheduler objects (jobs, job chains, orders, schedules, process classes and locks) are all configured using plain XML files. These files are saved in a so-called Hot Folders (the default location of this folder or directory is <INSTALLATION_PATH>/config/live ). There is a file in this directory for every JobScheduler object, in which the object’s configuration is saved.

  • JobScheduler Object Editor

JOE (JobScheduler Object Editor) is an application to create, edit and maintain the JobScheduler objects (jobs, job chains, orders and schedules).

$SCHEDULER_HOME/bin/jobeditor.sh

  • JobScheduler Information Dashboard

JID (JobScheduler Information Dashboard) is an application to provide an overview of the jobs planned and those that have successfully been completed.

$SCHEDULER_HOME/bin/dashboard.sh

  • JobScheduler Operations Center

JOC (JobScheduler Operations Center) is the interface that permit saw the states of the scheduled jobs and interact with them.

Building your first Job

The first thing to do is create a new Hot Folder inside <JOBSCHEDULER>/config/live

[root@scheduler]# cd /config/live
[root@scheduler]# mkdir first-job

Run JobScheduler Object Editor

$SCHEDULER_HOME/bin/jobeditor.sh

Open your Hot Folder directory, use the key combination ctrl+D or File/Open Live Folder.

If you open the JOC (JobScheduler Operations Center), you can see the Hot Folder directory.

The are two types of jobs:
Standalone Jobs can be started independently of other Jobs but cannot be used in Job Chains.
Order Jobs are used in Job Chains and are are started in a Job Chain by an Order for that particular Job Chain.

Continue with the JOE, create new Standalone Job clicking in Jobs/New Standalone job.

Now define your Job name and this shell script

#!/bin/sh
echo "$SCHEDULER_JOB_NAME is running"
echo "Current Timestamp: `date`"
exit $?

After saving the configuration JOE will create a new XML file inside <JOBSCHEDULER>/config/live/your/Hot_folder/firstjob.job.xml

Now change to JOC and click in the Update button, you can see your “firstjob” Job.

The task will now start to run and you can follow its progress in the Task history by clicking on the  Show log  button.

This will open a new browser window with the log output, where, amongst other things, you will be able to read the output from the shell script.

And this is all, I hope this information will be useful.

In future post I’ll explain how to squeeze this utility in all respects, like a future Nagios Integration ;-).

See you in other Sysadmin & DBA tips

Leave a Reply

Your email address will not be published. Required fields are marked *