Page History

Table of Contents

Scope

JobScheduler Master is operated with a Docker container.
Prerequisites
- The JobScheduler Master requires a database to be available
.
x

Build

Dockerfile

- from any physical or virtual host or from a Docker container.
- Consider to prepare the files indicated with chapter Build.

Build

The following files are required for the build context:

Dockerfile
JobScheduler Master installer tarball as available from SOS for download.
Installer Response file jobscheduler_install.xml with individual installation settings. A template of this file is available when extracting the installer tarball.
Start Script start_jobscheduler.sh

Dockerfile

Download: Dockerfile

Code Block

language	bash
title	Dockerfile
linenumbers	true
collapse	true

FROM openjdk:8
LABEL maintainer="Software- und Organisations-Service GmbH"

# default user id has to match later run-time user
ARG USER_ID=$UID

# provide build arguments for release information
ARG JS_MAJOR=1.13
ARG JS_RELEASE=1.13.3-SNAPSHOT

# add installer archive file
COPY jobscheduler_linux-x64.${JS_RELEASE}.tar.gz /usr/local/src/
RUN test -e /usr/local/src/jobscheduler_linux-x64.${JS_RELEASE}.tar.gz && \
    tar zxvf /usr/local/src/jobscheduler_linux-x64.${JS_RELEASE}.tar.gz -C /usr/local/src/ && \
    rm -f /usr/local/src/jobscheduler_linux-x64.${JS_RELEASE}.tar.gz && \
    ln -s /usr/local/src/jobscheduler.${JS_RELEASE} /usr/local/src/jobscheduler

# for JDK < 12, /dev/random does not provide sufficient entropy, see https://kb.sos-berlin.com/x/lIM3
RUN rm /dev/random && ln -s /dev/urandom /dev/random

# copy installer response file and run installer
COPY jobscheduler_install.xml /usr/local/src/jobscheduler/jobscheduler_install.xml
RUN /usr/local/src/jobscheduler/setup.sh -u /usr/local/src/jobscheduler/jobscheduler_install.xml

# make default user the owner of directories
RUN groupadd --gid ${USER_ID:-1000} jobscheduler && \
    useradd --uid ${USER_ID:-1000} --gid jobscheduler --home-dir /home/jobscheduler --no-create-home --shell /bin/bash jobscheduler && \
    chown -R jobscheduler:jobscheduler /var/sos-berlin.com

# copy and prepare start script
COPY start_jobscheduler.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/start_jobscheduler.sh

# create volumes for data persistence
VOLUME /var/sos-berlin.com/jobscheduler/testsuite/config/live

# allow incoming traffic to port
EXPOSE 40444

# run-time user, can be overwritten when running the container
USER jobscheduler

CMD ["/usr/local/bin/start_jobscheduler.sh"]

Explanations
- Line 1: We start from a CentOS image that includes JDK 8. Newer Java version can be used, see Which Java versions is JobScheduler available for?
- Line 5: Consider that $UID provides the numeric ID of the account that the JobScheduler Master installation inside the Docker container is performed for. This numeric ID typically starts above 1000 and should correspond to the account that is used on the Docker host, i.e. the account on the Docker Host and the account inside the container should use the same numeric ID. This mechanism simplifies exposure of the Docker container's file system.
- Line 8-9: Adjust the JobScheduler release number as required.
- Line 12-16: The installer tarball is copied and extracted to the container.
- Line 22-23: The installer response file is copied to the container, for details of this file see next chapter. Then the installer is executed for the current user.
- Line 26-28: An account and group "jobscheduler" is created that is handed over ownership of installed files.
- Line 31-32: The start script is copied to the container, see below chapter Start Script.
- Line 35: A volume is indicated for later mapping to a mount point at run-time.
- Line 38: Port 40444 is exposed for later mapping. This port is used for the connection between JOC Cockpit and JobScheduler Master.
- Line 41: The account "jobscheduler" that is the owner of the installation is exposed for later mapping. This account should be mapped at run-time to the account in the Docker Host that intends to will mount the exposed volume.
- Line 43: The start script is executed to launch the JobScheduler Master daemon.

...

Download: jobscheduler_install.xml

Code Block

language	xml
title	Installer Response File
linenumbers	true
collapse	true

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- 
XML configuration file for JobScheduler setup

The JobScheduler is available with a dual licensing model.
- GNU GPL 2.0 License (see http://www.gnu.org/licenses/gpl-2.0.html)
- JobScheduler Commercial License (see licence.txt)

The setup asks you for the desired license model 
(see <entry key="licenceOptions" .../> below).

If you call the setup with this XML file then you accept 
at the same time the terms of the chosen license agreement. 
-->
<AutomatedInstallation langpack="eng">
    <com.izforge.izpack.panels.UserInputPanel id="home">
        <userInput/>
    </com.izforge.izpack.panels.UserInputPanel>
    <com.izforge.izpack.panels.UserInputPanel id="licences">
        <userInput>
        
            <!-- Select the license model (GPL or Commercial) -->
            <entry key="licenceOptions" value="GPL"/>
            
            <!-- If you selected GPL as license model than the licence must be empty.
                 Otherwise please enter a license key if available.
                 It is also possible to modify the license key later. -->
            <entry key="licence" value=""/>
            
        </userInput>
    </com.izforge.izpack.panels.UserInputPanel>
    <com.izforge.izpack.panels.HTMLLicencePanel id="gpl_licence"/>
    <com.izforge.izpack.panels.HTMLLicencePanel id="commercial_licence"/>
    <com.izforge.izpack.panels.TargetPanel id="target">
        
        <!-- SELECT THE INSTALLATION PATH FOR THE BINARIES AND LIBRARIES
             The installation expands this path with the Scheduler ID as subdirectory.
             The path must be absolute!
             Default paths are
             /opt/sos-berlin.com/jobscheduler for Unix
             C:\Program Files\sos-berlin.com\jobscheduler for Windows -->
        <installpath>/opt/sos-berlin.com/jobscheduler</installpath>
        
    </com.izforge.izpack.panels.TargetPanel>
    <com.izforge.izpack.panels.UserPathPanel id="userpath">
        
        <!-- SELECT THE DATA PATH FOR CONFIGURATION AND LOG FILES
             The installation expands this path with the Scheduler ID as subdirectory.
             The path must be absolute!
             Default paths are
             /home/[user]/sos-berlin.com/jobscheduler for Unix
             C:\ProgramData\sos-berlin.com\jobscheduler for Windows -->
        <UserPathPanelElement>/var/sos-berlin.com/jobscheduler</UserPathPanelElement>
        
    </com.izforge.izpack.panels.UserPathPanel>
    <com.izforge.izpack.panels.PacksPanel id="package">
    
        <!-- SELECT THE PACKS WHICH YOU WANT INSTALL -->
           
        <!-- Package: JobScheduler
             JobScheduler Basic Installation
             THIS PACK IS REQUIRED. IT MUST BE TRUE -->
        <pack index="0" name="Job Scheduler" selected="true"/>
        
        <!-- Package: Database Support
             Job history and log files can be stored in a database. Database support is 
             available for MySQL, PostgreSQL, Oracle, SQL Server, DB2.
             THIS PACK IS REQUIRED. IT MUST BE TRUE -->
        <pack index="2" name="Database Support" selected="true"/>
        
        <!-- Package: Housekeeping Jobs
             Housekeeping Jobs are automatically launched by the Job Scheduler, e.g. to send 
             buffered logs by mail, to remove temporary files or to restart the JobScheduler. -->
        <pack index="5" name="Housekeeping Jobs" selected="true"/>
        
    </com.izforge.izpack.panels.PacksPanel>
    <com.izforge.izpack.panels.UserInputPanel id="network">
        <userInput>
            <!-- Network Configuration -->
            
            <!-- Enter the port for TCP communication (e.g. 4444) 
                 No longer required! -->
            <entry key="schedulerPort" value=""/>
            
            <!-- Enter the port for HTTP communication -->
            <entry key="schedulerHTTPPort" value="40444"/>
            
            <!-- To enter a JobScheduler ID is required. 
                 The IDs of multiple instances of the JobScheduler must be unique per server. 
                 The JobScheduler ID expands the above installation paths as subdirectory.
                 Please omit special characters like: / \ : ; * ? ! $ % & " < > ( ) | ^ -->
            <entry key="schedulerId" value="testsuite"/>
            
            <!-- Only for Linux (root permissions required) -->
            <entry key="schedulerInstallAsDaemon" value="no"/>
            	
            <!-- To enter a JobScheduler User (default=current User). 
                 Only for Linux (root permissions required) -->
            <entry key="runningUser" value=""/>
            
            <!-- Specify optional Java options here -->
            <entry key="jsJavaOptions" value=""/>
            
            <!-- It is recommended to enable TCP access for the host where the JobScheduler will install, 
                 optionally enter additional host names or ip addresses. To enable all hosts in your 
                 network to access the JobScheduler enter '0.0.0.0'. -->
            <entry key="schedulerAllowedHost" value="0.0.0.0"/>
            
            <!-- Choose (yes or no) wether the JobScheduler should be started at the end of the installation -->
            <entry key="launchScheduler" value="no"/>
            
        </userInput>
    </com.izforge.izpack.panels.UserInputPanel>
    <com.izforge.izpack.panels.UserInputPanel id="cluster">
        <userInput>
            <!-- Cluster Configuration / Job Streams Plugin -->
            
            <!-- The JobScheduler can be installed independent of other possibly JobSchedulers, 
                 as a primary JobScheduler in a backup system or as a backup JobScheduler. 
                 Use '' for a standalone, '-exclusive' for a primary 
                 or '-exclusive -backup' for a backup JobScheduler.
                 A database is required for a backup system. All JobSchedulers in a backup system 
                 must have the same JobScheduler ID and the same database. 
                 Further you can set '-distributed-orders' for a load balancing cluster.
                 For more information see
                 http://www.sos-berlin.com/doc/de/scheduler.doc/backupscheduler.xml
                 http://www.sos-berlin.com/doc/de/scheduler.doc/distributed_orders.xml -->
            <entry key="clusterOptions" value=""/>
            
            <!-- Enable JobStreams plugin.
                 For more information see https://kb.sos-berlin.com/x/uoC2AQ -->
            <entry key="jobStreamsPlugin" value="on"/>
            
        </userInput>
    </com.izforge.izpack.panels.UserInputPanel>
    <com.izforge.izpack.panels.UserInputPanel id="smtp">
        <userInput>
            <!-- Mail Recipients Configuration / SMTP Authentication -->
            
            <!-- Enter the ip address or host name and port (default: 25) of your SMTP server -->
            <entry key="mailServer" value=""/>
            <entry key="mailPort" value="25"/>
            
            <!-- Configure the SMTP authentication if necessary. -->
            <entry key="smtpAccount" value=""/>
            <entry key="smtpPass" value=""/>
            
            <!-- Enter the addresses of recipients to which mails with log files are automatically
                 forwarded. Separate multiple recipients by commas -->
            
            <!-- Account from which mails are sent -->
            <entry key="mailFrom" value=""/>
            
            <!-- Recipients of mails -->
            <entry key="mailTo" value=""/>
            
            <!-- Recipients of carbon copies: -->
            <entry key="mailCc" value=""/>
            
            <!-- Recipients of blind carbon copies -->
            <entry key="mailBcc" value=""/>
            
        </userInput>
    </com.izforge.izpack.panels.UserInputPanel>
    <com.izforge.izpack.panels.UserInputPanel id="email">
        <userInput>
            <!-- Mail Configuration / Event Handler -->
            
            <!-- Choose in which cases mails with log files are automatically forwarded. -->
            <entry key="mailOnError" value="yes"/>
            <entry key="mailOnWarning" value="yes"/>
            <entry key="mailOnSuccess" value="no"/>
            
            <!-- The Housekeeping package is required for configure JobScheduler as event handler
                 Choose this option if you intend to use JobScheduler Events and
                 - this JobScheduler instance is the only instance which processes Events
                 - this JobScheduler instance is a supervisor for other JobSchedulers which submit Events -->
            <entry key="jobEvents" value="off"/> 
            
        </userInput>
    </com.izforge.izpack.panels.UserInputPanel>
    <com.izforge.izpack.panels.UserInputPanel id="database">
        <userInput>
            <!-- JobScheduler Database Configuration -->
            
            <!-- Database connection settings can be specified with following entries such as
                 databaseHost, databasePort, ... or by a hibernate configuration file 
                 Posible values are 'withoutHibernateFile' (default) and 'withHibernateFile'. -->
            <entry key="databaseConfigurationMethod" value="withoutHibernateFile"/>
                 
            <!-- Choose the database management system. Supported values are 'mysql' for MySQL,
                 'oracle' for Oracle, 'mssql' for MS SQL Server, 'pgsql' for PostgreSQL,
                 'db2' for DB2 and 'sybase' for Sybase. 
                 Only if databaseConfigurationMethod=withoutHibernateFile -->
            <entry key="databaseDbms" value="mysql"/>
            
            <!-- Path to a hibernate configuration file if databaseConfigurationMethod=withHibernateFile -->
            <entry key="hibernateConfFile" value=""/>
            
            <!-- You can choose between 'on' or 'off' to create the database tables.
                 If you have modified the initial data of an already existing installation, 
                 then the modifications will be undone. Data added remains unchanged. 
                 This entry should be only 'off', when you sure, that all tables are already created. -->
            <entry key="databaseCreate" value="on"/>
            
        </userInput>
    </com.izforge.izpack.panels.UserInputPanel>
    <com.izforge.izpack.panels.UserInputPanel id="dbconnection">
        <userInput>
            <!-- JobScheduler Database Configuration if databaseConfigurationMethod=withoutHibernateFile -->
                 
            <!-- Enter the name or ip address of the database host 
                 This entry can also be used to configure the URL(s) for Oracle RAC databases.
                 For example:
                 <entry key="databaseHost" value="(DESCRIPTION=(ADDRESS_LIST=(LOAD_BALANCE=OFF)(FAILOVER=ON)
                    (ADDRESS=(PROTOCOL=TCP)(HOST=tst-db1.myco.com)(PORT=1604))
                    (ADDRESS=(PROTOCOL=TCP)(HOST=tst-db2.myco.com)(PORT=1604)))
                    (CONNECT_DATA=(SERVICE_NAME=mydb1.myco.com)(SERVER=DEDICATED)))"/>
                 The "databaseSchema" and "databasePort" entries should then be left empty. -->
            <entry key="databaseHost" value="mysql-5-7"/>
            
            <!-- Enter the port number for the database instance. Default ports are for MySQL 3306, 
                 Oracle 1521, MS SQL Server 1433, postgreSQL 5432, DB2 50000, Sybase 5000. -->
            <entry key="databasePort" value="3306"/>
            
            <!-- Enter the schema -->
            <entry key="databaseSchema" value="jobscheduler113"/>
            
            <!-- Enter the user name for database access -->
            <entry key="databaseUser" value="jobscheduler"/>
            
            <!-- Enter the password for database access -->
            <entry key="databasePassword" value="jobscheduler"/>
            
        </userInput>
    </com.izforge.izpack.panels.UserInputPanel>
    <com.izforge.izpack.panels.UserInputPanel id="jdbc">
        <userInput>
            <!-- JobScheduler Database Configuration -->
                 
            <!-- You can specify an external JDBC connector then set internalConnector = no
                 For license reasons MySQL, Sybase, MS SQL Server and Oracle ojdbc7 JDBC 
                 drivers are not provided. 
                 Alternatively you can use the mariadb JDBC Driver for MySQL and 
                 the jTDS JDBC Driver for MS SQL Server and Sybase which is provided. 
                 An Oracle ojdbc6 JDBC driver is also provided.
                 An internal JDBC connector for DB2 is not available -->
                 
            <!-- You can choose between 'yes' or 'no' for using the internal JDBC connector
                 or not -->
            <entry key="internalConnector" value="yes"/>
            
            <!-- Select the path to JDBC Driver -->
            <entry key="connector" value=""/>
            
            <!-- Only for DB2: Select the path to DB2 license file for JDBC Driver -->
            <entry key="connectorLicense" value=""/>
            
        </userInput>
    </com.izforge.izpack.panels.UserInputPanel>
    <com.izforge.izpack.panels.UserInputPanel id="reportingDatabase">
        <userInput>
            <!-- Reporting Database Configuration 
                 NOT SUPPORTED FOR SYBASE AND DB2 -->
            
            <!-- Set 'yes' if the JobScheduler and the Reporting database are the same.
                 If 'yes' then further Reporting database variables are ignored. -->
            <entry key="sameDbConnection" value="yes"/>
            
            <!-- Database connection settings can be specified with following entries such as
                 databaseHost, databasePort, ... or by a hibernate configuration file 
                 Posible values are 'withoutHibernateFile' (default) and 'withHibernateFile'. -->
            <entry key="reporting.databaseConfigurationMethod" value="withoutHibernateFile"/>            
                 
            <!-- Choose the database management system. Supported values are 'mysql' for MySQL,
                 'oracle' for Oracle, 'mssql' for MS SQL Server, 'pgsql' for PostgreSQL. 
                 only if reporting.databaseConfigurationMethod=withoutHibernateFile-->
            <entry key="reporting.databaseDbms" value="mysql"/>
            
            <!-- Path to a hibernate configuration file if reporting.databaseConfigurationMethod=withHibernateFile -->
            <entry key="reporting.hibernateConfFile" value=""/>
            
        </userInput>
    </com.izforge.izpack.panels.UserInputPanel>
    <com.izforge.izpack.panels.UserInputPanel id="reportingDbconnection">
        <userInput>
            <!-- Reporting Database Configuration if reporting.databaseConfigurationMethod=withoutHibernateFile
                 NOT SUPPORTED FOR SYBASE AND DB2 -->
                 
            <!-- Enter the name or ip address of the database host 
                 This entry can also be used to configure the URL(s) for Oracle RAC databases.
                 For example:
                 <entry key="reporting.databaseHost" value="(DESCRIPTION=(ADDRESS_LIST=(LOAD_BALANCE=OFF)(FAILOVER=ON)
                    (ADDRESS=(PROTOCOL=TCP)(HOST=tst-db1.myco.com)(PORT=1604))
                    (ADDRESS=(PROTOCOL=TCP)(HOST=tst-db2.myco.com)(PORT=1604)))
                    (CONNECT_DATA=(SERVICE_NAME=mydb1.myco.com)(SERVER=DEDICATED)))"/>
                 The "reporting.databaseSchema" and "reporting.databasePort" entries should then be left empty. -->
            <entry key="reporting.databaseHost" value="mysql-5-7"/>
            
            <!-- Enter the port number for the database instance. Default ports are for MySQL 3306, 
                 Oracle 1521, MS SQL Server 1433, postgreSQL 5432. -->
            <entry key="reporting.databasePort" value="3306"/>
            
            <!-- Enter the schema -->
            <entry key="reporting.databaseSchema" value="jobscheduler113"/>
            
            <!-- Enter the user name for database access -->
            <entry key="reporting.databaseUser" value="jobscheduler"/>
            
            <!-- Enter the password for database access -->
            <entry key="reporting.databasePassword" value="jobscheduler"/>
            
        </userInput>
    </com.izforge.izpack.panels.UserInputPanel>
    <com.izforge.izpack.panels.UserInputPanel id="reportingJdbc">
        <userInput>
            <!-- Reporting Database Configuration 
                 NOT SUPPORTED FOR SYBASE AND DB2 -->
                 
            <!-- You can specify an external JDBC connector then set reporting.internalConnector = no
                 For license reasons MySQL, MS SQL Server and Oracle ojdbc7 JDBC 
                 drivers are not provided. 
                 Alternatively you can use the mariadb JDBC Driver for MySQL and 
                 the jTDS JDBC Driver for MS SQL Server and Sybase which is provided. 
                 An Oracle ojdbc6 JDBC driver is also provided. -->
                 
            <!-- You can choose between 'yes' or 'no' for using the internal JDBC connector
                 or not -->
            <entry key="reporting.internalConnector" value="yes"/>
                 
            <!-- Select the path to JDBC Driver -->
            <entry key="reporting.connector" value=""/>
            
        </userInput>
    </com.izforge.izpack.panels.UserInputPanel>
    <com.izforge.izpack.panels.UserInputPanel id="end">
        <userInput/>
    </com.izforge.izpack.panels.UserInputPanel>
    <com.izforge.izpack.panels.InstallPanel id="install"/>
    <com.izforge.izpack.panels.ProcessPanel id="process"/>
    <com.izforge.izpack.panels.FinishPanel id="finish"/>
</AutomatedInstallation>

Explanations
- The above installer response file works for releases 1.13. Other releases ship with different versions of this file. You should pick-up a template of this file that matches your JobScheduler release by extracting the installer tarball.
- Generally all Generally all defaults of the response file can be maintained.
  - This includes use of port 40444 for the connection of JOC Cockpit to the Master. At run-time this port can be mapped, see Dockerfile.
- Line 20182-25233: The database connection makes use of a hostname "mysql-5-7" that is assumed to be the hostname of a Docker container running the MySQL database.
  - Modify the database connection settings as required for other use with your DBMS and access credentials.

...

Download: start_jobscheduler.sh

Code Block

language	bash
title	Start Script
linenumbers	true

#!/bin/sh

/opt/sos-berlin.com/jobscheduler/testsuite/bin/jobscheduler.sh start without-change-user && tail -f /dev/null

Explanations
- Line 3: The standard start script jobscheduler.sh is used. The tail command prevents the start script from terminating in order to keep the container alive.

Build Command

There are a number of ways how to write a build command, find the following example:

A typical build command could look like this:

Code Block

language	bash
title	Build Command
linenumbers	true

#!/bin/sh

IMAGE_NAME="Master-1-13"

docker build --no-cache --rm --tag=$IMAGE_NAME --file=$SCRIPT_HOME/build/Dockerfile --network=js --build-arg="USER_ID=$UID" ./build

Explanations
- Using a common network for JobScheduler components allows direct access to resources such as ports within the network. The network is required at build time to allow the installer to create and populate the JobScheduler database.
- Consider use of the --build-arg that injects the USER_ID environment variable to the Dockerfile with the numeric ID of the account running the build command. This simplifies access to the volume exposed by the Dockerfile as the same numeric user ID and group ID inside and outside of the container are used.

Run

There are a number of ways how to write a run command, find the following example:

A typical run command could look like this:

Code Block

language	bash
title	Run Command
linenumbers	true

#!/bin/sh

IMAGE_NAME="Master-1-13"

RUN_USER_ID="$(id -u $USER):$(id -g $USER)"

mkdir -p /some/path/logs

docker run -dit --rm --user=$RUN_USER_ID --hostname=$IMAGE_NAME --network=js --publish=50444:40444 --mount="type=volume,src=$IMAGE_NAME,dst=/var/sos-berlin.com/jobscheduler/testsuite/config/live" --volume=/some/path/logs:/var/log/sos-berlin.com/jobscheduler/testsuite:Z --name=$IMAGE_NAME $IMAGE_NAME

Explanations
- Using a common network for JobScheduler components allows direct access to resources such as ports within the network.
- The RUN_USER_ID variable is populated with the numeric ID of the account and the group that executes the run command. This value is assigned the --user option to inject the account into the image (replacing the account specified with the USE jobscheduler instruction in the Dockerfile.
- Specify a logs directory to be created that is referenced with the --volume option to expose the log directory of the JobScheduler Master. Consider that the testsuite sub-directory is created from the value of the JobScheduler ID that is specified with the installer response file.
- The --mount option is used in order to map a previously created Docker volume (assumed to be the value of the src= option) e.g. to the JobScheduler Master's live folder. This allows to read and write job related files to this directory that will automatically be picked up by JobScheduler Master.

Space shortcuts

Page tree

Versions Compared

Old Version 6

New Version 7

Key

Scope

Build

Dockerfile

Build

Dockerfile

Build Command

Run