Introduction
JobScheduler can reference job chain node parameters - i.e. parameters that are valid for a specific node in a job chain.
Job chain node parameters can be defined for orders and for the job chain:
- Job chain node parameters for orders are defined in a similar manner to conventional order parameters - but using a special syntax.
- Job chain node parameters for the job chain itself are defined in a job chain parameter file. The jobs in the job chain use a special monitor to read the job chain parameter file.
Job chain node parameters for both orders and for the job chain can reference other (global) parameters.
Usage Notes
In contrast with job parameters, job chain node parameters are independent of the the job specified for a job chain. Their use provides a greater degree of flexibility in job chain configuration than can be achieved with job or order parameters. Order parameters are - logically - tied to an order and have to be redefined each time an order is created. Similarly, job parameters are tied to a job. This means that if a job is to be used at multiple steps in a job chain or in different job chains, multiple instances of that job will have do be defined if different parameters are to be set each time the job is to be run, even though it is the same job implementation is being used each time.
Usage example
As an illustration of the flexibility brought with job chain node parameters, consider the following simple example:
For a job chain made up of three nodes:
- transfer a file by FTP to host a,
- transfer a file by FTP to host b,
- delete file.
the use of job chain node parameters would allow the same job to be used for both nodes 1 and 2 but with different parameters each time.
Article Scope
Two example job chains are described in this article:
- The first uses job chain node parameters defined for an order.
- The second example uses job chain node parameters defined for the job chain itself and also demonstrates referencing of global parameters.
Example showing job chain node parameters defined for an order
SECTION IS WORK IN PROGRESS
REDO SCREENSHOTS
This example shows how parameters can be defined in the order configuration and assigned to a specific job chain nodes.
Example Download
- You can download a sample from here.
Parameter definition
- The syntax for order parameters that are specific to a job chain node is <node>/<name>.
(This is in contrast to the standard order parameter syntax of <name>.)
A parameter named order_param to be made available to a job chain node with a state 100 is specified a parameter with the name 100/order_param. The job in the node 100 then has access to the parameter order_param. This parameter will not be available at other nodes in the job chain.
- Substitution will be done for the order node parameter.
- A job chain node parameter with the same name will be overwritten by the order node parameter.
- Order node parameters can be used if job chain node parameters have not been defined and a job chain configuration file has not been created.
Here is the job chain with two nodes:
The order defines the parameters for the job chain node.
Use with API Jobs
At the time of writing (current releases are 1.8.4, 1.9.8 and 1.10.2) the above example works with shell jobs and not with API jobs - which includes many of the JITL jobs.
A fix for this issue is planned - for more information see
-
JS-1573Getting issue details...
STATUS
Workaround for API & JITL Jobs
To activate the order node function please add the configuration monitor to the pre-processing element of the job at the job node.
Example for assigning a configuration monitor
<job stop_on_error="no" order="yes" name="job2"> .... <monitor name="configuration_monitor" ordering="0"> <script language="java" java_class_path="" java_class="sos.scheduler.managed.configuration.ConfigurationOrderMonitor"/> </monitor> /job>
In the above example the same job is used at both nodes of the sample job chain. The configuration monitor is defined in JOE as shown in the next screenshot:
Example showing job chain node parameters defined for the job chain
Downloads
- job_chain_node_params.zip - configuration files
Instructions
- Unzip all files into the
./config
folder of your JobScheduler installation. Open
config/scheduler.xml
and add the following lines before the<security>
element:<params> <param name="global_configuration_params" value="config/global_scheduler_vars.xml"/> </params>
- Restart the JobScheduler
- Open the JobScheduler Operating Center, JOC, in your browser using http://scheduler_host:scheduler_port
- Open the JOB CHAINS tab and enable "Show orders".
- Find the job chain
samples/configuration/chain_a
. - Find the order
my_order
, open the order menu and choose "Show log".
A window will open showing the log of the order. As the order isn't running yet, the window will show nothing more than information about the configuration file, the next start time of this order and its initial state. Leave the window open and move it to a place where you can see it along with the main browser window.
- Start the order by choosing "Start order now" from the Order menu.
- Watch the log grow in the other window
The order will run through both steps of the job chain. In each step the order parameters will be logged by the job. The parameters param1 - param4 should be shown in the log with the following values:
State:first param1=10 param2=abc param3=foo param4=abc---def
and
State:second param1=10 param2=xyz param4=xyz---def
The next section explains how these resulting parameters were configured.
How it works
The job chain chain_a
consists of two steps, "first" and "second". Both steps use the same job readparam
. The readparam
job is a small JavaScript job which reads all order parameters and logs them on info
-level.
The Java monitor implementation
The sos.scheduler.managed.configuration.ConfigurationOrderMonitor
is configured as a monitor for the job. This is a special pre/post-execution script which is included in the JobScheduler distribution. It can be used for all jobs.
In the JobScheduler Object Editor, JOE, the monitor implementation is called under JobScheduler Objects.Jobs.Jobname.Pre-/Post-Processing as a Java configuration_monitor. The monitor reads special job chain parameter files which set parameters for a job chain and are able to configure each node of a job chain individually. These parameter files are named by default according to the pattern myJobChain.config.xml
.
The screenshot below shows the monitor configuration in JOE.
The XML code generated by JOE for the readparam.job is:
The following screen-shot shows the job chain node parameters for the example:
The next screen-shot shows the Details for JobChain form, which can be used to edit the param1 and param2 parameters. This form is opened with the Parameter button shown in the previous screenshot (top right):
Note that the node parameters - param3 and param4 are set and edited using JOE's XML editor, which is accessed using the OpenXML button shown in the screenshot.
(See also the Example in Detail section below.)
The example in detail:
When using hot folders, the configuration monitor looks for the job chain parameter file in the current hot folder and expects the name job_chain_name.config.xml. So in this case, the name of the job chain parameter file is chain_a.config.xml
.
The full chain_a.config.xml
file is listed in the next block. This file is opened and edited by clicking on the Open XML button in the job chain node parameter form (the Details for JobChain form) shown in the screenshot above.
<?xml version="1.0" encoding="ISO-8859-1"?> <?xml-stylesheet type="text/xsl" href="scheduler_configuration_documentation.xsl"?> <settings> <job_chain name="chain_a"> <order> <params> <param name="param1" value="10"/> <param name="param2" value="abc"/> </params> <process state="first"> <params> <!-- create parameter only for this state --> <param name="param3" value="foo"/> <!-- use global and chain parameter in value of new parameter --> <param name="param4" value="${param2}---${global_test_var}"/> </params> </process> <process state="second"> <params> <!-- overwrite param2 parameter --> <param name="param2" value="xyz"/> <!-- use global and chain parameter in value of new parameter --> <param name="param4" value="${param2}---${global_test_var}"/> </params> </process> </order> </job_chain> </settings>
The chain_a.config.xml
file has three <params>
elements:
- The first one is a child of the
<order>
element. It configures parameters for all steps of the job chain. That's why the log showsparam1
as10
in both steps.
Parameters here were set in the job chain node parameter form (the Details for JobChain form) shown in the screenshot above. - The second
<params>
element is a child of<process state="first">
. It configures parameters for the first node (called "first") of the job chain. It sets two additional parametersparam3
andparam4
.param4
uses a special${paramname}
-syntax to reference other parameters. It referencesparam2
which was set in the first params element. As you see in the log, this part is later resolved toabc
. - The third is the
global_test_var
parameter that is a variable defined inconfig/scheduler_global_vars.xml
.
By configuring the global scheduler variableglobal_configuration_params
inscheduler.xml
to point to that file (scheduler_global_vars.xml
), we tell the configuration monitor to always read that file and process it's parameters. Inconfig/scheduler_global_vars.xml
the value forglobal_test_var
is set todef
, so the reference inparam4
is resolved to that value.
Note that:
- The second
<params>
element is a child of<process state="second">
. It configures parameters for the second node (called "second") of the job chain. It overwrites theparam2
parameter and sets it toxyz
.param3
is not set at all, thus it doesn't appear in the log for statesecond
. param4
is defined in the same way as in statefirst
, but this time the resulting value is different becauseparam2
has been overwritten for this state.
Scope / Application
In job chain parameter files the ${PARAMNAME}
-syntax can be used to reference:
- order parameters
- order parameters defined in the job chain parameter file
- global JobScheduler variables
- environment variables
Further References
Job Chain and Job Chain Node Parameters