How to control substitution in Job Parameters

Scope

• This article describes how the substitution of parameter values works in JobScheduler. It also describes how to use parameters in shell script, API Jobs and pre- and postprocessing scripts.
• JobScheduler ships with a Monitor that performs substitution for Node Parameters:
  • The "classic" implementation makes use of the following configuration monitor:

    • <monitor name="configuration_monitor">
    <script java_class="sos.scheduler.managed.configuration.ConfigurationOrderMonitor" language="java"/>
</monitor>

    • The implementation remains available throughout release 1.11 and will be discontinued with release 1.12.
      • JITL-411 - Getting issue details... STATUS
      • FEATURE AVAILABILITY STARTING FROM RELEASE 1.6 FEATURE AVAILABILITY ENDING WITH RELEASE 1.12
  • The optimized implementation makes use of the following monitor configuration:

    • <monitor name="configuration_monitor">
    <script java_class="com.sos.jitl.jobchainnodeparameter.monitor.JobchainNodeSubstituteMonitor" language="java"/>
</monitor>

    • The implementation becomes available with release 1.11:
    • JITL-276 - Getting issue details... STATUS
    • FEATURE AVAILABILITY STARTING FROM RELEASE 1.11

Substitution of parameter values by the JobScheduler

• The JobScheduler substitutes the values of environment variables in task parameters using ${env_var}, where env_var is the name of an environment variable.
  • The JobScheduler will substitute ${env_var} with an empty value if the environment variable is unknown.

  • For Unix systems case sensitivity is considered for environment variables.
• To prevent substitution in task parameters the environment variable can be quoted with a backslash, i.e. \${env_var}
• JobScheduler does not substitute environment variables in order parameters.

Example

<job  name="job_environment">
    <params >
        <param  name="param_from_environment" value="the value of scheduler_home is ${SCHEDULER_HOME} ${test}"/>
    </params>

    <script  language="shell">
        <![CDATA[
echo %SCHEDULER_PARAM_PARAM_FROM_ENVIRONMENT%
        ]]>
    </script>

    <monitor  name="process0" ordering="0">
        <script  language="java:javascript">
            <![CDATA[
function spooler_process_before(){
    spooler_log.info("param_from_environment=" + spooler_task.params.var("param_from_environment"));
    return true;
}
            ]]>
        </script>
    </monitor>

    <run_time />
</job>

This job will create the following output:

2015-12-07 10:39:32.458+0100 [info]   SCHEDULER-918  state=starting (at=2015-12-07 10:39:32.403+0100)
2015-12-07 10:39:33.984+0100 [info]   param_from_environment=the value of scheduler_home is C:/Program Files/sos-berlin.com/jobscheduler/scheduler_current 
2015-12-07 10:39:33.987+0100 [info]   SCHEDULER-987  Starting process: "C:\WINDOWS\TEMP\\sos-A4CFC1B980.cmd"
2015-12-07 10:39:34.019+0100 [info]   [stdout]  
2015-12-07 10:39:34.019+0100 [info]   [stdout] C:\Users\ur\Documents\sos-berlin.com\jobscheduler\scheduler_current>echo the value of scheduler_home is C:/Program Files/sos-berlin.com/jobscheduler/scheduler_current   
2015-12-07 10:39:34.019+0100 [info]   [stdout] the value of scheduler_home is C:/Program Files/sos-berlin.com/jobscheduler/scheduler_current 

Please note:

• ${test} has been substituted to an empty value as this environment variable has not prevously been set.
• To access the value of the parameter in a shell script, the parameter name has to be prefixed. The default value of the prefix is:
  • SCHEDULER_PARAM_
  • This prefix can be set by the scheduler.variable_name_prefix parameter which is configured in the $scheduler_data/config/scheduler.xml file. JobScheduler has to be restarted after the value of the prefix has been changed.

<?xml version="1.0" encoding="ISO-8859-1" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="scheduler_documentation.xsl" ?>
<spooler>
 
        <config mail_xslt_stylesheet="config/scheduler_mail.xsl" port="4102">
        
                <params>
                        <param name="scheduler.variable_name_prefix" value="SCHEDULER_PARAM_"/>
                        <param name="scheduler.order.keep_order_content_on_reschedule" value="false"/>
                </params>
         ...

Substitution of parameter values by use of the Configuration Monitor

An additional substitution mechanism for jobs running in a job chain is provided by the sos.scheduler.managed.configuration.ConfigurationOrderMonitor preprocessing Java class that is added as a Monitor to jobs.

Scope of substitution

This Java class implements the following steps:

• Before processing
  • it copies the node parameters that are configured in jobchain.config.xml file to the order parameters.
  • if an order parameter exists with the same name then it will be overwritten.
  • substitutes all ${param} variables with:
    • JobScheduler parameters that are configured in the $scheduler_data/config/scheduler.xml file,
    • task parameters that are configured in the currently running job,
    • order parameters that are configured with the order,
    • node parameters that have been copied to the order parameters.
• After processing
  • deletes all node parameters from the order.

Please note:

• If a task parameter value is to be substituted by the Configuration Monitor then the value has to be quoted with a backslash to prevent its substitution by the JobScheduler
  • Example: a value \${param}
• With the parameter setting scheduler.order.keep_order_content_on_reschedule=true the values will be substituted only in the first run.
• When using node parameters then the Configuration Monitor has to be assigned to the job
  • JOE will assign the Configuration Monitor automatically when configuring a node parameter in a job chain
  • Should no substitution be performed then please check if the Configuration Monitor is assigned to the job. Otherwise assign parameters individually by using JOE.

Handling of substitution up to release 1.10

FEATURE AVAILABILITY ENDING WITH RELEASE 1.10

• Performance of parameter subsitution is an issue:
  • 10 node parameter: 4s
  • 20 node parameter: 6s

Handling of substitution starting with release 1.11

FEATURE AVAILABILITY STARTING FROM RELEASE 1.11

• Performance of parameter substitution is restored to a normal range:
  • 20 node parameters <= 2s
• JITL-276 - Getting issue details... STATUS

Substitution of parameter values by JITL Jobs that extend the JobSchedulerJobAdapter base class

Some JITL Jobs extend the JobSchedulerJobAdapter base class and some older jobs extend JobSchedulerJob. For a detailed list see JITL Jobs - Status.

This article describes the substitution mechanisms for JITL Jobs that extend the JobSchedulerJobAdapter base class.

Scope of substitution

Substitution is carried out for:

• task parameters and
• order parameters

The following patterns are recognized as parameters to be substituted:

• %param%
• ${param}

Where param can be

• a task parameter
• an order parameter
• a node parameter
• a JobScheduler parameter
• some special parameter (see below list)

Note: Substitution applies exclusively to the current job run. This means that the values of the parameters will be unchanged after the execution of the job. If you have an order parameter param_x whose value is value is ${param_y} and there are two steps in the job chain, the value of param_x after the execution of the first node will still be value is ${param_y}

Handling of substitution up to release 1.10

FEATURE AVAILABILITY ENDING WITH RELEASE 1.10

• Parameters with the name param  and scheduler_param_param will be handled as being identical.

• Parameters with the name param_plus_any_understrikes and paramplusanyunderstrikes will be handled as being identical.

Handling of substitution starting with release 1.11

FEATURE AVAILABILITY STARTING FROM RELEASE 1.11

• No alias parameters as up to release 1.10 are supported.

• JITL-276 - Getting issue details... STATUS

List of special parameters

References

Change Management References - Agent Usage