JS7 JobScheduler

Space shortcuts

Page tree

Introduction

  • Instructions to execute jobs are basic to workflow automation.
  • Any number of jobs can be executed in a workflow. Jobs can be reused within the same workflow.

Job

  • A job is the basic unit that is executed by an Agent.
  • Job implementations come in the following flavors:
    • Shell Jobs are executed with the operating system shell, e.g. the Windows Shell or one of the Unix Shells such as /bin/sh, /bin/bash etc.
      • Shell jobs include calls to any shell commands, scripts and executable files.
      • Such jobs behave similarly to individual commands being executed in the command line of the operating system.
      • Shell jobs allow the use of scripting languages such as Perl, Ruby, Python, PowerShell etc.. The use of these languages requires an interpreter to be installed with the OS and that can be executed from the command line.
      • Consider that jobs executed by an Agent cannot successfully request interaction from a console or from a user. If the application, command, script or binary file that is executed tries to bring up a popup window for user interaction then this will fail. This is because in scheduling mode there is no console window to display messages and request that a user makes a confirmation in a popup window.
    • JVM Jobs can be implemented in a number of supported languages that can be operated for a Java Virtual Machine and for which the Agent exposes JS7 - Order Variables and job node arguments. Job implementation languages include:
      • Java®: such jobs are executed in the JVM provided by the Agent.
      • JavaScript®, Ruby, Python etc.: such jobs require use of a GraalVM® Java Virtual Machine that provides the compiler for the relevant scripting language.

Job Relationships

Job Parameterization

Jobs can be parameterized from the following sources:

  • Shell Jobs
  • JVM Jobs
    • Jobs running in a JVM provided by the Agent are forwarded a parameterization object that holds similar information as the environment variables for shell jobs.
  • Both Shell Jobs and JVM jobs can be assigned JS7 - Job Resources to propagate environment variables and arguments.

Job Configuration

Jobs are managed from the JOC Cockpit Configuration->Inventory View.

The tree panel offers the Workflows category to select and to display a workflow with the JS7 - Workflow Editor.

  • Jobs can be added from the toolbar by dragging and dropping the  icon.
  • Selecting a job causes the Job Property Editor to be displayed.


Required Job Properties

The required job properties are available with the Job Properties tab of the Workflow Editor.

  • Name: A job name is required. No spaces are allowed and a number of characters are not allowed. 
  • Label: A label assigns a name to the workflow node that is assigned the job.
    • A job can be re-used in a workflow, however, the job label must be unique.
  • Agent: Assignment to an Agent is required.
  • Return Code: The meaning of the job's return code has to be specified. 
    • Return Codes
      • For shell jobs the OS exit code determines the return code. By default an exit code 0 signals success, any other exit codes signal failure.
      • For JVM jobs the return code is specified by the job implementation.
      • If a number of return codes are used to indicate success or failure then
        • they can be specified from a list of comma separated values, for example 1,2,4,8.
        • they can be specified from a range of return codes using two dots between return codes, for example, 1,2..4,8.
      • Negative return codes can be indicated
        • by specifying negative numbers, for example 1,2,4,-1,-2,-4.
        • by specifying a range of negative numbers, for example 1,-1,-2..-4.
      • Some operating systems use 32-bit unsigned integers as exit codes. Therefore positive numbers in the range between 0x80000000 to 0xffffffff are considered negative return codes, for example the values -1 and 4294967295 are considered the same.
      • JS-2035 - Getting issue details... STATUS
        JOC-1463 - Getting issue details... STATUS
    • On Success / On Error
      • Either successful return codes can be specified or return codes signaling failure.
      • If a job terminates with a return code that is not indicated with the successful return codes or that is stated with return codes signaling failure then the job is considered to have failed. For further details see the section below on Error Handling.
      • A JS7 - Notifications is created for failed jobs.
    • On Warning
      • For return codes that signal warnings the job execution is considered successful and no error handling applies.
      • A JS7 - Notifications is created for jobs raising warnings.
  • Executable Type: Allows the job implementation to be selected as a shell job or a JVM job.
    • For Shell jobs the Job Property Editor displays the Script subtab and Environment Variables sub-tab.
    • For JavaScript jobs the Job Property Editor displays the Script sub-tab.
    • For Java jobs and JITL jobs the Job Property Editor displays the Java sub-tab.
  • Script Sub-tab
    • This tab holds the input field for the job script. 



    • The right upper corner of this tab offers a pencil icon to invoke the script editor. Any number of commands can be added to a job script:



    • The script editor offers resizing, syntax highlighting, capabilities for search & replace, undo & redo etc. Closing the script editor with the Submit button will add the script content to the script sub-tab.
  • Java Sub-tab
    • For jobs assigned the JVM job class the Java sub-tab is displayed to enter the name of the Java class used for this job.
    • The JS7 releases include JS7 - Job Templates that can be specified by their Java class name.

Optional Job Properties

Job Properties Tab

  • Title: A title that will be displayed in the relevant views can be added to a job.
  • For Shell Jobs: Environment Variables Sub-tab
    • Environment variables for job scripts have to be specified by a mapping:



    • The mapping includes specification of a name for the environment variable that has to comply with OS requirements. Consider that Unix operating systems treat environment variables case-sensitive. The GUI provides a list of order variable names for the mapping to environment variables.
    • The value of an environment variable is an expression that can be specified:
      • as a reference to a variable using the $variable syntax
        • consider that leading or trailing spaces are not allowed for references.
      • as a string which is a sequence of characters
        • enclosed with single quotes to allow literal use of any characters, including the $ character. The only character not allowed for a string is a single quote.
        • enclosed with double quotes to allow mixed use of characters and variables. A $ character is considered as a prefix for a variable.
    • The advantage of the above mapping is the fact that:
      • users publish only the environment variables that are required by a job instead of creating an arbitrary number of environment variables from any order variables and job arguments,
      • users limit the risk of overwriting existing shell environment variables, e.g. passing an order variable with the name "PATH" will overwrite a shell environment variable of the same name,
      • the Configuration view offers to search for workflows and jobs that carry specific names and/or values of environment variables.

Job Options Tab

  • Job options include the following configuration items:



  • Parallelism: Should more than one order pass the workflow at any given  time or should parallel child orders execute the same job, then this setting determines the maximum number of tasks that are available for the job. This limits parallelism of tasks to the specified number. If more than the specified number of tasks is requested then orders have to wait until a task becomes free.
  • Timeout / Grace Timeout: Specifies the maximum duration that a job should run.
    • If the timeout is exceeded then the Agent will send a SIGTERM signal to a shell job. This corresponds to executing the kill -15 command in Unix environments. The purpose of the SIGTERM signal is to let the job know that it should terminate and to leave it up to the job implementation to perform cleanup tasks such as disconnecting from a database or removing temporary files.
    • After sending the SIGTERM signal the Agent will wait for the duration of the Grace Timeout to allow the task to complete. If the Grace Timeout is exceeded then the Agent will send a SIGKILL signal that will forcibly terminate the task. This corresponds to executing the kill -9 command in Unix environments.
    • For details see JS7 - FAQ - How does JobScheduler terminate Jobs.
    • The timedOut function can be used in an JS7 - If Instruction to check if a job terminated due to timeout constraints. The function returns a Boolean value true that indicates if the previous job was terminated due to a timeout constraint. If the job did terminate for other reasons then the function returns the false value.

      • JS-2164 - Getting issue details... STATUS

      • Use of the timedout function for a failed job requires use of the If Instruction inside the catch block of a Try/Catch Instruction.
  • Warn on shorter execution: Specifies that a warning is created if job execution time falls below this limit. Jobs with warnings are considered successful and are not subject to error handling.
    • The setting can specify an absolute value in seconds, for example 3s.
      • Alternatively to specifying seconds the duration can be specified using the format hh:mm:ss, for example 01:30:00 for one and a half hours.
      • Alternatively to specifying absolute values a percentage can be specified that is calculated from successful past job executions. A value 30% indicates that a warning is raised if the execution time falls below the average by this ratio.
    • The job will raise a notification when falling below this limit.
  • Warn on longer execution: Specifies that a warning is created if job execution exceeds this limit. Jobs with warnings are considered successful and are not subject to error handling.
    • The setting can specify an absolute value in seconds, for example 3s.
      • Alternatively to specifying seconds the duration can be specified using the format hh:mm:ss, for example 01:30:00 for one and a half hours.
      • Alternatively to specifying absolute values a percentage can be specified that is calculated from successful past job executions. A value 30% indicates that a warning is raised if the execution time exceeds the average by this ratio.
    • The job will raise a notification when exceeding this limit.
  • Compatibility: For users of JobScheduler branch 1.x a compatibility mode is available.
    • For use of environment variables compatibility enables:
      • the automatic creation of environment variables for shell jobs from any available order variables and job arguments.
      • the use of the SCHEDULER_PARAM_ prefix for any environment variable created.
    • For use of job arguments the compatibility mode offers a corresponding tab.
  • Criticality: This option allows a level to be selected that can be used for monitoring purposes to prioritize alerts depending on a job's criticality.
  • Credential Key / Load User Profile: Specifies for jobs executed with Agents for Windows that the job should switch user context, see JS7 - Running Jobs as a different User.
  • Fail on output to stderr / Warn on output to stderr: Checks for output in the stderr channel and raises an error or a warning.
    • Fail: If output in the stderr channel is identified then an error is raised and the job is subject to error handling.
    • Warn: If output in the stderr channel is identified then the job run is considered successful and a warning is created.
    • In both situations a JS7 - Notifications is created.
  • Skip job if no admission time / Show periods: Specifies that the job should be skipped if the order's daily plan date does not match the days specified with the JS7 - Admission Times for Jobs.
    • If the checkbox is no checked then the job will wait for the next admission time.
    • The  Show Periods link allows to manage the job's admission times.
  • Job options for JVM jobs do not offer all above options:

Node Properties Tab

  • Node properties include the list of default arguments that are available from a job node:



  • The use case for node arguments includes using the same job a number of times in a given workflow with different parameter sets.
    • Assume that the job requires the same parameter names to be used.
    • However, for each occurrence of the job in a workflow different parameter values should be used.
  • In this situation node arguments can be introduced that specify variables with the same name but with different values for each occurrence of a job in the workflow.
    • Names of node arguments should not interfere with order variables as otherwise the node argument values will be overwritten by order variable values.
    • For shell jobs node arguments have to be mapped to environment variables similarly to order variables. 
    • For JVM jobs node arguments are available from Java® objects exposed to the job implementation.

Notification Tab

Notifications can be sent in the event of job errors, warnings and success.


Explanation:

  • The Notification tab of the job properties editor allows settings to be specified that overwrite the notification setup.
  • Attribute Mail on:  The attribute specifies one or more events for which notifications are sent, in the event of errors, warnings and success.
  • Attributes Mail To, Mail Cc, Mail Bcc: These attributes specify the recipients of mail notifications. Multiple recipients can be separated by a comma.
  • If job specific notification settings are used, then none of the attributes with the same name specified by the notification setup will be used.
    • Therefore both the Mail on attribute and a Mail To recipient have to be specified in order to change the recipient of notifications specifically for a job.
    • If the Mail on attribute is used but no Mail To recipient is specified then this will suppress mail notification for the job.

Error Handling

The Job Instruction allows job errors to be tracked. The Job Instruction:

  • determines if a job's return code signals success or error. From a number of successful return codes some can be selected to signal warnings that create a JS7 - Notifications.
  • can be configured to check if a job has written any output to the stderr channel and to fail the job if such output exists or to raise warnings.

Error handling for jobs is performed by workflow instructions:

  • A job can be added to a JS7 - Try-Catch Instruction block to execute follow-up jobs from a Catch block in the event of job failure.
  • A job can be added to a JS7 - Retry Instruction block to retry execution of one or more jobs in the event of a specific job failing.
  • A job can be added to a JS7 - If Instruction to check the job's return code or return values and to decide about further processing such as failing, finishing the underlying order or continuing with a different set of jobs in the workflow. This particularly applies in a situation when a job terminates with a non-zero return code which is not considered an error but e.g. an informational warning or in a situation when return codes are used to control workflow execution, e.g. indicating by specific return codes that successor jobs should be skipped. The If Instruction is the perfect means to handle this situation.

Find details from the JS7 - How To - Error Handling section.

How to use the same job in a workflow a number of times

A job can occur a number of times in a workflow by using references to the same job. For example, if a job should executed twice in a workflow then it can be referenced twice in the workflow. This includes to create different Labels for the job as labels assign a name to the workflow node that is assigned the job. A job can be reused using the same job name and different labels with each occurrence of the label being unique within the workflow.

For example, a workflow contains a job with the name job1. Then use the name of the job as job1 for any occurrence of the job and assign different labels to a number of nodes. As in the example, the labels used are job1 and job1-copy. Now any changes applied to job job1 will be applied to both nodes in the workflow as both labels point to the same job.



Resources


  • No labels