Page History

...

• Number of jobs, job chains and orders
  • This is about the number of job-related objects that are available in the system, independently from the fact that they are running or not.
  • JobScheduler has to track events for jobs, e.g. when to start and to stop jobs. Therefore a high number of job related objects creates some performance impact. Common scenarios used in enterprise level environments include up to 20000 jobs and 10000 job chains in a single JobScheduler instance.
• Number of job nodes
  • This is about the number of jobs that are used in job nodes for job chains. Jobs can be re-used for any number of job chains. 
    • You could operate e.g. 1000 job chains with each using 5 job nodes with individual jobs which results in a total of 5000 individual jobs.
    • You could operate e.g. 100 individual jobs that are used in 1000 job chains each using a individual sequence of 5 out the 100 jobs.
  • The length of a job chain, i.e. the number of job nodes, is important:
    • In average scenarios job chains with up to 30 job nodes are used.
    • You can operate a single job chain with a medium number of e.g. 4000 job nodes. In fact this will have a slight effect on performance as JobScheduler has to check predecessor and successor nodes for each job node.

...

JobScheduler is designed for parallelism as the most effective means to improve performance.

...

Order Queue

• When using the order queue setting <job_chain max_orders="number"> you will restrict the number of parallel orders in a job chain to number. Any additional orders will be queued.
• By default this attribute is not effective which allows an unlimited number of parallel orders in a job chain.
• When using a value <job_chain max_orders="1"> then this will result in strict serializaton of orders. The next order will enter the job chain only after the first order has completed the final node of a job chain.
• Consider The recommendation is not to use the order queue setting. Better performance is achieved by enabling orders to be processed in parallel. Consider use of this setting if your business requirements force orders to be serialized . Better performance is achieved by enabling orders to be processed in parallel.

Parallel Tasks

• tasks=

...

• which is a somewhat contradicting requirement concerning performance.

Parallel Tasks

By default JobScheduler executes a job in a single task. Consider to enable parallel tasks to be started for the following scenarios:

• Parallelized Orders
  • A job chain can hold a high number of orders. This is limited by the above-mentioned max_orders attribute of the job chain order queue.
  • Consider to allow e.g. 5 tasks to be started in parallel for a job in a job node. This will enable JobScheduler to run 5 orders in parallel for the respective job node.
  • By default only 1 task is started for a job node which results in the fact that orders waiting for that job node will be queued for later execution.
• Re-used jobs
  • If individual jobs are re-used in multiple job nodes of different job chains then multiple tasks can be started in parallel for instances of these job nodes.
  • By default only 1 task is started and any additional orders in other job chains would have to wait for this task to become free.

The syntax to enable multiple tasks for a job is:

• <job tasks="number"/> where number is an integer between 1 and the maximum number of tasks.
• The effective number of tasks that are started is restricted by process classes, see below.

Consider use of resources for parallel tasks:

• For long-running jobs it might be helpful to have a higher number of tasks.
• For short-running jobs it might be more efficient to have a lower number of tasks as the orders would anyway pass the job node quickly and would not have to wait. However, if your system provides sufficient resources then high parallelism of tasks is the recommended measure.

Process Classes

Process classes restrict the number of parallel tasks for jobs.

• A default process class is active with the JobScheduler installation that enforces a maximum of 30 parallel tasks:

  • The configuration item is available in ./config/scheduler.xml

    Code Block
    language xml title Default process class in configuration file scheduler.xml collapse true
    <?xml version="1.0" encoding="ISO-8859-1" standalone="no"?> <?xml-stylesheet type="text/xsl" href="scheduler_documentation.xsl" ?> <spooler> <config mail_xslt_stylesheet="config/scheduler_mail.xsl" port="4444"> <process_classes> <process_class max_processes="30"/> </process_classes> </config> </spooler>

  • Modify this value to a higher number of tasks for jobs that are not assigned an individual process class.
  • Modifications to ./config/scheduler.xml become active after a JobScheduler restart.
• Consider use of individual process classes.
  • You can create any number of process classes, e.g. by use of the JOE editor, and assign them individually to your jobs.
  • The syntax for assignment is: <job process_class="name"/> where name is the name or path of your process class.
  • This allows to manage groups of jobs individually that are guaranteed the specified number of tasks.
  • Process limits from individual process classes are independent from the limit of the above-mentioned default process class.
• Process class limit
  • With the maximum number of processes of a process class being exceeded the jobs will have to wait until a process becomes free.
  • Without the max_processes attribute in <process_class max_processes="number"/> being used no limit applies to the number of parallel tasks. This is usually a bad idea as any system has some resource limit for execution of parallel tasks.
  • The same behavior applies to jobs that are executed with the JobScheduler Master and with Agents. Jobs that are shared for execution with multiple Agents respect the process class limit with the tasks are technically executed on the Agent server. For use with Agents the max_processes attribute should therefore consider the resources of the server that the Agent is operated for.

Pre-loading of tasks

• min_tasks=
• idle_timeout=

...