How to Execute Jobs in a Job Chain in Parallel

Scope

This article is about methods that can be used with the JobScheduler for executing Jobs within a Job Chain at the same time.

See the How To Synchronize Job Execution Across Job Chains article for methods about synchronizing the execution of Jobs in different Job Chains.

Using JITL Jobs for Parallel Execution within a Job Chain

Three JITL Jobs are available for configuring the parallel execution of Jobs in a Job Chain:

• JobSchedulerJobChainSplitter
  • The Splitter Job is used to divide processing of a main Job Chain into a number of child Job Chain segments that are then processed in parallel.
  • A Join or Sync Job may be defined later in the Job Chain to bring processing of these child segments back to a single thread.
  • The Splitter Job generates separate Orders for each of the parallel child Job Chain segments. The Splitter Job will then move processing of the main Job Chain to a Join or Sync Job, if such a Job is defined later in the main Job Chain. See the Job Chains article for more information about the patterns that can be executed with the Splitter Job.
• JobSchedulerJoinOrders FEATURE AVAILABILITY STARTING FROM RELEASE 1.11.4
  • The Join Job is used to start or continue processing of a main Job Chain after a predefined number of Orders running on child segments of a Job Chain have been completed.
  • The Join Job is designed for a large number of parallel Orders and is significantly faster for this purpose than the Sync Job described below. Note that the Join Job can only be used within a Job Chain.
  • Note also that the Join Job waits for a predefined number of Orders to be completed - it simply counts orders and does not require specific Orders.
  • The Join Job provides a plug-in replacement for the Sync Job described below for sync/join operations within a Job Chain. This means that users of JobScheduler 1.11.4 or newer wishing to improve performance of the Sync Job can simply exchange the Jobs.
• JobSchedulerSynchronizeJobChains
  • The main function of the Sync Job is to synchronize the execution of Jobs in different Job Chains.
    
    • See the How To Synchronize Job Execution Across Job Chains article for information about using the Sync Job to synchronize the execution of Jobs in different Job Chains.
  • Note that the Sync Job can also be used with JobScheduler versions 1.11.3 and older to synchronize the execution of Jobs within a Job Chain. However, the Sync Job is only suitable for use in situations where performance is not an issue - i.e. situations with only a small number of parallel Orders - typically one Order for each Job Chain.
    • See the  How to split and sync jobs in a job chain FEATURE AVAILABILITY ENDING WITH RELEASE 1.11.4 article for an example showing the use of the Sync job within a Job Chain.
    • The Join Job described above provides a plug-in replacement for the Sync Job for sync/join operations within a Job Chain. This means that users of JobScheduler 1.11.4 or newer wishing to improve the performance of the Sync Job can simply exchange the Jobs. 
      

Examples for Static configuration

For the static configuration of the parallel execution of different Jobs:

• Using the JITL split and join Jobs FEATURE AVAILABILITY STARTING FROM RELEASE 1.11.4 :
  • This pattern is used for parallelization within a single Job Chain.
  • See the following articles for examples and downloads using the split and join Jobs:
    
    • How to split and join jobs in a job chain
    • Example showing how to run parallel jobs repeatedly in a job chain
    • How to Nest Parallel Executing Jobs in a Job Chain  

Examples for Dynamic configuration

Any number of instances of a Job can be dynamically generated and executed in parallel in a Job Chain. This is done using a custom Generate Job and the JITL Join Job.

The following articles describe examples and provide download files demonstrating the dynamic configuration of parallel job instances:

• How to split and join a dynamic number of job instances in a job chain
• How to split and join a dynamic number of job instances for execution with Agents

See also

• For more information about configuring parallel workflow dependencies see also:
  • Use Cases - Dependency Patterns
  • How to Serialize the Execution of Jobs and Orders
  • How to Configure Backward Dependencies in Jobs and Job Chains

Further Resources

Frequently Asked Questions

Examples in detail

Troubleshooting