Product Knowledge Base

Space shortcuts

Page tree

Versions Compared

Old Version 12

changes.mady.by.user Andreas Püschel

Saved on

compared with

New Version Current

changes.mady.by.user Alan Amos

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Minor changes to 'Parallel Processing'

Table of Contents

...

Use of Job Chains

Excerpt
  • Job chains can be seen as an assembly line on which multiple job nodes are passed, with each job making up one step in the processing of a chain.
  • Job chains allow the reuse of jobs in different job chains with different parameter sets.
  • Job chains allow dependencies to be mapped by, for example, causing processing to continue at different job nodes according to result returned from the processing of a job node.
  • Job chains enable complex workflow patterns such as Split & Sync: see Example showing the synchronization of multiple job chains
  • Nested job chains can be used to control and parallelize the execution of multiple job chains.

Read more on the features of Job Chain Objects in the order and job chain documentation.

Job Chain Patterns

Forward Dependency Patterns

Job chains represent a sequence of jobs that implements forward dependency.

Job chain with single end nodes for success and failure

  • The jobs within the job chain will result in a unique successful end state or otherwise will use the same end state for failure.
  • Each job would immediately cause the job chain to be completed in case of error.

 

Flowchart
Job1 [label="Job 1",fillcolor="lightskyblue"]
Job1_successful [shape=diamond,label="execution successful?",fillcolor="white"]
Job2 [label="Job 2",fillcolor="lightskyblue"]
Job2
Graphviz
flowchart {
Job1 [Label="Job 1"]
Job1_successful [shape=diamond,Labellabel="execution successful?",fillcolor="white"]
Job1_success [Label="yesJob3 [label="Job 3",fillcolor="lightskyblue"]
Job3_successful [shape=diamond,label="execution successful?",fillcolor="white"]
EndStateSuccess [label="End state: success",fillcolor="palegreen"]
EndStateError [label="End state: error",fillcolor="lightcoral"]
 
Job1 -> Job1_successful
Job1_successful -> Job2 [label=" yes "]
Job1_errorsuccessful -> EndStateError [Labellabel=" no "]
Job2 -> Job2 [Label_successful
Job2_successful -> Job3 [label=" yes "]
Job2_successful -> EndStateError [label=" no "]
Job3 -> Job3_successful
Job3_successful -> EndStateSuccess [label=" yes "]
Job3_successful -> EndStateError [label=" no "]

Job chain with multiple end nodes for success and failure

  • The jobs will use a common end state in case of success and individual end states for failure
  • Each job would immediately cause the job chain to be completed in case of error.

 

Flowchart
Job1 [label="Job 1",fillcolor="lightskyblue"]
Job1="Job 2"]
Job2_successful [shape=diamond,Labellabel="execution successful?",fillcolor="white"]
Job2_success [Labellabel="Job 2",fillcolor="yeslightskyblue"]
Job2_errorsuccessful [Labelshape=diamond,label="execution successful?",fillcolor="nowhite"]
Job3 [Labellabel="Job 3",fillcolor="lightskyblue"]
Job3_successful [shape=diamond,Labellabel="execution successful?",fillcolor="white"]
Job3_successEndStateSuccess [Labellabel="End state: success",fillcolor="yespalegreen"]
Job3_error [Label="noEndStateError1 [label="End state: error 1",fillcolor="lightcoral"]
EndStateSuccessEndStateError2 [label="End state: success error 2",fillcolor="lightcoral"]
EndStateErrorEndStateError3 [label="End state: error 3",fillcolor="lightcoral"]
 
Job1 -> Job1_successful
Job1_successful -> Job2 Job1_success[label=" yes "]
Job1_successful -> EndStateError1 [label="yes no "]
Job2 -> Job2_successful
Job1Job2_successful -> Job1_errorJob3 [label="yes"]
Job2_successful -> EndStateError2 [label=" no "]
Job3 -> Job3_successful
Job3_successful -> EndStateSuccess [label=" yes "]
Job3_successful -> EndStateError3 [label=" no "]

Job chain with individual successor nodes

  • The jobs are organized in a way that in case of error a job node is skipped an processing is continued with a later job node.
  • This sample represents the usage of a job for error handling of previous jobs.

 

Flowchart
Job1 [label="Job 1",fillcolor="lightskyblue"]
Job1_successful [shape=diamond,label="execution successful?",fillcolor="white"]
Job2 [label="Job 2",fillcolor="lightskyblue"]
Job2_successful [shape=diamond,label="execution successful?",fillcolor="white"]
Job3 [label="Job 3",fillcolor="lightskyblue"]
Job3_successful [shape=diamond,label="execution successful?",fillcolor="white"]
EndStateSuccess [label="End state: success",fillcolor="palegreen"]
EndStateError
Job2 [label="End state: error",fillcolor="lightcoral"]
 
Job1 -> Job2Job1_successful
Job2Job1_successful -> Job2_success [label=" yes "]
Job1_successful -> Job3 [label=" no \n continue with Job 3 "]
Job2 -> Job2_successful
Job2_successful -> Job3 [label=" yes "]
Job2_errorsuccessful -> EndStateError [label=" no "]
Job3 -> Job3_successful
Job3_successful -> EndStateSuccess Job3_success -> EndStateSuccess
Job3_successful -> Job3_error -> EndStateError
}

...

[label=" yes "]
Job3_successful -> EndStateError [label=" no "]

Nested Job Chain Pattern

  • Two Job Chains B and C are integrated into a Job Chain A for sequential processing.
  • Job Chain A starts Job Chain B and waits for completion. 
  • Once Job Chain B has terminated then Job Chain A is continued and starts Job Chain C.
  • After completion of Job Chain C the processing of Job Chain A is continued.

 

Flowchart
JobA1 [label="Job A1",fillcolor="lightskyblue"]
JobA2 [label="Job A2",fillcolor="lightskyblue"]
JobA3 [label="Job A3",fillcolor="lightskyblue"]
JobB1 [label="Job B1",fillcolor="orange"]
JobB2 [label="Job B2",fillcolor="orange"]
JobB3 [label="Job B3",fillcolor="orange"]
JobC1 [label="Job C1",fillcolor="lavender"]
JobC2 [label="Job C2",fillcolor="lavender"]
JobC3 [label="Job C3",fillcolor="lavender"]
 
JobA1 -> JobB1 [label=" start job chain B "]
JobA1 -> JobA2 [label=" skip job chain B "]

JobB1 -> JobB2 -> JobB3
JobB3 -> JobA2 [label=" continue job chain A "] 

JobA2 -> JobC1 [label=" start job chain C "]
JobA2 -> JobA3 [label=" skip job chain C "]
 
JobC1 -> JobC2 -> JobC3
JobC3 -> JobA3 [label=" continue job chain A "]

Parallel Processing Job Chain Pattern

'Split & Sync' or 'Split & Join'

  • Two Job Chains B and C are executed in parallel and are controlled by the initiating Job Chain A.
  • The initiating Job Chain A splits execution into two Job Chains B and C.
  • The execution of Job Chain B and C is synchronized and Job Chain A is continued.

 

Flowchart
JobA1 [label="Job A1",fillcolor="lightskyblue"]
JobA2 [label="Job A2",fillcolor="lightskyblue"]
JobB1 [label="Job B1",fillcolor="orange"]
JobB2 [label="Job B2",fillcolor="orange"]
JobB3 [label="Job B3",fillcolor="orange"]
JobC1 [label="Job C1",fillcolor="lavender"]
JobC2 [label="Job C2",fillcolor="lavender"]
JobC3 [label="Job C3",fillcolor="lavender"]
 
JobA1 -> JobB1 [label=" split processing for job chain B "]
JobA1 -> JobC1 [label=" split processing for job chain C "]

JobB1 -> JobB2 -> JobB3
JobB3 -> JobA2 [label=" merge processing for job chain A "] 

JobC1 -> JobC2 -> JobC3
JobC3 -> JobA2 [label=" merge processing for job chain A "]

Management of Job Chains

Job chains are created either using the JobScheduler Object Editor (JOE), or using API methods in individual other programs:

  • Firstly you create a job chain and the jobs that are part of this chain. Every position in a job chain is assigned a state and a job.
  • Then you create an order, i.e. a simple token that An order is then created for the job chain. This is a simple token that is assigned a state in a job chain (normally the beginning of a job chain). Optionally an order has a payload consisting of parameters that are processed by the jobs. When an order is added to the job chain, it is enqueued by JobScheduler according to the state of the order. The job assigned to this position then carries out the order. Additionally, each position in a job chain has a successor state and an error state. JobScheduler changes the state of an order after each job in the job chain has been processed. If the job step was successful, JobScheduler sets the successor state; otherwise it sets the error state.
  • The order then moves to another position in the job chain as defined by the new state.

Jobs within a job chain can be executed in multiple processes to handle parallel processing of orders.