Page History

...

• Use Case:
  • Consider the situation where:
    
    • A job chain is to start at a number of parallel job chains
    • Subsequent jobs in started job chains check if jobs from parallel chains completed successfully.
  • Usually a split & sync pattern is used for similar use cases, however, synchronizing includes that both jobs have to be completed.
  • Instead the current use case is about check and control if a job of a parallel chain has completed without forcing both jobs to run synchronously.

Solution

• Download: final_chain_job_chain_historycontrolParallelExecutionOfJobChainsExample.zip
• Extract the archive to a folder in your JobScheduler installation named ./config/live.
• The archive will extract the files to a sub-folder parallel_job_chain_controlcontrolParallelExecutionOfJobChainsExample. 
• Note that you can store the sample files in any folder you like.

Pattern

# order_A [shape="ellipse",label="Order A\n start time: Friday 08:00 ",fillcolor="violet"]
# order_B [shape="ellipse",label="Order B\n start time: Friday 07:00 ",fillcolor="violet"]
# order_Z [shape="ellipse",label="Order Z\n start time: Friday 15:00 ",fillcolor="violet"]
 
job job_chain_1 [label="Job Chain 1\nhas no relevant dependencies",fillcolor="orange"]
job_chain_2 [label="Job Chain 2\nis started by Job Chain 1",fillcolor="violet"]
job_chain_3 [label="Job Chain 3\nis started by Job Chain 1",fillcolor="lightskyblue"]
job_chain_4 [label="Job Chain 4\nis started by Job Chain 1",fillcolor="aquamarine"]
job_chain_5 [label="Job Chain 5\nis started by Job Chain 1",fillcolor="orangered"]
job_chain_6 [label="Job Chain 6\nis started by Job Chain 4",fillcolor="yellow"]


split1 [label="Split 1", fillcolor="orange"]
job1 [label="Job 1", fillcolor="orange"]
job2 [label="Job 2", fillcolor="orange"]
sync1 [label="Sync 1", fillcolor="orange"]
job1_2 [label="Job 1.2", fillcolor="orange"]

job3 [label="Job 3", fillcolor="violet"]
job4 [label="Job 4", fillcolor="violet"]

job5 [label="Job 5", fillcolor="lightskyblue"]
job6 [label="Job 6", fillcolor="lightskyblue"]

job7 [label="Job 7", fillcolor="aquamarine"]
job8 [label="Job 8", fillcolor="aquamarine"]
job9 [label="Job 9", fillcolor="aquamarine"]

job10 [label="Job 10", fillcolor="orangered"]
job11 [label="Job 11", fillcolor="orangered"]
job12 [label="Job 12", fillcolor="orangered"]

job13 [label="Job 13", fillcolor="yellow"]

job_chain_1 -> split1
split1 -> job1 -> sync1
split1 -> job2 -> sync1
sync1 -> job1_2
 
job1 -> job_chain_2 -> job3 -> job4
job2 -> job_chain_3 -> job5 -> job6

job1_2 -> job_chain_4 -> job7 -> job8 -> job9
job1_2 -> job_chain_5 -> job10 -> job11 -> job12


job9 -> job_chain_6 -> job13

# job8job12 -> job4
# job11 -> job4
# job11 -> job6
job4 -> job8job_chain_6 [arrowhead="normal", color="lightslategrey"]

job4 -> job11job8 [arrowhead="normal", color="lightslategrey"]
job6job4 -> job11 [arrowhead="normal", color="lightslategrey"]
job13job6 -> job12job11 [arrowhead="normal", color="lightslategrey"]

Implementation

Components

• The solution contains six job chains:
  • JobChain1 runs Job1 and Job2 in parallel.
    • On successful completion of Job1 then JobChain2 is run
    • On successful completion of Job2 then JobChain3 is run
  • JobChain2 runs Job3 and Job4
  • JobChain3 runs Job5 and Job6
  • JobChain4 runs after successful completion of JobChain1
    • JobChain4 runs Job7, Job8, Job9
    • Job7 runs 1st
    • Job8 runs once Job7 and Job4 have run successfully
  • JobChain5 runs after successful completion of JobChain1
    • JobChain5 runs Job10, Job11 and Job12
    • Job10 runs 1st
    • Job11 runs once Job10, Job4 and Job6 have run successfully
    • Job12 runs once Job11 has completed successfully

Usage

• • JobChain6 runs once Job9 and Job12 have completed successfully

Usage

• The first job split1 sets a number of parameters by use of a pre-processing Monitor for later control of successful completion of jobs. For each job to be controlled a pair of parameters is used:
  
  • The first parameter of the pair is a flag which is used for later checking if the dependent job is successfully completed.
    • The value of this parameter is initialised with "0" and is modified by the dependent jobs to "1".

    • In the above example the following parameters are used:

      • job4_successfully_completed=0 which will be checked by Job8 and Job11
      • job6_successfully_completed=0 which will be checked by Job11
      • job12_successfully_completed=0 which will be checked by Job13
  • The second parameter of the pair describes to which job chains the information about successful completion is broadcasted.
    • The example specifies the job chains for which a suspended order is continued if the current job completes successfully.
    • The current job chain  (here JobChain1) always is the first in the list.
      
    • The job chains can be specified by an absolute path or by a relative path to the current job chain.
      

    • In the above example the following values are used:

      • job4_commits_completion_to="/controlParallelExecutionOfJobChainsExample/JobChain1;JobChain4;JobChain5"

      • job6_commits_completion_to="/controlParallelExecutionOfJobChainsExample/JobChain1;JobChain5"

      • job12_commits_completion_to="/controlParallelExecutionOfJobChainsExample/JobChain1;JobChain4;JobChain6"

• The dependent jobs (here Job4, Job6 and Job12) use a post-processing monitor that implements commands to resume the orders
  
  • in all job chains which are specified by the commits_completion_to parameter
  • where the value of the successfully_completed parameter is set to "1".
• The jobs that wait for dependent jobs make use of a pre-processing monitor that checks the corresponding successfully_completed parameters.
  • These jobs use a task parameter dependent_jobs that specifies a list of dependent jobs.
  • In above example these are
    • Job8 : dependent_jobs="Job4"
    • Job11: dependent_jobs="Job4;Job6"
    • Job13: dependent_jobs="Job12"

• The above described pre- and post-processing monitors include the attached control_parallel_execution_of_job_chains.js JavaScript like this:

  Code Block
  title pre-processing monitor for split1 job collapse true
  <monitor name="set_params_for_check_jobs" ordering="1"> <script language="java:javascript"> <include live_file="control_parallel_execution_of_job_chains.js"/> <![CDATA[ function spooler_process_before() { //Specify the list of job chains for which Job4 commits that it has run successfully //These job chains can specify relative to the current job chain setParams4CheckingJobs( "Job4", ["Chain4", "Chain5"] ); //Specify the list of job chains for which Job6 commits that it has run successfully //These job chains can specify relative to the current job chain setParams4CheckingJobs( "Job6", ["Chain5"] ); //Specify the list of job chains for which Job12 commits that it has run successfully //These job chains can specify relative to the current job chain setParams4CheckingJobs( "Job12", ["Chain4", "Chain6"] ); return true; } ]]> </script> </monitor>

  Code Block
  title post-processing monitor for jobs Job4, Job6 and Job12 collapse true
  <monitor name="commit_successfully_run" ordering="0"> <script language="java:javascript"> <include live_file="control_parallel_execution_of_job_chains.js"/> <![CDATA[ function spooler_process_after(spooler_process_result) { return commit_successful_run(spooler_process_result); } ]]> </script> </monitor>

  Code Block
  title pre-processing monitor for job Job8, Job11 and Job13 collapse true
  <monitor name="check_dependent_jobs" ordering="0"> <script language="java:javascript"> <include live_file="control_parallel_execution_of_job_chains.js"/> <![CDATA[ function spooler_process_before() { return check_dependent_jobs(); } ]]> </script> </monitor>

• Positive Check
  • Add an order to job_chain_A using JOC's Add Order context menu.
    • The order should pass through the job chain without errors.
  • Add an order to job_chain_B using JOC's Add Order context menu.
    • The order should also pass through the job chain without errors.
  • Add an order to job_chain_Z using JOC's Add Order context menu.
    • The order should pass through the job chain without errors as all pre-conditions are met.
• Negative Check
• Modify the job do_something_job_chain_A to include an error, e.g. by adding the command exit 1 as the final line in the job script.
• Add an order to job_chain_A:
  
  • The order should proceed with an error.
• Add an order to job_chain_Z
  • The order should proceed with an error that is visible in the order log. This error will state that the predecessor job did not complete successfully.
  • The order will be set back and will be repeated according to the setback intervals that have been specified for the job check_predecessor_job_chain_Z.
Removing the modification to job do_something_job_chain_A and adding an order to job_chain_A that then runs successfully should result in a successful run of job_chain_Z - either by a new order added to job_chain_Z or by the next execution of the previous order that has been set back.

See also

• How to check the job chain history for previous job chain and order runs
• How to add a condition to check previous jobs before the execution of the current job chain

...