...
- Use Case
Create Job Stream #1
- 1. Batch Job #1 runs only at 10pm. If the job fails to run successfully, is should be rerun up to a maximum 2 times.without pre-condition.
2. Batch Job #2 will run IF AND ONLY IF Job #1 completes successfully by 11pm and only if today is Wednesday.
3. Batch Jobs #3 and #4 will run only if Job #2 ran successfully before 11.30pm and only if it’s before the 10th of the month (meaning the run date is < 10th OCT-2015 for example).
4. Batch Job #5 will run after midnight and if Jobs #3 and #4 completed running within 45 minutes of their individual start times.
5. Batch Job #6 will run only on Thursdays and Fridays of the week, and IF AND ONLY IF it’s after the 5th of the current month AND before 15th of the month (meaning the run date is > 5th OCT-2015 && < 15th OCT-2015 for example), #3, #4, #5, #6 will run if the respective predecessor Batch Job completed successfully.
Create Job Stream #2:
63. Run Batch Job #7 all by itself in a separate Job Stream #2 during the 1st week of the 3rd month of every quarter in a calendar yearruns without pre-condition.
Combine the 2 Job Streams into a new Job Stream #3:
74. Batch Job #8 and Batch Job #9 (in Job Stream #3) will be automatically started and running at the same time IF AND ONLY IF Job #6 from Job Stream #1 and Job #7 from Job Stream #2 ran successfully.
Create Job Stream # 4:
85. Run Batch Job #10 (in Job Stream #4) runs only after the successful completion of Job #8 and #9 in Job Stream #3.
- Feature Availability
Display feature availability StartingFromRelease 1.10.2 Jira server SOS JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 6dc67751-9d67-34cd-985b-194a8cdc9602 key JITL-228
...
Flowchart |
---|
order1 [shape="ellipse",label="Order start_job_stream1\n start time: 10:00 every day ",fillcolor="violet"] order2 [shape="ellipse",label="Order start_job_stream2\n start time: 12:00 every day ",fillcolor="violet"] job_chain1 [label="Job Stream 1\nhas no relevant dependencies",fillcolor="orange"] job_chain2 [label="Job Stream 2\nhas no relevant dependencies",fillcolor="orange"] job_chain3 [label="Job Stream 3\ndepends on the execution of\nJob Stream 1 and Job Stream 2",fillcolor="orange"] job_chain4 [label="Job Stream 4\ndepends on the execution of\nJob Stream 3",fillcolor="orange"] job1 [label="Job 1\nConditions: none", fillcolor="lightskyblue"] job2 [label="Job 2\nConditions: Job #1 completed successfully", fillcolor="lightskyblue"] # job2 -> job1 job3 [label="Job 3\nConditions: Job #2 completed successfully", fillcolor="lightskyblue"] # job3 -> job2 job4 [label="Job 4\nConditions: Job #3 completed successfully", fillcolor="lightskyblue"] # job4 -> job3 job5 [label="Job 5\nConditions: Job #4 completed successfully", fillcolor="lightskyblue"] # job5 -> job4 job6 [label="Job 6\nConditions: Job #5 completed successfully", fillcolor="lightskyblue"] job7 [label="Job 7\nConditions: none", fillcolor="lightskyblue"] job_splitter [label="Job Splitter\nConditions: Job Stream 1 and Job Stream 2\ncompleted successfully", fillcolor="lightskyblue"] # job_splitter -> job_chain1 # job_splitter -> job_chain2 job8 [label="Job 8\nConditions: none", fillcolor="lightskyblue"] job9 [label="Job 9\nConditions: none", fillcolor="lightskyblue"] job_sync [label="Job Sync\nConditions: Splitted orders for Job 8 and Job 9\ncompleted successfully", fillcolor="lightskyblue"] # job_sync -> job8 # job_sync -> job9 job_dispatcher [label="Job Dispatcher\nConditions: Job Stream 1 and Job Stream 2\ncompleted Sync completed successfully", fillcolor="lightskyblue"] job10 [label="Job 10\nConditions: Job Stream 3 completed successfully", fillcolor="lightskyblue"] # job10 -> job_chain3 order1 -> job_chain1 -> job1 -> job2 -> job3 -> job4 -> job5 -> job6 -> job_chain3 order2 -> job_chain2 -> job7 -> job_chain3 job_chain3 -> job_splitter job_splitter -> job8 -> job_sync job_splitter -> job9 -> job_sync job_sync -> job_dispatcher -> job_chain4 job_chain4 -> job10 |
...
- Hints
- The solution makes use of Scripting with Pre-/Post-Processing Monitors and date calculation.
- The term completed successfully as subsequently used includes that
- if not specified otherwise the last run of a job took place within the current period that is the current day (24pm to 0am)chain assigned an order an end state that is not used as an error state in the job chain..
- the last run of a job chain was successful (not any previous runs within the same period).
- The conditions assigned to jobs job chains are handled in a way that the successor job execution is skipped and processing is continued with the next job. The default behavior of JobScheduler would be to stop processing and wait for a condition to chain is executed if the conditoins can be met.
- Except for
job8
andjob9
all jobs implement a simple "echo hello world
" command.
- The solution contains four job streams with dependencies:
job_stream1
andjob_stream2
represent two job chains that have to be executed successfully to makejob_stream3
proceed.job_stream3
splits the processing into parallel jobs (job8
,job9
).job_stream4
is the final job chain that checks ifjob8
andjob9
fromjob_stream3
did complete successfully.
- The solution implements the following job dependencies:
job_stream1
job1
- runs without dependencies.
- is configured for setback, i.e. will repeat iself in case of error up to two times.
job2
- implements an individual Pre-Processing Monitor to check if the current day is Wednesday and if the predecessor job
job1
completed successfully before 11:00pm. - will be skipped and the order moved to the next job node if the check fails.
- implements an individual Pre-Processing Monitor to check if the current day is Wednesday and if the predecessor job
job3
- implements an individual Pre-Processing Monitor to check if the current date is before the 10th of month and if the predecessor job
job2
completed successfully before 11.30pm. - will be skipped and the order moved to the next job node if the check fails.
- implements an individual Pre-Processing Monitor to check if the current date is before the 10th of month and if the predecessor job
job4
- implements the same check as
job3
.
- implements the same check as
job5
- implements an individual Pre-Processing Monitor to check if two predecessor jobs (
job3
,job4
) completed successfully and consumed less than 45 min. execution time. - will be skipped and the order moved to the next job node if the check fails.
- implements an individual Pre-Processing Monitor to check if two predecessor jobs (
job2, job3, job4, job5, job6
- run if their predecessor job completed successfully
job6
job6
- implements an individual Pre-Processing Monitor to check if the current day is a Thursday or Friday and if the date is after the 5th and before the 15th of month.
- will be skipped and the order moved to the
success
end node if the check fails. - makes use of the Named Post-Processing Monitor
start_job_stream
(by a<monitor.use>
element):- the Post-Processing Monitor will not start job chains if the job completes with a return code > 0 signalling an error or if one of the checks fails.
- this Monitor checks if a number of jobs completed successfully (
job6
,job7
job_stream2
):- the job chain names are specified by use of the
check_job_jobschains
job parameter that is assigned a semicolon separated list of job chain names. - job chain names can be specified with an absolute path (starting from the
live
folder) or with a path relative to the directory of this job.the current job is exempted from checks as it is running, i.e. not completed, and resulted in a successful return code 0.
- the job chain names are specified by use of the
- if the checks are successful then this Monitor will start a number of job chains (
job_stream3
):- the job chain names are specified by use of the
start_job_chains
job parameter that is assigned a semicolon separated list of job chain names. - job chain names can be specified with an absolute path (starting from the
live
folder) or with a path relative to the directory of this job.
- the job chain names are specified by use of the
job_stream2
job7
- implements an individual Pre-Processing Monitor to check if the current day is within the first week of a month.
- will be skipped and the order moved to the
success
end node if the check fails. - makes use of the Named Post-Processing Monitor
start_job_stream
(by a<monitor.use>
element) in the same way asjob6
. - the same job parameterization applies as for
job6
with the exeception that the check is performed forjob_stream1
.
job_stream3
splitter
job8
- makes use of the Named Post-Processing Monitor
start_job_stream
(by a<monitor.use>
element):- the Monitor checks if a number of jobs completed successfully (
job8
,job9
). - if the checks are successful then this Monitor will start a number of job chains (
job_stream4
).
- the Monitor checks if a number of jobs completed successfully (
- implements a
ping
command to the localhost that is executed once. - in the same way as
job6
. - the same job parameterization applies as for
job6
with the exeception that the check is performed forjob_stream1
andjob_stream2
. - splits processing of the current order into parallel execution of jobs
job8
andjob9
.
- makes use of the Named Post-Processing Monitor
job8, job9
- run without dependencies.
- run in parallel.
- sync
- waits for
job8
andjob9
to complete successfully.
- waits for
dispatcher
job9
- makes use of the Named Post-Processing Monitor
start_job_stream
(by a<monitor.use>
element) in the same way asjob8
job6
. - the same job parameterization applies as for
job8.
implements aping
command to the localhost that is executed ten times. This is intended to have different execution times forjob8
andjob9
.job6
with the exception that no check is performed but
- makes use of the Named Post-Processing Monitor
job_stream4
- job10
- makes use of the Named Pre-Processing Monitor
check_job_chain_history
by a<monotor.use>
element) in- this Monitor checks if a number of jobs job chains completed successfully (
job9
,job9
job_stream3
):- the job chain names are specified by use of the
check_jobsjob_chains
job parameter that is assigned a semicolon separated list of job chain names. - job chain names can be specified with an absolute path (starting from the
live
folder) or with a path relative to the directory of this job.the current job is exempted from checks as it is running during the check, i.e. not completed, and resulted in a successful return code 0.
- the job chain names are specified by use of the
- this Monitor checks if a number of jobs job chains completed successfully (
- will be skipped and the order moved to the next job node fail if the check fails.
- makes use of the Named Pre-Processing Monitor
- job10
isJobRunEnabled = ( Date.today().getDate() < 10 );
*/
- Please remove comments to enable the respective checks.
Usage
- Positive Check
- Start the order
start_job_stream1
using JOC'sStart Order now
context menu. Alernatively add a new order.- The order should pass through the job chain without errors.
- Start the order
start_job_stream2
using JOC'sStart Order now
context menu. Alternatively add a new order.- The order should also pass through the job chain without errors.
- As the job chains
job_stream1
andjob_stream2
completed successfully the job chainjob_stream3
will be kicked off byjob7
ofjob_stream2
. - Job chain
job_stream3
runsjob8
andjob9
in parallel. job9
injob_stream3
will take longer thanjob8
to complete and will start job chainjob_stream4
.- Job chain
job_stream4
withjob10
will complete successfully.
- Start the order
- Negative Check
- Modify the job
job1
in job chainjob_stream1
to include an error, e.g. by adding the commandexit 1
as the final line in the job script. - Start the order
start_job_stream1
using JOC'sStart Order now
context menu. Alernatively add a new order.- The order should complete with an error having passed
job1
, no further jobs are executed.
- The order should complete with an error having passed
- Start the order
start_job_stream2
using JOC'sStart Order now
context menu. Alernatively add a new order.job7
will be executed succesfully, however,job_stream3
will not be started due to the fact thatjob_stream1
completed with error
job_stream1
and specifyjob2
as the initial order state.- This approach simulates
job2
being started independently. - The jobs
job2
up tojob6
in job chainjob_stream1
are skipped due to failed or missing execution of predecessor jobs. - Should you previously have performed the Positive Check then the job chains
job_stream3
andjob_stream4
will be kicked off. The fact that job chainjob_stream2
has previously completed successfully during the Positive Check run is sufficient to allow execution ofjob_stream3
andjob_stream4
..
- Modify the job