Page History

...

A demonstration of this solution is available for download from:

• sequential_execution_of_jobs.zip

Demo Installation

• Unpack the zip file to a local directory

• Copy the 'SQLLoaderProc' folder to your JobScheduler 'live' folder.

  No Format
  ... TODO - ADD LINK TO DOKU

• Copy the 'Data' folder to the a suitable local location.

  No Format
  The default location for this folder, which is specified in the configurations in the demo jobs, is: {{C:\sandbox}} Note that the following paths have to be modified if the location of the 'Data' folder is changed:

• The 'load_files' File Order Source directories in the load_files.job_chain.xml job chain object
• the 'source_file' and 'target_file' paths specified as parameters in the move_file_suc.job.xml and move_file_error.job.xml objects

...

• Just copy files from the 'Data/__test-files' folder to the 'in' folder, JobScheduler will automatically start processing within a few seconds.

How does the Demo Work?

{{DiagramBoxRight

...

...

digraph "Example: File Transfer using JadeJob"{

graph [rankdir=TB]
node [shape=box,style="rounded,filled",fillcolor=aquamarine]
node [fontsize=10]
ranksep=0.3

FolderIn_a ->  start
FolderIn_b ->  start

start  -> aquire_lock  -> load_file_process  -> move_file_suc  -> suc_release_lock  -> end_suc  -> success
move_file_err -> error_release_lock -> end_err -> error

}

The demo job chain ('load_files') is shown in the diagram to the right.

 TODO - ADD SCREENSHOT OF JOB CHAIN IN JOE
 The 'Folderin_a' and 'Folderin_b' symbols represent the 'File Order Source' directory monitored by the 'load_files' job chain.
 The remaining jobs in the chain shown the individual chain nodes shown the job chain 'Steps/Nodes' form.
 Note that the secondary job chain shown the behaviour in the event of an error.

• JobScheduler starts as soon as file matching with Regular Expression found in the File Order Source directory.
• JobScheduler's aquire_lock job matches file with regular expression and decide file's category i.e. Berlin or Munich.
• Once aquire_lock finds the matching category its try to set an Semaphore (Flag) using JobScheduler's inbuilt LOCK mechanism
• There is only one instance on LOCK is allowed or once LOCK is assigned to first file of Berlin category, next file has to wait or setback until the LOCK is free.
• The same mechanism will be repeated for files from category Munich but since the LOCK is not acquired ( or Semaphore (Flag)) is not set for Munich, file from category Munich will be allowed to be processed.
• Once process is finished depending upon success or error , JS will move the file from the 'in' folder to either the 'done' (on success) or 'failed' (on error).
• After moving input file to correct target directory JobScheduler job release_lock will be called which will remove the lock/Semaphore from JS and next file from same category will be allowed.

See also:

• Our Using locks FAQ.
• The Locks section in the JobScheduler reference documentation.
• Our Best Practice FAQ.

 Note that the secondary job chain shown on the right of the diagram is only relevant of an error occurs.

The 'Folderin_a' and 'Folderin_a' symbols represent the 'File Order Source' directory monitored by the job chain.

• JobScheduler starts as soon as file matching with Regular Expression found in in directory.
• JobScheduler's aquire_lock job matches file with regular expression and decide file's category i.e. Berlin or Munich.
• Once aquire_lock finds the matching category its try to set an Semaphore (Flag) using JobScheduler's inbuilt LOCK mechanism
• There is only onc instance on LOCK is allowed or once LOCK is assigned to first file of Berlin category, next file has to wait or setback until the LOCK is free.
• THe same mechanism will be repeated for files from category Munich but since the LOCK is not acquired ( or Semaphore (Flag)) is not set for Munich, file from category Munich will be allowed to be processed.
• Once process is finished depending upon success or error , JS will move the file from the 'in' folder to either the 'done' (on success) or 'failed' (on error).
• After moving input file to correct target directory JobScheduler job release_lock will be called which will remove the lock/Semaphore from JS and next file from same category will be allowed.

Graphical Representation of the Solution

digraph "Example: File Transfer using JadeJob"\{

graph [rankdir=TB]
node [shape=box,style="rounded,filled",fillcolor=aquamarine]
node [fontsize=10]
ranksep=0.3

FolderIn_a ->  start
FolderIn_b ->  start

start  -> aquire_lock  -> load_file_process  -> move_file_suc  -> suc_release_lock  -> end_suc  -> success
move_file_err -> error_release_lock -> end_err -> error

\}

...