Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The demo job chain ('load_files') is shown in the diagram to the right.

Show If
groupsos-members
Info
titlerework content
  • Could someone provide a Unix example?
  • The following section of text is somewhat orphaned: should it be removed?

 

TODO - ADD SCREENSHOT OF JOB CHAIN IN JOE

The 'Folderin_a' and 'Folderin_b' symbols represent the 'File Order Source' directory monitored by the 'load_files' job chain. The remaining jobs in the chain shown the individual chain nodes shown the job chain 'Steps/Nodes' form. Note that the secondary job chain shown the behaviour in the event of an error.

  • JobScheduler starts as soon as file matching with Regular Expression found in the File Order Source directory.
  • JobScheduler's aquire_lock job matches file with regular expression and decide file's category i.e. Berlin or Munich.
  • Once aquire_lock finds the matching category its try to set an Semaphore (Flag) using JobScheduler's inbuilt LOCK mechanism
  • There is only one instance on LOCK is allowed or once LOCK is assigned to first file of Berlin category, next file has to wait or setback until the LOCK is free.
  • The same mechanism will be repeated for files from category Munich but since the LOCK is not acquired ( or Semaphore (Flag)) is not set for Munich, file from category Munich will be allowed to be processed.
  • Once process is finished depending upon success or error , JS will move the file from the 'in' folder to either the 'done' (on success) or 'failed' (on error).
  • After moving the input file to correct target directory the JobScheduler job release_lock will be called which will remove the lock/semaphore from JobScheduler and the next file from same category will be allowed.

See also