Page History

...

• JS starts as soon as file matching with Regular Expression found in in directory
• JS's aquire_lock job matches file with regular expression and decide file's category i.e. Berlin or Munich
• Once aquire_lock finds the matching category its try to set an Semaphore (Flag) using JS's inbuilt LOCK mechanism
• There is only one instance on LOCK is allowed or once LOCK is assigned to first file of Berlin category, next file has to wait or setback until the LOCK is free.
• THe same mechanism will be repeated for files from category Munich but since the LOCK is not acquired ( or Semaphore (Flag)) is not set for Munich, file from category Munich will be allowed to be processed.
• Once process is finished depending upon success or error , JS will move the file from in to either done (success) or failed(error).
• After moving input file to correct target directory JS job release_lock will be called which will remove the lock/Semaphore from JS and next file from same category will be allowed.

...

Limitations of this solution

• The trick with limitation of this approach is to ensure is that there is no guarantee that the files are processed sequentially, should will be processed in their intended order if more than one file should arrive from a subsidary at once because there is .
  This situation could typically occur after an unplanned loss of file transfer connection. After the connection has been restored, there is a) no guarantee that a 'group' of the files arriving as a batch will be written to the file systen in a particular order b) no way for JobScheduler to know how many files will arrive as a batch.
  • One approach would be to wait until a steady state in the incoming directory has been reached (no new file has been added and the size of all files remains constant over a period such as a minutesuitable period of time) before starting to identify files and forward them for processing. JobScheduler could then sort files according to their names before forwarding them for processing.

    No Format
    The downside of this second approach is that it brings a delay in starting processing with it, due to the need wait to check whether all members of a batch of files has arrived by checking for a steady state.

Demo Package

A demonstration of this solution is available for download from:

...