Example showing an architecture for forwarding files from multiple nodes

Model architecture showing collection of files delivered to a large number of server nodes and forwarded by a central JobScheduler to a predefined set of destinations.

The key components of this architecture, which is illustrated in the diagram, are:

• The central JobScheduler, which actually carries out the file transfers.
• The server nodes, which are each equipped with a JobScheduler. Each of these JobSchedulers monitors a number of folders for incoming files and has a job chain that prepares the forwarding instructions for the central JobScheduler.

This configuration provides a central operation point for the file transfer jobs in JOC.

This architecture functions as follows:

• A file is added to one of the folders that are being monitored by a JobScheduler on one of the server nodes. This monitoring JobScheduler then starts a job chain.
• This job chain has three jobs with an initial file order source node:

1. Job 1: Checks if the file is fully loaded? (i.e, Has it achieved a steady state?)
2. Job 2: Sets file transfer parameters for the server and target. Different parameters can be set as required
3. Job 3: Forwards the file transfer parameters to the central JobScheduler in form of an "add order" command using TCP.

• The central JobScheduler starts file transfer when the order arrives from the server JobScheduler. It transfers the file using the YADE JITL job and either FTP or SFTP protocols.
  After file transfer has been completed, the central JobScheduler starts a housekeeping/clean-up job on the server JobScheduler in 'Agent-mode'. This is also done using one of the JITL jobs provided with JobScheduler.

See also:

• Directory Monitoring with File Orders
• file_order_source XML Element
• FAQ - Directory Monitoring and File Watching
• JITL - JobScheduler Integrated Template Library