Page History

Problem

Within a job chain file_order_:
File order source starts the an order when the file is created, not when the file is ready.

In some file transfer scenarios has the receiver of a file has no knowledge about the time when the sender creates the a file. In case of a (very) large file it can be the situation that , it is possible for the receiver tries to try to read the a file but before the sender has not finished to write writing it. If the receiver get then attempts to use the file at the this moment the sender is still writing, as a result he will get a corrupted, incomplete file.

Solution:

...

Solutions

Using file_order_source

There are three ways how to use file_order_source:

• The sender creates a file

...

• named abc.txt~.

...

• After the transfer is completed, the sender renames the file to abc.txt. You would use a regular expression

...

• such as ^.*\.txt$

...

• to check for the presence of files.

• The sender creates a file named abc.txt. When it is ready, a second file with 0 byte will be created. The name of the second file is abc.txt.trigger

...

• . Here, you would use a regular expression

...

• such as ^.*\.txt.trigger$

...

• . Note that with this approach you have the disadvantage

...

• that

...

• the name of the trigger file is

...

• listed under scheduler_file_path, not the name of the file that should be executed

...

• .

• Set-up a

...

• job chain

...

• where the file size

...

• is checked in the first node. Then carry out a setback if the file size is changing. This can be done with the job JobSchedulerExistsFile

See also: Directory Monitoring with File Orders

Using the JobSchedulerExistsFile job

• This job has the advantage over file_order_source solutions that it allows the use of parameters, for example for the name of the target directory, and it allows you to configure the polling rate.
• The JobSchedulerExistsFile job also checks whether the file size is constant - i.e. Is the file still being written? - and will only proceed if the file size is not changing.
• The JobSchedulerExistsFile job has three parameters to manage the check steady state behaviour.
  • check_steady_state_of_files: If true, job will check the steady state.
  • check_steady_state_interval: Interval in seconds between two checks
  • steady_state_count: If set, this is maximum number of intervals. If the maximum is reached, the task will be terminated with an error.

See also: Job JobSchedulerExistsFile

Related Downloads

You can download example files covering both file_order_source and JobSchedulerExistsFile job solutions