| Table of Contents | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
Problem:
File file_order_source starts the an order when the file is created, not when the file is ready.
In some file transfer scenarios has the receiver of a file has no knowledge about the time when the sender creates the a file. In case of a (very) large file, it can be the situation that the receiver tries to read the file but is possible for the receiver to try to read a file before the sender has not finished to write writing it. If the receiver get then attempts to use the file at the this moment the sender is still writing, as a result he will get a corrupted, incomplete file. Solution
Solutions:
Using file_order_source:
...
- The sender creates a file
...
- named
abc.txt~. When transmission is complete, the sender renames the file toabc.txt. You would use a regular expression
...
- such as
^.*\.txt$
...
- to check for the presence of files.
- The sender creates a file named
abc.txt. When it is ready, a second file with 0 byte will be created. The name of the second file isabc.txt.trigger
...
- . Here, you would use a regular expression
...
- such as
^.*\.txt.trigger$
...
- . Note that with this approach you have the disadvantage
...
- that
...
- the name of the trigger file is
...
- listed under
scheduler_file_path, not the name of the file that should be executed
...
- .
- Set up a
...
- job chain
...
- where the file size
...
- is checked in the first node. Then carry out a setback if the file size is changing.
Using the JobSchedulerExistsFile job:
| No Format |
|---|
This job has the advantage over {{file_order_source}} that it allows the use of parameters, for example, for the name of the target directory and it allows you define your own polling rate.
|
The JobSchedulerExistsFile job also checks whether the file size is constant - i.e. Is the file still being written? - and will only proceed if the file size is not changing.
| No Format |
|---|
Please note that at the moment this only job on works on Linux systems.
|
You can download example files covering both file_order_source and JobSchedulerExistsFile job solutions