...
File Completeness - Atomic File Transfer
...
ExampleDescription
The principle of atomic file transfer was described at the beginning of this article.
The local_2_local_atomic profile listed below is will carry out atomic file transfer from our test server to a local folder. Three relatively large files (each 200KB) will be downloaded and theses may be sufficiently large for you to see that they are first of all written to the local file system with the ~ suffix while the download is taking place and that the suffix is removed once the download has bee completed.
Download file
The configuration used for this example can be downloaded using the following link and then directly opened in the XML Editor:
XML Editor Configuration
The configuration used to demonstrate atomic file transfer is uses the ftp_demo_sos-berlin Protocol Fragment already used in several of the tutorials in this series. The two FileSpecSelection elements in the Profile point to three large files in a subfolder on the test server. A tilde (~) is used as an AtomicSuffix and an AtomicPrefix is not required.
The Profile Code
The following code boxes can be opened to show the Profile used in this example in XML and in settings.ini formats.
...
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
[protocol_fragment_ftp@ftp_demo_sos-berlin]
protocol = ftp
host = test.sos-berlin.com
user = demo
password = demo
[ftp_server_2_local_atomic]
operation = copy
source_include = protocol_fragment_ftp@ftp_demo_sos-berlin
file_spec = ^test_large_.\.txt$
source_dir = ./large
target_protocol = local
target_dir = ${USERPROFILE}\jade_demo\a\large
atomic_suffix = ~ |
Running the Atomic Transfer Profile
This profile is called on Windows systems using the following command, depending on the JADE version being used:
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
./jade.sh -settings="${HOME}/jade_demo/sos-berlin_demo_2_local_atomic.xml" -profile="ftp_server_2_local_atomic"
./jade.sh -settings="${HOME}/jade_demo/sos-berlin_demo_2_local_atomic.xml" -profile="ftp_server_2_local_atomic" |
Behavior - Atomic File Transfer
When atomic file transfer is specified JADE will add the characters specified in the AtomicPrefix and AtomicSuffix parameters to the file name. Once the file transfer has been completed, JADE deletes the prefix and suffix.
...
File Completeness - Check Steady State
Example Description
The local_2_local_check_steady_state profile listed below is intended to be used in JADE together with a second file transfer program that allows the file transfer rate to be restricted. This second file transfer program is used to transfer a series of files from our test server at a relatively slow rate to the local jade_demo/a folder used in previous examples. In parallel, JADE is used to check the size of files arriving in the folder jade_demo/a. JADE will transfer files to the target folder jade_demo/b once a steady state for each file has been found.
...
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
[local_2_local_steady_state]
operation = copy
source_protocol = local
file_spec = ^test_large_.\.txt$
source_dir = ${USERPROFILE}\jade_demo\a
check_steady_state_of_files = true
check_steady_state_interval = 1
steady_state_count = 30
target_protocol = local
target_dir = ${USERPROFILE}\jade_demo\b |
Running the Check Steady State Profile
This profile is called on Windows systems using the following command, depending on the JADE version being used:
...