Page History

...

This is the fifth in a series of articles describing how to get started with using the JADE Client via its Command Line Interface.The articles in this series are:

JADE Tutorials List

JADE Client Command Line Interface

1. Getting Started and Downloading files
2. Simple File Selection
3. More Advanced File Selection
   

4. File Transfer

5. Checking Files files for Completenesscompleteness
6. Public / Private Key Authentication
7. Transfer via a Jump Host / DMZ

JADE Background Service

1. Installation, Configuration and Use

Prerequisites

See the For setting-up and running the examples see Using the tutorials with the JADE Client Command Line Interface

Checking files for completeness

article for guidelines to setting up and running the tutorial examples.

Instructions for installing, configuring and using the XML Editor can be found in the XML Editor series of articles.

Download file

The configuration described in this tutorial can be downloaded and then directly opened in the XML Editor using the following link:

• coming ....

Checking files for completeness

A A number of methods for checking whether file transfer has been completed are presented in this document:

• MD5 hash checksums, which are generated before file transfer starts and can be used to check files after transfer.  
• Atomic file transfer, which is used to Atomic file transfer, which is used to hide files being transferred from target directory monitors until the file transfer has been completed.
  • A prefix and/or suffix is added to the name of the file for the duration of the transfer.
    This prefix/suffix is chosen so that the file being transferred will not be matched by any target host file monitor.
  • Once transfer has been completed, JADE removes the prefix/suffix and the file becomes visible to a file monitor.
  MD5 hash checksums, which are generated before file transfer starts can be used to check files after transfer
  • .
    
• Regularly checking the file size until it reaches a steady state. Here it is assumed that the transfer has been completed when two checks in series return the same size.

...

• Repeating the file transfer and comparing the sizes of the file transfers. When both transfers have the same size then it is assumed that the transfer is complete.
  See the check_retry the CheckSteadyState parameter documentation for more information.
• Checking file size after transfer with the check-size the CheckSize parameter.

If more than one file is to be transferred then the use of the transaction parameter is generally recommended (TBD - LINK)

File Completeness - MD5 Hash Checksums

Example a) -

...

Creating an MD5 checksum file

JADE can write a file's MD5 checksum into a file which it gives the ending .MD5. This file generate an MD5 checksum file for a file. This file which is given the ending .md5 and can then be transferred along with the 'original' file. Note however that JADE only generates the checksum file during a file transfer operation, whilst it is reading the 'original' file from the transfer source. This means that this feature has a limited validity and that in practice JADE checks files for completeness using integrity hash files that have been generated by another source.

Note that to create and save an integrity hash file before a file transfer operation, JADE requires write permissions for the source directory. In turn, this means that this feature cannot be However, to do this, JADE requires write permissions for the source directory. In turn, this means that this feature cannot be demonstrated by downloading files from our test server, where the demo user only has read permissions. Instead it will be demonstrated through local transferdemonstrated through local transfer.

Note

• This example uses a set of 5 test_*.txt files that have been saved into the user's \jade_demo\a directory by an earlier tutorial in this series - for example The JADE Client Command Line Interface - Tutorial 1 - Getting Started .
• A ProtocolFragment is not required in the configuration of this example, as local to local transfer is being carried out.

The Profile Code

The following code boxes can be opened to show the Profile used in this example in XML and in settings.ini formats.

...

[local_2_local_create_md5]
operation                           = copy
source_protocol                     = local
file_spec                           = ^(test)(_)[0-9]\.txt
source_dir                          = ${USERPROFILE}\jade_demo\a
target_protocol                     = local
target_dir                          = ${USERPROFILE}\jade_demo\b
check_security_hash                 = true
create_security_hash_file           = true

Run the example using:

jade.cmd -settings="%USERPROFILE%\jade_demo\jade_settings.ini" -profile="local_2_local_create_md5"      = true
create_security_hash_file           = true

Running the Profile

This profile is called on Windows systems using the following command, depending on the JADE version being used:

...

./jade.sh -settings="${HOME}/jade_demo/local_2_local_file_complete.xml" -profile="local_2_local_create_md5"
./jade.sh -settings="${HOME}/jade_demo/local_2_local_file_complete.xml" -profile="local_2_local_create_md5"

Behavior

If you have already run some of the previous examples, you will have a series of files in the the ${USERPROFILE}\jade_demo\a folder on your computer, sequentially named test_1.txt through to test_5.txt.

JADE will carry out the following steps when executing the local_2_local_create_md5 profile: Profile:

• Read each of the files in the source directory matching the FileSpec regular expression into memory
• Generate an integrity hash file Save the MD5 hash for each 'original' file files it finds matching the file_spec parameter regex in the source directory in a temporary file. FileSpec.
• Save the integrity hash files along with Transfer the 'original' files to in the transfer target directory.
• Move the files containing the MD5 hash tags to the target directory after transfer of the 'main' files has been completed.
  An MD5 file corresponding to a file test_1.txt would carry the name test_1.txt.MD5md5.

The next example shows the use of the MD5 file to verify the transferred files.

...