Table of Contents |
---|
Introduction
This is the second in a series of articles and describes how to perform simple file selection with the JADE YADE Client via its Command Line Interface.
...
YADE Tutorials List
...
YADE Client Command Line Interface
- Getting Started and Downloading files
- Simple File Selection
- More Advanced File Selection
- Checking files for completeness
- Public / Private Key Authentication
- Transfer via a Jump Host / DMZ
...
YADE Background Service
Prerequisites
See the Using the tutorials with the JADE YADE Client Command Line Interface article for guidelines to setting up and running the tutorial examples.
Instructions for installing, configuring and using the XML Editor can be found in the XML Editor series of articles.
Download file
The configuration described in this tutorial can be downloaded and then directly opened in the XML Editor from the following link:
Scope of this tutorial
Before starting to transfer files JADE YADE reads the contents of the folder specified using one or more child elements of the Selection parameter.
...
The basic file selection methods will be described in this tutorial:
File Name Filtering and Recursive File Selection
The following profile shows how JADE YADE can be used to:
- Filter the names of files to be transferred using a regular expression
- Transfer files from sub-directories recursively
Configuration structure
ProtocolFragment
The ftp_demo_sos-berlin ProtocolFragment used in the Client Command Line Interface - Getting Started tutorial will be reused.
Profile
The example presented in this tutorial uses a Profile based on the ftp_server_to_local Profile described in the Getting Started tutorial.
...
The value of the file_spec defines the regular expression used to filter file names
- A recursive child element has been added to the FileSpecSelection element and its value set to true (the default value is false).
- The source_dir parameter has also been changed to:
- /rec
XML Editor Configuration
The Profile Code
The following code boxes can be opened to show the Profile and ProtocolFragments used in this example in XML and in settings.ini
formats.
Code Block | ||||
---|---|---|---|---|
| ||||
<?xml version="1.0" encoding="utf-8"?> <Configurations xsi:noNamespaceSchemaLocation="http://www.sos-berlin.com/schema/jadeyade/JADEYADE_configuration_v1.0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <Fragments> <ProtocolFragments> <FTPFragment name="ftp_demo_sos-berlin"> <BasicConnection> <Hostname><![CDATA[test.sos-berlin.com]]></Hostname> </BasicConnection> <BasicAuthentication> <Account><![CDATA[demo]]></Account> <Password><![CDATA[demo]]></Password> </BasicAuthentication> </FTPFragment> </ProtocolFragments> </Fragments> <Profiles> <Profile profile_id="ftp_server_2_local_select_recursive"> <Operation> <Copy> <CopySource> <CopySourceFragmentRef> <FTPFragmentRef ref="ftp_demo_sos-berlin" /> </CopySourceFragmentRef> <SourceFileOptions> <Selection> <FileSpecSelection> <FileSpec><![CDATA[^test_.\.txt$]]></FileSpec> <Directory><![CDATA[./rec]]></Directory> <Recursive>true</Recursive> </FileSpecSelection> </Selection> </SourceFileOptions> </CopySource> <CopyTarget> <CopyTargetFragmentRef> <LocalTarget /> </CopyTargetFragmentRef> <Directory><![CDATA[${USERPROFILE}\jade_demo\a]]></Directory> </CopyTarget> </Copy> </Operation> </Profile> </Profiles> </Configurations> |
Code Block | ||||
---|---|---|---|---|
| ||||
[protocol_fragment_ftp@ftp_demo_sos-berlin] protocol = ftp host = test.sos-berlin.com user = demo password = demo [ftp_server_2_local_select_recursive] operation = copy source_include = protocol_fragment_ftp@ftp_demo_sos-berlin file_spec = ^test_.\.txt$ source_dir = ./rec recursive = true target_protocol = local target_dir = ${USERPROFILE}\jade_demo\a |
Running the Profile
This profile is called on Windows systems using one of the following commands, depending on the JADE YADE version being used:
Code Block | ||
---|---|---|
| ||
jade.cmd -settings="%USERPROFILE%\jade_demo\sos-berlin_demo_2_local_select.ini" -profile="ftp_server_2_local_select_recursive" |
Code Block | ||
---|---|---|
| ||
jade.cmd -settings="%USERPROFILE%\jade_demo\sos-berlin_demo_2_local_select.xml" -profile="ftp_server_2_local_select_recursive" |
On Unix systems the profile is called using one of the following commands, depending on the JADE YADE version being used:
Code Block | ||
---|---|---|
| ||
./jade.sh -settings="${HOME}/jade_demo/sos-berlin_demo_2_local_select.ini" -profile="ftp_server_2_local_select_recursive" |
Code Block | ||
---|---|---|
| ||
./jade.sh -settings="${HOME}/jade_demo/sos-berlin_demo_2_local_select.xml" -profile="ftp_server_2_local_select_recursive" |
Behavior
The relevant file structure on the source server is:
/
/rec
test.txt
test_1.txt/sd
test_2.txt
- JADE YADE will download the test_1.txt from the /rec folder on the server and ignore the test.txt file whose name does not match with the FileSpec regular expression.
- JADE YADE will create a /sd folder in the target a directory if this folder does not already exist.
- It will the open the /rec/sd sub-folder on the server, create a corresponding folder on the target and download the test_2.txt file to this folder.
Behavior with DisableErrorOnNoFilesFound = true
The default behavior of JADE YADE is to throw an error if a file transfer is attempted and no files are found. The DisableErrorOnNoFilesFound element is used to allow JADE YADE to attempt a file transfer without an error being generated if no files are found. This behavior can be useful when, for example, polling for files.
The DisableErrorOnNoFilesFound element is specified as a Directives child element, the next screenshot shows how this element is integrated in a configuration:
...
If, however, the DisableErrorOnNoFilesFound element is set to true, JADE YADE will not throw an error.
Note:
- The meaning of the values true and false for the DisableErrorOnNoFilesFound element are the opposite to the meaning for the element's
settings.ini
file predecessor, force_files.
Zero-Byte Files
The JADE YADE Client can handle files with zero bytes in a number of ways, depending on the setting used for the TransferZeroByteFiles parameter, which is specified as a child of the Directives element.
Test Example
The default TransferZeroByteFiles setting is true. The screenshot below shows how this element is integrated in a duplicate of the ftp_server_2_local file transfer profile that was described in the first tutorial in this series. This new Profile has profile_id = ftp_server_2_local_zero_byte.
...
The screenshot also shows the relevant extract from the parameter reference for the TransferZeroByteFiles element with a description of the meaning of the different values the TransferZeroByteFiles element can have.
XML Editor Configuration
The Profile Code
The following code boxes can be opened to show the Profile and ProtocolFragments used in this example in XML and in settings.ini
formats.
Code Block | ||||
---|---|---|---|---|
| ||||
<?xml version="1.0" encoding="utf-8"?> <Configurations xsi:noNamespaceSchemaLocation="http://www.sos-berlin.com/schema/jadeyade/JADEYADE_configuration_v1.0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <Fragments> <ProtocolFragments> <FTPFragment name="ftp_demo_sos-berlin"> <BasicConnection> <Hostname><![CDATA[test.sos-berlin.com]]></Hostname> </BasicConnection> <BasicAuthentication> <Account><![CDATA[demo]]></Account> <Password><![CDATA[demo]]></Password> </BasicAuthentication> </FTPFragment> </ProtocolFragments> </Fragments> <Profiles> <Profile profile_id="ftp_server_2_local_zero_byte"> <Operation> <Copy> <CopySource> <CopySourceFragmentRef> <FTPFragmentRef ref="ftp_demo_sos-berlin" /> </CopySourceFragmentRef> <SourceFileOptions> <Selection> <FileSpecSelection> <FileSpec><![CDATA[.*]]></FileSpec> <Directory><![CDATA[./]]></Directory> </FileSpecSelection> </Selection> <Directives> <DisableErrorOnNoFilesFound>true</DisableErrorOnNoFilesFound> <TransferZeroByteFiles><![CDATA[false]]></TransferZeroByteFiles> </Directives> </SourceFileOptions> </CopySource> <CopyTarget> <CopyTargetFragmentRef> <LocalTarget /> </CopyTargetFragmentRef> <Directory><![CDATA[${USERPROFILE}\jade_demo\a]]></Directory> </CopyTarget> </Copy> </Operation> </Profile> </Profiles> </Configurations> |
Code Block | ||||
---|---|---|---|---|
| ||||
[protocol_fragment_ftp@ftp_demo_sos-berlin] protocol = ftp host = test.sos-berlin.com user = demo password = demo [ftp_server_2_local_zero_byte] operation = copy source_include = protocol_fragment_ftp@ftp_demo_sos-berlin file_spec = .* source_dir = ./ force_files = false zero_byte_transfer = false target_protocol = local target_dir = ${USERPROFILE}\jade_demo\a |
Running the Profile
This profile is called on Windows systems using one of the following commands, depending on the JADE YADE version being used:
Code Block | ||
---|---|---|
| ||
jade.cmd -settings="%USERPROFILE%\jade_demo\sos-berlin_demo_2_local_select.ini" -profile="ftp_server_2_local_zero_byte" |
Code Block | ||
---|---|---|
| ||
jade.cmd -settings="%USERPROFILE%\jade_demo\sos-berlin_demo_2_local_select.xml" -profile="ftp_server_2_local_zero_byte" |
On Unix systems the profile is called using one of the following commands, depending on the JADE YADE version being used:
Code Block | ||
---|---|---|
| ||
./jade.sh -settings="${HOME}/jade_demo/sos-berlin_demo_2_local_select.ini" -profile="ftp_server_2_local_zero_byte" |
Code Block | ||
---|---|---|
| ||
./jade.sh -settings="${HOME}/jade_demo/sos-berlin_demo_2_local_select.xml" -profile="ftp_server_2_local_zero_byte" |
Behavior
- With the TransferZeroByteFiles element set to false or relaxed (and with the FileSpec set to
^test_.\.txt$
as listed above) neither will the test_4.txt zero byte file be downloaded nor will an error be raised. - Setting the TransferZeroByteFiles element to false and rerunning the example will:
- Cause the the test_4.txt file to be ignored as with the relaxed setting and
- Cause an error to be raised if no files with more than zero bytes are found - i.e. if no files are to be transferred.
This can be demonstrated by changing the FileSpec regex to^test_[4]\.txt$
which will only match for the zero byte file.
- A setting of TransferZeroByteFiles = strict will cause an error to be raised if any zero byte files are matched.
...