Table of Contents |
---|
Introduction
This is the first in a series of articles describing how to get started with using the the JADE YADE Client via its Command Line Interface .Articles in this series are:and configuring simple file management tasks.
YADE Tutorials List
YADE Client Command Line Interface
- Getting Started and Downloading files
- Simple File Selection
- More Advanced File Selection
- Checking files for completeness
- Public / Private Key Authentication
...
JADE Command Line Interface
...
JADE API
...
JADE JITL Jobs
...
Requirements
The JADE Client should be installed
- You can download the JADE Client from our JADE Download Page or from SourceForge.
- Installation of JADE on Unix systems is covered in the How to install JADE article.
About these examples
- This article describes operation of the JADE Client from the command line, e.g. by using a batch script. The other methods of operating the Client are:
- using the JADE API
- using the JobScheduler's JADE JITL Jobs
- The examples described in this article have been written for use on a Windows computer but can easily be adapted for Unix environments. Settings files for Windows and Unix systems are available for download via the link in the next section.
- The examples presented on this page use public FTP and SFTP servers that we have configured for this purpose on our web site.
- Both servers have the same address:
- A user demo can log on with password demo and download demonstration files. They cannot upload or delete files.
- Comments in JADE settings profiles are added using double semi-colons (";;").
Setting up the examples:
- For Windows users:
- Create a jade_demo folder in your user home directory with a and b sub-folders like this:
mkdir %USERPROFILE%\jade_demo
mkdir %USERPROFILE%\jade_demo\a
mkdir %USERPROFILE%\jade_demo\b
- Download the jade_settings_windows.ini file containing the example profiles presented in this article to the %USERPROFILE%/jade_demo folder and rename it to jade_settings.ini
- Create a jade_demo folder in your user home directory with a and b sub-folders like this:
- For Unix users:
- Create a jade_demo folder in your user home directory with a and b sub-folders like this:
mkdir ~/jade_demo
mkdir ~/jade_demo/a
mkdir ~/jade_demo/b
- Download the jade_settings_unix.ini file containing the example profiles presented in this article to the ~/jade_demo folder and rename it to jade_settings.ini
- Create a jade_demo folder in your user home directory with a and b sub-folders like this:
- Copy the appropriate jade_settings_*.ini file (Windows or Unix) to the jade_demo sub-folder in your user home directory.
:
- For Windows users the location for the jade.cmd file is:
C:\Program Files\sos-berlin.com\jade\client\bin
- or the location that was specified when installing JADE
- for Unix users the location for the jade.sh file is:
~/sos-berlin.com/jade/client/bin
- or the location that was specified when installing JADE
Note that comments in the JADE settings profiles are added using double semi-colons (";;").
Running the examples:
- For the examples is assumed that jade.cmd or jade.sh are available in your path or are prefixed by their path name.
- The command line parameterization is identical for use with the Windows and Unix command line
- Run the examples by entering the following in the command line, e.g. for the first of the examples listed below:
YADE Background Service
Prerequisites
See the Using the tutorials with the YADE Client Command Line Interface article for guidelines to setting up and running the tutorial examples.
Instructions for installing, configuring and using the XML Editor can be found in the XML Editor series of articles.
Download file
The configuration described in this file can be downloaded and then directly opened in the XML Editor from the following link:
Simple YADE file transfer operations - download by FTP and SFTP
Transferring the contents of a folder by FTP
In this example the contents of a folder on a public FTP server that the SOS GmbH has configured for demonstration purposes are copied and saved to the local file system.
The full configuration for this operation is shown in the XML Editor screenshot below.
Configuration structure
The Configurations element shown has two child nodes Fragments and Profiles, each containing a different group of configuration elements.
- The Fragments elements cover the protocol-specific parts of the configuration - in the current configuration these are that the protocol that is to be used, and the connection and authentication.
- Note that a file transfer operation usually requires two Fragments - one for the source part of the transfer and one for the target. However a target fragment is not required here as the the target is the local file system.
- Any number of Fragments can be predefined within a configuration and specified as required using their name attribute - in the example configuration this is ftp_demo_sos-berlln. This predefinition of Fragments allows their reuse and keeps the configuration well structured.
- Note that it is not necessary to specify a port here as YADE will automatically use the required value of 21 for FTP operations unless specified otherwise.
- The Profile elements contain the parameters that are specific to the current operation
- the Operation to be carried out is copy,
- the ftp_demo_sos-berlin Fragment that is to be used for the source part of the operation,
- the files that are to be transferred:
- all files defined using the .* FileSpec
- the directory in which the files to be transferred is looked for:
- defined using / file path,
- the directory that files are to be transferred to: ${USERPROFILE}\jade_demo\a
(on a Linux system the equivalent location would be ${HOME}/jade_demo/a ).
- The ref attribute in the FTPFragmentRef element is used to specify the Fragment to be used - in this case for the source part of the operation.
When the configuration is saved it will be validated against the XSD schema specified for the configuration (see XML Editor - Operation for more information)
The structure of the configuration is described in more detail in the YADE User Manual - Configuring and Running File Transfers article and its child articles.
Configuration Files
The configuration will be converted by default from the XML Editor's native XML format to the settings.ini
format required for YADE up to versions 1.10 when the configuration is saved in the XML Editor.
The following code boxes can be opened to show the simple configuration described above in XML and in settings.ini
formats:
Code Block | ||||
---|---|---|---|---|
| ||||
<?xml version="1.0" encoding="utf-8"?>
<Configurations xsi:noNamespaceSchemaLocation="http://www.sos-berlin.com/schema/yade/YADE_configuration_v1.0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<Fragments>
<ProtocolFragments>
<FTPFragment name="ftp_demo_sos-berlin">
<BasicConnection>
<Hostname><![CDATA[test.sos-berlin.com]]></Hostname>
</BasicConnection>
<BasicAuthentication>
<Account><![CDATA[demo]]></Account>
<Password><![CDATA[demo]]></Password>
</BasicAuthentication>
</FTPFragment>
</ProtocolFragments>
</Fragments>
<Profiles>
<Profile profile_id="ftp_server_2_local">
<Operation>
<Copy>
<CopySource>
<CopySourceFragmentRef>
<FTPFragmentRef ref="ftp_demo_sos-berlin" />
</CopySourceFragmentRef>
<SourceFileOptions>
<Selection>
<FileSpecSelection>
<FileSpec><![CDATA[.*]]></FileSpec>
<Directory><![CDATA[./]]></Directory>
</FileSpecSelection>
</Selection>
</SourceFileOptions>
</CopySource>
<CopyTarget>
<CopyTargetFragmentRef>
<LocalTarget />
</CopyTargetFragmentRef>
<Directory><![CDATA[${USERPROFILE}\jade_demo\a]]></Directory>
</CopyTarget>
</Copy>
</Operation>
</Profile>
</Profiles>
</Configurations> |
Code Block | ||||
---|---|---|---|---|
| ||||
[protocol_fragment_ftp@ftp_demo_sos-berlin]
protocol |
Code Block | ||||
---|---|---|---|---|
| ||||
jade.cmd -settings="%USERPROFILE%\jade_demo\jade_settings.ini" -profile="ftp_server_2_local" |
Code Block | ||||
---|---|---|---|---|
| ||||
./jade.sh -settings="${HOME}/jade_demo/jade_settings.ini" -profile="ftp_server_2_local" |
The following examples are given for Windows environment. Unix users will find the samples in the respective configuration file jade_settings.ini.
Simple JADE file transfer operations - download by FTP and SFTP
Transferring the contents of a folder by FTP
In this example the contents of a folder on a remote host are downloaded from our demonstration FTP server and saved on your local file system.
Example
The following profile from your settings file will be used to run the example on your computer:
Code Block | ||
---|---|---|
| ||
[ftp_server_2_local] ;; operation = copyftp ;; source_hosthost = test.sos-berlin.com source_protocoluser = demo password = demo [ftp_server_2_local] source_portoperation = 21copy source_userinclude = protocol_fragment_ftp@ftp_demo_sos-berlin source_passwordfile_spec = demo.* source_dir = ./ ;; target_hostprotocol = localhostlocal target_protocoldir = local target_dir = ${USERPROFILE}\jade_demo\a |
Running the configuration
The simple configuration described above is started from the command line or in a batch file on Windows systems using one of the following calls, where sos-berlin_demo_2_local is the name given to the configuration file on saving, depending on the YADE version being used:
Code Block | ||||
---|---|---|---|---|
| ||||
jade.cmd -settings="%USERPROFILE%\jade_demo\sos-berlin_demo_2_local.ini" -profile="ftp_server_2_local_pass" |
Code Block | ||||
---|---|---|---|---|
| ||||
jade.cmd -settings="%USERPROFILE%\jade_demo\sos-berlin_demo_2_local.xml" -profile="ftp_server_2_local_pass" |
On Unix systems the profile is called using one of the following commands, depending on the YADE version being used parameters listed above should not require further description and are the minimum that are required to copy files from the demo user's root directory on the remote source to a local target directory. By default JADE will download all the files it finds in the directory.Run the example by entering the following in the command line:
Code Block | |||||
---|---|---|---|---|---|
| |||||
./jade.shjade.cmd -settings="%USERPROFILE%\jade_demo\jade_settings.ini${HOME}/jade_demo/sos-berlin_demo_2_local.ini" -profile="ftp_server_2_local_pass" |
Code Block | ||||
---|---|---|---|---|
| ||||
./jade.sh -settings="${HOME}/jade_demo/sos-berlin_demo_2_local.xml" -profile="ftp_server_2_local_pass" |
More information can be found in the YADE User Manual - Configuring and Running File Transfers article.
Behavior
JADE YADE will log onto the remote server and download all the files it finds in the source _dir folder Directory folder (six in total) before stopping.
Note that by default JADE YADE will create the target folder and/or path if it does not exist and if JADE YADE has the necessary permissions. It will also overwrite any files it finds in the target directory with the same names as files it is downloading.
Note also that the transfer mode JADE YADE uses for FTP transfer can be specified using the transfer_mode parameter (not used in the example). This can be either ascii or binary (the default setting).
Transferring the contents of a folder by SFTP with password authentication
This example uses the second profile in our examples settings file, sftp_server_2_local_pass, which is quite similar to the ftp_server_2_local profile used in the previous example.
As the new profile name suggests, the SFTP protocol is used in this profile and authentication is by user/password - key pair authentication is described later in this series of articles.
Example
The profile is included in the jade_settings_windows.ini file as follows:
...
language | bash |
---|
...
the SFTP protocol and password authentication to copy the files in a folder on a public SFTP server that has been set up in parallel to the FTP server used in the first example to the local file system.
Configuration
While this operation is similar to that described in the first example, it is necessary to create a new Fragment element in the configuration. In addition a new Profile element will be created, to keep use of the Profiles elements as flexible as possible.
Fragment element
The new SFTPFragment is generated by selecting the ProtocolFragments element and using the Add child context menu item (right hand mouse click) in the same way as the FTPFragment was added in the previous example. The completed SFTPFragment configuration is shown in the firat screenshot below. Please note:
- It is not necessary to specify a port here as YADE will automatically use the required value of 22 for SFTP unless specified otherwise.
- The new fragment will be addressed with the sftp_demo_sos-berlin_passname attribute.
- The Password element is not a direct descendant of the Authentication element as was the case with FTP - this allows the use of an alternative authentication method (key pair authentication, described in the Public / Private Key Authentication tutorial)
XML Editor Configuration
Profile element
The Profile element required for this example will be almost identical to the FTP Profile from the previous example, with the exception that an SFTPFragmentRef element is required this time. In this case we suggest that the following procedure is used:
- generate the new Profile by duplicating the ftp_server_2_local element using the XML Editor's Copy and Paste functions, which are available using the XML Editor's context menu,
- delete the FTPFragmentRef element and add a new SFTPFragmentRef element,
- select the SFTPFragment as the value for the ref attribute of the SFTPFragmentRef element.
The completed sftp_server_2_local_pass
...
profile is shown below:
Running the Profile
This profile is called on Windows systems using one of the following commands, depending on the YADE version being used:
Code Block | ||||
---|---|---|---|---|
| ||||
jade.cmd -settings="%USERPROFILE%\jade_demo\sos-berlin_demo_2_local.ini" -profile="sftp_server_2_local_pass" |
Code Block | ||||
---|---|---|---|---|
| ||||
jade.cmd -settings="%USERPROFILE%\jade_demo\sos-berlin_demo_2_local.xml" -profile="sftp_server_2_local_pass" |
On Unix systems the profile is called using one of the following commands, depending on the YADE version being used:
Code Block | ||||
---|---|---|---|---|
| ||||
./jade.sh -settings="${HOME}/jade_demo/sos-berlin_demo_2_local.ini" -profile="sftp_server_2_local_pass" |
Code Block | ||||
---|---|---|---|---|
| ||||
./jade.sh -settings="${HOME}/jade_demo/sos-berlin_demo_2_local.xml |
Then call this profile by entering the following in the Windows command line:
Code Block | ||
---|---|---|
| ||
jade.cmd -settings="%USERPROFILE%\jade_demo\jade_settings.ini" -profile="sftp_server_2_local_pass" |
Behavior
As with the FTP transfer profile, JADE YADE will log onto the remote server and download all the files (FileSpec = .*) in the source_dir Directory folder before stopping. As with the previous example, six files will be downloaded.