Table of Contents |
---|
Pre- & Post-Processing Options
YADE can carry out both pre- and post-processing at the source and at the target parts of the transfer.
A typical post-processing operation at the source would be to move or rename a file after it has been copied to the target.
Pre- and post-processing can be triggered for:
- Transfer operations.
- Individual file transfers, so when three files are to be transferred, processing would be carried out three times.
In addition, post-processing can be triggered for:
- File Rename operations.
Pre- and post-processing can be carried out using the FTP, SFTP and local file system protocols: they are not possible with HTTP, HTTPS, SMBF or WebDAV.
Pre- and post-processing commands
The commands that can be executed as part of a pre- or post-processing operation are protocol-dependant:
- FTP commands have to be used with the FTP protocol and
- Shell commands have to be used with the SFTP protocol.
Pre- and post-processing operations
The pre- and post-processing operations available are:
- Pre-processing operations:
- CommandBeforeFile:
- commands are executed before each individual file is transferred
- CommandBeforeOperation
- commands are executed before the transfer operation is started
- CommandBeforeFile:
- Post-processing operations:
- CommandAfterFile:
- commands are executed after each file transfer has been completed
- CommandAfterOperation
- commands are executed after the transfer operation has been completed
- CommandBeforeRename
- commands are executed before each individual file is renamed
- CommandAfterFile:
Note that special variables such as $target_dir and $date are available for pre- or post-processing commands. These variables are listed under the Parameter Reference article for the relevant protocol *FragmentRef element. See the list of links provided in the Parameter reference for pre- and post-processing operations section on this page for more information.
Specifying pre- and post-processing operations
The commands that can be executed as part of a pre- or post-processing operation are protocol specific - meaning that they are specified as child elements of a protocol fragment element such as SFTPFragmentRef.
The XML hierarchy used to specify the pre- and post-processing operations around a typical Copy operation using SFTP would be:
- Profile
- Operation
- CopySource
- CopySourceFragmetRef
- SFTPFragmentRef
- SFTPPreProcessing
- CommandBeforeFile
- CommandBeforeOperation
- SFTPPostProcessing
- CommandBeforeFile
- CommandBeforeOperation
- CommandBeforeRename
- etc. (optional)
- SFTPPreProcessing
- SFTPFragmentRef
- CopySourceFragmetRef
- CopySource
- Operation
Parameter reference for pre- and post-processing operations
Detailed information about pre- and post-processing operations can be found in the relevant parameter reference article:
Variables
Variables can be used in pre- and post-processing commands.
Such variables provide the path and name of files in a transfer.
Refactored Variables
See
Jira | ||||||||
---|---|---|---|---|---|---|---|---|
|
- File separator (all paths) are provided as follows:
\
: if the pre- and post-processing commands are executed on a Windows machine./
: if the pre- and post-processing commands are not executed on a Windows machine.
- All directory paths provide the path without a trailing slash or backslash.
Variables:
- Directory variables:
${TargetDirFullName}
${SourceDirFullName
}- Target file variables:
${TargetFileFullName}
${TargetFileRelativeName}
${TargetFileBaseName}
${TargetFileParentFullName}
${TargetFileParentBaseName}
- Target transfer file variables:
${TargetTransferFileFullName}
${TargetTransferFileRelativeName}
${TargetTransferFileBaseName}
${TargetTransferFileParentFullName}
${TargetTransferFileParentBaseName}
- Source file variables:
${SourceFileFullName}
${SourceFileRelativeName}
${SourceFileBaseName}
${SourceFileParentFullName}
${SourceFileParentBaseName}
- Source renamed file variables:
${SourceFileRenamedFullName}
${SourceFileRenamedRelativeName}
${SourceFileRenamedBaseName}
${SourceFileRenamedParentFullName}
${SourceFileRenamedParentBaseName}
Directory | ${SourceDirFullName} | ${TargetDirFullName} | ||||
---|---|---|---|---|---|---|
The directory where files are stored on the source or target systems |
|
| ||||
Target File Name | Target Settings | ${TargetFileFullName} | ${TargetFileRelativeName} | ${TargetFileBaseName} | ${TargetFileParentFullName} | ${TargetFileParentBaseName} |
The name of a file on the target host. |
|
|
|
|
|
|
|
|
|
|
|
| |
Target Transfer File Name | Target Settings | ${TargetTransferFileFullName} | ${TargetTransferFileRelativeName} | ${TargetTransferFileBaseName} | ${TargetTransferFileParentFullName} | ${TargetTransferFileParentBaseName} |
During transfer of a file its name can temporary differ: files can be prefixed or suffixed to indicate temporary names.
of the target |
|
|
|
|
|
|
|
|
|
|
|
| |
Source File Name | Source Settings | ${SourceFileFullName} | ${SourceFileRelativeName} | ${SourceFileBaseName} | ${SourceFileParentFullName} | ${SourceFileParentBaseName} |
The name of a file on the source host.
|
|
|
|
|
| |
Source File Renamed Name | Source Settings | ${SourceFileRenamedFullName} | ${SourceFileRenamedRelativeName} | ${SourceFileRenamedBaseName} | ${SourceFileRenamedParentFullName} | ${SourceFileRenamedParentBaseName} |
The name of a file on the source host after Rename Operation. |
| No values are provided for the variables | ||||
|
|
|
|
|
|
Deprecated Variables
See
Jira | ||||||||
---|---|---|---|---|---|---|---|---|
|
- File separator (all paths) are provided as follows
\
: if the pre- and post-processing commands are executed on a Windows machine./
: if the pre- and post-processing commands are not executed on a Windows machine.
- All directory paths provide the path with a trailing slash or backslash.
Variables:
- Directory variables:
${TargetDirName}
${SourceDirName}
- Target file variables:
$TargetFileName
${TargetFileName}
- Target transfer file variables:
$TargetTransferFileName
${TargetTransferFileName}
- Source file variables:
$SourceFileName
${SourceFileName}
$SourceTransferFileName
${SourceTransferFileName}
- Source renamed file variables:
$RenamedSourceFileName
${RenamedSourceFileName}
Directory | ${SourceDirName} | ${TargetDirName} | ||
---|---|---|---|---|
The directory where files are stored on the source or target systems |
|
| ||
Target File Name | $TargetFileName | ${TargetFileName} | $TargetTransferFileName | ${TargetTransferFileName} |
The name of a file on the target host (TargetFileName) without replacements being applied. During transfer of a file its name can temporarily differ (TargetTransferFileName): files can be prefixed or suffixed to indicate temporary names. Depending on the variables used absolute and relative paths are provided for files. |
|
| During transfer files can be prefixed or suffixed to indicate temporary names.
|
|
The target file name including replacements applied with the following sample settings:
Replacements are not applied to to temporary names of files during transfer. |
|
|
|
|
Source File Name | $SourceFileName | ${SourceFileName} | $SourceTransferFileName | ${SourceTransferFileName} |
The name of a file on the source host without replacements being applied. During transfer of a file the path is tripped from the file name. |
|
|
|
|
No replacements are applied to source and target file name variables. |
|
|
|
|
Source File Name after Rename Operation | $RenamedSourceFileName | ${RenamedSourceFileName} | Note | |
If no source replacement is specified then no values are provided fore the variables. | ||||
The source file name with replacements being applied that include the following sample settings:
|
|
|
|
6.5.10.1 Commands for the transfer
Commands for the transfer
6.5.10.1.1 pre_transfer_commmands - pre_transfer_commmands
Data-Type: SOSOptionCommandString
The default value for this parameter is .
6.5.10.1.2 post_transfer_commmands - commands, which has to be executed after the transfer ended
Data-Type: SOSOptionCommandString
The default value for this parameter is .
6.5.10.1.3 Cumulate File Parameters
With the options described in this chapter one can be determined that the files from the resultlist not should be transferred separately, but are to be combined prior to transfer to another, further file (target file) consecutively.
The name of the file is set with the parameter cumulative_filename.
With the parameter cumulative_file_separator is set a text that will be inserted between the individual files in the target file.
The files can be deleted after transfer.
6.5.10.1.3.1 cumulate_files - cumulate (all) files into a single file
This option determines whether the individual files from the results list should be individually transferred and then cumulated (i.e. packed together in a single file) at the transfer target.
The name of the cumulative file is specified using the cumulative_filename parameter.
The cumulative_file_separator parameter is used to specify a text that is placed in the target file, between the individual files, in order to be able to separate these files.
The individual files can be deleted after transfer if required. This is determined using the cumulative_file_delete or remove_files parameters.
Data-Type: SOSOptionBoolean
The default value for this parameter is false.
6.5.10.1.3.2 cumulative_filename - cumulate (all) files into a single file
This option determines whether the individual files from the results list should be individually transferred and then cumulated (i.e. packed together in a single file) at the transfer target.
The name of the cumulative file is specified using the cumulative_filename parameter.
The cumulative_file_separator parameter is used to specify a text that is placed in the target file, between the individual files, in order to be able to separate these files.
The individual files can be deleted after transfer if required. This is determined using the cumulative_file_delete or remove_files parameters.
Data-Type: SOSOptionOutFileName
The default value for this parameter is .
Use together with parameter: cumulative_file_separator, cumulative_file_delete, cumulate_files
6.5.10.1.3.3 cumulative_file_separator - cumulate (all) files into a single file
This option causes files from the results list to be cumulated together into a single target file, one after the other, before being transferred. This is in contrast to the default transfer mode in which files are transferred individually.
The name of the cumulative file is set using the cumulative_filename parameter.
The cumulative_file_separator parameter is used to specify a text that is placed in the target file, between the individual files, in order to be able to separate these files.
The individual files can be deleted after transfer if required. This is determined using the cumulative_file_delete or remove_files parameters.
Data-Type: SOSOptionString
The default value for this parameter is .
Use together with parameter: cumulative_filename, cumulative_file_separator, cumulative_file_delete, cumulate_files
6.5.10.1.3.4 cumulative_file_delete - cumulate (all) files into a single file
This option determines whether the individual files from the results list should be individually transferred and then cumulated (i.e. packed together in a single file) at the transfer target.
The name of the cumulative file is specified using the cumulative_filename parameter.
The cumulative_file_separator parameter is used to specify a text that is placed in the target file, between the individual files, in order to be able to separate these files.
The individual files can be deleted after transfer if required. This is determined using the cumulative_file_delete or remove_files parameters.
Data-Type: SOSOptionString
The default value for this parameter is .
Use together with parameter: cumulative_filename, cumulative_file_separator, cumulative_file_delete, cumulate_files
6.5.10.1.3.5 Examples: Cumulate Files
[cumulate_using_cumulative_file] include=local_source_host,local_target_host,jadeHistory file_spec=^.*\.txt$ operation=copy source_dir=R:/backup/sos/java/junittests/testdata/JADE/ target_dir=c:/temp/a cumulate_files=true cumulative_file_name=t.1 cumulative_file_separator=--- File: %{SourceFileName} --cumulative_file_delete=true |
Profile "sample_transfer
6.5.10.1.4 Compressing Files
Compressing Files
6.5.10.1.4.1 compress_files
This parameter specifies whether or not the content of the source fil es should be compressed by using a
zip-algorithm.
In case of sending files each files which has to be selected for sending will be compressed in a single zip file. The
extension of the file-name is defined with the parameter compress_filename_extension.
A gzip-compatible compression is used, no further software components are required.
If compress_files is used the parameter append_files is not allowed.
Data-Type: SOSOptionBoolean
The default value for this parameter is false.
6.5.10.1.4.2 compressed_file_extension -
This parameter specifies the file extension should target file compression be specified using the compress_files parameter.
Data-Type: SOSOptionString
The default value for this parameter is .gz.
6.5.10.1.4.3 decompress_after_transfer - Unpack Files after
Data-Type: SOSOptionString
The default value for this parameter is false.
6.5.10.2 Compressing Files
Commands for each object
6.5.10.2.1 Pre_Command - Command to be executed before transfer
Data-Type: SOSOptionString
The default value for this parameter is .
6.5.10.2.2 Post_Command - Command to be executed after transfer
Data-Type: SOSOptionString
The default value for this parameter is .