| Table of Contents |
|---|
| outlinh1. true |
|---|
| outlinh1. true |
|---|
| 1 | printablefalse |
|---|
| 2 | stylh1. none |
|---|
| 3 | indent20px |
|---|
|
Scenario
In this example user want to poll for an incoming file on the ServerA, once file matching with the reguler expression is found on ServerA, JobScheduler should transfer the file(s) from ServerA to ServerB.
After successfully transfer of file on ServerB, JobScheduler should start an processing script on ServerB via SSH.
| Graphviz |
|---|
digraph "Example: Server to Server Transfer with polling" \{
rankdir=LR;
edge [
color="#31CEF0"
];
source_server [label="Data Source (FTP)"];
target_server [label="Data Target (sFTP)"];
jobscheduler [label="JobScheduler \n JADE JITL job \n SSH JITL Job"];
source_server->target_server [label="2. data transfer"];
source_server->jobscheduler [dir=both label="commands" color="grey"];
jobscheduler->source_server [dir=right label="1. polling" color="blue"];
jobscheduler->target_server [dir=both label="commands" color="grey"];
jobscheduler->target_server [dir=righ label="3. ssh command" color="red"];
} |
Solution
For such a file transfer and processing workflow, JobScheduler has built in JITL jobs, namely JADE job and SSH job.
The JADE JITL jobs can be configured to transfer data from Server-to-Server without touch down, more information about Server-to-Server transfer can be found here Server to Server Transfer.
The JADE has inbuilt polling feature which can be configured using polling parameter of JADE i.e. poll_interval : frequency of polling in seconds and poll_timeout : maximum time in seconds for polling. The job_chain can be configured by an order to run at predefined time, repeat execution or can be started manually.
...
- jade_s2s_file_transfer : for server to server data transfer and
- ssh_start_proccessing : starting the processing script using ssh
| Code Block |
|---|
|
<?xml version="1.0" encoding="ISO-8859-1"?>
<job_chain orders_recoverable="yes" visible="yes" title="Tranasfer Data from ServerA to ServerB and start Processing on ServerB vis SSH" max_orders="1" name="DataTransferProcessing">
<job_chain_node state="jade_s2s_file_transfer" job="jade_s2s_file_transfer" error_state="error" next_state="ssh_start_proccessing" on_error="suspend"/>
<job_chain_node state="ssh_start_proccessing" job="ssh_start_proccessing" next_state="success" error_state="error" on_error="suspend"/>
<job_chain_node state="success"/>
<job_chain_node state="error"/>
</job_chain>
|
...
This job is an JADE JITL job which is configured to poll source server (ftp) for file matching with file_speh1. ^TEST*\.log$ for every 30 seconds for maximum of 600 seconds.
All the matching file(s) during polling will be copy(ied) to the target server (sftp). For more information about JADE JITL job can be found here JADE_JITL_Parameters.
| Code Block |
|---|
|
<?xml version="1.0" encoding="ISO-8859-1"?>
<job title="S2S transfer with polling (API Job for JobScheduler Advanced Data Exchange)" order="yes" stop_on_error="no" name="jade_s2s_file_transfer">
<description >
<include file="jobs/jadeJob.xml"/>
</description>
<params >
<param name="file_spec" value="\.log$"/>
<param name="operation" value="copy"/>
<param name="verbose" value="9"/>
<param name="poll_interval" value="30"/>
<param name="poll_minfiles" value="1"/>
<param name="poll_timeout" value="600"/>
<param name="source_protocol" value="ftp"/>
<param name="source_host" value="mp.sos"/>
<param name="source_user" value="test"/>
<param name="source_password" value="super_secret"/>
<param name="source_dir" value="/outbound"/>
<param name="target_protocol" value="sftp"/>
<param name="target_host" value="homer.sos"/>
<param name="target_user" value="test"/>
<param name="target_password" value="super_secret"/>
<param name="target_ssh_auth_method" value="password"/>
<param name="target_dir" value="/home/test/inbound"/>
</params>
<script language="java" java_class="sos.scheduler.jade.JadeJob"/>
<run_time />
</job>
|
...
| Code Block |
|---|
|
<?xml version="1.0" encoding="ISO-8859-1"?>
<job title="Launch commands or executable files by SSH" order="yes" name="ssh_start_proccessing">
<description >
<include file="jobs/SOSSSHJob2JSAdapter.xml"/>
</description>
<params >
<param name="host" value="homer.sos"/>
<param name="port" value="22"/>
<param name="user" value="test"/>
<param name="auth_method" value="password"/>
<param name="command" value="uname -an"/>
<param name="password" value="super_secret"/>
</params>
<script language="java" java_class="sos.scheduler.job.SOSSSHJob2JSAdapter"/>
<run_time />
</job>
|
...
<mscgen>
// poll file and ssh execution
msc {
a, b, c;
}
</mscgen>