Example for YADE file transfer using JITL Job

Example: File Transfer using JITL Job

The following scenario is given:

• JobScheduler reads a status information from an Oracle database table scenario1 where we store values every 5 minutes
  • For this demo we handle the table as if  there were just one row of data.

If the column value of scenario1.Status = “GO” then

• read additional FTP attributes from an database table, i.e. SourceFilePath, SourceFileName, TargetFilePath, TargetFileName, EncryptedYN.
• store FTP attributes as memory variables for later use.
• call YADE to transfer files by SFTP using its configuration from the stored memory variables.
• If SourceFile does not exist
  • send e-mail indicating the error including the file properties (source file name, source file size, source file date/time created, date/time of failure, target file name, target directory, failure detail)
• If SourceFile does exist
  • If EncryptedYN has the value Y
    • execute encryption routine
  • perform SFTP file transfer to target system
  • confirm that the file has been successfully received at the destination.
  • If the file has not been successfully received at the destination
    • send mail indicating the error that occurred including the file properties (source file name, source file size, source file date/time created, date/time of failure, target file name, target directory, failure detail)
  • If the file has been successfully received
    • send a confirmation mail including the file properties (source file name, source file size, source file date/time created, date/time of failure, target file name, target director)
• Log All transport activities
• Create PDF report of all transport activities between two dates/times

Representation of the Solution

Components of the Solution

checkDB: Access to Oracle database using pl/sql

For the connection to the Oracle™ DBMS we use the pl/sql script plsql_script.txt which is executed by the Java adapter class sos.scheduler.db.JobSchedulerPLSQLJobJSAdapterClass.

The pl/sql script is defined as:

declare
	status 			varchar2(64);
	sourcefilepath 	varchar2(2048);
	sourcefilename 	varchar2(2048);
	targetfilepath 	varchar2(2048);
	targetfilename 	varchar2(2048);
	encryptedyn 	varchar2(16);

  begin 
	select STATUS, SOURCEFILEPATH, SOURCEFILENAME, TARGETFILEPATH, TARGETFILENAME, ENCRYPTEDYN
	into   status, sourcefilepath, sourcefilename, targetfilepath, targetfilename, encryptedyn
	from scenario1;
	
	dbms_output.put_line('status=' || status);
	dbms_output.put_line('sourcefilepath=' || sourcefilepath);
	dbms_output.put_line('sourcefilename=' || sourcefilename);
	dbms_output.put_line('targetfilepath=' || targetfilepath);
	dbms_output.put_line('targetfilename=' || targetfilename);
	dbms_output.put_line('encryptedyn=' || encryptedyn);
  end;

The parameters used by the adapter class are

• name of the pl/sql script and where it is found
• name of the database and access parameters

For the interpretation of the column value of scenario1.Status being "GO" some post-processing routine is used (here programmed in javax.script:rhino)

function spooler_process_after(spooler_process_result)
{
  var params = spooler_task.order().params();
  var status = params.value("status");
					
  if ( status == "GO" )
	{
	spooler_log.info("Table status is \"GO\", continue!");
	}
  else
	{
	spooler_log.info("Waiting until table status is \"GO\" ...");
	spooler_task.order().set_state("WAITING");
	}

  return spooler_process_result;
}

CheckFileExists: check if file exists

• The file defined by the parameters SourceFilePath and SourceFileName is checked with the JITL job sos.scheduler.file.JobSchedulerExistsFile.

checkEncrypted: Interpretation of Parameter encryptedyn

The value of the parameter encryptedyn decides whether the file is to be encrypted or not. The encryption is done for a local copy of the file, therefore different steps of the job chain are used. The functions are part of the job checkEncrypted programmed in javax.script:rhino.

function spooler_process() {

	spooler_log.info("------- " + spooler_job.name() + " -------");
				
	var encryptedyn = spooler_task.order().params().value("encryptedyn");
				
	if ( "Y" == encryptedyn )
	{
		spooler_log.info("Encryption required");
		spooler_task.order().set_state("transportSourceToLocal");
	}
	else if ( "N" == encryptedyn )
	{
		spooler_log.info("No encryption required");
		spooler_task.order().set_state("transportSourceToTarget");
	}
	else
	{
		spooler_log.error("encryptedyn = \"" + encryptedyn + "\" (must be Y or N)");
		return false;
	}

	return true;
}

In case of encryption we use the following steps:

• transportSourceToLocal: copy file from source system to local system
• encrypt: encrypt file
• transportLocalToTarget: copy file from local system to target system

Without encryption only one transfer is used to copy the file from source to target in step

• transportSourceToTarget

Transport Steps

The transport steps are using the JITL job sos.scheduler.jade.JadeJob.

The following parameters are required as per step:

encrypt: encryption of the local file

In our example the encryption is only simulated. An own routine may be used here.

Report: Mailing a report

A mail is generated to send the YADE history file as an example for a report. External report features may be used here. The JITL job sos.scheduler.managed.JobSchedulerManagedMailJob is used here to manage the mail.

JobChain in JOE