...
- JobScheduler's
aquire_lockjob matches files using regular expressions and determines the file's category - for example, Berlin or Munich.No Format The regular expressions are also defined in the "File Order Sources" area shown in the screenshot. Note that for simplicity the regular expressions match the prefixes "a" and "b" in the file names and not directly "Berlin" or "Munich". The {{aquire_lock}} job uses a Rhino JavaScript to try to aquire the lock and either wait if the lock is not available or proceed to the next node if it is. This script is listed below.:
| Code Block | ||
|---|---|---|
| ||
function spooler_process() \{
try \{
var parameters = spooler_task.order().params();
var filePath = "" + String(parameters.value("scheduler_file_path"));
spooler_log.info( " scheduler_file_path : " + filePath );
//
var param1 = "" + String(parameters.value("param1"));
spooler_log.info( " param1 : " + param1 );
//
var fileParts = filePath.split("\\");
var fileName = fileParts[fileParts.length-1];
spooler_log.info( "fileName : " + fileName );
if(fileName.match("^a[A-Za-z0-9_]*\.csv$")) \{
var lockName = "BERLIN_PROC";
var lock_name = "BERLIN_PROC";
spooler_log.info( "File matched with berlin lock_name : "+ lockName );
\}
if(fileName.match("^b[A-Za-z0-9_]*\.csv$")) \{
var lockName = "MUNICH_PROC";
spooler_log.info( "File matched with berlin lock_name : "+ lockName );
\}
spooler_task.order().params().set_value("file_spec",fileName);
spooler_task.order().params().set_value("lock_name",lockName);
if (!spooler.locks().lock_or_null( lockName )) \{
var lock = spooler.locks().create_lock();
lock.set_name( lockName );
spooler.locks().add_lock( lock );
if (spooler_task.try_hold_lock( lockName )) \{
return true;
\} else \{
spooler_task.call_me_again_when_locks_available();
\}
\} else \{
spooler_task.order().setback();
spooler_log.info( " lock is already aquired , lock_name : "+ lockName +" , setback , will retry after some time");
\}
return true;
\} catch (e) \{
spooler_log.warn("error occurred : " + String(e));
return false;
\}
\}
|
- Once aquire_lock finds the matching category it will try to set a semaphore (flag) using JobScheduler's inbuilt LOCK mechanism
- Only one instance of each LOCK is allowed as can be seen in
Max Non Exclusiveparameter shown in the screenshot of JobScheduler's JOE interface below:
- below. Once a LOCK has been assigned to a file from a category (either Berlin or Munich), all subsequent files for this category haves to wait with a setback until the LOCK has been freed.
- The same mechanism will be repeated for files from other categories. As long as a file of any given category is not being processed and therefore the corresponding LOCK not been set, the way will be free for the file from the other category to be allowed to be processed.
No Format This can be seen in the following screenshot of JobScheduler'sthe JOEJOC interface showing the progression of file orders along the {{load_files}} job chain.
...
The following screenshot from the JobScheduler's JOE interface showshows the nodes in the load_files job chain and their related states:
| Code Block | ||
|---|---|---|
| ||
function spooler_process() \{
try \{
var parameters = spooler_task.order().params();
var filePath = "" + String(parameters.value("scheduler_file_path"));
spooler_log.info( " scheduler_file_path : " + filePath );
var lockName = "" + String(parameters.value( "lock_name" ));
spooler_log.info( " lock_name : " + lockName );
/*
var fileParts = filePath.split("\\");
var fileName = fileParts[fileParts.length-1];
spooler_log.info( "fileName : " + fileName );
if(fileName.match("^a[A-Za-z0-9_]*\.csv$")) \{
var lockName = "BERLIN_PROC";
var lock_name = "BERLIN_PROC";
spooler_log.info( "File matched with berlin lock_name : "+ lockName );
\}
if(fileName.match("^b[A-Za-z0-9_]*\.csv$")) \{
var lockName = "MUNICH_PROC";
spooler_log.info( "File matched with berlin lock_name : "+ lockName );
\}
*/
if (spooler.locks().lock_or_null( lockName )) \{
spooler.locks().lock( lockName ).remove();
\}
return true;
\} catch (e) \{
spooler_log.warn("error occurred: " + String(e));
return false;
\}
\}
|
...