Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Links added

...

This job is creating a result set. A result set contain the names of all files which are selected as specified by the filter criteria. The content of the result set is returned as a parameter, but can be written to a file, too.
Parameters, which are useful for creating a result-set, are:

ParameterNameParameter Name

TitleDescription

raise_error_if_result_set_is

 Raise error on expected size of result-set

result_list_file 

Name of the result-list file

expected_size_of_result_set

 Number of expected hits in result-list

on_empty_result_set

 Set next node on empty result set

scheduler_sosfileoperations_resultset 

The result of the operation as a list of items

scheduler_sosfileoperations_resultsetsize 

The amount of hits in the result set of the operation

An example for a job-xml file:

Code Block
  <job order='no' >
     <params>
       <param name="[[#file|file]]" value="." />
       <param name="[[#file_spec|file_spec]]" value="" />
       <param name="[[#gracious|gracious]]" value="false" />
       <param name="[[#max_file_age|max_file_age]]" value="0" />
       <param name="[[#min_file_age|min_file_age]]" value="0" />
       <param name="[[#max_file_size|max_file_size]]" value="-1" />
       <param name="[[#min_file_size|min_file_size]]" value="-1" />
       <param name="[[#skip_first_files|skip_first_files]]" value="0" />
       <param name="[[#skip_last_files|skip_last_files]]" value="0" />
       <param name="[[#count_files|count_files]]" value="false" />
       <param name="[[#create_order|create_order]]create_order" value="false" />
       <param name="[[#create_orders_for_all_files|create_orders_for_all_files]]file" value="false" />
       <param name="[[#order_jobchain_name|order_jobchain_name]]" value="" />
       <param name="[[#next_state|next_state]]" value="" />
       <param name="[[#on_empty_result_set|on_empty_result_set]]" value="empty" />
       <param name="[[#expected_size_of_result_set|expected_size_of_result_set]]" value="0" />
       <param name="[[#raiseraise_error_if_result_set_is|raise_error_if_result_set_is]]" " value="0" />
       <param name="[[#result_list_file|result_list_file]]" value="empty" />
     </params>
     <script language="java" java_class="sos.scheduler.file.JobSchedulerExistsFile" />
  </job>

This job can be used standalone, as a single job, or as an order driven job in a jobchain as a jobchain node. Parameters are respectively accepted as job- or as order-parameters.
A job can process multiple parameters that are analyzed when the job starts. Parameters are defined in the configuration of the job or of the order. Parameters can also be submitted by API methods. Parameters are optional or mandatory and may contain default values.
This job is creating a result set. A result set contain the names of all files which are selected as specified by the filter criteria. The content of the result set is returned as a parameter, but can be written to a file, too.
Parameters, which are useful for creating a result-set, are

ParameterName

Title

raise_error_if_result_set_is

 

result_list_file

 

expected_size_of_result_set

 

on_empty_result_set

 

scheduler_sosfileoperations_resultset

 

scheduler_sosfileoperations_resultsetsize

 

This job can create file-orders. It can be specified that a file-order is created for the first file of the result set only or for all files of the result seta file, too.

Parameter Definitions

Parameters Used by JobSchedulerExistsFile

 Name

TitleDescription

Mandatory

Default

file

File or Folder to watch for

true

.

file_spec

Regular Expression for filename filtering

false

 

gracious

Specify error message tolerance

false

false

max_file_age

Maximum age of a file

false

0

min_file_age

Minimum age of a file

false

0

max_file_size

Maximum size of a file

false

-1

min_file_size

Minimum size of one or multiple files

false

-1

skip_first_files

Number of files to remove from the top of the result-set

false

0

skip_last_files

Number of files to remove from the bottom of the result-set

false

0

count_files

Return the size of resultset

false

false

create_order

Activate file-order creation

false

false

create_orders_for_all_files

Create a file-order for every file in the result-list

false

false

create_orders_for_new_files

Create a file-order for every new file in the result-list 

falsefalse
param_name_file_path The name of the parameter that contains the name of the file to be transferredfalse---

order_jobchain_name

The name of the jobchain which belongs to the order

false

 

next_state

The first node to execute in a jobchain

false

 

merge_order_parameterMerge actual order parameter into new created orderfalsefalse

on_empty_result_set

Set next node on empty result set

false

empty

expected_size_of_result_set

Number of expected hits in result-list

false

0

raise_error_if_result_set_is

Raise error on expected size of result-set

false

0

result_list_file

Name of the result-list file

false

empty

check_steady_state_of_files

Check the completeness of a file (steady state)

falsefalse
steady_state_countMaximum Number of Checkpointsfalse30
check_steady_state_intervalTemporal distance between checkpointsfalse1

...

Regular Expression for filename filtering
Regular Expression for file filtering. The behaviour is CASE_INSENSITIVE.
Only effective if the parameter file is a directory.
Some remarks on regular expression, as used in JobScheduler. :

  • A regular expression is not a wildcard . To get an impression of the differences one have a look on the meaning of the wildcard .txt, which will select all filenames with the filename-extension ".txt". A regular expression to match, e.g. works the same way, this "wildcard" must look like "^.\.txt$". That looks a little bit strange but it is much more flexible and powerfull on filtering filenames than the concept of wildcards, if one want to filter more complex names or pattern.
  • The general syntax of an regular expression , also referred to as regex or regexp, is described here . It is different to other RegExp definitions, e.g. as for Perl.

Data-Type : SOSOptionRegExp

...

The result set is sorted according to the used constraining parameters used: +

  • min_file_age, max_file_age: in ascending order by date of last modification, the newest file first.

...

  • min_file_size, max_file_size: in ascending order by file size, the smallest file first.

+ if If parameters for file age as well as file size are given the set is sorted by file age.

...

DataType: SOSOptionBoolean

Default: false

Anchor
param_name_file_path
param_name_file_path
Parameter param_name_file_path:

...

The name of the parameter containing the name of the file to be transferred

...

 The name of the parameter containing the name of the file to be transferred

This parameter sets the name of the parameter that contains the name of of the transferred file. The default value is scheduler_file_path. The name should be changed from the default if it is not desired to create file_orders that have to handle a file sink.

...

The name of the jobchain which belongs to the order
The name of the job chain which has to be launched by the order is the value of this parameter.
One must take into account, that the name of the jobchain must contain a subfolder structure if the jobchain is not in the folder "live". An example: the jobchain "Test" is located in "live/sample/FileOperations/". The value which has to be specfied is then "/sample/FileOperations/Test".
Data-Type : SOSOptionString
Use together with parameterparameters:

Anchor
next_state
next_state
Parameter next_state: The first node to execute in a jobchain

...

The first node to execute in a jobchain
The name of the node of a jobchain, with which the execution of the chain must be started, is the value of this parameter.
Data-Type : SOSOptionJobChainNode
Use together with parameterparameters:

Anchor
merge_order_parameter
merge_order_parameter
Parameter merge_order_parameter: Merge actual order parameter into new created order  

...

Data-Type : SOSOptionInteger
The default value for this parameter is 0.
Use together with parameter:

Anchor
raise_error_if_result_set_is
raise_error_if_result_set_is
Parameter raise_error_if_result_set_is: Raise error on expected size of result set

...

Data-Type : SOSOptionRelOp
The default value for this parameter is 0.
Use together with parameter:

Anchor
result_list_file
result_list_file
Parameter result_list_file: Name of the result list file

...

Data-Type : SOSOptionstring
The default value for this parameter is empty.
Use together with parameter:

...

...

Anchor
scheduler_sosfileoperations_resultsetsize
scheduler_sosfileoperations_resultsetsize
Parameter scheduler_sosfileoperations_resultsetsize: The amount of hits in the result set of the operation

...

Data-Type : SOSOptionsInteger
The default value for this parameter is empty.
Use together with parameter:

...

...

Anchor
scheduler_sosfileoperations_file_count
scheduler_sosfileoperations_file_count
Parameter scheduler_sosfileoperations_file_count: Return the size of the result set after a file operation

...

Data-Type : SOSOptionInteger
The default value for this parameter is 0.
Use together with parameter:

...

...