Page History

...

A job should monitor the size of a file, e.g. a log file.
If the file size is not steady after some predefined timeout then an error should be raised or a predefined exit code or order state should be set.
File size monitoring is implemented by a Monitor script that can be used by any job in a job chain.

Download, Installation and

...

Operation

Download
- Download: file_size_monitor.zip
Installation
- Unzip the archive to the ./config/live folder of your JobScheduler Master.

...

- A job chain job_chain1 is included with two jobs job1 and job2 and an order order1 with respective parameter settings.
- The File Size Monitor is provided with the monitor_file_change monitor.

Operation
- Modify the parameter monitor_file_change_location of order1 to point to a file in your environment.
- Start order1 from the JOC user interface.
  - Immediately modify and save the monitored file. Changes of the file size are checked every 10s for a max. duration of 30s.
  - After 30 seconds order1 will either be successfully moved to job2 or it will fail depending on the monitored file being considered steady within the given timeout.

Configuration

The File Size Monitor can be configured from job or order parameters.

The following parameters are used:

Parameter	Required	Default	Description
`monitor_file_change_location`	yes		The full path of a file that should be monitored.
`monitor_file_change_timeout`	yes		The max. duration in seconds that repeated checks for steady file size are performed.
`monitor_file_change_interval`	no	1	Specifies the number of seconds that the monitor sleeps between repeated checks of the file size.
`monitor_file_change_exit_code`	no	1	Optionally sets the exit code of the current job to the specified value if the monitored files is not being found steady. This works for shell jobs exclusively.
`monitor_file_change_error_state`	no		Optionally sets the order to the specified state if the monitored file is not being found steady

Implementation

Order Implementation

An order can be used to carry the parameters for the File Size Monitor.

The order can be implemented like this:

Code Block

language	xml
title	Order Implementation Sample
linenumbers	true
collapse	true

<order>
    <params >
        <param  name="monitor_file_change_location" value="/tmp/jobscheduler/file/some_changing_file.txt"/>
        <param  name="monitor_file_change_timeout" value="30"/>
        <param  name="monitor_file_change_error_state" value="error"/>
    </params>

    <run_time />
</order>

Job Implementation

A job that makes use of the File Size Monitor can reference the monitor by the <monitor.use> element.

The job can be implemented as a shell job or as an API job for any of the supported languages, e.g.

Code Block

language	xml
title	Job Implementation Sample
linenumbers	true
collapse	true

<job  order="yes" stop_on_error="no">
    <script  language="shell">
        <![CDATA[
echo hello world
        ]]>
    </script>

    <monitor.use  ordering="0" monitor="monitor_file_change"/>

    <run_time />
</job>

Explanations

...

- - line 2 to 6: implements the job script
  - line 8: references the monitor script

Monitor Implementation

The File Size Monitor is implemented as follows:s

Code Block

language	js
title	Monitor implementationImplementation
linenumbers	true
collapse	true

function spooler_process_before() {

    var rc = true;

    // merge parameters from task and order
    var params = spooler_task.params;
    params.merge( spooler_task.order.params );

    // required parameters
    var monitorLocation = params.value( 'monitor_file_change_location' );
    var monitorTimeout = params.value( 'monitor_file_change_timeout' );

    // optional parameters
    var monitorInterval = params.value( 'monitor_file_change_interval' );
    var monitorExitCode = params.value( 'monitor_file_change_exit_code' );
    var monitorErrorState = params.value( 'monitor_file_change_error_state' );
 
    if ( !monitorLocation ) {
        spooler_log.error( '.. parameter missing: monitor_file_change_location' );
        return false;
    }

    if ( !monitorTimeout ) {
        spooler_log.error( '.. parameter missing: monitor_file_change_timeout' );
        return false;
    } else {
        monitorTimeout = parseInt( monitorTimeout );
    }

    if ( !monitorInterval ) {
        monitorInterval = 1;
    } else {
        monitorInterval = parseInt( monitorInterval );
    }

    if ( !monitorExitCode ) {
        monitorExitCode = 1;
    } else {
        monitorExitCode = parseInt( monitorExitCode );
    }

    // let's use a Java package for file operations
    var monitorFile = new java.io.File( monitorLocation );
    if ( !monitorFile.isFile() ) {
        spooler_log.error( 'file does not exist: ' + monitorLocation );
        return false;
    } else {
        spooler_log.info( '.. monitoring file for max. ' + monitorTimeout + ' seconds: ' + monitorLocation );
    }

    var currentDuration = 0;
    var fileSize = 0;
    var lastFileSize = 0;

    do {
        lastFileSize = monitorFile.length();
        spooler_log.info( '.. current file size: ' + lastFileSize + ' bytes' );
        java.lang.Thread.sleep( monitorInterval * 1000 );
        fileSize = monitorFile.length();
        currentDuration = currentDuration + monitorInterval;
       } while ( (fileSize != lastFileSize) && (currentDuration < monitorTimeout) )

    if ( fileSize === lastFileSize ) {
        spooler_log.info( '.. file size is steady after ' + currentDuration + ' seconds (' + fileSize + ' bytes): ' + monitorLocation );
    } else {
        spooler_log.info( '.. giving up as file size is not steady after ' + currentDuration + ' seconds (' + fileSize + ' bytes): ' + monitorLocation );
        rc = false;

        if ( monitorExitCode ) {
            spooler_log.info( '.. setting task exit code: ' + monitorExitCode );
            spooler_task.exit_code = monitorExitCode;
        }

        if ( monitorErrorState ) {
            spooler_log.info( '.. moving order to error state: ' + monitorErrorState );
            spooler_task.order.state = monitorErrorState;
        }
    }

    return rc;
}

x

Explanations
- line 1: the monitor implements the spooler_process_before() function, i.e. it is executed for orders (not for standalone jobs) before the order is processed by the associated job.
- line 6-7: the monitor parameters are merged from task and order parameters. Precedence is granted to order parameters.
- line 10-16: parameter values are assigned to local variables.
- line 18-28: the required parameters are checked and an error is raised if parameters are missing.
- line 30-40: default values for optional parameters are assigned.
- line 43-49: the monitor makes use of the Java File class as JavaScript does not provide file operations. Integrating Java with JavaScript works seemlessly as both are provided by the JVM. This integrations allows to use any Java classes from JavaScript. The monitor checks the existence of the monitored file and will raise an error if the file does not exist.
- line 51-61: a loop is implemented that considers changes in the file size for the max. duration of repeated checks. Between checks a sleep interval is implemented.
- line 63-78: finally if the file size is found to be steady the respective informative message is written to the log and otherwise the return code is set accordingly.

Space shortcuts

Page tree

Versions Compared

Old Version 2

New Version Current

Key

Download, Installation and

Operation

Configuration

Implementation

Order Implementation

Job Implementation

Monitor Implementation