How can I trigger a job or a jobchain based on a (series of) event(s)?

Events

Question: How can I trigger a job or a job chain based on a (series of) event(s)?

Introduction

In some cases it is necessary to trigger a job (or a job chain) based on (multiple) events (other jobs, monitored files...) which might even occur on different hosts.

JobScheduler provides a solution to set off events and to handle these events on an active instance of the eventservice JobScheduler.
You find the complete documentation in scheduler_events.pdf - Documentation. Especially how to define event handler with JOE is described there in chapter 4.1.3 XML Event Handlers
To demonstrate the usage of events we will draw up a simple scenario:

start a single job which will do something,
watch a folder for an incoming file with the extension ".txt" and start at job to process and/or move this file
watch the same folder for an incoming file with the extension ".pdf" and start a job chain to process and move this file

after these three actions are executed, in a randomly sequence, and successfully finished, another job chain must be started and all events must be cleared.

For this example, two different JobScheduler instances may be configured.
An eventservice (or event-supervisor) and a workload JobScheduler which is registered with that eventservice.
The eventservice is able to receive events from other JS instances on different servers and platforms.
It is possible also to have the eventservice and the workload JobScheduler running in the same JobScheduler instance.
If one will use different JS for eventing it is needed that these both JS can communicate.

Requirements

The JobScheduler must run using a database. The events are stored permanently in the table "SCHEDULER_EVENTS". To customize an eventhandler experience with xml and xslt are required.

Instructions

Unzip all files of events.zip or events_windows.zip into the ./config/live folder of your workload JobScheduler installation. After unzipping you will have in the subfolder "samples" an subfolder "events" with the required JS objects to run the example.
Edit config/scheduler.xml of the workload JobScheduler and configure a JobScheduler as a supervisor. The supervisor is an attribute of the tag config. See supervisor.
In the installation directory of your workload JobScheduler create a folder files with two subfolders in and processed
Unzip all files of events_supervisor.zip into the ./config folder of your active instance of the eventservice JobScheduler installation. After unzipping you will have in the subfolder "live/sos" a subfolder "events" with all the objects needed to run the eventservice. And a folder "config/events" which is used by the eventhandler.
Edit config/events/example.event_class.xsl on the active instance of the eventservice JobScheduler and adjust the host and port parameters to those of the workload JobScheduler. Copy the file to the folder config/events.

          <xsl:template match="events[event[@job_name='simple_shell_job'] and event[@job_chain='txt_chain'] and event[@job_chain='pdf_chain']]">
            <xsl:call-template '''name="run_job"'''>
              <xsl:with-param name="job">'''samples/events/done_job<'''/xsl:with-param>
              <xsl:with-param '''name="host">localhost'''</xsl:with-param>
              <xsl:with-param '''name="port">4444'''</xsl:with-param>
            </xsl:call-template>
            <remove_event>
              <event event_class="example"/>
            </remove_event>
          </xsl:template>

The config/events/example.event_class.xsl file, which is shown above, customizes an event handler which will start the job samples/events/done_job
if all three expected events have arrived.

In the next steps, these events will be initiated:

Open the JS operations gui of the workload JobScheduler in your browser using http://scheduler_host:scheduler_port
Find the job samples/events/simple_shell_job and click on it.

          <job>
            <script language="shell">
                <![CDATA[
          # run any shell script or command(s)
          echo hello world!
          
          # submit event to active instance of the eventservice JobScheduler
          $\{SCHEDULER_HOME\}/bin/jobscheduler_event.sh -x $? -e "example"
                ]]>
            </script>
            <run_time/>
          </job>

In the Job menu press "Start task now" (1st event)
Put a .txt file in the files/in directory. This should cause samples/events/file_job to run and move the file to the files/processed directory. (2nd event)

          <job order="yes">
              <script language="shell">
                  <![CDATA[
          # run any shell script or command(s)
          echo processing file $SCHEDULER_PARAM_SCHEDULER_FILE_PATH
          
          mv $SCHEDULER_PARAM_SCHEDULER_FILE_PATH files/processed
          
          # submit event to Supervisor Job JobScheduler
          $\{SCHEDULER_HOME\}/bin/jobscheduler_event.sh -x $? -e "example"
                  ]]>
              </script>
              <run_time/>
          </job>

Put a .pdf file in the files/in directory.
This will cause samples/events/file_job to run and move the file to the files/processed directory. (3rd event)

When all three events (coming from simple_shell_job, .txt file, .pdf file) were thrown, the job samples/events/done_job will immediately start and executes the requested action: delete the files from the files/processed directory.

        <job name="done_job">
            <script language="shell">
                <![CDATA[
        echo content of files/processed:
        ls -la files/processed
        
        echo deleting contents of files/processed
        rm -f files/processed/*
                ]]>
            </script>
            <run_time/>
        </job>

How it works

The job samples/events/simple_shell_job calls the script

  {{$\{SCHEDULER_HOME\}/bin/jobscheduler_event.sh -x $? -e "example"}}

to signal an event of call "example" to the active instance of the eventservice JobScheduler.
The -x switch sets the exit code of the previous shell-command (echo in this case), and the -e switch sets the event class as properties of the event.
Other properties (which do not need to be set as parameters for the script jobscheduler_event.sh) include the name of the job which created the event, the name of the job chain (if the job was called in a chain), name of the order ...

Similarly the samples/events/file_job creates events which are collected in the active instance of the eventservice JobScheduler.
The samples/events/file_job is used in two separate jobchains: samples/events/txt_chain (which triggers .txt files) and samples/events/pdf_chain (which triggers .pdf files).
Thus, two more events are submitted to the JobScheduler, with the event class example and with two different jobchain properties.

Each time the active instance of the eventservice JobScheduler receives an event it runs the JobScheduler_event_service job .
The job will then process all event handlers which follow the naming scheme:

 *[_job_name_].*.job.xsl will be run for events created by the job [_job_name_]
 *[_job_chain_name_].*.job_chain.xsl will be run for events created by a job in the job chain [_job_chain_name_]
 *[_event_class_name_].*.event_class.xsl will be run for events with the event class [_event_class_name_]

In this example, all events are assigned to the event class example, so for each event the eventhandler (aka the xsl stylesheet) config/events/example.event_class.xsl is executed and parsed by a running job of the active instance of the eventservice JobScheduler.

This stylesheet processes an xml which contains all events currently known to the active instance of the eventservice JobScheduler, e.g.:

   <events current_date="2011-08-06 11:14:01" expiration_date="2011-08-06 23:14:01">
     <event created="2011-08-06 11:13:29" event_class="example"
       event_id="0" exit_code="0" expires="2011-08-06 23:13:30"
       job_chain="" job_name="simple_shell_job" order_id=""
       remote_scheduler_host="wilma" remote_scheduler_port="4444" JobScheduler_id="scheduler.supervisor"/>
     <event created="2011-08-06 11:14:01" event_class="example"
       event_id="0" exit_code="0" expires="2011-08-06 23:14:01"
       job_chain="txt_chain" job_name="file_job"
       order_id="files/in/sample2.txt" remote_scheduler_host="wilma"
       remote_scheduler_port="4444" JobScheduler_id="scheduler.supervisor"/>
    </events>

The event handler (stylesheet) has a template which tries to find 3 events, one from the simple_shell_job job, and one from each of the file handling job chains:

   <xsl:template match="events[
                     event[@job_name='simple_shell_job'] and
                     event[@job_chain='txt_chain'] and event[@job_chain='pdf_chain']
                     ]">

If these events are not found, nothing happens. The events remain in the active instance of the eventservice JobScheduler.
If the events are found, the event handler stylesheet returns two commands to the active instance of the eventservice JobScheduler:

The first command starts the samples/events/done_job on the workload JobScheduler:

     <xsl:call-template name="run_job">
       <xsl:with-param name="job">samples/events/done_job</xsl:with-param>
       <xsl:with-param name="host">localhost</xsl:with-param>
       <xsl:with-param name="port">4444</xsl:with-param>
     </xsl:call-template>

The run_job template is a template from the imported stylesheet scheduler_event_functions.xsl.inc which is provided to facilitate creating (basic) JobScheduler commands.

The second command cleans up the current set of events of the event class example to avoid triggering samples/events/done_job
again before all 3 events have occurred:

     <remove_event>
       <event event_class="example"/>
     </remove_event>

The scheduler_event_functions.xsl.inc stylesheet

It is recommended to include scheduler_event_functions.xsl.inc into your own event handler stylesheets by adding:

     <xsl:import href="scheduler_events.xsl.inc"/>

to the stylesheet. (see example.event_class.xsl)

This included stylesheet will ensure:

that a root level <commands> tag (element) is created in the resulting xml and
that the stylesheet will return a valid (and well formed) xml result (returning nothing would result in an error)

Examples for match statements in event handlers

        {{<xsl:template match="events[event[@job_name='simple_shell_job' and @exit_code=0]]">}}

Matches if an event exists for job simple_shell_job with exit code 0.

        {{<xsl:template match="events[event[@job_name='simple_shell_job' and @exit_code>0]]">}}

Matches if an event exists for job simple_shell_job with an exit code greater than 0.

        {{<xsl:template match="events[event[@job_name='simple_shell_job'] and not(event[@job_name='file_job'])]">}}

Matches if an event exists for job simple_shell_job but no event for job file_job.

        {{<xsl:template match="events[event[@job_name='simple_shell_job'] or count(event[@event_class='other_events'])>=3]">}}

Matches if an event exists for job simple_shell_job or at least three events of event class other_events.

    {{<xsl:template match="events[event[@job_name='simple_shell_job']/params/param[@name='foo' and @value='bar'] ]">}}

Matches if an event exists for job simple_shell_job with an event parameter "foo" with the value "bar".

Downloads:

scheduler_events.pdf - Documentation

events.zip - job configuration files
events_windows.zip - job configuration files (windows version)
events_supervisor.zip - event handler files

Space shortcuts

Page tree