Page History

Events

Question: How can I trigger a job or a jobchain job chain based on a (series of) event(s)?

Introduction

In some cases it is necessary to trigger a job (or a jobchainjob chain) based on (multiple) events (other jobs, monitored files...) which might even occur on different hosts.

JobScheduler provides a solution to set off events and to handle these events on an active instance of the eventservice event service JobScheduler.
To demonstrate You find the complete documentation in scheduler_events.pdf - Documentation. Especially how to define event handlers with JOE is described in chapter 4.1.3 XML Event Handlers
To show the usage of events we will draw up a simple scenario:

1. start a single job which will do something,
2. watch a folder for an incoming file with the extension ".txt" and start at job to process and/or move this file
3. watch the same folder for an incoming file with the extension ".pdf" and start a jobchain job chain to process and move this file

after After these three actions are executed, in a randomly sequence, and successfully finished, another jobchain job chain must be started and all events must be cleared.

For this example, two different JobScheduler instances may be configured.
An eventservice event service (or event - supervisor) and a workload Workload JobScheduler which is registered with that eventserviceevent service.
The eventservice event service is able to receive events from other JS JobScheduler instances on different servers and platforms.
It is possible also to have the eventservice event service and the workload Workload JobScheduler running in the same JobScheduler instance.
If one will use different JS different JobScheduler instances were used for eventing then it is needed required that these both JS instances can communicate.

Requirements

The JobScheduler must run using a database. The events are stored permanently in the table "SCHEDULER_EVENTS". To customize an eventhandler event handler some experience with xml XML and xslt XSLT are required.

Instructions

• Unzip all files of events.zip or events_windows.zip into the ./config/live folder of your workload Workload JobScheduler installation. After unzipping you will have in the subfolder "samples" an subfolder "events" with the required JS JobScheduler objects to run the example.
• Edit ./config/scheduler.xml xml of the workload Workload JobScheduler and configure a JobScheduler as a supervisorSupervisor. The supervisor Supervisor is an attribute of the tag config. See supervisor Supervisor.
• In the installation directory of your workload JobScheduler create a folder files with two subfolders in and processed
• Unzip all files of events_supervisor.zip into the ./config folder of your active instance of the eventservice event service JobScheduler installation. After unzipping you will have in the subfolder "./config/live/sos" a subfolder "events" with all the objects needed required to run the eventserviceevent service. And a folder "./config/events" which that is used by the eventhandlerevent handler.
• Edit config/events/example.event_class.xsl on the active instance of the eventservice event service JobScheduler and adjust the host and port parameters to those of the workload Workload JobScheduler. Copy the file to the folder ./config/events.
   

          <xsl:template match="events[event[@job_name='simple_shell_job'] and event[@job_chain='txt_chain'] and event[@job_chain='pdf_chain']]">
            <xsl:call-template '''name="run_job"'''>
              <xsl:with-param name="job">'''samples/events/done_job<'''/xsl:with-param>
              <xsl:with-param '''name="host">localhost'''</xsl:with-param>
              <xsl:with-param '''name="port">4444'''</xsl:with-param>
            </xsl:call-template>
            <remove_event>
              <event event_class="example"/>
            </remove_event>
          </xsl:template>

The config/events/example.event_class.xsl file, which is shown above, customizes an event handler which will start the job samples/events/done_job
if all three expected events have arrived.

In the next steps , these events will be initiated:

• Open the JS operations gui JobScheduelr Operations GUI of the workload Workload JobScheduler in your browser using http://scheduler_host:scheduler_port
• Find the job samples/events/simple_shell_job and click on it.:
   

          <job>
            <script language="shell">
                <![CDATA[
          # run any shell script or command(s)
          echo hello world!
          
          # submit event to active instance of the eventservice JobScheduler
          $\{SCHEDULER_HOME\}/bin/jobscheduler_event.sh -x $? -e "example"
                ]]>
            </script>
            <run_time/>
          </job>

• In the Job menu press "Start task now" (1st event)
• Put a .txt file in the files/in directory. This should cause samples/events/file_job to run and move the file to the files/processed directory. (2nd event):
   

          <job order="yes">
              <script language="shell">
                  <![CDATA[
          # run any shell script or command(s)
          echo processing file $SCHEDULER_PARAM_SCHEDULER_FILE_PATH
          
          mv $SCHEDULER_PARAM_SCHEDULER_FILE_PATH files/processed
          
          # submit event to Supervisor Job JobScheduler
          $\{SCHEDULER_HOME\}/bin/jobscheduler_event.sh -x $? -e "example"
                  ]]>
              </script>
              <run_time/>
          </job>

• Put a .pdf file in the files/in directory.
  This will cause samples/events/file_job to run and move the file to the files/processed directory. (3rd event)

When all three events (comming coming from simple_shell_job, .txt file, .pdf file) were thrown, the job samples/events/done_job will immediately start and executes execute the requested action: delete the files from the files/processed directory.

        <job name="done_job">
            <script language="shell">
                <![CDATA[
        echo content of files/processed:
        ls -la files/processed
        
        echo deleting contents of files/processed
        rm -f files/processed/*
                ]]>
            </script>
            <run_time/>
        </job>

How it works

The job samples/events/simple_shell_job calls the script

  {{$\{SCHEDULER_HOME\}/bin/jobscheduler_event.sh -x $? -e "example"}}

to signal an event of call "example" to the active instance of the eventservice even tservice JobScheduler.
The -x switch sets the exit code of the previous shell - command (echo in this case), and the -e switch sets the event class as properties of the event.
Other properties (which do not need to be set as parameters for the script jobscheduler_event.sh) include the name of the job which that created the event, the name of the jobchain job chain (if the job was called in a chain), name of the order ..etc.

Similarly the samples/events/file_job creates events which that are collected in the active instance of the eventservice event service JobScheduler.
The samples/events/file_job is used in two separate jobchainsjob chains: samples/events/txt_chain (which triggers .txt files) and samples/events/pdf_chain (which triggers .pdf files).
Thus, two more events are submitted to the JobScheduler , with the event class example and with two different jobchain job chain properties.

Each time the active instance of the eventservice event service JobScheduler receives an event it runs the JobScheduler_event_service job .
The job will then process all event handlers which that follow the naming scheme:

 *[_job_name_].*.job.xsl will be run for events created by the job [_job_name_]
 *[_job_chain_name_].*.job_chain.xsl will be run for events created by a job in the job jobchainchain [_job_chain_name_]
 *[_event_class_name_].*.event_class.xsl will be run for events with the event class [_event_class_name_]

In this example, all events are assigned to the event class example, so for each event the eventhandler (aka the xsl event handler (i.e.the XSLT stylesheet) config/events/example.event_class.xsl is executed and parsed by a running job of the active instance of the eventservice event service JobScheduler.

This stylesheet processes an xml which object that contains all events currently known to the active instance of the eventservice event service JobScheduler, e.g.:

   <events current_date="2011-08-06 11:14:01" expiration_date="2011-08-06 23:14:01">
     <event created="2011-08-06 11:13:29" event_class="example"
       event_id="0" exit_code="0" expires="2011-08-06 23:13:30"
       job_chain="" job_name="simple_shell_job" order_id=""
       remote_scheduler_host="wilma" remote_scheduler_port="4444" JobScheduler_id="scheduler.supervisor"/>
     <event created="2011-08-06 11:14:01" event_class="example"
       event_id="0" exit_code="0" expires="2011-08-06 23:14:01"
       job_chain="txt_chain" job_name="file_job"
       order_id="files/in/sample2.txt" remote_scheduler_host="wilma"
       remote_scheduler_port="4444" JobScheduler_id="scheduler.supervisor"/>
    </events>

The event handler (stylesheet) has a template which that tries to find 3 events, one from the simple_shell_job job, and one from each of the file handling jobchainsjob chains:

   <xsl:template match="events[
                     event[@job_name='simple_shell_job'] and
                     event[@job_chain='txt_chain'] and event[@job_chain='pdf_chain']
                     ]">

If these events are not found, then nothing happens. The events remain in the active instance of the eventservice event service JobScheduler.
If the events are found , then the event handler stylesheet returns two commands to the active instance of the eventservice event service JobScheduler:

The first command starts the samples/events/done_job on the workload Workload JobScheduler:

     <xsl:call-template name="run_job">
       <xsl:with-param name="job">samples/events/done_job</xsl:with-param>
       <xsl:with-param name="host">localhost</xsl:with-param>
       <xsl:with-param name="port">4444</xsl:with-param>
     </xsl:call-template>

...

The second command cleans up the current set of events of the event class example to avoid triggering samples/events/done_job
again before all 3 events have occuredoccurred:

     <remove_event>
       <event event_class="example"/>
     </remove_event>
 

The scheduler_event_functions.xsl.inc stylesheet

It is recommended to include scheduler_event_functions.xsl.inc into your own event handler stylesheets by adding:

     <xsl:import href="scheduler_events.xsl.inc"/>

...

• that a root level <commands> tag (element) is created in the resulting xml and
• that the stylesheet will return a valid (and well formed) xml result (returning nothing would result in an error)

Examples for match statements in event handlers

        {{<xsl:template match="events[event[@job_name='simple_shell_job' and @exit_code=0]]">}}

Matches if an event exists for job simple_shell_job with exit code 0.

        {{<xsl:template match="events[event[@job_name='simple_shell_job' and @exit_code>0]]">}}

Matches if an event exists for job simple_shell_job with an exit code greater than 0.

        {{<xsl:template match="events[event[@job_name='simple_shell_job'] and not(event[@job_name='file_job'])]">}}

Matches if an event exists for job simple_shell_job but no event for job file_job.

        {{<xsl:template match="events[event[@job_name='simple_shell_job'] or count(event[@event_class='other_events'])>=3]">}}
        

Matches if an event exists for job simple_shell_job or at least three events of event class other_events.

    {{<xsl:template match="events[event[@job_name='simple_shell_job']/params/param[@name='foo' and @value='bar'] ]">}}
  

Matches if an event exists for job simple_shell_job with an event parameter "foo" with the value "bar".

Downloads:

• scheduler_events.pdf - Documentation

• events.zip - job configuration files
• events_windows.zip - job configuration files (windows version)
• events_supervisor.zip - event handler files