Introduction

JS7 - Notifications are managed from the Configuration->Notification sub-view of JOC Cockpit.

  • The configuration format of notifications is XML. The JOC Cockpit offers an XSD schema-aware XML Editor for configuration purposes.
  • The configuration is applied by the JS7 - Monitor Service:

    • inspects the workflow execution history and creates notifications as required,

    • receives warnings and errors from the JOC Cockpit log files and creates related notifications,
    • receives warnings and errors from the JS7 - Log Notification Service.
  • Resulting notifications are displayed with the JS7 - Monitor view in the Notifications tab. They can be sent by mail and they can be processed from the command line.
  • Changes to the configuration are immediately applied by the Monitor Service.

Notification View

The Configuration->Notification sub-view allows to manage the configuration like this:


Explanation:

  • If this view shows an empty configuration then the "New" button can be used to created a configuration.
  • Alternatively, a configuration can be added with the "Upload" button. A configuration sample is available from
    SET-212 - Getting issue details... STATUS .

Configuration

Structure and Elements

The configuration makes use of the following base elements, which are available from the Configurations root element.

  • Fragments: Define re-usable configuration blocks for Messages, Monitors and monitored Objects.
    • MessageFragments
      • Message
        • Defines the content which is sent, for example, by mail to a user or which is used to parameterize a command line utility, such as content to be forwarded to a System Monitor.
          • Messages for use with E-Mail represent the mail body, either for use with plain text or with HTML e-mails.
            • Messages for use with the Command Line represent a string that can be used with the Notifications/Notification/NotificationMonitors/CommandFragmentRef element, see below.
          • Message elements can include Monitor Variables that are placeholders for values, e.g. for the Workflow Path, Order ID etc.
          • Any number of Message elements can be added.
      • MonitorFragments. These fragments come in a number of flavors for the following notifcation types.
        • MailFragment
          • The following elements are required to send mail:
            • MessageRef: Specifies the reference to a Message element that provides the mail body.
            • Subject: Specifies the mail subject and can include Monitor Variables.
            • To: Specifies the e-mail address of the recipient. Multiple recipients can be separated by comma.
          • The following elements are optional to send mail:
            • CC: The recipient of carbon copies. Multiple recipients can be separated by comma.
            • BCC: The recipient of blind carbon copies. Multiple recipients can be separated by comma.
            • From: The e-mail address of the account that is used to send mail. Consider that your mail server configuration determines whether a specific or an arbitrary account can be used.
          • For details see JS7 - Notifications - Configuration Element MailFragment
        • CommandFragment
          • MessageRef: Specifies the reference to a Message element which provides the content that is to be forwarded with the Command element. The message content is available from the ${MESSAGE} Monitor Variable for use with later elements.
          • Command: Specifies the shell command for Linux/Windows which is used to forward notifications, to, for example, a System Monitor Agent utility.
            • The example notification view above makes use of a shell command like this:
              • echo "${MESSAGE}" >> /tmp/notification.log
            • The echo shell command appends the content of the ${MESSAGE} Monitor Variable to a file in the /tmp directory.
          • For details see JS7 - Notifications - Configuration Element CommandFragment
        • JMSFragment
        • NSCAFragment
          • This fragment type is used for direct connections to System Monitors that operate a Nagios-compatible NSCA service for passive checks such as Nagios®, Icinga®, Opsview®. This fragment type makes use of the JSend NSCA Java API.
          • The required connection attributes such as host, port, timeouts are specific to the System Monitor being used.
          • Note that JSend NSCA can be used without encryption or with an XOR or TRIPLE_DES encryption. Additional encryption schemes are not supported.
          • For details see JS7 - Notifications - Configuration Element NSCAFragment
      • ObjectFragments
        • Workflows: Any umber of workflow configurations can be added and are distinguished by a unique name that is added to this element.
          • Workflow: A specific workflow can be specified by, for example, its name. The path attribute allows a regular expression to specify a part of the workflow path.
            • WorkflowJob: This element can be used to limit notifications to specific jobs in a workflow.
              • This includes the option of specifying the job name attribute and/or its label attribute. For both attributes constant values and regular expressions can be used, for example .* to specify e-mail to be sent for any jobs.
              • For releases earlier to 2.7.1:
                • It is required that the criticality, which can be one of ALL, NORMAL or CRITICAL, is specified when using this element.
              • For releases starting from 2.7.1:
                • The criticality can be one or more of MINOR, NORMAL, MAJOR, CRITICAL.
                • The ALL criticality is deprecated.
                • JOC-1819 - Getting issue details... STATUS
              • The return_code_from and return_code_to attributes can be optionally used to further limit notifications to jobs which complete with the given return code. The return code for shell jobs corresponds to the OS exit code.
            • Empty: If no WorkflowJob element is specified then the Notification applies to any JS7 - Workflow Instructions including the JS7 - Fail Instruction, otherwise it will be applied to occurrences of the JS7 - Job Instruction.
  • Notifications: Define the effective notifications by references to the Fragments elements described above.
    • Notification: Any number of notifications can be added with each norification being distinguished by a unique name. A notification is assigned a type which can be any one of SUCCESS, WARNING or ERROR. This allows notifications which are used, for example, in the event of job errors and warnings to be specified. This also allows notifications to be specified for successful workflow execution. Note that successful execution includes both the absence of job errors and optionally the presence of job warnings.
      • SystemNotification: Selects one or more of the above MonitorFragments. It is possible to select multiple fragments of the same fragment type.
        • Such notifications are created from system errors and warnings that are identified from JOC Cockpit log files.
      • NotificationMonitors: Selects one or more of the above MonitorFragments. It is possible to select multiple fragments of the same fragment type.
        • CommandFragmentRef: Selects the CommandFragment to be used.
          • MessageRef: Selects the Message element to be used with the Command.
        • MailFragmentRef: Selects the MailFragment to be used in order to send notifications by mail. If multiple MailFragment elements are referenced then different types of mail, e.g. for different recipients or with different content and layout of the mail body can be used.
        • NSCAFragmentRef: Selects the NSCAFragment to be used for direct integration with a NSCA compatible System Monitor.
        • JMSFragmentRef: Selects the JMSFragment to be used to send notifications to a Java Message Queue compatible product.
      • NotificationObjects: Selects the workflows which notifications are created for.
        • WorkflowRef: Selects a Workflows element that limits notifications to their respective workflows. Any number of workflow references can be added.

Monitor Variables

  • Syntax
    • All variables have to be used with the following syntax, independently of the operating system in use: ${VARIABLE_NAME}
  • Scope
    • The Message element can include Monitor Variables: users can select the variable names from the below list of variables and use for example ${MON_N_CREATED}.
    • The Command elements can include Operating System Variables: users have to prefix variable names from the below list with JS7_ to use for example ${JS7_MON_N_CREATED}.

Common Variables

The following variables are available from the configuration and can be used with Message elements and with Command elements.

NameDescription  Possible ValuesNote
MON_TIME_ZONETime zone used for formatting notification dates.

Any valid time zone.

JOC-1307

Common JOC Cockpit Variables

The following variables are available for access to JOC Cockpit and can be used with Command elements and with Message elements.

NameDescription  Note
JOC_ID
The identifier assigned the JOC Cockpit instanceAvailability starting with JOC-1447.
JOC_TITLEThe title assigned the JOC Cockpit instanceThe title is visible from the JOC Cockpit widget in the Dashboard view.
JOC_URIThe URL by which JOC Cockpit is available
JOC_REVERSE_PROXY_URIThe URL to access JOC Cockpit by a proxyAvailability starting with JOC-1381.

Order Notifications

Notification Variables

The following variables are available from the configuration and can be used with Message elements and with Command elements.

NameDescription  Possible Values
MON_N_IDThe unique identifier of the monitoring entry.
MON_N_TYPEThe message type corresponds to the specified type with the Notification element.SUCCESS, WARNINGERROR, RECOVERED
MON_N_STATUSA numeric status identifier that can be used for integration with a System Monitor.0-OK, 1-WARNING, 2-CRITICAL, 3- UNKNOWN
MON_N_RANGEAn indicator for the monitor scope including jobs and workflows.WORKFLOW, WORKFLOW_JOB
MON_N_NOTIFICATION_ID

The notification_id of the Notification XML element in use.


MON_N_RECOVERED_IDThe identifier of a monitoring entry that previously reported an error and that is considered recovered by the current monitoring entry.
MON_N_WARNThe indicator for the type of warning, e.g. LONGER_THAN.

NONE, LONGER_THAN, SHORTER_THAN, RETURN_CODE,STDERR

Availability starting with JS7 2.4.1:

JOC-1350 - Getting issue details... STATUS

JOC-1363 - Getting issue details... STATUS

MON_OS_WARN_TEXTThe warning message in the event of a warning occurring.

Availability starting with JS7 2.4.1:

JOC-1350 - Getting issue details... STATUS

JOC-1363 - Getting issue details... STATUS

MON_N_CREATEDThe timestamp that the monitoring entry has been added.

Order Variables

The following variables are available from orders and can be used with Message elements and with Command elements.

MON_O_HISTORY_IDMON_O_NAMEMON_O_WORKFLOW_POSITIONMON_O_START_WORKFLOW_POSITIONMON_O_END_WORKFLOW_POSITIONMON_O_PARENT_IDMON_O_MAIN_PARENT_ID
318207#2021-07-23#T7052187293-Test00000000700020318207
320121branch11/branch1:011/fork+branch1:2318207318207
320122branch21/branch2:011/fork+branch2:2318207318207
321560branch2b1/fork+branch2:1/branch2b:01/fork+branch2:11/fork+branch2:1/fork+branch2b:0320122318207
321561branch2c1/fork+branch2:1/branch2c:01/fork+branch2:11/fork+branch2:1/fork+branch2c:0320122318207
321682branch1b1/fork+branch1:1/branch1b:01/fork+branch1:11/fork+branch1:1/fork+branch1b:0320121318207
321683branch1c1/fork+branch1:1/branch1c:01/fork+branch1:11/fork+branch1:1/fork+branch1c:0320121318207
NameDescription  Note
MON_O_HISTORY_IDThe Order History ID (unique identifier).
MON_O_CONTROLLER_ID
The JS7 Controller ID.
MON_O_ORDER_IDThe Order ID (unique identifier).
MON_O_WORKFLOW_VERSION_IDThe version identifier of the workflow.
MON_O_WORKFLOW_POSITIONThe order's current position in the workflow.
MON_O_WORKFLOW_PATHThe inventory path of the workflow that the order is executed for.
MON_O_WORKFLOW_FOLDERThe parent folder of a workflow in the inventory.
MON_O_WORKFLOW_NAMEThe identifier of the workflow that the order is executed for.
MON_O_WORKFLOW_TITLEThe title of the workflow that the order is executed for.
MON_O_HAS_CHILDRENThe Boolean value true or false indicates that the current order is a parent order with forked child orders.
MON_O_MAIN_PARENT_IDIf the current order is a nested child order then this identifier points to the monitoring identifier of the main parent order.
MON_O_PARENT_ID

If the current order is a child order then this identifier points to the monitoring identifier of the direct parent order.

Note that child orders can be nested.
MON_O_PARENT_ORDER_IDThe Order ID of the parent order if the current order is a child order.
MON_O_NAME

The Order ID of the current order if this is a main order. If the current order is a child order then the branch name of the child order is returned.


MON_O_SEVERITYThe Order History status.SUCCESSFUL, FAILED, INCOMPLETE
MON_O_STATEThe current order state, see JS7 - Order State Transitions
MON_O_STATE_TIMEThe point in time when the order transitioned to its current state.
MON_O_CURRENT_HOS_IDThe monitoring identifier of the current order step.
MON_O_START_CAUSEThe reason for the start of an order.ORDER, FORK
MON_O_START_TIME_SCHEDULEDThe point in time that the order was scheduled for.
MON_O_START_TIMEThe timestamp when the order started.
MON_O_START_WORKFLOW_POSITIONFor a main order the position in the workflow that the order was started for is returned. For a child order the workflow position related to its branch is returned.
MON_O_START_VARIABLESThe list of order variables in JSON format.{"file":"/tmp/my_file.txt"}
MON_O_END_TIMEThe timestamp for the end of order execution.Note that in the event of job errors, an order might have halted in a failed state and without  providing an end time as the order did not completely execute the workflow.
MON_O_TIME_ELAPSEDThe difference in seconds between the end time and the start time of order execution.
MON_O_END_WORKFLOW_POSITIONFor a main order the last position in the workflow is returned. For a child order the last position of its branch is returned.
MON_O_END_HOS_ID

MON_O_ERRORThe Boolean value true or false that indicates if an error has occurred for the current order.
MON_O_ERROR_CODEA JS7 error code in case that a known error has occurred.
MON_O_ERROR_REASONA predefined reason that indicates the cause of an error, e.g. FAILED.DISRUPTED, FAILED, TIMEDOUT
MON_O_ERROR_RETURN_CODEIn case of errors for shell jobs the operating system exit code is returned. For JITL jobs and for JVM jobs the return code optionally is implemented by the respective Java class.
MON_O_ERROR_STATE

MON_O_ERROR_TEXTThe error message in the event of an error occurring.
MON_O_LOG_IDThe database identifier of the order log.
MON_O_CREATEDThe timestamp that the order monitoring entry has been added.
MON_O_MODIFIEDThe timestamp that the order monitoring entry has been modified.

Order Step Variables

The following variables are available from order steps, e.g. from jobs executed with a workflow and can be used with Message elements and with Command elements.

NameDescription  Note
MON_OS_HISTORY_IDThe Order History identifier for the current order step.
MON_OS_WORKFLOW_POSITIONThe textual position indicator of the order step in the workflow including the branch identifier.
MON_OS_HO_MAIN_PARENT_IDFor a main order the monitoring identifier is returned. For a child the Order History identifier of the main order is returned.
MON_OS_HO_IDThe Order History identifier of the current order.
MON_OS_POSITIONThe numeric position index of the order step in an instruction block within the workflow, e.g. within a Fork Instruction or Lock Instruction.
MON_OS_JOB_NAMEThe name of the job that the order is executing.
MON_OS_JOB_TITLEThe title of the job that the order is executing.
MON_OS_JOB_LABELThe unique label of the job instruction in the workflow.The label is unique for each occurrence of a job in a workflow.
MON_OS_JOB_CRITICALITYThe criticality as defined with the job.MINOR, NORMAL, MAJOR, CRITICAL
MON_OS_AGENT_IDThe unique identifier of an Agent that is specified when registering an Agent.
MON_OS_AGENT_URIThe URI of the Agent that executes the current job.
MON_OS_SEVERITYThe Order History status of the current order step.SUCCESSFUL, FAILED, INCOMPLETE
MON_OS_START_CAUSEThe cause for the start of the order step.ORDER, FORK
MON_OS_START_TIMEThe timestamp for the start of execution in the current order step.
MON_OS_START_VARIABLESThe list of order variables in JSON format before execution of the current workflow instruction.{"file":"/tmp/my_order_file.txt"}
MON_OS_END_TIMEThe timestamp for the end of execution of the current workflow instruction.
MON_OS_END_VARIABLESThe list of order variables in JSON format resulting after execution of the current workflow instruction.{"file":"/tmp/my_job_file.txt","my_log":"/logs/my_log.log"}
MON_OS_TIME_ELAPSEDThe difference in seconds between the end time and the star time of execution of the current order step.
MON_OS_ERRORThe Boolean value true or false that indicates if an error occurred with the current workflow instruction.
MON_OS_ERROR_CODEA JS7 error code in the event of a known error occuring.
MON_OS_ERROR_REASONA predefined reason that indicates the cause of error, e.g. FAILED.DISRUPTED, FAILED, TIMEDOUT
MON_OS_ERROR_STATEThe order state for the current order step caused by the error, e.g. FAILED, BROKEN.
MON_OS_ERROR_TEXTThe error message in the event of an error occurring in the current order step.
MON_OS_RETURN_CODEFor shell jobs, the operating system exit code is returned.
For JITL jobs and for JVM jobs, the return code optionally is implemented by the relevant Java class.

MON_OS_WARNThe indicator for the type of warning, e.g. LONGER_THAN.

NONE, LONGER_THAN, SHORTER_THAN

RETURN_CODE - Availability starting with JS7 2.4.1:

JOC-1350 - Getting issue details... STATUS

STDERR - Availability starting with JS7 2.4.1:

JOC-1363 - Getting issue details... STATUS


Deprecated since JS7 2.4.1 JOC-1350 - Getting issue details... STATUS .

Use Notification Variables:  MON_N_WARN

MON_OS_WARN_TEXTThe warning message in the event of a warning occurring.

Deprecated since JS7 2.4.1 JOC-1350 - Getting issue details... STATUS .

Use Notification Variables:  MON_N_WARN_TEXT

MON_OS_LOG_IDThe identifier of the task log.
MON_OS_CREATEDThe timestamp that the monitoring entry has been added.
MON_OS_MODIFIEDThe timestamp that the monitoring entry has been modified.


MON_OS_JOB_TAGS

The list of tags assigned a job. The syntax is:

MON_OS_JOB_TAGS=[<group>:]<tag[,tag]>[[;<group>:]<tag>]

Example:

MON_OS_JOB_TAGS=critical_without_group,priority_without_group;Notification:critical,priority;SLA:level1,penalty

The environment variable holds

  • two tags critical_without_group and priority_without_group
  • one group Notification that includes two tags critical and priority
  • one group SLA that includes two tags level1 and penalty

Available from  JOC-1812 - Getting issue details... STATUS

Order Notification JOC Cockpit Variables

The following variables are available for access to JOC Cockpit and can be used with Command elements and with Message elements.

NameDescription  Note
JOC_HREF_JOB
The JOC Cockpit URL that points to the Workflows view with the respective job.This URL can be used to navigate in JOC Cockpit to the respective Task History entry.
JOC_HREF_JOB_LOGThe JOC Cockpit URL that is used view the job's task log.This URL can be used to navigate in JOC Cockpit to the respective Task History entry and to immediately view the log.
JOC_HREF_ORDERThe JOC Cockpit URL that points to the Order History view.This URL can be used to navigate in JOC Cockpit to the respective Order History entry.
JOC_HREF_ORDER_LOGThe JOC Cockpit URL that is used to view the Order log.This URL can be used to navigate in JOC Cockpit to the respective Order History entry and to immediately view the log.
JOC_HREF_WORKFLOWThe JOC Cockpit URL that points to the Workflows view for the given order.This URL can be used to navigate in JOC Cockpit to the respective entry in the Workflows view.

Reverse Proxy. Availability starting with JOC-1381.

NameDescription  Note
JOC_REVERSE_PROXY_HREF_JOB
The JOC Cockpit URL that points to the Workflows view with the respective job.This URL can be used to navigate in JOC Cockpit to the respective Task History entry.
JOC_REVERSE_PROXY_HREF_JOB_LOGThe JOC Cockpit URL that is used view the job's task log.This URL can be used to navigate in JOC Cockpit to the respective Task History entry and to immediately view the log.
JOC_REVERSE_PROXY_HREF_ORDERThe JOC Cockpit URL that points to the Order History view.This URL can be used to navigate in JOC Cockpit to the respective Order History entry.
JOC_REVERSE_PROXY_HREF_ORDER_LOGThe JOC Cockpit URL that is used to view the Order log.This URL can be used to navigate in JOC Cockpit to the respective Order History entry and to immediately view the log.
JOC_REVERSE_PROXY_HREF_WORKFLOWThe JOC Cockpit URL that points to the Workflows view for the given order.This URL can be used to navigate in JOC Cockpit to the respective entry in the Workflows view.

System Notifications

Notification Variables

The following variables are available for System Notifications and can be used with Command elements and with Message elements.

NameDescription  Note
MON_SN_CATEGORY
The category is JOC for messages related to JOC Cockpit classes and System for OS related messages
MON_SN_TYPEThe message type is WARNING or ERROR
MON_SN_SOURCEThe identifier within a category that specifies the source in which a problem occurred

For example source ControllerApi for problems related to communication with the Controller.

Availability starting with JOC-1447.

Old name: MON_SN_SECTION

MON_SN_NOTIFIERThe Java class that caused the notification
MON_SN_TIMEThe point in time when an error or warning did occur
MON_SN_MESSAGEThe message provided with the error or warning
MON_SN_EXCEPTIONThe exception name provided that the System error or warning is related to Java classes

Logging

Processing of notifications is logged by the Monitor Service:

Further Resources

Feature

How To ... Instructions

Troubleshooting


  • No labels