Introduction

Order logs and task logs are stored in the JS7 - Database in a compressed format and are available from the JS7 - Order History and JS7 - Task History views respectively. Both views allow display of the log output in a log view window.

JS7 does not make use of files for log output of order logs and task logs.

• Files are a frequent source of error as they may not be created due to disk space restrictions or lack of permissions.
• Job processing could be blocked if a log file cannot be created. Even worse, if log files are not available then users are unable to see job execution results.

JS7 therefore streams log output from Agents to the Controller and to JOC Cockpit without the use of files.

• Streaming makes job log output available to JOC Cockpit within 1s. 
• Streaming allows a running log which translates to the fact that users observe ongoing log output of orders and jobs in the log view window.
• In case of missing connections:
  • between Controller and Agent - the Agent stores log output in its journal and forwards logs when reconnected to a Controller later on. 
  • between JOC Cockpit and Controller - the Controller stores log output in its journal and forwards logs when reconnected to JOC Cockpit at a later point in time.

Log Size

Jobs can write log output of arbitrary size to the job's stdout/stderr channel.

• The JS7 Agent and Controller handle log output based on a streaming implementation that can handle any size of log output.
• The JOC Cockpit adds log output to the JS7 database in a compressed format. This processing is affected by large log size by the following aspects:
  • The compression ratio depends on the individual log output, however, it is safe to assume the resulting size of logs in the database to be 10-20% of the original size.
  • The compression process takes more time and more system resources such as CPU and Java Heap Space for larger log output.

The JS7 - History Service that handles logs in JOC Cockpit works with the following threshold settings, see JS7 - Settings:

• A setting for the applicable size of a task log (name: log_applicable_size, default: 500 MB). A task log that exceeds this size will be handled like this:
  • The first 100 KB and the last 100 KB of log output will be used as the task log.
    • A banner information is added to the begin of log output:
      • LOG OUTPUT EXCEEDS APPLICABLE SIZE OF <log_applicable_size> MB AND IS TRUNCATED TO THE FIRST AND LAST 100 KB.
    • The truncated task log will be compressed and added to the database.
  • The original log file will be removed.
  • Occurrence of log output exceeding this threshold raises a warning, however, the task execution history and truncated log output are available from the JOC Cockpit API and GUI.
• A setting for the maximum size of a task log (name: log_maximum_size, default: 1000 MB). A task log exceeding this size will be handled like this:
  • The History Service will stop to write larger log files. Log output exceeding the threshold value is ignored.
  • The first 100 KB of the log file will be used as the task log.
    • A banner information is added to the begin of log output:
      • LOG OUTPUT EXCEEDS MAXIMUM SIZE OF <log_maximum_size> MB AND IS TRUNCATED TO 100 KB.
    • The truncated task log will be compressed and added to the database.
  • The original log file will be removed.

The History Service similarly considers the above settings when performing switch-over and fail-over operations between JOC Cockpit instances (compressing and storing current logs to the database).

The JOC Cockpit GUI makes use of the following threshold setting, see JS7 - Settings:

• A setting for the maximum display size of a task log (name: log_maximum_display_size, default: 10 MB). An uncompressed task log exceeding this size is not offered for display with the Log View window but is offered for download only.

The threshold settings can be modified from the JOC Cockpit Settings page.

Log Files

A number of users prefer to have log files available for each order execution and job execution, for example to consolidate log files on a file server or to submit log files to specific tools for log analysis. Typical shortcomings of log file analysis include:

• Log analysis is useful but usually comes too late. It's more a reporting activity that is not intended to replace near-real time monitoring.
• Log analysis starts when the log is completed. In Unix environments parallel read and write operations on log files can occur. 
• Log analysis is useful when it comes to parsing logs for unexpected output or errors. However, it might prove tedious to teach a log analyzer recovery of errors from log output in case of a job being successfully restarted for a specific order.

The JS7 - REST Web Service API allows access to log output of orders and jobs and thereby supports a situation when log output should be consolidated or should be analyzed by external tools.

The JS7 - PowerShell Module is a lightweight wrapper for the REST Web Service API that can be used with Linux and Windows to provide order logs and task logs from files as described in the linked articles below. Users are free to use the JS7 REST Web Service from their preferred scripting language to provide similar functionality.

Links

How To ... Instructions