Page History

...

User might be interested to automatically receive reports about past job executions.
- The reports include the same information as available from the JOC Cockpit History view for tasks.
- The reports are provided as Excel® files similar to what is available for export from the JOC Cockpit History view.
forward task logs to some external location.
- This allows to use log analysis tools such as Splunk, Elasticsearch etc.
- You are strongly advised not to access task logs from their original location with the JobScheduler Master's ./logs directory for analysis with 3rd party tools.
  - Log files are constantly updated, approx. every 5s, therefore you cannot decide when the log is completed.
  - Log files are stored to the database on task completion and will be overwritten on disk with the next task starting from the same job.
The forwarding of logs The report can be scheduled e.g. on a daily basis or more frequently to report about past job execution resultsallow immediate log analysis.

Use Cases

...

Forward Task

...

Logs from a job

The PowerShell CLI is can be used by jobs to create reportsforward logs. Two modules are applied for this purpose:

the JobScheduler PowerShell Modulea reporting PowerShell Module. This example makes use of the ImportExcel PowerShell Module that can be used to create Excel® reports on Windows and Linux.

The Get-JobSchedulerTaskHistory cmdlet is used to retrieve task history items and to forward them to the ImportExcel module Get-JobSchedulerTaskLog cmdlet within a job. Two flavors of the job are available for Windows and Linux. The difference is not about the handling of cmdlets or parameters but due to the fact that PowerShell is invoked differently on Windows and Linux. For Windows environments usually PowerShell is available with the OS, for Linux the job has to call pwsh to invoke the PowerShell.

Find a sample report: jobscheduler_reporting.xlsx

Please consider that below jobs are examples that have to be adjusted for your environment.

...

Code Block

language	powershell
title	Forward Task History Report Logs (Windows version)
linenumbers	true
collapse	true

<?xml version="1.0" encoding="UTF-8" ?>
<job title="ReportForward Task HistoryLogs" process_class="agent_windows">
  <params>
    <param name="history_results_directory" value="/users/js/history"/>
  </params>
  <script language="powershell"><![CDATA[
Import-Module $env:SCHEDULER_DATA/config/powershell/Modules/ImportExcel;
Import-Module $env:SCHEDULER_DATA/config/powershell/Modules/JobScheduler;

Connect-JS -Url $JOCCockpitUrl -Credential $JOCCockpitCredential | Out-Null;

# Datesretrieve inlast localhistory timezone,results output includes local date format
Get-JSTaskHistory -Timezone (Get-Timezone ) `
                |  Select-Object -Property @{name="JobScheduler ID"; expression={$_.jobschedulerId}}, `
                                           @{name="Task ID"; expression={$_.taskId}}, `
                                           @{name="Job"; expression={$_.job}}, `
                                           @{name="Status"; expression={$_.state._text}}, `
                                           @{name="Start Time"; expression={ Get-Date $_.startTime }}, `
                                           @{name="End Time"; expression={ if available
if ( Test-Path -Path "$env:SCHEDULER_PARAM_HISTORY_RESULTS_DIRECTORY/$($env:SCHEDULER_JOB_NAME).history" -ErrorAction continue )
{
	$lastHistory = Import-Clixml -Path "$env:SCHEDULER_PARAM_HISTORY_RESULTS_DIRECTORY/$($env:SCHEDULER_JOB_NAME).history"
} else {
	$lastHistory = Get-JobSchedulerTaskHistory -RelativeDateFrom -8h | Sort-Object -Property startTime
}

# Copy log files to target directory
Get-JSTaskHistory -DateFrom $lastHistory[0].startTime | Tee-Object -Variable lastHistory | Get-JobSchedulerTaskLog | Select-Object @{name='path'; expression={ "$env:SCHEDULER_PARAM_HISTORY_RESULTS_DIRECTORY/$(Get-Date $_.endTime }}, `
                                           @{name="Duration (sec.)"; expression={ (New-Timespan -Start "$($_.startTime)" -End "$($_.endTime)").Seconds }}, `
                                           @{name="Criticality"startTime -f 'yyyyMMdd-hhmmss')-$([io.path]::GetFileNameWithoutExtension($_.job)).log"}}, @{name='value'; expression={ $_.criticalitylog }}, `
                                           @{name="Exit Code"; expression={$_.exitCode}} `
                | Export-Excel -Path /tmp/jobscheduler_reporting.xlsx -WorksheetName "Task-History" -ClearSheet; | Set-Content

# store last history results to a file for later retrieval
Export-Clixml -Path "$env:SCHEDULER_PARAM_HISTORY_RESULTS_DIRECTORY/$($env:SCHEDULER_JOB_NAME).history"
				
Write-Output ".. reportlogs forwarded createdto: /tmp/jobscheduler_reporting.xls"$env:SCHEDULER_PARAM_HISTORY_RESULTS_DIRECTORY";
]]></script>
  <run_time/>
</job>

...

Space shortcuts

Page tree

Versions Compared

Old Version 1

New Version 2

Key

Use Cases

Forward Task

Logs from a job