Page History

...

User might be interested to automatically forward order logs to some external location.
- This allows to use log analysis tools such as Splunk® , Elasticsearch® etc.
- You are strongly advised not to access order logs from their original location with the JobScheduler Master's ./logs directory for analysis with 3rd party tools.
  - Log files are constantly updated, approx. every 5s, therefore you cannot decide when the log is completed . and you should not block files that JobScheduler requires to access.
  - Log files are stored to the database on order completion and will be overwritten on disk with the next order starting for the same job chain.
The forwarding of logs therefore should start from the database and can be scheduled e.g. on a daily basis or more frequently to allow immediate log analysis.

...

The PowerShell CLI can be used by jobs a job to forward logs. This requires prior installation ofof the JobScheduler PowerShell Module.

The Get-JobSchedulerOrderHistory cmdlet is used to retrieve order history items and to forward them to the Get-JobSchedulerOrderLog cmdlet within a job. Two flavors of the job are available for Windows and Linux. The difference is not about the handling of cmdlets or parameters but due to the fact that PowerShell is invoked differently on Windows and Linux. For Windows environments usually PowerShell is available with the OS, for Linux the job has to call pwsh to invoke the PowerShell.

Please consider that below jobs are examples that have to be adjusted for your environment.

Windows Version for use with Agents

Download: forward_taskorder_logs_windows.job.xml

Code Block

language	powershell
title	Forward Order Logs (Windows Agent version)
linenumbers	true
collapse	true

<?xml version="1.0" encoding="UTF-8" ?>
<job title="Forward Order Logs" process_class="agent_windows">
  <params>
    <param name="history_results_directory" value="/userstmp/js/history"/>
  </params>
  <script language="powershell"><![CDATA[
Import-Module $env:SCHEDULER_DATA/config/powershell/Modules/JobScheduler;
Connect-JS -Url $JOCCockpitUrl -Credential $JOCCockpitCredential | Out-Null;

mkdirNew-Item -ItemType Directory -Force -Path $env:SCHEDULER_PARAM_HISTORY_RESULTS_DIRECTORY | Out-Null;

# retrieve last history results if available
if ( Test-Path -Path "$($env:SCHEDULER_PARAM_HISTORY_RESULTS_DIRECTORY)/taskorder.history" -ErrorAction continue )
{
    $lastHistory = Import-Clixml -Path "$($env:SCHEDULER_PARAM_HISTORY_RESULTS_DIRECTORY)/taskorder.history";
} else {
    $lastHistory = Get-JobSchedulerTaskHistoryJobSchedulerOrderHistory -RelativeDateFrom -8h | Sort-Object -Property startTime;
}
 
# Copy log files to target directory
Get-JSTaskHistoryJSOrderHistory -DateFrom $lastHistory[0].startTime | Tee-Object -Variable lastHistory | Get-JobSchedulerTaskLogJobSchedulerOrderLog | Select-Object @{name='path'; expression={ "$env:SCHEDULER_PARAM_HISTORY_RESULTS_DIRECTORY/$(Get-Date $_.startTime -f 'yyyyMMdd-hhmmss')-$([io.path]::GetFileNameWithoutExtension($_.jobjobChain))-$($_.orderId).log"}}, @{name='value'; expression={ $_.log }} | Set-Content;
 
# store last history results to a file for later retrieval
$lastHistory | Export-Clixml -Path "$env:SCHEDULER_PARAM_HISTORY_RESULTS_DIRECTORY/taskorder.history";
                 
Write-Output ".. logs forwarded to: $env:SCHEDULER_PARAM_HISTORY_RESULTS_DIRECTORY";
]]></script>
  <run_time/>
</job>

...

Code Block

language	powershell
title	Forward Order Logs (Linux version)
linenumbers	true
collapse	true

<?xml version="1.0" encoding="UTF-8" ?>
<job title="Forward Order Logs" process_class="agent_linux">
  <params>
    <param name="history_results_directory" value="/home/jstmp/history"/>
  </params>
  <script language="shell"><![CDATA[
pwsh -NoLogo -NonInteractive -Command '& {
    . $env:SCHEDULER_DATA/config/powershell/JobScheduler.PowerShell_profile.ps1;
    Import-Module $env:SCHEDULER_DATA/config/powershell/Modules/JobScheduler;
    Connect-JS -Url $JOCCockpitUrl -Credential $JOCCockpitCredential | Out-Null;
 
    # retrieve last history results if available
    if ( Test-Path -Path "$($env:SCHEDULER_PARAM_HISTORY_RESULTS_DIRECTORY)/order.history" -ErrorAction continue )
    {
        $lastHistory = Import-Clixml -Path "$($env:SCHEDULER_PARAM_HISTORY_RESULTS_DIRECTORY)/order.history";
   } else {
        $lastHistory = Get-JobSchedulerOrderHistory -RelativeDateFrom -8h | Sort-Object -Property startTime;
    }
 
    # Copy log files to target directory
    Get-JobSchedulerOrderHistory -DateFrom $lastHistory[0].startTime | Tee-Object -Variable lastHistory | Get-JobSchedulerOrderLog | Select-Object @{name="path"; expression={ "$env:SCHEDULER_PARAM_HISTORY_RESULTS_DIRECTORY/$(Get-Date $_.startTime -f 'yyyyMMdd-hhmmss')-$([io.path]::GetFileNameWithoutExtension($_.orderId)).log"}}, @{name="value"; expression={ $_.log }} | Set-Content;
 
    # store last history results to a file for later retrieval
    $lastHistory | Export-Clixml -Path "$env:SCHEDULER_PARAM_HISTORY_RESULTS_DIRECTORY/order.history";
                 
    Write-Output ".. logs forwarded to: $env:SCHEDULER_PARAM_HISTORY_RESULTS_DIRECTORY";
}'
]]></script>
  <run_time/>
</job>
</job>

...

Space shortcuts

Page tree

Versions Compared

Old Version 4

New Version 5

Key

Windows Version for use with Agents