PowerShell CLI 1.2 - Use Cases - Task Log

Starting Situation

User might be interested to automatically forward task logs to some external location.
- This allows to use log analysis tools such as Splunk® , Elasticsearch® etc.
- You are strongly advised not to access task logs from their original location with the JobScheduler Master's ./logs directory for analysis with 3rd party tools.
  - Log files are constantly updated, approx. every 5s, therefore you cannot decide when the log is completed and you shouldn't block files that JobScheduler requires to access.
  - Log files are stored to the database on task completion and will be overwritten on disk with the next task starting from the same job.
The forwarding of logs therefore should start from the database and can be scheduled e.g. on a daily basis or more frequently to allow immediate log analysis.

Use Cases

Forward Task Logs from a job

The PowerShell CLI can be used by a job to forward logs. This requires prior installation of the JobScheduler PowerShell Module.

The Get-JobSchedulerTaskHistory cmdlet is used to retrieve task history items and to forward them to the Get-JobSchedulerTaskLog cmdlet within a job. Two flavors of the job are available for Windows and Linux. The difference is not about the handling of cmdlets or parameters but due to the fact that PowerShell is invoked differently on Windows and Linux. For Windows environments usually PowerShell is available with the OS, for Linux the job has to call pwsh to invoke the PowerShell.

Please consider that below jobs are examples that have to be adjusted for your environment.

Windows Version for use with Agents

Download: forward_task_logs_windows.job.xml

Forward Task Logs (Windows Agent version)

<?xml version="1.0" encoding="UTF-8" ?>
<job title="Forward Task Logs" process_class="agent_windows">
  <params>
    <param name="history_results_directory" value="/tmp/history"/>
  </params>
  <script language="powershell"><![CDATA[
Import-Module $env:SCHEDULER_DATA/config/powershell/Modules/JobScheduler;
Connect-JS -Url $JOCCockpitUrl -Credential $JOCCockpitCredential | Out-Null;

New-Item -ItemType Directory -Force -Path $env:SCHEDULER_PARAM_HISTORY_RESULTS_DIRECTORY | Out-Null;

# retrieve last history results if available
if ( Test-Path -Path "$($env:SCHEDULER_PARAM_HISTORY_RESULTS_DIRECTORY)/task.history" -ErrorAction continue )
{
    $lastHistory = Import-Clixml -Path "$($env:SCHEDULER_PARAM_HISTORY_RESULTS_DIRECTORY)/task.history";
} else {
    $lastHistory = Get-JobSchedulerTaskHistory -RelativeDateFrom -8h | Sort-Object -Property startTime;
}
 
# Copy log files to target directory
Get-JSTaskHistory -DateFrom $lastHistory[0].startTime | Tee-Object -Variable lastHistory | Get-JobSchedulerTaskLog | Select-Object @{name='path'; expression={ "$env:SCHEDULER_PARAM_HISTORY_RESULTS_DIRECTORY/$(Get-Date $_.startTime -f 'yyyyMMdd-hhmmss')-$([io.path]::GetFileNameWithoutExtension($_.job)).log"}}, @{name='value'; expression={ $_.log }} | Set-Content;
 
# store last history results to a file for later retrieval
$lastHistory | Export-Clixml -Path "$env:SCHEDULER_PARAM_HISTORY_RESULTS_DIRECTORY/task.history";
                 
Write-Output ".. logs forwarded to: $env:SCHEDULER_PARAM_HISTORY_RESULTS_DIRECTORY";
]]></script>
  <run_time/>
</job>

Explanations

Line 2: The job is executed with a Windows Agent that is assigned by a process class.
Line 3-5: Consider to adjust the history_results_directory parameter to point to a valid directory where to store the log files.
Line 6: The job is of type "powershell" and will use the Powershell version provided with the server.
Line 7: The JobScheduler PowerShell module is imported. The module could be installed with any location in the file system
Line 8: The Connect-JS cmdlet is used to authenticate with the JOC Cockpit REST Web Service. The required URL and credentials are specified in a PowerShell profile, see PowerShell CLI 1.2 - Use Cases - Credentials Management
Line 10: The target directory specified by the history_results_directory parameter is created should it not exist.
Line 13-18: Any previous execution of this job will create a file task.history that stores the latest history entries for which log files have been processed. If such a file exists then the value of the $lastHistory variable is restored and otherwise an initial query about the last 8 hours of task executions is used to populate the $lastHistory variable.
Line 21: The Get-JobSchedulerTaskHistory cmdlet is called
- with the parameter -DateFrom $lastHistory[0].startTime to specify the most recent history entry to be processed
  - optionally with additional parameters, e.g. to specify the date range for which logs are forwarded A value -DateFrom (Get-Date -Hour 0 -Minute 0 -Second 0).AddDays(-7).ToUniversalTime() specifies that logs should be forwarded for the last 7 days (from midnight). Keep in mind that dates have to be specified for the UTC time zone. Without this parameter logs will be forwarded for the last day.
  - see the Get-JobSchedulerTaskHistory cmdlet for a full parameter reference.
- the output of this cmdlet is pipelined to the Tee-Object cmdlet to store new values to the $lastHistory variable and to proceed with the pipeline.
- the following pipeline step includes to read the task log for each entry reported by the Get-JobSchedulerTaskHistory cmdlet.
- the next pipeline step includes to select properties from the pipeline that are forwarded to the Set-Content cmdlet that expects the Path and Value parameters. For the Path parameter the output directory and the file name for the log file are specified from the task's start time and job name.
Line 24: Finally the values returned by the $lastHistory variable are persistently written to disk for later retrieval.

Linux Version

Download: forward_task_logs_linux.job.xml

Forward Task Logs (Linux version)

<?xml version="1.0" encoding="UTF-8" ?>
<job title="Forward Task Logs" process_class="agent_linux">
  <params>
    <param name="history_results_directory" value="/tmp/history"/>
  </params>
  <script language="shell"><![CDATA[
pwsh -NoLogo -NonInteractive -Command '& {
    . $env:SCHEDULER_DATA/config/powershell/JobScheduler.PowerShell_profile.ps1;
    Import-Module $env:SCHEDULER_DATA/config/powershell/Modules/JobScheduler;
    Connect-JS -Url $JOCCockpitUrl -Credential $JOCCockpitCredential | Out-Null;
 
    New-Item -ItemType Directory -Force -Path $env:SCHEDULER_PARAM_HISTORY_RESULTS_DIRECTORY | Out-Null;
 
    # retrieve last history results if available
    if ( Test-Path -Path "$($env:SCHEDULER_PARAM_HISTORY_RESULTS_DIRECTORY)/task.history" -ErrorAction continue )
    {
        $lastHistory = Import-Clixml -Path "$($env:SCHEDULER_PARAM_HISTORY_RESULTS_DIRECTORY)/task.history";
   } else {
        $lastHistory = Get-JobSchedulerTaskHistory -RelativeDateFrom -8h | Sort-Object -Property startTime;
    }
 
    # Copy log files to target directory
    Get-JSTaskHistory -DateFrom $lastHistory[0].startTime | Tee-Object -Variable lastHistory | Get-JobSchedulerTaskLog | Select-Object @{name="path"; expression={ "$env:SCHEDULER_PARAM_HISTORY_RESULTS_DIRECTORY/$(Get-Date $_.startTime -f 'yyyyMMdd-hhmmss')-$([io.path]::GetFileNameWithoutExtension($_.job)).log"}}, @{name="value"; expression={ $_.log }} | Set-Content;
 
    # store last history results to a file for later retrieval
    $lastHistory | Export-Clixml -Path "$env:SCHEDULER_PARAM_HISTORY_RESULTS_DIRECTORY/task.history";
                 
    Write-Output ".. logs forwarded to: $env:SCHEDULER_PARAM_HISTORY_RESULTS_DIRECTORY";
}'
]]></script>
  <run_time/>
</job>
</job>