JS7 JobScheduler

Space shortcuts

Page tree

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »

Introduction


Directory Watching

Find a sample implementation from the following chapters.

Download (upload .json): pdfWatchDirectoryEvents.workflow.json

Workflow

The workflow includes a single job that implements Directory Watching.


Explanation:

  • The job is included in a JS7 - Cycle Instruction. The purpose of this instruction is to manage periods for which Directory Watching should be performed.
  • The workflow makes use of a number of variables.
    • Workflow variables without default value have to be specified when adding orders to the workflow.
      • move_destination: specifies the target directory to which incoming files will be moved.
      • monitor_path: specifies the source directory that is watched for incoming files
    • Workflow variables with default values can be omitted when adding orders to the workflow.
      • monitor_filter: specifies a wildcard expression to select files, for example a filter *.csv will limit files in consideration to this extension.
      • monitor_subdirectories: specifies true if sub-directories should be considered and otherwise false.
      • monitor_log_file: specifies the log file for events.
      • monitor_period: specifies the duration of job execution in seconds. If the specified duration is exceeded then the job will terminate and will be restated by the Cycle Instruction. This guarantees that the file watcher is renewed and that any sub-directories that might have become inaccessible, for example from a remote share that is rebooted, will be registered with the next job run.
      • monitor_interval: specifies the interval in seconds between checks of the monitor_log_file for changes.
      • monitor_steady: specifies the number of seconds that an incoming file is waited for to check that the file size is unchanged and the file can be considered stable.

Cycle

The Cycle Instruction is an option that users apply who want to watch directories during certain business hours only.

The instruction allows to specify weekday, month days etc. and the period during which the job will be repeatedly executed.

Clicking the Cycle Instruction in the Configuration view displays the cycle in the right panel with the property editor.


Clicking the period brings up the details of the cycle:

Job

The job is implemented with PowerShell:

  • PowerShell is available for Windows servers by default.
    • Consider the shebang in the first line of the job that indicates the PowerShell version in use:
      • the string @@findstr/v "^@@f.*&" "%~f0"|powershell.exe -&goto:eof indicates use of powershell.exe from version 5.1.
      • the string @@findstr/v "^@@f.*&" "%~f0"|pwsh.exe -&goto:eof indicates use of pwsh.exe from version 6 or 7.
  • PowerShell can be installed for Unix environments such as Linux and MacOS.
    • Consider to modify the shebang in the first line of the job like this:
      • the string #!/usr/bin/env pwsh can be used assuming that pwsh is the binary of the PowerShell interpreter.

Find the full source code of the job:

Job script source 
@@findstr/v "^@@f.*&" "%~f0"|powershell.exe -&goto:eof

# for Windows with PowerShell 7 use shebang: @@findstr/v "^@@f.*&" "%~f0"|pwsh.exe -&goto:eof
# for Unix use shebang: #!/usr/bin/env pwsh

# start watcher for a given path, file name filter and optionally any sub-directories
$watcher = New-Object System.IO.FileSystemWatcher
$watcher.Path = $env:MONITOR_PATH
$watcher.Filter = $env:MONITOR_FILTER
$watcher.IncludeSubdirectories = ($env:MONITOR_SUBDIRECTORIES -eq 'true')
$watcher.EnableRaisingEvents = $true   

# define action to move files
$moveFileAction = {
    if ( Test-Path $Event.SourceEventArgs.FullPath -PathType leaf )
    {
    	$lastSize,$size = -1
        While ($true)
        {
            $size = (Get-Item $Event.SourceEventArgs.FullPath).length
            if ( $size -ne $lastSize )
            {
                "$(Get-Date) [Check] $($Event.SourceEventArgs.FullPath) check steady state, size=$size" | Out-File $env:MONITOR_LOG_FILE -Append
				Start-Sleep -Seconds $env:MONITOR_STEADY
                $lastSize = $size
            } else {
                break
            }
        }
        "$(Get-Date) [$($Event.SourceEventArgs.ChangeType)] $($Event.SourceEventArgs.FullPath) moved to $($env:MOVE_DESTINATION)" | Out-File $env:MONITOR_LOG_FILE -Append
        Move-Item -Path $Event.SourceEventArgs.FullPath -Destination $env:MOVE_DESTINATION -Force
    }
}

# define action to log directory events
$logFileAction = {
   "$(Get-Date) [$($Event.SourceEventArgs.ChangeType)] $($Event.SourceEventArgs.FullPath)" | Out-File $env:MONITOR_LOG_FILE -Append
}

# register actions to be taken when events are raised
Register-ObjectEvent -InputObject $watcher -EventName "Created" -Action $moveFileAction | Out-Null
Register-ObjectEvent -InputObject $watcher -EventName "Changed" -Action $logFileAction | Out-Null
Register-ObjectEvent -InputObject $watcher -EventName "Deleted" -Action $logFileAction | Out-Null
Register-ObjectEvent -InputObject $watcher -EventName "Renamed" -Action $logFileAction | Out-Null

# move existing files found before file watcher reports events
Get-ChildItem -Path "$($env:MONITOR_PATH)/$($env:MONITOR_FILTER)" -Recurse | Move-Item -Destination $env:MOVE_DESTINATION -Force

# check existing log file
$sleep,$count,$lastCount = 0
if ( Test-Path $env:MONITOR_LOG_FILE -PathType leaf )
{
	$lastCount = $count = (Get-Content $env:MONITOR_LOG_FILE -ErrorAction silentlycontinue | Measure-Object -Line).Lines
}

# run file watchter for the given period
Write-Output ".. watching for $env:MONITOR_PERIOD seconds"
While( $sleep -le $env:MONITOR_PERIOD )
{
	Start-Sleep -Seconds $env:MONITOR_INTERVAL
    $sleep += $env:MONITOR_INTERVAL
	$count = (Get-Content $env:MONITOR_LOG_FILE -ErrorAction silentlycontinue | Measure-Object -Line).Lines 
    if ( $count -ne $lastCount )
    {
    	Get-Content $env:MONITOR_LOG_FILE -Tail ($count-$lastCount) -ErrorAction silentlycontinue
        $lastCount = $count
    }
}

# perform cleanup
Get-EventSubscriber | Unregister-Event
$watcher.Dispose


Explanation:

  • x






  • No labels