Introduction

JS7 offers more capabilities than AutoSys® Workload Automation. This includes that existing AutoSys® job configurations can be migrated to JS7.

For general information see JS7 - Migration of AutoSys Jobs.
Conversion of AutoSys® job configurations is offered as a professional service.

Converter

With Release 2.7.0 the Converter becomes available to migrate job configurations from AutoSys® JIL files to JS7:

JOC-1154 - Getting issue details... STATUS JOC-1921 - Getting issue details... STATUS

The Converter reads AutoSys® *.jil files and creates .json files that are added to a .tar.gz or .zip archive file for import into JS7, see JS7 - Inventory Export and Import.

Prerequisites

The Converter can be used for the following platforms.

The Converter Java classes can be used with Java LTS editions of JRE or JDK 17 and newer.
The Converter Start Script is available for
- Linux, MacOS® and AIX® using bash, dash, ksh and zsh shells.
- Windows 10, 11, Windows Server

Download

Download of the Converter from the js7_converter_autosys.tar.gz tarball and js7_converter_autosys.zip archive file is available within the scope of migration services.

After extraction of the .tar.gz archive for Unix or .zip archive for Windows find the following files:

js7_converter_autosys
- js7_convert_autosys.sh | .cmd (Converter Start Script)
- js7_convert_autosys.config (Converter Configuration File)
- lib (Directory for Java Classes)

Usage

Invoking the Converter Start Script without arguments displays the usage clause:

Converter Start Script: js7_convert_autosys.sh | .cmd

Usage: js7_convert_autosys.sh | .cmd [Options]

  Options:
    --input-dir=<location of input directory>                         | required argument
    --output-dir=<location of output directory>                       | default: ./output/js7_converted_autosys
    --report-dir=<location of report directory>                       | default: ./output/report
    --archive=<location of resulting .zip archive for JS7 import>     | default: ./js7_converted_autosys.zip | .zip
    --config=<location of config file>                                | default: ./js7_convert_autosys.config
 Switches:
    -h | --help                                                       | displays usage

Explanation:

Options
- --input-dir
  - Specifies the input directory of AutoSys® *.jil files for job configurations.
  - The Converter recursively traverses the input directory.
- --output-dir
  - Specifies the output directory to which converted .*json files are written. For a single *.jil file there can be a number of corresponding *.json files.
  - For each sub-directory of the input directory the corresponding sub-directory is created in the output directory.
  - On start-up the Converter removes existing files and sub-directories from the output directory.
- --report-dir
  - The Converter will report to the specified directory by use of a number of .csv files the extent to which the conversion is successful:
    - parser_summary.csv: reports the number of objects found in the input directory and sub-directories.
    - converter_summary.csv: reports the number converted objects per object type such as workflows and scheduled.
    - converter_analyzer.csv: includes information and warnings about JIL element conversion.
- --archive
  - Specifies the path to an archive file with the .tar.gz or .zip extension that will be created by the Converter. Depending on the file extension the related archive format will be used on any OS platform.
  - The archive includes the *.json files of the output directory and can be used for later import into JS7, see JS7 - Inventory Export and Import.
  - An existing archive file will be overwritten.
- --config
  - Specifies the location of the Converter Configuration File.. This file is required for the Converter to run. It includes a number of settings that can be adjusted.

Examples

Unix

Example for running the Converter Start Script for Unix with minimum Arguments

js7_converter_home=./js7_converter_autosys
${js7_converter_home}/js7_convert_autosys.sh --input-dir=./input --config=./js7_convert_autosys.config

# reads *.jil files from the input directory and creates the corresponding output directory hierarchy in the "output" sub-directory of the working directory
# report files are written to the "report" sub-directory, the resulting "js7_converted_autosys.zip" archive file is written to the working directory

Example for running the Converter Start Script for Unix with more Arguments

js7_converter_home=./js7_converter_autosys
${js7_converter_home}/js7_convert_autosys.sh \
    --input-dir=./input \
    --output-dir=./output \
    --report-dir=./report \
    --archive=./js7_converted_autosys.zip

# reads *.jil files from the input directory and creates the corresponding output directory hierarchy in the "output" sub-directory of the working directory
# report files are written to "report" sub-directory, the resulting "js7_converted_autosys.zip" archive file is written to the working directory

Windows

Example for running the Converter Start Script for Windows with minimum Arguments

js7_convert_autosys.cmd --input-dir=C:\TEMP\AutoSys

@rem reads *.jil files from the input directory and creates the corresponding directory hierarchy in the "output" sub-directory of the working directory
@rem report files are written to the "report" sub-directory, the resulting "js7_converted_autosys.zip" archive file is written to the working directory

Example for running the Converter Start Script for Windows with more Arguments

js7_convert_autosys.cmd ^
    --input-dir=C:\TEMP\AutoSys ^
    --output-dir=C:\TEMP\output ^
    --report-dir=C:\TEMP\report ^
    --archive=C:\TEMP\js7-import.zip

@rem reads *.jil files from the "C:\TEMP\AutoSys" input directory and creates the corresponding directory hierarchy in "C:\TEMP\output"
@rem report files are written to the "C:\TEMP\report" directory, the resulting archive file is written to "C:\TEMP\js7-import.zip"

Configuration

The Converter is configured from a properties file that can be specified with the --config option when invoking the Converter. By default the js7_convert_autosys.config file is assumed that ships with the Converter.

Users should adjust the configuration to their needs, for example, to map Agents and Calendars.

Download: js7_convert_autosys.config

Default Configuration from js7_convert_autosys.config file

########################################################################################################################################################
# Parser
########################################################################################################################################################
# default: jil
# case-insensitive
parserConfig.extensions                             = jil;txt
parserConfig.excluded.fileNames                     = machines.jil
########################################################################################################################################################
# Autosys
########################################################################################################################################################
# Creates separate XML files containing the original Autosys job definition for each standalone job (a job without any conditions) or for each BOX job.
#           Output directory: report/autosys.input.original/config
autosys.input.splitConfiguration                    = false

# diagram - Generates diagram files using the original Autosys definition. A Graphviz app installation is required.
#           Output directory: report/autosys.input.original/diagram
autosys.input.diagram.generate                      = false
# Attempts to resolve the Autosys conditions in order to reduce the displayed dependencies (arrows) between various jobs.
# Example: Display the dependencies as a sequence of jobs instead of showing multiple dependency arrows if such conditions are defined.
autosys.input.diagram.optimize.dependencies         = true
autosys.input.diagram.outputFormat                  = svg
autosys.input.diagram.size                          = 0
autosys.input.diagram.graphviz.executable           = C:/Program Files/Graphviz/bin/dot.exe
autosys.input.diagram.graphviz.cleanupDotFiles      = true
########################################################################################################################################################
# JS7
########################################################################################################################################################
# Generate
########################################################################################################################################################
;generateConfig.workflows                            = false
;generateConfig.agents                               = false
;generateConfig.schedules                            = false
;generateConfig.locks                                = false
;generateConfig.calendars                            = false
########################################################################################################################################################
# Workflow
########################################################################################################################################################
# default: Etc/UTC
workflowConfig.default.timezone                     = US/Pacific
########################################################################################################################################################
# Workflow Job
########################################################################################################################################################
;jobConfig.forced.graceTimeout                       = 15
;jobConfig.forced.parallelism                        = 2
jobConfig.forced.failOnErrWritten                   = true
;jobConfig.forced.warnOnErrWritten                   = true
;jobConfig.forced.v1Compatible                       = true

# Generate retry instructions for all jobs.
# default: Generate retry instructions if Autosys n_retrys is defined.
;jobConfig.forced.retry.maxTries                     = 10
;jobConfig.forced.retry.delays                       = 30;60;60

# default: empty
jobConfig.forced.shell.unix.commandPrefix           = source
;jobConfig.forced.shell.windows.commandPrefix        =

# default: #!/bin/bash
;jobConfig.default.shell.unix.shebang                = #!/bin/sh
########################################################################################################################################################
# Agent
########################################################################################################################################################
;agentConfig.forced.controllerId                     = js7
;agentConfig.forced.agent                            = {"platform":"WINDOWS"}
;agentConfig.forced.agent                            = {"platform":"UNIX"}
;agentConfig.forced.agent                            = {"platform":"WINDOWS","standaloneAgent":{"agentName":"primaryAgent","url":"http://localhost:4445"}}
;agentConfig.forced.agent                            = {"platform":"UNIX","standaloneAgent":{"agentName":"primaryAgent","url":"http://localhost:4445"}}
;agentConfig.forced.agent                            = examples/agent_standalone.json

;agentConfig.default.controllerId                    = js7
;agentConfig.default.agent                           = examples/cluster_agent.json

# 1) configure agent mappings in this configuration file
;agentConfig.mappings                                = autosys_agent1=agent_standalone.json; \
                                                      autosys_agent2=cluster_agent.json
# 2) use external agent mappings configuration file (should have .config extension)
#    example of the contents of the file agent_mappings.config:
#       autosys_agent1  = examples/agent_standalone.json
#       autosys_agent2  = {"platform":"UNIX","standaloneAgent":{"agentName":"primaryAgent","url":"http://localhost:4445"}}
#       autosys_agent3  = cluster_agent.json
#       autosys_agent4  = examples/agent_standalone.json
;agentConfig.mappings                                = agent_mappings.config
########################################################################################################################################################
# Mock
########################################################################################################################################################
;mockConfig.forced.shell.unixScript                  = $HOME/MockScript.sh
;mockConfig.forced.shell.windowsScript               = %UserProfile%\MockScript.cmd

# INFO      Log arguments and always end successfully
# ERROR     Log arguments and fail if required parameters are missing
;mockConfig.forced.jitl.mockLevel                    = INFO
########################################################################################################################################################
# Schedule
########################################################################################################################################################
;scheduleConfig.forced.workingDayCalendarName        = AnyDays
;scheduleConfig.forced.nonWorkingDayCalendarName     = AnyDays
# If date_conditions=y|1, these settings are used, otherwise "false" is set
scheduleConfig.forced.planOrders                    = true
scheduleConfig.forced.submitOrders                  = true

scheduleConfig.default.workingDayCalendarName       = AnyDays
scheduleConfig.default.nonWorkingDayCalendarName    = AnyDays
# default: Etc/UTC
;scheduleConfig.default.timezone                     = Etc/UTC
########################################################################################################################################################
# Calendar
########################################################################################################################################################
# All generated calendars will be placed in this folder; otherwise, they will be placed in the root directory
# default: root directory
calendarConfig.forced.folder                        = Calendars
########################################################################################################################################################
# Lock
########################################################################################################################################################
;lockConfig.forced.capacity                          = 9999

# Default capacity for locks is based on Autosys resources
# default: 1
;lockConfig.default.capacity                         = 9999
########################################################################################################################################################
# Boards
########################################################################################################################################################
# lifetime - in minutes or days
;boardConfig.forced.lifetime                          = 2 * 24 * 60

# Default lifetime if a condition does not define a lookback
# default: 24 * 60 (24h)
boardConfig.default.lifetime                         = 60d

Settings

Section: Generate

Setting	Default	Explanation
`generateConfig.workflows`	`true`	Specifies that workflows should be created.
`generateConfig.agents`	`true`	Specifies that Agent configurations should be created.
`generateConfig.schedules`	`true`	Specifies that schedules should be created.
`generateConfig.calendars`	`true`	Specifies that calendars should be created.
`generateConfig.locks`	`true`	Specifies that Resource Locks should be created.

Section: Parser

Setting Default Explanation

parserConfig.extensions

jil

The default file name extensions for looking up AutoSys® JIL files in the input directory. If more than one extension is specified, they are separated by semicolon.

Example:

parserConfig.extensions = jil;txt

parserConfig.excluded.fileNames

Specifies files from their name that should be excluded when looking up AutoSys® JIL files in the input directory. If more than one file name is specified, they are separated by semicolon.

Example:

parserConfig.excluded.fileNames = machines.jil

Section: Autosys

Setting	Default	Explanation
`autosys.input.splitConfiguration`	`false`	Creates separate XML files containing the original AutoSys® job definition for each standalone job (a job without any conditions) or for each box job. Example: `autosys.input.splitConfiguration = true`
`autosys.input.diagram.generate`	`false`	Specifies that diagram files in SVG format should be created. Example: `autosys.input.diagram.generate = true`
`autosys.input.diagram.optimize.dependencies`	`false`	Attempts to resolve AutoSys® job conditions in order to reduce the dependencies (arrows) between various jobs in diagrams.
`autosys.input.diagram.outputFormat`	`svg`	Specifies the output format of diagram files.
`autosys.input.diagram.size`	`0`
`autosys.input.diagram.graphviz.executable`		Specifies the location of the graphviz `dot` executable to create SVG output files for diagrams. Example: `autosys.input.diagram.graphviz.executable = C:/Program Files/Graphviz/bin/dot.exe`
`autosys.input.diagram.graphviz.cleanupDotFiles`	`false`	Specifies whether the converter should delete the .dot (internal Graphviz format) files once the conversion is complete. Example: `autosys.input.diagram.graphviz.cleanupDotFiles = true`

Section: Workflow

Setting Default Explanation

workflowConfig.default.timezone

Etc/UTC

Specifies the default time zone that is added to converted workflows. This time zone for example is applied to JS7 - Admission Times for Jobs.

See List of tz database time zones

Section: Workflow Job

Setting	Default	Explanation
`jobConfig.forced.GraceTimeout`		Specifies the grace timeout after which a running job is terminated if the cancel/force operation is used, see JS7 - Job Instruction.
`jobConfig.forced.parallelism`		Specifies the number of parallel tasks that a job can be running for, see JS7 - Job Instruction.
`jobConfig.forced.failOnErrWritten`	`false`	Specifies the handling of job output to the stderr channel. If set to `true` then output to stderr is considered an error, otherwise such output is logged and no error is raised, , see JS7 - Job Instruction.
`jobConfig.forced.warnOnErrWritten`	`false`	Specifies the handling of job output to the stderr channel. If set to `true` then output to stderr is considered a warning, otherwise such output is logged and no warning is raised, , see JS7 - Job Instruction.
`jobConfig.forcedV1Compatible`	`false`	JS7 workflows can be operated in a JS1 compatibility mode. If set to `true` then for order variables and workflow variables in shell jobs respective environment variables are created that are prefixed with `SCHEDULER_PARAM_` followed by the uppercase variable name. for job arguments of shell jobs accordingly environment variables are created. Consider that environment variables are not added by the Converter but are created on-the-fly by the Controller and Agents.
`jobConfig.forced.retry.maxTries`		Forces use of the indicated maximum for use of the JS7 - Retry Instruction. Without this setting the AutoSys® `n_retrys` option is used to specify the maximum number of retries.
`jobConfig.forced.retry.delays`		Forces the indicated delay between retries.
`jobConfig.forced.shell.unix.commandPrefix`		Forces a prefix to be used for Unix shell jobs. Example: `jobConfig.forced.shell.unix.commandPrefix = source`
`jobConfig.forced.shell.windows.commandPrefix`		Forces a prefix to be used for Windows shell jobs.
`jobConfig.default.shell.unix.shebang`	`#!/bin/bash`	Specifies the shebang to be inserted to the begin of job scripts: Example: `jobConfig.default.shell.unix.shebang = #!/bin/sh`

Section: Agent

Settings Default Explanation

agentConfig.forced.controllerId Forces use of the given ControllerId for all generated agents.

agentConfig.default.controllerId Specifies the ControllerId to be used for generated agents if no ControllerId is provided in the agentConfig (forced.agent, default.agent, mappings) for a specific agent.

agentConfig.forced.agent

Forces use of the given Agent for all converted jobs.

Agent definition:

Expects a JSON value.
The JSON can be provided as:
- JSON text
  or
- JSON include file (relative to the properties file)

JSON elements:

Name	Default	Explanation
`platform`	`UNIX`	Forces use of the UNIX or WINDOWS platform.
`subagentClusterId`		Forces use the given `subagentClusterId` for converted jobs when an agentCluster is used.
`standaloneAgent`		See agent-schema.json.
`agentCluster`		See clusterAgent-schema.json.

Examples:
- {"platform":"UNIX"}
  - Forces use of the UNIX platform for all converted jobs.
- {"standaloneAgent":{"url":"http://localhost:4445"}}
  - Forces use of the standalone Agents in all converted jobs with the given url.
- {"platform":"UNIX", "standaloneAgent":{"agentName":"forced_agent", "url":"http://localhost:4445"}}
  - Forces use of a standalone Agent forced_agent in all converted jobs with the given url.

agentConfig.default.agent

Name	URL
`default_agent`	`http://localhost:4445`

This agent is used when an AutoSys® job does not provide machine information.

The syntax for assignments: see agentConfig.forced.agent Agent definition.

agentConfig.mappings

By default the Converter handles the mapping of Agent Names found in AutoSys® job machine attribute:

The syntax for mapping includes:

<autosys-agent-name>=<Agent definition>

<autosys-agent-name> is the name of the AutoSys® Agent(machine).
<Agent definition> see agentConfig.forced.agent Agent definition..

Example:

autosys_agent1={"platform":"UNIX", "standaloneAgent":{"agentName":"primaryAgent", "url":"http://unix:4445"}};autosys_agent2={"platform":"WINDOWS", "standaloneAgent":{"agentName":"secondaryAgent", "url":"http://win:4445"}}

The machine autosys_agent1 is mapped to the JS7 Agent with the name primaryAgent that is operated for Unix.
The machine class autosys_agent2 is mapped to the JS7 Agent with the name secondaryAgent that is operated for Windows.

or

autosys_agent1=examples/agent_standalone.json; autosys_agent2=examples/cluster_agent.json

The machine autosys_agent1 is mapped to the JS7 Agent defined in the examples/agent_standalone.json file.
The machine autosys_agent2 is mapped to the JS7 Agent defined in the examples/cluster_agent.json file.

Any number of mappings are specified separated by a semicolon. This setting can be split across a number of lines by use the "\" line continuation character at the end of each line that should be continued.

Configuration of mappings in a file:

With each run, the converter creates a report/agent_mapping.config file that lists all machine names on the left and empty mapping values on the right.
After manual adjustments, the file can be moved to another location and assigned as a file to the agentConfig.mappings in the next conversion.

Section: Mock

Setting Default Explanation

mockConfig.forced.jitl.mockLevel

Mock levels are used for dry runs of the conversion for JS7 - Job Templates (JITL JVM Jobs). If a mock level is specified then JVM Jobs will not be executed but will log and will optionally check arguments only.

The following values can be specified for this setting:

INFO: arguments are logged, execution of JVM Jobs is considered successful.
ERROR: arguments are logged, required arguments of JVM Jobs are checked and errors are raised in case of missing arguments.

Without this setting being specified the JITL JVM Jobs will be executed from the converted jobs.

mockConfig.forced.shell.unixScript

Mock scripts are used for dry runs of the conversion. The job scripts of converted Shell Jobs are replaced by the call to a mock script for Unix platforms. Consider that mock scripts have to be provided by the user and have to be available for Agents at run-time.

This setting specifies the path to a mock script for Unix, for example, $HOME/MockScript.sh

A mock script for Unix can look like this:

Example for Mock Script on Unix

#!/bin/sh
echo "------------------------------------------------------------"
echo "Start Unix Mock Script: $0"
echo "------------------------------------------------------------"
echo "Arguments: $*"
echo "------------------------------------------------------------"

If no mock script is specified then the original AutoSys® job script is used for the converted JS7 job.

mockConfig.forced.shell.windowsScript

Mock scripts are used for dry runs of the conversion. The job scripts of converted Shell Jobs are replaced by the call to a mock script for Windows platforms. Consider that mock scripts have to be provided by the user and have to be available for Agents at run-time.

Specifies the path to a mock script for Windows, for example, %UserProfile%\MockScript.cmd

A mock script for Windows can look like this:

Example for Mock Script on Windows

@echo off
@echo ------------------------------------------------------------
@echo Start Windows Mock Script: %0
@echo ------------------------------------------------------------
@echo Arguments: %*
@echo ------------------------------------------------------------

If no mock script is specified then the original AutoSys® job script is used for the converted JS7 job.

Section: Schedule

Setting	Default	Explanation
`scheduleConfig.forced.WorkingDayCalendarName`	`AnyDays`	Use of JS7 - Calendars is required. This setting forces use of the same calendar for working days with any schedules that are created. By default a calendar with the name `AnyDays` is assumed. Users have to create a calendar with this name prior to running the Converter.
`scheduleConfig.forced.NonWorkingDayCalendarName`		This setting forces use of the same calendar for non-working days with any schedules that are created. Use of a calendar for non-working days is optional. Users have to create a calendar with this name prior to running the Converter with this setting. If the setting is not used, then no non-working day calendar will be applied by the Converter.
`scheduleConfig.default.WorkingDayCalendarName`	`AnyDays`	This setting specifies a default calendar for working days for schedules that do not specify a calendar on their own. By default a calendar with the name `AnyDays` is assumed. Users have to create a calendar with this name prior to running the Converter.
`scheduleConfig.default.NonWorkingDayCalendarName`	`AnyDays`	This setting specifies use of a default calendar for non-working days with schedules that do not specify a calendar on their own. Use of a calendar for non-working days is optional. If this setting is used, then users have to create a calendar with this name prior to running the Converter.
`scheduleConfig.default.timezone`	`Etc/UTC`	With the JS7 a time zone is required for scheduled. This setting specifies the default time zone that is applied to Schedules. See List of tz database time zones
`scheduleConfig.forced.planOrders`	`false`	Orders are automatically planned for the JS7 - Daily Plan. The setting is applied if a JIL file's `date_condition` setting specifies `y` or `1`.
`scheduleConfig.forced.submitOrders`	`false`	Orders are automatically submitted to Controllers for the JS7 - Daily Plan. The setting is applied if a JIL file's `date_condition` setting specifies `y` or `1`.

Section: Calendar

Setting Default Explanation

calendarConfig.forced.folder

Root folder

Specifies the folder where the calendars are generated.

Example:

calendarConfig.forced.folder = Calendars

Section: Lock

Setting	Default	Explanation
`lockConfig.forced.capacity`		Specifies the capacity of JS7 - Resource Locks.
`lockConfig.default.capacity`	`1`	Default capacity for Resource Locks is based on AutoSys® resources

Section: Boards

Setting Default Explanation

boardConfig.forced.lifetime

Specifies the lifetime of Notices created by JS7 - Notice Boards

boardConfig.default.lifetime

24*60

Specifies the default lifetime for Notices. The default value is 24*60 (24 hours).

Example:

boardConfig.default.lifetime = 60d

Recommendations

It is recommended to create the following objects in the JS7 inventory prior to conversion:

Agents
- At least one Agent has to be available.
- The Converter offers to map Agent Names from AutoSys® to JS7 Agents. Such Agents have to be available in the JS7 prior to conversion. Users can register Agents as Standalone Agents and as Cluster Agents, see JS7 - Agent Management.
Calendars
- At lease one working day calendar has to be available. The calendar can be stored in an arbitrary folder. The name of the calendar can be specified with the scheduleConfig.forced.workingDayCalendarName and scheduleConfig.default.workingDayCalendarName settings or it will default to AnyDays.
- Optionally a non-working day celandar can be available. The calendar can be stored in an arbitrary folder. The name of the calendar can be specified with the
- scheduleConfig.forced.nonWorkingDayCalendarName and scheduleConfig.default.nonWorkingDayCalendarName settings. By default no non-working day Calendar will be applied.
- If the calendar specified by converted schedules is not available in the JS7 then imported schedules become invalid. They can be individually assigned a calendar in the JS7 inventory.

Further Resources

JS7 - Migration of AutoSys Jobs

Space shortcuts

Page tree

Introduction

Converter

Prerequisites

Download

Usage

Examples

Unix

Windows

Configuration

Settings

Section: Generate

Section: Parser

Section: Autosys

Section: Workflow

Section: Workflow Job

Section: Agent

Section: Mock

Section: Schedule

Section: Calendar

Section: Lock

Section: Boards

Recommendations

Further Resources

Space shortcuts

Page tree

JS7 - Migration of AutoSys Jobs - Converter Usage

Introduction

Converter

Prerequisites

Download

Usage

Examples

Unix

Windows

Configuration

Settings

Section: Generate

Section: Parser

Section: Autosys

Section: Workflow

Section: Workflow Job

Section: Agent

Section: Mock

Section: Schedule

Section: Calendar

Section: Lock

Section: Boards

Recommendations

Further Resources