Introduction

• The JS7 offers to perform operations on Controllers and Agents by the JS7 - REST Web Service API.
  
  • For detailed information see the Technical Documentation of the REST Web Service API.
• The REST Web Service API can be accessed from Shell utilities such as curl.
• In addition, a PowerShell module is available for simplified access to the REST Web Service API. This is described in the JS7 - PowerShell Module article.

The Controller Management Script offered for Unix Shell can be applied to perform frequently used operations on Controllers and Agents.

Controller Management Script

The script is offered for download and can be applied for frequently used management operations:

• The script is available for Linux and MacOS® using bash shell.
• The script terminates with exit code 0 to signal successful execution, with exit code 1 for command line argument errors and with exit code 4 for non-recoverable errors. Exit code 3 signals that no matching objects have been found.
• The script is intended as a baseline example for customization by JS7 users and by SOS within the scope of professional services. Examples make use of JS7 Release 2.7.2, bash 4.2, curl 7.29.0 and jq 1.6.0.
  

Prerequisites

The Script requires the curl utility and the jq utility to be available from the operating system. 

jq ships with the MIT license, see https://opensource.org/licenses/MIT.

Download

Download: manage-controller.sh

Usage

Invoking the script without arguments displays the usage clause:

Usage: manage-controller.sh [Command] [Options] [Switches]

  Commands:
    register          --primary-url    [--primary-cluster-url]   [--primary-title]
                     [--secondary-url] [--secondary-cluster-url] [--secondary-title]
    unregister        --controller-id
    check             --controller-id --controller-url
    store-agent       --controller-id --agent-id --agent-name --agent-url [--title] [--alias] [--process-limit] [--hide]
    ..                --controller-id --agent-id --agent-name [--title] [--alias] [--process-limit]
                      --primary-subagent-id   --primary-url   [--primary-title]
                      --secondary-subagent-id --secondary-url [--secondary-title]
    delete-agent      --controller-id --agent-id
    deploy-agent      --controller-id --agent-id [--cluster]
    revoke-agent      --controller-id --agent-id [--cluster]
    store-subagent    --controller-id --agent-id --subagent-id --subagent-url [--title] [--role]
    delete-subagent   --controller-id --subagent-id
    store-cluster     --controller-id --cluster-id --agent-id --subagent-id [--priority] [--title]
    delete-cluster    --controller-id --cluster-id
    deploy-cluster    --controller-id --cluster-id
    revoke-cluster    --controller-id --cluster-id
    export-agent      --controller-id --file [--format] [--agent-id]
    import-agent      --controller-id --file [--format] [--overwrite]

  Options:
    --url=<url>                        | required: JOC Cockpit URL
    --user=<account>                   | required: JOC Cockpit user account
    --password=<password>              | optional: JOC Cockpit password
    --ca-cert=<path>                   | optional: path to CA Certificate used for JOC Cockpit login
    --client-cert=<path>               | optional: path to Client Certificate used for login
    --client-key=<path>                | optional: path to Client Key used for login
    --timeout=<seconds>                | optional: timeout for request, default: 60
    --controller-id=<id[,id]>          | required: Controller ID
    --controller-url=<url>             | optional: Controller URL for connection test
    --primary-url=<url>                | optional: Primary Controller/Director Agent URL
    --primary-cluster-url=<url>        | optional: Primary Controller Cluster URL
    --primary-title=<string>           | optional: Primary Controller/Director Agent title
    --primary-subagent-id=<id>         | optional: Primary Director Agent Subagent ID
    --secondary-url=<url>              | optional: Secondary Controller/Director Agent URL
    --secondary-cluster-url=<url>      | optional: Secondary Controller Cluster URL
    --secondary-title=<string>         | optional: Secondary Controller/Director Agent title
    --secondary-subagent-id=<id>       | optional: Secondary Director Agent Subagent ID
    --file=<path>                      | optional: path to export file or import file
    --format=<ZIP|TAR_GZ>              | optional: format of export file or import file
    --agent-id=<id[,id]>               | optional: Agent IDs
    --agent-name=<name>                | optional: Agent name
    --agent-url=<url>                  | optional: Agent URL
    --title=<string>                   | optional: Agent title or Subagent Cluster title
    --alias=<name[,name]>              | optional: Agent alias name
    --process-limit=<number>           | optional: Agent max. number of parallel processes
    --role=<primary|secondary|no>      | optional: Subagent role acting as Primary/Secondary Director Agent
    --subagent-id=<id[,id]>            | optional: Subagent ID
    --subagent-url=<url>               | optional: Subagent URL
    --cluster-id=<id>                  | optional: Subagent Cluster ID
    --priority=<first|next>            | optional: Subagent Cluster priority: active-passive, active-active
    --audit-message=<string>           | optional: audit log message
    --audit-time-spent=<number>        | optional: audit log time spent in minutes
    --audit-link=<url>                 | optional: audit log link
    --log-dir=<directory>              | optional: path to directory holding the script's log files

  Switches:
    -h | --help                        | displays usage
    -v | --verbose                     | displays verbose output, repeat to increase verbosity
    -p | --password                    | asks for password
    -o | --overwrite                   | overwrites objects on import
    -i | --hide                        | hides Agent
    -c | --cluster                     | signals a Cluster Agent
    --show-logs                        | shows log output if --log-dir is used
    --make-dirs                        | creates directories if they do not exist

see https://kb.sos-berlin.com/x/9YZvCQ

Commands

• export
  • Allows to export objects such as workflows to an archive file in .zip or .tar.gz format. The command comes in two flavors:
    • export individual objects specified by the --path and --type options.
      • Should relative paths be used in the archive file then the --start-folder option and --use-short-path switch can be applied.
    • export objects from folders using the --folder option and --recursive switch.
      • Optionally one or more object types can be specified and otherwise all objects will be exported, see --type option.
      • Should relative paths be used in the archive file then the  --use-short-path switch can be applied.
      • Export of objects can further be limited by use of the --no-* switches, see section Switches.
  • The archive file is specified from the --file and --format options.
  • If JOC Cockpit is operated for the High Security Level then the --for-signing switch can be used to export Controller Objects that should be digitally signed. Objects and signatures can be imported using the import-deploy command.
• import
  • Imports an archive file to the inventory. The operation applies to use of JOC Cockpit with the Low and Medium Security Level.
  • Users can specify if existing objects will be overwritten or if duplicate objects from the import file will be assigned a prefix or suffix or will be ignored.
• import-deploy
  
  • Imports an archive file to the inventory and deploys the included objects. The operation is applicable if JOC Cockpit is operated for the High Security Level.
    • As a prerequisite the archive file must be exported using the --for-signing switch.
    • Workflows and Job Resources from the archive file are digitally signed by the user. Signature files are added to the archive file.
  • On import the objects in the archive file are deployed to related Controllers as specified during export.
• deploy
  • Allows to deploy Controller Objects such as workflows. The command can be used in two flavors:
    • deploy individual objects specified by the --path and --type options.
    • deploy objects from folders using the --folder option and --recursive switch.
  • Deploying objects forwards them to Controllers and Agents.
    • More than one Controller ID can be specified like this: --controller-id=controller-uat-1,controller-uat-2
• revoke
  • Allows to undeploy Controller Objects such as workflows. The command can be used in two flavors:
    • revoke individual objects specified by the --path and --type options.
    • revoke objects from folders using the --folder option and --recursive switch.
  • Revoking Controller objects deletes them from the Controller and Agents, objects remain in draft status in the inventory.
    • More than one Controller ID can be specified like this: --controller-id=controller-uat-1,controller-uat-2
• release
  • Allows to release Automation Objects such as schedules. The command can be used in two flavors:
    • release individual objects specified by the --path and --type options.
    • release objects from folders using the --folder option and --recursive switch.
  • Releasing objects activates them for example for use by the Daily Plan.
• recall
  • Allows to unrelease Automation Objects such as schedules. The command can be used in two flavors:
    • recall individual objects specified by the --path and --type options.
    • recall objects from folders using the --folder option and --recursive switch.
  • Recalling objects deactivates them from further use, objects remain in draft status in the inventory.
• store
  • Allows to store an object such as a workflow or schedule from a file to the inventory.
    • The --file option specifies the file that holds the JSON representation of an object.
    • The --type option specifies the object type. 
    • The --path option specifies the folders and object name of the objects inventory location.
  • Objects are stored to the inventory in draft status and can be deployed or released using the related commands.
• remove
  • Allows to remove objects such as workflows or schedules from the inventory. The command can be used in two flavors:
    • remove individual objects specified by the --path and --type options.
    • remove objects from folders recursively using the --folder option.
  • Controller objects such as workflows are removed from the Controller and from the inventory. Automation objects such as schedules are removed from the inventory.
  • Removing objects moves them to the trash from which they can be restored or deleted
• restore
  • Allows to restore objects such as workflows or schedules from the trash. The command can be used in two flavors:
    • restore individual objects specified by the --path and --type options.
    • restore objects from folders recursively using the --folder option.
  • Restoring objects moves them from the trash to the inventory from which they can be deployed or released.
• delete
  • Allows to delete objects such as workflows or schedules from the trash. The command can be used in two flavors:
    • delete individual objects specified by the --path and --type options.
    • delete objects from folders recursively using the --folder option.
  • Deleting objects will permanently wipe them from the trash.
• revalidate
  • Allows to revalildate objects such as workflows or schedules from the inventory, for example after import. The command can be used for inventory folders.

Options

• --url
  • Specifies the URL by which JOC Cockpit is accessible using <http|https>://<host>:<port>.
  • Example: http://centostest-primary.sos:4446
  • Example: https://centostest-primary.sos:4443
• --user
  • Specifies the user account for login to JOC Cockpit. If JS7 - Identity Services are available for Client authentication certificates that are specified with the --client-cert and --client-key options then their common name (CN) attribute has to match the user account.
  • If a user account is specified then a password can be specified using the --password option or interactive keyboard input can be prompted using the -p switch.
• --password
  • Specifies the password used for the account specified with the --user option for login to JOC Cockpit.
  • Consider use of the -p switch offering a secure option for interactive keyboard input.
• --controller-id
  • Specifies the identification of the Controller that holds related orders.
  • More than one Controller ID can be specified, separated by comma, for the export operation when using the --for-signing switch.
• --ca-cert
  • Specifies the path to a file in PEM format that holds the Root CA Certificate and optionally Intermediate CA Certificates to verify HTTPS connections to JOC Cockpit.
• --client-cert
  • Specifies the path to a file in PEM format that holds the Client Certificate if HTTPS mutual authentication is used..
• --client-key
  • Specifies the path to a file in PEM format that holds the Client Private Key if HTTPS mutual authentication is used..
• --timeout
  • Specifies the maximum duration for requests to the JS7 REST Web Service. Default: 60 seconds.
• --file
  • Specifies the location of an archive file that is used with export, import and import-deploy commands.
  • On export an existing archive file will be overwritten.
• --format
  • Specifies the format of the archive file indicated with the --file option.
  • The format can be one of ZIP or TAR_GZ. Default: ZIP. The JS7 can process archive files in .zip format on Unix.
• --folder
  • Specifies the inventory folder used for the related operation.
    • Folder specification starts from a / followed by one or more sub-folders.
    • More than one folder can be specified using comma as in --folder=/ProductDemo/AgentCluster,/ProductDemo/ScheduledExecution.
  • When used with the import and import-deploy commands, a single folder can be specified that is prepended the folders included with the archive file.
• --start-folder
  • Specifies the inventory folder used for relative paths in archive files when using the export command with the --path option, see --use-short-path switch.
• --path
  • Specifies the path of an object such as a workflow, job resource, schedule. A path starts from a /, optionally followed by a hierarchy of sub-folders, and the object name.
  • Objects are identified from thier path and object type.
• --type
  • Specifies the object type such as a workflow or schedule that is indicated together with the --path option to identify an object.
    • Controller Object types include: WORKFLOW,FILEORDERSOURCE,JOBRESOURCE,NOTICEBOARD,LOCK
    • Automation Object types include: SCHEDULE,WORKINGDAYSCALENDAR,NONWORKINGDAYSCALENDAR,JOBTEMPLATE,INCLUDESCRIPT,REPORT
  • When used with the export command for folders then more than one object type can be specified separated by comma, for example --type=WORKFLOW,JOBRESOURCE
• --new-path
  • When used with the restore command, the new path is specified to which the object will be restored in the inventory.
• --prefix
  • When used with the import command, a prefix can be specified that is prepended all objects that are imported.
  • If an object with the same name and prefix exists, then the object will not be imported.
• --suffix
  • When used with the import command, a suffix can be specified that is appended all objects that are imported.
  • If an object with the same name and suffix exists, then the object will not be imported.
• --algorithm
  • When used with the import-deploy command, the signature algorithm is specified that was used to digitally sign objects. Default: SHA512withECDSA.
  • The algorithm name is made up of the hash algorithm name such as SHA256, SHA512 and the encryption type of the Private Key such as ECDSA or RSA.
  • This offers to specify the following algorithm names: SHA256withECDSA, SHA256withRSA, SHA512withECDSA, SHA512withRSA.
• --date-from
  • Specifies the date starting from which the Daily Plan will be updated:
    • The --date-from=now option value specifies that the Daily Plan will be updated for orders starting from now.
    • The Daily Plan date in ISO date format can be specified, for example --date-from=2023-10-23.
    • If omitted then the Daily Plan will not be updated.
  • Orders in the Daily Plan can be updated for example if the underlying workflow or schedule is changed.
• --audit-message
  • Specifies a message that is made available to the Audit Log.
  • Specification of Audit Log messages can be enforced on a per user basis and for a JS7 environment.
• --audit-time-spent
  • Specifies the time spent to perform an operation which is added to the Audit Log.
  • The option can be specified if the --audit-message option is used.
• --audit-link
  • Specifies a link (URL) which is added to the Audit Log.
  • The option can be specified if the --audit-message option is used.
• --log-dir
  • If a log directory is specified then the script will log information about processing steps to a log file in this directory.
  • File names are created according to the pattern: deploy-workflow.<yyyy>-<MM>-<dd>T<hh>-<mm>-<ss>.log
  • For example: deploy-workflow.2022-03-19T20-50-45.log

Switches

• -h | --help
  • Displays usage.
• -v | --verbose
  • Displays verbose log output that includes requests and responses with the JS7 REST Web Service.
  • When used twice as with -v -v then curl verbose output will be displayed.
• -p | --password
  • Asks the user for interactive keyboard input of the password used for the account specified with the --user option..
  • The switch is used for secure interactive input as an alternative to use of the option --password=<password>.
• -r | --recursive
  • Specifies that folders will be looked up recursively if the --folder option is used.
• -o | --overwrite
  • Specifies that objects with the same name and type will be overwritten when used with the import command.
• -s | --for-signing
  • Specifies that objects are exported for digital signing when used with the export command. For JS7 environments operated for the High Security Level digitally signed objects can be imported using the import-deploy command.
• -u | --use-short-path
  • Specifies that relative object paths will be applied to archive files when used with the export command.
  • An object path /a/b/c/workflow will be added as /c/workflow to the archive file. A folder /a/b/c will be added as /c to the archive file.
• --no-draft
  • Specifies that draft objects will be excluded when used with the export and deploy command.
• --no-deployed
  • Specifies that deployed objects will be excluded when used with the export and deploy command.
• --no-released
  • Specifies that released objects will be excluded when used with the export command.
• --no-invalid
  • Specifies that invalid objects will be excluded when used with the export command.
• --show-logs
  • Displays the log output created by the script if the --log-dir option is used.
• --make-dirs
  • If directories are missing that are indicated with the --log-dir option then they will be created.

Exit Codes

• 0: operation successful
• 1: argument errors
• 3: no objects found
• 4: JS7 REST Web Service is not reachable or reports errors

Examples

The following examples illustrate use cases for automated setup up of an environment with Controller and Agents.

Registering, Checking and Unregistering Controller

Controller instances are assigned a Controller ID on installation. When registering the Controller it will be registered with the given Controller ID.

Users can register a Controller from its URL. Users can check if the connection between JOC Cockpit and Controller can be established.

# common options for connection to JS7 REST API
request_options=(--url=http://localhost:4446 --user=root --password=root)  

# register Standalone Controller
./manage-controller.sh register ${request_options[@]} \
    --primary-url=http://localhost:4444 --primary-title="Standalone Controller"

# check Standalone Controller Connection
./manage-controller.sh check ${request_options[@]} --controller-url=http://localhost:4444

# unregister Standalone Controller
./manage-controller.sh unregister ${request_options[@]} --controller-id=controller

In a Controller Cluster both Controller instances are installed using the same Controller ID.

# common options for connection to JS7 REST API
request_options=(--url=http://localhost:4446 --user=root --password=root)

# register Controller Cluster
./manage-controller.sh register ${request_options[@]} \
    --primary-url=http://localhost:4544   --primary-title="Primary Controller" \
    --secondary-url=http://localhost:4644 --secondary-title="Secondary Controller"

# check Controller Cluster Connections
./manage-controller.sh check ${request_options[@]} --controller-url=http://localhost:4544
./manage-controller.sh check ${request_options[@]} --controller-url=http://localhost:4644

# unregister Controller Cluster
./manage-controller.sh unregister ${request_options[@]} \
    --controller-id=controller --controller-url=http://localhost:4544

Storing, Deploying, Revoking and Deleting Agents

Storing and Deploying Agents

When Agents are stored, then they are added or updated in the inventory and are set to the not deployed status.

When Agents are deployed, then the Agent configuration will be made available to the Controller which connects to the related Agent. As a prerequisite for deployment the Agent must be up & running and must be reachable for the Controller to establish a connection.

# common options for connection to JS7 REST API
request_options=(--url=http://localhost:4446 --user=root --password=root --controller-id=controller)

# store Standalone Agent
./manage-controller.sh store-agent ${request_options[@]} \
    --agent-id=StandaloneAgent --agent-name=StandaloneAgent \
    --agent-url="http://localhost:4445" --title="Standalone Agent"

# deploy Standalone Agent
./manage-controller.sh deploy-agent ${request_options[@]} --agent-id=StandaloneAgent

An Agent Cluster includes the Primary and Secondary Director Agents. As a prerequisite for deployment the Director Agents must be up & running and must be reachable for the Controller to establish a connection.

# common options for connection to JS7 REST API
request_options=(--url=http://localhost:4446 --user=root --password=root --controller-id=controller)

# store Agent Cluster with Primary/Secondary Director Agent
./manage-controller.sh store-agent ${request_options[@]} \
    --agent-id=AgentCluster --agent-name=AgentCluster --title="Agent Cluster" \
    --primary-subagent-id=primary-director     --primary-url=http://localhost:4545   --primary-title="Primary Director" \
    --secondary-subagent-id=secondary-director --secondary-url=http://localhost:4645 --secondary-title="Secondary Director"

# deploy Agent Cluster
./manage-controller.sh deploy-agent ${request_options[@]} --agent-id=AgentCluster --cluster

Revoking and Deleting Agents

When Agents are revoked, then they will be deleted from the Controller. The Agent configuration remains in place with the inventory and will be set to the not deployed status.

When Agents are removed, then they will be deleted from the Controller and from the inventory.

# common options for connection to JS7 REST API
request_options=(--url=http://localhost:4446 --user=root --password=root --controller-id=controller)

# revoke Standalone Agent
./manage-controller.sh revoke-agent ${request_options[@]} --agent-id=StandaloneAgent

# delete Standalone Agent
./manage-controller.sh delete-agent ${request_options[@]} --agent-id=StandaloneAgent

Revoking an Agent Cluster includes to revoke the Subagents and Subagent Clusters from the Controller using a single operation.

Deleting an Agent Cluster wipes the Agent configuration from both Controller and inventory.

# common options for connection to JS7 REST API
request_options=(--url=http://localhost:4446 --user=root --password=root --controller-id=controller)

# revoke Agent Cluster
./manage-controller.sh revoke-agent ${request_options[@]} --agent-id=AgentCluster --cluster

# delete Agent Cluster
./manage-controller.sh delete-agent ${request_options[@]} --agent-id=AgentCluster

Storing and Deleting Subagents

When Subagents are stored, then they are added or updated in the inventory and will be set to the not deployed status.

When Subagents are deployed, then they are made available to the Agent Cluster which connects to the related Subagent. As a prerequisite for deployment the Subagent must be up & running and must be reachable for the Agent Cluster to establish a connection.

# common options for connection to JS7 REST API
request_options=(--url=http://localhost:4446 --user=root --password=root --controller-id=controller)

# store Subagents
./manage-controller.sh store-subagent ${request_options[@]} \
    --agent-id=AgentCluster --subagent-id=Subagent_01 \
    --subagent-url=http://localhost:4745 --title="Subagent 01"

./manage-controller.sh store-subagent ${request_options[@]} \
    --agent-id=AgentCluster --subagent-id=Subagent_02 \
    --subagent-url=http://localhost:4845 --title="Subagent 02"

# delete Subagents
./manage-controller.sh delete-subagent ${request_options[@]} --subagent-id=Subagent_01
./manage-controller.sh delete-subagent ${request_options[@]} --subagent-id=Subagent_02

Storing, Deploying, Revoking and Deleting Subagent Clusters

Storing and Deploying Subagent Clusters

When Subagent Clusters are stored, then they are added or updated in the inventory and will be set to the not deployed status.

When Subagent Clusters are deployed, then the cluster configuration will be made available to the Agent Cluster.

# common options for connection to JS7 REST API
request_options=(--url=http://localhost:4446 --user=root --password=root --controller-id=controller)

# store active-passive (fixed-priority) Subagent Cluster
./manage-controller.sh store-cluster ${request_options[@]} \
    --agent-id=AgentCluster --cluster-id=active-passive \
    --subagent-id=Subagent_01,Subagent_02 --priority=first --title="Active-Passive Cluster"

# store active-active (round-robin) Subagent Cluster
./manage-controller.sh store-cluster ${request_options[@]} \
    --agent-id=AgentCluster --cluster-id=active-active \
    --subagent-id=Subagent_01,Subagent_02 --priority=next --title="Active-Active Cluster"

# deploy Subagent Clusters
./manage-controller.sh deploy-cluster ${request_options[@]} --cluster-id=active-passive
./manage-controller.sh deploy-cluster ${request_options[@]} --cluster-id=active-active

Revoking and Deleting Subagent Clusters

When Subagent Clusters are revoked, then they are deleted from the Agent Cluster. The cluster configuration remains in place with the inventory and will be set to the not deployed status.

When Subagent Clusters are deleted, then they are deleted from the Agent Cluster and from the inventory.

# common options for connection to JS7 REST API
request_options=(--url=http://localhost:4446 --user=root --password=root --controller-id=controller)

# revoke Subagent Clusters
./manage-controller.sh revoke-cluster ${request_options[@]} --cluster-id=active-passive
./manage-controller.sh revoke-cluster ${request_options[@]} --cluster-id=active-active

# delete Subagent Clusters
./manage-controller.sh delete-cluster ${request_options[@]} --cluster-id=active-passive
./manage-controller.sh delete-cluster ${request_options[@]} --cluster-id=active-active

Exporting and Importing Agents

Agent configurations can be exported to an archive file. They can be imported, for example when setting up next JS7 environments.

For import users can specify that existing Agent configurations with the same name should be overwritten.

# common options for connection to JS7 REST API
request_options=(--url=http://localhost:4446 --user=root --password=root --controller-id=controller)

# export Agents from their Agent ID
./manage-controller.sh export-agent ${request_options[@]} \
    --file=export_agents.zip --agent-id=StandaloneAgent,AgentCluster

# import Agents
./manage-controller.sh import-agent ${request_options[@]} --file=export_agents.zip

Resources

• API
  • JS7 - REST Web Service API
  • JS7 - PowerShell Module
• Controller Status Operations
  • JS7 - Unix Shell CLI for Controller Status Operations
• Workflow Deployment Operations
  • JS7 - Unix Shell CLI for Workflow Deployment Operations
• Workflow Status Operations
  • JS7 - Unix Shell CLI for Workflow Status Operations