...

Workflow Deployment Script

The script is offered for download and can be applied for frequently used deployment operations:

• The script is available for Linux and MacOS® using bash shell.
• The script terminates with exit code 0 to signal successful execution, with exit code 1 for command line argument errors and with exit code 4 for non-recoverable errors. Exit code 3 signals that no matching objects have been found.
• The script is intended as a baseline example for customization by JS7 users and by SOS within the scope of professional services. Examples make use of JS7 Release 2.7.2, bash 4.2, curl 7.29.0 and jq 1.6.0.
  

...

Usage: deploy-workflow.sh [Command] [Options] [Switches]

  Commands:
    export            --file [--format] --path --type [--use-short-path] [--start-folder] [--for-signing] 
    ..                --file [--format] --folder [--recursive] [--type] [--use-short-path] [--for-signing]]
                             [--no-draft] [--no-deployed] [--no-released] [--no-invalid]
    import            --file [--format] [--folder] [--overwrite] [--prefix] [--suffix]
    import-deploy     --file [--format] [--folder] [--algorithm]
    deploy            --path --type [--date-from] [--no-draft] [--no-deployed]
    ..                --folder [--recursive] [--date-from] [--no-draft] [--no-deployed]
    revoke            --path --type
    ..                --folder [--recursive]
    release           --path --type [--date-from]
    ..                --folder [--recursive] [--date-from]
    recall            --path --type
    ..                --folder [--recursive] [--type]
    store             --path --type --file
    remove            --path --type [--date-from]
    ..                --folder [--date-from]
    restore           --path --type --new-path [--prefix] [--suffix]
    ..                --folder --new-path [--prefix] [--suffix]
    delete            --path --type
    ..                --folder
    revalidate        --folder [--recursive]
    sign              --file      [--keystore |--key --cert] [--key-password] [--hash]
    ..                --directory [--keystore |--key --cert] [--key-password] [--hash]
    encrypt           --in [--infile --outfile] --cert [--java-home] [--java-lib]
    decrypt           --in [--infile --outfile] --key [--key-password] [--java-home] [--java-lib]

  Options:
    --url=<url>                        | required: JOC Cockpit URL
    --controller-id=<id[,id]>          | required: Controller ID
    --user=<account>                   | required: JOC Cockpit user account
    --password=<password>              | optional: JOC Cockpit password
    --ca-cert=<path>                   | optional: path to CA Certificate used for JOC Cockpit login
    --client-cert=<path>               | optional: path to Client Certificate used for login
    --client-key=<path>                | optional: path to Client Key used for login
    --timeout=<seconds>                | optional: timeout for request, default: 60
    --file=<path>                      | optional: path to export file or import file
    --format=<ZIP|TAR_GZ>              | optional: format of export file or import file
    --folder=<folder[,folder]>         | optional: list of inventory folders holding objects
    --start-folder=<folder>            | optional: start folder for export with relative paths
    --path=<path[,path]>               | optional: list of inventory paths to objects
    --type=<type[,type]>               | optional: list of object types such as WORKFLOW,SCHEDDULE
    --new-path=<path>                  | optional: new object path on restore
    --prefix=<string>                  | optional: prefix for duplicate objects on import
    --suffix=<string>                  | optional: suffix for duplicate objects on import
    --algorithm=<identifier>           | optional: signature algorithm for import, default: SHA512withECDSA
    --date-from=<date>                 | optional: update daily plan start date for deploy/release operation
    --directory=<directory>            | optional: path to directory with files that should be signed
    --keystore=<path>                  | optional: path to keystore file in PKCS12 format
    --key=<path>                       | optional: path to private key file in PEM format
    --key-password=<password>          | optional: password for keystore/private key file
    --cert=<path>                      | optional: path to certificate file in PEM format
    --hash=<sha256|sha512>             | optional: hash algorithm for signing such as sha256, sha512, default: sha256
    --in=<string>                      | optional: input string for encryption/decryption
    --infile=<path>                    | optional: input file for encryption/decryption
    --outfile=<path>                   | optional: output file for encryption/decryption
    --java-home=<directory>            | optional: Java Home directory for encryption/decryption, default: $JAVA_HOME
    --java-lib=<directory>             | optional: Java library directory for encryption/decryption, default: ./lib
    --audit-message=<string>           | optional: audit log message
    --audit-time-spent=<number>        | optional: audit log time spent in minutes
    --audit-link=<url>                 | optional: audit log link
    --log-dir=<directory>              | optional: path to directory holding the script's log files

  Switches:
    -h | --help                        | displays usage
    -v | --verbose                     | displays verbose output, repeat to increase verbosity
    -p | --password                    | asks for password
    -k | --key-password                | asks for keystore/key password
    -r | --recursive                   | specifies folders to be looked up recursively
    -o | --overwrite                   | overwrites objects on import
    -s | --for-signing                 | exports objects for digital signing
    -u | --use-short-path              | exports relative paths
    --no-draft                         | excludes draft objects
    --no-deployed                      | excludes deployed objects
    --no-released                      | excludes released objects
    --no-invalid                       | excludes invalid objects
    --show-logs                        | shows log output if --log-dir is used
    --make-dirs                        | creates directories if they do not exist

...

• export
  • Allows to export objects such as workflows to an archive file in .zip or .tar.gz format. The command comes in two flavors:
    • export individual objects specified by the --path and --type options.
      • Should relative paths be used in the archive file then the --start-folder option and --use-short-path switch can be applied.
    • export objects from folders using the --folder option and --recursive switch.
      • Optionally one or more object types can be specified and otherwise all objects will be exported, see --type option.
      • Should relative paths be used in the archive file then the  --use-short-path switch can be applied.
      • Export of objects can further be limited by use of the --no-* switches, see section Switches.
  • The archive file is specified from the --file and --format options.
  • If JOC Cockpit is operated for the High Security Level then the --for-signing switch can be used to export Controller Objects that should be digitally signed. Objects and signatures can be imported using the import-deploy command.
• import
  • Imports an archive file to the inventory. The operation applies to use of JOC Cockpit with the Low and Medium Security Level.
  • Users can specify if existing objects will be overwritten or if duplicate objects from the import file will be assigned a prefix or suffix or will be ignored.
• sign
  • Digitally signs workflows and job resources from an export archive file. The operation applies to use of JOC Cockpit with the High Security Level.
  • Signing includes to specify the Private Key and Certificate from files in PEM format or from a keystore. Optionally the hash algorithm sha256 or sha512 is specified.
  • The sequence of operations includes to export, to sign and to import-deploy signed objects.
  • The sign command works without access to the JS7 REST Web Service and does not require to specify options for connecting to JOC Cockpit.
• import-deploy
  
  • Imports an archive file to the inventory and deploys the included objects. The operation is applicable if JOC Cockpit is operated for the High Security Level.
    • As a prerequisite the archive file must be exported using the --for-signing switch.
    • Workflows and Job Resources from the archive file are digitally signed by the user. Signature files are added to the archive file.
  • On import the objects in the archive file are deployed to related Controllers as specified during export.
• deploy
  • Allows to deploy Controller Objects such as workflows. The command can be used in two flavors:
    • deploy individual objects specified by the --path and --type options.
    • deploy objects from folders using the --folder option and --recursive switch.
  • Deploying objects forwards them to Controllers and Agents.
    • More than one Controller ID can be specified like this: --controller-id=controller-uat-1,controller-uat-2
• revoke
  • Allows to undeploy Controller Objects such as workflows. The command can be used in two flavors:
    • revoke individual objects specified by the --path and --type options.
    • revoke objects from folders using the --folder option and --recursive switch.
  • Revoking Controller objects deletes them from the Controller and Agents, objects remain in draft status in the inventory.
    • More than one Controller ID can be specified like this: --controller-id=controller-uat-1,controller-uat-2
• release
  • Allows to release Automation Objects such as schedules. The command can be used in two flavors:
    • release individual objects specified by the --path and --type options.
    • release objects from folders using the --folder option and --recursive switch.
  • Releasing objects activates them for example for use by the Daily Plan.
• recall
  • Allows to unrelease Automation Objects such as schedules. The command can be used in two flavors:
    • recall individual objects specified by the --path and --type options.
    • recall objects from folders using the --folder option. The --type option and --recursive switch can optionally be specified.
  • Recalling objects deactivates them from further use, objects remain in draft status in the inventory.
• store
  • Allows to store an object such as a workflow or schedule from a file to the inventory.
    • The --file option specifies the file that holds the JSON representation of an object.
    • The --type option specifies the object type. 
    • The --path option specifies the folders and object name of the objects inventory location.
  • Objects are stored to the inventory in draft status and can be deployed or released using the related commands.
• remove
  • Allows to remove objects such as workflows or schedules from the inventory. The command can be used in two flavors:
    • remove individual objects specified by the --path and --type options.
    • remove objects from folders recursively using the --folder option.
  • Controller objects such as workflows are removed from the Controller and from the inventory. Automation objects such as schedules are removed from the inventory.
  • Removing objects moves them to the trash from which they can be restored or deleted
• restore
  • Allows to restore objects such as workflows or schedules from the trash. The command can be used in two flavors:
    • restore individual objects specified by the --path and --type options.
    • restore objects from folders recursively using the --folder option.
  • Restoring objects moves them from the trash to the inventory from which they can be deployed or released.
• delete
  • Allows to delete objects such as workflows or schedules from the trash. The command can be used in two flavors:
    • delete individual objects specified by the --path and --type options.
    • delete objects from folders recursively using the --folder option.
  • Deleting objects will permanently wipe them from the trash.
• revalidate
  • Allows to revalildate objects such as workflows or schedules from the inventory, for example after import. The command can be used for inventory folders.
• encrypt
  • Allows to encrypt a value using the --in option. If used to encrypt a file then --infile and --outfile options must be specified.
  • The --cert option specifies the path to a file holding the Certificate used for encryption.
  • Encryption is performed by Java libraries that are looked up in the ./lib sub-directory of the JS7 Unix Shell CLI. For details see JS7 - Encryption and Decryption.
• decrypt
  • Allows to decrypt a value using the --in option. If used to decrypt a file then --infile and --outfile options must be specified.
  • The --key option specifies the path to the Private Key used for decryption. If the Private Key is protected by a password, then the --key-password option or switch must be specified.
  • Decryption is performed by Java libraries that are looked up in the ./lib sub-directory of the JS7 Unix Shell CLI.

...

• --url
  • Specifies the URL by which JOC Cockpit is accessible using <http|https>://<host>:<port>.
  • Example: http://centostest-primary.sos:4446
  • Example: https://centostest-primary.sos:4443
• --user
  • Specifies the user account for login to JOC Cockpit. If JS7 - Identity Services are available for Client authentication certificates that are specified with the --client-cert and --client-key options then their common name (CN) attribute has to match the user account.
  • If a user account is specified then a password can be specified using the --password option or interactive keyboard input can be prompted using the -p switch.
• --password
  • Specifies the password used for the account specified with the --user option for login to JOC Cockpit.
  • Password input from the command line is considered insecure.
    • Consider use of the -p switch offering a secure option for interactive keyboard input.
    • Consider use of the encrypt command to encrypt a password: ./deploy-workflow.sh encrypt --in=root --cert=encrypt.crt.
      • The encryption result will include the prefix enc: followed by the encrypted symmetric key, initialization vector and encrypted secret separated by space.
      • If an encrypted password is specified, then it will be decrypted using the Private Key file: ./deploy-workflow.sh <command> --password="enc:BF8J8KP7TPlxy..." --key=encrypt.key.
• --controller-id
  • Specifies the identification of the Controller that holds related orders.
  • More than one Controller ID can be specified, separated by comma, for the export operation when using the --for-signing switch.
• --ca-cert
  • Specifies the path to a file in PEM format that holds the Root CA Certificate and optionally Intermediate CA Certificates to verify HTTPS connections to JOC Cockpit.
• --client-cert
  • Specifies the path to a file in PEM format that holds the Client Certificate if HTTPS mutual authentication is used..
• --client-key
  • Specifies the path to a file in PEM format that holds the Client Private Key if HTTPS mutual authentication is used..
• --timeout
  • Specifies the maximum duration for requests to the JS7 REST Web Service. Default: 60 seconds.
• --file
  • Specifies the location of an archive file that is used with export, import and import-deploy commands.
  • On export an existing archive file will be overwritten.
• --format
  • Specifies the format of the archive file indicated with the --file option.
  • The format can be one of ZIP or TAR_GZ. Default: ZIP. The JS7 can process archive files in .zip format on Unix.
• --folder
  • Specifies the inventory folder used for the related operation.
    • Folder specification starts from a / followed by one or more sub-folders.
    • More than one folder can be specified using comma as in --folder=/ProductDemo/AgentCluster,/ProductDemo/ScheduledExecution.
  • When used with the import and import-deploy commands, a single folder can be specified that is prepended the folders included with the archive file.
• --start-folder
  • Specifies the inventory folder used for relative paths in archive files when using the export command with the --path option, see --use-short-path switch.
• --path
  • Specifies the path of an object such as a workflow, job resource, schedule. A path starts from a /, optionally followed by a hierarchy of sub-folders, and the object name.
  • Objects are identified from thier path and object type.
• --type
  • Specifies the object type such as a workflow or schedule that is indicated together with the --path option to identify an object.
    • Controller Object types include: WORKFLOW,FILEORDERSOURCE,JOBRESOURCE,NOTICEBOARD,LOCK
    • Automation Object types include: SCHEDULE,WORKINGDAYSCALENDAR,NONWORKINGDAYSCALENDAR,JOBTEMPLATE,INCLUDESCRIPT,REPORT
  • When used with the export command for folders then more than one object type can be specified separated by comma, for example --type=WORKFLOW,JOBRESOURCE
• --new-path
  • When used with the restore command, the new path is specified to which the object will be restored in the inventory.
• --prefix
  • When used with the import command, a prefix can be specified that is prepended all objects that are imported.
  • If an object with the same name and prefix exists, then the object will not be imported.
• --suffix
  • When used with the import command, a suffix can be specified that is appended all objects that are imported.
  • If an object with the same name and suffix exists, then the object will not be imported.
• --algorithm
  • When used with the import-deploy command, the signature algorithm is specified that was used to digitally sign objects. Default: SHA512withECDSA.
  • The algorithm name is made up of the hash algorithm name such as SHA256, SHA512 and the encryption type of the Private Key such as ECDSA or RSA.
  • This offers to specify the following algorithm names: SHA256withECDSA, SHA256withRSA, SHA512withECDSA, SHA512withRSA.
• --date-from
  • Specifies the date starting from which the Daily Plan will be updated:
    • The --date-from=now option value specifies that the Daily Plan will be updated for orders starting from now.
    • The Daily Plan date in ISO date format can be specified, for example --date-from=2023-10-23.
    • If omitted then the Daily Plan will not be updated.
  • Orders in the Daily Plan can be updated for example if the underlying workflow or schedule is changed.
• --directory
  • When used with the sign command, specifies the directory in which workflow files with the extension *.workflow.json and job resources holding the extension .jobresource.json are looked up for signing. Sub-directories are looked up recursively.
  • All files found will be digitally signed by creating a signature file with the extension *.json.sig that holds the signature of the related object.
• --keystore
  • When used with the sign command, specifies the path to a keystore file in PKCS12 format. The keystore is expected to hold the Private Key and Certificate.
  • Only one of the options --keystore and --key can be specified.
• --key
  • When used with the sign and decrypt commands, specifies the path to a file that holds the Private Key  in PEM format used for signing/decrypting in PEM format.
  • Only one of the options --keystore and --key can be specified.
• --cert
  
  • When used with the sign and encrypt commands, specifies the path to a file that holds the CA-signed or self-signed X.509 Certificate. Alternatively, the path to a file holding the Public Key can be specified. The Certificate/Public Key is expected in PEM format.
  • For signing the argument is required if the --key option is used. The argument is optional If the --keystore option is used. The --cert option has precedence if used with the --keystore option.
  • For encryption the Certificate/Public Key must match the Private Key used for later decryption specified with the --key option.
• --key-password
  • When used with the sign and decrypt commands, specifies the password for access to the keystore using the --keystore option or key file using the --key option.
  • Password input from the command line is considered insecure.
    • Consider use of the -k switch or more elaborate mechanisms, for example by temporarily populating the system keystore form a security key such as a YubiKey® or similar.
    • Consider use of encrypted passwords as explained with the --password option.
• --hash
  • When used with the sign command,  specifies specifies the hash algorithm used to create a hash from a file that is to should be signed.
  • Possible values include sha256 and sha512. Default: sha256
• --in
  • When used with the encrypt and decrypt commands, specifies the input value that should be encrypted or decrypted.,
  • One of the options --in or --infile can be specified.
• --infile
  • When used with the encrypt and decrypt commands, specifies the path to the input file that should be encrypted/decrypted.
  • One of the options --in or --infile can be specified. This option requires use of the --outfile option.
• --outfile
  • When used with the encrypt command, specifies the path to the output file that will be created holding the encrypted content of the input file.
  • When used with the decrypt command, specifies the path to the output file that will be created holding the decrypted content of the input file.
  • The option is required if the --infile option is specified
• --java-home
  • When used with the encrypt and decrypt commands or with encrypted passwords, specifies the Java home directory. By default the JAVA_HOME environment variable is used to determine the location of Java.
  • The Java home directory is the top-level directory of a Java installation. The directory includes the bin sub-directory and java executable.
• --java-lib
  • When used with the encrypt and decrypt commands or with encrypted passwords, a number of Java libraries are required to perform encryption/decryption.
  • The Java libraries are expected in the lib sub-directory of the JS7 Unix Shell CLI. Default: ./lib.
• --audit-message
  • Specifies a message that is made available to the Audit Log.
  • Specification of Audit Log messages can be enforced on a per user basis and for a JS7 environment.
• --audit-time-spent
  • Specifies the time spent to perform an operation which is added to the Audit Log.
  • The option can be specified if the --audit-message option is used.
• --audit-link
  • Specifies a link (URL) which is added to the Audit Log.
  • The option can be specified if the --audit-message option is used.
• --log-dir
  • If a log directory is specified then the script will log information about processing steps to a log file in this directory.
  • File names are created according to the pattern: deploy-workflow.<yyyy>-<MM>-<dd>T<hh>-<mm>-<ss>.log
  • For example: deploy-workflow.2022-03-19T20-50-45.log

...

# common options for connection to the JS7 REST API
request_options=(--url=http://localhost:4446 --user=root --password=root --controller-id=controller)

# restore object from trash, using suffix for restored objectd
./deploy-workflow.sh restore "${request_options[@]}" \
    --path=/ap/NewFolder01/NewWorkflow01 --type=WORKFLOW --new-path=/ap/NewFolder01/NewWorkflow01 --suffix=restored

# delete object from trash
./deploy-workflow.sh delete "${request_options[@]}" \
    --path=/ap/NewFolder01/NewWorkflow01 --type=WORKFLOW

# delete objects from trash by folder
./deploy-workflow.sh delete "${request_options[@]}" \
    --folder=/ap/NewFolder01

...

Using encrypted Passwords

• Secrets can be encrypted and decrypted as explaind with JS7 - Encryption and Decryption.
• Encryption/decryption require related Java libraries that are available from the ./lib sub-directory.
• For creation of a Private Key and Certificate see JS7 - How to create X.509 Encryption Keys.
  
  

# create Private Key
openssl ecparam -name secp384r1 -genkey -noout -out encrypt.key

# create Certificate Signing Request, adjust the subject to your needs
openssl req -new -sha512 -nodes -key encrypt.key -out encrypt.csr -subj "/C=DE/ST=Berlin/L=Berlin/O=SOS/OU=IT/CN=Encrypt"

# create Certificate, adjust the validity period to your needs
openssl x509 -req -sha512 -days 1825 -signkey encrypt.key -in encrypt.csr -out encrypt.crt -extfile <(printf "keyUsage=critical,keyEncipherment,keyAgreement\n")


# encrypt athe secretpassword such as a password "root" using the Certificate, the encryption result will be returned and will look like: enc:BEXbHYa...
./MY_JS7_PASSWORD=$(./deploy-workflow.sh  encrypt --in="root" --cert=encrypt.crt)

# store the environment variable to your profile ($HOME/.bash_profile or similar) to make the encrypted password available to the shell
# export MY_JS7_PASSWORD=enc:BEXbHYa...


# options for connection to the JS7 REST API can specify the encryption result as password and the Private Key for decryption
request_options=(--url=http://localhost:4446 --user=root --password="enc:BEXbHYa...$MY_JS7_PASSWORD" --key=encrypt.key --controller-id=controller)

 # for example, when exporting workflows the Private Key is used to decrypt the password for access to the REST API on-the-fly
./deploy-workflow.sh export "${request_options[@]}" --file=export.zip --path=/ap/ap3jobs,/ap/Agent/apRunAsUser --type=WORKFLOW

# decrypt an encrypted secret using the Private Key
./deploy-workflow.sh decrypt --in="enc:BEXbHYa..." --key=encrypt.key

Resources

• API
  • JS7 - REST Web Service API
  • JS7 - PowerShell Module
• Controller Deployment Operations
  • JS7 - Unix Shell CLI for Controller Deployment
• Controller Status Operations
  • JS7 - Unix Shell CLI for Controller Status Operations
• Workflow Status Operations
  • JS7 - Unix Shell CLI for Workflow Status Operations
• Identity Service Deployment Operations
  • JS7 - Unix Shell CLI for Workflow Identity Service Deployment 
• Workflow JOC Cockpit Status Operations
  • JS7 - Unix Shell CLI for Workflow JOC Cockpit Status Operations