You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 36 Current »

Introduction

The Workflow Operation Script offered for Unix Shell can be applied to perform frequently used status operations on workflows, jobs and orders.

Workflow Operation Script

OperationObjectDocumentation
add / cancelOrder

JS7 - Workflows - Status Operations on Orders

suspend / resume
let run
transfer
suspend / resumeWorkflow

JS7 - Workflows - Status Operations on Workflows

stop / unstopJob, Instructions

JS7 - Workflows - Status Operations on Jobs
JS7 - Workflows - Status Operations on Instructions

skip / unskip
post / get / deleteNotice

JS7 - Notice Boards

encrypt / decryptNone

Encrypt / decrypt strings and files


The script is offered for download and can be applied for frequently used operations on workflows, jobs and orders:

  • The script is available for Linux and MacOS® using bash shell.
  • The script terminates with exit code 0 to signal successful execution, with exit code 1 for command line argument errors and with exit code 4 for non-recoverable errors. Exit code 3 signals that no matching objects have been found.
  • The script is intended as a baseline example for customization by JS7 users and by SOS within the scope of professional services. Examples make use of JS7 Release 2.7.2, bash 4.2, curl 7.29.0 and jq 1.6.0.

Prerequisites

The Script requires the curl utility and the jq utility to be available from the operating system. 

jq ships with the MIT license, see https://opensource.org/licenses/MIT.

Download

Download: JS7 - Download (Section: Unix Shell ClI)

Usage

Invoking the script without arguments displays the usage clause:


Usage
Usage: operate-workflow.sh [Command] [Options] [Switches]

  Commands:
    add-order         --workflow  [--date-to] [--order-name] [--block-position] [--start-position] [--end-position] [--variable] [--force]
    cancel-order     [--workflow] [--folder] [--recursive] [--order-id] [--state] [--date-from] [--date-to] [--time-zone] [--force]
    suspend-order    [--workflow] [--folder] [--recursive] [--order-id] [--state] [--date-from] [--date-to] [--time-zone] [--force]
    resume-order     [--workflow] [--folder] [--recursive] [--order-id] [--state] [--label] [--variable]
    letrun-order     [--workflow] [--folder] [--recursive] [--order-id] [--state]
    transfer-order    --workflow] [--folder] [--recursive]
    suspend-workflow  --workflow  [--folder] [--recursive]
    resume-workflow   --workflow  [--folder] [--recursive]
    stop-job          --workflow --label
    unstop-job        --workflow --label
    skip-job          --workflow --label
    unskip-job        --workflow --label
    post-notice       --notice-board  [--notice-id] [--notice-lifetime]
    get-notice       [--notice-board] [--notice-id] [--folder] [--recursive] [--date-to]
    delete-notice    [--notice-board] [--notice-id] [--folder] [--recursive] [--date-to]
    encrypt           --in [--infile --outfile] --cert [--java-home] [--java-lib]
    decrypt           --in [--infile --outfile] --key [--key-password] [--java-home] [--java-lib] 

  Options:
    --url=<url>                        | required: JOC Cockpit URL
    --controller-id=<id>               | required: Controller ID
    --user=<account>                   | required: JOC Cockpit user account
    --password=<password>              | optional: JOC Cockpit password
    --ca-cert=<path>                   | optional: path to CA Certificate used for JOC Cockpit login
    --client-cert=<path>               | optional: path to Client Certificate used for login
    --client-key=<path>                | optional: path to Client Key used for login
    --timeout=<seconds>                | optional: timeout for request, default: 15
    --order-name=<string>              | optional: name for order, default: <current user>
    --block-position=<label>           | optional: label for block instruction that holds start position
    --start-position=<label>           | optional: label from which the order will be started
    --end-position=<label[,label]>     | optional: list of labels before which the order will terminate
    --variable=<key=value[,key=value]> | optional: list of variables holding key/value pairs
    --date-from=<date>                 | optional: order past scheduled date
    --date-to=<date>                   | optional: order scheduled date or notice date, default: now
    --time-zone=<tz>                   | optional: time zone for dates, default: <current-time-zone>
                                                   see https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
    --state=<state[,state]>            | optional: list of states limiting orders to be processed such as
                                                   SCHEDULED, INPROGRESS, RUNNING, SUSPENDED, WAITING, FAILED
    --folder=<path[,path]>             | optional: list of folders holding workflows, orders, notice boards
    --workflow=<name[,name]>           | optional: list of workflow names
    --order-id=<id[,id]>               | optional: list of order identifiers
    --label=<label[,label]>            | optional: list of labels for jobs
    --notice-board=<name[,name]>       | optional: list of notice boards
    --notice-id=<id>                   | optional: notice identifier, default: <current date>
    --notice-lifetime=<period>         | optional: lifetime for notice
    --key=<path>                       | optional: path to private key file in PEM format
    --key-password=<password>          | optional: password for private key file
    --cert=<path>                      | optional: path to certificate file in PEM format
    --in=<string>                      | optional: input string for encryption/decryption
    --infile=<path>                    | optional: input file for encryption/decryption
    --outfile=<path>                   | optional: output file for encryption/decryption
    --java-home=<directory>            | optional: Java Home directory for encryption/decryption, default: $JAVA_HOME
    --java-lib=<directory>             | optional: Java library directory for encryption/decryption, default: ./lib
    --audit-message=<string>           | optional: audit log message
    --audit-time-spent=<number>        | optional: audit log time spent in minutes
    --audit-link=<url>                 | optional: audit log link
    --log-dir=<directory>              | optional: path to directory holding the script's log files

  Switches:
    -h | --help                        | displays usage
    -v | --verbose                     | displays verbose output, repeat to increase verbosity
    -p | --password                    | asks for password
    -k | --key-password                | asks for key password
    -r | --recursive                   | specifies folders to be looked up recursively
    -f | --force                       | specifies forced start or termination of jobs
    --show-logs                        | shows log output if --log-dir is used
    --make-dirs                        | creates directories if they do not exist 

Commands

  • add-order
    • Adds an order to a workflow. Allows to specify the start position and end positions in the workflow in case that specific jobs only should be executed.
    • Orders can be forced to start and to ignore JS7 - Admission Times for Jobs using the --force switch.
  • cancel-order
    • Terminates an order which optionally includes to forcibly terminate running jobs using the --force switch.
  • suspend-order, resume-order
    • Suspends an order which optionally includes to forcibly terminate running jobs using the --force switch.
    • Resumes a previously suspended order, optionally from a different position in the workflow using the --label option.
  • letrun-order
  • transfer-order
    • Orders attached a previous version of a workflow for which a newer version exists are transferred to the latest version of the workflow.
  • suspend-workflow, resume-workflow
    • Suspends the worfklow and freezes orders in the given workflow.
    • Resumes a previously suspended workflow. Frozen orders will awake and will continue.
  • stop-job, unstop-job
    • Stopping a job will prevent the job from running. A stopped job will suspend orders arriving at the job label.
    • Unstopping a previously stopped job will allow orders to execute the job. Suspended orders must be resumed to execute the job.
  • skip-job, unskip-job
    • Skipping a job will prevent the job from running. Orders will skip the job and will continue with the next instruction.
    • Unskipping a previously skipped job will allow orders to execute the job.
  • post-notice, get-notice, delete-notice
  • encrypt
    • Allows to encrypt a value using the --in option. If used to encrypt a file then --infile and --outfile options must be specified.
    • The --cert option specifies the path to a file holding the Certificate used for encryption.
    • Encryption is performed by Java libraries that are looked up in the ./lib sub-directory of the  JS7 Unix Shell CLI. For details see JS7 - Encryption and Decryption.
  • decrypt
    • Allows to decrypt a value using the --in option. If used to decrypt a file then --infile and --outfile options must be specified.
    • The --key option specifies the path to the Private Key used for decryption. If the Private Key is protected by a password, then the --key-password option or switch must be specified.
    • Decryption is performed by Java libraries that are looked up in the ./lib sub-directory of the JS7 Unix Shell CLI.

Options

  • --url
  • --user
    • Specifies the user account for login to JOC Cockpit. If JS7 - Identity Services are available for Client authentication certificates that are specified with the --client-cert and --client-key options then their common name (CN) attribute has to match the user account.
    • If a user account is specified then a password can be specified using the --password option or interactive keyboard input can be prompted using the -p switch.
  • --password
    • Specifies the password used for the account specified with the --user option for login to JOC Cockpit.
    • Password input from the command line is considered insecure.
      • Consider use of the -p switch offering a secure option for interactive keyboard input.
      • Consider use of the encrypt command to encrypt a password: ./deploy-workflow.sh encrypt --in=root --cert=encrypt.crt.
        • The encryption result will include the prefix enc: followed by the encrypted symmetric key, initialization vector and encrypted secret separated by space.
        • If an encrypted password is specified, then it will be decrypted using the Private Key file: ./deploy-workflow.sh <command> --password="enc:BF8J8KP7TPlxy..." --key=encrypt.key.
  • --controller-id
    • Specifies the identification of the Controller that holds related orders.
  • --ca-cert
    • Specifies the path to a file in PEM format that holds the Root CA Certificate and optionally Intermediate CA Certificates to verify HTTPS connections to JOC Cockpit.
  • --client-cert
    • Specifies the path to a file in PEM format that holds the Client Certificate if HTTPS mutual authentication is used..
  • --client-key
    • Specifies the path to a file in PEM format that holds the Client Private Key if HTTPS mutual authentication is used..
  • --timeout
    • Specifies the maximum duration for requests to the JS7 REST Web Service. Default: 60 seconds.
  • --order-name
    • Specifies a qualifier that is added to newly created orders. Used with add-order command.
  • --block-position
    • Specifies the label of a block instruction such as JS7 - Resource Locks, should the --start-position option be used for a position inside the block instruction. Used with add-order command.
  • --start-position
    • Specifies the label of an instruction in the workflow from which the order is started, for example from a job. Used with add-order and resume-order commands.
    • The label of a top-level instruction can be specified. For start positions from nested instructions the --block-position option has to be specified too.
  • --end-position
    • Specifies the label of an instruction before which the order will terminate. More than one label can be specified separated by comma. Used with add-order command.
    • If the --block-position option is used then the end position is inside the block. Otherwise the order will terminate when reaching the block's end.
  • --variable
    • One or more variables can be specified that hold key/value pairs separated by comma. Used with add-order and resume-order commands.
    • Example: --variable="myVar1=myValue1,myVar2=myValue2"
  • --date-from
    • Specifies the date and time in ISO format of the daily plan date, for example 2023-10-23. Used with cancel-order and suspend-order commands.
    • Dates can be calculated from the date command, for example:
      • --date-to="$(TZ=Europe/London date +'%Y-%m-%d')" specifies the daily plan date before the current day in the Europe/London time zone.
      • --date-to="$(TZ=Europe/London date --date="1 day ago" +'%Y-%m-%d')" specifies the daily plan date before yesterday.
  • --date-to
    • Specifies the date and time in ISO format of the daily plan date, for example 2023-10-23. Used with add-order, cancel-order, suspend-order, get-notice and delete-notice commands.
    • Dates can be calculated from the Unix OS date command, for example:
      • --date-to="$(TZ=Europe/London date +'%Y-%m-%d')" specifies the daily plan date before the current day in the Europe/London time zone.
      • --date-to="$(TZ=Europe/London date --date="1 day ago" +'%Y-%m-%d')" specifies the daily plan date before yesterday.
  • --time-zone
  • --state
    • Specifies one or more states - separated by comma - for which orders should be processed. Used with cancel-order, suspend-order, resume-order and letrun-order commands.
    • Valid states include PENDING, SCHEDULED, INPROGRESS, RUNNING, SUSPENDED, WAITING, PROMPTING, FAILED, BLOCKED.
    • For example --state=SCHEDULED,SUSPENDED,FAILED will process orders holding any of the states specified.
  • --folder
    • Specifies one or more inventory folders from absolute paths - separated by comma - holding workflows that should be processed.
    • For example --folder=/ProductDemo/CyclicExecution,/ProductDemo/ScheduledExecution will process workflows in the given folders.
    • If the --recursive switch is used then sub-folders will be looked up recursively.
  • --workflow
    • Specifies one or more workflows - separated by comma - that should be processed. Used with most of the commands.
    • For example --workflow=Cyclic-Check,Daily-EOD will consider the Cyclic-Check and Daily-EOD workflows.
  • --order-id
    • Specifies one or more order identifiers - separated by comma - for which orders should be processed. Used with cancel-order, suspend-order, resume-order and letrun-order commands.
    • For example --order-id=#2024-08-25#T54565139012-sos,#2024-08-25#T56189833113-sos will process the indicated orders.
  • --label
    • Specifies one or more labels - separated by comma - for jobs that should be processed. Used with stop-job, unstop-job, skip-job and unskip-job commands.
    • For example --label=job1,job2 will consider jobs assigned the indicated labels.
  • --notice-board
    • Specifies one or more JS7 - Notice Boards - separated by comma - that should be processed. Used with the post-notice, get-notice and delete-notice commands.
  • --notice-id
    • Specifies the identifier of a notice, frequently the daily plan date is used, for example 2024-08-21. Used with the post-notice, get-notice and delete-notice commands.
  • --notice-lifetime
    • Specifies the max. period for which the notice will be available. Periods such as 1h, 30m can be specified. Used with the post-notice command.
  • --key
    • When used with the decrypt command, specifies the path to a file that holds the Private Key used for decrypting in PEM format.
  • --cert
    • When used with the encrypt command, specifies the path to a file that holds the CA signed or self-signed X.509 Certificate. Alternatively the path to a file holding the Public Key can be specified. The Certificate is expected in PEM format.
    • For encryption the Certificate must match the Private Key used for later decryption specified with the --key option.
  • --key-password
    • When used with the decrypt command, specifies the password for access to the key file using the --key option.
    • Password input from the command line is considered insecure.
      • Consider use of the -k switch or more elaborate mechanisms, for example by temporarily populating the system keystore form a security key such as a YubiKey® or similar.
      • Consider use of encrypted passwords as explained with the --password option.
  • --in
    • When used with the encrypt and decrypt commands, specifies the input value that should be encrypted or decrypted.,
    • One of the options --in or --infile can be specified.
  • --infile
    • When used with the encrypt and decrypt commands, specifies the path to the input file that should be encrypted/decrypted.
    • One of the options --in or --infile can be specified. This option requires use of the --outfile option.
  • --outfile
    • When used with the encrypt command, specifies the path to the output file that will be created holding the encrypted content of the input file.
    • When used with the decrypt command, specifies the path to the output file that will be created holding the decrypted content of the input file.
    • The option is required if the --infile option is specified
  • --java-home
    • When used with the encrypt and decrypt commands or with encrypted passwords, specifies the Java home directory. By default the JAVA_HOME environment variable is used to determine the location of Java.
    • The Java home directory is the top-level directory of a Java installation. The directory includes the bin sub-directory and java executable.
  • --java-lib
    • When used with the encrypt and decrypt commands or with encrypted passwords, a number of Java libraries are required to perform encryption/decryption.
    • The Java libraries are expected in the lib sub-directory of the JS7 Unix Shell CLI. Default: ./lib.
  • --audit-message
    • Specifies a message that is made available to the Audit Log.
    • Specification of Audit Log messages can be enforced on a per user basis and for a JS7 environment.
  • --audit-time-spent
    • Specifies the time spent to perform an operation which is added to the Audit Log.
    • The option can be specified if the --audit-message option is used.
  • --audit-link
    • Specifies a link (URL) which is added to the Audit Log.
    • The option can be specified if the --audit-message option is used.
  • --log-dir
    • If a log directory is specified then the script will log information about processing steps to a log file in this directory.
    • File names are created according to the pattern: operate-workflow.<yyyy>-<MM>-<dd>T<hh>-<mm>-<ss>.log
    • For example: operate-workflow.2022-03-19T20-50-45.log

Switches

  • -h | --help
    • Displays usage.
  • -v | --verbose
    • Displays verbose log output that includes requests and responses with the JS7 REST Web Service.
    • When used twice as with -v -v then curl verbose output will be displayed.
  • -p | --password
    • Asks the user for interactive keyboard input of the password used for the account specified with the --user option..
    • The switch is used for secure interactive input as an alternative to use of the option --password=<password>.
  • -k | --key-password
    • Asks the user for interactive keyboard input of the password used for access to a keystore or key file specified with the --keystore or --key options.
    • The switch is used for secure interactive input as an alternative to use of the --key-password=<password> option.
  • -r | --recursive
    • Specifies that folders will be looked up recursively if the --folder option is used.
  • -f | --force
  • --show-logs
    • Displays the log output created by the script if the --log-dir option is used.
  • --make-dirs
    • If directories are missing that are indicated with the --log-dir option then they will be created.

Exit Codes

  • 0: operation successful
  • 1: argument errors
  • 3: no objects found
  • 4: JS7 REST Web Service is not reachable or reports errors

Examples

The following examples illustrate typical use cases.

Adding Orders

Examples for Adding Orders
# common options for connection to JS7 REST API
request_options=(--url=http://localhost:4446 --user=root --password=root --controller-id=controller)

# add ad hoc order
./operate-workflow.sh add-order "${request_options[@]}" \
	--workflow=ap3jobs

# add ad hoc order for specific job
./operate-workflow.sh add-order "${request_options[@]}" \
	--workflow=ap3jobs --start-position=job2 --end-position=job3

# add pending order
./operate-workflow.sh add-order "${request_options[@]}" \
	--workflow=ap3jobs --date-to=never

# add scheduled order and force admission
./operate-workflow.sh add-order "${request_options[@]}" \
	--workflow=ap3jobs --order-name=sample-1 --date-to=now+15 --force

Cancelling Orders

Examples for Cancelling Orders
# common options for connection to JS7 REST API
request_options=(--url=http://localhost:4446 --user=root --password=root --controller-id=controller)

# cancel orders by state
./operate-workflow.sh cancel-order "${request_options[@]}" \
    --workflow=ap3jobs --date-to=-2h --state=SCHEDULED,PROMPTING,SUSPENDED,INPROGRESS,RUNNING

# cancel orders by state, yesterday's start date, folder recursively
./operate-workflow.sh cancel-order "${request_options[@]}" \
	--folder=/ap --recursive --date-to="$(TZ=Europe/London date + '%Y-%m-%d')" --state=SCHEDULED,PROMPTING,SUSPENDED,INPROGRESS,RUNNING

Suspending and Resuming Orders

Examples for Suspending and Resuming Orders
# common options for connection to JS7 REST API
request_options=(--url=http://localhost:4446 --user=root --password=root --controller-id=controller)

# suspend orders in workflow
./operate-workflow.sh suspend-order "${request_options[@]}" \
	--workflow=ap3jobs

# suspend orders in workflow and terminate running job
./operate-workflow.sh suspend-order "${request_options[@]}" \
	--workflow=ap3jobs --force

# resume suspended orders in workflow
./operate-workflow.sh resume-order "${request_options[@]}" \
	--workflow=ap3jobs --state=SUSPENDED

Letting run Orders

Example for Letting Run Orders
# common options for connection to JS7 REST API
request_options=(--url=http://localhost:4446 --user=root --password=root --controller-id=controller)

# let run waiting orders in workflow
./operate-workflow.sh letrun-order "${request_options[@]}" \
	--workflow=ap3jobs --state=WAITING

Transferring Orders

Example for Transferring Orders
# common options for connection to JS7 REST API
request_options=(--url=http://localhost:4446 --user=root --password=root --controller-id=controller)

# transfer orders to latest workflow version
./operate-workflow.sh transfer-order "${request_options[@]}" \
	--workflow=ap3jobs

Suspending and Resuming Workflows

Example for Suspending and Resuming Workflows
# common options for connection to JS7 REST API
request_options=(--url=http://localhost:4446 --user=root --password=root --controller-id=controller)

# suspend workflow
./operate-workflow.sh suspend-workflow "${request_options[@]}" \
	--workflow=ap3jobs

# resume workflow
./operate-workflow.sh resume-workflow "${request_options[@]}" \
	--workflow=ap3jobs

Stopping and Unstopping Jobs

Example for Stopping and Unstopping Jobs
# common options for connection to JS7 REST API
request_options=(--url=http://localhost:4446 --user=root --password=root --controller-id=controller)

# stop jobs from labels
./operate-workflow.sh stop-job "${request_options[@]}" \
	--workflow=ap3jobs --label=job1,job2

# unstop jobs from labels
./operate-workflow.sh unstop-job "${request_options[@]}" \
	--workflow=ap3jobs --label=job1,job2

Skipping and Unskipping Jobs

Example for Skipping and Unskipping Jobs
# common options for connection to JS7 REST API
request_options=(--url=http://localhost:4446 --user=root --password=root --controller-id=controller)

# skip jobs from labels
./operate-workflow.sh skip-job "${request_options[@]}" \
	--workflow=ap3jobs --label=job1,job2

# unskip jobs from labels
./operate-workflow.sh unskip-job "${request_options[@]}" \
	--workflow=ap3jobs --label=job1,job2

Posting, Reading and Deleting Notices

Example for Posting, Reading and Deleting Notices
# common options for connection to JS7 REST API
request_options=(--url=http://localhost:4446 --user=root --password=root --controller-id=controller)

# post notice for current daily plan date
./operate-workflow.sh post-notice "${request_options[@]}" \
	--notice-board=ap3jobs

# post notice for specific daily plan date and lifetime
./operate-workflow.sh post-notice "${request_options[@]}" \
	--notice-board=ap3jobs --notice-id=2024-08-26 --notice-lifetime=6h


# reading notices from notice board
./operate-workflow.sh get-notice "${request_options[@]}" \
    --notice-board=ap3jobs

# reading specific notice from notice board
./operate-workflow.sh get-notice "${request_options[@]}" \
    --notice-board=ap3jobs --notice-id=2024-08-26

# reading notices by folder
./operate-workflow.sh get-notice "${request_options[@]}" \
    --folder=/ap --recursive


# delete notices from notice board for current daily plan date
./operate-workflow.sh delete-notice "${request_options[@]}" \
	--notice-board=ap3jobs

# delete specific notices from notice board using notice identifiers
./operate-workflow.sh delete-notice "${request_options[@]}" \
	--notice-board=ap3jobs --notice-id=2024-08-25,2024-08-26

# delete notices by folder
./operate-workflow.sh delete-notice "${request_options[@]}" \
	--folder=/ap --recursive

Encrypting and Decrypting

Example for Encrypting and Decrypting
# create Private Key
openssl ecparam -name secp384r1 -genkey -noout -out encrypt.key

# create Certificate Signing Request
openssl req -new -sha512 -nodes -key encrypt.key -out encrypt.csr -subj "/C=DE/ST=Berlin/L=Berlin/O=SOS/OU=IT/CN=Encrypt"

# create Certificate
openssl x509 -req -sha512 -days 1825 -signkey encrypt.key -in encrypt.csr -out encrypt.crt -extfile <(printf "keyUsage=critical,keyEncipherment,keyAgreement\n")


# encrypt a secret such as a password using the Certificate, the encryption result will be returned and will look like: enc:BEXbHYa...
./deploy-controller.sh encrypt --in="root" --cert=encrypt.crt

# options for connection to the JS7 REST API can specify the encryption result as password and the Private Key for decryption
request_options=(--url=http://localhost:4446 --user=root --password="enc:BEXbHYa..." --key=encrypt.key --controller-id=controller)

# for example, when adding an ad hoc order the Private Key is used to decrypt the password for access to the REST API on-the-fly
./operate-workflow.sh add-order "${request_options[@]}" --workflow=ap3jobs

# decrypt an encrypted secret using the Private Key
./deploy-controller.sh decrypt --in="enc:BEXbHYa..." --key=encrypt.key

Resources



  • No labels