Introduction
Jobs might require variables for parameterization that hold secrets. We find a number of requirements for such variables:
- The secret should not be exposed to JS7 logging and to any instance of JS7 products that track variables.
- For example, if a variable created by some job should be forwarded to a next job executed with a different Agent then the Controller and JOC Cockpit keep track of the variable.
- The variable is available in the Controller's memory.
- The variable is available in the JOC Cockpit's JS7 - History.
- At no point in time the secret should be from clear text to any involved JS7 component, database or OS.
- For example, if a variable created by some job should be forwarded to a next job executed with a different Agent then the Controller and JOC Cockpit keep track of the variable.
- The secret should not be exposed to OS mechanisms that allow a 3rd-party to identify the secret
- For example, the following command to encrypt a secret can be tracked by any account capable of executing a
ps -aux
command:
echo "secret" | openssl enc -aes256 -salt -pass pass:"secret-key"
- For example, the following command to encrypt a secret can be tracked by any account capable of executing a
- We find a number of No-Go approaches that do not make it for a secure solution:
- Symmetric keys are a No-Go as they are available in two places and leave it up to the implementation where to store the key.
- Obfuscation is a No-Go as it does not resist to any serious attack.
The preferred solution with JS7 is to use asymmetric keys.
Asymmetric Keys
The basic proceeding works like this:
- Consider the parties involved and related use cases:
- A job executed on Agent A should be parameterized by a variable holding a secret.
- A job executed on Agent B retrieves a secret that should be forwarded to a job on Agent A and possibly to other Agents too.
- Use of asymmetric keys allows
- to create and to store a private key on Agent A.
- to use Agent A's public key on Agent B or any other system involved.
- to manage encryption like this:
- create a symmetric one-time key and an encrypted copy of the key derived from Agent A's public key.
- encrypt a variable's value with the one-time key.
- drop the one-time key and forward the encrypted copy of the one-time key and the variable holding the encrypted value to Agent A.
- only Agent A will be able to decrypt the encrypted one-time key which reveals the symmetric key required to decrypt the variable value.
Solution
Solution for Unix Jobs
Managing the private/public key pair
Step 1: Create a private/public key pair
The following step is performed on the server hosting the Agent that should decrypt variables:
# navigate to the Agent's <agent-data>/config/private directory cd /var/sos-berlin.com/js7/agent/config/private # create the agent.key private key file using "jobscheduler" as a passphrase openssl genrsa -aes256 -passout pass:"jobscheduler" -out agent.key 4096 # extract agent.pub public key file from private key openssl rsa -passin pass:"jobscheduler" -in agent.key -pubout > agent.pub
Step 2: Make public key available
Copy the public key to the server(s) hosting the Agent(s) that should encrypt variables:
# navigate to the Agent's <agent-data>/config directory cd /var/sos-berlin.com/js7/agent/config # copy the agent.pub public key file from the Agent in Step 1 to the target Agent using scp or a file transfer tool
Setting up the Workflow
The workflow example is straightforward from two jobs that create and that read encrypted variables:
Download sample workflow (upload .json): pduVariableCrypto.workflow.json
The fist job encrypt-variables looks like this:
#!/bin/bash set -e # encrypt variable values using the Agent's public key ##!include Crypto EncryptVariable myVar1 "secret1" EncryptVariable myVar2 "secret2"
Explanation:
- The job makes use of JS7 - Script Includes: the Crypto Script Include holds the Shell functions used in the job.
- The
##!include Crypto
inserts the shell code available from the indicated Crypto Script Include. - The Script Include is invoked once per job and can be parameterized to specify the location of the public key.
## include Crypto --replace="<public-key>","/var/sos-berlin.com/js7/agent/config/agent.pub"
- The above value represents the default value that will be used if the Script Include is invoked without replacement options.
- The
- The
EncryptVariable
shell function expects the name of the variable and the value that should be encrypted.EncryptVariable
<name> <value> [<key-name> [,<public-key>]]
<name>
: The name of the variable is required.<value>
: The value of the variable is required.<key-name>
: The name of a second variable holding the encrypted symmetric key. Defaults to<name>_key
.<public-key>
: The path to the public key file is specified. Defaults to<agent-data>/config/agent.pub
.
- The shell function will encrypt the variable with the public key.
- The encrypted variable will be forwarded to subsequent jobs and instructions in the workflow.
The second job decrypt-variables looks like this:
set -e # decrypt variable values using the Agent's private key ##!include Crypto secret1=$(DecryptVariable "${MY_VAR1}" "${MY_VAR1_KEY}") echo "${secret1}" secret2=$(DecryptVariable "${MY_VAR2}" "${MY_VAR2_KEY}") echo "${secret2}"
Explanation:
- The job makes use of JS7 - Script Includes: the Crypto Script Include holds the Shell functions used in the job.
- The
##!include Crypto
inserts the shell code available from the indicated Crypto Script Include. - The Script Include can be parameterized to specify the location of the private key.
## include Crypto --replace="<private-key>","/var/sos-berlin.com/js7/agent/config/private/agent.key"
- The above value represents the default value that will be used of the Script Include is invoked without replacements.
- The Script Include can be parameterized to specify a passphrase used by the private key.
## include Crypto --replace="<passphrase>","jobscheduler"
- The Script Include can be invoked with any number of
--replace=<what>,<with>
options.
- The
- The
DecryptVariable
function expects the encrypted value of the variable and the encrypted value of the symmetric key.DecryptVariable
<value> <key-value> [<private-key> [,<passphrase>]]
<value>
: The encrypted value of the variable is required.<key-value>
: The value of the variable holding the encrypted symmetric key is required.<private-key>
: The path to the private key file is specified. Defaults to<agent-data>/config/private/agent.key
.<passphrase>
: The passphrase of the private key.
- The function will decrypt the encrypted symmetric key.
- The function will decrypt the encrypted variable value using the decrypted symmetric key.
- The
DecryptVariable
function returns the secret that can be assigned an environment variable. - It is recommended not to write the secret to a file or to perform any operation that will expose the secret to logging of output in the stdout and stderr channels.