Page History
Table of Contents |
---|
Introduction
Jobs might require variables for parameterization that hold secrets. We find a number of requirements for management of such variables:
- The secret should not be exposed to JS7 logging and or to any instance of JS7 products that track variables.
- For example, if a variable created by some job should be forwarded to a next job executed with a different Agent then the Controller and JOC Cockpit keep track of the variable.
- The variable is available in the Agent's and the Controller's memory.
- The variable is available in the Controller's journal, in JOC Cockpit's s JS7 - History and in the JS7 - Database.
- At no point in time the secret should be from clear text available to any involved JS7 componentproducts, to the database or OS.
- For example, the following command to encrypt a secret can be tracked by any account capable of executing a
ps -aux
command:
echo "secret" | openssl enc -aes256 -salt -pass pass:"secret-key"
to the OS.
- For example, if a variable created by some job should be forwarded to a next job executed with a different Agent then the Controller and JOC Cockpit keep track of the variable.
- We find a number of invalid inadequate approaches that do not make it for a secure solution:
- Symmetric keys are a No-Go as they are available in two places and leave it up to the implementation where to store the key.
- Obfuscation is a No-Go as it does not resist to any serious attackattacks.
The preferred solution with JS7 is to use of asymmetric keys. :
...
- Decryption can be performed directly by
...
Asymmetric Keys
The basic proceeding works like this:
- Consider the parties involved and related use cases:
- A job executed on Agent A should be parameterized by a variable holding a secret.
- A job executed on Agent B retrieves a secret that should be forwarded to a job on Agent A and possibly to other Agents too.
- Use of asymmetric keys allows
- to create and to store a private key on Agent A.
- to use Agent A's public key on Agent B or any other system involved.
- to manage encryption and decryption like this:
- create a symmetric one-time key and an encrypted copy of the key derived from Agent A's public key.
- encrypt a variable's value with the one-time key.
- drop the one-time key and forward the encrypted copy of the one-time key and the variable holding the encrypted value to Agent A.
- only Agent A will be able to decrypt the encrypted one-time key using its private key which reveals the symmetric key required to decrypt the variable value.
Solution Outline
The solution is provided for download and can be used to automate encryption and decryption of variables.
- The solution is available for Linux and MacOS® using bash shell.
- The solution is intended as a baseline example for customization by JS7 users and by SOS within the scope of professional services.
Solution for Unix Jobs
Managing the private/public key pair
Step 1: Create a private/public key pair
...
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
# navigate to the Agent's <agent-data>/config/private directory
cd /var/sos-berlin.com/js7/agent/config/private
# create the agent.key private key file using "jobscheduler" as a passphrase
openssl genrsa -aes256 -passout pass:"jobscheduler" -out agent.key 4096
# extract the agent.pub public key file from agent.key private key file
openssl rsa -passin pass:"jobscheduler" -in agent.key -pubout > agent.pub |
Step 2: Make public key available
Copy the public key to the server(s) hosting the Agent(s) that should encrypt variables:
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
# navigate to the Agent's <agent-data>/config directory
cd /var/sos-berlin.com/js7/agent/config
# copy the agent.pub public key file from the Agent in Step 1 to target Agents using scp or a file transfer tool |
Setting up the Workflow
The workflow example introduces two jobs that encrypt and decrypt variables.
...
First Job: encrypt-variables
The fist job encrypt-variables looks like this:
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
#!/bin/bash
set -e
# encrypt variable values using the Agent's public key
##!include Crypto
EncryptVariable myVar1 "secret1"
EncryptVariable myVar2 "secret2" |
Explanation:
- The job makes use of JS7 - Script Includes: the Crypto Script Include holds the shell functions used in the job.
- The
##!include Crypto
inserts the shell code available from the indicated Crypto Script Include. - The Script Include is invoked once per job and can be parameterized to specify the location of the public key.
##!include Crypto --replace="<public-key>","/var/sos-berlin.com/js7/agent/config/agent.pub"
- The above value represents the default value that will be used if the Script Include is invoked without replacement options.
- The
- The
EncryptVariable
shell function expects the name of the variable and the value that should be encrypted.EncryptVariable
<name> <value> [<key-name> [,<public-key>]]
<name>
: The name of the variable is required.<value>
: The value of the variable is required.<key-name>
: The name of a second variable holding the encrypted symmetric key. Defaults to<name>_key
.<public-key>
: The path to the public key file is specified. Defaults to<agent-data>/config/agent.pub
.
- The shell function will encrypt the variable using the public key.
- The encrypted variable will be forwarded to subsequent jobs and instructions in the workflow.
Second Job: decrypt-variables
The second job decrypt-variables looks like this:
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
#!/bin/bash
set -e
# decrypt variable values using the Agent's private key
##!include Crypto
secret1=$(DecryptVariable "${MY_VAR1}" "${MY_VAR1_KEY}")
echo "${secret1}"
secret2=$(DecryptVariable "${MY_VAR2}" "${MY_VAR2_KEY}")
echo "${secret2}" |
Explanation:
- The job makes use of JS7 - Script Includes: the Crypto Script Include holds the shell functions used in the job.
- The
##!include Crypto
inserts the shell code available from the indicated Crypto Script Include. - The Script Include can be parameterized to specify the location of the private key.
##!include Crypto --replace="<private-key>","/var/sos-berlin.com/js7/agent/config/private/agent.key"
- The above value represents the default value that will be used of the Script Include is invoked without replacements.
- The Script Include can be parameterized to specify a passphrase required by the private key.
##!include Crypto --replace="<passphrase>","jobscheduler"
- The Script Include can be invoked with any number of
--replace=<what>,<with>
options.
- The
- The
DecryptVariable
function expects the encrypted value of the variable and the encrypted value of the symmetric key.DecryptVariable
<value> <key-value> [<private-key> [,<passphrase>]]
<value>
: The encrypted value of the variable is required.<key-value>
: The value of the variable holding the encrypted symmetric key is required.<private-key>
: The path to the private key file is specified. Defaults to<agent-data>/config/private/agent.key
.<passphrase>
: The passphrase of the private key is specified.
- The function will decrypt the encrypted symmetric key and will decrypt the encrypted variable value using the decrypted symmetric key.
- The
DecryptVariable
function returns the secret that can be assigned an environment variable. - It is recommended not to write the secret to a file or to perform any operation that will expose the secret to logging of output in the stdout and stderr channels.
...
- scripts that are used outside of JS7 products or by related jobs.
- No JS7 product is directly involved in encryption/decryption as otherwise the JS7 product would know the keys involved that potentially could be compromised by logging, database persistence etc.
- Performing encryption/decryption by jobs limits the attack surface to the OS process executing the job. The job implementation is controlled by the user who can verify secure operation.
For creation of Encryption Keys see JS7 - How to create X.509 Encryption Keys.
Asymmetric Keys
Encryption and decryption use asymmetric keys, for details see JS7 - Encryption and Decryption:
Encryption
Use with Variables
The value of a variable can be encrypted using shell scripts and cmdlets:
- JS7 - How to encrypt and decrypt using Unix Shell
- JS7 - How to encrypt and decrypt using Windows Shell
- JS7 - How to encrypt and decrypt using PowerShell
When executed from jobs then the encrypted secret can be stored to a variable that is made available to later jobs in the workflow.
Graphviz | ||
---|---|---|
| ||
digraph structs {
compound=true;
rankdir=LR;
Secret [label=" Secret ",style="filled",fillcolor="limegreen",fontname="Arial",fontsize="12pt"]
Encrypted_Secret [label=" Variable holding Encrypted Secret ",style="filled",fillcolor="dodgerblue",fontname="Arial",fontsize="12pt"]
Certificate [shape="ellipse",label="Certificate / Public Key",style="filled",fillcolor="orange",fontname="Arial",fontsize="12pt"]
Encrypt [shape="rectangle",label="Encrypt\njs7_encrypt.sh | .cmd\nInvoke-JS7Encrypt",fontname="Arial",fontsize="10pt",style="filled",fillcolor="white"]
subgraph encrypt {
fontname="Arial";
fontsize="12pt";
Secret -> Encrypt;
Certificate -> Encrypt;
Encrypt -> Encrypted_Secret [label="encrypt",fontname="Arial",fontsize="10pt"];
}
} |
Use with Job Resources
The value of a variable is encrypted and is stored to JS7 - Job Resources using shell scripts or cmdlets:
- JS7 - How to update a Job Resource using Unix Shell
- JS7 - How to update a Job Resource using PowerShell
The Job Resource can be accessed by any jobs in the same or in other workflows.
Graphviz | ||
---|---|---|
| ||
digraph structs {
compound=true;
rankdir=LR;
Secret [label=" Secret ",style="filled",fillcolor="limegreen",fontname="Arial",fontsize="12pt"]
Encrypted_Secret [label=" Job Resource Variable holding Encrypted Secret ",style="filled",fillcolor="dodgerblue",fontname="Arial",fontsize="12pt"]
Certificate [shape="ellipse",label="Certificate / Public Key",style="filled",fillcolor="orange",fontname="Arial",fontsize="12pt"]
Encrypt [shape="rectangle",label="Encrypt\njs7_set_job_resource.sh\nSet-JS7-JobResource",fontname="Arial",fontsize="10pt",style="filled",fillcolor="white"]
subgraph encrypt {
fontname="Arial";
fontsize="12pt";
Secret -> Encrypt;
Certificate -> Encrypt;
Encrypt -> Encrypted_Secret [label="encrypt",fontname="Arial",fontsize="10pt"];
}
} |
Decryption
Encrypted secrets are made available from variables. This similarly works for variables that are created by predecessor jobs on the fly and by Job Resources. For shell jobs workflow variables are made available from OS environment variables.
Graphviz | ||
---|---|---|
| ||
digraph structs {
compound=true;
rankdir=LR;
Encrypted_Secret [label=" Variable holding Encrypted Secret ",style="filled",fillcolor="dodgerblue",fontname="Arial",fontsize="12pt"]
Secret [label=" Secret ",style="filled",fillcolor="limegreen",fontname="Arial",fontsize="12pt"]
PrivateKey [shape="ellipse",label="Private Key",style="filled",fillcolor="orange",fontname="Arial",fontsize="12pt"]
Decrypt [shape="rectangle",label="Decrypt\njs7_decrypt.sh | .cmd\nInvoke-JS7Decrypt",fontname="Arial",fontsize="10pt",style="filled",fillcolor="white"]
subgraph decrypt {
fontname="Arial";
fontsize="12pt";
Encrypted_Secret -> Decrypt;
PrivateKey -> Decrypt;
Decrypt -> Secret [label="decrypt",fontname="Arial",fontsize="10pt"];
}
} |
The basic proceeding works like this:
- Consider the parties involved and related use cases:
- A job executed on Agent A should be parameterized by a variable holding a secret.
- A job executed on Agent B retrieves a secret that should be forwarded to the job on Agent A and possibly to other Agents too.
- Use of asymmetric keys allows
- to create and to store a Private Key on Agent A.
- to use Agent A's Certificate or Public Key on Agent B or on any other system involved.
- to manage encryption and decryption like this:
- create a symmetric one-time key and an encrypted copy of the key from Agent A's Certificate/Public Key.
- encrypt the value of a variable with the one-time key.
- drop the one-time key and forward the encrypted copy of the one-time key and the variable holding the encrypted value to Agent A.
- only Agent A will be able to decrypt the encrypted one-time key using its Private Key which provides the symmetric key required to decrypt the variable's value.
- Find details from JS7 - Encryption and Decryption.
Resources
Display children header |
---|
...
Explanation:
...