Introduction

JS7 - Encryption and Decryption are integrated with jobs in a number of ways. Basic requirements include that

Jobs can be subject to a number of scenarios:
- Jobs might want to read secrets from specific sources such as arguments, configuration files etc.
- Jobs might want to create secrets on-the-fly that should be processed by successor jobs.
Jobs should consider not to expose secrets to logging, database persistence etc.
- Jobs can use the scripts and cmdlets explained with the JS7 - How to encrypt and decrypt article section to encrypt and to decrypt values.
- Jobs can use JS7 - Job Resources to store encrypted values, see JS7 - How to update a Job Resource using Unix Shell

For creation of Encryption Keys see JS7 - How to create X.509 Encryption Keys.

Integration

Use with Workflow Variables

Encrypting Secrets to Workflow Variables

Explanation:

The job encrypts a secret using the target Agent's certificate.
A new variable is added to the workflow that holds the encrypted secret.

Examples:

For details see JS7 - How to encrypt and decrypt using Unix Shell

Example for Encryption using Unix Shell

# encrypt secret
result=$($JS7_AGENT_HOME/bin/js7_encrypt.sh --cert=$JS7_AGENT_CONFIG_DIR/foobar.crt --in="12345678")

# forward "new_var" workflow variable holding the encrypted result
echo "new_var=$result" >> $JS7_RETURN_VALUES

For details see JS7 - How to encrypt and decrypt using Windows Shell

Example for Encryption using Windows Shell

@rem encrypt secret and return result with JS7_ENCRYPT_VALUE environment variable
call "%JS7_AGENT_HOME%\bin\js7_encrypt.cmd" "--cert=%JS7_AGENT_CONFIG_DIR%\foobar.crt" "--in=12345678"

@rem forward "new_var" workflow variable holding the encrypted result
echo new_var=%JS7_ENCRYPT_VALUE% >> %JS7_RETURN_VALUES%

For details see JS7 - How to encrypt and decrypt using PowerShell

Example for Encryption using PowerShell

# encrypt secret and return result with JS7_ENCRYPT_VALUE environment variable
$result = Invoke-JS7Encrypt -CertificatePath $env:JS7_AGENT_CONFIG_DIR/foobar.crt -Value '12345678' -JavaLib $env:JS7_AGENT_HOME/lib

# forward "new_var" workflow variable holding the encrypted result
"new_var=$result" | Out-File $env:JS7_RETURN_VALUES -Append

Decrypting Secrets from Workflow Variables

Explanation:

Workflow variables are provided from environment variables for shell jobs, see JS7 - Job Instruction.
The job decrypts a secret using the current Agent's Private Key.

Examples:

Example for Decryption using Unix Shell

# encrypted result is assumed being available from NEW_VAR environment variable
secret=$($JS7_AGENT_HOME/bin/js7_decrypt.sh \
    --key=$JS7_AGENT_CONFIG_DIR/private/foobar.key \
    --in="$NEW_VAR")
echo $secret

Example for Decryption using Windows Shell

@rem encrypted result is assumed being available from NEW_VAR environment variable

call "%JS7_AGENT_HOME%\bin\js7_decrypt.cmd" ^
    "--key=%JS7_AGENT_CONFIG_DIR%\private\foobar.key" ^
    "--in=%NEW_VAR%"
@echo %JS7_DECRYPT_VALUE%

Example for Decryption using PowerShell

# encrypted result is assumed being available from NEW_VAR environment variable
$secret = Invoke-JS7Decrypt -Value $env:NEW_VAR -KeyPath $env:JS7_AGENT_CONFIG_DIR/private/foobar.key -JavaLib $env:JS7_AGENT_HOME/lib
Write-Output $secret

Use with Job Resources

Encrypting Secrets to Job Resources

Explanation:

The job encrypts a secret using the target Agent's certificate and stores the encrypted result to a Job Resource variable.

Examples:

For details see JS7 - How to update a Job Resource using Unix Shell

Example for Encryption using Unix Shell

$JS7_AGENT_HOME/bin/js7_set_job_resource.sh \
    --url=http://joc-2-0-primary:7446 \
    --controller-id=controller \
    --user=root \
    --password=root \
    --job-resource=/ProductDemo/Variables/pdBusinessSecret \
    --key=businessSecret \
    --value='12345678' \
    --env-var=BUSINESS_SECRET \
    --encrypt-cert=$JS7_AGENT_CONFIG_DIR/foobar.crt

For details see JS7 - How to update a Job Resource using PowerShell

Example for Encryption using PowerShell

Set-JS7JobResource `
    -Path /ProductDemo/Variables/pdBusinessSecret `
    -Key 'businessSecret' `
    -Value '12345678' `
    -EnvVar 'BUSINESS_SECRET' `
    -EncryptCertificatePath $env:JS7_AGENT_CONFIG_DIR/foobar.crt `
    -JavaLib $env:JS7_AGENT_HOME/lib

Decrypting Secrets from Job Resources

Variables from Job Resources are available from environment variables similar to workflow variables.

The Job Resource is assigned the workflow to make environment variables available to all jobs or is assigned individual jobs. The environment variables specified with the Job Resource are automatically available for shell jobs.

Decryption of secrets is the same as for Decrypting Secrets from Workflow Variables.

Encrypting Configuration Files to Job Resources

Explanation:

An external application encrypts a configuration file using the target Agent's certificate. The encrypted configuration file is added to a Job Resource.
When the Job Resource is assigned a workflow or job then JS7 takes care to transfer the Job Resource to all Agents operating related jobs.

Examples:

For details see JS7 - How to update a Job Resource using Unix Shell

Example for Encryption using Unix Shell

./js7_set_job_resource.sh \
    --url=http://joc-2-0-primary:7446 \
    --controller-id=controller \
    --user=root \
    --password=root \
    --job-resource=/ProductDemo/Variables/pdConfigurationData \
    --key=configurationData \
    --file=application.conf \
    --env-var=CONFIGURATION_DATA \
    --encrypt-cert=foobar.crt

For details see JS7 - How to update a Job Resource using PowerShell

Example for Encryption using PowerShell

Set-JS7JobResource `
    -Path /ProductDemo/Variables/pdConfigurationData `
    -Key 'configurationData' `
    -File application.conf `
    -EnvVar 'CONFIGURATION_DATA' `
    -EncryptCertificatePath foobar.crt `
    -JavaLib /js7/js7.encryption/lib

Decrypting Configuration Files from Job Resources

Explanation;:

JS7 takes care to transfer the Job Resource to all Agents operating workflows or jobs which are assigned the Job Resource. The encrypted configuration file included with the Job Resource variable is stored to a temporary file per instance of a job (task, process) and is automatically removed on termination of the job instance.
Environment variables from the Job Resource denote the location of the temporary file and the encryption result:
- <env-var>: location of temporary file
- <env-var>_KEY: encryption result holding the encrypted symmetric key and intialization vector
The job makes use of the js7_decrypt.sh | .cmd scripts to decrypt the encrypted configuration file by use of its Private Key. Similarly the Invoke-JS7Decrypt PowerShell cmdlet can be used.

Examples:

Example for Decryption using Unix Shell

# previous encryption is assumed to create the CONFIGURATION_DATA and CONFIGURATION_DATA_KEY environment variables in the Job Resource:
# ./js7_set_job_resource.sh --job-resource=/ProductDemo/Variables/pdConfigurationData --key=configurationData --file=application.conf --env-var=CONFIGURATION_DATA ...

# CONFIGURATION_DATA environment variable specifies the path to a temporary file provided by JS7 that holds the encrypted configuration data
echo "$CONFIGURATION_DATA"
# CONFIGURATION_DATA_KEY environment variable holds the encrypted symmetric key and initialization vector
echo "$CONFIGURATION_DATA_KEY"

# decrypt configuration file and store to a temporary file
$JS7_AGENT_HOME/bin/js7_decrypt.sh \
    --key=$JS7_AGENT_CONFIG_DIR/private/foobar.key \
    --in="$CONFIGURATION_DATA_KEY" \
    --infile="$CONFIGURATION_DATA" \
    --outfile=application-$$.conf

# on job termination JS7 automatically performs cleanup of the temporary file denoted by CONFIGURATION_DATA
# user's job implementation has to clean up temporary files created by the job
rm -f application-$$.conf

Example for Decryption using Windows Shell

@rem previous encryption is assumed to create the CONFIGURATION_DATA and CONFIGURATION_DATA_KEY environment variables in the Job Resource:

@rem CONFIGURATION_DATA environment variable specifies the path to a temporary file provided by JS7 that holds the encrypted configuration data
@echo %CONFIGURATION_DATA%
@rem CONFIGURATION_DATA_KEY environment variable holds the encrypted symmetric key and initialization vector
@echo %CONFIGURATION_DATA_KEY%

set "TempFile=%TEMP%\application-%DATE%-%TIME::=.%.%RANDOM%.conf"
call "%JS7_AGENT_HOME%\bin\js7_decrypt.cmd" ^
    "--key=%JS7_AGENT_CONFIG_DIR%\private\foobar.key" ^
    "--in=%CONFIGURATION_DATA_KEY%" ^
    "--infile=%CONFIGURATION_DATA%" ^
    "--outfile=%TempFile%"

# on job termination JS7 automatically performs cleanup of the temporary file denoted by CONFIGURATION_DATA
# user's job implementation has to clean up temporary files created by the job
del /F %TempFile%

Example for Decryption using PowerShell

# previous encryption is assumed to create the CONFIGURATION_DATA and CONFIGURATION_DATA_KEY environment variables in the Job Resource:
# Set-JS7JobResource -Path /ProductDemo/Variables/pdConfigurationData -Key 'configurationData' -File application.conf -EnvVar 'CONFIGURATION_DATA' ...

# CONFIGURATION_DATA environment variable specifies the path to a temporary file provided by JS7 that holds the encrypted configuration data
Write-Output $env:CONFIGURATION_DATA
# CONFIGURATION_DATA_KEY environment variable holds the encrypted symmetric key and initialization vector
Write-Output $env:CONFIGURATION_DATA_KEY

# decrypt configuration file
$tempFile = New-TemporaryFile
Invoke-JS7Decrypt `
    -Value $env:CONFIGURATION_DATA_KEY `
    -File $env:CONFIGURATION_DATA `
    -OutFile $tempFile `
    -KeyPath $env:JS7_AGENT_CONFIG_DIR/private/foobar.key `
    -JavaLib $env:JS7_AGENT_HOME/lib

# on job termination JS7 automatically performs cleanup of the temporary file denoted by CONFIGURATION_DATA
# user's job implementation has to clean up temporary files created by the job
Remove-Item -Path $tempFile -Force

Use with Configuration Items from Files

Encrypting Secrets to Configuration Files

Explanation;:

A secret should be stored to a configuration file, for example a password for access to a database,
An external application creates/updates the secret, for example triggered by password rotation.
The application makes use of the js7_encrypt.sh | .cmd scripts to encrypt the secret and stores the encrypted secret to the configuration file. Similarly the Invoke-JS7Encrypt PowerShell cmdlet can be used.
Find details from JS7 - How to encrypt and decrypt Database Credentials.

Examples:

Example for Encryption using Unix Shell

# encrypt secret and return result
result=$(./js7_encrypt.sh --cert=foobar.crt --in="12345678")

# update <password> placeholder in application.conf file
sed -i'' -e "s@<password>@${result}@g" application.conf

Example for Encryption using Windows Shell

@rem encrypt secret and return result with JS7_ENCRYPT_VALUE environment variable
call .\js7_encrypt.cmd "--cert=foobar.crt" "--in=12345678"

@rem update <password> placeholder in application.conf file
powershell.exe -Command "((Get-Content application.conf) -replace '<password>', $env:JS7_ENCRYPT_VALUE) | Set-Content -Path application.conf"

Example for Encryption using PowerShell

# encrypt secret and return result
$result = Invoke-JS7Encrypt -CertificatePath foobar.crt -Value '12345678' -JavaLib /js7/js7.encryption/lib

# update <password> placeholder in application.conf file
((Get-Content application.conf) -replace '<password>', $result) | Set-Content -Path application.conf

Decrypting Secrets from Configuration Files

Explanation;:

A job reads the encrypted secret from the configuration file.
The job makes use of the js7_decrypt.sh | .cmd scripts to decrypt the secret by use of its Private Key. Similarly the Invoke-JS7Decrypt PowerShell cmdlet can be used.

Examples:

Example for Decryption using Unix Shell

# read secret from application.conf file that is assumed to look like this:
#     username=foo
#     password=gE4NPEbrrUFFdjbnvtX9ZWF9owz1ghwbjl2WLAP19k6ps1tXX+fug== KeU1ZYgNfF1kNfYQaIp/Wg== osXNvG2evsSnf3WUE8I8TQ==

result=$(grep -E 'password=.*' application.conf | cut -d '=' -f 2-)

secret=$($JS7_AGENT_HOME/bin/js7_decrypt.sh \
    --key=$JS7_AGENT_CONFIG_DIR/private/foobar.key \
    --in="$result")
echo $secret

Example for Decryption using Windows Shell

# read secret from application.conf file that is assumed to look like this:
#     username=foo
#     password=gE4NPEbrrUFFdjbnvtX9ZWF9owz1ghwbjl2WLAP19k6ps1tXX+fug== KeU1ZYgNfF1kNfYQaIp/Wg== osXNvG2evsSnf3WUE8I8TQ==

@set "TempFile=%TEMP%\application-%DATE%-%RANDOM%.tmp"
@findstr /c:"password=" application.conf >> %TempFile%
for /f "tokens=1,* delims='='" %%i in (%TempFile%) do set RESULT=%%j
@del /F %TempFile%

call "%JS7_AGENT_HOME%\bin\js7_decrypt.cmd" ^
    "--key=%JS7_AGENT_CONFIG_DIR%\private\foobar.key" ^
    "--in=%RESULT%"
@echo %JS7_DECRYPT_VALUE%

Example for Decryption using PowerShell

# read secret from application.conf file that is assumed to look like this:
#     username=foo
#     password=gE4NPEbrrUFFdjbnvtX9ZWF9owz1ghwbjl2WLAP19k6ps1tXX+fug== KeU1ZYgNfF1kNfYQaIp/Wg== osXNvG2evsSnf3WUE8I8TQ==

$matches = ( Get-Content application.conf | Select-String "password=(.*)" ).Matches
$result = if ( $matches.Groups.count -gt 1 ) { $matches.Groups[1].Value }

$secret = Invoke-JS7Decrypt -Value $result -KeyPath $env:JS7_AGENT_CONFIG_DIR/private/foobar.key -JavaLib $env:JS7_AGENT_HOME/lib
Write-Output $secret

Use with Configuration Files

Encrypting Configuration Files

Explanation;:

Instead of encrypting individual items in a configuration file, the file will be encrypted.
An external application updates and encryptes the configuration file in case of changes.
The application makes use of the js7_encrypt.sh | .cmd scripts to encrypt the configuration file. Similarly the Invoke-JS7Encrypt PowerShell cmdlet can be used.

Examples:

Example for Encryption using Unix Shell

# encrypt configuration file and return result
result=$(./js7_encrypt.sh --cert=foobar.crt --infile=application.conf --outfile=application.conf.enc)
echo "$result"

Example for Encryption using Windows Shell

@rem encrypt configuration file and return result with JS7_ENCRYPT_VALUE environment variable
call .\js7_encrypt.cmd "--cert=foobar.crt" "--infile=application.conf" "--outfile=application.conf.enc"
echo %JS7_ENCRYPT_VALUE%

Example for Encryption using PowerShell

# encrypt configurationn file and return result
$result = Invoke-JS7Encrypt -CertificatePath foobar.crt -File application.conf -OutFile application.conf.enc -JavaLib /js7/js7.encryption/lib

# update <password> placeholder in application.conf file
((Get-Content application.conf) -replace '<password>', $result) | Set-Content -Path application.conf

Decrypting Configuration Files

Explanation;:

The job makes use of the js7_decrypt.sh | .cmd scripts to decrypt the encrypted configuration file by use of its Private Key. Similarly the Invoke-JS7Decrypt PowerShell cmdlet can be used.
On termination the job will drop the decrypted configuration file.

Examples:

Example for Decryption using Unix Shell

# previous encryption assumed to create the output file like this:
# result=$(./js7_encrypt.sh --cert=foobar.crt --infile=application.conf --outfile=application.conf.enc)

# decrypt configuration file
$JS7_AGENT_HOME/bin/js7_decrypt.sh \
    --key=$JS7_AGENT_CONFIG_DIR/private/foobar.key \
    --in="$result" \
    --infile=application.conf.enc \
    --outfile=application.conf

Example for Decryption using Windows Shell

@rem previous encryption assumed to create the output file like this:
@rem call .\js7_encrypt.cmd "--cert=foobar.crt" "--infile=application.conf" "--outfile=application.conf.enc"
@echo %JS7_DECRYPT_VALUE%

call "%JS7_AGENT_HOME%\bin\js7_decrypt.cmd" ^
    "--key=%JS7_AGENT_CONFIG_DIR%\private\foobar.key" ^
    "--in=%JS7_DECRYPT_VALUE%" ^
    "--infile=application.conf.enc" ^
    "--outfile=application.conf"

Example for Decryption using PowerShell

# prevous encryption assumed to create the output file like this:
# $result = Invoke-JS7Encrypt -CertificatePath foobar.crt -File application.conf -OutFile application.conf.enc -JavaLib /js7/js7.encryption/lib

# decrypt configuration file
Invoke-JS7Decrypt -Value $result -File application.conf.enc -OutFile application.conf -KeyPath $env:JS7_AGENT_CONFIG_DIR/private/foobar.key -JavaLib $env:JS7_AGENT_HOME/lib

Key Distribution

Keys can be distributed in a number of ways. Frequently used scenarios include that

Users create individual Private Keys and Certificates for encryption/decryption of secrets per Agent.
Users create Private Keys and Certificates that are shared amongst a number of Agents.
- This applies to use of an Agent Cluster that allows to execute jobs on any Agent in the cluster.

It is possible that jobs access an Agent's Private Key and SSL Certificate that are used to secure HTTPS connections, see JS7 - Agent HTTPS Connections. This requires the Agent's SSL certificate to be created with the dataEncipherment key usage option. Many users consider it more secure to use separate keys for HTTPS connections and for encryption/decryption of secrets.

Further Resources

Space shortcuts

Page tree

Introduction

Integration

Use with Workflow Variables

Encrypting Secrets to Workflow Variables

Decrypting Secrets from Workflow Variables

Use with Job Resources

Encrypting Secrets to Job Resources

Decrypting Secrets from Job Resources

Encrypting Configuration Files to Job Resources

Decrypting Configuration Files from Job Resources

Use with Configuration Items from Files

Encrypting Secrets to Configuration Files

Decrypting Secrets from Configuration Files

Use with Configuration Files

Encrypting Configuration Files

Decrypting Configuration Files

Key Distribution

Further Resources

Space shortcuts

Page tree

JS7 - Encryption - Integration with Shell CLI

Introduction

Integration

Use with Workflow Variables

Encrypting Secrets to Workflow Variables

Decrypting Secrets from Workflow Variables

Use with Job Resources

Encrypting Secrets to Job Resources

Decrypting Secrets from Job Resources

Encrypting Configuration Files to Job Resources

Decrypting Configuration Files from Job Resources

Use with Configuration Items from Files

Encrypting Secrets to Configuration Files

Decrypting Secrets from Configuration Files

Use with Configuration Files

Encrypting Configuration Files

Decrypting Configuration Files

Key Distribution

Further Resources