Page History
...
Introduction
Sensitive information that is used in job scripts can be stored in a Credential Store and retrieved at run-time.
This feature is similar to the approach used by the YADE file transfer job (and command line utility) to store credentials with the YADE Credential Store.
- Starting Point
- Users frequently operate jobs that require credentials, e.g. to access a database, a file transfer SFTP server etc.
- Such jobs are implemented as simple shell jobs.
- Security Considerations
- Sensitive information in jobs should not be hard-coded, should not be used from parameters and should not be disclosed, e.g. written to log files. Therefore credentials cannot be forwarded by parameters to jobs.
- Instead, a run-time interface is offered that allows to retrieve sensitive information from a credential store. References to credential store entries can safely be stored with parameter values.
- Credential Store
- A credential store allows the secure storage and retrieval of credentials for authentication, as well as connection and other parameters, for a detailed features and supported products see YADE Credential Store.
- Solution Outline
- Access to the credential store is provided by a Java class that can be loaded from shell jobs and from JS7 - Job Templates.
- The Java class is parameterized with the path that identifies the requested entry from the credential store.
Usage
TheJS7 Agents provide the com.sos.keepass.SOSKeePassDatabase
Java class that can be invoked
- in a Shell Job shell job by calling the
java
command line utility with the class name:- If the class is executed successfully:
- return code = 0, output is sent to stdoutto stdout
- If execution of the class ends in error:
- return code = 99, exception output is sent to stderr
- in a JavaScript Job
- by directly instantiating the Java class from JavaScript.
- if execution of the class ends in error then an exception is raised.
- in Powershell Jobs (for use with Agents only) by calling the java command line utility with the class name:
- A return code is provided similar to Shell jobs.
- to stderr
from the command line like this:
java com.sos
from the command line like this:
- java com.sos
.keepass.SOSKeePassDatabase
"cs://server/SFTP/homer.sos@password?file=credential_store.kdbx"
When invoking the class then the path to the entry in the credential store is has to be specified.
- If the class is executed successfully:
- by use of a shell script provided with JS7 Agents
- JobScheduler e.g. for Unix environments:
$SCHEDULER$JS7_AGENT_HOME/bin/jobscheduleragent_credential_value.sh "cs://server/SFTP/homer.sos@password?file=credential_store.kdbx"
- for Windows environments:
%JS7_AGENT_HOME%\bin\agent_credential_value.cmd "cs://server/SFTP/homer.sos@password?file=credential_store.kdbx"
- The script hides the call to the
java
command line utility.
- JobScheduler e.g. for Unix environments:
Syntax
- A call to the
SOSKeePassDatabase
class syntactically uses a single parameter string that holds
- the URI and a number of query parameters:
URI
cs://<entry_path>@<property_name>
- required- The URI based syntax includes the protocol
cs://
- followed by the
<entry_path>
that specifies the directory structure and entry name in the credentials store file. - followed by the @ character
- followed by the
<property_name>
that should be retrieved:- frequently used properties include credential store field names such as
title, user, password
. Custom field names are supported.- starting with JITL-585 the
attachment
property is supported
- starting with JITL-585 the
- for detailed explanations of available properties see the Using Credential Store to securely store authentication, connection and other parameters article.
- frequently used properties include credential store field names such as
- The URI based syntax includes the protocol
Query Parameters
Standard Parameters
file
- required- the path to the credential store database file. This file can be stored anywhere in the file system.
- The path can be specified either relatively or absolutely. As a path separator both forward slashes and backslashes can be used. For example:
cs://databases/mysql_localhost@password?file=config/credential_store.kdbx
cs://databases/mysql_localhost@password?file=C:/jobscheduler/dataProgramData/sos-berlin.com/js7/agent/config/credential_store.kdbx
- Relative values with a Master paths are considered with respect to the Master's SCHEDULER_DATA directory.Relative values with an Agent are with respect to the SCHEDULER_HOME (install) directory.Agent's working directory, see
JS7_AGENT_WORK_DIR
from the JS7 - Job Environment Variables
password
- optional- the password for the credential store database file.
- It is recommended not to use this parameter and instead to use a
key_file
to access the credential store.
key_file
- optional, default: <credential_store_database_filename_without_extension>.key- a key file for the credential store database file.
- If this parameter is set:
- this path can be specified either relatively or absolutely. See the explanations explanation for the
file
parameter. - An exception will be thrown raised if the .key file is not cannot be found.
- this path can be specified either relatively or absolutely. See the explanations explanation for the
- If this parameter is not set:
- a <credential_store_database_filename_without_extension>.key file such as credential_store.kdbx -> credential_store.key will be sought looked up in the directory where the credential store database file is located.
- The .key file will be used if it is foundexists.
- An exception will be thrown raised if a .key file is not found and the
password
parameter is not used.
- a <credential_store_database_filename_without_extension>.key file such as credential_store.kdbx -> credential_store.key will be sought looked up in the directory where the credential store database file is located.
ignore_expired
- optional (boolean 0-1), default: 0- ignore_expired=0 - an exception is thrown when raised if the entry has expired.
- ignore_expired=1 - expiring expiration of an entry is ignored.
attachment
- optional (boolean 0-1)- attachment=1 - a attachment field is readused.
Advanced Parameters
create_entry
- optional (boolean 0-1), default:0
create_entry
=0 - an exception is thrown when raised if the entry not cannot be found.create_entry
=1 - creates an entry if it does not exist.- an exception is thrown when raised if the root group of the
cs://<entry_path>
URI does not match with the database credential store root group. - creates the full path to the entry if it does not exist.
- an exception is thrown when raised if the root group of the
set_property
- optional- set value of a string property:
cs://<entry_path>@<property_name>?...&
set_property=<value><property_name>
property exists:- property value will be updated.
<property_name>
property does not exist:<property_name>
property will be created with the given value.
- set value of a binary property:
...&set_property=<file path>&attachment=1cs://<entry_path>@<property_name>?
<property_name>
property exists:- property value will be updated with the
<file path>
file content.
- property value will be updated with the
<property_name>
property does not exist:<property_name>
binary property will be created with the<file path>
file content.
cs://<entry_path>@attachment?...&set_property=<file path>
- creates a new binary property with the given file content if it does not exist (property name is a file name), or updates the binary property value with from the given file content.
- set value of a string property:
stdout_on_set_binary_property
- optional (boolean 0-1), default:0
stdout_on_set_binary_property
=0- does not create the any output of for the binary property set with the
set_property
to the STDOUT..
- does not create the any output of for the binary property set with the
stdout_on_set_binary_property
=1- creates the output of to stdout for the binary property set with the
set_property
to the STDOUT.
- creates the output of to stdout for the binary property set with the
Examples
Shell JobExample for
Master/Agent (Windows)Unix
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
<job order="no" stop_on_error="no" stderr_log_level="error">
<script language="shell">
<![CDATA[
@echo off
rem Sample 1 %SCHEDULER_CREDENTIAL_VALUE%
call "%SCHEDULER_HOME%/bin/jobscheduler_credential_value.cmd" "cs://server/SFTP/homer.sos@password?file=%SCHEDULER_DATA%/config/credential_store.kdbx"
if ERRORLEVEL 1 exit /b %ERRORLEVEL%
echo %SCHEDULER_CREDENTIAL_VALUE%
rem Sample 2 stdout
call "%SCHEDULER_HOME%/bin/jobscheduler_credential_value.cmd" "cs://server/SFTP/homer.sos@password?file=%SCHEDULER_DATA%/config/credential_store.kdbx" stdout
if ERRORLEVEL 1 exit /b %ERRORLEVEL%
]]>
</script>
<run_time />
</job> |
Explanations
- The
<job>
element makes use of thestderr_log_level
attribute to cause the job to fail in case of errors being reported to stderr, e.g. from the Java classSOSKeePassDatabase
. - The script
jobscheduler_credential_value.cmd
is available from the Master's or Agent's./bin
directory. The environment variable%SCHEDULER_HOME%
is automatically provided. - The credential store file is located in the
./config
directory of a Master or Agent. The environment variable%SCHEDULER_DATA%
is automatically provided. However, the credential store file can be located anywhere in the file system. - The credential value to be retrieved is returned by the built-in environment variable %
SCHEDULER_CREDENTIAL_VALUE%
by the scriptjobscheduler_credential_value.cmd
(seeSample 1 %SCHEDULER_CREDENTIAL_VALUE%
).- %
SCHEDULER_CREDENTIAL_VALUE%
contains the last row of the possible multiple rows value.
- %
- The additional parameter
stdout
(seeSample 2 stdout
) controls that all outputs are forwarded to stdout.
Shell Job Example for Master/Agent (Unix)
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
<job order="no" stop_on_error="no" stderr_log_level="error">
<script language="shell">
<![CDATA[
SCHEDULER_CREDENTIAL_VALUE=`"$SCHEDULER_HOME/bin/jobscheduler_credential_value.sh" "cs://server/SFTP/homer.sos@password?file=$SCHEDULER_DATA/config/credential_store.kdbx"`
RETURNCODE=$?
if [ $RETURNCODE -ne 0 ]
then
exit $RETURNCODE
fi
echo $SCHEDULER_CREDENTIAL_VALUE
]]>
</script>
<run_time />
</job> |
Explanations
- The
<job>
element makes use of thestderr_log_level
attribute to cause the job to fail in case of errors being reported to stderr, e.g. from the Java classSOSKeePassDatabase
. - The script
jobscheduler_credential_value.sh
is available from the Master's or Agent's./bin
directory. The environment variable$SCHEDULER_HOME
is automatically provided. - The credential store file is located in the
./config
directory of a Master or Agent. The environment variable$SCHEDULER_DATA
is automatically provided. However, the credential store file can be located anywhere in the file system. - The credential value to be retrieved is returned to stdout by the script
jobscheduler_credential_value.sh
. The above example makes use of the environment variable$SCHEDULER_CREDENTIAL_VALUE
that holds output to stdout of the script, i.e. receives the credential value.
JavaScript Job Example for Master/Agent (all platforms)
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
<job order="no" stop_on_error="no">
<script language="java:javascript"><![CDATA[
function getCredentialStoreProperty( uri ) {
try {
return Packages.com.sos.keepass.SOSKeePassDatabase.getProperty( uri );
}
catch (e) {
throw new Error( "can't get property: " + e.message );
}
}
function exportCredentialStoreAttachment2File( uri, targetFile ) {
var fos = null;
try {
var data = Packages.com.sos.keepass.SOSKeePassDatabase.getBinaryProperty( uri );
fos = new Packages.java.io.FileOutputStream( targetFile );
fos.write( data );
} catch (e) {
throw new Error( "[" + targetFile + "] can't write attachment to file: " + e.message );
}
finally {
if ( fos !== null ) {
fos.close();
}
}
}
function spooler_process() {
// find credential store from a Master's ./config directory
var file = "config/credential_store.kdbx";
spooler_log.info( "--- get string property ---" );
var property = "server/SFTP/homer.sos@password";
var uri = "cs://" + property + "?file=" + file;
var val = getCredentialStoreProperty( uri );
spooler_log.info( "[" + property + "]=" + val );
spooler_log.info( "--- get binary property as string ---" );
property = "server/SFTP/homer.sos@homer.privat.dsa";
uri = "cs://" + property + "?file=" + file + "&attachment=1";
val = getCredentialStoreProperty( uri );
spooler_log.info( "[" + property + "]=" + val );
spooler_log.info( "--- get binary property as byte array and write to file ---" );
property = "server/SFTP/homer.sos@homer.privat.dsa";
uri = "cs://" + property + "?file=" + file;
var targetFile = "D:/my_homer.privat.dsa";
exportCredentialStoreAttachment2File( uri, targetFile );
spooler_log.info( "[" + property + "] written to " + targetFile );
return false;
}
]]></script>
<run_time />
</job> |
Explanations
Two methods can be used:
com.sos.keepass.SOSKeePassDatabase.getProperty( uri )
- is used to read a value from a credential store entry, e.g. an account, password etc.
- returns a string that holds the requested value
com.sos.keepass.SOSKeePassDatabase.getBinaryProperty( uri )
- is used to read a file attachment from a credential store entry, e.g. if the entry is attached a private key file.
- returns a byte array from a credential store entry that specifies an attachment
PowerShell Job Example (Agent for Windows)
The recommended way is to call the same script scheduler_credential_value.cmd
as explained above for shell jobs:
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
<job order="no" stop_on_error="no" stderr_log_level="error" process_class="/Agent">
<script language="powershell"><![CDATA[
$file = "$env:SCHEDULER_DATA/config/credential_store.kdbx";
$property = "server/SFTP/homer.sos@password";
$uri = "cs://" + $property + "?file=" + $file;
$val = Invoke-Expression "&`"$env:SCHEDULER_HOME\bin\jobscheduler_credential_value.cmd`" '`"$uri`"' stdout"
$spooler_log.info( "[" + $property + "]=" + $val);
]]></script>
<run_time />
</job> |
Alternatively the Java class SOSKeePassDatabase
can be invoked directly.
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
<job order="no" stop_on_error="no" stderr_log_level="error" process_class="/Agent">
<script language="powershell"><![CDATA[
$file = "$env:SCHEDULER_DATA/config/credential_store.kdbx";
$property = "server/SFTP/homer.sos@password";
$uri = "cs://" + $property + "?file=" + $file;
$val = java.exe com.sos.keepass.SOSKeePassDatabase $uri
$spooler_log.info( "[" + $property + "]=" + $val);
]]></script>
<run_time />
</job> |
Explanations
- The
<job>
element makes use of thestderr_log_level
attribute to cause the job to fail in case of errors being reported to stderr, e.g. from the Java classSOSKeePassDatabase
.
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
<job order="no" stop_on_error="no" stderr_log_level="error" process_class="/Agent">
<script language="powershell"><![CDATA[
$file = "$env:SCHEDULER_DATA/config/credential_store.kdbx";
$property = "server/SFTP/homer.sos@password";
$uri = "cs://" + $property + "?file=" + $file;
$val = Invoke-Expression "&`"${env:JAVABIN}`" com.sos.keepass.SOSKeePassDatabase '`"$uri`"'"
$spooler_log.info( "[" + $property + "]=" + $val);
]]></script>
<run_time />
</job> |
Explanations
- The
<job>
element makes use of thestderr_log_level
attribute to cause the job to fail in case of errors being reported to stderr, e.g. from the Java classSOSKeePassDatabase
. - The environment variable
%JAVABIN%
is automatically provided on an Agent.
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
<job order="no" stop_on_error="no" stderr_log_level="error" process_class="/Agent">
<script language="powershell"><![CDATA[
$file = "$env:SCHEDULER_DATA/config/credential_store.kdbx";
$property = "server/SFTP/homer.sos@password";
$uri = "cs://" + $property + "?file=" + $file;
$javaExe = "C:\Program Files\Java\jre1.8.0_171\bin\java.exe"
$val = Invoke-Expression "&`"$javaExe`" com.sos.keepass.SOSKeePassDatabase '`"$uri`"'"
$spooler_log.info( "[" + $property + "]=" + $val);
]]></script>
<run_time />
</job> |
Explanations
- The
<job>
element makes use of thestderr_log_level
attribute to cause the job to fail in case of errors being reported to stderr, e.g. from the Java classSOSKeePassDatabase
.
Finally an individual process can be started to invoke the Java class:
language | powershell |
---|---|
title | Powershell Job Example (invoke process) |
collapse | true |
| |
#!/usr/bin/bash
JS7_CREDENTIAL_VALUE=`"$JS7_AGENT_HOME/bin/agent_credential_value.sh" "cs://jobs/SFTP/sftp_server@password?file=$JS7_AGENT_CONFIG_DIR/jobs.kdbx"`
RC=$?
if [ $RC -ne 0 ]
then
exit $RC
fi
echo $JS7_CREDENTIAL_VALUE |
Explanation:
- The script
agent_credential_value.sh
is available from the Agent's./bin
folder in the installation directory. The environment variable$JS7_AGENT_HOME
is automatically provided. - The credential store file is located in the
./config
directory of the Agent. The environment variable$JS7_AGENT_CONFIG_DIR
is automatically provided. - The credential value to be retrieved is returned to stdout by the script. The example makes use of the environment variable
$JS7_CREDENTIAL_VALUE
to hold the output to stdout of the script, i.e. it receives the credential value. - As a prerequisite the
JAVA_HOME
environment variable has to be set. As the Agent is operated for Java the script will find this environment variable if it is exported when starting the Agent.
Example for Windows
Code Block | ||||
---|---|---|---|---|
| ||||
@rem Example 1: use of built-in variable %JS7_CREDENTIAL_VALUE%
@call "%JS7_AGENT_HOME%\bin\agent_credential_value.cmd" "cs://jobs/SFTP/sftp_server@password?file=%JS7_AGENT_CONFIG_DIR%\jobs.kdbx"
if ERRORLEVEL 1 exit /b %ERRORLEVEL%
@echo %JS7_CREDENTIAL_VALUE%
@rem Example 2: output to stdout
@call "%JS7_AGENT_HOME%\bin\agent_credential_value.cmd" "cs://jobs/SFTP/sftp_server@password?file=%JS7_AGENT_CONFIG_DIR%\jobs.kdbx" stdout
if ERRORLEVEL 1 exit /b %ERRORLEVEL% |
Explanation:
- The script
agent_credential_value.cmd
is available from the Agent's./bin
folder in the installation directory. The environment variable%JS7_AGENT_HOME%
is automatically provided. - The credential store file is located in the
./config
directory of the Agent. The environment variable%JS7_AGENT_CONFIG_DIR%
is automatically provided. - The credential value to be retrieved is returned via the built-in environment variable %
JS7_CREDENTIAL_VALUE%
by the script (seeExample 1: use of built-in variable %JS7_CREDENTIAL_VALUE%
).%JS7_CREDENTIAL_VALUE%
contains the last row of output to stdout.
- The additional argument
stdout
(seeExample 2: output stdout
) controls that any output is forwarded to stdout. - As a prerequisite the
JAVA_HOME
environment variable has to be set. As the Agent is operated for Java the script will find this environment variable if it is exported when starting the Agent.