Using Credential Store to securely store authentication, connection and other parameters

Introduction

The Credential Store (CS) allows sensitive data to be encrypted and stored securely and independently of the application(s) such as YADE and the JobScheduler YADE JITL Jobs that use this data. Access to the CS is only possible with access methods such as a key file and/or password.
The CS requires the use of a .kdb or .kdbx database and the installation of a kdb-compatible user interface such as "KeePass", "KeePass 2" or "KeePass-X".
The advantage of using a CS is that the CS stores sensitive information such as credentials in a standardized, secure and fully encrypted database and sensitive authentication information is not exposed in use. Applications access the CS database by using password, encryption-key file or a combination of both.
The CS can be used to securely store information of parameters, database connection URLs, private key files and other sensitive data.

Scope

This article describes the use of the Credential Store with the YADE Client.

The configuration of the Credential Store for use with the YADE JITL job is .

Example Description

The example presented in this article illustrates the configuration and use of the Credential Store in a simple file transfer operation that is carried out with the YADE Client.

The file transfer operation is based on the file transfer example described in The YADE Client Command Line Interface - Tutorial 1 - Getting Started article. The tutorial describes the configuration required to download a number of files from an online server provided by the SOS GmbH and save these files on the user's local file system. Using this server means that users can get a working example up and running with a minimum of effort. A simplified version of the configuration used in the tutorial (only specifying transfer by FTP) is available as a download: sos-berlin_demo_2_local.xml.

In the current example, the Credential Store is to store configuration information for the online server - i.e. for the file transfer source. The principle described can be equally well used for the configuration of multiple file transfer source, target, proxy and jump-host servers and for the other file transfer protocols that can be used by the YADE Client.

Note that a YADE Client is required to carry out the example file transfer. Instructions for installing and configuring the YADE Client can be found in the YADE - Tutorials article.

Configuration Procedure

Installing the Credential Store and configuring the database

KeePass 2 has been used in the current article to implement the Credential Store database. The installation and use of KeePass is described on the KeePass Web Site.

Feature Availability

FEATURE AVAILABILITY STARTING FROM RELEASE 1.12.2

Credential Store features such as secure, compliant and password-free use of the Credential Store as well as compatibility with Keepass .kdb databases require the YADE Client in version 1.12.2 or newer.

Database Configuration

Credential Store databases are stored as a file on the file system.

For the examples described in the current article the following database was configured (on a Windows system):

Path: %USERPROFILE%\jade_demo\keepass\demo_cred_store.kdbx
Master Password: sos

Note that a Master Key file (not used in the example below) can be generated using the Files/Change Master Key KeePass menu option and then selecting the Show expert options checkbox.

Adding the configuration information to the Credential Store

Configuration information is stored in the Credential Store as an Entry and Entries can be organized into Groups.

The following information can be stored in a CS Entry:

Title: The identifier for the Entry This could be a string containing, for example, the host name/server name.
User name: The user identification of a user who is authenticated for the operation.
Password: Assigned password for a user account or passphrase for a private key.
URL: The host name/server name or IP address of the server.
Notes: This block can be used to specify additional parameters for the file transfer. The syntax ... < TO REWORK
File Attachment & Custom Fields: Files such as PGP or SSH private keys can be stored as attachments. A first attachment is added as an attachment and further attachments are added using my_custom_field parameters
YADE will retrieve the contents of an attached file at run-time - intermediate or temporary files are not created when reading attachments.
Note that attachments are specified in the KeePass GUI via the AdvancedEdit Entry tab.

The following information needs to be specified for the current example:

Groups: demo,ftp (optional)
Title: demo_on_test.sos-berlin.com
User name: demo
Password: demo
URL: test.sos-berlin.com (Alternatively, the IP address could have been specified here.)

The following screenshot shows that two Groups have been configured for the current example, named "demo" and "ftp", along with the Entry "demo_on_test.sos-berlin.com".

The next screenshot shows the configuration of the parameters in the "demo_on_test.sos-berlin.com" Entry:

Integrating the Credential Store in a File Transfer Configuration

The use of the Credential Store is specified in YADE Client file transfer configuration files, which are written in XML. We recommend using the SOS XML Editor to edit these files. Instructions for downloading, installing and using the XML Editor are linked from this page.

In the remainder of the current article, it is assumed that readers have made themselves familiar with the organization of the YADE Client file transfer configurations into Profiles and Fragments. This is described in The YADE Client Command Line Interface - Tutorial 1 - Getting Started article mentioned above.

The current example uses the XML configuration from the Getting Started tutorial article above and describes the necessary configuration elements required to move the sensitive information such as user name and password from the XML file to the Credential Store. Users wishing implement the current example should download the tutorial file transfer configuration file linked above and open it in their XML Editor.

The information required to use the Credential Store falls into two "areas":

Information about the Credential Store itself (location, how to access its contents, etc.).
This information is configured in the XML file in a CredentialStoreFragment.
Information about how the information in the Credential Store (server address, password, etc.) is to be accessed for the file transfer.
This information is stored as a string in each of the relevant XML ProtocolFragment child elements (Hostname, Account. etc.).

In addition, the ProtocolFragment element has a reference specifying that the Credential Store is to be used.

Specifying the Credential Store

Fragments
- ProtocolFragments
  - FTPFragment name=""
    - ....
    - CredentialStoreFragmentRef ref ="ftp_demo"
- CredentialStoreFragments
  - CredentialStoreFragment name ="ftp_demo"
    - CSFile file path ....
    - CSAuthentication
      - ...
    - CSEntryPath

A CredentialStoreFragments element at the same level in the XML hierarchy as the Protocol Fragments elements.
- This element can have one or more child Credential Store Fragment elements (described below).
A CredentialStoreFragment element that is referenced from the ProtocolFragment. This Fragment specifies the location of and authentication required for the Credential Store.
- Password, key file and combined password/key file authentication methods are possible.
A CredentialStoreFragmentRef element as a child element of the Protocol Fragment element - in the current example this is the FTPFragment.
- The values of the connection and authentication elements are modified to refer to elements stored within the Credential Store.

Addressing the information in the Credential Store

Parameters stored in a Credential Store database Entry can be addressed in the CredentialStoreFragment XML element as follows:

The CSEntryPath element is used to specify the base path in the Credential Store database to the Entry.
In the current example this would be set to:
- demo/ftp/demo_on_test.sos-berlin.com
  where demo and ftp are (optional) Group names, as already mentioned, and demo_on_test.sos-berlin.com is the Title of the KeePass database Entry.

The Credential Store Entry parameters are addressed using one of the following syntaxes:

relative:
- cs://@parameter_name, where the parameter_name is the name of the relevant parameter specified for the Entry - for example, url and the CSEntryPath element is filled as shown above
fully specified:
- cs://demo/ftp/demo_on_test.sos-berlin.com@parameter_name, and where the CSEntryPath element, which is a required element, is left blank

The following parameters are set in the Credential Store in the current example:

Hostname: cs://demo/ftp/demo_on_test.sos-berlin.com@url (where @url specifies the URL element stored in the database )
Account: cs://demo/ftp/demo_on_test.sos-berlin.com@user (where @user specifies the User name element stored in the database)
Password: cs://demo/ftp/demo_on_test.sos-berlin.com@password (where @password specifies the Password element stored in the database)

Note that a full list of parameters is described in the Adding an Entry to the Credential Store section above.

Alternative Syntax

Note also that although the CSEntryPath element is a required element, it can be left empty and that fully specified paths can be used for each parameter. For example, the Password could be specified using:

cs://demo/ftp/demo_on_test.sos-berlin.com@password

Configuration in the XML Editor

The parts of the XML configuration relevant to the use of the Credential Store are shown in the following screenshot of the configuration for the current example, with parameter values highlighted in yellow:

MP Please add new scrennshot with fully qualified,

Running the YADE Client with the Credential Store

The use of the Credential Store is contained within the settings file and is not exposed when calling the YADE Client. For example, on Windows systems, the YADE Client is called for the current example using:

Call on Windows systems

C:\Program Files\sos-berlin.com\jade\client\bin>jade.cmd -settings="%USERPROFILE%\jade_demo\sos-berlin_demo_2_local_cs.xml" -profile="ftp_server_2_local_cs"

MP after executing the YADE command you will see the server information used from credential store.

The following listing shows the output produced by the example configuration. Note that the problem that occurred with the transfer of one of the files has nothing to do with the use of the Credential Store.

MP please consider to delete the log file.

Output

C:\Program Files\sos-berlin.com\jade\client\bin>jade.cmd -settings="%USERPROFILE%\jade_demo\sos-berlin_demo_2_local_cs.xml" -profile="ftp_server_2_local_cs"

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+                       START  : JADE.CMD                         +
+                       -----------------                         +
+ DATE     : 18.04.2018 15:32:45,54
+ HOSTNAME : JS-PC
+ USER     : aa
+ CALL     : C:\Program Files\sos-berlin.com\jade\client\bin\jade.cmd -settings="C:\Users\aa\jade_demo\sos-berlin_demo_2_local_cs.xml" -profile="ftp_server_2_local_cs"
+                                                                 +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

main INFO  15:32:49,319   (SOSDataExchangeEngineMain.java:76) ::Execute SOSDataExchange - Kommandozeilenprogram startet ....
main INFO  15:32:49,709   (SOSDataExchangeEngine.java:536) ::showBanner
************************************************************************
*                                                                      *
*                     YADE - Managed File Transfer                     *
*                     -----www.sos-berlin.com-----                     *
*                                                                      *
************************************************************************
  Version                 = 1.12.3-SNAPSHOT (2018-04-15 23:09, revision f156fa1144fe219789e9bf2ad1d3a4b52a68cd24) Copyright 2003-2018 SOS GmbH Berlin
  Date                    = 2018-04-18 15:32:49
  SettingsFile            = C:\Users\aa\jade_demo\sos-berlin_demo_2_local_cs.xml
  Profile                 = ftp_server_2_local_cs
  Operation               = copy
  Transactional           = false

  +------------Source------------
  | Protocol              = ftp
  | Host                  = test.sos-berlin.com
  | IP                    = 93.157.51.161
  | User                  = demo
  | Password              = ***
  | Passive               = false
  | TransferMode          = binary
  | Directory             = ./
  | FileSpec              = .*
  | ErrorWhenNoFilesFound = true
  | Recursive             = false
  | Remove                = false

  +------------Target------------
  | Protocol              = local
  | Host                  = JS-PC
  | IP                    = 192.11.0.85
  | Directory             = C:\Users\aa\jade_demo\transfer_receive/
  | OverwriteFiles        = true


main INFO  15:32:49,803   (SOSVfsFtpBaseClass.java:242) ::doConnect SOSVfs_D_0102: Verbunden mit Rechner 'test.sos-berlin.com' ³ber Port-Nummer '21'.
main INFO  15:32:49,866   (SOSVfsFtpBaseClass.java:958) ::login (demo@test.sos-berlin.com:21) SOSVfs_D_133: Benutzer 'demo' eingeloggt.
main INFO  15:32:49,928   (SOSVfsFtpBaseClass.java:1295) ::transferMode SOSVfs_D_123: Antwort des FTP-Servers ['binary']: '200 Type set to I'.
main INFO  15:32:50,272   (SOSDataExchangeEngine.java:897) ::setInfo 6 files found for regexp '.*'.
main ERROR 15:32:51,131   (SOSFileListEntry.java:1150) ::run SOSVfs_E_229: Fehler. Daten³bertragung nicht m÷glich. Grund: com.sos.JSHelper.Exceptions.JobSchedulerException: unable to get inputstream for file './test_5.txt'
main ERROR 15:32:51,131   (SOSDataExchangeEngine.java:1140) ::transfer SOSDataExchangeEngine.TRANSFER_ABORTED
main INFO  15:32:51,131   (SOSFileList.java:464) ::rollback Rollback aborted files.
main INFO  15:32:51,147   (SOSDataExchangeEngine.java:359) ::showResult
*************************************************************************
 Ausf³hrungsstatus                 = Fehlerhaft.
 Erfolgreiche ▄bertragungen        = 5
 ▄bersprungene ▄bertragungen       = 0
 Fehlgeschlagene ▄bertragungen     = 1
 letzter aufgetretener Fehler      = unable to get inputstream for file './test_5.txt'

*************************************************************************
main ERROR 15:32:51,147   (SOSDataExchangeEngineMain.java:81) ::Execute Execute: Fehler aufgetreten: unable to get inputstream for file './test_5.txt', Programm wird mit Exit-Code 99 beendet.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+                        END   : JADE.CMD                         +
+                        ----------------                         +
+ DATE     : 18.04.2018 15:32:51,20
+ HOSTNAME : JS-PC
+ USER     : aa
+ CALL     : C:\Program Files\sos-berlin.com\jade\client\bin\jade.cmd -settings="C:\Users\aa\jade_demo\sos-berlin_demo_2_local_cs.xml" -profile="ftp_server_2_local_cs"
+ EXIT     : 99
+                                                                 +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

As with all YADE jobs, the number of successful file transfers can be seen in the log file.

Space shortcuts

Page tree