Introduction
The JobScheduler Universal Agent Batch Installer is a set of files that is used by a JobScheduler Master to install one or more Universal Agents on local and remote servers.
Article Scope
This article is one of a series describing how to configure the JobScheduler Universal Agent Batch Installer. It describes a simple use case for the Universal Agent Batch Installer - to install a single Universal Agent on a remote server. This article describes the installation procedure in detail and is intended as a "getting started" guide. The other articles in this series describe more complex use cases and are listed in the introductory JobScheduler Universal Agent - Batch Installation and Update of Agents article.
A general introduction to installation of the JobScheduler Universal Agent can be found in the JobScheduler Universal Agent - Installation & Operation article.
The JobScheduler Universal Agent - Batch Installation and Update of Agents article provides an overview of how the Batch Installer works. New users are strongly recommended to read this article before proceeding with the instructions presented here.
The Batch Installation - Reference Documentation article provides a detailed guide to configuration of the batch installer with, for example, full lists of parameters.
Installation Environment
This article describes the use of the JobScheduler Universal Agent Batch Installer on a Linux server, working from a Windows system.
Note that the Batch Installer is only available for the current JobScheduler Universal Agent version.
Installation Procedure
General requirements:
- The JobScheduler Master to be used for the installation of the Universal Agents should be up and running.
- A Java Run-time Environment in version 1.8 or newer should already be installed on the system(s) the Agent is to be installed on.
- We recommend that you use the SOS XML Editor to manage the configuration files required for the Batch Installer. More information about the installation and use of the XML editor will be provided later in this article.
1. Download and Extract the Batch Installer and Universal Agent Files
Files can be downloaded from http://www.sos-berlin.com/jobscheduler-downloads.
Unix Target Systems
- Download the "JobScheduler Universal Agent" tarball for Unix.
- Download the "Universal Agent Batch Installer".
- This archive contains the following folders and sub-folders:
live/batch_install_universal_agent
- This folder contains the Jobs and Job Chain used to carry out the installation.
Extract these files to the thelive
folder of the JobScheduler Master:SCHEDULER_DATA/config/live
.
- This folder contains the Jobs and Job Chain used to carry out the installation.
batch_install
- This folder contains the configuration files for the Batch Installer.
Extract these files to theconfig
directory of the JobScheduler Master:SCHEDULER_DATA
/config
.
- This folder contains the configuration files for the Batch Installer.
- This archive contains the following folders and sub-folders:
- Copy the JobScheduler Universal Agent installation file to the
SCHEDULER_DATA/config/
batch_install
folder.
Windows Target Systems
- Download the "JobScheduler Universal Agent" for Windows.
- Download the "Universal Agent Batch Installer".
- This archive contains the following folders and sub-folders:
- the
live/batch_install_universal_agent
folder- This folder contains the Jobs and Job Chain used to carry out the installation.
Extract these files to thelive
folder of the JobScheduler Master:SCHEDULER_DATA/config/live
.
- This folder contains the Jobs and Job Chain used to carry out the installation.
- the
batch_install
folder- This folder contains the configuration files for the Batch Installer.
Extract these files to the JobScheduler configuration directory:SCHEDULER_DATA/config
.
- This folder contains the configuration files for the Batch Installer.
- the
- This archive contains the following folders and sub-folders:
- Copy the JobScheduler Universal Agent installation file to the
SCHEDULER_DATA/config/
batch_instal
folderl
.
2. Configure the Transfer of the Universal Agent Installation Files and the Agent Installation
Hints
Configuring complex XML files with a text editor is often a tedious and error-prone procedure. To get around this, we recommend that you use our XML Editor to manage the configuration files required for the Batch Installer. The XML Editor automatically validates configuration files and therefore reduces the possibility of configuration errors.
After downloading the XML Editor, extract the contents of the downloaded zip file to a convenient folder (an installation routine is not required) and start the
sos.xml.editor.exe
program.The XML Editor is available for Windows.
- An overview of the use of the XML Editor is provided in the Using the XML Editor section at the end of this article.
A more detailed description of the use of the XML Editor can be found in the XML Editor - Operation article.
- The JobScheduler Master comes with its own managed file transfer solution, YADE. YADE will be used here for transferring the files from the JobScheduler Master to the Agent servers.
For more information about YADE, see the YADE product site.
2.1 Configure transfer of the Universal Agent installation files
The transfer of the Universal Agent installation files is configured in the yade_agent_batch_installer_settings.xml
file. This file will be located in the
folder unpacked in step 1 above.SCHEDULER_DATA
/config/batch_installer
- Open your
yade_agent_batch_installer_settings.xml
YADE configuration file in the XML Editor.- Assign the following schema to the configuration file:
Two configuration elements are required for the transfer of the files required for each Universal Agent installation:
- a Protocol Fragment that defines how the file transfer is to be carried out and
- a Profile that defines the file transfer itself.
Protocol Fragment Configuration
A Protocol Fragment is required in the YADE configuration file for each installation. The easiest way to add a Protocol Fragment element using the XML Editor is to copy an already existing element and pasting it to the parent element - in this case <ProtocolFragments>
.
Each Protocol Fragment has to have a name attribute that is unique to the YADE configuration file and is made up of <protocol>_<host>:<port>
where:
<protocol>
identifies the protocol to be used for making the transfer<host>
is the transfer target host name<port>
is the transfer target port number.
An example fragment name would be SFTP_192.11.0202:4441
A complete typical Protocol Fragment configuration would be (shown from the root Configurations element):
- Configurations
- Fragments
- ProtocolFragments
- SFTPFragment name="
SFTP_192.11.0.202:4441
"- BasicConnection
- Hostname
192.11.0.202
- Hostname
- SSHAuthentication
- Account sos
- AuthenticationMethodPassword
- Password
********
- Password
- BasicConnection
- SFTPFragment name="
- ProtocolFragments
- Fragments
Profile Configuration
A separate Profile in the YADE configuration file is required for each Agent installation.
Each Profile has to have a profile_id attribute that is unique to the YADE configuration file and is made up of <host>:<port>
where:
<host>
is the transfer target host name and<port>
is the transfer target port number.
A typical profile_id attribute would be 192.11.0.202:4441
.
A typical Profile configuration could contain the following elements:
- The CopySource ... Directory element, which points to the
batch_install
folder which was unpacked as described in Section 1 of this article above. - The SFTPFragmentRef element, which contains the name of the SFTPFragment element configured in the Protocol Fragment Configuration section above.
- The CopyTarget ... Directory, which points to a folder on the target server where the Account specified in the SSHAuthentication.Account element (configured in the Protocol Fragment Configuration section above) will be able to carry out the installation.
A complete typical Profile configuration for a single Agent would be (listed from the root Configurations element):
- Configurations
- Profiles
- Profile profile_id="
192.11.0.202:4441
"- Operation
- Copy
- CopySource
- CopySourceFragmentRef
- LocalSource
- SourceFileOptions
- Selection
- FileSpecSelection
- FileSpec
.*\.(sh|gz)$
- Directory
C:\ProgramData\sos-berlin.com\jobscheduler\jobscheduler_1.12\config\batch_install
- FileSpec
- FileSpecSelection
- Selection
- CopySourceFragmentRef
- CopyTarget
- CopyTargetFragmentRef
- SFTPFragmentRef ref="
SFTP_192.11.0.202:4441
"
- SFTPFragmentRef ref="
- Directory
/home/sos/temp/js_agent_batch_install
- CopyTargetFragmentRef
- CopySource
- Copy
- Operation
- Profile profile_id="
- Profiles
The following code block lists an example YADE configuration file for installing two agents. This file has therefore two SFTPFragment elements and two Profile elements.
The following files will be transferred from the JobScheduler Master to the Agent servers:
- the JobScheduler Universal Agent installation file:
jobscheduler_unix_universal_agent.<version>.tar.gz
- a start script for every Agent instance:
jobscheduler_agent_<port>.sh
- an installation script:
jobscheduler_universal_agent_install.sh
YADE Settings File for File Transfer
Note that the file shown above is in the YADE XML format and that YADE versions 1.10 and older require a .ini
file to perform the transfer that is specified in the <globals><install_path><transfer><settings>
element in the Batch Installer configuration file.
- We recommended that you use the XML Editor to convert configuration files from XML to the
.ini
format (see the XML Editor for converting YADE configuration files (XML) into YADE setting files (.ini) section below for more information). - The following example shows as an example a
.ini
file that has been converted from the XML file above. You can download the example using this link: yade_agent_batch_installer_settings.ini.
2.2 Configure the installation of the Universal Agents
The installation of the Universal Agents on the target servers is configured in the batch_installer_config_file.xml
file in the XML Editor. This file will be located in the
folder unpacked in step 1 above.SCHEDULER_DATA
/config/batch_installer
- Open your
batch_installer_config_file.xml
configuration file in the XML Editor.- Assign the
scheduler_universal_agent_installations.xsd
schema, which will also be located in the
folder.
config/batch_installerSCHEDULER_DATA
/
- Assign the
The root XML element of the Batch Installer configuration file is the <installations>
element. This contains an <installation>
element for each Agent instance to be installed. In addition, the <global>
element can be used in a homogeneous environment to specify parameters that will be applied to all Agent installations. The majority of configuration elements can be specified as <global>
and as <installation>
elements. If a parameter is specified in both the <global>
element and in an individual <installation>
element then the setting for the individual element will overwrite the global setting.
Post-processing commands
- Note that mandatory elements are not indicated in the XML Editor as it is possible to define a configuration items at either the globals level or at the individual agent installation level.
- In addition, there are no default parameter settings for the batch installer. The installation will fail if mandatory configuration items are not set. Mandatory configuration items are indicated in the list of parameters below.
The parameters to be configured are:
- The
<install_path>
element, which points to a location where the Agent should be installed. (Mandatory)- A typical value would be
/opt/sos-berlin.com/jobscheduler
- A typical value would be
- The <transfer><settings> element which points to the location where the YADE transfer configuration file that was configured in Section 2.1 above. (Mandatory)
- A typical value would be
C:\ProgramData\sos-berlin.com\jobscheduler\jobscheduler_1.12\config\batch_install\yade_agent_batch_installer_settings.xml
- A typical value would be
- The
<ssh>
elements (often partly configured as<global>
elements):<auth_method>
: the authentication method to be used for the SSH connection. (Mandatory)<port>
: SSH port. (Mandatory)<user>
: SSH login account name. (Mandatory)<password>
: Password for the SSH login account.- <
auth_method>
: SSH authentication method. (Mandatory) <host>
: the host name of the target server. (Mandatory)
- The
<installation_file>
element - the name of the Universal Agent installation file. This filename contains the version of the Agent which you are installing and it is helpful for recognizing the version of the Agents that are installed. (Mandatory)- For Unix systems this is usually
jobscheduler_unix_universal_agent.<version>.tar.gz
- For Unix systems this is usually
- The
<agent_options>
elements:<java_home>
: the location of the Java Runtime Environment (JRE)- See here for more information.
<java_options>
: the Java options, e.g the Java memory settings- See here for more information
<scheduler_http_port>
: the http port that the Universal Agent is to listen to. (Mandatory)<
the IP address of the target system. (Mandatory)scheduler_
ip_address>
: the user name the Agent is to run under on the target system. (Mandatory)<
scheduler_user>
: the directory to be used for the Agent Log files.<
scheduler_log_dir>- A typical location would be
/var/log/sos-berlin.com/jobscheduler/agent
. - See here for more information
- A typical location would be
<
the kill scripts allows a task and it's child processes to be killed.scheduler_
kill_script>- See here for more information.
<scheduler_pid_file_dir>
the- See here for more information.
<agent_installation_script>
the script to be used for the installation of the Agent (Mandatory)- This is usually
jobscheduler_universal_agent_install.sh
- This is usually
- The
<postprocessing>
elements:- Post-processing commands for the Agent server's operating system. The default commands for an Ubuntu distribution will place the Agent start script under
/etc/init.d
and will setup the autostart for the Universal Agent:echo ${ssh.sudo_password} | sudo -S cp ${install_path}/jobscheduler_agent/bin/jobscheduler_agent_${agent_options.scheduler_http_port}.sh /etc/init.d/jobscheduler_agent_${agent_options.scheduler_http_port}
echo ${ssh.sudo_password} | sudo -S
update-rc.d jobscheduler_agent_${agent_options.scheduler_http_port} defaults
- These two commands will be executed by the
PerformInstall
job node, which is the last job node in the batch installer job chain. If you want to add more commands, please add the call to the job node.
- Post-processing commands for the Agent server's operating system. The default commands for an Ubuntu distribution will place the Agent start script under
Post-processing commands
- The SSH login user has to be set to
root
in order to execute these commands. Therefore the SSH login user should be included insudoers
. Note that the values for the elements defined in the Batch Installer configuration file can be used for the postprocessing commands, for instance:
${install_path}
: the value for the<install_path>
element${agent_options.scheduler_http_port}
: the value for the<
scheduler_http_port>
element. If this element is not defined, the default value (4445) will be taken, as in the example full XML code listing below.
A complete typical installation configuration for a single Agent would be (shown from the root installations element):
- installations lastRun="2018-01-02 13:01"
- globals
- install_path
/opt/sos-berlin.com/jobscheduler
- transfer
- settings
C:\ProgramData\sos-berlin.com\jobscheduler\jobscheduler_1.12\config\batch_install\yade_agent_batch_installer_settings.xml
- settings
- ssh
- auth_method
password
- port
22
- auth_method
- installation_file
jobscheduler_unix_universal_agent.1.12.0.tar.gz
- install_path
- installation lastRun="
2018-01-02 13:01
"- agent_options
- scheduler_http_port
4441
- scheduler_user
sos
- scheduler_log_dir
/var/log/sos-berlin.com/jobscheduler/agent
- scheduler_http_port
- ssh
- user
sos
- password
sos
- sudo_password
sos
- host
192.11.0.202
- user
- postprocessing
- command
echo ${ssh.sudo_password} | sudo -S cp ${install_path}/jobscheduler_agent/bin/jobscheduler_agent_${agent_options.scheduler_http_port}.sh /etc/init.d/jobscheduler_agent_${agent_options.scheduler_http_port}
- command
echo ${ssh.sudo_password} | sudo -S update-rc.d jobscheduler_agent_${agent_options.scheduler_http_port} defaults
- command
- agent_options
- globals
Note that in the above example two of the SSH elements (<auth_method>
and <port>
) are configured globally and four (<user>
, <password>
, <sudo_password>
and <host>
) are configured individually for each Agent.
Hints
Note that if the port is not specified with the
<installations><installation><agent_options>
<scheduler_http_port>
element the default value of4445
will be applied to all Agent installations.- For a homogeneous environment parameters can be specified with the
<global>
element and applied to all Agent installations.
The following code block shows Batch Installer configuration file for installing two agents. This file has therefore two installation elements and fewer elements defined in the <globals>
section than the typical installation configuration for a single Agent shown above.
The Batch Installer configuration file for the example can be downloaded here: batch_installer_config_file.xml.
2.3 Configure the JobScheduler Jobs
Agent installation is carried out by the four Jobs in the universal_agent_installer
Job Chain. Execution of this Job Chain is started by an Order that is generated by the AgentBatchInstaller
Job. A more detailed description of the installation procedure can be found in the JobScheduler Universal Agent - Batch Installation and Update of Agents article.
Two parameters have to be set for the AgentBatchInstaller
Job which generates the Order for the universal_agent_installer
Job Chain.
Open the JobScheduler Master's
folder in the JobScheduler Object Editor, JOE and edit the parameters for the
live/batch_install_universal_agentSCHEDULER_DATA/
AgentBatchInstaller
Job:
Set the
installation_definition_file
Job parameter to the path that points to the Batch Installer configuration file.
If the paths recommended in Step 1 of this procedure above were followed this will be:SCHEDULER_DATA/config/
batch_install/batch_installer_config_file.xml
- Similarly, the
installation_job_chain
parameter should be set to:/batch_install_universal_agent/universal_agent_installer
- Leave the
filter_install_host
andfilter_install_port
parameters blank. - Leave the update parameter set to
true
.
3 Installation of the Agents
The Agents can be installed after the XML configuration files and JobScheduler Jobs have been configured as described above.
Agent installation is carries out by the four Jobs in the universal_agent_installer
Job Chain. Execution of this Job Chain is started by an Order that is generated by the AgentBatchInstaller
Job. A more detailed description of the installation procedure can be found in the JobScheduler Universal Agent - Batch Installation and Update of Agents article.
Carry out the following steps to start the
Job:AgentBatchInstaller
- Sign in to the JOC Cockpit operating interface with a User Account that has the permissions necessary to start a Job.
- Open the "Jobs" view and open the "batch_installer_universal_agent" folder.
- Click on Action symbol (the ellipsis) alongside the AgentBatchInstaller Job and select the Start Task now option.
- This job will create the appropriate Orders for the JobScheduler Master as explained in the description of the Batch Installer Application and start execution of the Job Chain.
- Progress of the installation procedure can be followed either by clicking on the Expand Details button or by opening the same folder in the JOC Cockpit's Job Chains view, clicking on the Action symbol and selecting the Show Job Chain Details menu item.
The XML Editor can be used to manage the Batch Installer configuration file and the YADE configuration file. An example Batch Installer configuration file for the installation of a single Agent is shown in the following screenshot: An example YADE configuration file for the installation of a single Agent is shown in the next screenshot: The "YADE Xml2Ini Converter" function in the XML Editor is available for converting YADE files from XML to the Note that it is not necessary to create a Profile for the Batch Installer, as conversion will take place automatically once the "Autotransform" option has been chosen. Every time a change in the XML file is saved the XML file will be automatically converted and saved in A pop-up window will show every time a change is saved if the option "Show notification on autotransform" is chosen. Using the XML Editor
XML Editor for Batch Installer configuration file
XML Editor for YADE configuration file
XML Editor for converting YADE configuration files (XML) into YADE setting files (.ini)
.ini
format required by versions of YADE before 1.11. Clicking on the button of the same name will open the following form:.ini
format.