Installing the JS7 Agent
Prerequisites
- A Java Runtime Environment starting from version 1.8 is required.
Preparation
- Choose the Agent archive for the target system from the JobScheduler Downloads page.
- For Unix systems:
- A tarball archive is available that can be used for:
- For Windows systems:
- A .zip archive is available that can be used for:
- manual installation on a small number of computers.
(Described in the current article.) - the installation of a large number of Agents using third party deployment tools.
- An installer that can be used for manual installation a small number of computers.
(Explained in detail with the JobScheduler Universal Agent - Installation with Windows Installer article.)
Installation
- Unzip the downloaded file to an arbitrary directory.
- Directory structure (only files and folders directly relevant):
bin
agent.cmd
- The start script for Windows platforms.
agent.sh
- The start script for Unix platforms.
agent_instance.sh-example
lib
- The directory for Java libraries.
- Configure the settings in
log4j2.xml
file if you want to adjust the logger settings.
var_4445
service
(for Windows)
- On Windows Systems:
- You have to change the directory permissions for the
.\logs
and .\service
directories if you have extracted the JobScheduler Agent under e.g. C:\Program Files.
- If multiple instances are configured then every instance must have its own
./var_4445
data directory (e.g. ./var_<port of the instance>
)
Update of a an Agent
- Preparations for an update
- Stop all JobScheduler Universal Agent instances
- Remove the ./lib directory
- Unzip the downloaded file to the installation directory
- If you use the Windows installer of JobScheduler Universal Agent then the ./lib directory is updated automatically during the setup. It is not necessary to remove the ./lib directory before you start the installer.
Running the Agent
- SOS does not recommend running the JobScheduler Agent as
root
(for Unix) or as Administrator
(for Windows). - Instead the user account should be used that jobs are executed for. Should jobs be executed for a number of user accounts then consider the chapter Running multiple instances of JobScheduler Universal Agent.
Usage
Running the start script without parameters shows the usage clause:
...
Code Block |
---|
language | text |
---|
title | Usage for Windows |
---|
|
Usage: agent.cmd command [options] [web services]
command:
start [options]
stop [options]
abort [options]
restart [options]
status [options] [web services]
debug [options]
kill [options]
install-service [options]
remove-service [options]
start-service [options]
options:
-http-port=<[hostname or ip address:]number> | to listen to a specific host name or ip address if it is specified; default port: 4445,
-https-port=<[hostname or ip address:]number> |
-data-directory=<location of data directory> | default: ./var_4445
-timeout=<number> | in seconds; only for stop and restart
-kill-script=<location of kill script> | default: ".\bin\agent_kill_task.cmd"; only for start, debug
-java-options=<java options> | default: -Xms100m; see https://kb.sos-berlin.com/x/aIC9
-job-java-options=<java options> | see https://kb.sos-berlin.com/x/aIC9
web services:
overwiew | default
task | running JobScheduler Agent tasks
tunnel | tunnels summary
tunnel/ | list of tunnels
command | running command summary
command/ | list of running commands (watching file_order_source) |
Command Line Options
-http-port=<[hostname or ip address:]number>
- is the HTTP port that the Agent is listening to in order to receive requests from a JobScheduler Master:
- Without this option being used the port defaults to 4445 and the Agent listens to all available network interfaces.
- Should you want to specify a port then the following precedence applies:
- First precedence: command line option
- Second precedence: environment variable
JS7_AGENT_HTTP_PORT
(see below) - Third precedence: use of default value
-https-port=<[hostname or ip address:]number>
- is the HTTPS port that the Agent is listening to in order to receive requests from a Controller:
- Without this option being used the port defaults to 4445and the JobScheduler Agent listens to all available network interfaces.
- Should you want to specify a port then the following precedence applies:
- First precedence: command line option
- Second precedence: environment variable
SJS7_AGENT_HTTP_PORT
(see below) - Third precedence: use of default value
-data-directory=<number>
- Location of the data directory.
- It has to be unique over all JobScheduler Universal Agent instances
- Should you want to specify a data directory then the following precedence applies:
- First precedence: command line option
- Second precedence: environment variable
JS7_AGENT_DATA
(see below) - Third precedence: use of default value (=
JS7_AGENT_HOME/var_<JS7_AGENT_PORT>
)
-timeout=<number>
- This option can be used to specify the number of seconds that the Agent will wait for tasks to stop.
- This option can be applied for
stop
and restart
commands. - The Agent sends a SIGTERM signal to the task and having reached the timeout a SIGKILL signal will be sent to stop any tasks immediately.
-kill-script=<location of kill script>
The kill scripts provide the functionality to kill a task and it's child processes.
- Two kill scripts are provided
This option can be used to specify the location of a different "kill script" if necessary
Should you want to specify a different "kill script" then the following precedence applies:
- First precedence: command line option
- Second precedence: environment variable
JS7_AGENT_KILL_SCRIPT
(see below) - Third precedence: use of default value
-java-options=<java options>
With Java 1.8 the initial memory allocation has changed, for details see How to manage the Java heap space.
This option can be used to apply Java options for the Agent, e.g. the memory settings.
Without this option being used the Java options default to '-Xms100m'.
Should you want to specify the Java options then the following precedence applies:
- First precedence: command line option
- Second precedence: environment variable
JAVA_OPTIONS
(see below) - Third precedence: use of default value
-job-java-options=<java options>
Without this option being used the Java options for each job which is started by the JobScheduler Universal Agent.
Should you want to specify the Java options for the jobs then the following precedence applies:
- First precedence: command line option
- Second precedence: environment variable
JS7_AGENT_JOB_JAVA_OPTIONS
(see below)
Start the Agent
Code Block |
---|
|
agent.cmd|sh start [options] |
Stop the Agent
Code Block |
---|
|
agent.cmd|sh stop [options] |
...
- This corresponds to sending SIGKILL with a kill command.
- Should jobs be running that started detached child processes then it is not guaranteed that child processes will be killed.
Restart the Agent
Code Block |
---|
|
agent.cmd|sh restart [options] |
Check the Agent Status
Code Block |
---|
|
agent.cmd|sh status [options] |
...
Code Block |
---|
ERROR: spray.can.Http$ConnectionAttemptFailedException: Connection attempt to localhost:4445 failed
...JS7 Agent(4445) not started! |
Monitoring Agents
See the How to perform active checks with a System Monitor such as Nagios/op5 article for information about configuring an automated status check of Agents for monitoring purposes.
Master/Agent Compatibility
The Agent will raise an error if it is contacted by a Controller that is not compatible with the Agent's version.
Working Directory for Jobs
The JS7_AGENT_WORK_DIR
environment variable can be used to change the jobs working directory from the default value of JS7_AGENT_HOME
to a value such as $HOME (Unix) or %USEERPROFILE% (Windows).
Windows Service Interface: Usage
The following information applies to batch installation on Windows systems. For installation with a GUI and user dialog see JobScheduler Universal Agent - Installation with Windows Installer.
The Agent is operable as a Windows Service. The start script of the Agent allows to install/remove the Windows Service.
Install the Windows Service
Code Block |
---|
|
agent.cmd install-service [-http-port=<number>] [-ip-address=<hostname or ip address>] |
...
Warning |
---|
During the service installation it tries to copy the executable file for the Windows Service to the .\service directory. This operation could fail with the error "Access denied" if you have extracted the JobScheduler Universal Agent to e.g. C:\Program Files\. In this case you can change the permissions of the .\service directory or open the command prompt with elevated administrator rights and execute the above command once more. |
Start the Windows Service
Code Block |
---|
|
agent.cmd start-service [-http-port=<number>] |
...
Info |
---|
The stop command contains more than a simple stop-service command: the stop command checks whether the Agent was started through the CLI or as a Windows Service and stops the Agent accordingly. Therefore there is no stop-service command. |
Remove the Windows Service
Code Block |
---|
|
agent.cmd remove-service [-http-port=<number>] |
...
Warning |
---|
During removal of the service it tries to remove the executable file of the Windows Service from the .\service directory. This operation could fail with the error "Access denied" if you have extracted the Agent to e.g. C:\Program Files\. In this case you can change the permissions of the .\service directory or open the command prompt with elevated administrator rights and execute the above command once more. |
After the installation of the Windows Service you will find the .\service\sos_js7_agent_<http-port>w.exe
file. Start this program to configure the Windows Service.
Image Modified | For example goto the "Startup" tab to modifiy the start parameter |
Logging
- Log File
- On startup the Agent creates a log file in the directory that is pointed to by the environment variable
JS7_AGENT_LOG_DIR
or in the var_4445/logs
subdirectory of the Agent installation directory. - Log file names are created from a prefix and the port used by the Agent like this:
- Log files are rotated for each day (see ./
lib/log4j2.xml
) for which job activities occur. - Rotated log files are assigned file names like this:
agent.log.<yyyy-MM-dd>
- For days where the Agent has no jobs to execute no log rotation will be performed.
- PID File
- On startup the Agent creates a PID file in the directory that is pointed to by the environment variable
JS7_AGENT_PID_FILE_DIR
or in the log directory. The PID file contains the Process ID of the system process that the Agent is running in. - The PID file is used in order to prevent the Agent to be started twice with the same settings and it can be used for shutdown scripts that require the PID to terminate the process.
- PID file names are created like this:
- Further References
Anchor |
---|
| js_agent_env_vars |
---|
| js_agent_env_vars |
---|
|
Environment Variables
The following environment variables can be used:
JAVA_HOME
- points to the location of the Java Runtime Environment (JRE).
- Without setting this environment variable Java will be used from the location specified by the system path.
- Please consider that
JAVA_HOME
does not point to the location of a JDK but to a JRE directory where the bin/java
executable resides.
JAVA_OPTIONS
- sets Java options, e.g. the Java memory settings for the Agent.
- Without setting this environment variable the Java options default to '-Xms100m'.
JS7_AGENT_JOB_JAVA_OPTIONS
- sets Java options for each job that is started by the Agent.
JS7_AGENT_HOME
- points to the directory where the JobScheduler Agent has been installed.
- Without setting this environment variable the default value is the parent directory of the start script.
- Should you want to start the Agent from a directory different to the Agent installation directory, e.g. by copying the start script to some other location, then this environment variable has to be set in order to locate the JobScheduler Agent installation directory.
JS7_DAGENT_ATA
- points to the directory where the JobScheduler Agent has its data directory.
- Without setting this environment variable the default value is
JS7_HOME/var_<JS7_HTTP_PORT>
.
JS7_AGENT_HTTP_PORT
- sets the http port that the JobScheduler Agent is listening to.
- indicates which network interfaces the JobScheduler Agent should listen to if a host or ip address specified.
- if only a port number is specified then the JobScheduler Agent listens to all available network interfaces via http.
- Without setting this environment variable the port defaults to 4445.
JS7_AGENT_HTTPS_PORT
- sets the https port that the JobScheduler Agent is listening to.
- indicates which network interfaces the JobScheduler Agent should listen to if a host or ip address specified.
- if only a port number is specified then the JobScheduler Agent listens to all available network interfaces via https.
- Without setting this environment variable the https protocol doesn't use.
JS7_AGENT_USER
- sets the user account that the JobScheduler Agent is operated for. This includes running jobs with the permissions of the specified user.
- This setting is available for Unix systems only. For Windows systems the user account that runs the start script is used.
- Without setting this environment variable the user account that runs the start script is used.
- This setting can be used when running the Agent start script in system start-up and shutdown configurations that are executed by
root
, e.g. in /etc/init.d
or corresponding locations.
JS7_AGENT_LOG_DIR
- sets the directory where the JobScheduler Agent log file is created.
- This setting defaults to the directory
logs
in the Agent installation directory. - For Windows systems for which the Agent is installed in the program directory that is pointed to by the
%ProgramFiles%
environment variable it is recommended not to use the default setting. Instead specify a different path via the SCHEDULER_LOG_DIR
environment variable, e.g. some location in the data directory that is pointed to by the %ProgramData%
environment variable.
JS7_AGENT_WORK_DIR
- sets the working directory for the jobs started by the JobScheduler Agent, e.g. ${HOME} or %USERPROFILE%.
- This setting defaults to the SCHEDULER_HOME.
JS7_AGENT_KILL_SCRIPT
JS7_AGENT_PID_FILE_DIR
- sets the directory where the JobScheduler Agent pid file is created.
- This setting defaults to the directory that is specified with the
JS7_AGENT_LOG_DIR
environment variable or the log directory default value.
Usage Examples
Running the Agent on Windows
For Windows® operating systems the location of the Java Runtime Environment and of the log directory can be specified like this:
Code Block |
---|
|
set JAVA_HOME=%ProgramFiles%\Java\jre8
set SCHEDULER_LOG_DIR=%ProgramData%\sos-berlin.com\js7\agent\agent_2.0\logs
"%ProgramFiles%\sos-berlin.com\agent\js7\agent_2.0\bin\agent.cmd" start |
Running the Agent on Mac OS X
For Mac® OS X the location of the Java Runtime Environment can be specified like this:
Code Block |
---|
|
JAVA_HOME=/Library/Internet\ Plug-Ins/JavaAppletPlugin.plugin/Contents/Home
export JAVA_HOME
/Users/ap/Documents/js7/agent/bin/agent.sh start |
Automated Start-up and Shutdown of an Agent
- For Unix systems the start-up and shutdown configurations apply that are executed by
root
, e.g. in /etc/init.d
or corresponding locations.- Consider use of the
JS7_USER
environment variable to run an Agent that is started by root
for a different user account.
- For Windows systems the start-up of the Agent by installing it as a Windows Service is recommended.
Debugging
- The Agent log level can be increased using the Agent's Apache ProcRun Demon Service Manager demon/service.
- On Windows systems this is installed in the Agent
service
Folder and will have a name such as sos_agent_4444w.exe
where 4444
is the port the agent can be addressed over. - Start the ProcRun Manager, select the Logging tab in the Manager interface and set the level to Debug.
- The location of the log files has already been described above.
- (Do not forget to set the debug level back to Info once finished.)
Anchor |
---|
| running_multiple_instances |
---|
| running_multiple_instances |
---|
|
Running multiple instances of the Agent- Multiple instances of the JS7 Agent on the same computer can be operated, e.g. for different user accounts that jobs should be executed for.
- Consider detailed instructions with the JobScheduler Universal Agent - Running multiple instances article.
Show If |
---|
|
Examples for Windows Code Block |
---|
language | bash |
---|
title | Set port by commands and options |
---|
| set JAVA_HOME=%ProgramFiles%\Java\jre8
"%ProgramFiles%\sos-berlin.com\js7\agent\agent_2.0\bin\agent.cmd" start -http-port=4447 |
Code Block |
---|
language | bash |
---|
title | Set log directory and port by environment variables |
---|
| set JAVA_HOME=%ProgramFiles%\Java\jre8
set JS7_LOG_DIR=%ProgramData%\sos-berlin.com\agent\jobscheduler_agent_1.10\logs
set JS7_HTTP_PORT=4445
"%ProgramFiles%\sos-berlin.com\js7\agent\agent_2.0\bin\agent.cmd" start |
Examples for Unix Code Block |
---|
language | bash |
---|
title | Switch user account and port by commands and options |
---|
| su - js
agent.sh start -http-port=4446 |
Code Block |
---|
language | bash |
---|
title | Switch user account and port by environment variables |
---|
| SCHEDULER_USER=js
SCHEDULER_HTTP_PORT=4446
export SCHEDULER_USER SCHEDULER_HTTP_PORT
jobscheduler_agent.sh start |
|
Testing the JobScheduler Universal Agent Operability
A simple way to test if the installed Agent works as expected is to carry out a test using Process Classes. The Process Class and the corresponding job will be defined in your JobScheduler Master, which will connect to the JobScheduler Agent. SOS recommends to do the test locally to exclude connection problems, e.g. firewall settings, since the goal of the test is to see whether the JobScheduler Agent is working properly.
Process Class and Standalone Job
First create a Process Class, for example the following (find attached example: agent1.process_class.xml):
...
Find more details and use cases from the article: How to execute Jobs and Job Chains with Agents
Logs
The easiest way to check if the test was successful is to verify the contents of the job log in JOC and to see if the job completed successfully. Otherwise an error will be reported in JOC.
...
Code Block |
---|
language | bash |
---|
title | JobScheduler Universal Agent Log |
---|
|
2020-04-28 16:11:24.462 +0200 [DEBUG] akka.io.TcpListener - New connection accepted
......
......
2020-04-28 16:13:39.231 +0200 [DEBUG] spray.can.server.HttpServerConnection - TcpConnection terminated, stopping |
Behavior in the event of the Universal Agent crashing
It is important that all the tasks running on an Agent are killed if the Agent should crash or otherwise terminate abnormally while executing tasks. To this end, every task that is being executed by an Agent is noted by a agent_watchdog.sh agent watchdog process in a kill_tasks_after_crash.sh
script. This script is located in the Agent's tmp
folder and tasks are dynamically added to and deleted from the script as they are started and completed. This script is dynamically created when a first task is started by the Agent and is deleted when no tasks are running.
If an Agent crashes then the Agent Watchdog will start this script which will then cause all tasks to be killed that were running when the Agent crashed.
See also
...