Introduction

• Sizing is an important step when planning the installation and operation of JS7.
• The following guidelines are recommendations (best practices) that require to be mapped to the requirements of your environment.
• SOS recommends deciding on the desired target architecture in a first step and then determining the sizing accordingly.

Architecture

• First of all, a decision about the architecture has to be taken:
  • Use of JOC Cockpit and Controller standalone instances
  • Use of JOC Cockpit and Controller high-availability cluster instances
  • For details see the JS7 - System Architecture article.
• Then the scheduling mode should be identified:
  • Use of JS7 Agents
  • Use of JS7 - Agentless Scheduling

Scalability

The JS7 adjusts to both vertical and horizontal scaling, see the JS7 - Performance article:

• Vertical Scalability
  • There are no built-in limits as to the overall number of jobs and number of parallel job executions per Controller and Agent.
  • A single Controller is tested for some 50 000 parallel tasks, a single Agent is tested for approx. 15 000 parallel tasks.
  • Adding more power to a single machine (CPU, Memory, SSD Disks) will improve performance as the products will make use of all available cores.
• Horizontal Scalability
  • Splitting the job load across a number of Agents will push the number of parallel tasks.

Users have a choice to size for vertical or horizontal scalability.

Sizing

Server Resources

• The following indicators should be considered:
  • The overall number of workflows and jobs that will be available.
  • The number of jobs that will be executed in parallel.
• Number of Jobs
  • Small environments are assumed to use < 1 000 jobs, medium sized environments use < 20 000 jobs, larger environments use larger numbers.
• Parallelism of Jobs
  • Small environments run < 100 parallel jobs, medium sized environments run < 1 000 parallel jobs, larger environments run larger numbers of jobs.

Basic Recommendation

The recommendation indicates dedicated resources available to JS7 products. Resource usage by additional software running on a machine come on top.

Overall number of jobs

• Impact
  • JS7 is designed to operate up to 1 000 000 overall job objects. However, to share duties users might split the load across multiple JS7 instances.
  • A high number of orders, workflows and jobs affects the responsiveness of the JOC Cockpit GUI in displaying the objects in question.
• Recommendations
  • For small and medium sized environments no specific measures are required.
  • For larger environments a number of measures are applied that are tailored to the user's needs. SOS provides Consulting Services to assist in designing workflows and jobs for performance.

CPU

• Impact
  • The JOC Cockpit will use a small amount of CPU during idle times due to the fact that a number of background processes are running, see the JS7 - Services article. When load is increased by parallel users or by parallel use of the JS7 - REST Web Service API then more threads will be used that map to processes and cores.
  • The Controller and Agent will hardly use any CPU power during idle times. The Controller and Agent handle thread pools in way that balances CPU consumption across all available cores.
• Recommendations
  • Do not use a single core CPU system. Situations could occur when individual jobs might misbehave and consume more CPU than expected. A multi-core CPU system means that process resources will be available in a situation when a single core CPU is blocked. The more cores that are available the better the performance results will be.

Memory

• Impact
  • The number of parallel orders and jobs increases memory consumption of an active Controller instance and the relevant Agents.
    • The JOC Cockpit can be operated from 512 MB heap size. This value should be increased if more users are working in parallel with JOC Cockpit or if a larger number of orders are available in the JS7 - Daily Plan.
    • The Controller can be operated starting from 256 MB memory. Higher parallelism of orders and jobs will require higher values, e.g. for 100 000 orders approx. 4 GB will be required.
    • The Agent has a footprint of about 100 MB memory. As with the Controller higher parallelism of orders and jobs will push this value. As a rule of thumb 2 GB main memory should be sufficient to execute 5 000 jobs in parallel on an Agent.
  • Refer to the JS7 - FAQ - Which Java Options are recommended article for default values and details how to specify memory consumption.
• Recommendations
  • Calculate 500 MB for JOC Cockpit for 5 to 10 parallel users. Calculate 2 GB each for a Controller and Agent if < 50 000 task executions per day and < 5000 parallel task executions are performed. You can reduce the values for Controller and Agent to 500 MB if < 10 000 task executions per day and < 1000 parallel task executions are performed.

Initial Installation

• File System
  • The size of installed files should not exceed approx. 250 MB for each JOC Cockpit, Controller and Agent.
• Database
  • The database initially can be used with 200 MB tablespace.

Ongoing Operation

File System

• Journal files
  • Journal files are used by Controller instances and Agent instances to store information about state transitions and log output. Typically such information is immediately forwarded from an Agent to the Controller and from the Controller to JOC Cockpit. However, in case of high load or outage of a product such as a Controller, then the Agent will continue to execute jobs and store such information in its journal. Journal files therefore can grow, technically to the max. size imposed by the OS. Journal files will shrink after JS7 products are recoupled. For most situations with <100 000 job executions per day and an outage period < 24h it is sufficient to assume 5 GB diskspace for journal files. 
• Log files
  • Log files are by default configured for log rotation to limit diskspace consumption and to restrict the number of files in use, see the JS7 - Log Rotation article for more information.
    
    • All log files are controlled by Log4j2 and allow individual configuration.
    • By default diskspace consumption is limited to 5 GB and log files are retained for 30 days. This applies to each JOC Cockpit, Controller and Agent.
  • Recommendations
    • It is suggested that not less than 1 GB disk space is assigned for logs. Use of more than 5 GB disk space should be required only if the retention period for log files is prolonged.
    • When JS7 products cannot write to a log file then they will be blocked from further operation. To prevent such a situation add a disk space monitoring task to your system monitor.

Database

• Shared/Dedicated Database Server
  • JS7 does not require a dedicated Database Server. At the same time in a shared scenario users should be aware of the impact of the Database Server being subject to high load.
  • If the JS7 faces insufficient database performance then this will slow down the performance of the JOC Cockpit, preferably the JS7 - History might not be able to report job execution results and log output in good time. Practically speaking, if the job execution history is typically available within 1-2s after completion of a job then in the event of high job execution load and poor database performance this can take longer, maybe minutes. However, the data will not be lost but will be added with a delay.
•  Memory
  • Some DBMS allow tables to be stored in memory. This can improve the overall performance for frequently used tables. 
  • Recommendation
    • SOS recommends leaving this subject to your DBA. This is about database optimization and will result in a noticeable effect only in high performance environments.
• Tablespace
  • JS7
    • stores the job execution history in the database,
    • stores the task logs in a zipped format in the database,
    • can be configured to automatically purge the database after configurable object retention periods: see the JS7 - Cleanup Service article.
  • Recommendations
    • Calculate the increase in tablespace size based on the overall number of job executions. Assume 2 KB for job history and task log for each job execution. Add a factor for individual log output of scheduled scripts and programs.
    • A practical approach includes closely monitoring tablespace growth during the first few months of operation and then adjusting the configuration of the Cleanup Service to purge for shorter or longer retention periods.
    • As a rule of thumb do not start JS7 operation with a tablespace <200 MB.

Further Resources

• JS7 - Performance
• JS7 - FAQ - Which Java Options are recommended
• JS7 - How To - Apply Java Options