| Table of Contents |
|---|
Scope
- API jobs are job that make use of the JobScheduler API Interface
- Such jobs should can be optimized considering the requirements and capabilities of the specific environment that they are operated for.
- This article explains the impact of the quantity structure, resource consumption and parallelism on the overall performance.
- The explanations in this article are intended to make users understand the factors that impact peformance and possible measures. Performance optimization for individual environments is part of our the SOS Consulting Services.
Performance Goals
- Performance is about effective use of resources such as memory and CPU. Resources are shared across jobs. Therefore , therefore we can speed up the execution of certain processes but would will have to accept a performance degradation for other processes.
- Performance optimization requires a clear goal to be achieved. There is no such thing as an "overall speed improvement", instead performance improvements are oriented towards balanced use of resources for specific use cases.
...
- For starters let's define the exact meaning when using the following terms:
- low number: 1 - 1000 objects
- medium number: 1000 - 4000 objects
- high number: 4000 - 10000 objects
- Then consider the impact of a JobScheduler single instance environment and a distributed environment:
- In a single instance environment all job related objects are managed and executed within the same instance ( JobScheduler Master )instance.
- In a distributed environment all job related objects are managed by the same instance ( JobScheduler Master )instance, but are executed on in distributed JobScheduler Agent instances (JobScheduler Agents) at run-time.
Quantity Structure
...
- Number of jobs, job chains and orders
- This is about the number of job-related objects that are available in the system, independently from the fact that they are running or not.
- JobScheduler has to track events for jobs, e.g. when to start and to stop jobs. Therefore a high number of job related objects causes a creates some performance impact. Some commonly used Common scenarios used in enterprise level environments include up to 15000 20000 jobs and 5000 10000 job chains in a single JobScheduler instance.
- Number of job nodes
- This is about the number of jobs that are used in job nodes for job chains.
- Jobs can be re-used for any number of job chains.
- You could operate e.g. 1000 job chains with each using 5 job nodes with individual jobs which results in a total of 5000 jobs.
- You could operate e.g. 100 different jobs that are
organized - used in 1000 job chains each using a different sequence of 5 jobs.
- The length of a job chain, i.e. the number of job nodes, is important:
- In
common environment - average scenarios job chains with up to 30 job nodes are used.
- You can operate a job chain with a high number of e.g. 4000 job nodes. In fact this will have a slight effect on performance as JobScheduler has to check predecessor and successor job nodes.
Resource Consumption
Consider what resources are consumed when operating jobs:
- Resources used per job
- Depending on the nature of your job this job will consume
- an instance of a JVM
- memory and CPU as required by your job implementation
- Depending on the nature of your job this job will consume
- Total of running jobs
- This number has less impact on JobScheduler than you would expect, but it affects the available resources like memory and CPU.
- Consider the information from the article How to determine the sizing of a JobScheduler environment for memory and CPU
. - consumption.
- A common observation is that a system behaves performant as long it's capacity is not used up. Exceeding e.g. the memory limit of a server will result in the operating system swapping the memory and degrading the performance in an unacceptable way.
Parallelism
- x
Measures for Performance Optimization
...