Product Knowledge Base

Space shortcuts

Page tree

Scope

  • Reference test cases provide an idea about the achievable performance for a given use case.
  • They are not intended to guarantee absolute reference values as such values depend on the system environment and resources. Instead they provide information on the ratio of reference values when using different optimization settings.
  • Consider the article Performance Optimization for API jobs when applying optimizations to your environment.

Test Suites

  • Three test suites are provided:
    • Test Suite 1: Zero Optimization
      • Optimization: Jobs are configured without optimization, i.e. no tasks are pre-loaded or re-used
      • Configuration: <job tasks="1"/>
    • Test Suite 2: Full Optimization
      • Optimization: Jobs are fully optimized, i.e. tasks are pre-loaded and re-used for processing of parallel orders
      • Configuration: <job tasks="3" min_tasks="3"/>
    • Test Suite 3: Balanced Optimization
      • Optimization: Jobs are medium optimized, i.e. tasks are not pre-loaded but re-used
      • Configuration: <job tasks="3" idle_timeout="30"/>
  • Recommendation
    • Try to use balanced optimization for your environment. 
      • There is only a small delay when initially loading tasks for the first order of a job chain.
      • When running a medium or high number of orders then the performance is approx. equivalent to full optimization.
      • This optimization respects use of resources (memory) by unloading tasks that are not used for a longer time (idle timeout).
    • For high performance environments use the full optimization scenario.
      • Guarantee that resources (memory) are available to pre-load and to retain the required number of tasks for parallel orders.
      • Should available memory be exhausted then this scenario will create extremely poor performance due to memory swapping.

Test Objects

  • Download
    • Download Jobs and Job Chains: performance.zip
    • Download Java Job: test.jar
      • The Java archive is expected in the ./lib/user_lib folder of a JobScheduler installation
      • The Java Job implements a single line of logging by use of spooler_log.info()
  • Quantity 
    • 400 jobs
    • 40 job chains with 10 job nodes each
  • Usage
    • 120 jobs and 12 job chains are used for testing
      • 10 job chains for test suite 1
      • 1 job chain for test suite 2
      • 1 job chain for test suite 3
    • 280 jobs and 28 job chains are not actively used for testing. Their purpose is to create some basic load for JobScheduler as each object has to be retained in memory.

Test Cases

  • Prerequisites
    • All job chains make use of unique jobs, i.e. no jobs are shared by multiple job chains
    • All jobs use the same Java class
  • Test Cases
    • Test Case 1:
      • 1 order executed in 1 job chain with 10 job nodes
      • This forces sequential processing without the possibility to re-use a task.
    • Test Case 2:
      • 3 orders executed in parallel in 1-3 job chains with 10 job nodes each
      • This allows parallel processing of 3 orders
      • For a zero optimization scenario 3 job chains are used, for other scenarios the same job chain is used to allow parallelism.
    • Test Case 3:
      • 10 orders executed in parallel in 1-10 job chains with 10 job nodes each
      • Same as test case 2 with a higher number of parallel orders.
    • Test Case 4:
      • 30 orders executed in parallel in 1 job chain with 10 job nodes
      • Same as test case 3 with a higher number of parallel orders.

Test Results

  • Download Document: jobscheduler_performance.xls
  • The applied metrics include 
    • the average number of tasks that are executed per minute in a given test case; for re-used tasks the number of task steps is used.
    • the absolute number of tasks that are executed during a test case, re-used tasks are counted as a single task.

 

Loading

 

  • No labels