Product Knowledge Base

Space shortcuts

Page tree

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 12 Next »

Assume you want to create a job chain but some parts of the chain should run in parallel. You want to define all nodes in one chain.

  • Please define some nodes at the beginning of the chain
  • When you want to parallize the execution add a node with the splitter job
  • Define the next node for the splitter. This could be the sync node or any other node in the chain.
  • Define the number of orders that should run in parallel. Also define the node where these order should start. Use a list like 100;200;300. That means that you will get 3 additional orders all running in paralell.
  • Define the sync node with the JITL sync job. All orders must reach this node. When all orders are in this node, the execution proceeds with the original order. The new orders will end in the sync.
  • please note that every parallel processing needs an own unique sync job.

You can download an example from here.Splitter.zip

     *****under construction*****

Best Practices for job chains with parallel processing

Example: Multiple parallel processing in a single job chain
*
Beispiel: Wir wollen eine Jobkette erstellen die zunächst 6 insert table jobs parallel ausführt. Wenn alle 6 Jobs fertig sind soll ein create Index Job gestartet werden. Danach sollen mehrere test Jobs über die erstellten Daten laufen. Die tes Jobs sollen wieder parallel ausgeführt werden.

Case: We like to create a job chain which runs six insert table jobs in parallel.
After processing those 6 jobs we want to process a create index job.
Then we want to run several test jobs on the data. The processing of the test jobs should be parallel.
*
Diagram der Jobkette (erstellt mit einer experimentellen JOE-Version, die zukünftig ausgeliefert wird).

Diagram created with JOE (JobScheduler object editor - this is a sandbox, coming with one of the next versions)


*
Das nächste Bild zeigt eine Liste der Jobs der Jobkette in der Darstellung in JOE

This picture shows the jobs of the chain displayed by JOE Steps/Nodes

    • Verwende für jede parallele Verarbeitung einen split Job einen dazugehörenden eindeutigen sync Job.
      Dieser sync Job darf in einer JobScheduler Instanz in keiner anderen Jobkette verwendet werden.
      Um den sync Job eindeutig zu halten empfehlen wir den Namen der Jobkette als Prefix in dem Jobnamen des sync Jobs zu verwenden.
      Beispiel: insert_to_core_parallel.sync1
  • Für die Namensgebung der State Namen für die parallel zu verarbeitenden Knoten empfehlen wir die Syntax state Name Splitter : state Name Job
    Beispiel: split:partition_1
    • Use a split job to start parallel processing with a JS instance. There has to be a corresponding sync job for each split job. This sync job has to be unique (i.e. have a unique name) and cannot be used anywhere else in the JS instance.
  • Use an unique sync job for each parallel processing within one JobScheduler instance. One sync job evaluates the information from all splitter jobs where this sync job is used.
  • To identify the sync jobs in JOE job list and to make sure to use a unique sync job name we strongly recommend to combine the synchronization job name with the name of the job chain using the sync job.
  • We recommend to combine the parameter state_names in the splitter job as node name of splitter:jobname

Display detail from JOE showing the parameters for the first splitter job split

Display detail from JOE showing the parameters for the second splitter job split_4_test

we also recommend to use the startjob /sos/jitl/JobChainStart and the endjob /sos/jitl/JobChainEnd in {}every{*} job chain.

You can download the example from here.MultiParallel.zip

  • No labels