Introduction
Dependencies between workflows include that JS7 - Notice Boards are used by a JS7 - PostNotices Instruction to create Notices that other workflows are waiting for using the JS7 - ExpectNotices Instruction and JS7 - ConsumeNotices Instruction.
- By default such dependencies remain in place for an arbitrary duration until they can be resolved.
- Example:
- Workflow A includes a PostNotices Instruction to create a Notice.
- Workflow B includes a ConsumeNotices Instruction to expect and to remove the Notice.
- Assume that Workflow A is delayed - possibly for days - and that no related Notice is created. This makes the order in Workflow B wait until the Notice arrives. Should the order in Workflow A be cancelled then the order in Workflow B will wait endlessly unless it is cancelled too.
For some users this is the desired behavior, for some it is not: users handle dependencies in the JS7 - Daily Plan individually.
- Some users want dependencies to remain in place until they can be resolved.
- Example:
- Assume Workflow A to load reporting data to a Data Warehouse on a daily basis. Assume Workflow B to run a report from the Data Warehouse.
- If reporting data are sequential and build upon each other then the order in Workflow B has to wait until reporting data are present before creating a report.
- In this scenario users want Workflow B to be delayed for hours or days until data from Workflow A are ready as the sequential nature of data deliveries forces this behavior.
- Example:
- Some users want dependencies to be dropped after a certain period.
- Example:
- Assume a scenario of classic batch processing when Workflow A loads data to a database that are processed by Workflow B with both workflows running on a daily basis.
- If Workflow A is late within a day then the order for Workflow B should wait. However, if Workflow A exceeds the 24 hours' period of the Daily Plan it should be cancelled as the same Workflow A is executed by a new order on the next day. In this scenario Workflow B should not wait longer for Workflow A than the Daily Plan's period.
- Example:
The following explanations focus on the scenario to clean up the Daily Plan from past dependencies of a previous day.
Cleanup
Cleanup includes two operations for which script solutions are available:
- Drop Orders scheduled for past Daily Plan dates.
- Drop Notices created for past Daily Plan dates.
Both operations to clean up the Daily Plan can be combined in a workflow for automated execution on a daily basis.
Workflow
The workflow implements cleanup from two shell jobs introduced with the above articles.
- cancel-orders.sh
- delete-notices.sh
Download (upload .json): pdwCleanupDailyPlan.workflow.json
Explanation:
- The workflow makes use of the
Default
job resource, see JS7 - Job Environment Variables and JS7 - Job Resources.- It is not required to use the job resource that is available from JS7 - Download.
- However, the job resource simplifies parameterization of the job.
- Error handling includes to use a JS7 - Try-Catch Instruction that will continue processing with the Catch Block should one of the jobs in the Try Block fail.
Job: Cancel-Orders
The job makes use of the cancel-order.sh
shell script to cancel orders waiting for past notices from JS7 - Notice Boards.
For details see JS7 - How to cancel orders using the REST Web Service API from the Shell.
Explanations:
- Parameterization of
cancel-orders.sh
includes-url="${JS7_JOC_URL}
: The JOC Cockpit URL is provided from an environment variable that users can specify with theDefault
job resource.-user=root
,--password=root
: The example makes use of user/password authentication.Anumberofwaysareoffered- x
Job: Delete Notices
x
Further Resources
- JS7 - How to cancel orders using the REST Web Service API from the Shell
- JS7 - How to delete notices using the REST Web Service API from the Shell