JS7 JobScheduler

Space shortcuts

Page tree

Versions Compared

Old Version 13

changes.mady.by.user Andreas Püschel

Saved on

compared with

New Version 14

changes.mady.by.user Andreas Püschel

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

Info
under construction!!!

Introduction

  • JS7 JOC Cockpit can be configured to connect to Git repositories to store its configuration objects such as workflows and jobs.
  • Configuration objects can be versioned (tagged) and can be rolled out to different JS7 instances via Git. This allows to create and to test workflows in a development environment and to perform a rollout to a production environment by use of a Git repository.
  • JOC Cockpit is the leading system to store configuration objects in its database. A connection to a Git repository is considered an additional option to manage versions and rollouts of JS7 configuration objects.

Object Mappings

Configuration Objects

Configuration JS7 configuration objects that can be stored in added to a Git repository include

  • Workflows
  • Locks
  • File Order SourcesJob Resources
  • Notice Boards
  • Calendars
  • Schedules

Settings

Settings are properties for JOC Cockpit to work properly. The configuration items for JOC Cockpit to work with a Git repository include

  • git_repository_url
    • URL of the remote repository to connect to
  • git_account
    • account that it used to connect to Git
  • git_key_file
    • file name of the private key file
  • git_access_token
    • access token for Git access as an alternative to use of a private key file

The JOC Cockpit user menu offers the "Settings" menu item that displays the settings group git:

Status
colourYellow
titleTODO: INSERT SCREENSHOT

Git Repository Usage

The JS7 - REST Web Service API executes a set of Git operations by use of the JGit implementation.

Management

JS7 configuration objects are managed in a Git repository via the JS7 REST Web Service API.

The following operations are available:

  • git-add
    • adds (new) configuration objects from a working copy to a local repository
  • git-delete
    • removes an already added configuration object from the local repository
  • git-commit
    • commits newly added, updated or removed configuration objects to a local repository
  • git-push
    • pushes changes from the local repository to the remote repository

Versioning

Versioning of JS7 configuration objects is performed by use of the tagging capabilities of Git. 

The JS7 REST Web Service API manages the Git tagging process.

The following operations are available:

  • git-tag
    • tags the current commit with a version label
  • git-push
    • see above

Rollout

Rollout between different JS7 instances can be performed by use of the Git functionality.

The JS7 REST Web Service API manages the processes for checkout and cloning.

The following operations are available:

  • git-checkout
    • checks out a specific branch or tag from the repository
  • git-clone
    • initially clones a repository
  • git-pull
    • pulls the recent changes from the remote repository to the local repository

Force Overwrite

Forcefully overwrites a target configuration with the source configuration in use.

The JS7 REST Web Service API manages to forcefully overwrite configuration objects in the Git repository.

The following operations are available depending on the force direction:

  • JS7 JOC Cockpit instance => Git repository
    • git-add
      • adds (new) configuration objects from a working copy to a local repository
    • git-delete
      • removes an already added configuration object from the local repository
    • git-commit
      • commits newly added, changed or removed configuration objects to a local repository
    • git-push
      • pushes the changes from the local repository to the connected remote repository
  • Git repository => JS7 JOC Cockpit instance
    • git-checkout
      • checks out a specific branch or tag from the repository
    • git-clone
      • initially clones a repository
    • git-pull
      • pulls recent changes from the remote repository to the local repository
Note
  • If merging of configuration objects between different Git repositories or between different branches of a single GIT repository is required then additional tools such as Git Extensions can be used.

Use Cases

Single Client - Single Branch

Scenario

  • The Git repository is used to hold a copy of JS7 configuration objects.
  • Additionally, the Git repository is used to version JS7 configuration objects.

Such objects can be managed with JS7 in a way that allows to re-use the objects - without any changes - across rollout environments such as dev, test, prod.

The following configuration objects typically hold configuration items that can be specific for an individual rollout environment and therefore optionally can be excluded from a Git repository.

  • Job Resources
  • Calendars
  • Schedules

Configuration Mappings

The JS7 inventory is organized by folders.

  • Folders hold configuration objects such as workflows etc. and any level of sub-folders.
  • Folders are the basic unit for JS7 - Permissions, for example, users can be granted access to specific folders only.
  • Folders allow to perform bulk operations such as to deploy, to release, to remove and to rename configuration objects.

Folders can be mapped to Git repositories managed by the same or by different Git servers like this:


Flowchart
USER1 [label="   User: 1   ",fillcolor="lightskyblue"]
USER2 [label="   User: 2   ",fillcolor="lightskyblue"]
USER3 [label="   User: 3   ",fillcolor="lightskyblue"]

FOLDER1 [label="   Folder: 1   ",fillcolor="white"]
FOLDER1a [label="   Sub-folder: 1.1   ",fillcolor="white"]
FOLDER1b [label="   Sub-folder: 1.2   ",fillcolor="white"]
FOLDER2 [label="   Folder: 2   ",fillcolor="white"]
FOLDER3 [label="   Folder: 3   ",fillcolor="white"]

REPOSITORY1 [label="   Git Repository: 1   ",fillcolor="orange"]
REPOSITORY2 [label="   Git Repository: 2   ",fillcolor="orange"]
REPOSITORY3 [label="   Git Repository: 3   ",fillcolor="orange"]

USER1 -> FOLDER1 [label="  can access  "]
USER2 -> FOLDER1 [label="  can access  "]
USER2 -> FOLDER2 [label="  can access  "]
USER3 -> FOLDER3 [label="  can access  "]

FOLDER1 -> FOLDER1a [label="  includes  "]
FOLDER1 -> FOLDER1b [label="  includes  "]
FOLDER1a -> REPOSITORY1
Flowchart
USER1a [label="   User: 1   ",fillcolor="lightskyblue"]
USER1b [label="   User: 2   ",fillcolor="lightskyblue"]
USER1c [label="   User: 3   ",fillcolor="lightskyblue"]
JOC1 [label="   JOC Cockpit maps to  ",fillcolor="lightskyblue"]
DATABASEFOLDER1b -> REPOSITORY1 [shape="ellipse",label="     Database     ",fillcolor="lightskyblue"]
REPO  maps to  "]

FOLDER2 -> REPOSITORY2 [label="    Git Repository    ",fillcolor="orange"]
BRANCH1 [shape="ellipse",label="    Single Branch    ",fillcolor="orange"]

JOC1  maps to  "]
FOLDER3 -> BRANCH1REPOSITORY3 [label="  maps to Commit/Push\nPull   "]
# BRANCH1 -> JOC1 [label="   Pull   "]
BRANCH1 -> REPO

JOC1 -> DATABASE

USER1a -> JOC1
USER1b -> JOC1
USER1c -> JOC1

Repository Operations

  • management
  • versioning
  • rollout

Requirements

  • The JOC Cockpit database is the leading system holding the configuration objects.

Single Client - Multiple Branches (dev, test, prod)

Scenario

  • The Git repository is used to hold a copies of JS7 configuration objects in branches.
  • Additionally, the Git repository is used to version JS7 configuration objects.
  • The Git repository is used to merge branches and to rollout JS7 configuration objects to different JS7 instances, such as Dev => Test => Prod.
Flowchart
USER1a [label="   User: 1a   ",fillcolor="lightskyblue"]
USER1b [label="   User: 1b   ",fillcolor="lightskyblue"]
USER2 [label="   User: 2   ",fillcolor="lightskyblue"]
USER3 [label="   User: 3   ",fillcolor="lightskyblue"]

JOC1 [label="   JOC Cockpit: Dev   ",fillcolor="lightskyblue"]
JOC2 [label="   JOC Cockpit: Test   ",fillcolor="lightskyblue"]
JOC3 [label="   JOC Cockpit: Prod   ",fillcolor="lightskyblue"]

DATABASE1 [shape="ellipse",label="     Database: Dev     ",fillcolor="lightskyblue"]
DATABASE2 [shape="ellipse",label="     Database: Test     ",fillcolor="lightskyblue"]
DATABASE3 [shape="ellipse",label="     Database: Prod     ",fillcolor="lightskyblue"]
REPO [label="    Git Repository    ",fillcolor="orange"]

BRANCH1 [shape="ellipse",label="    Branch: Dev    ",fillcolor="orange"]
BRANCH2 [shape="ellipse",label="    Branch: Test    ",fillcolor="orange"]
BRANCH3 [shape="ellipse",label="    Branch: Prod    ",fillcolor="orange"]

JOC1 -> DATABASE1
JOC2 -> DATABASE2
JOC3 -> DATABASE3

JOC1 -> BRANCH1 [label="   Commit/Push\nPull   "]
JOC2 -> BRANCH2 [label="   Commit/Push\nPull   "]
JOC3 -> BRANCH3 [label="   Commit/Push\nPull   "]

BRANCH1 -> BRANCH2 [label="  merge   "] 
BRANCH2 -> BRANCH3 [label="   merge   "]

BRANCH1 -> REPO
BRANCH2 -> REPO
BRANCH3 -> REPO

USER1a -> JOC1
USER1b -> JOC1
USER2 -> JOC2
USER3 -> JOC3

Repository Operations

  • management
  • versioning
  • rollout
  • force overwrite

Requirements

  • The JOC Cockpit database holds the configuration objects:
    • each JOC Cockpit instance is connected to its own database.
    • each branch is connected to an individual JOC Cockpit instance.
    • the database can be updated from a branch.
  • each branch makes use of its own version of configuration objects.
  • force overwrite is possible.

Multiple Clients - Multiple Branches (dev, test, prod)

Scenario

  • The Git repository is used to hold a copy of JS7 configuration objects.
  • Additionally, the Git repository is used to version JS7 configuration objects.
  • The Git repository is used to merge branches and to rollout JS7 configuration objects to different JS7 instances such as Dev => Test => Prod.
"]


The mapping of folder objects to repositories implies

  • user access:
    • users with JS7 access permissions to manage folders and to access the Git repository can manage configuration objects in JOC Cockpit and can commit, push, pull configuration objects to/from Git,
    • users with JS7 access permissions to manage folders but without access to the Git repository can manage configuration objects in JOC Cockpit,
    • users with JS7 access permissions to view folders do not require Git access.
  • folder usage:
    • if a single Git repository should be used then all folders have to be available from a single root folder,
    • for use with a number of Git repositories each folder maps to a different repository,
    • folders can remain without Git integration for local management and deployment.

JOC Cockpit maps folders to Git repositories and stores the respective Git repository URL for each folder in its inventory.

Git Access

User Access

Each user holds an individual set of credentials to connect to a Git server.

The following settings are stored with a user's profile:

  • git_account
    • account that is used to connect to a Git server
  • git_key
    • the user's private key for Git access
  • git_access_token
    • the access token for Git access that works as an alternative to use of a private key

If repositories are used from a number of Git servers that require different sets of user credentials then separate user accounts in JOC Cockpit have to be used.

Repository Access

  • The JS7 - REST Web Service API executes a set of Git operations by use of the JGit implementation.
  • Any operations to add configuration objects to a repository, to commit, push, pull and merge objects are available from the JS7 REST Web Service API.
  • The JOC Cockpit offers the user interface for respective Git operations.
  • Users are free to use additional tools such as Git Extensions to manage repositories and branches.

Git Operations

Management

JS7 configuration objects are managed in a Git repository via the JS7 REST Web Service API.

The following operations are available:

  • git-add
    • adds (new) configuration objects from a working copy to a local repository
  • git-delete
    • removes an already added configuration object from the local repository
  • git-commit
    • commits newly added, updated or removed configuration objects to a local repository
  • git-push
    • pushes changes from the local repository to the remote repository

Versioning

Versioning of JS7 configuration objects is performed by use of the tagging capabilities of Git. 

The JS7 REST Web Service API manages the Git tagging process.

The following operations are available:

  • git-tag
    • tags the current commit with a version label
  • git-push
    • see above

Rollout

Rollout between different JS7 instances can be performed by use of the Git functionality.

The JS7 REST Web Service API manages the processes for checkout and cloning.

The following operations are available:

  • git-checkout
    • checks out a specific branch or tag from the repository
  • git-clone
    • initially clones a repository
  • git-pull
    • pulls the recent changes from the remote repository to the local repository

Force Overwrite

Forcefully overwrites a target configuration with the source configuration in use.

The JS7 REST Web Service API manages to forcefully overwrite configuration objects in the Git repository.

The following operations are available depending on the force direction:

  • JS7 JOC Cockpit instance => Git repository
    • git-add
      • adds (new) configuration objects from a working copy to a local repository
    • git-delete
      • removes an already added configuration object from the local repository
    • git-commit
      • commits newly added, changed or removed configuration objects to a local repository
    • git-push
      • pushes the changes from the local repository to the connected remote repository
  • Git repository => JS7 JOC Cockpit instance
    • git-checkout
      • checks out a specific branch or tag from the repository
    • git-clone
      • initially clones a repository
    • git-pull
      • pulls recent changes from the remote repository to the local repository


Info
titleNote

If merging of configuration objects between different Git repositories or between different branches of a single Git repository is required then additional tools such as Git Extensions can be used.

Use Cases

Single Client - Single Branch

Scenario

  • Architecture
    • A single JOC Cockpit instance is used with optionally different Controllers and Agents for rollout environments such as dev, test, prod.
    • The Git repository is used per folder to hold a copy of JS7 configuration objects and to create versions of JS7 configuration objects in a single branch.
  • Users
    • All users act in the same JOC Cockpit instance that is a single client to the Git server.
    • Changes to the Git repository are performed by the users' individual accounts to a single branch.
  • Rollout
    • Deployments are performed from a single source (JOC Cockpit) to a number of Controllers and Agents (dev, test, prod).
    • The Git repository can be used to fallback (pull) to previous versions of configuration objects, however, only one version is available with JOC Cockpit at a given point in time.


Flowchart
USER1a [label="   User: 1   ",fillcolor="lightskyblue"]
USER1b [label="   User: 2   ",fillcolor="lightskyblue"]
USER1c [label="   User: 3   ",fillcolor="lightskyblue"]
FOLDER1 [label="   Folder: 1   ",fillcolor="white"]
JOC1 [label="   JOC Cockpit   ",fillcolor="lightskyblue"]
DATABASE [shape="ellipse",label="     Database     ",fillcolor="lightskyblue"]
REPO [label="    Git Repository    ",fillcolor="orange"]
BRANCH1 [shape="ellipse",label="    Single Branch    ",fillcolor="orange"]

JOC1 -> FOLDER1 -> BRANCH1 [label="   Commit/Push\nPull   "]
# BRANCH1 -> JOC1 [label="   Pull   "]
BRANCH1 -> REPO

JOC1 -> DATABASE

USER1a -> JOC1
USER1b -> JOC1
USER1c -> JOC1

Repository Operations

  • management
  • versioning
  • rollout

Requirements

  • The JOC Cockpit database is the leading system holding the configuration objects.

Single Client - Multiple Branches (dev, test, prod)

Scenario

  • Architecture
    • A dedicated JOC Cockpit instance is used with each Controller and Agents for each rollout environment such as dev, test, prod.
    • The same Git repository is used per folder by all JOC Cockpit instances, however, each instance is acting on a specific branch per rollout environment.
    • The Git repository is used to hold a copy of JS7 configuration objects, to create versions of JS7 configuration objects and to merge branches such as dev => test, test => prod.
  • Users
    • All users act in the same JOC Cockpit instance of the respective rollout environment that is a single client to the Git server.
    • Changes to the Git repository are performed by the users' individual accounts to a each branch.
  • Rollout
    • Deployments are performed from a single source (JOC Cockpit) to the connected Controller and Agents per rollout environment (dev, test, prod).
    • The Git repository is used to merge branches and to rollout JS7 configuration objects to different Controllers and Agents, such as dev => test => prod.
    • The Git repository can be used to fallback (pull) to previous versions of configuration objects, however, only one version is available with a given JOC Cockpit instance at a given point in time.


Flowchart
USER1a [label="   User: 1a   ",fillcolor="lightskyblue"]
USER1b [label="   User: 1b   ",fillcolor="lightskyblue"]
USER2 [label="   User: 2   ",fillcolor="lightskyblue"]
USER3 [label="   User: 3   ",fillcolor="lightskyblue"]

JOC1 [label="   JOC Cockpit: dev   ",fillcolor="lightskyblue"]
JOC2 [label="   JOC Cockpit: test   ",fillcolor="lightskyblue"]
JOC3 [label="   JOC Cockpit: prod   ",fillcolor="lightskyblue"]

FOLDER1 [label="   Folder: 1 (dev)   ",fillcolor="white"]
FOLDER2 [label="   Folder: 1 (test)  ",fillcolor="white"]
FOLDER3 [label="   Folder: 1 (prod)   ",fillcolor="white"]

DATABASE1 [shape="ellipse",label="     Database: dev     ",fillcolor="lightskyblue"]
DATABASE2 [shape="ellipse",label="     Database: test     ",fillcolor="lightskyblue"]
DATABASE3 [shape="ellipse",label="     Database: prod     ",fillcolor="lightskyblue"]
REPO [label="    Git Repository    ",fillcolor="orange"]

BRANCH1 [shape="ellipse",label="    Branch: dev    ",fillcolor="orange"]
BRANCH2 [shape="ellipse",label="    Branch: test    ",fillcolor="orange"]
BRANCH3 [shape="ellipse",label="    Branch: prod    ",fillcolor="orange"]

JOC1 -> DATABASE1
JOC2 -> DATABASE2
JOC3 -> DATABASE3

JOC1 -> FOLDER1 -> BRANCH1 [label="   Commit/Push\nPull   "]
JOC2 -> FOLDER2 -> BRANCH2 [label="   Commit/Push\nPull   "]
JOC3 -> FOLDER3 -> BRANCH3 [label="   Commit/Push\nPull   "]

BRANCH1 -> BRANCH2 [label="  merge   "] 
BRANCH2 -> BRANCH3 [label="   merge   "]

BRANCH1 -> REPO
BRANCH2 -> REPO
BRANCH3 -> REPO

USER1a -> JOC1
USER1b -> JOC1
USER2 -> JOC2
USER3 -> JOC3

Repository Operations

  • management
  • versioning
  • rollout
  • force overwrite

Requirements

  • The JOC Cockpit database holds the configuration objects:
    • each JOC Cockpit instance is connected to its own database.
    • each Git branch is mapped to an individual JOC Cockpit instance.
    • the database can be updated from a Git branch.
  • each Git branch makes use of its own version of configuration objects.

Multiple Clients - Multiple Branches (dev, test, prod)

Scenario

  • Architecture
    • A dedicated JOC Cockpit instance is used with each Controller and Agents for each user in the dev rollout environment.
    • A dedicated JOC Cockpit instance is used with each Controller and Agents for all users in the test and prod rollout environments.
    • The same Git repository is used per folder by all JOC Cockpit instances, however, each instance is acting on a specific branch per rollout environment.
    • The Git repository is used to hold a copy of JS7 configuration objects, to create versions of JS7 configuration objects and to merge branches such as dev => test, test => prod.
  • Users
    • Users act in individual JOC Cockpit instances of the dev rollout environment as multiple clients to the Git server.
    • Users act in the same JOC Cockpit instance of the test and prod rollout environment as a single client to the Git server.
    • Changes to the Git repository are performed by the users' individual accounts to a each branch.
  • Rollout
    • Deployments are performed from a single source (JOC Cockpit) to the connected Controller and Agents per rollout environment (dev, test, prod).
    • The Git repository is used to merge branches and to rollout JS7 configuration objects to different Controllers and Agents, such as dev => test => prod.
    • The Git repository can be used to fallback (pull) to previous versions of configuration objects, however, only one version is available with a given JOC Cockpit instance at a given point in time.


Flowchart
USER1a [label="   User: 1a   ",fillcolor="lightskyblue"]
USER1b [label="   User: 1b   ",fillcolor="lightskyblue"]
# USER1c [label="   User: 1c   ",fillcolor="lightskyblue"]
USER2a [label="   User: 2a   ",fillcolor="lightskyblue"]
USER2b [label="   User: 2b   ",fillcolor="lightskyblue"]
USER3 [label="   User: 3   ",fillcolor="lightskyblue"]

JOC1a [label="   JOC Cockpit: dev 1a   ",fillcolor="lightskyblue"]
JOC1b [label="   JOC Cockpit: dev 1b   ",fillcolor="lightskyblue"]
# JOC1c [label="   JOC Cockpit: dev 1c   ",fillcolor="lightskyblue"]
JOC2  [label="   JOC Cockpit: test   ",fillcolor="lightskyblue"]
JOC3  [label="   JOC Cockpit: prod
Flowchart
USER1a [label="   User: 1a   ",fillcolor="lightskyblue"]
USER1b [label="   User: 1b   ",fillcolor="lightskyblue"]
USER1c [label="   User: 1c   ",fillcolor="lightskyblue"]
USER2 [label="   User: 2   ",fillcolor="lightskyblue"]
USER3 [label="   User: 3   ",fillcolor="lightskyblue"]

JOC1a# FOLDER1a [label="   JOC   Folder: Cockpit: Dev 1a  1a (dev)   ",fillcolor="lightskybluewhite"]
JOC1b# FOLDER1b [label="   JOC   Folder: Cockpit: Dev 1b  1b (dev)   ",fillcolor="lightskybluewhite"]
JOC1c# FOLDER1c [label="   JOC   Folder: Cockpit: Dev 1c  1c (dev)   ",fillcolor="lightskybluewhite"]
JOC2# FOLDER2  [label="   JOC Cockpit: Test      Folder: 1 (test)   ",fillcolor="lightskybluewhite"]
JOC3# FOLDER3  [label="   JOC Cockpit   Folder: Prod   1 (prod)   ",fillcolor="lightskybluewhite"]

DATABASE1a [shape="ellipse",label="      DatabaseDatabase: Devdev 1a      ",fillcolor="lightskyblue"]
DATABASE1b [shape="ellipse",label="      DatabaseDatabase: Devdev 1b      ",fillcolor="lightskyblue"]
# DATABASE1c [shape="ellipse",label="      DatabaseDatabase: Devdev 1c      ",fillcolor="lightskyblue"]
DATABASE2  [shape="ellipse",label="      DatabaseDatabase: Testtest      ",fillcolor="lightskyblue"]
DATABASE3  [shape="ellipse",label="      DatabaseDatabase: Prodprod      ",fillcolor="lightskyblue"]
REPO [label="    Git Repository    ",fillcolor="orange"]

BRANCH1 [shape="ellipse",label="    Branch: Devdev    ",fillcolor="orange"]
BRANCH2 [shape="ellipse",label="    Branch: Test test   ",fillcolor="orange"]
BRANCH3 [shape="ellipse",label="    Branch: Prod prod   ",fillcolor="orange"]

JOC1a -> DATABASE1a
JOC1b -> DATABASE1b
# JOC1c -> DATABASE1c
JOC2 -> DATABASE2
JOC3 -> DATABASE3

JOC1a -> BRANCH1 [label="   Commit/Push\nPull   "]
JOC1b -> BRANCH1 [label="   Commit/Push\nPull   "]
# JOC1c -> BRANCH1 [label="   Commit/Push\nPull   "]
JOC2 -> BRANCH2 [label="   Commit/Push\nPull   "]
JOC3 -> BRANCH3 [label="   Commit/Push\nPull   "]

BRANCH1 -> BRANCH2 [label="   merge   "] 
BRANCH2 -> BRANCH3 [label="   merge   "[label="   merge   "] 

BRANCH1 -> REPO
BRANCH2 -> REPO
BRANCH3 -> REPO


USER1aUSER3 -> JOC3
USER2a -> JOC1aJOC2
USER1bUSER2b -> JOC1bJOC2
USER1cUSER1a -> JOC1cJOC1a
USER2USER1b -> JOC2JOC1b
USER3# USER1c -> JOC3JOC1c

Operations

  • management
  • versioning
  • rollout
  • force overwrite

...

  • The JOC Cockpit database holds the configuration objects:
    • each JOC Cockpit instance is connected to its own database.
    • each Git branch is connected to one or more JOC Cockpit instanceinstances.
  • Each branch makes use of an individual version.
  • Each user (client) makes use of an individual JS7 instance using JOC Cockpit and database, Controller and Agents
    If merging of configuration objects is required then additional tools such as Git Extensions can be used.