JS7 JobScheduler

Space shortcuts

Page tree

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Next »

Introduction

In the JS7 - Deployment process a Deployment Descriptor specifies

  • which JS7 components of JOC Cockpit, Controller instances and Agents should be installed,
  • where to install each JS7 component,
  • which installation options to use, for example the installation directory,
  • which configuration options to use, for example port usage.

Deployment Descriptors are specified from files in JSON format and include instructions, for example to deploy a larger number of Agents on target machines.

The JS7 - Deployment Packaging Script makes use of the Deployment Descriptor to install and to package JS7 components.

Deployment Descriptor

Deployment Descriptors are specified from .json files.

  • Users are strongly encouraged to validate their .json files against the Deployment Descriptor Schema: JS7_Deployment_Descriptor_v1.0.json
  • The basic structure includes to specify an array of Agent objects.

Deployment Descriptor Schema for Agents

TODO

Examples

Deployment Descriptor for Agents

Agents with HTTP Connections

The Deployment Descriptor specifies two Agents to be installed on two hosts:

  • Host centostest-primary and centostest-secondary each are operating an Agent.
  • Agents make use of HTTP connections.

Download example: agent-http-2022-12-04.json


Example for Deployment Descriptor 
{
    "descriptor": {
        "descriptorId": "agent-http-2022-12-04",
        "title": "Install or update Agents for HTTP connections to release 2.5.0",
        "account": "sos",
        "scheduled": "2022-12-02T06:19:12+01:00",
        "created": "2022-11-30T23:19:12+01:00"
    },
    "agents": [
        {
            "agent_001": {
                "target": {
                    "connection": {
                        "host": "centostest-primary",
                        "port": 22
                    },
                    "authentication": {
                        "method": "publickey",
                        "user": "sos",
                        "keyFile": "/home/sos/.ssh/sos_rsa"
                    },
                    "packageLocation": "/tmp",
                    "execPre": "sudo systemctl stop js7_agent.service",
                    "execPost": "sudo systemctl start js7_agent.service"
                },
                "media": {
                    "release": "2.5.0",
                    "tarball": "/mnt/releases/scheduler_setups/2.5.0/js7_agent_unix.2.5.0.tar.gz"
                },
                "installation": {
                    "home": "/opt/sos-berlin.com/js7/agent-primary",
                    "data": "/var/sos-berlin.com/js7/agent-primary",
                    "httpPort": "31445"
                }
            }
        },
        {
            "agent_002": {
                "target": {
                    "connection": {
                        "host": "centostest-secondary",
                        "port": 22
                    },
                    "authentication": {
                        "method": "publickey",
                        "user": "sos",
                        "keyFile": "/home/sos/.ssh/sos_rsa"
                    },
                    "packageLocation": "/tmp",
                    "execPre": "sudo systemctl stop js7_agent.service",
                    "execPost": "sudo systemctl start js7_agent.service"
                },
                "media": {
                    "release": "2.5.0",
                    "tarball": "/mnt/releases/scheduler_setups/2.5.0/js7_agent_unix.2.5.0.tar.gz"
                },
                "installation": {
                    "home": "/opt/sos-berlin.com/js7/agent-secondary",
                    "data": "/var/sos-berlin.com/js7/agent-secondary",
                    "httpPort": "32445"
                }
            }
        }
    ],
    "controllers": [],
    "joc": []
}

Agents with HTTPS Connections

The Deployment Descriptor specifies the same Agents as from the previous example to be installed:

  • Host centostest-primary and centostest-secondary each are operating an Agent.
  • Agents use HTTPS connections and are equipped with certificates.

Download example: agent-https-2022-12-04.json


Example for Deployment Descriptor 
{
    "descriptor": {
        "descriptorId": "agent-https-2022-12-04",
        "title": "Install or update Agents for HTTPS connections to release 2.5.0",
        "account": "sos",
        "scheduled": "2022-12-02T06:19:12+01:00",
        "created": "2022-11-30T23:19:12+01:00"
    },
    "certificates": {
        "controller": {
            "primaryControllerCert": "controllers/instances/testsuite.primary/config/private/centostest-primary.crt",
            "secondaryControllerCert": "controllers/instances/testsuite.secondary/config/private/centostest-secondary.crt"
        }
    },
    "agents": [
        {
            "agent_001": {
                "target": {
                    "connection": {
                        "host": "centostest-primary",
                        "port": 22
                    },
                    "authentication": {
                        "method": "publickey",
                        "user": "sos",
                        "keyFile": "/home/sos/.ssh/sos_rsa"
                    },
                    "packageLocation": "/tmp",
                    "execPre": "sudo systemctl stop js7_agent.service",
                    "execPost": "sudo systemctl start js7_agent.service"
                },
                "media": {
                    "release": "2.5.0",
                    "tarball": "/mnt/releases/scheduler_setups/2.5.0/js7_agent_unix.2.5.0.tar.gz"
                },
                "installation": {
                    "home": "/opt/sos-berlin.com/js7/agent-primary",
                    "data": "/var/sos-berlin.com/js7/agent-primary",
                    "httpPort": "localhost:31443",
                    "httpsPort": "centostest-primary.sos:31443"
                },
                "configuration": {
                    "controller": {
                        "controllerId": "testsuite"
                    },
                    "certificates": {
                        "keyStore": "agents/instances/agent_001/config/private/https-keystore.p12",
                        "keyStorePassword": "jobscheduler",
                        "keyPassword": "jobscheduler",
                        "trustStore": "agents/instances/agent_001/config/private/https-truststore.p12",
                        "trustStorePassword": "jobscheduler"
                    },
                    "templates": [
                        "agents/templates/https/config"
                    ]
                }
            }
        },
        {
            "agent_002": {
                "target": {
                    "connection": {
                        "host": "centostest-secondary",
                        "port": 22
                    },
                    "authentication": {
                        "method": "publickey",
                        "user": "sos",
                        "keyFile": "/home/sos/.ssh/sos_rsa"
                    },
                    "packagelocation": "/tmp",
                    "execPre": "sudo systemctl stop js7_agent.service",
                    "execPost": "sudo systemctl start js7_agent.service"
                },
                "media": {
                    "release": "2.5.0",
                    "tarball": "/mnt/releases/scheduler_setups/2.5.0/js7_agent_unix.2.5.0.tar.gz"
                },
                "installation": {
                    "home": "/opt/sos-berlin.com/js7/agent-secondary",
                    "data": "/var/sos-berlin.com/js7/agent-secondary",
                    "httpPort": "localhost:32443",
                    "httpsPort": "centostest-secondary.sos:32443"
                },
                "configuration": {
                    "controller": {
                        "controllerId": "testsuite"
                    },
                    "certificates": {
                        "keyStore": "agents/instances/agent_002/config/private/https-keystore.p12",
                        "keyStorePassword": "jobscheduler",
                        "keyPassword": "jobscheduler",
                        "trustStore": "agents/instances/agent_002/config/private/https-truststore.p12",
                        "trustStorePassword": "jobscheduler"
                    },
                    "templates": [
                        "agents/templates/https/config"
                    ]
                }
            }
        }
    ],
    "controllers": [],
    "joc": []
}

Deployment Descriptor for Controllers

Single Controllers with HTTP Connections

Controller Cluster with HTTPS Connections

Deployment Descriptor for JOC Cockpit

Single JOC Cockpit with HTTP Connections

JOC Cockpit Cluster with HTTPS Connections

Deployment Descriptor for Agents, Controllers, JOC Cockpit

Deployment Descriptor for Standalone Agent, Controller and JOC Cockpit Cluster using HTTP Connections

The Deployment Descriptor specifies a complete job scheduling environment including

  • Host centostest-primary operating an Agent, a Controller instance and a JOC Cockpit instance.
  • All components use HTTP connections.

Download example: agent-controller-joc-http-2022-12-04.json


Example for Deployment Descriptor 
{
    "descriptor": {
        "descriptorId": "agent-controller-joc-http-20221204",
        "title": "Install or update JOC Cockpit, Controller and Agents to release 2.5.0",
        "account": "sos",
        "scheduled": "2022-12-02T06:19:12+01:00",
        "created": "2022-11-30T23:19:12+01:00"
    },
    "agents": [
        {
            "agent_001": {
                "target": {
                    "connection": {
                        "host": "centostest-primary",
                        "port": 22
                    },
                    "authentication": {
                        "method": "publickey",
                        "user": "sos",
                        "keyFile": "/home/sos/.ssh/sos_rsa"
                    },
                    "packageLocation": "/tmp",
                    "execPre": "sudo systemctl stop js7_agent.service",
                    "execPost": "sudo systemctl start js7_agent.service"
                  },
                "media": {
                    "release": "2.5.0",
                    "tarball": "/mnt/releases/scheduler_setups/2.5.0/js7_agent_unix.2.5.0.tar.gz"
                },
                "installation": {
                    "home": "/home/sos/js7.target/opt/agent-primary",
                    "data": "/home/sos/js7.target/var/agent-primary",
                    "httpPort": "31445"
                },
                "configuration": {
                    "controller": {
                        "controllerId": "testsuite"
                    },
                    "templates": [
                        "agents/templates/https/config"
                    ]
                }
            }
        }
    ],
    "controllers": [
        {
            "testsuite": {
                "primary": {
                    "target": {
                        "connection": {
                            "host": "centostest-primary",
                            "port": 22
                        },
                        "authentication": {
                            "method": "publickey",
                            "user": "sos",
                            "keyFile": "/home/sos/.ssh/sos_rsa"
                        },
                        "packageLocation": "/tmp",
                        "execPre": "sudo systemctl stop js7_controller.service",
                        "execPost": "sudo systemctl start js7_controller.service"
                      },
                    "media": {
                        "release": "2.5.0",
                        "tarball": "/mnt/releases/scheduler_setups/2.5.0/js7_controller_unix.2.5.0.tar.gz"
                    },
                    "installation": {
                        "home": "/home/sos/js7.target/opt/controller-primary",
                        "data": "/home/sos/js7.target/var/controller-primary",
                        "httpPort": "21444"
                    },
                    "configuration": {
                        "templates": [
                            "controllers/templates/https.primary/config"
                        ]
                    }
                }
            }
        }
    ],
    "joc": [
        {
            "joc_001": {
                "primary": {
                    "target": {
                        "connection": {
                            "host": "centostest-primary",
                            "port": 22
                        },
                        "authentication": {
                            "method": "publickey",
                            "user": "sos",
                            "keyFile": "/home/sos/.ssh/sos_rsa"
                        },
                        "packageLocation": "/tmp",
                        "execPre": "sudo systemctl stop js7_joc.service",
                        "execPost": "sudo systemctl start js7_joc.service"
                    },
                    "media": {
                        "release": "2.5.0",
                        "tarball": "/mnt/releases/scheduler_setups/2.5.0/js7_joc_linux.2.5.0.tar.gz"
                    },
                    "installation": {
                        "setupDir": "/home/sos/js7.target/opt/joc-primary.setup",
                        "home": "/home/sos/js7.target/opt/joc-primary",
                        "data": "/home/sos/js7.target/var/joc-primary",
                        "httpPort": "11446",
                        "isUser": true,
                        "isPreserveEnv": true,
                        "javaOptions": "-Xmx512m -Djava.security.egd=file:///dev/urandom"
                    },
                    "configuration": {
                        "responseDir": "joc/instances/joc_001.primary/response",
                        "templates": [
                            "joc/templates/https/resources"
                        ]
                    }
                }
            }
        }
    ]
}

Deployment Descriptor for Agents, Controller Cluster and JOC Cockpit Cluster using HTTPS Connections

The Deployment Descriptor specifies a complete job scheduling environment including

  • Host centostest-primary and centostest-secondary each are operating an Agent, a Controller instance and a JOC Cockpit instance.
  • Controller and JOC Cockpit instances on both hosts act as a cluster.
  • Use of clustering includes to deploy a JS7 license.
  • All components use HTTPS connections and are equipped with certificates.

Download example: agent-controller-joc-https-2022-12-04.json


Example for Deployment Descriptor 
{
    "descriptor": {
        "descriptorId": "joc-controller-agents-all-20221204",
        "title": "Install or update JOC Cockpit, Controller and Agents to release 2.5.0",
        "account": "sos",
        "scheduled": "2022-12-02T06:19:12+01:00",
        "created": "2022-11-30T23:19:12+01:00"
    },
    "license": {
        "licenseKeyFile": "licenses/sos.pem",
        "licenseBinFile": "licenses/js7-license.jar"
    },
    "certificates": {
        "controller": {
            "primaryControllerCert": "controllers/instances/testsuite.primary/config/private/centostest-primary.crt",
            "secondaryControllerCert": "controllers/instances/testsuite.secondary/config/private/centostest-secondary.crt"
        },
        "joc": {
            "primaryJocCert": "joc/instances/joc_001.primary/resources/centostest-primary.crt",
            "secondaryJocCert": "joc/instances/joc_001.primary/resources/centostest-secondary.crt"
        }
    },
    "agents": [
        {
            "agent_001": {
                "target": {
                    "connection": {
                        "host": "centostest-primary",
                        "port": 22
                    },
                    "authentication": {
                        "method": "publickey",
                        "user": "sos",
                        "keyFile": "/home/sos/.ssh/sos_rsa"
                    },
                    "packageLocation": "/tmp",
                    "execPre": "sudo systemctl stop js7_agent.service",
                    "execPost": "sudo systemctl start js7_agent.service"
                  },
                "media": {
                    "release": "2.5.0",
                    "tarball": "/mnt/releases/scheduler_setups/2.5.0/js7_agent_unix.2.5.0.tar.gz"
                },
                "installation": {
                    "home": "/home/sos/js7.target/opt/agent-primary",
                    "data": "/home/sos/js7.target/var/agent-primary",
                    "httpPort": "localhost:31445",
                    "httpsPort": "centostest-primary.sos:31443",
                    "javaOptions": "-Xmx256m"
                },
                "configuration": {
                    "controller": {
                        "controllerId": "testsuite"
                    },
                    "certificates": {
                        "keyStore": "agents/instances/agent_001/config/private/https-keystore.p12",
                        "keyStorePassword": "jobscheduler",
                        "keyPassword": "jobscheduler",
                        "trustStore": "agents/instances/agent_001/config/private/https-truststore.p12",
                        "trustStorePassword": "jobscheduler"
                    },
                    "templates": [
                        "agents/templates/https/config"
                    ]
                }
            }
        },
        {
            "agent_002": {
                "target": {
                    "connection": {
                        "host": "centostest-secondary",
                        "port": 22
                    },
                    "authentication": {
                        "method": "publickey",
                        "user": "sos",
                        "keyFile": "/home/sos/.ssh/sos_rsa"
                    },
                    "packageLocation": "/tmp",
                    "execPre": "sudo systemctl stop js7_agent.service",
                    "execPost": "sudo systemctl start js7_agent.service"
                  },
                "media": {
                    "release": "2.5.0",
                    "tarball": "/mnt/releases/scheduler_setups/2.5.0/js7_agent_unix.2.5.0.tar.gz"
                },
                "installation": {
                    "home": "/home/sos/js7.target/opt/agent-secondary",
                    "data": "/home/sos/js7.target/var/agent-secondary",
                    "httpPort": "localhost:32445",
                    "httpsPort": "centostest-secondary.sos:32443"
                },
                "configuration": {
                    "controller": {
                        "controllerId": "testsuite"
                    },
                    "certificates": {
                        "keyStore": "agents/instances/agent_002/config/private/https-keystore.p12",
                        "keyStorePassword": "jobscheduler",
                        "keyPassword": "jobscheduler",
                        "trustStore": "agents/instances/agent_002/config/private/https-truststore.p12",
                        "trustStorePassword": "jobscheduler"
                    },
                    "templates": [
                        "agents/templates/https/config"
                    ]
                }
            }
        }
    ],
    "controllers": [
        {
            "testsuite": {
                "primary": {
                    "target": {
                        "connection": {
                            "host": "centostest-primary",
                            "port": 22
                        },
                        "authentication": {
                            "method": "publickey",
                            "user": "sos",
                            "keyFile": "/home/sos/.ssh/sos_rsa"
                        },
                        "packageLocation": "/tmp",
                        "execPre": "sudo systemctl stop js7_controller.service",
                        "execPost": "sudo systemctl start js7_controller.service"
                      },
                    "media": {
                        "release": "2.5.0",
                        "tarball": "/mnt/releases/scheduler_setups/2.5.0/js7_controller_unix.2.5.0.tar.gz"
                    },
                    "installation": {
                        "home": "/home/sos/js7.target/opt/controller-primary",
                        "data": "/home/sos/js7.target/var/controller-primary",
                        "httpPort": "localhost:21444",
                        "httpsPort": "centostest-primary.sos:21443",
                        "javaOptions": "-Xmx256m"
                    },
                    "configuration": {
                        "certificates": {
                            "keyStore": "controllers/instances/testsuite.primary/config/private/https-keystore.p12",
                            "keyStorePassword": "jobscheduler",
                            "keyPassword": "jobscheduler",
                            "trustStore": "controllers/instances/testsuite.primary/config/private/https-truststore.p12",
                            "trustStorePassword": "jobscheduler"
                        },
                        "templates": [
                            "controllers/templates/https.primary/config"
                        ]
                    }
                },
                "secondary": {
                    "target": {
                        "connection": {
                            "host": "centostest-secondary",
                            "port": 22
                        },
                        "authentication": {
                            "method": "publickey",
                            "user": "sos",
                            "keyFile": "/home/sos/.ssh/sos_rsa"
                        },
                        "packageLocation": "/tmp",
                        "execPre": "sudo systemctl stop js7_controller.service",
                        "execPost": "sudo systemctl start js7_controller.service"
                     },
                    "media": {
                        "release": "2.5.0",
                        "tarball": "/mnt/releases/scheduler_setups/2.5.0/js7_controller_unix.2.5.0.tar.gz"
                    },
                    "installation": {
                        "home": "/home/sos/js7.target/opt/controller-secondary",
                        "data": "/home/sos/js7.target/var/controller-secondary",
                        "httpPort": "localhost:22444",
                        "httpsPort": "centostest-secondary.sos:22443",
                        "javaOptions": "-Xmx256m"
                    },
                    "configuration": {
                        "certificates": {
                            "keyStore": "controllers/instances/testsuite.secondary/config/private/https-keystore.p12",
                            "keyStorePassword": "jobscheduler",
                            "keyPassword": "jobscheduler",
                            "trustStore": "controllers/instances/testsuite.secondary/config/private/https-truststore.p12",
                            "trustStorePassword": "jobscheduler"
                        },
                        "templates": [
                            "controllers/templates/https.secondary/config"
                        ]
                    }
                }
            }
        }
    ],
    "joc": [
        {
            "joc_001": {
                "primary": {
                    "target": {
                        "connection": {
                            "host": "centostest-primary",
                            "port": 22
                        },
                        "authentication": {
                            "method": "publickey",
                            "user": "sos",
                            "keyFile": "/home/sos/.ssh/sos_rsa"
                        },
                        "packageLocation": "/tmp",
                        "execPre": "sudo systemctl stop js7_joc.service",
                        "execPost": "sudo systemctl start js7_joc.service"
                    },
                    "media": {
                        "release": "2.5.0",
                        "tarball": "/mnt/releases/scheduler_setups/2.5.0/js7_joc_linux.2.5.0.tar.gz"
                    },
                    "installation": {
                        "setupDir": "/home/sos/js7.target/opt/joc-primary.setup",
                        "home": "/home/sos/js7.target/opt/joc-primary",
                        "data": "/home/sos/js7.target/var/joc-primary",
                        "httpPort": "localhost:11446",
                        "httpsPort": "centostest-primary.sos:11443",
                        "isUser": true,
                        "isPreserveEnv": true,
                        "javaOptions": "-Xmx256m -Djava.security.egd=file:///dev/urandom"
                    },
                    "configuration": {
                         "responseDir": "joc/instances/joc_001.primary/response",
                         "certificates": {
                            "keyStore": "joc/instances/joc_001.primary/resources/https-keystore.p12",
                            "keyStorePassword": "jobscheduler",
                            "keyPassword": "jobscheduler",
                            "trustStore": "joc/instances/joc_001.primary/resources/https-truststore.p12",
                            "trustStorePassword": "jobscheduler"
                        },
                        "templates": [
                            "joc/templates/https/resources"
                        ]
                    }
                },
                "secondary": {
                    "target": {
                        "connection": {
                            "host": "centostest-secondary",
                            "port": 22
                        },
                        "authentication": {
                            "method": "publickey",
                            "user": "sos",
                            "keyFile": "/home/sos/.ssh/sos_rsa"
                        },
                        "packageLocation": "/tmp",
                        "execPre": "sudo systemctl stop js7_joc.service",
                        "execPost": "sudo systemctl start js7_joc.service"
                    },
                    "media": {
                        "release": "2.5.0",
                        "tarball": "/mnt/releases/scheduler_setups/2.5.0/js7_joc_linux.2.5.0.tar.gz"
                    },
                    "installation": {
                        "setupDir": "/home/sos/js7.target/opt/joc-secondary.setup",
                        "home": "/home/sos/js7.target/opt/joc-secondary",
                        "data": "/home/sos/js7.target/var/joc-secondary",
                        "httpPort": "localhost:12446",
                        "httpsPort": "centostest-secondary.sos:12443",
                        "isUser": true,
                        "isPreserveEnv": true,
                        "javaOptions": "-Xmx256m -Djava.security.egd=file:///dev/urandom"
                    },
                    "configuration": { 
                         "responseDir": "joc/instances/joc_001.secondary/response",
                         "certificates": {
                            "keyStore": "joc/instances/joc_001.secondary/resources/https-keystore.p12",
                            "keyStorePassword": "jobscheduler",
                            "keyPassword": "jobscheduler",
                            "trustStore": "joc/instances/joc_001.secondary/resources/https-truststore.p12",
                            "trustStorePassword": "jobscheduler"
                        },
                        "templates": [
                            "joc/templates/https/resources"
                        ]
                    }
                }
            }
        }
    ]
}

Resources



  • No labels