JS7 JobScheduler

Space shortcuts

Page tree

Versions Compared

Old Version 3

changes.mady.by.user Andreas Püschel

Saved on

compared with

New Version 4

changes.mady.by.user Andreas Püschel

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Storage
    • Storage capacity has to match the number of packages that are stored per JS7 release, per JS7 component and target machines.
      • Example
        • Assume that the Deployment Server is used to deploy JS7 Agents to 100 target machines.
        • The average disk space consumption of a JS7 Agent package is around 175 MB.
        • If you intend to keep the latest three JS7 releases then this will make 300 packages with an overall disk space consumption of ca. 53 GB.
    • The working area in the Deployment Server is used to temporarily store JS7 installations. A rough estimate of 10 GB will be sufficient for most purposes.
    • Storage for JS7 release tarballs should be considered. For Unix environments the tarballs for JOC Cockpit, Controller and Agent require approx. 600 MB per release.
  • Computing Power
    • Speed matters only when it comes to packaging. This process is performed once per new JS7 release that should be deployed.
    • It can be considered a waste of resources having a powerful server being used <1% of the time.
  • Network
    • The Deployment Server requires decent connectivity as it has to connect to any target machines machine to which it deploys JS7 components.
      • Considering firewalls users could think about "opening the gates" for the timeslot of deployment only.
    • Transfer of tarball packages to target machines preferably happens in parallel in order to have a short period of time in which mixed releases of JS7 components are in place.

Security

It is essential to have a consistent and secure deployment process in place.

  • Security is the basic reason why the JS7 deployment scripts ship as shell scripts and not from a high level programming language that might require installation and updates. Transparency and adjustability are key. Users are free to modify deployment scripts at their will and to request changes within the scope of SOS professional services.
  • The source of JS7 installation tarballs is a major concern. Basically the installer scripts for JS7 components offer to download components on demand from the SOS Web Site. Users are strongly discouraged to make use of this feature for automated deployment. Instead, the JS7 components should be separately downloaded and checked that no tampered versions of the installer files have been download to exclude man-in-the-middle attacks.
  • Good understanding of the deployment process and of the working of deployment scripts is key.

Directory Layout

The suggested directory layout for the Deployment Server includes the following directories and files:

  • archive (holds deployment packages)
    • <deployment-descriptor> (specifies the Deployment Descriptor)
      • agents (holds Agent deployment packages)
        • <agent-id>(specifies the Agent ID)
          • js7_deploy_agent_unix.<agent-id>.<release>.config.tar.gz (deployment package for the Agent's configuration directory)
          • js7_deploy_agent_unix.<agent-id>.<release>.install.tar.gz (deployment package for the Agent's installation directory)
          • run_deploy_agent.sh (deployment script)
          • run_install_agent.sh (wrapper script for parameterized call to js7_install_agent.sh)
        • <agent-id>(specifies the Agent ID)
        • ...
      • controllers (holds Controller deployment packages)
        • <controller-id> (specifies the Controller ID)
          • <controller-type> (specifies the type of the Controller instance which is primary or secondary) 
            • js7_deploy_controller_unix.<controller-id>.<controller-type>.<release>.config.tar.gz (deployment package for the Controller instance's configuration directory)
            • js7_deploy_controller_unix.<controller-id>.<controller-type>.<release>.install.tar.gz (deployment package for the Controller instance's installation directory)
            • run_deploy_controller.sh (deployment script)
            • run_install_controller.sh (wrapper script for parameterized call to js7_install_controller.sh)
          • <controller-type> (specifies the type of the Controller instance which is primary or secondary) 
      • js7_import_tar.gz (holds the Deployment Workflow for import to JS7 JOC Cockpit)
    • <deployment-descriptor> (specifies the Deployment Descriptor)
    • ...
  • bin (holds executable files, preferably individual scripts, deployment scripts and installer scripts available from JS7 - Download)
  • ca  (holds the Certificate Authority as explained from  JS7 - How to create self-signed Certificates, not used if an external Certificate Authority is in place)
    • certs (holds CA-signed Certificates)
    • csr (holds Certificate Signing Requests)
    • private (holds Private Keys)
  • config (holds configuration files)
    • agents (holds Agent configuration files)
      • instances (holds configuration files specific for an Agent)
        • <agent-id>(specifies the Agent ID for directories and files that are specific to an Agent)
          • config (general configuration)
            • private (specific configuration)
              • trusted-pgp-keys (optionally holds PGP public key files and keyring files used for signing, see JS7 - Deployment of Scheduling Objects)
                • <pgp-public-key> (public key file or keyring file)
                • <pgp-public-key> (public key file or keyring file)
                • ...
              • trusted-x509-keys (optionally holds X.509 certificate files used for signing, see JS7 - Deployment of Scheduling Objects)
                • <x509-certificate> (X.509 certificate file)
                • <x509-certificate> (X.509 certificate file)
                • ...
              • https-keystore.p12 (optional default location and file name of a PKCS12 truststore)
              • https-truststore.p12 (optional default location and file name of a PKCS12 truststore)
              • private.conf (optional configuration file, for example to specify keystore, truststore and Distinguished Names of Controller certificate, see JS7 - Agent Configuration Items)
              • log4j2.xml (optional log configuration file, see JS7 - Log Levels and Debug Options)
            • agent.conf (optional configuration file, see JS7 - Agent Configuration Items)
        • <agent-id>(specifies the Agent ID for directories and files that are specific to an Agent)
        • ...
      • templates (holds configuration files that act as templates for a number of Agents)
        • <ag
    • certs
      • ca  (the Root Certificate Authority used for self-signed certificates)
        • <root-ca-certificate> (the Root CA Certificate file, frequently available with a .pem, .crt extension)
      • server
        • <server-certificate>(Server Certificate file, frequently available with a .pem, .crt extension).
        • <server-certificate>(Server Certificate file, frequently available with a .pem, .crt extension).
        • ....
      • client
        • <client-certificate>(Client Certificate file, frequently available with a .pem, .crt extension).
        • <client-certificate>(Client Certificate file, frequently available with a .pem, .crt extension).
        • ...
    • controllers
      • <controller-id>(specifies the Controller ID for directories and files that are specific to a Controller)
    • joc
  • desc
    • <deployment-descriptor>.json (Deployment Descriptor .json file)
    • <deployment-descriptor>.json (Deployment Descriptor .json file)
    • ...
  • release  (holds the installation tarballs for JS7 releases)
    • ... (users can apply an arbitrary directory hierarchy at this level)
      • js7_agent_unix.<release>.tar.gz (JS7 Agent installation tarball as download from the SOS Web Site)
      • js7_controller_unix.<release>.tar.gz (JS7 Controller installation tarball as download from the SOS Web Site)
      • js7_joc_linux.<release>.tar.gz (JS7 JOC Cockpit installation tarball as download from the SOS Web Site)
  • work (the working area is preferably used to perform installation of JS7 components during packaging)
    • agents (directory for Agent installation during packaging)
      • <agent-id>(specifies the Agent ID for directories and files that are specific to an Agent)
      • <agent-id>(specifies the Agent ID for directories and files that are specific to an Agent)
      • ...
    • controllers (directory for Controller installation during packaging)
      • <controller-id>(specifies the Controller ID for directories and files that are specific to a Controller)
      • <controller-id>(specifies the Controller ID for directories and files that are specific to a Controller)
      • ...
    • tmp (temporary files are written to this directory, if the --keep-work switch is used when invoking the JS7 - Deployment Packaging Script then files will remain in this directory which suggests cleanup by the user)
  • env.sh (Environment Script, see next chapter)

Explanation on Individual Files

Environment Script env.sh

The Environment Script initializes a number of environment variables that map to directories as explained above:

Code Block
titleExample for Environment File env.sh
linenumberstrue
#!/bin/sh

SCRIPT_HOME=$HOME/js7.deploy

DEP_ARCHIVE="${DEP_ARCHIVE:-$SCRIPT_HOME/archive}"
DEP_BIN="${DEP_BIN:-$SCRIPT_HOME/bin}"
DEP_CONFIG="${DEP_CONFIG:-$SCRIPT_HOME/config}"
DEP_DESC="${DEP_DESC:-$SCRIPT_HOME/desc}"
DEP_INSTALL="${DEP_INSTALL:-$SCRIPT_HOME/install}"
DEP_RELEASE=/mnt/releases/scheduler_setups
DEP_TARGET="/home/sos/js7.target"
DEP_WORK="${DEP_WORK:-$SCRIPT_HOME/work}"

JAVA_HOME="${SCRIPT_HOME}"/jdk-11.0.2
PATH=${JAVA_HOME}/bin:${PATH}

export JAVA_HOME PATH DEP_ARCHIVE DEP_BIN DEP_CONFIG DEP_DESC DEP_INSTALL DEP_RELEASE DEP_TARGET DEP_WORK


The Environment Script has to be sourced to make environment variables available to shell scripts in the Deployment Server:

Code Block
titleExample for Environment File env.sh
linenumberstrue
. ./env.sh