Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

Introduction

For JS7 - Automated Deployment On Premises use of a Deployment Area is recommended.

...

  • to hold the script environment for JS7 - Automated Installation and Update and the JS7 - Deployment Packaging,
  • to hold the configuration files and certificates for deployment of JS7 componentsproducts,
  • to hold the JS7 installations per JS7 release and component product such as JOC Cockpit, Controller, Agent,
  • to hold the archive of deployment packages per JS7 releases and target machine,
  • to operate a JS7 Agent that is used to perform the JS7 - Deployment Workflow.

...

  • archive (holds Deployment Packages)
    • <deployment-descriptor> (specifies the Deployment Descriptor)
      • agents (holds Agent Deployment Packages)
        • <agent-id>(specifies the directory for a specific Agent identified by its Agent ID)
          • js7_deploy_agent_unix.<agent-id>.<release>.config.tar.gz (tarball for the Agent's configuration directory)
          • js7_deploy_agent_unix.<agent-id>.<release>.install.tar.gz (tarball for the Agent's installation directory)
          • run_deploy_agent.sh (Deployment Script for execution on the target host)
          • run_install_agent.sh (wrapper script for the parameterized call to the js7_install_agent.sh Installer Script)
          • run_rollout_agent.sh (Rollout Script for transfer of tarballs and Deployment Script to the target host)
        • <agent-id>(specifies the directory for a specific Agent identified by its Agent ID)
        • ...
      • controllers (holds Controller Deployment Packages)
        • <controller-id> (specifies the directory for a specific Controller identified by its Controller ID)
          • <instance-type> (specifies the type of the Controller instance which is primary or secondary
            • js7_deploy_controller_unix.<controller-id>.<instance-type>.<release>.config.tar.gz (tarball for the Controller instance's configuration directory)
            • js7_deploy_controller_unix.<controller-id>.<instance-type>.<release>.install.tar.gz (tarball for the Controller instance's installation directory)
            • run_deploy_controller.sh (Deployment Script for execution on the target host)
            • run_install_controller.sh (Wrapper Script for the parameterized call to the js7_install_controller.sh Installer Script)
            • run_rollout_controller.sh (Rollout Script for transfer of tarballs and Deployment Script to the target host)
          • <instance-type> (specifies the type of the Controller instance which is primary or secondary
        • <controller-id> (specifies the (directory for a specific Controller identified by its Controller ID)
        • ...
      • joc (holds JOC Cockpit Deployment Packages)
        • <joc-id>(specifies the directory for a specific JOC Cockpit identified by its Cockpit ID)
          • <instance-type> (specifies the type of the JOC Cockpit instance which is primary or secondary
            • js7_deploy_joc_linux.<joc-id>.<instance-type>.<release>.config.tar.gz (tarball for the JOC Cockpit instance's configuration directory)
            • js7_deploy_joc_linux.<joc-id>.<instance-type>.<release>.install.tar.gz (tarball for the JOC Cockpit instance's installation directory)
            • run_deploy_joc.sh (Deployment Script for execution on the target host)
            • run_install_joc.sh (Wrapper Script for the parameterized call to the js7_install_joc.sh Installer Script.
            • run_rollout_joc.sh (Rollout Script for transfer of tarballs and Deployment Script to the target host)
          • <instance-type> (specifies the type of the JOC Cockpit instance which is primary or secondary
        • <joc-id>(specifies the directory for a specific JOC Cockpit identified by its JOC Cockpit ID)
        • ...
      • js7_import_tar.gz (holds the Deployment Workflow for import to JS7 JOC Cockpit, see JS7 - Deployment Workflow)
      • run_deploy.sh (Wrapper Script to run all run_deploy_*.sh Deployment Scripts for Agent, Controller and JOC Cockpit instances )in the same Deployment Descriptor)
    • <deployment-descriptor> (specifies the Deployment Descriptor)
    • ...
  • bin (holds executable files, preferably individual scripts, Deployment Scripts and Installer Scripts available from JS7 - Download)
  • ca  (holds the Certificate Authority as explained from  JS7 - How to create self-signed X.509 SSL TLS Certificates, not used if an external Certificate Authority is in place)
    • certs (holds self-signed certificates)
    • csr (holds Certificate Signing Requests)
    • private (holds Private Keys)
  • config (holds configuration files)
    • agents (holds Agent configuration files)
      • instances (holds configuration files specific for an Agent in a user environment)
        • <agent-id>(specifies the Agent ID for directories and files that are specific to an Agent)
          • config (general configuration)
            • private (specific configuration)
              • https-keystore.p12 (default location and file name of a PKCS12 keystore holding the Agent's private key and certificate)
              • https-truststore.p12 (default location and file name of a PKCS12 truststore holding the Root CA Certificate)
        • <agent-id>(specifies the Agent ID for directories and files that are specific to an Agent)
        • ...
      • templates (holds configuration files that ship as templates for a number of Agents)
        • <template-name> (an arbitrary directory name for templates can be used)
        • <template-name> (an arbitrary directory name for templates can be used)
        • ...
    • controllers (holds Controller configuration files)
      • instances (holds configuration files specific for Controller instances in a user environment)
        • <controller-id>.<controller-type>(specifies the Controller ID for directories and files that are specific to a Controller instance with the instance type being primary or secondary)
          • config (general configuration)
            • private (specific configuration)
              • <controller-cert> (the Controller instance's server authentication certificate)
              • https-keystore.p12 (default location and file name of a PKCS12 keystore holding the Controller instance's private key and certificate)
              • https-truststore.p12 (default location and file name of a PKCS12 truststore holding the Root CA Certificate)
        • <controller-id>.<controller-type>(specifies the Controller ID for directories and files that are specific to a Controller)
        • ...
      • templates (holds configuration files that ship as templates for a number of Controllers)
        • <template-name> (an arbitrary directory name for templates can be used)
        • <template-name> (an arbitrary directory name for templates can be used)
        • ...
    • joc  (holds JOC Cockpit configuration files)
      • instances (holds configuration files that are specific for JOC Cockpit instance in a user environment)
        • <joc-id>.<instance-type> (holds configuration files for a JOC Cockpit instance with the instance type being primary or secondary)
          • resources (optionally holds configuration files such as the joc.properties file, keystore, truststore files etc.)
          • response (optionally holds response files, mainly the joc_install.xml response file, that will be copied to the JOC Cockpit's setup directory)
        • <joc-id>.<instance-type> (holds configuration files for a JOC Cockpit instance with the instance type being primary or secondary)
        • ...
      • templates (holds configuration files that ship as templates for a number of JOC Cockpit instances)
        • dbms (holds sample response files per supported DBMS, mainly the joc_install.xml response file and hibernate.cfg.xml file that will be copied to the JOC Cockpit's setup directory)
          • h2 (H2®)
          • mariadb (MariaDB®)
          • mssql (SQL Server®)
          • mysql (MySQL®)
          • oracle (Oracle®)
          • pgsql (PostgreSQL®)
        • http (holds sample configuration files for use with HTTP connections)
          • resources (holds configuration files such as the joc.properties file)
          • start.d (holds configuration files such as http.ini, https.ini, ssl.ini)
        • https (holds sample configuration files for use with HTTPS connections)
          • resources (holds configuration files such as the joc.properties file with keystore and truststore settings)
          • start.d (holds configuration files such as http.ini, https.ini, ssl.ini)
        • <template-name> (an arbitrary directory name for templates can be used)
        • ...
    • licenses (optionally holds files related to JS7 licenses in a user environment)
  • desc (holds Deployment Descriptors)
    • <deployment-descriptor>.json (Deployment Descriptor .json file)
    • standalone (holds sample files related to Standalone instances of JS7 Agents, Controller, JOC Cockpit)
    • cluster (holds sample files related to Cluster instances of JS7 Agents, Controller, JOC Cockpit)
    • <deployment-descriptor>.json (Deployment Descriptor .json file specific for a user environment)
    • <deployment-descriptor>.json (Deployment Descriptor .json file specific for a user environment)
    • ...
  • logs (optionally holds log files)
    • deployment_package.<deployment-descriptor>.<host>.<timestamp>.log (Packaging Script log files)
    • install_js7_agent.<host>.<timestamp>.log (Agent Installer log files)
    • install_js7_controller.<host>.<timestamp>.log (Controller Installer log files)
    • install_js7_joc.<host>.<timestamp>.log (JOC Cockpit Installer log files)
  • release  (holds the installation tarballs of JS7 releases)
    • ... (users can apply an arbitrary directory hierarchy at this level)
      • js7_agent_unix.<release>.tar.gz (JS7 Agent installation tarball as download from the SOS Web Site)
      • js7_controller_unix.<release>.tar.gz (JS7 Controller installation tarball as download from the SOS Web Site)
      • js7_joc_linux.<release>.tar.gz (JS7 JOC Cockpit installation tarball as download from the SOS Web Site)
  • work (the working area is preferably used to perform installation of JS7 components products during packaging)
    • agents (directory for Agent installation during packaging)
      • <agent-id>(specifies the Agent ID for directories and files that are specific to an Agent)
        • ... (sub-directories used for Agent installation)
      • <agent-id>(specifies the Agent ID for directories and files that are specific to an Agent)
      • ...
    • controllers (directory for Controller installation during packaging)
      • <controller-id>(specifies the Controller ID for directories and files that are specific to a Controller)
        • ... (sub-directories used for Controller installation)
      • <controller-id>(specifies the Controller ID for directories and files that are specific to a Controller)
      • ...
    • tmp (temporary files are written to this directory, if the --keep-work switch is used when invoking the JS7 - Deployment Packaging then files will remain in this directory which suggests cleanup by the user)
  • env.sh (Environment Script, see next chapter)

...

Code Block
titleExample for Environment File env.sh
linenumberstrue
#!/bin/sh

SCRIPT_HOME=${HOME}/js7.deploy

DEP_ARCHIVE="${DEP_ARCHIVE:-$SCRIPT_HOME/archive}"
DEP_BIN="${DEP_BIN:-$SCRIPT_HOME/bin}"
DEP_CA="${DEP_CA:-$SCRIPT_HOME/ca}"
DEP_CONFIG="${DEP_CONFIG:-$SCRIPT_HOME/config}"
DEP_DESC="${DEP_DESC:-$SCRIPT_HOME/desc}"
DEP_RELEASE="${DEP_RELEASE:-$SCRIPT_HOME/release}"
DEP_WORK="${DEP_WORK:-$SCRIPT_HOME/work}"

JAVA_HOME="/usr/lib/jvm/jdk-11.0.2"
PATH=${DEP_BIN}:${JAVA_HOME}/bin:${PATH}

export JAVA_HOME PATH DEP_ARCHIVE DEP_BIN DEP_CA DEP_CONFIG DEP_DESC DEP_RELEASE DEP_WORK

...

The Installer Wrapper Scripts are created by the JS7 - Deployment Packaging Script to parameterize calls to the Installer Scripts. The scripts are generated by the Packaging Script.

An example An example of a Wrapper Script for installation of an Agent using HTTPS connections looks like this:

Code Block
titleExample for Installer Wrapper Script run_install_agent.sh
linenumberstrue
collapsetrue
#!/bin/sh

set -e

DEP_CONFIG="/home/sos/js7.deploy/config"
DEP_BIN="/home/sos/js7.deploy/bin"
DEP_WORK_TEMP="/home/sos/js7.deploy/work/agents/agent_001"

mkdir -p "${DEP_WORK_TEMP}"

"${DEP_BIN}"/js7_install_agent.sh \
    --real-path-prefix="${DEP_WORK_TEMP}" \
    --release="2.5.0" \
    --tarball="/mnt/releases/scheduler_setups/2.5.0/js7_agent_unix.2.5.0.tar.gz" \
    --home="${DEP_WORK_TEMP}/opt/sos-berlin.com/js7/agent-primary" \
    --data="${DEP_WORK_TEMP}/var/sos-berlin.com/js7/agent-primary" \
    --http-port="localhost:31443" \
    --https-port="centostest-primary.sos:31443" \
    --controller-id="cluster" \
    --controller-primary-cert="${DEP_CONFIG}/controllers/instances/cluster.primary/config/centostest-primary.crt" \
    --controller-secondary-cert="${DEP_CONFIG}/controllers/instances/cluster.secondary/config/private/centostest-secondary.crt" \
    --keystore="${DEP_CONFIG}/agents/instances/agent_001/config/private/https-keystore.p12" \
    --keystore-password="jobscheduler" \
    --truststore="${DEP_CONFIG}/agents/instances/agent_001/config/private/https-truststore.p12" \
    --truststore-password="jobscheduler" \
    --deploy-dir="${DEP_CONFIG}/agents/templates/https/config" \
    --make-dirs

...

Anchor
deployment_scripts
deployment_scripts
Deployment Scripts: run_deploy_agent.sh, run_deploy_controller.sh, run_deploy_joc.sh

The Deployment Scripts perform transfer of deployment tarballs to target machines and extracts are created by the JS7 - Deployment Packaging Script. The scripts are called by Rollout Scripts on target hosts to extract the respective tarballs. The parameterization is created from the Deployment Descriptor.

Code Block
titleExample for Deployment Script run_deploy_agent.sh
linenumberstrue
collapsetrue
#!/bin/sh

set -e

scp -i "/home/sos/.ssh/sos_rsa" -P 22 "/home/sos/js7.dep/archive/standalone-agent-http-2022-12-04/agents/agent_001/js7_deploy_agent_unix.agent_001.2.5.0.install.tar.gz" sos@centostest-primary:"/tmp"
ssh -i "/home/sos/.ssh/sos_rsa" -p 22 -t -t -o LogLevel=QUIET sos@centostest-primary << 'EOF'
if [ $(id -u) -eq 0 ]
then
    use_sudo=""
force_sudo="0"
if [ $(id -u) -eq 0 ]
then
    use_sudo=""
else
    use_sudo="sudo"
fi

systemd_service_dir="/usr/lib/systemd/system"
systemd_service_name="js7_agent_31445.service"

...

mkdir -p " Log ".. executing pre-extraction command: StopService"
StopService

if [ -d "/opt/sos-berlin.com/js7/agent-primary"
cd " ]
then
    Log ".. removing home directory: /opt/sos-berlin.com/js7/agent-primary"
   > /dev/null
test -e "/tmp/js7_deploy_agent_unix.agent_001.2.5.0.install.tar.gz" if [ "${force_sudo}" -eq 0 ] && gzip[ -cw -d < "/tmp/opt/sos-berlin.com/js7_deploy_agent_unix.agent_001.2.5.0.install.tar.gz" | tar -xf -
rc=$?
cd - > /dev/null

exit $rc
EOF


scp -i "/home/sos/.ssh/sos_rsa" -P 22 "/home/sos/js7.dep/archive/standalone-agent-http-2022-12-04/agents/agent_001/js7_deploy_agent_unix.agent_001.2.5.0.config.tar.gz" sos@centostest-primary:"/tmp"
ssh -i "/home/sos/.ssh/sos_rsa" -p 22 -t -t -o LogLevel=QUIET sos@centostest-primary << 'EOF'
if [ $(id -u) -eq 0 ]
then
    use_sudo=""
else
    use_sudo="sudo"
fi

/agent-primary" ]
    then
        rm -f -r "/opt/sos-berlin.com/js7/agent-primary"
    else
        ${use_sudo} rm -f -r "/opt/sos-berlin.com/js7/agent-primary"
    fi
fi

if [ ! -d "/opt/sos-berlin.com/js7" ]
then
    Log ".. creating parent home directory: /opt/sos-berlin.com/js7"
    if [ "${force_sudo}" -eq 0 ] && [ -w "/opt/sos-berlin.com" ]
    then
        mkdir -p "/varopt/sos-berlin.com/js7"
cd    else
        ${use_sudo} mkdir -p "/varopt/sos-berlin.com/js7"
 > /dev/null
test -e "/tmp/js7_deploy_agent_unix.agent_001.2.5.0.config.tar.gz" && gzip -c -d < "/tmp/js7_deploy_agent_unix.agent_001.2.5.0.config.tar.gz" | tar -xf -
rc=$?
cd - > /dev/null

systemd_service_dir="/usr/lib/systemd/system"
systemd_service_name="js7_agent_31445.service"

...

if [ "${rc}" -eq 0 ]
then
    MakeService " fi

    if [ "${force_sudo}" -eq 1 ] || [ ! -w "/opt/sos-berlin.com" ]
    then
        Log ".. owning parent home directory to: sos"
        ${use_sudo} chown -R sos "/opt/sos-berlin.com/js7"
    fi
fi

Log ".. extracting tarball to home directory: /opt/sos-berlin.com/js7/agent-standalone/bin/agent_31445.service"
    rc=$?
fi
"
cd "/opt/sos-berlin.com/js7" > /dev/null

Log ".. extracting tarball: js7_deploy_agent_unix.agent_001.2.5.1.install.tar.gz"
if [ "${rcforce_sudo}" -eq 01 ]
then 
	StartService
fi

exit $rc
EOF

Explanation:

...


    test -e "/tmp/js7_deploy_agent_unix.agent_001.2.5.1.install.tar.gz" && gzip -c -d < "/tmp/js7_deploy_agent_unix.agent_001.2.5.1.install.tar.gz" | ${use_sudo} tar -xf -
else
    test -e "/tmp/js7_deploy_agent_unix.agent_001.2.5.1.install.tar.gz" && gzip -c -d < "/tmp/js7_deploy_agent_unix.agent_001.2.5.1.install.tar.gz" | tar -xf -
fi
rc=$?
cd - > /dev/null

if [ ! -d "/var/sos-berlin.com/js7" ]
then
    Log ".. creating parent data directory: /var/sos-berlin.com/js7"
    if [ "${force_sudo}" -eq 0 ] && [ -w "/var/sos-berlin.com" ]
    then
        mkdir -p "/var/sos-berlin.com/js7"
    else
        ${use_sudo} mkdir -p "/var/sos-berlin.com/js7"
    fi

    if [ "${force_sudo}" -eq 1 ] || [ ! -w "/var/sos-berlin.com" ]
    then
        Log ".. owning parent data directory to: sos"
        ${use_sudo} chown -R sos "/var/sos-berlin.com/js7"
    fi
else
    if [ -d "/var/sos-berlin.com/js7/agent-primary" ]
    then
        if [ "${force_sudo}" -eq 1 ] || [ ! -w "/var/sos-berlin.com/js7/agent-primary" ]
        then
            Log ".. owning data directory to: sos"
            ${use_sudo} chown -R sos "/var/sos-berlin.com/js7/agent-primary"
        fi
    fi
fi

Log ".. extracting tarball to data directory: /var/sos-berlin.com/js7"
cd "/var/sos-berlin.com/js7" > /dev/null

Log ".. extracting tarball: js7_deploy_agent_unix.agent_001.2.5.1.config.tar.gz"
if [ "${force_sudo}" -eq 1 ]
then
    test -e "/tmp/js7_deploy_agent_unix.agent_001.2.5.1.config.tar.gz" && gzip -c -d < "/tmp/js7_deploy_agent_unix.agent_001.2.5.1.config.tar.gz" | ${use_sudo} tar -xf -
else
    test -e "/tmp/js7_deploy_agent_unix.agent_001.2.5.1.config.tar.gz" && gzip -c -d < "/tmp/js7_deploy_agent_unix.agent_001.2.5.1.config.tar.gz" | tar -xf -
fi
rc=$?
cd - > /dev/null

if [ "${rc}" -eq 0 ]
then
    Log ".. owning home directory to: sos1:sos1"
    ${use_sudo} chown -R sos1:sos1 "/opt/sos-berlin.com/js7/agent-primary"
    rc=$?
fi

if [ "${rc}" -eq 0 ]
then
    Log ".. owning data directory to: sos2:sos2"
    ${use_sudo} chown -R sos2:sos2 "/var/sos-berlin.com/js7/agent-primary"
    rc=$?
fi

if [ "${rc}" -eq 0 ]
then
    Log ".. creating systemd service: js7_agent_31445.service"
    MakeService "/opt/sos-berlin.com/js7/agent-primary/bin/agent_31445.service"
    rc=$?
fi

Log ".. executing post-extraction command: StartService"
StartService

Log ".. deployment completed, return code: ${rc}"
exit ${rc}


Explanation:

  • Authentication is performed by use of private/public key.
  • For each deployed tarball transfer and extraction are individually applied.
  • The example includes commands for pre- and post-processing to stop and to start an Agent from iJS7 - systemd Service Files for automated Startup and Shutdown with Unix Systems.
  • The example leaves out (...) a number of shell functions that are created to implement the MakeService, StopService and StartService operations.

Anchor
rollout_scripts
rollout_scripts
Rollout Scripts: run_rollout_agent.sh, run_rollout_controller.sh, run_rollout_joc.sh

The Rollout Scripts perform transfer of deployment tarballs to target hosts and extract the respective tarballs by invoking the Deployment Script. The parameterization is created from the Deployment Descriptor.

Code Block
titleExample for Rollout Script run_rollout_agent.sh
linenumberstrue
collapsetrue
#!/bin/sh

set -e

scp -i "/home/sos/.ssh/sos_rsa" -P 22 "/home/sos/js7.deploy/archive/user-joc-controller-agent-https-2022-12-04/agents/agent_001/js7_deploy_agent_unix.agent_001.2.5.1.install.tar.gz" sos@centostest-primary:"/tmp"
scp -i "/home/sos/.ssh/sos_rsa" -P 22 "/home/sos/js7.deploy/archive/user-joc-controller-agent-https-2022-12-04/agents/agent_001/js7_deploy_agent_unix.agent_001.2.5.1.config.tar.gz" sos@centostest-primary:"/tmp"
scp -i "/home/sos/.ssh/sos_rsa" -P 22 "/home/sos/js7.deploy/archive/user-joc-controller-agent-https-2022-12-04/agents/agent_001/run_deploy_agent.sh" sos@centostest-primary:"/tmp"
ssh -i "/home/sos/.ssh/sos_rsa" -p 22 -t -t -o LogLevel=QUIET sos@centostest-primary 'chmod u+x /tmp/run_deploy_agent.sh && /tmp/run_deploy_agent.sh'

Resources

...