Installation as Container / with Docker

Cadenza can be executed in a container. This method of execution has a number advantages compared with other installation methods:

A container is immutable and all configuration must be injected from externally. This means that executing the container is absolutely reproducible with a given configuration
Containers are versioned, meaning that you can easily upgrade to newer versions without the need to change files in an existing installation
Containers are bundling all necessary dependencies for the execution of Cadenza. This means that you can change your execution platform without the need to install other dependencies.

While the term "container" is very generic there are very exact specifications which you have to follow in order to be able to let your application run using one of the container runtimes which are able to run OCI-compliant containers.

This document assumes you have the CadenzaWeb Docker ZIP file accessible through your Cadenza distribution. You can also use the disy Registry to download disy Cadenza.

You will need credentials (username/password) to be able to pull the CadenzaWeb-Image from the disy Registry. Please get in touch with your account representative to receive these credentials.

Purpose of this Document

These instructions are meant to help getting started with installing, configuring, running, managing and updating Cadenza Web in a Docker environment.

Contents of the .zip File

The README file: This document.
The cadenza-image.tar file: A Cadenza Web image to be used in Docker environments.
The example-config folder: A directory with example configuration for Cadenza Web, which also includes an example script (runcadwebdocker.sh) that shows how the configuration is applied.

Requirements for Cadenza Web

The host machine executing Cadenza Web inside a Docker environment has the following requirements:

At least 4 GB of free disk space, depending on the number of concurrently used Cadenza Web Docker images.
At least 1 GB of available main memory, 2 to 32 GB recommended, depending on the number of concurrent users and the complexity of the data.
A working 64bit installation of Linux version 3.10 or newer, with the following software installed:
- docker (>= 24) (refer to the Installing Docker section for additional information)

Installing Docker

If Docker is not installed, this is the first step that needs to be done.

Docker can either be installed

from the Linux distribution’s default repository (if it is included) using the distributions package manager, e.g. sudo apt install docker.io on Ubuntu/Debian, sudo dnf install docker on Fedora, etc.
by adding a Linux distribution-specific repository (if Docker provides official packages for the Linux distribution in question), e.g. CentOS, Debian, Fedora, Ubuntu and then using the distributions package manager
by using the statically-linked binaries provided by Docker (in cases other options are not available): Binaries

To verify that Docker is installed and running successfully, the following command can be run:

docker images lists the available images in your Docker installation (table with no entries if you just installed Docker)
docker container ls lists the currently running containers (and the images they are based on).

Installing Cadenza Web

To "install" the provided Cadenza Web image (cadenza-image.tar), it has to be loaded into the docker environment. To facilitate this, the following command can be run:

docker image load -i cadenza-image.tar

Configuring Cadenza Web

Defining Options

Settings are defined by environment variables and/or by Java system properties.

Both types of settings are supported and can be added to the ./runcadenzadocker.sh script, in fact the script file already contains examples for setting both environment variables and Java system properties:

Environment variables can be defined by adding an -e option to the ./runcadenzadocker.sh script. For instance, if the environment variable FOO should be set to the value BAR, then the following line needs to be added:
```
-e FOO=BAR \
```
Java system properties can be defined by extending the existing -e JAVA_OPTS="-Xmx2g" \ option with the desired properties. For instance, if the Java system property xyz.abc=true should be added, the existing line with JAVA_OPTS should be extended with the new option like this (note the mandatory -D):
```
-e JAVA_OPTS="-Xmx2g -Dxyz.abc=true" \
```

Adding custom libraries to Tomcat

Sometimes it is necessary to add additional libraries to the Tomcat running Cadenza. In order to allow this, we added the directory /usr/local/tomcat/shared/lib to the shared.loader property in catalina.properties. This allows you to mount an own directory from outside at this place in order to allow access to additional libraries:

Additional part in Docker command to mount an outside directory at /usr/local/tomcat/shared/lib

--mount src="<somedir-with-jars>",target="/usr/local/tomcat/shared/lib",type=bind

Running Cadenza Web

After that, Cadenza Web can be started with

./runcadenzadocker.sh

(which sets up various configuration options and executes docker run).

Cadenza Web runs in the background by default (detached) and neither receives input from the command line nor displays output on it.

Attaching your command line to a running, detached Cadenza Web can be accomplished with
```
docker attach <name>|<id>
```
Shutting down Cadenza Web can be accomplished with
```
docker stop <name>|<id>
```

(<name>|<id> is either the name or the id of the container in question.)

Managing Cadenza

Status, Health & Restart

The health of individual cadenza web instances running in docker can be checked with

docker inspect --format='{{.State.Health.Status}}' <name>|<id>

in which <name>|<id> is either the name or the id of the container in question.

The names and ids of containers can be found with docker container ls, which also displays the status of running containers.

To automatically restart containers on failure, the restart parameter can be added to docker run in runcadwebdocker.sh. For instance, adding

--restart=on-failure:5

means that a container is restarted if it exits with a non-zero exit status, but there will only be 5 tries to restart it before giving up.

See Docker Restart Policies for more information.

Resource Limits

It is recommended to define resource limits for a docker container running Cadenza Web, depending on the specific requirements and deployment scenarios.

To facilitate this, additional options can be added to the docker run command in runcadwebdocker.sh.

The options supported by Docker are described in Docker Resource Constraints, while a more general description of resource control capabilities on Linux exists at Linux cgroups. (Please take note that the official docker documentation only links to documentation of outdated version of the cgroup feature.

Improving Security

There are parameters that can be added to docker run in runcadwebdocker.sh, which can help prevent or mitigate security issues:

--security-opt no-new-privileges prevents processes in the container from obtaining escalated privileges via suid or sgid binaries like su or sudo. See Docker Security Configuration for details.
seccomp (secure computing mode) allows to further restrict available system calls within the container, but requires a Linux kernel that supports this feature. See Seccomp security profiles for Docker for details.

Refer to Docker Security for a general overview of available security options.

Additionally, it is recommended to explicitly bind incoming container traffic to a specific host interface if the machine running the container has multiple network interfaces.

By default the container is bound to 0.0.0.0 (localhost), allowing access from all network interfaces.

To restrict traffic to a specific network interface, the IP address of the network interface can be explicitly added to the -p option in the docker run command in runcadwebdocker.sh.

See Docker Run – Expose for more information.

Commands to check/audit some settings with security implications:

# Avoid privileged containers
# Good: Privileged=false       Bad: Privileged=true
docker ps -q | xargs docker inspect --format '{{ .Id }}: Privileged={{ .HostConfig.Privileged }}'

# Avoid network mode "host" on container
# Good: NetworkMode=default    Bad: NetworkMode=host
docker ps -q | xargs docker inspect --format '{{ .Id }}: NetworkMode={{ .HostConfig.NetworkMode }}'

# Avoid sharing process namespace of host
# Good: PidMode=               Bad: PidMode=host
docker ps -q | xargs docker inspect --format '{{ .Id }}: PidMode={{ .HostConfig.PidMode }}'

Accessing a Running Cadenza Web Instance with JMX

This Cadenza Web Docker image can be configured to allow access to Java Management Extensions (JMX).

To facilitate this, the configuration in the runcadwebdocker.sh has to be adapted, by extending the following JVM configuration line with additional settings:

-e JAVA_OPTS="-Xmx2g " \

The additional settings need to be added after -Xmx2g and before the second ".

-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=9090     // this is the port used to access the JMX interface
-Dcom.sun.management.jmxremote.rmi.port=9090 // can be the same port as the one above
-Djava.rmi.server.hostname=127.0.0.1         // restricts access to users on the same host

To make JMX remote connections available from other machines, the configured port needs to be published when running the container. This can be done by adding the line

--publish 9090:9090 \

to the docker run command in runcadwebdocker.sh

The following settings are only appropriate in a debug environment, as they deactivate any kind of security checks.

-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false

They nonetheless may be useful in some circumstances; for instance to test and establish whether a JMX-based tool is able to connect to the JMX-enabled Cadenza Web docker image before a more complex and secure configuration is set up.

To use JMX in a production environment, it is recommended to follow the section Using SSL of the Monitoring and Management Using JMX Technology document on setting up encryption keys and creating a password file:

Updating Cadenza

Stop the current instance of Cadenza Web by
- executing docker containers ls to find the ID of the running Cadenza Web instance.
- executing docker stop <ID> (where <ID> is replaced by the ID found in the above step).
Load the updated Cadenza Web image with docker image load -i cadenza-image.tar
Compare the existing ./runcadenzadocker.sh script with the version from the update. This is done to verify that settings, options and other customizations that might have been added are preserved and no changes are unintentionally lost or discarded.
Run the new instance with the ./runcadenzadocker.sh script.

Docker Troubleshooting

It is possible to use Docker packages from Linux distributions (e.g. docker.io in Ubuntu) as well as manually installing Docker from upstream sources for a supported distribution (e.g. Install instructions for Ubuntu)
- It is important to keep in mind that Docker officially only supports Ubuntu LTS versions, so if a non-LTS version is used, the install instructions from the Docker website using add-apt-repository in combination with lsb_release -cs will most likely point to the wrong version.
- The most common symptom of this is that the package manager complains about a missing RELEASE file. This can be addressed by replacing the wrong distribution codename in the apt repository list file with the preceding LTS distribution’s code name.
Most likely it will be necessary to add a user to the "docker" group to run docker commands without being root. This can be done with: sudo usermod -a -G docker <username>
On distributions running systemd, the docker container might not have internet connectivity. The most common symptom of this is the failure to retrieve package updates during the docker build. This is caused by systemd-resolved, which generates a /etc/resolv.conf using its own local DNS cache. See "docker container has no internet" for workarounds.
If you mount an external volume into the container where Cadenza needs to access or store files then you must make sure to set the correct permissions. Cadenza runs as the cadenza user of the group cadenza inside the container. The userid is 1000 and the groupid as well. There are a few ways to achieve this. Either (as root) you change ownership to this user (e.g. chown 1000:1000 /directory/tomount) or you make it world writeable (chmod 777 /directory/tomount).