The default shipping configuration of Teamwork Cloud is not a hardened configuration.
When hardening an installation, there are variables that can render the installation inoperative, such as incompatibility of the supported ciphers in a certificate and the supported ciphers in the hardened configuration.
Furthermore, the default configurations assume that the deployment is behind a secure infrastructure, and therefore required ports are globally allowed.
Since some of Teamwork Cloud's infrastructure relies on available components, newly discovered vulnerabilities need to be mitigated during the life-cycle of the installation.
Below, we will cover the potentially exploitable vulnerabilities of the different components, as well as various steps to mitigate depending on the policies of the deploying organization.
Cassandra Port Access
When installing on Linux using our deployment scripts, all of the ports required by Cassandra for inter-node communication, as well as for the Teamwork Cloud nodes to communicate with Cassandra nodes are opened globally. This configuration is deployed mostly to facilitate testing of the environment upon installation, prior to taking any measures to harden the installation. If we check the firewall upon installation, we will see an output similar to the one below:
Code Block
# firewall-cmd --list-all
public (active)
target: default
icmp-block-inversion: no
interfaces: eth0
sources:
services: cassandra lmadmin ssh twcloud
ports:
protocols:
masquerade: no
forward-ports:
source-ports:
icmp-blocks:
rich rules:
In our deployment we create a firewall service definition to facilitate management of the rules. This file is located in/etc/firewalld/services/cassandra.xml, and contains the following:
The first step in securing Cassandra is to limit traffic only to Cassandra and Teamwork Cloud Nodes. In the example below, we have a single node Cassandra/Teamwork Cloud installation, with its IP address set to 10.254.254.56.
As can be seen below, the firewall configuration has been modified to only allow access from itself.
Code Block
# firewall-cmd --list-all
public (active)
target: default
icmp-block-inversion: no
interfaces: eth0
sources:
services: lmadmin ssh twcloud
ports:
protocols:
masquerade: no
forward-ports:
source-ports:
icmp-blocks:
rich rules:
rule family="ipv4" source address="10.254.254.56" service name="cassandra" accept
The process to follow is to remove the general firewall allowance to the Cassandra service. After that, we want to ensure that if direct port rule assignments were made, they are removed. Finally, we want to create a set of rich rules which will allow access only to the required nodes.
If your deployment consists of a single-node or multi-node cluster where both Teamwork Cloud and Cassandra reside on the same nodes, we can automate this process by using the following script.
Code Block
title
harden_cassandra_ports.sh
#!/bin/bash
echo "Hardening Cassandra Ports"
echo "Creating Cassandra firewall service profile"
cat <<EOF | tee /etc/firewalld/services/cassandra.xml &> /dev/null
<?xml version="1.0" encoding="utf-8"?>
<service version="1.0">
<short>cassandra</short>
<description>cassandra</description>
<port port="7000" protocol="tcp"/>
<port port="7001" protocol="tcp"/>
<port port="9042" protocol="tcp"/>
<port port="9160" protocol="tcp"/>
<port port="9142" protocol="tcp"/>
</service>
EOF
echo "Removing existing firewall rules on the cassandra ports or service profile"
sleep 5
zone=$(firewall-cmd --get-default) &> /dev/null
firewall-cmd --zone=$zone --remove-port=7000/tcp --permanent &> /dev/null
firewall-cmd --zone=$zone --remove-port=7001/tcp --permanent &> /dev/null
firewall-cmd --zone=$zone --remove-port=7199/tcp --permanent &> /dev/null
firewall-cmd --zone=$zone --remove-port=9042/tcp --permanent &> /dev/null
firewall-cmd --zone=$zone --remove-port=9160/tcp --permanent &> /dev/null
firewall-cmd --zone=$zone --remove-port=9142/tcp --permanent &> /dev/null
firewall-cmd --zone=$zone --remove-service=cassandra --permanent &> /dev/null
echo "Creating ruch rules for Cassandra nodes discovered via nodetool gossipinfo"
set -f
local_list=($(nodetool gossipinfo | grep '/' | cut -f 2 -d /))
set +f
for i in "${local_list[@]}" ; do
cmd=" firewall-cmd --zone=$zone --add-rich-rule='rule family=\"ipv4\" source address=\"$i\" service name=\"cassandra\" accept' --permanent &> /dev/null"
echo $cmd
eval $cmd
done
firewall-cmd --reload &> /dev/null
The above script is structured in such a way that it will output the rich rule commands to the screen to facilitate copying and modifying to add nodes, which is needed if you have Teamwork Cloud nodes which are not part of the Cassandra cluster.
The above script should be executed on all nodes of a multi-node Cassandra cluster, with all nodes being in an operational state.
Windows Users: Create firewall rules restricting access on the aforementioned ports only to authorized nodes - Cassandra nodes and Teamwork Cloud Nodes.
Replication Strategy
Teamwork Cloud presently does not support Multi-DC replication strategies. As such, the environment must be configured with SimpleStrategy.
Cassandra Authentication
By default, Cassandra is deployed with the AllowAllAuthenticator. This authenticator, as the name implies, is an anonymous authenticator which performs no checks.
If you require authenticated connections, you will need to make changes to yourcassandra.yamland change the authenticator to PasswordAuthenticator. If you are running a multi-node Cassandra cluster, you need to change the replication factor of the system_auth keyspace.
Open source Cassandra does not support encryption at rest. If you require encryption at rest, you need to use a disk encryption layer.
Teamwork Cloud Protocols and Ciphers
Teamwork Cloud consists of 3 Java based services - Teamwork Cloud (twcloud), Authserver (authserver) and WebApp (webapp).
twcloud and authserver require Java 11 (its location varies depending on how it was deployed), whereas webapp uses a bundled Java 14, located in<install_root>/WebAppPlatform/jre/.
Therefore, in order to harden these services we must begin by hardening the JVM. The default settings for the JVM are located injava.security.
We can check the ciphers/protocols being used by the applications using nmap (version 7.x) or TestSSLServer.jar, available fromhttps://community.rsa.com/docs/DOC-45511
As an example, below is a scan using both tools against a default installation. In this example, we will be testing port 8111, the TWCloud port.
Code Block
# nmap --script ssl-enum-ciphers -p 8111 127.0.0.1
Starting Nmap 7.80 ( https://nmap.org ) at 2020-04-21 17:32 MDT
Nmap scan report for localhost (127.0.0.1)
Host is up (0.00014s latency).
PORT STATE SERVICE
8111/tcp open unknown
| ssl-enum-ciphers:
| TLSv1.0:
| ciphers:
| TLS_DHE_RSA_WITH_AES_128_CBC_SHA (dh 1024) - A
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256r1) - A
| TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A
| compressors:
| NULL
| cipher preference: client
| warnings:
| Key exchange (dh 1024) of lower strength than certificate key
| TLSv1.1:
| ciphers:
| TLS_DHE_RSA_WITH_AES_128_CBC_SHA (dh 1024) - A
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256r1) - A
| TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A
| compressors:
| NULL
| cipher preference: client
| warnings:
| Key exchange (dh 1024) of lower strength than certificate key
| TLSv1.2:
| ciphers:
| TLS_DHE_RSA_WITH_AES_128_CBC_SHA (dh 1024) - A
| TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 (dh 1024) - A
| TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (dh 1024) - A
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (secp256r1) - A
| TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A
| TLS_RSA_WITH_AES_128_CBC_SHA256 (rsa 2048) - A
| TLS_RSA_WITH_AES_128_GCM_SHA256 (rsa 2048) - A
| compressors:
| NULL
| cipher preference: client
| warnings:
| Key exchange (dh 1024) of lower strength than certificate key
|_ least strength: A
Nmap done: 1 IP address (1 host up) scanned in 0.76 seconds
# java -jar TestSSLServer.jar 127.0.0.1 8111
Supported versions: TLSv1.0 TLSv1.1 TLSv1.2
Deflate compression: no
Supported cipher suites (ORDER IS NOT SIGNIFICANT):
TLSv1.0
RSA_WITH_AES_128_CBC_SHA
DHE_RSA_WITH_AES_128_CBC_SHA
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
(TLSv1.1: idem)
TLSv1.2
RSA_WITH_AES_128_CBC_SHA
DHE_RSA_WITH_AES_128_CBC_SHA
RSA_WITH_AES_128_CBC_SHA256
DHE_RSA_WITH_AES_128_CBC_SHA256
TLS_RSA_WITH_AES_128_GCM_SHA256
TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
----------------------
Server certificate(s):
a241e1c3b957bd14e6e0242fd012f75853eef243: CN=X.X.X.X
----------------------
Minimal encryption strength: strong encryption (96-bit or more)
Achievable encryption strength: strong encryption (96-bit or more)
BEAST status: vulnerable
CRIME status: protected
As can be observed above, the default configuration using OpenJDK 1.8.0_242 is allowing TLS v1.0 and v1.1, which are deprecated. Additionally, we can see that several key exchanges are taking place using dh1024.
We then proceed to harden the configuration.
Since we are dealing with ciphers, you need to make sure that you do not disable a cipher required by your certificates.
After hardening the VM, we end up with a different set of allowed ciphers and protocols, as shown below.
Code Block
# nmap --script ssl-enum-ciphers -p 8111 127.0.0.1
Starting Nmap 7.70 ( https://nmap.org ) at 2020-04-21 17:44 MDT
Nmap scan report for localhost (127.0.0.1)
Host is up (0.00015s latency).
PORT STATE SERVICE
8111/tcp open unknown
| ssl-enum-ciphers:
| TLSv1.2:
| ciphers:
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (secp256r1) - A
| TLS_RSA_WITH_AES_128_CBC_SHA256 (rsa 2048) - A
| TLS_RSA_WITH_AES_128_GCM_SHA256 (rsa 2048) - A
| compressors:
| NULL
| cipher preference: client
|_ least strength: A
Nmap done: 1 IP address (1 host up) scanned in 0.95 seconds
# java -jar TestSSLServer.jar 127.0.0.1 8111
Supported versions: TLSv1.2
Deflate compression: no
Supported cipher suites (ORDER IS NOT SIGNIFICANT):
TLSv1.2
RSA_WITH_AES_128_CBC_SHA256
TLS_RSA_WITH_AES_128_GCM_SHA256
----------------------
Server certificate(s):
71ed3969e41a94877c51aacb87d995af4a12b6d9: CN=x.x.x.x
----------------------
Minimal encryption strength: strong encryption (96-bit or more)
Achievable encryption strength: strong encryption (96-bit or more)
BEAST status: protected
CRIME status: protected
The process of hardening the JVM requires making some changes to thejava.securityfile. While these can be made directly, the downside is that if you upgrade your JVM, you will have to reapply your changes.
However, we can place our modifications in our own file, and simply pass a parameter to the JVM upon invocation so that it will apply our changes.
For example, we can create a file/home/twcloud/twc.java.security, and pass a parameter to the JVM in the form of-Djava.security.properties=/home/twcloud/twc.java.security.
Our hardened security settings are as shown below:
On Windows, you need to edit the registry keyComputer\HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Apache Software Foundation\Procrun 2.0\TeamworkCloud\Parameters\Java\Optionsand append the setting pointing to your security overrides to the bottom of the settings.
For the Authserver service, you need to edit<install_root>/AuthServer/authserver-runby inserting the directive before the call to theauthentication-server-XXXXXX.jaras shown below:
On Windows, you need to edit the registry keyComputer\HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Apache Software Foundation\Procrun 2.0\AuthServer\Parameters\Java\Optionsand append the setting pointing to your security overrides to the bottom of the settings.
For the Webapp service, under Linux you need to edit<install_root>/WebAppPlatform/bin/setenv.shand add the directive to theJVM_OPTSvariable as shown below:
On Windows, you need to edit the registry keyComputer\HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Apache Software Foundation\Procrun 2.0\WebApp\Parameters\Java\Optionsand append the setting pointing to your security overrides to the bottom of the settings.
JMX
By default, the twcloud service activates a JMX remote port to facilitate application monitoring. The default configuration does not contain any form of authentication.
On Linux, the configuration is located in<install_root>/jvm.options.
On Windows, it is located in registry keyComputer\HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Apache Software Foundation\Procrun 2.0\TeamworkCloud\Parameters\Java\Options.
These settings can be removed, thereby removing JMX remote access.
If you would like to allow remote JMX access, but require authentication, you can do so by adding settings. For complete documentation, please refer to the Java documentation.
As an example, the below configuration adds password authentication:
As can be seen, we are pointing to a set of files (/home/twcloud/jmx.passwordand/home/twcloud/jmx.access) which control who can access these files.
The vulnerability vector is one whereby JMX could be exploited to execute code. To prevent this, we allow only an authenticated user (jmx.password) who has read-only rights (jmx.access).
Code Block
title
jmx.password
monitoring DqzbksT4ET
Code Block
title
jmx.access
monitoring readonly
In this example, we created a user (monitoring) with a password (DqzbksT4ET), who can only read values via Remote JMX, but cannot write or execute anything via JMX.
Warning
The password and access files have a very stringent ownership requirement. They need to be owned by the user running the process, and be accessible exclusively to that user.
For example, in our default installation, the twcloud user is running the twcloud service. Therefore, the files need to be owned by twcloud, and have full rights (rwx) by twcloud, and only twcloud.
The Webapp Platform (webapp), as deployed by our installer, runs on a bundled Apache Tomcat. As such, best practices for hardening Apache Tomcat should be followed.
Although we have already constrained ciphers and protocols at the JVM level, it is best practice to do so at the Tomcat configuration level. We also need to address issues such as secure cookies, disable XSS on foreign sites, and also remove default directories published as part of the default installation. The official Tomcat documentation covers a large portion of this (recommended reading) -https://tomcat.apache.org/tomcat-9.0-doc/security-howto.html. Additionally, there are a plethora of documents online covering all aspects of securing Tomcat to OWASP standards.
server.xml
There are various changes which can be made to<install_root>/WebAppPlatform/conf/server.xmlin order to harden the system.
The first step (do not do this if running on Windows) is to disable the shutdown port.