Sysdig Monitor On-Prem Installation Troubleshooting

Docker Connectivity Issues (IPv4/IPv6)

We have seen some issues with IPv4 and IPv6 interconnectivity issues between our on-premises containers and the outside world.

IP packet forwarding is governed by the ip_forward system parameter. Packets can only pass between containers if this parameter is 1. Usually, you will simply leave the Docker server at its default setting --ip-forward=true and Docker will go set ip_forward to 1 for you when the server starts up. If you set --ip-forward=false and your system’s kernel has it enabled, the --ip-forward=false option has no effect.

To check the setting on your kernel use:

sysctl net.ipv4.conf.all.forwarding

To turn it on use:

sysctl net.ipv4.conf.all.forwarding=1

Please see this article from docker for more details on Docker Connectivity.

 

Proxy/Firewall Issues

Prior to installing ensure your proxy settings are valid for the session.  You can curl, lynx, or wget to test internet connectivity: 

export http_proxy="http://user:password@proxy_server:port" 
export https_proxy="https://user:password@proxy_server:port"
echo $http_proxy

You can then attempt a curl or docker hub call to ensure outside connectivity

 

Firewall

Prior to installation, you may want to disable local firewall (iptables) to rule-out local connectivity issues. However here are some details around Sysdig connectivity and backend connectivity requirements.

Sysdig Connectivity:


6443  Agent communication
  443  Sysdig Monitor UI access
8800  Mananagement console access

Here are specifics around what is used for connectivity for our backend for on-premises Solution:
https://www.replicated.com/docs/kb/supporting-your-customers/firewalls/

 

File Write Permissions Issues (SELINUX or APP ARMOR)

During the install, you may see errors writing to volumes such as (/var or /opt) from either the onprem install scripts or Docker.  You should disable SELINUX (CENTOS/RHEL) or Apparmor (UBUNTU/DEBIAN) during the course of install so the valid directories can be created.  This can be accomplished by:

Centos (SELINUX)

From the command line, edit the /etc/sysconfig/selinux file. This file is a symlink to /etc/selinux/config. The configuration file is self-explanatory. Changing the value of SELINUX or SELINUXTYPEchanges the state of SELinux and the name of the policy to be used the next time the system boots.

[root@host2a ~]# cat /etc/sysconfig/selinux
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#       enforcing - SELinux security policy is enforced.
#       permissive - SELinux prints warnings instead of enforcing.
#       disabled - SELinux is fully disabled.
SELINUX=permissive
# SELINUXTYPE= type of policy in use. Possible values are:
#       targeted - Only targeted network daemons are protected.
#       strict - Full SELinux protection.
SELINUXTYPE=targeted

# SETLOCALDEFS= Check local definition changes
SETLOCALDEFS=0

More info is available here.

 

UBUNTU/Debian (AppArmor)

AppArmor can be disabled, and the kernel module unloaded by entering the following:

sudo systemctl stop apparmor.service
sudo update-rc.d -f apparmor remove

To re-enable AppArmor enter:

sudo systemctl start apparmor.service
sudo update-rc.d apparmor defaults

 

Advanced Troubleshooting - Firewall, IPtables, IP forwarding

In the preflight check step with Replicated, if you come across the error:

getsockopt: no route to host

Please do the following:

For CentOS 7/RedHat:

Log in as root or run these commands via sudo:

service firewalld stop 
systemctl disable firewalld
sysctl -w net.ipv4.ip_forward=1
iptables -F
setenforce 0
service docker restart

For Ubuntu:

Log in as root or run these commands via sudo:

sysctl -w net.ipv4.ip_forward=1
systemctl stop apparmor.service
update-rc.d -f apparmor remove
ufw disable
iptables -F
service docker restart
Have more questions? Submit a request