Openshift integrates Project Calico

Endocode contributed to improve OpenShift-Ansible Calico roles and configurability of Project Calico

by Vincent Schwarzer | August 14, 2017

Today was announced that with version OpenShift 3.6, Project Calico is fully supported as a network solution. Endocode and idealo Internet GmbH contributed to this integration.

We started a project in mid of April in which we would use RedHat OpenShift Origin with Project Calico on BareMetal as the solution to move internal applications to a container based infrastructure.

At this point in time the integration of Project Calico into the RedHat OpenShift-Ansible installer was in its early stages. Over the course of the project, we rolled out and tested RedHat OpenShift Origin with Project Calico. In cooperation with internal development teams prototypical migrations of selected applications to the OpenShift platform have been performed. This served as a starting point to implement full Calico support in OpenShift.

OpenShift officially supports Red Hat Enterprise Linux (RHEL) and CentOS as Linux distributions. These Linux distributions and Fedora are using firewalld as their default firewall solution.

One of the first issues we encountered was related to the fact that Project Calico is using Border Gateway Protocol (BGP)1 to distribute routes between nodes.2 The route distribution in BGP requires access to port 179/tcp. Another similar issue was that the docker registry provided by OpenShift couldn’t be reached because it requires access to port 5000/tcp.

By default firewalld is blocking the traffic on all ports and ports have to be whitelisted individually. To configure firewalld the command line tool firewall-cmd is used. Firewalld allows to define different zones that can be associated with different network interfaces and their own firewall rules.

Without any additional configuration all interfaces / rules are associated with the public zone. Whitelisting of a port in firewalld can be done by:

firewall-cmd --list-all-zones //show configuration of all firewalld zones
firewall-cmd --permanent --zone public --add-port 179/tcp
//add rule to allow port 179 for TCP

The flag --permanent is required so that the rule is still in place when the firewalld service is restarted. These rules can be defined in openshift-ansible as well.

Beyond those fixes we added the possibility to configure the Project Calico Ansible roles using Ansible inventory variables.3

In Project Calico itself we integrated that NAT-Outgoing in Default IPPools can be configured at startup with flags / environment variables which was a requirement of our customer.4

Many thanks to the maintainers of Project Calico and RedHat OpenShift-Ansible who were super-helpful and worked closely together with us over the course of the project. All the mentioned changes were made available upstream as part of our commitment to use and contribute to OpenSource projects.

Image credit: Alan Levine, CC-BY 2.0