Add admin guide for configuring cluster-wide IPsec encryption #3236
Conversation
| already communicate with each other via the Internet Protocol (IP). The simplest | ||
| way of protecting traffic within an Openshift cluster is to ensure that the | ||
| master and all nodes use IPsec to encrypt their communication. This document | ||
| describes the simplest way to do this. |
There was a problem hiding this comment.
Can we reword the last sentence to avoid repetition of 'simplest'.
Perhaps:
This document describes how to achieve that.
|
|
||
| Now that the configuration has been defined, you need to tell `libreswan` | ||
| what IP subnets and hosts to apply each policy to. This is done through policy | ||
| files in /etc/ipsec.d/policies/ where each configured connection has a |
There was a problem hiding this comment.
Do the paths need to be made into fixed width text?
| what IP subnets and hosts to apply each policy to. This is done through policy | ||
| files in /etc/ipsec.d/policies/ where each configured connection has a | ||
| corresponding policy file. In our example above, we have two connections, | ||
| `private` and `clear` and each will have a file in /etc/ipsec.d/policies. |
There was a problem hiding this comment.
Above policies has a trailing /, here it doesn't. We should be consistent.
| Save the nickname for later. | ||
|
|
||
| The client certificate, CA certificate, and private key files must be combined | ||
| into a PKCS#12 file, which is a common container format for multiple |
There was a problem hiding this comment.
Container is a loaded term w.r.t.openshift/docker. Maybe use something else?
| == Overview | ||
| IPsec is a mechanism for encrypting communication between hosts which can | ||
| already communicate with each other via the Internet Protocol (IP). The simplest | ||
| way of protecting traffic within an Openshift cluster is to ensure that the |
There was a problem hiding this comment.
"OpenShift" (everywhere)
| In this example, we will secure communication of an entire IP subnet from which | ||
| the Openshift hosts receive their IP addresses. Since you are securing the | ||
| host-to-host communication, this automatically includes all cluster management | ||
| and pod data traffic. |
There was a problem hiding this comment.
Management traffic is already over https, so this adds a second layer of encryption...
|
|
||
| [[procedure]] | ||
| == Procedure | ||
| This procedure should be repeated for each host in your cluster. Any host |
There was a problem hiding this comment.
maybe explicitly clarify that "host" means both nodes and masters here
| == Procedure | ||
| This procedure should be repeated for each host in your cluster. Any host | ||
| in the cluster that does not have IPsec enabled will not be able to communicate | ||
| with a host that does. It is suggested to perform these steps on the master |
| By default, Openshift secures cluster management communication with mutually | ||
| authenticated HTTPS communication. This means that both the client (like an | ||
| openshift node) and the server (like the openshift api-server) send each other | ||
| their certificates, which are checked against a know Certificate Authority (CA). |
|
|
||
| /etc/ipsec.d/policies/private should contain the IP subnet of your cluster, | ||
| which your hosts receive IP addresses from. This, by default, will cause all | ||
| communication between hosts in the 172.16.0.0/16 subnet to be encrypted if |
There was a problem hiding this comment.
Makes it sound like regardless of what you put in the file, it will affect traffic to 172.16.0.0/16... I think you rearranged some text here and didn't fully fix up everything for it.
| communication between hosts in the 172.16.0.0/16 subnet to be encrypted if | ||
| the remote host's client certificate authenticates against the local host's | ||
| Certificate Authority certificate. If the remote host's certificate does not | ||
| authenticate, all traffic between the two hosts it will be blocked. |
| 172.16.0.0/16 | ||
| ---- | ||
|
|
||
| Next encryption between all hosts and the subnet gateway must be un-encrypted, |
|
|
||
| ---- | ||
| 172.16.0.0/16 | ||
| ---- |
There was a problem hiding this comment.
Presumably one can add additional IP ranges or specific addresses here? It would be good to explain that.
| from another host in the cluster. | ||
|
|
||
| The most common failure is certificates that do not verify against a common | ||
| Certificate Authority certificate. To verify this: |
There was a problem hiding this comment.
It seems unlikely that this would happen in an OpenShift cluster, since the certificates are being created by automated processes; if people were ending up with node certs that used the wrong CA, then that would cause https failures even if they weren't using ipsec.
|
@openshift/networking PTAL, thanks! |
|
@knobunc done |
| the OpenShift hosts receive their IP addresses. Since you are securing the | ||
| host-to-host communication, this automatically includes all cluster management | ||
| and pod data traffic. Note that as OpenShift management traffic uses HTTPS, | ||
| enabling IPsec will encrypt management traffic a second time. |
There was a problem hiding this comment.
I was pointing that out before as more of a question: do we really want to do that? (Do our customers really want to do that?) Would it be better to just specifically encrypt the currently-unencrypted traffic (which is just VXLAN?) rather than encrypting all traffic between the hosts? (Does libreswan even let us configure something like that?)
(If we do actually want to do this, I don't think it needs to be pointed out in the docs; you can go back to whatever this said before.)
|
On Wed, 30 Nov 2016, Dan Winship wrote:
+In this example, we will secure communication of an entire IP subnet from which
+the OpenShift hosts receive their IP addresses. Since you are securing the
+host-to-host communication, this automatically includes all cluster management
+and pod data traffic. Note that as OpenShift management traffic uses HTTPS,
+enabling IPsec will encrypt management traffic a second time.
I was pointing that out before as more of a question: do we really want to do that? (Do our customers really want to do that?) Would it be better to just specifically encrypt the currently-unencrypted traffic
(which is just VXLAN?) rather than encrypting all traffic between the hosts? (Does libreswan even let us configure something like that?)
(If we do actually want to do this, I don't think it needs to be pointed out in the docs; you can go back to whatever this said before.)
We are working on supporting port exclusion in an easier way that
currently. So I would advise to ignore the double encryption for now.
If you really want to exclud it, you can add excluding port 443 using something
like:
conn passthrough-outgoing-https
left=%defaultroute
leftprotoport=tcp/%any
right=0.0.0.0
rightsubnet=0.0.0.0/0
rightprotoport=tcp/443
auto=route
type=passthrough
priority=1
conn passthrough-incoming-https
left=%defaultroute
leftprotoport=tcp/443
right=0.0.0.0
rightsubnet=0.0.0.0/0
rightprotoport=tcp/%any
priority=1
auto=route
type=passthrough
This will result in high priority kernel policy that will match packets
to/from port 443 and will let that traffic go out in the clear. It has
a higher priority that the regular opportunistic connections.
Paul
|
|
@openshift/team-documentation this has passed tech review. Over to you. Thanks. |
|
I created #3398 as a follow-up to this (for 3.3 ->). I'll close this and continue the edits there, and put that onto peer review. Let me know if there's any issues. |
This document is relevant for OpenShift 3.3 and later.
@openshift/networking