Skip to content

Commit 633e8f5

Browse files
dcbwdcbw
committed
Add admin guide for configuring cluster-wide IPsec encryption
1 parent a863c80 commit 633e8f5

File tree

2 files changed

+176
-0
lines changed

2 files changed

+176
-0
lines changed

‎_topic_map.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -497,6 +497,9 @@ Topics:
497497
- Name: IPtables
498498
File: iptables
499499
Distros: openshift-origin,openshift-enterprise
500+
- Name: IPsec
501+
File: ipsec
502+
Distros: openshift-origin,openshift-enterprise
500503
- Name: Securing Builds by Strategy
501504
File: securing_builds
502505
Distros: openshift-origin,openshift-enterprise

‎admin_guide/ipsec.adoc

Lines changed: 173 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,173 @@
1+
[[admin-guide-ipsec]]
2+
= ipsec
3+
{product-author}
4+
{product-version}
5+
:data-uri:
6+
:icons:
7+
:experimental:
8+
:toc: macro
9+
:toc-title:
10+
11+
toc::[]
12+
13+
== Overview
14+
IPsec is a mechanism for encrypting communication between hosts which can
15+
already communicate with each other via the Internet Protocol (IP). The simplest
16+
way of protecting traffic within an OpenShift cluster is to ensure that the
17+
master and all nodes use IPsec to encrypt their communication. This document
18+
describes how to achieve this.
19+
20+
In this example, we will secure communication of an entire IP subnet from which
21+
the OpenShift hosts receive their IP addresses. Since you are securing the
22+
host-to-host communication, this automatically includes all cluster management
23+
and pod data traffic. Note that as OpenShift management traffic uses HTTPS,
24+
enabling IPsec will encrypt management traffic a second time.
25+
26+
[[requirements]]
27+
== Requirements
28+
This guide requires that you have the `libreswan` package, version 3.19 or later,
29+
installed on cluster hosts. Only 3.19 and later include the necessary
30+
opportunistic group functionality that allows hosts to be configured without
31+
knowledge of every other host in the cluster.
32+
33+
[[procedure]]
34+
== Procedure
35+
This procedure should be repeated for each host (both nodes and masters) in your
36+
cluster. Any host in the cluster that does not have IPsec enabled will not be
37+
able to communicate with a host that does. It is suggested to perform these steps
38+
on masters first, and then each node.
39+
40+
[[certificates]]
41+
=== Certificates
42+
By default, OpenShift secures cluster management communication with mutually
43+
authenticated HTTPS communication. This means that both the client (like an
44+
openshift node) and the server (like the openshift api-server) send each other
45+
their certificates, which are checked against a known Certificate Authority (CA).
46+
These certificates are generated at cluster setup time and typically live on
47+
each host.
48+
49+
These certificates can also be used to secure pod communications with IPsec. You
50+
need three files on each host:
51+
52+
- cluster Certificate Authority file
53+
- host client certificate file
54+
- host private key file
55+
56+
First, determine what the certificate's nickname will be after it has been
57+
imported into the `libreswan` certificate database. The nickname is taken
58+
directly from the certificate's subject's Common Name (CN):
59+
60+
----
61+
openssl x509 -in /path/to/client-certificate -subject -noout | sed -n 's/.*CN=\(.*\)/\1/p'
62+
----
63+
64+
Save the nickname for later.
65+
66+
The client certificate, CA certificate, and private key files must be combined
67+
into a PKCS#12 file, which is a common file format for multiple
68+
certificates and keys. To do this, you can use the `openssl` program:
69+
70+
----
71+
openssl pkcs12 -export \
72+
-in /path/to/client-certificate \
73+
-inkey /path/to/private-key \
74+
-certfile /path/to/certificate-authority \
75+
-passout pass: \
76+
-out certs.p12
77+
----
78+
79+
This PKCS#12 file must then be imported into the `libreswan` certificate
80+
database. The -W option is left empty because we do not need to assign a
81+
password to the PKCS#12 file as it is only temporary.
82+
83+
----
84+
ipsec initnss
85+
pk12util -i certs.p12 -d sql:/etc/ipsec.d -W ""
86+
rm certs.p12
87+
----
88+
89+
[[ipsec-policy]]
90+
=== libreswan IPsec Policy
91+
92+
Now that the necessary certificates have been imported into the `libreswan`
93+
certificate database, you can create a policy that uses them to secure
94+
communication between hosts in your cluster. The following configuration
95+
creates two `libreswan` connections. The first encrypts traffic using the
96+
OpenShift certificates, while the second creates exceptions to the encryption
97+
for cluster-external traffic.
98+
99+
Place the following text into the file `/etc/ipsec.d/openshift-cluster.conf`:
100+
101+
----
102+
conn private
103+
left=%defaultroute
104+
leftid=%fromcert
105+
# our certificate
106+
leftcert="NSS Certificate DB:openshift-node-2" <1>
107+
right=%opportunisticgroup
108+
rightid=%fromcert
109+
# their certificate transmitted via IKE
110+
rightca=%same
111+
ikev2=insist
112+
authby=rsasig
113+
failureshunt=drop
114+
negotiationshunt=hold
115+
auto=ondemand
116+
117+
conn clear
118+
left=%defaultroute
119+
right=%group
120+
authby=never
121+
type=passthrough
122+
auto=route
123+
priority=100
124+
----
125+
<1> replace the text after the colon (eg `openshift-node-2`) with the certificate
126+
nickname you saved above. For example, on a different host, the full line might be
127+
`leftcert="NSS Certificate DB:openshift-master"`.
128+
129+
Now that the configuration has been defined, you need to tell `libreswan`
130+
what IP subnets and hosts to apply each policy to. This is done through policy
131+
files in `/etc/ipsec.d/policies/` where each configured connection has a
132+
corresponding policy file. In our example above, we have two connections,
133+
`private` and `clear` and each will have a file in `/etc/ipsec.d/policies/`.
134+
135+
`/etc/ipsec.d/policies/private` should contain the IP subnet of your cluster,
136+
which your hosts receive IP addresses from. This, by default, will cause all
137+
communication between hosts in the cluster subnet to be encrypted if
138+
the remote host's client certificate authenticates against the local host's
139+
Certificate Authority certificate. If the remote host's certificate does not
140+
authenticate, all traffic between the two hosts will be blocked.
141+
142+
For example, if all your hosts are configured to use addresses in the
143+
172.16.0.0/16 address space, your `private` policy file would contain:
144+
145+
----
146+
172.16.0.0/16 <1>
147+
----
148+
<1> any number of additional subnets to encrypt may be added to this file, which
149+
will result in all traffic to those subnets using IPsec as well.
150+
151+
Next, encryption between all hosts and the subnet gateway must be un-encrypted,
152+
to ensure that traffic can enter and exit the cluster. To do this, add the
153+
gateway to the `/etc/ipsec.d/policies/clear` file like so:
154+
155+
----
156+
172.16.0.1/32 <1>
157+
----
158+
<1> additional hosts and subnets may be added to this file, which will result in
159+
all traffic to these hosts and subnets being unencrypted.
160+
161+
Finally, restart the `libreswan` service to load the new configuration and
162+
policies, and begin encrypting:
163+
164+
----
165+
systemctl restart ipsec
166+
----
167+
168+
== Troubleshooting
169+
When authentication cannot be completed between two hosts, you will not even
170+
be able to ping between them as all IP traffic will be rejected. If the `clear`
171+
policy is not configured correctly, you will also not be able to SSH to the host
172+
from another host in the cluster. You can use the 'ipsec status' command to check
173+
that the `clear` and `private` policies have been loaded.

0 commit comments

Comments
 (0)