Skip to content

Commit 5ed7ea6

Browse files
author
Traci Morrison
authored
Merge pull request #9633 from tsmetana/local-block-volume-3.10
Local volume: add raw block devices support
2 parents f77f2c3 + a02776e commit 5ed7ea6

File tree

1 file changed

+235
-2
lines changed

1 file changed

+235
-2
lines changed

‎install_config/configuring_local.adoc

+235-2
Original file line numberDiff line numberDiff line change
@@ -148,12 +148,12 @@ ifdef::openshift-origin[]
148148
-p PROVISIONER_IMAGE=quay.io/external_storage/local-volume-provisioner:v1.0.1 \
149149
endif::[]
150150
ifndef::openshift-origin[]
151-
-p PROVISIONER_IMAGE=registry.access.redhat.com/openshift3/local-storage-provisioner:v3.9 \ <1>
151+
-p PROVISIONER_IMAGE=registry.access.redhat.com/openshift3/local-storage-provisioner:v3.10 \ <1>
152152
endif::[]
153153
local-storage-provisioner
154154
----
155155
ifndef::openshift-origin[]
156-
<1> Replace `v3.9` with the right {product-title} version.
156+
<1> Replace `v3.10` with the right {product-title} version.
157157
+
158158
endif::[]
159159

@@ -208,3 +208,236 @@ $ chcon -R unconfined_u:object_r:svirt_sandbox_file_t:s0 /mnt/local-storage/
208208
====
209209
Omitting any of these steps may result in the wrong PV being created.
210210
====
211+
212+
[[local-volume-raw-block-devices]]
213+
=== Raw Block Devices
214+
It is possible to statically provision also raw block devices using the local
215+
volume provisioner. This feature is disabled by default and requires additional
216+
configuration.
217+
218+
. Enable the `BlockVolume` feature gate on all masters.
219+
Edit or create the master configuration file on all masters
220+
(*_/etc/origin/master/master-config.yaml_* by default) and add `BlockVolume=true`
221+
under the `apiServerArguments` and `controllerArguments` sections:
222+
+
223+
[source, yaml]
224+
----
225+
apiServerArguments:
226+
feature-gates:
227+
- BlockVolume=true
228+
...
229+
230+
controllerArguments:
231+
feature-gates:
232+
- BlockVolume=true
233+
...
234+
----
235+
236+
. Enable the feature gate on all nodes by editing the node configuration `ConfigMap`:
237+
+
238+
[source, bash]
239+
----
240+
$ oc edit configmap node-config-compute --namespace openshift-node
241+
$ oc edit configmap node-config-master --namespace openshift-node
242+
$ oc edit configmap node-config-infra --namespace openshift-node
243+
----
244+
+
245+
Ensure all the configmaps contain `BlockVolume=true` in the feature-gates
246+
array of the `kubeletArguments`:
247+
+
248+
.Example node configmap feature-gates setting
249+
[source, yaml]
250+
----
251+
kubeletArguments:
252+
feature-gates:
253+
- RotateKubeletClientCertificate=true,RotateKubeletServerCertificate=true,BlockVolume=true
254+
----
255+
256+
. Restart the master. The nodes should be restarted automatically after the
257+
configuration change (it may take several minutes).
258+
259+
[[local-volume-prepare-block-devices]]
260+
==== Prepare the Block Devices
261+
Before starting the provisioner all the block devices that should be available
262+
to the pods need to be linked to the *_/mnt/local-storage/<storage class>_* directory
263+
structure. Example: to make a device *_/dev/dm-36_* available:
264+
265+
. Create a directory for its `StorageClass` in *_/mnt/local-storage_*:
266+
+
267+
[source, bash]
268+
----
269+
$ mkdir -p /mnt/local-storage/block-devices
270+
----
271+
272+
. Create a symbolic link that would point to the device:
273+
+
274+
[source, bash]
275+
----
276+
$ ln -s /dev/dm-36 dm-uuid-LVM-1234
277+
----
278+
+
279+
[NOTE]
280+
====
281+
It is a good practice to use the same name for the symbolic link as the link
282+
from *_/dev/disk/by-uuid_* or *_/dev/disk/by-id_* directory to avoid possible name conflicts.
283+
====
284+
285+
. Create or update the `ConfigMap` configuring the provisioner:
286+
+
287+
[source, yaml]
288+
----
289+
kind: ConfigMap
290+
metadata:
291+
name: local-volume-config
292+
data:
293+
storageClassMap: |
294+
block-devices: <1>
295+
hostDir: /mnt/local-storage/block-devices <2>
296+
mountDir: /mnt/local-storage/block-devices <3>
297+
----
298+
<1> Name of the StorageClass.
299+
<2> Path to the directory on the host. It must be a subdirectory of *_/mnt/local-storage_*.
300+
<3> Path to the directory in the provisioner pod. We recommend using the same directory structure as used on the host and `mountDir` can be omitted in this case.
301+
. Change the SELinux label of the device and the *_/mnt/local-storage/_*:
302+
+
303+
[source, bash]
304+
----
305+
$ chcon -R unconfined_u:object_r:svirt_sandbox_file_t:s0 /mnt/local-storage/
306+
$ chcon unconfined_u:object_r:svirt_sandbox_file_t:s0 /dev/dm-36
307+
----
308+
309+
. Create `StorageClass` for the block-devices:
310+
+
311+
[source, yaml]
312+
----
313+
apiVersion: storage.k8s.io/v1
314+
kind: StorageClass
315+
metadata:
316+
name: block-devices
317+
provisioner: kubernetes.io/no-provisioner
318+
volumeBindingMode: WaitForFirstConsumer
319+
----
320+
321+
Now the block device *_/dev/dm-36_* is ready to be used by the provisioner and
322+
provisioned as a PV.
323+
324+
==== Deploy the Provisioner
325+
The deployment of the provisioner is similar as with the usual filesytem-type of
326+
volumes. There are two differences: the provisioner needs to be run in a
327+
privileged container and have access to the *_/dev_* filesystem from the host.
328+
Download the template from the
329+
link:https://raw.githubusercontent.com/openshift/origin/release-3.9/examples/storage-examples/local-examples/local-storage-provisioner-template.yaml[*_local-storage-provisioner-template.yaml_*]
330+
file and make the following changes:
331+
332+
. Set the `privileged` attribute of the `securityContext` of the container spec
333+
to `true`:
334+
+
335+
[source, yaml]
336+
----
337+
...
338+
containers:
339+
...
340+
name: provisioner
341+
...
342+
securityContext:
343+
privileged: true
344+
...
345+
----
346+
347+
. Ensure the host *_/dev/_* filesystem would be mounted into the container using `hostPath`:
348+
+
349+
[source, yaml]
350+
----
351+
...
352+
containers:
353+
...
354+
name: provisioner
355+
...
356+
volumeMounts:
357+
- mountPath: /dev
358+
name: dev
359+
...
360+
volumes:
361+
- hostPath:
362+
path: /dev
363+
name: dev
364+
...
365+
----
366+
367+
. Create the template from the modified yaml file:
368+
+
369+
[source, bash]
370+
----
371+
$ oc create -f local-storage-provisioner-template.yaml
372+
----
373+
374+
. Start the provisioner the same way as in the case without the block devices
375+
support:
376+
+
377+
[source, bash]
378+
----
379+
$ oc new-app -p CONFIGMAP=local-volume-config \
380+
-p SERVICE_ACCOUNT=local-storage-admin \
381+
-p NAMESPACE=local-storage \
382+
ifdef::openshift-origin[]
383+
-p PROVISIONER_IMAGE=quay.io/external_storage/local-volume-provisioner:v1.0.1 \
384+
endif::[]
385+
ifndef::openshift-origin[]
386+
-p
387+
PROVISIONER_IMAGE=registry.access.redhat.com/openshift3/local-storage-provisioner:v3.10 \
388+
endif::[]
389+
local-storage-provisioner
390+
----
391+
392+
==== Using the raw block PV
393+
To use the block device in the pod, create a PVC with `volumeMode: Block` and
394+
the `storageClass` of the block device:
395+
396+
[source, yaml]
397+
----
398+
apiVersion: v1
399+
kind: PersistentVolumeClaim
400+
metadata:
401+
name: block-pvc
402+
spec:
403+
storageClassName: block-devices
404+
accessModes:
405+
- ReadWriteOnce
406+
volumeMode: Block
407+
resources:
408+
requests:
409+
storage: 1Gi
410+
----
411+
412+
An example pod using the block device PVC:
413+
414+
[source, yaml]
415+
----
416+
apiVersion: v1
417+
kind: Pod
418+
metadata:
419+
name: busybox-test
420+
labels:
421+
name: busybox-test
422+
spec:
423+
restartPolicy: Never
424+
containers:
425+
- resources:
426+
limits :
427+
cpu: 0.5
428+
image: gcr.io/google_containers/busybox
429+
command:
430+
- "/bin/sh"
431+
- "-c"
432+
- "while true; do date; sleep 1; done"
433+
name: busybox
434+
volumeDevices:
435+
- name: vol
436+
devicePath: /dev/xvda
437+
volumes:
438+
- name: vol
439+
persistentVolumeClaim:
440+
claimName: block-pvc
441+
----
442+
Note the volume is not mounted in the pod but exposed as the
443+
*_/dev/xvda_* block device.

0 commit comments

Comments
 (0)