Skip to content

Commit 0888b96

Browse files
committed
Updates for OCP 3.4 cluster upgrade process (automated+manual in-place)
Also logging + cluster upgrade tweaks per QE feedback Add etcd v2 API note to release notes Add manual etcd upgrade steps (RPM+containerized) Add upgrade steps for excluder pkgs
1 parent 180a163 commit 0888b96

File tree

7 files changed

+669
-311
lines changed

7 files changed

+669
-311
lines changed

_topic_map.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -38,8 +38,8 @@ Distros: openshift-enterprise,openshift-dedicated
3838
Topics:
3939
- Name: Overview
4040
File: index
41-
- Name: OpenShift Container Platform 3.3 Release Notes
42-
File: ocp_3_3_release_notes
41+
- Name: OpenShift Container Platform 3.4 Release Notes
42+
File: ocp_3_4_release_notes
4343
Distros: openshift-enterprise
4444
- Name: Latest Product Updates
4545
File: osd_latest_product_updates

install_config/install/disconnected_install.adoc

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,8 @@
22
= Disconnected Installation
33
{product-author}
44
{product-version}
5-
:latest-tag: v3.3.1.5
6-
:latest-int-tag: 3.3.1
5+
:latest-tag: v3.4.0.37
6+
:latest-int-tag: 3.4.0
77
:data-uri:
88
:icons:
99
:experimental:

install_config/upgrading/automated_upgrades.adoc

Lines changed: 203 additions & 60 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
= Performing Automated In-place Cluster Upgrades
33
{product-author}
44
{product-version}
5-
:latest-tag: v3.3.1.5
5+
:latest-tag: v3.4.0.37
66
:data-uri:
77
:icons:
88
:experimental:
@@ -28,8 +28,8 @@ endif::[]
2828
The automated upgrade performs the following steps for you:
2929

3030
* Applies the latest configuration.
31-
* Upgrades and restart master services.
32-
* Upgrades and restart node services.
31+
* Upgrades master and etcd components and restarts services.
32+
* Upgrades node components and restarts services.
3333
* Applies the latest cluster policies.
3434
* Updates the default router if one exists.
3535
* Updates the default registry if one exists.
@@ -66,7 +66,7 @@ To upgrade from OpenShift Origin 1.0 to 1.1, run the following playbook:
6666

6767
----
6868
# ansible-playbook \
69-
[-i </path/to/inventory/file>] \
69+
-i </path/to/inventory/file> \
7070
playbooks/byo/openshift-cluster/upgrades/v3_0_to_v3_1/upgrade.yml
7171
----
7272

@@ -89,7 +89,7 @@ run the following playbook:
8989

9090
----
9191
# ansible-playbook \
92-
[-i </path/to/inventory/file>] \
92+
-i </path/to/inventory/file> \
9393
playbooks/byo/openshift-cluster/upgrades/v3_1_minor/upgrade.yml
9494
----
9595

@@ -110,32 +110,47 @@ ifdef::openshift-enterprise[]
110110

111111
[IMPORTANT]
112112
====
113-
Before upgrading your cluster to {product-title} 3.3, the cluster must be
113+
Before upgrading your cluster to {product-title} 3.4, the cluster must be
114114
already upgraded to the
115-
link:https://docs.openshift.com/enterprise/3.2/release_notes/ose_3_2_release_notes.html#ose-32-asynchronous-errata-updates[latest asynchronous release of version 3.2]. Cluster upgrades cannot span more than one
116-
minor version at a time, so if your cluster is at version 3.0 or 3.1, you must
117-
first upgrade incrementally (e.g., 3.0 to 3.1, then 3.1 or 3.2).
115+
link:https://docs.openshift.com/container-platform/3.3/release_notes/ocp_3_3_release_notes.html#ocp-33-asynchronous-errata-updates[latest asynchronous release of version 3.3]. Cluster upgrades cannot span more than one
116+
minor version at a time, so if your cluster is at a version earlier than 3.3,
117+
you must first upgrade incrementally (e.g., 3.1 to 3.2, then 3.2 to 3.3).
118118
====
119119

120120
To prepare for an automated upgrade:
121121

122-
. If you are upgrading from version 3.2 to 3.3, manually disable the 3.2 channel and enable the 3.3 channel on each master and node host:
122+
. If you are upgrading from {product-title} 3.3 to 3.4, manually disable the 3.3
123+
channel and enable the 3.4 channel on each master and node host:
123124
+
124125
----
125-
# subscription-manager repos --disable="rhel-7-server-ose-3.2-rpms" \
126-
--enable="rhel-7-server-ose-3.3-rpms"\
126+
# subscription-manager repos --disable="rhel-7-server-ose-3.3-rpms" \
127+
--enable="rhel-7-server-ose-3.4-rpms"\
127128
--enable="rhel-7-server-extras-rpms"
128129
# yum clean all
129130
----
131+
130132
. For any upgrade path, always ensure that you have the latest version of the
131-
*atomic-openshift-utils* package, which should also update the
133+
*atomic-openshift-utils* package on each RHEL 7 system, which also updates the
132134
*openshift-ansible-** packages:
133135
+
134136
----
135137
# yum update atomic-openshift-utils
136138
----
137-
. Lastly, you must be logged in as a cluster administrative user on the master
138-
host for the upgrade to succeed:
139+
140+
. Install or update to the following latest available **-excluder* packages on
141+
each RHEL 7 system, which helps ensure your systems stay on the correct versions
142+
of *atomic-openshift* and *docker* packages when you are not trying to upgrade,
143+
according to the {product-title} version:
144+
+
145+
----
146+
# yum install atomic-openshift-excluder atomic-openshift-docker-excluder
147+
----
148+
+
149+
These packages add entries to the `exclude` directive in the host's
150+
*_/etc/yum.conf_* file.
151+
152+
. You must be logged in as a cluster administrative user on the master host for
153+
the upgrade to succeed:
139154
+
140155
----
141156
$ oc login
@@ -159,8 +174,8 @@ you should have an installation configuration file located at
159174
start an upgrade.
160175

161176
The installer supports upgrading between minor versions of {product-title}
162-
(one minor version at a time, e.g., 3.2 to 3.3) as well as between
163-
xref:../../release_notes/ocp_3_3_release_notes.adoc#ocp-33-asynchronous-errata-updates[asynchronous errata updates] within a minor version (e.g., 3.3.z).
177+
(one minor version at a time, e.g., 3.3 to 3.4) as well as between
178+
xref:../../release_notes/ocp_3_3_release_notes.adoc#ocp-33-asynchronous-errata-updates[asynchronous errata updates] within a minor version (e.g., 3.4.z).
164179

165180
If you have an older format installation configuration file in
166181
*_~/.config/openshift/installer.cfg.yml_* from an installation of a previous
@@ -172,17 +187,33 @@ xref:../../install_config/install/quick_install.adoc#defining-an-installation-co
172187
To start an upgrade with the quick installer:
173188

174189
. Satisfy the steps in xref:preparing-for-an-automated-upgrade[Preparing for an Automated Upgrade] to ensure you are using the latest upgrade playbooks.
190+
191+
. Run the following command on each host to remove the *atomic-openshift* packages
192+
from the list of yum excludes on the host:
193+
+
194+
----
195+
# atomic-openshift-excluder unexclude
196+
----
197+
175198
. Run the installer with the `upgrade` subcommand:
176199
+
177200
----
178201
# atomic-openshift-installer upgrade
179202
----
180203
. Then, follow the on-screen instructions to upgrade to the latest release.
181204
// tag::automated_upgrade_after_reboot[]
182-
. When the upgrade finishes, a recommendation will be printed to reboot all hosts.
183-
After rebooting, if there are no additional features enabled, you can
184-
xref:verifying-the-upgrade[verify the upgrade]. Otherwise, the next step depends
185-
on what features are enabled.
205+
. Run the following command on each host to add the *atomic-openshift* packages
206+
back to the list of yum excludes on the host:
207+
+
208+
----
209+
# atomic-openshift-excluder exclude
210+
----
211+
212+
. After all master and node upgrades have completed, a recommendation will be
213+
printed to reboot all hosts. After rebooting, if there are no additional
214+
features enabled, you can xref:verifying-the-upgrade[verify the upgrade].
215+
Otherwise, the next step depends on what additional features have you previously
216+
enabled.
186217
+
187218
[cols="1,4"]
188219
|===
@@ -197,66 +228,178 @@ on what features are enabled.
197228
// end::automated_upgrade_after_reboot[]
198229

199230
[[running-the-upgrade-playbook-directly]]
200-
== Running the Upgrade Playbook Directly
231+
== Running Upgrade Playbooks Directly
232+
233+
You can run automated upgrade playbooks using Ansible directly, similar to the
234+
advanced installation method, if you have an inventory file. Playbooks can be
235+
run using the `ansible-playbook` command.
236+
237+
The same *_v3_4_* upgrade playbooks can be used for either of the following
238+
scenarios:
201239

202-
You can run the automated upgrade playbook using Ansible directly, similar
203-
to the advanced installation method, if you have an inventory file.
240+
- Upgrading existing {product-title} 3.3 clusters to 3.4
241+
- Upgrading existing {product-title} 3.4 clusters to the latest
242+
xref:../../release_notes/ocp_3_4_release_notes.html#ocp-34-asynchronous-errata-updates[asynchronous
243+
errata updates]
204244

205-
The same *_v3_3_* upgrade playbook can be used to upgrade either of the
206-
following to the latest 3.3 release:
245+
[[upgrading-control-plane-nodes-separate-phases]]
246+
=== Upgrading the Control Plane and Nodes in Separate Phases
207247

208-
- xref:upgrading-to-ocp-3-3[Existing {product-title} 3.2 clusters]
209-
- xref:upgrading-to-ocp-3-3-asynchronous-releases[Existing {product-title} 3.3 clusters]
248+
An {product-title} cluster can be upgraded in one or more phases. You can choose
249+
whether to upgrade all hosts in one phase by running a single Ansible playbook,
250+
or upgrade the _control plane_ (master components) and nodes in multiple phases
251+
using separate playbooks.
210252

211-
[[upgrading-to-ocp-3-3]]
212-
=== Upgrading to {product-title} 3.3
253+
[NOTE]
254+
====
255+
Instructions on the full upgrade process and when to call these playbooks are
256+
described in xref:upgrading-to-ocp-3-4[Upgrading to the Latest {product-title}
257+
3.4 Release].
258+
====
259+
260+
When upgrading in separate phases, the control plane phase includes upgrading:
261+
262+
- etcd
263+
- master components
264+
- Docker on any stand-alone etcd hosts
265+
266+
It does does not include:
267+
268+
- node services running on masters
269+
- Docker running on masters
270+
- node services running on stand-alone nodes
271+
272+
When upgrading only the nodes, the control plane must already be upgraded. The
273+
node phase includes upgrading:
274+
275+
- node services running on masters and stand-alone nodes
276+
- Docker running on masters and nodes
213277

214-
To run an upgrade from {product-title} 3.2 to 3.3:
278+
[[customizing-node-upgrades]]
279+
=== Customizing Node Upgrades
280+
281+
Whether upgrading in a single or multiple phases, you can customize how the node
282+
portion of the upgrade progresses by passing certain Ansible variables to an
283+
upgrade playbook using the `-e` option.
284+
285+
[NOTE]
286+
====
287+
Instructions on the full upgrade process and when to call these playbooks are
288+
described in xref:upgrading-to-ocp-3-4[Upgrading to the Latest {product-title}
289+
3.4 Release].
290+
====
291+
292+
The `openshift_upgrade_nodes_serial` variable can be set to an integer or
293+
percentage to control how many node hosts are upgraded at the same time. The
294+
default is `1`, upgrading nodes one at a time.
295+
296+
For example, to upgrade 20 percent of the total number of detected nodes at a
297+
time:
298+
299+
----
300+
$ ansible-playbook -i <path/to/inventory/file> \
301+
</path/to/upgrade/playbook> \
302+
-e openshift_upgrade_nodes_serial="20%"
303+
----
304+
305+
The `openshift_upgrade_nodes_label` variable allows you to specify that only
306+
nodes with a certain label are upgraded. This can also be combined with the
307+
`openshift_upgrade_nodes_serial` variable.
308+
309+
For example, to only upgrade nodes in the *group1* region, two at a time:
310+
311+
----
312+
$ ansible-playbook -i <path/to/inventory/file> \
313+
</path/to/upgrade/playbook> \
314+
-e openshift_upgrade_nodes_serial="2" \
315+
-e openshift_upgrade_nodes_label="region=group1"
316+
----
317+
318+
See xref:../../admin_guide/manage_nodes.adoc#updating-labels-on-nodes[Manging
319+
Nodes] for more on node labels.
320+
321+
[[upgrading-to-ocp-3-4]]
322+
=== Upgrading to the Latest {product-title} 3.4 Release
323+
324+
To upgrade an existing {product-title} 3.3 or 3.4 cluster to the latest 3.4
325+
release:
215326

216327
. Satisfy the steps in xref:preparing-for-an-automated-upgrade[Preparing for an Automated Upgrade] to ensure you are using the latest upgrade playbooks.
217-
. Ensure the `*deployment_type*` parameter in your inventory file is set to
328+
329+
. Run the following command on each host to remove the *atomic-openshift* packages
330+
from the list of yum excludes on the host:
331+
+
332+
----
333+
# atomic-openshift-excluder unexclude
334+
----
335+
336+
. Ensure the `deployment_type` parameter in your inventory file is set to
218337
`openshift-enterprise`.
338+
219339
. If you have multiple masters configured and want to enable rolling, full system
220-
restarts of the hosts, you can set the `*openshift_rolling_restart_mode*`
340+
restarts of the hosts, you can set the `openshift_rolling_restart_mode`
221341
parameter in your inventory file to `system`. Otherwise, the default value
222342
*services* performs rolling service restarts on HA masters, but does not reboot
223343
the systems. See
224344
xref:../install/advanced_install.adoc#configuring-cluster-variables[Configuring
225345
Cluster Variables] for details.
226-
. Run the *_v3_3_* upgrade playbook. If your inventory file is located somewhere
227-
other than the default *_/etc/ansible/hosts_*, add the `-i` flag to specify the
228-
location. If you previously used the `atomic-openshift-installer` command to run
229-
your installation, you can check *_~/.config/openshift/hosts_* (previously
230-
located at *_~/.config/openshift/.ansible/hosts_*) for the last inventory file
231-
that was used, if needed.
346+
347+
. At this point, you can choose to run the upgrade in a single or multiple phases.
348+
See xref:upgrading-control-plane-nodes-separate-phases[Upgrading the Control
349+
Plane and Nodes in Separate Phases] for more details which components are
350+
upgraded in each phase.
351+
+
352+
If your inventory file is located somewhere other than the default
353+
*_/etc/ansible/hosts_*, add the `-i` flag to specify its location. If you
354+
previously used the `atomic-openshift-installer` command to run your
355+
installation, you can check *_~/.config/openshift/hosts_* for the last inventory
356+
file that was used, if needed.
357+
+
358+
[NOTE]
359+
====
360+
You can add `--tags pre_upgrade` to the following `ansible-playbook` commands to
361+
run the pre-upgrade checks for the playbook. This is a dry-run option that
362+
preforms all pre-upgrade checks without actually upgrading any hosts, and
363+
reports any problems found.
364+
====
365+
+
366+
*Option A) Upgrade control plane and nodes in a single phase.*
367+
+
368+
Run the *_upgrade.yml_* playbook to upgrade the cluster in a single phase using
369+
one playbook; the control plane is still upgraded first, then nodes in-place:
232370
+
233371
----
234-
# ansible-playbook [-i </path/to/inventory/file>] \
235-
/usr/share/ansible/openshift-ansible/playbooks/byo/openshift-cluster/upgrades/v3_3/upgrade.yml
372+
# ansible-playbook -i </path/to/inventory/file> \
373+
/usr/share/ansible/openshift-ansible/playbooks/byo/openshift-cluster/upgrades/v3_4/upgrade.yml \
374+
[-e <customized_node_upgrade_variables>] <1>
236375
----
237-
include::install_config/upgrading/automated_upgrades.adoc[tag=automated_upgrade_after_reboot]
238-
239-
[[upgrading-to-ocp-3-3-asynchronous-releases]]
240-
=== Upgrading to {product-title} 3.3 Asynchronous Releases
241-
242-
To apply
243-
xref:../../release_notes/ocp_3_3_release_notes.html#ocp-33-asynchronous-errata-updates[asynchronous errata updates] to an existing {product-title} 3.3 cluster:
376+
<1> See xref:customizing-node-upgrades[Customizing Node Upgrades] for any desired
377+
`<customized_node_upgrade_variables>`.
378+
+
379+
*Option B) Upgrade the control plane and nodes in separate phases.*
244380

381+
.. To upgrade only the control plane, run the *_upgrade_control_plane.yaml_*
382+
playbook:
383+
+
384+
----
385+
# ansible-playbook -i </path/to/inventory/file> \
386+
/usr/share/ansible/openshift-ansible/playbooks/byo/openshift-cluster/upgrades/v3_4/upgrade_control_plane.yml
387+
----
245388

246-
. Satisfy the steps in xref:preparing-for-an-automated-upgrade[Preparing for an Automated Upgrade] to ensure you are using the latest upgrade playbooks.
247-
. Run the *_v3_3_* upgrade playbook (the same playbook that is used for
248-
xref:upgrading-to-ocp-3-3[upgrading from {product-title} 3.2 to 3.3]). If your
249-
inventory file is located somewhere other than the default
250-
*_/etc/ansible/hosts_*, add the `-i` flag to specify the location. If you
251-
previously used the `atomic-openshift-installer` command to run your
252-
installation, you can check *_~/.config/openshift/hosts_* (previously located at
253-
*_~/.config/openshift/.ansible/hosts_*) for the last inventory file that was
254-
used, if needed.
389+
.. To upgrade only the nodes, run the *_upgrade_nodes.yaml_* playbook:
255390
+
256391
----
257-
# ansible-playbook [-i </path/to/inventory/file>] \
258-
/usr/share/ansible/openshift-ansible/playbooks/byo/openshift-cluster/upgrades/v3_3/upgrade.yml
392+
# ansible-playbook -i </path/to/inventory/file> \
393+
/usr/share/ansible/openshift-ansible/playbooks/byo/openshift-cluster/upgrades/v3_4/upgrade_nodes.yml \
394+
[-e <customized_node_upgrade_variables>] <1>
259395
----
396+
<1> See xref:customizing-node-upgrades[Customizing Node Upgrades] for any desired
397+
`<customized_node_upgrade_variables>`.
398+
+
399+
If you are upgrading the nodes in groups as described in
400+
xref:customizing-node-upgrades[Customizing Node Upgrades], continue invoking the
401+
*_upgrade_nodes.yml_* playbook until all nodes have been successfully upgraded.
402+
260403
include::install_config/upgrading/automated_upgrades.adoc[tag=automated_upgrade_after_reboot]
261404
endif::[]
262405

@@ -327,7 +470,7 @@ endif::[]
327470
common issues:
328471
+
329472
----
330-
# oadm diagnostics
473+
# oc adm diagnostics
331474
...
332475
[Note] Summary of diagnostics execution:
333476
[Note] Completed with no errors or warnings seen.

0 commit comments

Comments
 (0)