advanced_install: installer as system container #5258

sosiouxme · 2017-09-13T21:37:26Z

Simplify slightly and also note some things that were not apparent.

I would like to go on to describe using the container from docker, but I thought it would be good to fix up what's there first.

sosiouxme · 2017-09-13T21:38:28Z

install_config/install/advanced_install.adoc

+*docker* service. The installer image provides the same functionality
+as the traditional RPM-based installer, but it runs in a containerized
+environment that provides all of its dependencies rather than being
+installed directly on the host.


This is the only real advantage -- don't have to install a bunch of RPMs on your host. So I think it's worth mentioning.

sosiouxme · 2017-09-13T21:39:29Z

install_config/install/advanced_install.adoc

 +
 ----
-$ atomic pull --storage ostree \


It turns out this whole step is unnecessary as atomic install does it for you.

sosiouxme · 2017-09-13T21:41:13Z

install_config/install/advanced_install.adoc

@@ -2216,22 +2206,35 @@ ifdef::openshift-origin[]
 endif::[]
 ----
 <1> Sets the name for the systemd service.
-<2> Specify the location for your inventory file on your local workstation.
+<2> Specify the location for your local inventory file.


This is the only place we referred to local workstation... this can run anywhere that docker and systemd run, not that likely to be a workstation actually.

sosiouxme · 2017-09-13T21:42:03Z

install_config/install/advanced_install.adoc


 . Use the `systemctl` command to start the installer service as you would any
-other systemd service. This command initiates the cluster installation:
+other systemd service. This command initiates the cluster installation using
+the inventory file specified above and the root user's SSH configuration:


This is one thing that was not apparent before... ssh config comes from root for this use case (in the docker use case you have to mount it in).

sosiouxme · 2017-09-13T21:42:50Z

install_config/install/advanced_install.adoc

 link:https://access.redhat.com/documentation/en-us/red_hat_satellite/6.2/html/content_management_guide/managing_ostree_content[OSTree]
-instead of defaulting to *docker* daemon storage. Use the Atomic CLI to import
-the installer image from the local *docker* engine to OSTree storage:
+instead of *docker* daemon storage. As the root user, use the Atomic


atomic install and systemctl and journalctl are generally things you have to run as root. Is it customary to use the root prompt in docs?

sosiouxme · 2017-09-13T21:43:18Z

install_config/install/advanced_install.adoc

 +
 ----
 $ systemctl start openshift-installer
 ----
 +
+There is no output from this command while the installer service is


One potentially confusing thing the first time you run this.

sosiouxme · 2017-09-13T21:45:12Z

install_config/install/advanced_install.adoc

-playbook, but you can set it to the path of another playbook inside the
-container.
+To run any other playbooks (for example, to run the
+xref:configuring-cluster-pre-install-checks[pre-install checks]


The pre-install checks seemed a more relevant subject for this topic than the upgrade playbooks. Plus I'd really like to let people know they can run checks from this container.

sosiouxme · 2017-09-14T02:52:13Z

install_config/install/advanced_install.adoc

-    --set INVENTORY_FILE=/etc/ansible/hosts \
-    --set PLAYBOOK_FILE=playbooks/byo/openshift-cluster/upgrades/v3_6/upgrade.yml \//<1>
+    --set INVENTORY_FILE=/path/to/inventory \
+    --set PLAYBOOK_FILE=playbooks/byo/openshift-checks/pre-install.yml \//<1>


actually I have to update this - currently it won't work with a relative path.

sosiouxme · 2017-09-14T13:14:12Z

install_config/install/advanced_install.adoc

-    --set INVENTORY_FILE=/etc/ansible/hosts \
-    --set PLAYBOOK_FILE=playbooks/byo/openshift-cluster/upgrades/v3_6/upgrade.yml \//<1>
+    --set INVENTORY_FILE=/path/to/inventory \
+    --set PLAYBOOK_FILE=/usr/share/ansible/openshift-ansible/playbooks/byo/openshift-checks/pre-install.yml \//<1>


As currently implemented, you actually need the full path for the playbook. This will change "soon" with updates to the image and we'll be able to go back to relative paths (the full path should always work though).

sosiouxme · 2017-09-14T13:17:21Z

@openshift/team-documentation these are changes for 3.6 and origin ... hopefully by the time 3.7 rolls around we will have an updated atomic with a better workflow and no longer tech preview.

EDIT: actually the better workflow is here already :) Updated...

sosiouxme · 2017-09-14T15:16:42Z

install_config/install/advanced_install.adoc

    --storage=ostree \
    --name=openshift-installer \//<1>
    --set INVENTORY_FILE=/path/to/inventory \//<2>
 ifdef::openshift-enterprise[]
-    docker:registry.access.redhat.com/openshift3/ose-ansible:v3.6
+    registry.access.redhat.com/openshift3/ose-ansible:v3.6


@ashcrow @giuseppe check me on this -- it looks like the latest version of atomic on RHEL now has more of the functionality you wanted here.

Previously, it needed to pull through docker, and it would import into ostree every time, and created a regular systemd service (not one-time, and no output, just journald, and service definition was retained after running).

That is still the behavior AFAICS if I specify the image with the docker: prefix, however if I leave that off, then it directly imports the image to ostree (no docker even running), creates a one-time service, runs it, and uninstalls it, which I think is the behavior you were wanting to get to.

Just wanted you to check if I'm missing anything here.

[...]if I leave that off, then it directly imports the image to ostree (no docker even running), creates a one-time service, runs it, and uninstalls it, which I think is the behavior you were wanting to get to.

That is the atomic.run once feature that is much cleaner than the original one. Looks like RHEL 7.4 has atomic 1.18.1-3 which has the run once ability. I'm not sure if the docker import is still needed or not with that version. @giuseppe?

It looks like with that feature, it doesn't even use the service name provided. I don't think it ever creates a systemd service per se, doesn't seem to send anything to journald. So I'll take out all the discussion of systemd service, which should make this even simpler :)

sosiouxme

Updated per things I discovered about the atomic install command just now.

sosiouxme · 2017-09-14T15:21:45Z

install_config/install/advanced_install.adoc

+*docker* service. The installer image provides the same functionality
+as the traditional RPM-based installer, but it runs in a containerized
+environment that provides all of its dependencies rather than being
+installed directly on the host.


This is the main advantage -- don't have to install a bunch of RPMs on your host. So I think it's worth mentioning.

sosiouxme · 2017-09-14T15:22:28Z

install_config/install/advanced_install.adoc

-+
----
-$ atomic install --system \
+# atomic install --system \


atomic install I think you have to run as root. Is it customary to use the root prompt in docs?

Correct. It must be run as root.

yes, it must run as root. "atomic install --system" would fail anyway if not running as root.

sosiouxme · 2017-09-14T15:23:32Z

install_config/install/advanced_install.adoc

-$ systemctl start openshift-installer
----
+This command initiates the cluster installation using the inventory file specified and the root user's
+SSH configuration, logging to the terminal as well as *_/var/log/ansible.log_*. The first time this is executed, the image will be imported into


Things that were not apparent before... ssh config comes from root for this use case, and where to find logs.

ashcrow

Looks great!

ashcrow · 2017-09-14T15:32:17Z

install_config/install/advanced_install.adoc

+*docker* service. The installer image provides the same functionality
+as the traditional RPM-based installer, but it runs in a containerized
+environment that provides all of its dependencies rather than being
+installed directly on the host.


ashcrow · 2017-09-14T15:33:04Z

install_config/install/advanced_install.adoc

-+
----
-$ atomic install --system \
+# atomic install --system \


Correct. It must be run as root.

ashcrow · 2017-09-14T15:41:23Z

install_config/install/advanced_install.adoc

    --storage=ostree \
    --name=openshift-installer \//<1>
    --set INVENTORY_FILE=/path/to/inventory \//<2>
 ifdef::openshift-enterprise[]
-    docker:registry.access.redhat.com/openshift3/ose-ansible:v3.6
+    registry.access.redhat.com/openshift3/ose-ansible:v3.6


[...]if I leave that off, then it directly imports the image to ostree (no docker even running), creates a one-time service, runs it, and uninstalls it, which I think is the behavior you were wanting to get to.

That is the atomic.run once feature that is much cleaner than the original one. Looks like RHEL 7.4 has atomic 1.18.1-3 which has the run once ability. I'm not sure if the docker import is still needed or not with that version. @giuseppe?

ashcrow · 2017-09-14T15:45:29Z

@sosiouxme thank you for working on this!

sosiouxme · 2017-09-15T19:40:03Z

I added a commit to also discuss running the installer image via Docker. This can be worked on separately if preferred, though they are closely related.

sosiouxme · 2017-11-09T15:37:40Z

@openshift/team-documentation can this PR get some love?

vikram-redhat · 2017-11-09T20:25:54Z

@sosiouxme is this for 3.7?

sosiouxme · 2017-11-10T14:14:36Z

@vikram-redhat it's not specific to 3.7, it's been relevant since it was written two months ago. 3.7 does not block on getting this in, if that's what you're asking.

ncbaratta

LGTM

ncbaratta · 2017-12-18T19:54:22Z

install_config/install/advanced_install.adoc

+This installer image provides the same functionality as the RPM-based
+installer, but it runs in a containerized environment that provides all
+of its dependencies rather than being installed directly on the host.
+So, the only requirement to use it is the ability to run a container.


I'd remove the 'So,'

giuseppe · 2018-01-11T17:21:51Z

@sosiouxme could you please rebase this PR? It looks like this change didn't get it, and the documentation still reports the old way of using the system container installer.

ashcrow · 2018-01-11T18:55:40Z

👍 we've had some confusion over usage and I think this PR clears things up.

Update, simplify, and also note some things that were not apparent about running as a system container. Rearrange containerized install material to enable discussion of running it as a Docker container.

sosiouxme · 2018-01-11T20:31:25Z

@giuseppe @ashcrow sure thing guys.
@vikram-redhat can this get moving now?

gaurav-nelson · 2018-01-12T00:51:47Z

install_config/install/advanced_install.adoc

+
+* SSH key(s) so that Ansible can reach your hosts.
+* An Ansible inventory file.
+* The location of an Ansible playbook to run against that inventory.


gaurav-nelson · 2018-01-12T00:54:18Z

install_config/install/advanced_install.adoc

+Note that this must be run by a non-`root` user with access to *docker*.
+
+----
+$ docker run -t -u `id -u` \


Please use callous for explaining this code instead of the list. For example:

$ docker run -t -u `id -u` \ <1> -v $HOME/.ssh/id_rsa:/opt/app-root/src/.ssh/id_rsa:Z \ <2> ...and so on.

and after the code block use the callouts as:

<1> `-u `id -u`` makes the container run with the same UID as the current user, which allows that user to use the SSH key inside the container (SSH private keys are expected to be readable only by their owner). <2> `-v $HOME/.ssh/id_rsa:/opt/app-root/src/.ssh/id_rsa:Z` mounts your SSH key (`$HOME/.ssh/id_rsa`) under the container user's `$HOME/.ssh` (*_/opt/app-root/src_* is the `$HOME` of the user in the container). If you mount the SSH key into a non-standard location you can add an environment variable with `-e ANSIBLE_PRIVATE_KEY_FILE=/the/mount/point` or set `ansible_ssh_private_key_file=/the/mount/point` as a variable in the inventory to point Ansible at it. ... and so on.

The text portions used above are examples only please do not use it as it is.

This results in some really huge callouts, but I guess that's fine. I think originally there wasn't as clear a correspondence between the lines of command and the list explanation. Now the only real mismatch is that the third callout is about two lines, but I guess that's fine too.

gaurav-nelson · 2018-01-12T00:58:26Z

install_config/install/advanced_install.adoc

+or set `ansible_ssh_private_key_file=/the/mount/point` as a variable in
+the inventory to point Ansible at it.
+
+Note that the SSH key is mounted with the `:Z` flag: this is


s/flag:/flag**.**

gaurav-nelson · 2018-01-12T00:58:48Z

install_config/install/advanced_install.adoc

+
+Note that the SSH key is mounted with the `:Z` flag: this is
+required so that the container can read the SSH key under
+its restricted SELinux context; this also means that your


s/SELinux context; this also/SELinux context**.** This also

gaurav-nelson · 2018-01-12T01:20:53Z

install_config/install/advanced_install.adoc

+original SSH key file will be re-labeled to something like
+`system_u:object_r:container_file_t:s0:c113,c247`. For more details
+about `:Z` please check the `docker-run(1)` man page. Keep this in mind
+when providing these volume mount specifications because this could


s/this could/this might

gaurav-nelson · 2018-01-12T01:23:47Z

install_config/install/advanced_install.adoc

+mount a static Ansible inventory file into the container as
+*_/tmp/inventory_* and set the corresponding environment variable to
+point at it. As with the SSH key, the inventory file SELinux labels may
+need to be relabeled via the `:Z` flag to allow reading in the container,


s/via/by using

gaurav-nelson · 2018-01-12T01:28:00Z

@sosiouxme Thank you for the PR. I have suggested few updates, not big ones but multiple small ones.

Then I can merge this one after QE checks.

ganhuang · 2018-01-12T01:52:51Z

cc @wkshi

ashcrow · 2018-01-12T14:00:41Z

@gaurav-nelson @sosiouxme @wkshi thanks guys for working on this!

sosiouxme · 2018-01-12T14:41:56Z

Thanks, I'll fix those up today.

sosiouxme · 2018-01-12T21:34:54Z

@gaurav-nelson thanks for the review; I think I've addressed everything except #5258 (review) where I have a lingering question.

gaurav-nelson · 2018-01-14T23:32:03Z

Thank you @sosiouxme for the updates, I am waiting for QE review before merging this. After that's done, I will do a follow up PR and then we can work on better explanation for your question. Thanks again!

wkshi

Looks good to me.(QE side)

ashcrow · 2018-01-16T14:07:03Z

@gaurav-nelson looks like we got QE approval 🎆 🎆

sosiouxme · 2018-01-17T13:38:48Z

👏

adellape · 2018-02-09T21:25:33Z

@gaurav-nelson I'm getting a lot of conflicts trying to pick something else from master to 3.9-stage, and I think it's because this PR should have gotten picked to 3.9 when it was picked to 3.6 and 3.7. I'm gonna try doing that.

sosiouxme force-pushed the 20170913-containerized-installer branch from dfae5e5 to 9bbb592 Compare September 13, 2017 21:40

sosiouxme commented Sep 13, 2017

View reviewed changes

sosiouxme commented Sep 14, 2017

View reviewed changes

sosiouxme force-pushed the 20170913-containerized-installer branch from 9bbb592 to 495c0e4 Compare September 14, 2017 13:12

sosiouxme commented Sep 14, 2017

View reviewed changes

sosiouxme force-pushed the 20170913-containerized-installer branch from 495c0e4 to 7f9d6db Compare September 14, 2017 15:10

sosiouxme commented Sep 14, 2017

View reviewed changes

ashcrow reviewed Sep 14, 2017

View reviewed changes

sosiouxme force-pushed the 20170913-containerized-installer branch from 7f9d6db to ace08df Compare September 14, 2017 17:28

etsauer mentioned this pull request Oct 3, 2017

Rebasing openstack image on top of openshift-ansible image redhat-cop/casl-ansible#95

Merged

ncbaratta added the peer-review-needed Signifies that the peer review team needs to review this PR label Nov 9, 2017

vikram-redhat added branch/enterprise-3.6 branch/enterprise-3.7 labels Nov 10, 2017

ncbaratta reviewed Dec 18, 2017

View reviewed changes

ncbaratta added peer-review-done Signifies that the peer review team has reviewed this PR and removed peer-review-needed Signifies that the peer review team needs to review this PR labels Dec 18, 2017

sosiouxme force-pushed the 20170913-containerized-installer branch from 03aea35 to c46d96d Compare January 11, 2018 20:26

openshift-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jan 11, 2018

advanced_install: installer image

7c24c89

Update, simplify, and also note some things that were not apparent about running as a system container. Rearrange containerized install material to enable discussion of running it as a Docker container.

sosiouxme force-pushed the 20170913-containerized-installer branch from c46d96d to 7c24c89 Compare January 11, 2018 20:29

gaurav-nelson reviewed Jan 12, 2018

View reviewed changes

advanced_install: tweaks per pier review

d85436f

wkshi reviewed Jan 16, 2018

View reviewed changes

gaurav-nelson merged commit ebae7ec into openshift:master Jan 17, 2018

gaurav-nelson added the to_followup label Jan 17, 2018

gaurav-nelson added this to the Next Release milestone Jan 17, 2018

gaurav-nelson removed the to_followup label Jan 17, 2018

sosiouxme deleted the 20170913-containerized-installer branch January 17, 2018 13:37

This was referenced Jan 18, 2018

[enterprise-3.6] advanced_install: installer as system container #7222

Merged

[enterprise-3.7] advanced_install: installer as system container #7223

Merged

vikram-redhat modified the milestones: Next Release, Staging, Published 25/Jan/2018 Jan 25, 2018

This was referenced Feb 9, 2018

[enterprise-3.9] advanced_install: installer as system container #7701

Merged

Update 'byo' references based on playbook refactoring #7274

Merged

advanced_install: installer as system container #5258

advanced_install: installer as system container #5258

Conversation

sosiouxme commented Sep 13, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sosiouxme commented Sep 14, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sosiouxme left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ashcrow left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ashcrow commented Sep 14, 2017

sosiouxme commented Sep 15, 2017

sosiouxme commented Nov 9, 2017

vikram-redhat commented Nov 9, 2017

sosiouxme commented Nov 10, 2017 • edited Loading

ncbaratta left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

giuseppe commented Jan 11, 2018

ashcrow commented Jan 11, 2018

sosiouxme commented Jan 11, 2018

Choose a reason for hiding this comment

gaurav-nelson Jan 12, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gaurav-nelson Jan 12, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gaurav-nelson commented Jan 12, 2018

ganhuang commented Jan 12, 2018

ashcrow commented Jan 12, 2018

sosiouxme commented Jan 12, 2018

sosiouxme commented Jan 12, 2018

gaurav-nelson commented Jan 14, 2018

wkshi left a comment

Choose a reason for hiding this comment

ashcrow commented Jan 16, 2018

sosiouxme commented Jan 17, 2018

adellape commented Feb 9, 2018

sosiouxme commented Sep 14, 2017 •

edited

Loading

sosiouxme commented Nov 10, 2017 •

edited

Loading

gaurav-nelson Jan 12, 2018 •

edited

Loading

gaurav-nelson Jan 12, 2018 •

edited

Loading