Skip to content

OSDOCS#4348: Installing an IBM Cloud VPC cluster in restricted env #55526

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Feb 5, 2024
Merged

OSDOCS#4348: Installing an IBM Cloud VPC cluster in restricted env #55526

merged 1 commit into from
Feb 5, 2024

Conversation

mjpytlak
Copy link
Contributor

@mjpytlak mjpytlak commented Feb 2, 2023 •
edited
Loading

Version(s):
4.15+

Issue:
This issue addresses osdocs-4348.

Link to docs preview:
Installing a cluster on IBM Cloud VPC in a restricted network

A significant amount of the content in this topic is reused from existing approved IBM Cloud VPC doc. The following preview links are for areas where notable new content was added in support of a restricted installation:

QE review:

  • QE has approved this change.

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Feb 2, 2023
@openshift-ci-robot
Copy link

openshift-ci-robot commented Feb 2, 2023 •
edited by openshift-ci bot
Loading

@mjpytlak: This pull request references OSDOCS-4348 which is a valid jira issue.

In response to this:

Version(s):
4.13+

Issue:
This issue addresses osdocs-4348.

Link to docs preview:

QE review:

  • QE has approved this change.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Feb 2, 2023
@openshift-ci-robot
Copy link

openshift-ci-robot commented Feb 2, 2023 •
edited by openshift-ci bot
Loading

@mjpytlak: This pull request references OSDOCS-4348 which is a valid jira issue.

In response to this:

Version(s):
4.13+

Issue:
This issue addresses osdocs-4348.

Link to docs preview:

QE review:

  • QE has approved this change.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@mjpytlak mjpytlak added this to the Planned for 4.13 GA milestone Feb 2, 2023
@ocpdocs-previewbot
Copy link

ocpdocs-previewbot commented Feb 2, 2023 •
edited
Loading

🤖 Mon Feb 05 21:08:48 - Prow CI generated the docs preview: https://55526--ocpdocs-pr.netlify.app

@mjpytlak mjpytlak changed the title OSDOCS-4348: Installing an IBM Cloud VPC cluster in restricted env OSDOCS#4348: Installing an IBM Cloud VPC cluster in restricted env Feb 3, 2023
@openshift-ci-robot openshift-ci-robot removed the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Feb 3, 2023
@openshift-ci-robot
Copy link

@mjpytlak: No Jira issue is referenced in the title of this pull request.
To reference a jira issue, add 'XYZ-NNN:' to the title of this pull request and request another refresh with /jira refresh.

In response to this:

Version(s):
4.13+

Issue:
This issue addresses osdocs-4348.

Link to docs preview:

QE review:

  • QE has approved this change.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@mjpytlak
Copy link
Contributor Author

mjpytlak commented Feb 3, 2023

@MayXuQQ @jeffnowicki Ready for SME and QE review. PTAL. Thanks in advance.

@MayXuQQ
Copy link

MayXuQQ commented Feb 6, 2023

@mjpytlak now the test blocked by OCPBUGS-2363

@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 21, 2023
@openshift-merge-robot openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 27, 2023
@mjpytlak mjpytlak force-pushed the osdocs-4348 branch 2 times, most recently from ad1cae8 to ec6f257 Compare March 2, 2023 00:43
@kalexand-rh kalexand-rh removed this from the Planned for 4.13 GA milestone May 15, 2023
@kalexand-rh
Copy link
Contributor

The branch/enterprise-4.14 label has been added to this PR.

This is because your PR targets the main branch and is labeled for enterprise-4.13. And any PR going into main must also target the latest version branch (enterprise-4.14).

If the update in your PR does NOT apply to version 4.14 onward, please re-target this PR to go directly into the appropriate version branch or branches (enterprise-4.x) instead of main.

@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 16, 2023
@openshift-merge-robot openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 17, 2023
@mjpytlak mjpytlak force-pushed the osdocs-4348 branch 2 times, most recently from 7d463d6 to 80b0b2c Compare May 17, 2023 13:19
@openshift-bot
Copy link

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

@openshift-ci openshift-ci bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Aug 16, 2023
@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 16, 2023
@kalexand-rh
Copy link
Contributor

/remove-lifecycle stale

@jeffnowicki
Copy link

jeffnowicki commented Jan 25, 2024 •
edited
Loading

most looks good, the only comment is about the available region to run the restricted installation on IBMCloud, which is mentioned in #55526 (comment)

For now, per my testing, only us-east region supports the VPE for all the listed private endpoints.

serviceEndpoints:
  - name: IAM
    url: <iam_private_endpoint_url>
  - name: VPC
    url: <vpc_private_endpoint_url>
  - name: ResourceController
    url: <resource_controller_private_endpoint_url>
  - name: ResourceManager
    url: <resource_manager_private_endpoint_url>
  - name: DNSServices
    url: <dns_services_private_endpoint_url>
  - name: COS
    url: <cos_private_endpoint_url>
  - name: GlobalSearch
    url: <global_search_private_endpoint_url>
  - name: GlobalTagging
    url: <global_tagging_private_endpoint_url>

So I suggest to add a NOTE for that, e.g: https://github.com/openshift/openshift-docs/pull/55526/files#diff-f4e400c4dd55ea0769811aa54453b3a4393d1db5fe828272f880b574bceb69caR385 or some other place.

@jianlinliu IBM Cloud private service endpoints (regardless of region) resolve to the same private internal network. So regardless of region you are deploying your cluster, you can pick any available private service endpoint. Selecting one in same (if available) or nearest geographic location would be preferable to minimize network latency. I'm uncertain at the moment, why your test failed.

@jeffnowicki
Copy link

@cjschaef and I will re-evaluate services and their private service endpoints and whether there are some that have region-based restrictions.

@jianlinliu
Copy link

regardless of region you are deploying your cluster, you can pick any available private service endpoint. Selecting one in
same (if available) or nearest geographic location would be preferable to minimize network latency. I'm uncertain at the > moment, why your test failed.

@jeffnowicki @mjpytlak I think VPEs are supposed to be not regionally bound, but sounds there is some limit from IBMCloud provider. Let me take an example to make it be clear.

Assuming a user is installing a disconnected cluster in eu-gb region following the guide, the user need to override ResourceController endpoint to use a private url - https://private.resource-controller.cloud.ibm.com/, right? If true, how can I create the VPE (I did not see the service-controller service is listed on https://cloud.ibm.com/vpc-ext/provision/endpointGateway when eu-gb region is selected)?

@mjpytlak mjpytlak force-pushed the osdocs-4348 branch 2 times, most recently from 3283f74 to 3c01958 Compare January 29, 2024 16:49
@mjpytlak
Copy link
Contributor Author

mjpytlak commented Jan 29, 2024 •
edited
Loading

@jianlinliu - @jeffnowicki and I had a chance to connect on our guidance around service endpoints. And after he and Chris discussed it more, Jeff requested that the latest updates be made. You can find them here [1]

[1] 3c01958

@jeffnowicki
Copy link

@jianlinliu IBM Cloud is migrating off of "Cloud (aka classic) Service Endpoints" to "Virtual Private Endpoints". While CSE private endpoints are not necessarily regionally bound, VPE private endpoints are. With that being the case, a disconnected/restricted network installation will be constrained to those those regions that have VPE enablement for all required services. The current enabled regions are noted in the doc update @mjpytlak made. Over time, more regions will be enabled.

jianlinliu
jianlinliu reviewed Jan 30, 2024
View reviewed changes
installing/installing_ibm_cloud_public/installing-ibm-cloud-restricted.adoc Outdated
+
You cannot use the VPC that the installation program provisions by default.
* If you plan on configuring endpoint gateways to use {ibm-cloud-name} Virtual Private Endpoints, consider the following requirements:
** Endpoint gateway support is currently limited to the `us-east`, `us-south`, and `eu-de` regions.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mjpytlak @jeffnowicki can you confirm eu-de region here? When I am installing a disconnected cluster in eu-de region, then overriding ResourceController serviceEndpoints to https://private.resource-controller.cloud.ibm.com/, but I did not find a endpoint gateway targeted to ``private.resource-controller.cloud.ibm.comin the output ofibmcloud is endpoint-gateway-targets --output JSON` against `eu-de` region.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you confirm eu-de region here? When I am installing a disconnected cluster in eu-de region, then overriding ResourceController serviceEndpoints to https://private.resource-controller.cloud.ibm.com/, but I did not find a endpoint gateway targeted to ``private.resource-controller.cloud.ibm.com in the output ofibmcloud is endpoint-gateway-targets --output JSON` against `eu-de` region.

@jianlinliu you are correct, I confirmed that is the one service in eu-de that's missing a private endpoint and would be needed for a "fully disconnected" cluster installation.

@mjpytlak - please remove mention of eu-de from that section of the docs.

fyi @cjschaef

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please remove mention of eu-de from that section of the docs.

Removed. Thank you.

@mjpytlak mjpytlak force-pushed the osdocs-4348 branch 2 times, most recently from 874569b to 300326c Compare January 30, 2024 14:05
@jeffnowicki
Copy link

Followup approval and based on recent changes due to service endpoint (private) restrictions across regions.

  • lgtm
  • ibm-approved

@jianlinliu
Copy link

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jan 31, 2024
@mjpytlak mjpytlak added the peer-review-needed Signifies that the peer review team needs to review this PR label Feb 5, 2024
@bscott-rh bscott-rh added the peer-review-in-progress Signifies that the peer review team is reviewing this PR label Feb 5, 2024
bscott-rh
bscott-rh reviewed Feb 5, 2024
View reviewed changes
Copy link
Contributor

@bscott-rh bscott-rh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few nits and suggestions, otherwise LGTM.

/remove-label peer-review-in-progress
/remove-label peer-review-needed
/label peer-review-done

modules/installation-about-restricted-network.adoc Outdated
Comment on lines 23 to 26
ifeval::["{context}" == "installing-ibm-cloud-restricted"]
:ibm-cloud:
endif::[]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ifeval::["{context}" == "installing-ibm-cloud-restricted"]
:ibm-cloud:
endif::[]
ifeval::["{context}" == "installing-ibm-cloud-restricted"]
:ibm-cloud:
:ipi:
endif::[]

Can remove lines 47-49 by putting both conditions in this ifeval

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed good catch.

modules/installation-about-restricted-network.adoc Outdated

You complete the installation using a bastion host or portable device that can access both the internet and your closed network. You must use a host with internet access to:

* Download the installation program, the the OpenShift CLI (`oc`), and the CCO utility (`ccoctl`).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Download the installation program, the the OpenShift CLI (`oc`), and the CCO utility (`ccoctl`).
* Download the installation program, the OpenShift CLI (`oc`), and the CCO utility (`ccoctl`).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per the usual, eagle eye.

modules/installation-about-restricted-network.adoc Outdated

[NOTE]
====
If you are specifying an {ibm-name} Key Protect for {ibm-cloud-name} root key as part of the installation process, the service endpoint for Key Protect is also a required.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If you are specifying an {ibm-name} Key Protect for {ibm-cloud-name} root key as part of the installation process, the service endpoint for Key Protect is also a required.
If you are specifying an {ibm-name} Key Protect for {ibm-cloud-name} root key as part of the installation process, the service endpoint for Key Protect is also required.

modules/installation-about-restricted-network.adoc
If you are specifying an {ibm-name} Key Protect for {ibm-cloud-name} root key as part of the installation process, the service endpoint for Key Protect is also a required.
====

By default, the public endpoint is used to access the service. If network restrictions limit access to public service endpoints, you can override the default behavior.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to explain how/where to find docs on how to override the default behavior?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I follow. The very next paragraph states that you update install-config.yaml with the URI of an alternate service endpoint.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right! I left this review comment before I got to the next section

modules/installation-about-restricted-network.adoc Outdated
Comment on lines 145 to 146
ifeval::["{context}" == "installing-ibm-cloud-restricted"]
:!ibm-cloud:
endif::[]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ifeval::["{context}" == "installing-ibm-cloud-restricted"]
:!ibm-cloud:
endif::[]
ifeval::["{context}" == "installing-ibm-cloud-restricted"]
:!ibm-cloud:
:!ipi:
endif::[]

Can combine these conditionals and remove lines 169-171.

modules/installation-ibm-cloud-configure-vpc-for-endpoint-gateways.adoc Outdated

.Prerequisites

* You have installed the {ibm-cloud-name} Command Line Interface (`ibmcloud`).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* You have installed the {ibm-cloud-name} Command Line Interface (`ibmcloud`).
* You have installed the {ibm-cloud-name} Command Line Interface utility (`ibmcloud`).

I'm not sure if this is new content in this PR or reused from other areas.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Net new. I updated it. Thanks.

modules/installation-ibm-cloud-download-rhcos.adoc Outdated
[id="installation-ibm-cloud-download-rhcos_{context}"]
= Downloading the RHCOS cluster image

The installation program requires the {op-system-first} image to install the cluster. While optional, downloading the {op-system-first} before deploying, removes the need for internet access when creating the cluster.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The installation program requires the {op-system-first} image to install the cluster. While optional, downloading the {op-system-first} before deploying, removes the need for internet access when creating the cluster.
The installation program requires the {op-system-first} image to install the cluster. While optional, downloading the {op-system-first} before deploying removes the need for internet access when creating the cluster.

modules/installation-initializing-manual.adoc Outdated
+
[NOTE]
====
If you choose to leave the default value of `External`, your network must be able to access the public endpoint for {ibm-cloud-name} Internet Services (CIS). CIS is not enabled for Virtual Private Endpoints.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If you choose to leave the default value of `External`, your network must be able to access the public endpoint for {ibm-cloud-name} Internet Services (CIS). CIS is not enabled for Virtual Private Endpoints.
If you use the default value of `External`, your network must be able to access the public endpoint for {ibm-cloud-name} Internet Services (CIS). CIS is not enabled for Virtual Private Endpoints.

minimalism nit

modules/installation-initializing.adoc
@@ -548,6 +558,18 @@ vpcSubnets: <vpcSubnet>
+
For `platform.powervs.vpcName`, specify the name for the existing {ibm-cloud-name}. For `platform.powervs.vpcSubnets`, specify the existing subnets.
endif::ibm-power-vs+restricted[]
ifdef::ibm-cloud+restricted[]
.. Define the network and subnets for the VPC to install the cluster in under the parent `platform.ibmcloud` field:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
.. Define the network and subnets for the VPC to install the cluster in under the parent `platform.ibmcloud` field:
.. Define the VPC network and subnets for the cluster to use under the parent `platform.ibmcloud` field:

Tried rewording this because "in under" felt awkward.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. This is language that is used for every restricted installation assembly. I agree it reads a little awkward, but I am not sure if your suggestion changes the technical accuracy. I am inclined to leave this one alone.

modules/installation-initializing.adoc Outdated
+
[NOTE]
====
If you choose to leave the default value of `External`, your network must be able to access the public endpoint for {ibm-cloud-name} Internet Services (CIS). CIS is not enabled for Virtual Private Endpoints.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If you choose to leave the default value of `External`, your network must be able to access the public endpoint for {ibm-cloud-name} Internet Services (CIS). CIS is not enabled for Virtual Private Endpoints.
If you use the default value of `External`, your network must be able to access the public endpoint for {ibm-cloud-name} Internet Services (CIS). CIS is not enabled for Virtual Private Endpoints.

minimalism nit

@openshift-ci openshift-ci bot added peer-review-done Signifies that the peer review team has reviewed this PR and removed peer-review-in-progress Signifies that the peer review team is reviewing this PR peer-review-needed Signifies that the peer review team needs to review this PR labels Feb 5, 2024
@openshift-ci openshift-ci bot removed the lgtm Indicates that a PR is ready to be merged. label Feb 5, 2024
@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Feb 5, 2024

New changes are detected. LGTM label has been removed.

@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Feb 5, 2024

@mjpytlak: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@mjpytlak mjpytlak merged commit 3c07fea into openshift:main Feb 5, 2024
@mjpytlak
Copy link
Contributor Author

mjpytlak commented Feb 5, 2024

/cherrypick enterprise-4.15

@openshift-cherrypick-robot
Copy link

@mjpytlak: new pull request created: #71227

In response to this:

/cherrypick enterprise-4.15

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-cherrypick-robot openshift-cherrypick-robot mentioned this pull request Feb 5, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jianlinliu jianlinliu jianlinliu left review comments

@jeffnowicki jeffnowicki jeffnowicki requested changes

@bscott-rh bscott-rh bscott-rh left review comments

Assignees

@jianlinliu jianlinliu

Labels
branch/enterprise-4.15 peer-review-done Signifies that the peer review team has reviewed this PR size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Milestone
OCP 4.15 GA
Development

Successfully merging this pull request may close these issues.