Skip to content

Commit 58974ab

Browse files
Shabirmeangcf-owl-bot[bot]busunkim96
authored andcommitted
docs(samples): add usage samples to show handling of LRO response Operation (#191)
* improvement: add samples for the using the library * cleanup: update the git ignore * 🦉 Updates from OwlBot See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * chore: remove unnecessary comments * chore: update git ignore * 🦉 Updates from OwlBot See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * doc: add simple usage instructions to README * cleanup: add copyright headers * chore: update git ignore * 🦉 Updates from OwlBot See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * doc: add more details to retry function doc * chore: update git ignore * 🦉 Updates from OwlBot See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * doc: add seperate sections about the samples * chore: update git ignore * 🦉 Updates from OwlBot See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * cleanup: add region tags * doc: update the readme to remove inline code * chore: update git ignore * 🦉 Updates from OwlBot See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * doc: update readme * chore: update git ignore * 🦉 Updates from OwlBot See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * process: add nox and fix lint errors * chore: update git ignore * 🦉 Updates from OwlBot See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * process: add requirements.txt * chore: add lro samples to gitignore * test: add test for quickstart * test: add test for create cluster * test: add test for delete cluster * test: fix fixture in test * 🦉 Updates from OwlBot See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * lint: finx linting errors * cleanup: remove the main from being inside the region tags * cleanup: remove the main from being inside the region tags * cleanup: remove the main from being inside the region tags * 🦉 Updates from OwlBot See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * 🦉 Updates from OwlBot See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * 🦉 Updates from OwlBot See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * doc: fix typo Co-authored-by: Bu Sun Kim <[email protected]> * doc: pr comment doc update Co-authored-by: Bu Sun Kim <[email protected]> * doc: add license headers to missing files * 🦉 Updates from OwlBot See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Bu Sun Kim <[email protected]>
1 parent f1ece5e commit 58974ab

11 files changed

+867
-0
lines changed

container/snippets/README.md

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# Samples
2+
3+
All the samples are self contained unless they are placed inside their own folders. The samples use [Application Default Credentails (ADC)](https://cloud.google.com/docs/authentication/production#automatically) to authenticate with GCP. So make sure ADC is setup correctly _(i.e. `GOOGLE_APPLICATION_CREDENTIALS` environment variable is set)_ before running the samples. Some sample might require additional python modules to be installed.
4+
5+
You can run samples as follows:
6+
7+
```python
8+
python <sample_name.py> <arg1> <arg2> ...
9+
```
10+
11+
You can run the following command to find the usage and arguments for the samples:
12+
13+
```python
14+
python <sample_name.py> -h
15+
```
16+
```bash
17+
# example
18+
python quickstart.py -h
19+
20+
usage: quickstart.py [-h] project_id zone
21+
22+
positional arguments:
23+
project_id Google Cloud project ID
24+
zone GKE Cluster zone
25+
26+
optional arguments:
27+
-h, --help show this help message and exit
28+
```
29+
30+
### Quickstart sample
31+
- [**quickstart.py**](quickstart.py): A simple example to list the GKE clusters in a given GCP project and zone. The sample uses the [`list_clusters()`](https://cloud.google.com/python/docs/reference/container/latest/google.cloud.container_v1.services.cluster_manager.ClusterManagerClient#google_cloud_container_v1_services_cluster_manager_ClusterManagerClient_list_clusters) API to fetch the list of cluster.
32+
33+
34+
### Long running operation sample
35+
36+
The following samples are examples of operations that take a while to complete.
37+
For example _creating a cluster_ in GKE can take a while to set up the cluster
38+
nodes, networking and configuring Kubernetes. Thus, calls to such long running
39+
APIs return an object of type [`Operation`](https://cloud.google.com/python/docs/reference/container/latest/google.cloud.container_v1.types.Operation). We can
40+
then use the id of the returned operation to **poll** the [`get_operation()`](https://cloud.google.com/python/docs/reference/container/latest/google.cloud.container_v1.services.cluster_manager.ClusterManagerClient#google_cloud_container_v1_services_cluster_manager_ClusterManagerClient_get_operation) API to check for it's status. You can see the
41+
different statuses it can be in, in [this proto definition](https://github.com/googleapis/googleapis/blob/master/google/container/v1/cluster_service.proto#L1763-L1778).
42+
43+
- [**create_cluster.py**](create_cluster.py): An example of creating a GKE cluster _(with mostly the defaults)_. This example shows how to handle responses of type [`Operation`](https://cloud.google.com/python/docs/reference/container/latest/google.cloud.container_v1.types.Operation) that reperesents a long running operation. The example uses the python module [`backoff`](https://github.com/litl/backoff) to handle a graceful exponential backoff retry mechanism to check if the `Operation` has completed.
44+
45+
- [**delete_cluster.py**](delete_cluster.py): An example of deleting a GKE cluster. This example shows how to handle responses of type [`Operation`](https://cloud.google.com/python/docs/reference/container/latest/google.cloud.container_v1.types.Operation) that reperesents a long running operation.

container/snippets/create_cluster.py

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
# -*- coding: utf-8 -*-
2+
# Copyright 2022 Google LLC
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
16+
# [START gke_create_cluster]
17+
import argparse
18+
import sys
19+
from typing import Dict
20+
21+
import backoff
22+
from google.cloud import container_v1
23+
24+
25+
def on_success(details: Dict[str, str]) -> None:
26+
"""
27+
A handler function to pass into the retry backoff algorithm as the function
28+
to be executed upon a successful attempt.
29+
30+
Read the `Event handlers` section of the backoff python module at:
31+
https://pypi.org/project/backoff/
32+
"""
33+
print("Successfully created cluster after {elapsed:0.1f} seconds".format(**details))
34+
35+
36+
def on_failure(details: Dict[str, str]) -> None:
37+
"""
38+
A handler function to pass into the retry backoff algorithm as the function
39+
to be executed upon a failed attempt.
40+
41+
Read the `Event handlers` section of the backoff python module at:
42+
https://pypi.org/project/backoff/
43+
"""
44+
print("Backing off {wait:0.1f} seconds after {tries} tries".format(**details))
45+
46+
47+
@backoff.on_predicate(
48+
# the backoff algorithm to use. we use exponential backoff here
49+
backoff.expo,
50+
# the test function on the return value to determine if a retry is necessary
51+
lambda x: x != container_v1.Operation.Status.DONE,
52+
# maximum number of times to retry before giving up
53+
max_tries=20,
54+
# function to execute upon a failure and when a retry a scheduled
55+
on_backoff=on_failure,
56+
# function to execute upon a successful attempt and no more retries needed
57+
on_success=on_success,
58+
)
59+
def poll_for_op_status(
60+
client: container_v1.ClusterManagerClient, op_id: str
61+
) -> container_v1.Operation.Status:
62+
"""
63+
This function calls the Operation API in GCP with the given operation id. It
64+
serves as a simple retry function that fetches the operation and returns
65+
it's status.
66+
67+
We use the 'backoff' python module to provide us the implementation of the
68+
backoff & retry strategy. The function is annotated with the `backoff`
69+
python module to schedule this function based on a reasonable backoff
70+
algorithm.
71+
"""
72+
73+
op = client.get_operation({"name": op_id})
74+
return op.status
75+
76+
77+
def create_cluster(project_id: str, location: str, cluster_name: str) -> None:
78+
"""Create a new GKE cluster in the given GCP Project and Zone"""
79+
# Initialize the Cluster management client.
80+
client = container_v1.ClusterManagerClient()
81+
# Create a fully qualified location identifier of form `projects/{project_id}/location/{zone}'.
82+
cluster_location = client.common_location_path(project_id, location)
83+
cluster_def = {
84+
"name": cluster_name,
85+
"initial_node_count": 2,
86+
"node_config": {"machine_type": "e2-standard-2"},
87+
}
88+
# Create the request object with the location identifier.
89+
request = {"parent": cluster_location, "cluster": cluster_def}
90+
create_response = client.create_cluster(request)
91+
op_identifier = f"{cluster_location}/operations/{create_response.name}"
92+
# poll for the operation status and schedule a retry until the cluster is created
93+
poll_for_op_status(client, op_identifier)
94+
95+
96+
# [END gke_create_cluster]
97+
98+
if __name__ == "__main__":
99+
parser = argparse.ArgumentParser(
100+
description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter,
101+
)
102+
parser.add_argument("project_id", help="Google Cloud project ID")
103+
parser.add_argument("zone", help="GKE Cluster zone")
104+
parser.add_argument("cluster_name", help="Name to be given to the GKE Cluster")
105+
args = parser.parse_args()
106+
107+
if len(sys.argv) != 4:
108+
parser.print_usage()
109+
sys.exit(1)
110+
111+
create_cluster(args.project_id, args.zone, args.cluster_name)
Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
# -*- coding: utf-8 -*-
2+
# Copyright 2022 Google LLC
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
16+
import os
17+
import uuid
18+
19+
import backoff
20+
21+
from google.cloud import container_v1 as gke
22+
23+
import pytest
24+
25+
import create_cluster as gke_create
26+
27+
PROJECT_ID = os.environ["GOOGLE_CLOUD_PROJECT"]
28+
ZONE = "us-central1-b"
29+
CLUSTER_NAME = f"py-container-repo-test-{uuid.uuid4().hex[:10]}"
30+
31+
32+
@pytest.fixture(autouse=True)
33+
def setup_and_tear_down() -> None:
34+
35+
# nohing to setup here
36+
37+
# run the tests here
38+
yield
39+
40+
# delete the cluster
41+
client = gke.ClusterManagerClient()
42+
cluster_location = client.common_location_path(PROJECT_ID, ZONE)
43+
cluster_name = f"{cluster_location}/clusters/{CLUSTER_NAME}"
44+
op = client.delete_cluster({"name": cluster_name})
45+
op_id = f"{cluster_location}/operations/{op.name}"
46+
47+
# schedule a retry to ensure the cluster is deleted
48+
@backoff.on_predicate(
49+
backoff.expo, lambda x: x != gke.Operation.Status.DONE, max_tries=20
50+
)
51+
def wait_for_delete() -> gke.Operation.Status:
52+
return client.get_operation({"name": op_id}).status
53+
54+
wait_for_delete()
55+
56+
57+
def test_create_clusters(capsys: object) -> None:
58+
gke_create.create_cluster(PROJECT_ID, ZONE, CLUSTER_NAME)
59+
out, _ = capsys.readouterr()
60+
61+
assert "Backing off " in out
62+
assert "Successfully created cluster after" in out
63+
64+
client = gke.ClusterManagerClient()
65+
cluster_location = client.common_location_path(PROJECT_ID, ZONE)
66+
list_response = client.list_clusters({"parent": cluster_location})
67+
68+
list_of_clusters = []
69+
for cluster in list_response.clusters:
70+
list_of_clusters.append(cluster.name)
71+
72+
assert CLUSTER_NAME in list_of_clusters

container/snippets/delete_cluster.py

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
# -*- coding: utf-8 -*-
2+
# Copyright 2022 Google LLC
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
16+
# [START gke_delete_cluster]
17+
import argparse
18+
import sys
19+
from typing import Dict
20+
21+
import backoff
22+
from google.cloud import container_v1
23+
24+
25+
def on_success(details: Dict[str, str]) -> None:
26+
"""
27+
A handler function to pass into the retry backoff algorithm as the function
28+
to be executed upon a successful attempt.
29+
30+
Read the `Event handlers` section of the backoff python module at:
31+
https://pypi.org/project/backoff/
32+
"""
33+
print("Successfully deleted cluster after {elapsed:0.1f} seconds".format(**details))
34+
35+
36+
def on_failure(details: Dict[str, str]) -> None:
37+
"""
38+
A handler function to pass into the retry backoff algorithm as the function
39+
to be executed upon a failed attempt.
40+
41+
Read the `Event handlers` section of the backoff python module at:
42+
https://pypi.org/project/backoff/
43+
"""
44+
print("Backing off {wait:0.1f} seconds after {tries} tries".format(**details))
45+
46+
47+
@backoff.on_predicate(
48+
# the backoff algorithm to use. we use exponential backoff here
49+
backoff.expo,
50+
# the test function on the return value to determine if a retry is necessary
51+
lambda x: x != container_v1.Operation.Status.DONE,
52+
# maximum number of times to retry before giving up
53+
max_tries=20,
54+
# function to execute upon a failure and when a retry is scheduled
55+
on_backoff=on_failure,
56+
# function to execute upon a successful attempt and no more retries needed
57+
on_success=on_success,
58+
)
59+
def poll_for_op_status(
60+
client: container_v1.ClusterManagerClient, op_id: str
61+
) -> container_v1.Operation.Status:
62+
"""
63+
A simple retry function that fetches the operation and returns it's status.
64+
65+
The function is annotated with the `backoff` python module to schedule this
66+
function based on a reasonable backoff algorithm
67+
"""
68+
69+
op = client.get_operation({"name": op_id})
70+
return op.status
71+
72+
73+
def delete_cluster(project_id: str, location: str, cluster_name: str) -> None:
74+
"""Delete an existing GKE cluster in the given GCP Project and Zone"""
75+
76+
# Initialize the Cluster management client.
77+
client = container_v1.ClusterManagerClient()
78+
# Create a fully qualified location identifier of form `projects/{project_id}/location/{zone}'.
79+
cluster_location = client.common_location_path(project_id, location)
80+
cluster_name = f"{cluster_location}/clusters/{cluster_name}"
81+
# Create the request object with the location identifier.
82+
request = {"name": cluster_name}
83+
delete_response = client.delete_cluster(request)
84+
op_identifier = f"{cluster_location}/operations/{delete_response.name}"
85+
# poll for the operation status until the cluster is deleted
86+
poll_for_op_status(client, op_identifier)
87+
88+
89+
# [END gke_delete_cluster]
90+
91+
if __name__ == "__main__":
92+
parser = argparse.ArgumentParser(
93+
description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter,
94+
)
95+
parser.add_argument("project_id", help="Google Cloud project ID")
96+
parser.add_argument("zone", help="GKE Cluster zone")
97+
parser.add_argument("cluster_name", help="Name to be given to the GKE Cluster")
98+
args = parser.parse_args()
99+
100+
if len(sys.argv) != 4:
101+
parser.print_usage()
102+
sys.exit(1)
103+
104+
delete_cluster(args.project_id, args.zone, args.cluster_name)

0 commit comments

Comments
 (0)