Skip to content

Commit e951326

Browse files
committed
Squashed 'release-tools/' changes from 4133d1d..f8c8cc4
f8c8cc4 Merge pull request kubernetes-csi#237 from msau42/prow b36b5bf Merge pull request kubernetes-csi#240 from dannawang0221/upgrade-go-version adfddcc Merge pull request kubernetes-csi#243 from pohly/git-subtree-pull-fix c465088 pull-test.sh: avoid "git subtree pull" error 7b175a1 Update csi-test version to v5.2.0 987c90c Update go version to 1.21 to match k/k 2c625d4 Add script to generate patch release notes f9d5b9c Merge pull request kubernetes-csi#236 from mowangdk/feature/bump_csi-driver-host-path_version b01fd53 Bump csi-driver-host-path version up to v1.12.0 984feec Merge pull request kubernetes-csi#234 from siddhikhapare/csi-tools 1f7e605 fixed broken links of testgrid dashboard de2fba8 Merge pull request kubernetes-csi#233 from andyzhangx/andyzhangx-patch-1 cee895e remove windows 20H2 build since it's EOL long time ago 670bb0e Merge pull request kubernetes-csi#229 from marosset/fix-codespell-errors 35d5e78 Merge pull request kubernetes-csi#219 from yashsingh74/update-registry 63473cc Merge pull request kubernetes-csi#231 from coulof/bump-go-version-1.20.5 29a5c76 Merge pull request kubernetes-csi#228 from mowangdk/chore/adopt_kubernetes_recommand_labels 8dd2821 Update cloudbuild image with go 1.20.5 1df23db Merge pull request kubernetes-csi#230 from msau42/prow 1f92b7e Add ginkgo timeout to e2e tests to help catch any stuck tests 2b8b80e fixing some codespell errors c10b678 Merge pull request kubernetes-csi#227 from coulof/check-sidecar-supported-versions 72984ec chore: adopt kubernetes recommand label b055535 Header bd0a10b typo c39d73c Add comments f6491af Script to verify EOL sidecar version 901bcb5 Update registry k8s.gcr.io -> registry.k8s.io git-subtree-dir: release-tools git-subtree-split: f8c8cc4
1 parent 591fa77 commit e951326

6 files changed

+341
-63
lines changed

SIDECAR_RELEASE_PROCESS.md

Lines changed: 40 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ The release manager must:
1717
Whenever a new Kubernetes minor version is released, our kubernetes-csi CI jobs
1818
must be updated.
1919

20-
[Our CI jobs](https://k8s-testgrid.appspot.com/sig-storage-csi-ci) have the
20+
[Our CI jobs](https://testgrid.k8s.io/sig-storage-csi-ci) have the
2121
naming convention `<hostpath-deployment-version>-on-<kubernetes-version>`.
2222

2323
1. Jobs should be actively monitored to find and fix failures in sidecars and
@@ -46,63 +46,54 @@ naming convention `<hostpath-deployment-version>-on-<kubernetes-version>`.
4646
## Release Process
4747
1. Identify all issues and ongoing PRs that should go into the release, and
4848
drive them to resolution.
49-
1. Download the latest version of the
50-
[K8s release notes generator](https://github.com/kubernetes/release/tree/HEAD/cmd/release-notes)
51-
1. Create a
52-
[Github personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)
53-
with `repo:public_repo` access
54-
1. Generate release notes for the release. Replace arguments with the relevant
55-
information.
56-
* Clean up old cached information (also needed if you are generating release
57-
notes for multiple repos)
58-
```bash
59-
rm -rf /tmp/k8s-repo
60-
```
61-
* For new minor releases on master:
62-
```bash
63-
GITHUB_TOKEN=<token> release-notes \
64-
--discover=mergebase-to-latest \
65-
--org=kubernetes-csi \
66-
--repo=external-provisioner \
67-
--required-author="" \
68-
--markdown-links \
69-
--output out.md
70-
```
71-
* For new patch releases on a release branch:
72-
```bash
73-
GITHUB_TOKEN=<token> release-notes \
74-
--discover=patch-to-latest \
75-
--branch=release-1.1 \
76-
--org=kubernetes-csi \
77-
--repo=external-provisioner \
78-
--required-author="" \
79-
--markdown-links \
80-
--output out.md
81-
```
82-
1. Compare the generated output to the new commits for the release to check if
83-
any notable change missed a release note.
84-
1. Reword release notes as needed. Make sure to check notes for breaking
85-
changes and deprecations.
86-
1. If release is a new major/minor version, create a new `CHANGELOG-<major>.<minor>.md`
87-
file. Otherwise, add the release notes to the top of the existing CHANGELOG
88-
file for that minor version.
89-
1. Submit a PR for the CHANGELOG changes.
90-
1. Submit a PR for README changes, in particular, Compatibility, Feature status,
91-
and any other sections that may need updating.
9249
1. Check that all [canary CI
93-
jobs](https://k8s-testgrid.appspot.com/sig-storage-csi-ci) are passing,
50+
jobs](https://testgrid.k8s.io/sig-storage-csi-ci) are passing,
9451
and that test coverage is adequate for the changes that are going into the release.
9552
1. Check that the post-\<sidecar\>-push-images builds are succeeding.
96-
[Example](https://k8s-testgrid.appspot.com/sig-storage-image-build#post-external-snapshotter-push-images)
53+
[Example](https://testgrid.k8s.io/sig-storage-image-build#post-external-snapshotter-push-images)
54+
1. Generate release notes.
55+
1. Download the latest version of the [K8s release notes generator](https://github.com/kubernetes/release/tree/HEAD/cmd/release-notes)
56+
1. Create a
57+
[Github personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)
58+
with `repo:public_repo` access
59+
1. For patch release, use the script generate_patch_release_notes.sh. Read the instructions at the top of the
60+
script. The script also creates PRs for each branch.
61+
1. For new minor releases, follow these steps and replace arguments with the relevant
62+
information.
63+
* Clean up old cached information (also needed if you are generating release
64+
notes for multiple repos)
65+
```bash
66+
rm -rf /tmp/k8s-repo
67+
```
68+
* For new minor releases on master:
69+
```bash
70+
GITHUB_TOKEN=<token> release-notes \
71+
--discover=mergebase-to-latest \
72+
--org=kubernetes-csi \
73+
--repo=external-provisioner \
74+
--required-author="" \
75+
--markdown-links \
76+
--output out.md
77+
```
78+
1. Compare the generated output to the new commits for the release to check if
79+
any notable change missed a release note.
80+
1. Reword release notes as needed, ideally in the original PRs so that the
81+
release notes can be regnerated. Make sure to check notes for breaking
82+
changes and deprecations.
83+
1. If release is a new major/minor version, create a new `CHANGELOG-<major>.<minor>.md`
84+
file.
85+
1. Submit a PR for the CHANGELOG changes.
86+
1. Submit a PR for README changes, in particular, Compatibility, Feature status,
87+
and any other sections that may need updating.
9788
1. Make sure that no new PRs have merged in the meantime, and no PRs are in
9889
flight and soon to be merged.
9990
1. Create a new release following a previous release as a template. Be sure to select the correct
10091
branch. This requires Github release permissions as required by the prerequisites.
10192
[external-provisioner example](https://github.com/kubernetes-csi/external-provisioner/releases/new)
10293
1. If release was a new major/minor version, create a new `release-<minor>`
10394
branch at that commit.
104-
1. Check [image build status](https://k8s-testgrid.appspot.com/sig-storage-image-build).
105-
1. Promote images from k8s-staging-sig-storage to k8s.gcr.io/sig-storage. From
95+
1. Check [image build status](https://testgrid.k8s.io/sig-storage-image-build).
96+
1. Promote images from k8s-staging-sig-storage to registry.k8s.io/sig-storage. From
10697
the [k8s image
10798
repo](https://github.com/kubernetes/k8s.io/tree/HEAD/registry.k8s.io/images/k8s-staging-sig-storage),
10899
run `./generate.sh > images.yaml`, and send a PR with the updated images.
@@ -120,7 +111,7 @@ naming convention `<hostpath-deployment-version>-on-<kubernetes-version>`.
120111

121112
The following jobs are triggered after tagging to produce the corresponding
122113
image(s):
123-
https://k8s-testgrid.appspot.com/sig-storage-image-build
114+
https://testgrid.k8s.io/sig-storage-image-build
124115

125116
Clicking on a failed build job opens that job in https://prow.k8s.io. Next to
126117
the job title is a rerun icon (circle with arrow). Clicking it opens a popup

cloudbuild.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
# See https://github.com/kubernetes/test-infra/blob/HEAD/config/jobs/image-pushing/README.md
1414
# for more details on image pushing process in Kubernetes.
1515
#
16-
# To promote release images, see https://github.com/kubernetes/k8s.io/tree/HEAD/k8s.gcr.io/images/k8s-staging-sig-storage.
16+
# To promote release images, see https://github.com/kubernetes/k8s.io/tree/HEAD/registry.k8s.io/images/k8s-staging-sig-storage.
1717

1818
# This must be specified in seconds. If omitted, defaults to 600s (10 mins).
1919
# Building three images in external-snapshotter takes more than an hour.
@@ -26,7 +26,7 @@ steps:
2626
# The image must contain bash and curl. Ideally it should also contain
2727
# the desired version of Go (currently defined in release-tools/prow.sh),
2828
# but that just speeds up the build and is not required.
29-
- name: 'gcr.io/k8s-testimages/gcb-docker-gcloud:v20230424-910a2a439d'
29+
- name: 'gcr.io/k8s-testimages/gcb-docker-gcloud:v20230623-56e06d7c18'
3030
entrypoint: ./.cloudbuild.sh
3131
env:
3232
- GIT_TAG=${_GIT_TAG}
Lines changed: 170 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
# Copyright 2023 The Kubernetes Authors.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
import argparse
16+
import datetime
17+
import re
18+
from collections import defaultdict
19+
import subprocess
20+
import shutil
21+
from dateutil.relativedelta import relativedelta
22+
23+
def check_gh_command():
24+
"""
25+
Pretty much everything is processed from `gh`
26+
Check that the `gh` command is in the path before anything else
27+
"""
28+
if not shutil.which('gh'):
29+
print("Error: The `gh` command is not available in the PATH.")
30+
print("Please install the GitHub CLI (https://cli.github.com/) and try again.")
31+
exit(1)
32+
33+
def duration_ago(dt):
34+
"""
35+
Humanize duration outputs
36+
"""
37+
delta = relativedelta(datetime.datetime.now(), dt)
38+
if delta.years > 0:
39+
return f"{delta.years} year{'s' if delta.years > 1 else ''} ago"
40+
elif delta.months > 0:
41+
return f"{delta.months} month{'s' if delta.months > 1 else ''} ago"
42+
elif delta.days > 0:
43+
return f"{delta.days} day{'s' if delta.days > 1 else ''} ago"
44+
elif delta.hours > 0:
45+
return f"{delta.hours} hour{'s' if delta.hours > 1 else ''} ago"
46+
elif delta.minutes > 0:
47+
return f"{delta.minutes} minute{'s' if delta.minutes > 1 else ''} ago"
48+
else:
49+
return "just now"
50+
51+
def parse_version(version):
52+
"""
53+
Parse version assuming it is in the form of v1.2.3
54+
"""
55+
pattern = r"v(\d+)\.(\d+)\.(\d+)"
56+
match = re.match(pattern, version)
57+
if match:
58+
major, minor, patch = map(int, match.groups())
59+
return (major, minor, patch)
60+
61+
def end_of_life_grouped_versions(versions):
62+
"""
63+
Calculate the end of life date for a minor release version according to : https://kubernetes-csi.github.io/docs/project-policies.html#support
64+
65+
The input is an array of tuples of:
66+
* grouped versions (e.g. 1.0, 1.1)
67+
* array of that contains all versions and their release date (e.g. 1.0.0, 01-01-2013)
68+
69+
versions structure example :
70+
[((3, 5), [('v3.5.0', datetime.datetime(2023, 4, 27, 22, 28, 6))]),
71+
((3, 4),
72+
[('v3.4.1', datetime.datetime(2023, 4, 5, 17, 41, 15)),
73+
('v3.4.0', datetime.datetime(2022, 12, 27, 23, 43, 41))])]
74+
"""
75+
supported_versions = []
76+
# Prepare dates for later calculation
77+
now = datetime.datetime.now()
78+
one_year = datetime.timedelta(days=365)
79+
three_months = datetime.timedelta(days=90)
80+
81+
# get the newer versions on top
82+
sorted_versions_list = sorted(versions.items(), key=lambda x: x[0], reverse=True)
83+
84+
# the latest version is always supported no matter the release date
85+
latest = sorted_versions_list.pop(0)
86+
supported_versions.append(latest[1][-1])
87+
88+
for v in sorted_versions_list:
89+
first_release = v[1][-1]
90+
last_release = v[1][0]
91+
# if the release is less than a year old we support the latest patch version
92+
if now - first_release[1] < one_year:
93+
supported_versions.append(last_release)
94+
# if the main release is older than a year and has a recent path, this is supported
95+
elif now - last_release[1] < three_months:
96+
supported_versions.append(last_release)
97+
return supported_versions
98+
99+
def get_release_docker_image(repo, version):
100+
"""
101+
Extract docker image name from the release page documentation
102+
"""
103+
output = subprocess.check_output(['gh', 'release', '-R', repo, 'view', version], text=True)
104+
#Extract matching image name excluding `
105+
match = re.search(r"docker pull ([\.\/\-\:\w\d]*)", output)
106+
docker_image = match.group(1) if match else ''
107+
return((version, docker_image))
108+
109+
def get_versions_from_releases(repo):
110+
"""
111+
Using `gh` cli get the github releases page details then
112+
create a list of grouped version on major.minor
113+
and for each give all major.minor.patch with release dates
114+
"""
115+
# Run the `gh release` command to get the release list
116+
output = subprocess.check_output(['gh', 'release', '-R', repo, 'list'], text=True)
117+
# Parse the output and group by major and minor version numbers
118+
versions = defaultdict(lambda: [])
119+
for line in output.strip().split('\n'):
120+
parts = line.split('\t')
121+
# pprint.pprint(parts)
122+
version = parts[0]
123+
parsed_version = parse_version(version)
124+
if parsed_version is None:
125+
continue
126+
major, minor, patch = parsed_version
127+
128+
published = datetime.datetime.strptime(parts[3], '%Y-%m-%dT%H:%M:%SZ')
129+
versions[(major, minor)].append((version, published))
130+
return(versions)
131+
132+
133+
def main():
134+
manual = """
135+
This script lists the supported versions Github releases according to https://kubernetes-csi.github.io/docs/project-policies.html#support
136+
It has been designed to help to update the tables from : https://kubernetes-csi.github.io/docs/sidecar-containers.html\n\n
137+
It can take multiple repos as argument, for all CSI sidecars details you can run:
138+
./get_supported_version_csi-sidecar.py -R kubernetes-csi/external-attacher -R kubernetes-csi/external-provisioner -R kubernetes-csi/external-resizer -R kubernetes-csi/external-snapshotter -R kubernetes-csi/livenessprobe -R kubernetes-csi/node-driver-registrar -R kubernetes-csi/external-health-monitor\n
139+
With the output you can then update the documentation manually.
140+
"""
141+
parser = argparse.ArgumentParser(formatter_class=argparse.RawDescriptionHelpFormatter, description=manual)
142+
parser.add_argument('--repo', '-R', required=True, action='append', dest='repos', help='The name of the repository in the format owner/repo.')
143+
parser.add_argument('--display', '-d', action='store_true', help='(default) Display EOL versions with their dates', default=True)
144+
parser.add_argument('--doc', '-D', action='store_true', help='Helper to https://kubernetes-csi.github.io/docs/ that prints Docker image for each EOL version')
145+
146+
args = parser.parse_args()
147+
148+
# Verify pre-reqs
149+
check_gh_command()
150+
151+
# Process all repos
152+
for repo in args.repos:
153+
versions = get_versions_from_releases(repo)
154+
eol_versions = end_of_life_grouped_versions(versions)
155+
156+
if args.display:
157+
print(f"Supported versions with release date and age of `{repo}`:\n")
158+
for version in eol_versions:
159+
print(f"{version[0]}\t{version[1].strftime('%Y-%m-%d')}\t{duration_ago(version[1])}")
160+
161+
# TODO : generate proper doc output for the tables of: https://kubernetes-csi.github.io/docs/sidecar-containers.html
162+
if args.doc:
163+
print("\nSupported Versions with docker images for each end of life version:\n")
164+
for version in eol_versions:
165+
_, image = get_release_docker_image(repo, version[0])
166+
print(f"{version[0]}\t{image}")
167+
print()
168+
169+
if __name__ == '__main__':
170+
main()

0 commit comments

Comments
 (0)