Skip to content

Commit 9f118b1

Browse files
pohlypohly
committed
Squashed 'release-tools/' changes from c6a88c6..1d60e77
1d60e77 Merge pull request kubernetes-csi#131 from pohly/kubernetes-1.20-tag 9f10459 prow.sh: support building Kubernetes for a specific version fe1f284 Merge pull request kubernetes-csi#121 from kvaps/namespace-check 8fdf0f7 Merge pull request kubernetes-csi#128 from fengzixu/master 1c94220 fix: fix a bug of csi-sanity a4c41e6 Merge pull request kubernetes-csi#127 from pohly/fix-boilerplate ece0f50 check namespace for snapshot-controller dbd8967 verify-boilerplate.sh: fix path to script 9289fd1 Merge pull request kubernetes-csi#125 from sachinkumarsingh092/optional-spelling-boilerplate-checks ad29307 Make the spelling and boilerplate checks optional 5f06d02 Merge pull request kubernetes-csi#124 from sachinkumarsingh092/fix-spellcheck-boilerplate-tests 48186eb Fix spelling and boilerplate errors 71690af Merge pull request kubernetes-csi#122 from sachinkumarsingh092/include-spellcheck-boilerplate-tests 981be3f Adding spelling and boilerplate checks. 2bb7525 Merge pull request kubernetes-csi#117 from fengzixu/master 3b6d17b Merge pull request kubernetes-csi#118 from pohly/cloud-build-timeout 9318c6c cloud build: double the timeout, now 1 hour 4ab8b15 use the tag to replace commit of csi-test 5d74e45 change the csi-test import path to v4 7dcd0a9 upgrade csi-test to v4.0.2 86ff580 Merge pull request kubernetes-csi#116 from andyzhangx/export-image-name c3a9662 allow export image name and registry name git-subtree-dir: release-tools git-subtree-split: 1d60e7792624a9938c0bd1b045211fbb89e513d6
1 parent 6dfc366 commit 9f118b1

16 files changed

+480
-34
lines changed

‎.prow.sh

+16
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,23 @@
11
#! /bin/bash -e
2+
3+
# Copyright 2021 The Kubernetes Authors.
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License");
6+
# you may not use this file except in compliance with the License.
7+
# You may obtain a copy of the License at
28
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
317
# This is for testing csi-release-tools itself in Prow. All other
418
# repos use prow.sh for that, but as csi-release-tools isn't a normal
519
# repo with some Go code in it, it has a custom Prow test script.
620

721
./verify-shellcheck.sh "$(pwd)"
22+
./verify-spelling.sh "$(pwd)"
23+
./verify-boilerplate.sh "$(pwd)"

‎SIDECAR_RELEASE_PROCESS.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ The release manager must:
1111
kubernetes/org to request membership
1212
* Be a top level approver for the repository. To become a top level approver,
1313
the candidate must demonstrate ownership and deep knowledge of the repository
14-
through active maintainence, responding to and fixing issues, reviewing PRs,
14+
through active maintenance, responding to and fixing issues, reviewing PRs,
1515
test triage.
1616
* Be part of the maintainers or admin group for the repository. admin is a
1717
superset of maintainers, only maintainers level is required for cutting a

‎boilerplate/boilerplate.Dockerfile.txt

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Copyright YEAR The Kubernetes Authors.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.

‎boilerplate/boilerplate.Makefile.txt

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Copyright YEAR The Kubernetes Authors.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.

‎boilerplate/boilerplate.bzl.txt

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Copyright YEAR The Kubernetes Authors.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.

‎boilerplate/boilerplate.go.txt

+15
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
/*
2+
Copyright YEAR The Kubernetes Authors.
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
*/

‎boilerplate/boilerplate.py

+202
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,202 @@
1+
#!/usr/bin/env python
2+
3+
# Copyright 2019 The Kubernetes Authors.
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License");
6+
# you may not use this file except in compliance with the License.
7+
# You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
17+
from __future__ import print_function
18+
19+
import argparse
20+
import difflib
21+
import glob
22+
import json
23+
import mmap
24+
import os
25+
import re
26+
import sys
27+
from datetime import date
28+
29+
parser = argparse.ArgumentParser()
30+
parser.add_argument(
31+
"filenames",
32+
help="list of files to check, all files if unspecified",
33+
nargs='*')
34+
35+
# Rootdir defaults to the directory **above** the repo-infra dir.
36+
rootdir = os.path.dirname(__file__) + "./../../../"
37+
rootdir = os.path.abspath(rootdir)
38+
parser.add_argument(
39+
"--rootdir", default=rootdir, help="root directory to examine")
40+
41+
default_boilerplate_dir = os.path.join(rootdir, "csi-driver-nfs/hack/boilerplate")
42+
43+
parser.add_argument(
44+
"--boilerplate-dir", default=default_boilerplate_dir)
45+
46+
parser.add_argument(
47+
"-v", "--verbose",
48+
help="give verbose output regarding why a file does not pass",
49+
action="store_true")
50+
51+
args = parser.parse_args()
52+
53+
verbose_out = sys.stderr if args.verbose else open("/dev/null", "w")
54+
55+
def get_refs():
56+
refs = {}
57+
58+
for path in glob.glob(os.path.join(args.boilerplate_dir, "boilerplate.*.txt")):
59+
extension = os.path.basename(path).split(".")[1]
60+
61+
ref_file = open(path, 'r')
62+
ref = ref_file.read().splitlines()
63+
ref_file.close()
64+
refs[extension] = ref
65+
66+
return refs
67+
68+
def file_passes(filename, refs, regexs):
69+
try:
70+
f = open(filename, 'r')
71+
except Exception as exc:
72+
print("Unable to open %s: %s" % (filename, exc), file=verbose_out)
73+
return False
74+
75+
data = f.read()
76+
f.close()
77+
78+
basename = os.path.basename(filename)
79+
extension = file_extension(filename)
80+
if extension != "":
81+
ref = refs[extension]
82+
else:
83+
ref = refs[basename]
84+
85+
# remove build tags from the top of Go files
86+
if extension == "go":
87+
p = regexs["go_build_constraints"]
88+
(data, found) = p.subn("", data, 1)
89+
90+
# remove shebang from the top of shell files
91+
if extension == "sh" or extension == "py":
92+
p = regexs["shebang"]
93+
(data, found) = p.subn("", data, 1)
94+
95+
data = data.splitlines()
96+
97+
# if our test file is smaller than the reference it surely fails!
98+
if len(ref) > len(data):
99+
print('File %s smaller than reference (%d < %d)' %
100+
(filename, len(data), len(ref)),
101+
file=verbose_out)
102+
return False
103+
104+
# trim our file to the same number of lines as the reference file
105+
data = data[:len(ref)]
106+
107+
p = regexs["year"]
108+
for d in data:
109+
if p.search(d):
110+
print('File %s is missing the year' % filename, file=verbose_out)
111+
return False
112+
113+
# Replace all occurrences of the regex "CURRENT_YEAR|...|2016|2015|2014" with "YEAR"
114+
p = regexs["date"]
115+
for i, d in enumerate(data):
116+
(data[i], found) = p.subn('YEAR', d)
117+
if found != 0:
118+
break
119+
120+
# if we don't match the reference at this point, fail
121+
if ref != data:
122+
print("Header in %s does not match reference, diff:" % filename, file=verbose_out)
123+
if args.verbose:
124+
print(file=verbose_out)
125+
for line in difflib.unified_diff(ref, data, 'reference', filename, lineterm=''):
126+
print(line, file=verbose_out)
127+
print(file=verbose_out)
128+
return False
129+
130+
return True
131+
132+
def file_extension(filename):
133+
return os.path.splitext(filename)[1].split(".")[-1].lower()
134+
135+
skipped_dirs = ['Godeps', 'third_party', '_gopath', '_output', '.git',
136+
'cluster/env.sh', 'vendor', 'test/e2e/generated/bindata.go',
137+
'repo-infra/verify/boilerplate/test', '.glide']
138+
139+
def normalize_files(files):
140+
newfiles = []
141+
for pathname in files:
142+
if any(x in pathname for x in skipped_dirs):
143+
continue
144+
newfiles.append(pathname)
145+
return newfiles
146+
147+
def get_files(extensions):
148+
files = []
149+
if len(args.filenames) > 0:
150+
files = args.filenames
151+
else:
152+
for root, dirs, walkfiles in os.walk(args.rootdir):
153+
# don't visit certain dirs. This is just a performance improvement
154+
# as we would prune these later in normalize_files(). But doing it
155+
# cuts down the amount of filesystem walking we do and cuts down
156+
# the size of the file list
157+
for d in skipped_dirs:
158+
if d in dirs:
159+
dirs.remove(d)
160+
161+
for name in walkfiles:
162+
pathname = os.path.join(root, name)
163+
files.append(pathname)
164+
165+
files = normalize_files(files)
166+
167+
outfiles = []
168+
for pathname in files:
169+
basename = os.path.basename(pathname)
170+
extension = file_extension(pathname)
171+
if extension in extensions or basename in extensions:
172+
outfiles.append(pathname)
173+
return outfiles
174+
175+
def get_regexs():
176+
regexs = {}
177+
# Search for "YEAR" which exists in the boilerplate, but shouldn't in the real thing
178+
regexs["year"] = re.compile( 'YEAR' )
179+
# dates can be 2014, 2015, 2016, ..., CURRENT_YEAR, company holder names can be anything
180+
years = range(2014, date.today().year + 1)
181+
regexs["date"] = re.compile( '(%s)' % "|".join(map(lambda l: str(l), years)) )
182+
# strip // +build \n\n build constraints
183+
regexs["go_build_constraints"] = re.compile(r"^(// \+build.*\n)+\n", re.MULTILINE)
184+
# strip #!.* from shell scripts
185+
regexs["shebang"] = re.compile(r"^(#!.*\n)\n*", re.MULTILINE)
186+
return regexs
187+
188+
189+
190+
def main():
191+
regexs = get_regexs()
192+
refs = get_refs()
193+
filenames = get_files(refs.keys())
194+
195+
for filename in filenames:
196+
if not file_passes(filename, refs, regexs):
197+
print(filename, file=sys.stdout)
198+
199+
return 0
200+
201+
if __name__ == "__main__":
202+
sys.exit(main())

‎boilerplate/boilerplate.py.txt

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Copyright YEAR The Kubernetes Authors.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.

‎boilerplate/boilerplate.sh.txt

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Copyright YEAR The Kubernetes Authors.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.

‎build.make

+14-1
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@
2020

2121
# This is the default. It can be overridden in the main Makefile after
2222
# including build.make.
23-
REGISTRY_NAME=quay.io/k8scsi
23+
REGISTRY_NAME?=quay.io/k8scsi
2424

2525
# Can be set to -mod=vendor to ensure that the "vendor" directory is used.
2626
GOFLAGS_VENDOR=
@@ -275,3 +275,16 @@ test-shellcheck:
275275
.PHONY: check-go-version-%
276276
check-go-version-%:
277277
./release-tools/verify-go-version.sh "$*"
278+
279+
# Test for spelling errors.
280+
.PHONY: test-spelling
281+
test-spelling:
282+
@ echo; echo "### $@:"
283+
@ ./release-tools/verify-spelling.sh "$(pwd)"
284+
285+
# Test the boilerplates of the files.
286+
.PHONY: test-boilerplate
287+
test-boilerplate:
288+
@ echo; echo "### $@:"
289+
@ ./release-tools/verify-boilerplate.sh "$(pwd)"
290+

‎cloudbuild.sh

+14
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,19 @@
11
#! /bin/bash
22

3+
# Copyright 2021 The Kubernetes Authors.
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License");
6+
# you may not use this file except in compliance with the License.
7+
# You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
317
# shellcheck disable=SC1091
418
. release-tools/prow.sh
519

‎cloudbuild.yaml

+3-1
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,9 @@
1616
# To promote release images, see https://github.com/kubernetes/k8s.io/tree/master/k8s.gcr.io/images/k8s-staging-sig-storage.
1717

1818
# This must be specified in seconds. If omitted, defaults to 600s (10 mins).
19-
timeout: 1800s
19+
# Building three images in external-snapshotter takes roughly half an hour,
20+
# sometimes more.
21+
timeout: 3600s
2022
# This prevents errors if you don't use both _GIT_TAG and _PULL_BASE_REF,
2123
# or any new substitutions added in the future.
2224
options:

0 commit comments

Comments
 (0)