Skip to content

🌱 generalize tools Dockerfiles for arm64, ppc64le linux builds #1992

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 9, 2021
Merged

🌱 generalize tools Dockerfiles for arm64, ppc64le linux builds #1992

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ GCP Cloud Build watches this branch. On every push, it runs the pipeline define

- `_GOARCH=amd64 _GOOS=darwin`
- `_GOARCH=amd64 _GOOS=linux`
- `_GOARCH=arm64 _GOOS=linux`
- `_GOARCH=ppc64le _GOOS=linux`
Comment on lines +18 to +19
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do this need to be listed somewhere else? I mean you will need to configure this list of os/arch somewhere (maybe not in code though).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I think @DirectXMan12 needs to update these vars in gcloud.


(we may add more the in the future).

Expand Down
38 changes: 26 additions & 12 deletions build/cloudbuild_tools.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,22 +12,36 @@
# See the License for the specific language governing permissions and
# limitations under the License.

substitutions:
_KUBERNETES_VERSION: v1.19.2
steps:
- name: "gcr.io/cloud-builders/docker"
args: ["build", "-t", "gcr.io/kubebuilder/thirdparty-${_GOOS}:1.20.2", "./build/thirdparty/${_GOOS}"]
args: [
"build",
"--build-arg", "OS=${_GOOS}",
"--build-arg", "ARCH=${_GOARCH}",
"--build-arg", "KUBERNETES_VERSION=${_KUBERNETES_VERSION}",
"-t", "gcr.io/kubebuilder/thirdparty-${_GOOS}-${_GOARCH}:${_KUBERNETES_VERSION}",
"./build/thirdparty/${_GOOS}",
]
# darwin takes forever
timeout: 30m

- name: "gcr.io/kubebuilder/thirdparty-${_GOOS}:1.20.2"
args: ["cp", "/kubebuilder_${_GOOS}_${_GOARCH}.tar.gz", "/workspace/kubebuilder-1.20.2-${_GOOS}-${_GOARCH}.tar.gz"]
env:
- 'GOOS=${_GOOS}'
- 'GOARCH=${_GOARCH}'
- name: 'gcr.io/cloud-builders/gsutil'
args: ['-h', 'Content-Type:application/gzip', 'cp', '/workspace/kubebuilder-1.20.2-${_GOOS}-${_GOARCH}.tar.gz', 'gs://kubebuilder-tools/kubebuilder-tools-1.20.2-${_GOOS}-${_GOARCH}.tar.gz']
env:
- 'GOOS=${_GOOS}'
- 'GOARCH=${_GOARCH}'
images: ["gcr.io/kubebuilder/thirdparty-${_GOOS}:1.20.2"]
- name: "gcr.io/kubebuilder/thirdparty-${_GOOS}-${_GOARCH}:${_KUBERNETES_VERSION}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we keep something around that cross-tags this to the old thirdparty-${OS} format too, so we don't confuse people? (this is a question, not a suggestion -- I can't quite convince myself either way)

Copy link
Contributor Author

@estroz estroz Feb 8, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this image intended to be pulled/used outside of this build system? If so then I don’t think the old format should be kept around because it is now ambiguous.

EDIT: actually for v1.19 I think we can make an exception because released kubebuilder projects already use the old format.

Copy link
Contributor

@Adirio Adirio Feb 9, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Edit: nevermind, we are distributing tarballs, not docker images. Docker is just used as the build environment.

args: [
"cp",
"/kubebuilder_${_GOOS}_${_GOARCH}.tar.gz",
"/workspace/kubebuilder-tools-${_KUBERNETES_VERSION}-${_GOOS}-${_GOARCH}.tar.gz",
]

- name: "gcr.io/cloud-builders/gsutil"
args: [
"-h", "Content-Type:application/gzip",
"cp",
"/workspace/kubebuilder-tools-${_KUBERNETES_VERSION}-${_GOOS}-${_GOARCH}.tar.gz",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto here.

In either case, will need to update the go.kubebuilder.io shortlinks

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tarball name format hasn't changed, and if it has it shouldn't have since it already includes os and arch.

"gs://kubebuilder-tools/kubebuilder-tools-${_KUBERNETES_VERSION}-${_GOOS}-${_GOARCH}.tar.gz",
]

images: ["gcr.io/kubebuilder/thirdparty-${_GOOS}-${_GOARCH}:${_KUBERNETES_VERSION}"]
# darwin takes forever
timeout: 30m
61 changes: 35 additions & 26 deletions build/thirdparty/darwin/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,45 +13,54 @@
# limitations under the License.

# Build or fetch the following binaries for darwin and then host them in a tar.gz file in an alpine image
# - apiserver (build)
# - kube-apiserver (build)
# - kubectl (fetch)
# - etcd (fetch)

FROM golang:1.15 as darwin
# Install tools
ENV CGO 0
ENV DEST /usr/local/kubebuilder/bin/
FROM golang:1.15 as builder

RUN apt update && \
apt install rsync -y && \
apt-get install unzip && \
go get github.com/jteeuwen/go-bindata/go-bindata && \
( mkdir -p $DEST || echo "" )
# Version and platform args.
ARG KUBERNETES_VERSION
ARG ETCD_VERSION=v3.4.9
ARG OS=darwin
ARG ARCH

RUN git clone https://github.com/kubernetes/kubernetes $GOPATH/src/k8s.io/kubernetes --depth=1 -b v1.20.2
# Tools path.
ENV DEST=/usr/local/kubebuilder/bin/

WORKDIR /go/src/k8s.io/kubernetes
# Install dependencies.
RUN apt update && \
apt install unzip rsync -y && \
mkdir -p $DEST

# Build for linux first otherwise it won't work for darwin - :(
ENV KUBE_BUILD_PLATFORMS linux/amd64
RUN make WHAT=cmd/kube-apiserver
ENV KUBE_BUILD_PLATFORMS darwin/amd64
# kube-apiserver
WORKDIR /kubernetes
RUN git clone https://github.com/kubernetes/kubernetes . --depth=1 -b ${KUBERNETES_VERSION}
ENV CGO_ENABLED=0
ENV KUBE_BUILD_PLATFORMS=${OS}/${ARCH}
RUN make WHAT=cmd/kube-apiserver && \
cp _output/local/bin/$KUBE_BUILD_PLATFORMS/kube-apiserver $DEST
cp _output/local/bin/${KUBE_BUILD_PLATFORMS}/kube-apiserver $DEST

RUN curl -sLO https://storage.googleapis.com/kubernetes-release/release/v1.20.2/bin/darwin/amd64/kubectl && \
# kubectl
RUN curl -sLO https://storage.googleapis.com/kubernetes-release/release/${KUBERNETES_VERSION}/bin/${OS}/${ARCH}/kubectl && \
chmod +x kubectl && \
cp kubectl $DEST

ENV ETCD_VERSION="3.4.9"
ENV ETCD_DOWNLOAD_FILE="etcd-v${ETCD_VERSION}-darwin-amd64.zip"
RUN curl -sLO https://github.com/coreos/etcd/releases/download/v${ETCD_VERSION}/etcd-v${ETCD_VERSION}-darwin-amd64.zip -o ${ETCD_DOWNLOAD_FILE} && \
unzip -o ${ETCD_DOWNLOAD_FILE} && \
cp etcd-v${ETCD_VERSION}-darwin-amd64/etcd $DEST
# etcd
ENV ETCD_BASE_NAME=etcd-${ETCD_VERSION}-${OS}-${ARCH}
RUN curl -sLO https://github.com/coreos/etcd/releases/download/${ETCD_VERSION}/${ETCD_BASE_NAME}.zip && \
unzip -o ${ETCD_BASE_NAME}.zip && \
cp ${ETCD_BASE_NAME}/etcd $DEST

# Package into tarball.
WORKDIR /usr/local
RUN tar -czvf /kubebuilder_darwin_amd64.tar.gz kubebuilder/
RUN tar -czvf /kubebuilder_${OS}_${ARCH}.tar.gz kubebuilder/

# Host the tar.gz file in a thin image
# Copy tarball to final image.
FROM alpine:3.8
COPY --from=darwin /kubebuilder_darwin_amd64.tar.gz /kubebuilder_darwin_amd64.tar.gz

# Platform args.
ARG OS=darwin
ARG ARCH

COPY --from=builder /kubebuilder_${OS}_${ARCH}.tar.gz /
52 changes: 35 additions & 17 deletions build/thirdparty/linux/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,34 +13,52 @@
# limitations under the License.

# Fetch the following into binaries for linux and then host them in a tar.gz file in an alpine image
# - apiserver (fetch)
# - kube-apiserver (fetch)
# - kubectl (fetch)
# - etcd (fetch)

# no need for anything fancy
FROM alpine:3.8 as linux
# Install tools
ENV DEST /usr/local/kubebuilder/bin/
FROM alpine:3.8 as builder

# Version and platform args.
ARG KUBERNETES_VERSION
ARG ETCD_VERSION=v3.4.9
ARG OS=linux
ARG ARCH

# Tools path.
ENV DEST=/usr/local/kubebuilder/bin/

# Install dependencies.
RUN apk add --no-cache curl && \
mkdir -p $DEST || echo ""
mkdir -p $DEST

# kube-apiserver
ENV K8S_SERVER_BASE_NAME=kubernetes-server-${OS}-${ARCH}
RUN curl -sLO https://dl.k8s.io/${KUBERNETES_VERSION}/${K8S_SERVER_BASE_NAME}.tar.gz && \
tar xzf ${K8S_SERVER_BASE_NAME}.tar.gz && \
cp kubernetes/server/bin/kube-apiserver $DEST

RUN curl -sLO https://storage.googleapis.com/kubernetes-release/release/v1.20.2/bin/linux/amd64/kubectl && \
# kubectl
RUN curl -sLO https://storage.googleapis.com/kubernetes-release/release/${KUBERNETES_VERSION}/bin/${OS}/${ARCH}/kubectl && \
chmod +x kubectl && \
cp kubectl $DEST

RUN curl -sLO https://dl.k8s.io/v1.20.2/kubernetes-server-linux-amd64.tar.gz && \
tar xzf kubernetes-server-linux-amd64.tar.gz && \
cp kubernetes/server/bin/kube-apiserver $DEST

ENV ETCD_VERSION="3.4.9"
ENV ETCD_DOWNLOAD_FILE="etcd-v${ETCD_VERSION}-linux-amd64.tar.gz"
RUN curl -sLO https://github.com/coreos/etcd/releases/download/v${ETCD_VERSION}/etcd-v${ETCD_VERSION}-linux-amd64.tar.gz -o ${ETCD_DOWNLOAD_FILE} && \
tar xzf ${ETCD_DOWNLOAD_FILE} && \
cp etcd-v${ETCD_VERSION}-linux-amd64/etcd $DEST
# etcd
ENV ETCD_BASE_NAME=etcd-${ETCD_VERSION}-${OS}-${ARCH}
RUN curl -sLO https://github.com/coreos/etcd/releases/download/${ETCD_VERSION}/${ETCD_BASE_NAME}.tar.gz && \
tar xzf ${ETCD_BASE_NAME}.tar.gz && \
cp ${ETCD_BASE_NAME}/etcd $DEST

# Package into tarball.
WORKDIR /usr/local
RUN tar -czvf /kubebuilder_linux_amd64.tar.gz kubebuilder/
RUN tar -czvf /kubebuilder_${OS}_${ARCH}.tar.gz kubebuilder/

# Copy tarball to final image.
FROM alpine:3.8
COPY --from=linux /kubebuilder_linux_amd64.tar.gz /kubebuilder_linux_amd64.tar.gz

# Platform args.
ARG OS=linux
ARG ARCH

COPY --from=builder /kubebuilder_${OS}_${ARCH}.tar.gz /