gcp-auth addon does not create image pull secret for new namespace

[katsew]

What Happened?

I enabled gcp-auth addon, and I realized that this addon does not detect namespace creation.
I usually use skaffold with helm, and it automatically creates a namespace for me.
However, with this addon, I have to run minikube addons enable gcp-auth --refresh every time helm create new namespace, and It is not good workaround for my situation.

Here is the step to reproduce.

1. run minikube addons enable gcp-auth to install addon
2. run kubectl create ns new-namespace
3. run kubectl get secret -n new-namespace

no gcp-auth secret found on that namespace.

This problem may solved by adding webhook for namespace creation.
When new namespace created, the webhook creates a imagePullSecret for the namespace.
https://github.com/GoogleContainerTools/gcp-auth-webhook

However the code of creating imagePullSecret is in this repo.
https://github.com/kubernetes/minikube/blob/master/pkg/addons/addons_gcpauth.go#L140

I'm a bit confused where to report the issue, so let me know if it is a wrong place.

Thanks.

Attach the log file

no logs for minikube.

Operating System

macOS (Default)

Driver

Docker

[RA489]

/kind support

[sharifelgamal]

yeah, this is a known issue for gcp-auth, I appreciate you filing this. i took a shot at implementing this in the webhook repo awhile ago, but never finished it. i'll take another look soon.

[klaases]

Hi @katsew, I can confirm that I am able to reproduce this issue, and will be working on finding an adequate solution.

---

Enable gcp-auth:

klaases$ ./out/minikube addons enable gcp-auth
    ▪ Using image k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0
    ▪ Using image gcr.io/k8s-minikube/gcp-auth-webhook:v0.0.8
🔎  Verifying gcp-auth addon...
📌  Your GCP credentials will now be mounted into every pod created in the minikube cluster.
📌  If you don't want your credentials mounted into a specific pod, add a label with the `gcp-auth-skip-secret` key to your pod configuration.
📌  If you want existing pods to be mounted with credentials, either recreate them or rerun addons enable with --refresh.
🌟  The 'gcp-auth' addon is enabled

Create namespace:

klaases$ kubectl create ns new-namespace
namespace/new-namespace created

Got secret:

klaases$ kubectl get secret -n new-namespace
NAME                  TYPE                                  DATA   AGE
default-token-289v6   kubernetes.io/service-account-token   3      10s

Refresh also working:

klaases$ minikube addons enable gcp-auth --refresh
🔎  Verifying gcp-auth addon...
📌  Your GCP credentials will now be mounted into every pod created in the minikube cluster.
📌  If you don't want your credentials mounted into a specific pod, add a label with the `gcp-auth-skip-secret` key to your pod configuration.
🌟  The 'gcp-auth' addon is enabled

---

Create namespace 2:

klaases$ kubectl create ns new-namespace2
namespace/new-namespace2 created

Got secret 2:

klaases$ kubectl get secret -n new-namespace2
NAME                  TYPE                                  DATA   AGE
default-token-zcvl4   kubernetes.io/service-account-token   3      8s

---

Test for expected failure for non-existent namespace3:

klaases$ kubectl get secret -n new-namespace3
No resources found in new-namespace3 namespace.

---

Creating namespace 3:

klaases$ kubectl create ns new-namespace3
namespace/new-namespace3 created

Get secret 3:

klaases$ kubectl get secret -n new-namespace3
NAME                  TYPE                                  DATA   AGE
default-token-tnzg6   kubernetes.io/service-account-token   3      18s

[sharifelgamal]

So the challenge here is that the webhook itself is very narrow in scope. It intentionally doesn't have a lot of state about the cluster itself, it really only knows about the objects sent to it via the request. That makes trying to either (a) retrieve the secret from the cluster or (b) recreating the secret for the new namespace quite difficult.