Skip to content

EC2 controller - crash loop back off - when upgrading to 1.2.2 #2007

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
gecube opened this issue Jan 31, 2024 · 13 comments
Closed

EC2 controller - crash loop back off - when upgrading to 1.2.2 #2007

gecube opened this issue Jan 31, 2024 · 13 comments
Labels
service/ec2 Indicates issues or PRs that are related to ec2-controller.

Comments

@gecube
Copy link

gecube commented Jan 31, 2024

Good day!

Today I noticed that EC2 controller was updated:

  conditions:
    - lastTransitionTime: '2024-01-29T08:15:45Z'
      message: Release reconciliation succeeded
      reason: ReconciliationSucceeded
      status: 'True'
      type: Ready
    - lastTransitionTime: '2024-01-29T08:15:45Z'
      message: Helm upgrade succeeded
      reason: UpgradeSucceeded
      status: 'True'
      type: Released

to the version 1.2.2

  helmChart: ack-system/ack-system-ack-ec2-controller
  lastAppliedRevision: 1.2.2
  lastAttemptedRevision: 1.2.2

In the meantime EC2 controller pod begin to fail:

Screenshot 2024-01-31 at 12 12 20
@gecube
Copy link
Author

gecube commented Jan 31, 2024

Logs are attached

{"level":"info","ts":"2024-01-31T11:07:01.629Z","logger":"setup","msg":"initializing service controller","aws.service":"ec2"}
{"level":"info","ts":"2024-01-31T11:07:02.696Z","msg":"Waited for 1.042559256s due to client-side throttling, not priority and fairness, request: GET:https://172.20.0.1:443/apis/snapshot.storage.k8s.io/v1\n"}
{"level":"info","ts":"2024-01-31T11:07:07.860Z","logger":"setup","msg":"starting manager","aws.service":"ec2"}
{"level":"info","ts":"2024-01-31T11:07:07.860Z","logger":"controller-runtime.metrics","msg":"Starting metrics server"}
{"level":"info","ts":"2024-01-31T11:07:07.860Z","msg":"attempting to acquire leader lease ack-system/ack-ec2.services.k8s.aws...\n"}
{"level":"info","ts":"2024-01-31T11:07:07.860Z","logger":"controller-runtime.metrics","msg":"Serving metrics server","bindAddress":"0.0.0.0:8080","secure":false}
{"level":"info","ts":"2024-01-31T11:07:24.275Z","msg":"successfully acquired lease ack-system/ack-ec2.services.k8s.aws\n"}
{"level":"info","ts":"2024-01-31T11:07:24.275Z","msg":"Starting EventSource","controller":"adoptedresource","controllerGroup":"services.k8s.aws","controllerKind":"AdoptedResource","source":"kind source: *v1alpha1.AdoptedResource"}
{"level":"info","ts":"2024-01-31T11:07:24.275Z","msg":"Starting Controller","controller":"adoptedresource","controllerGroup":"services.k8s.aws","controllerKind":"AdoptedResource"}
{"level":"info","ts":"2024-01-31T11:07:24.275Z","msg":"Starting EventSource","controller":"vpc","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPC","source":"kind source: *v1alpha1.VPC"}
{"level":"info","ts":"2024-01-31T11:07:24.275Z","msg":"Starting EventSource","controller":"fieldexport","controllerGroup":"services.k8s.aws","controllerKind":"FieldExport","source":"kind source: *v1alpha1.FieldExport"}
{"level":"info","ts":"2024-01-31T11:07:24.275Z","msg":"Starting Controller","controller":"fieldexport","controllerGroup":"services.k8s.aws","controllerKind":"FieldExport"}
{"level":"info","ts":"2024-01-31T11:07:24.275Z","msg":"Starting Controller","controller":"vpc","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPC"}
{"level":"info","ts":"2024-01-31T11:07:24.275Z","msg":"Starting EventSource","controller":"flowlog","controllerGroup":"ec2.services.k8s.aws","controllerKind":"FlowLog","source":"kind source: *v1alpha1.FlowLog"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"flowlog","controllerGroup":"ec2.services.k8s.aws","controllerKind":"FlowLog"}
{"level":"info","ts":"2024-01-31T11:07:24.275Z","msg":"Starting EventSource","controller":"vpc","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPC","source":"kind source: *v1alpha1.VPC"}
{"level":"info","ts":"2024-01-31T11:07:24.275Z","msg":"Starting EventSource","controller":"routetable","controllerGroup":"ec2.services.k8s.aws","controllerKind":"RouteTable","source":"kind source: *v1alpha1.RouteTable"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"vpc","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPC"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"routetable","controllerGroup":"ec2.services.k8s.aws","controllerKind":"RouteTable"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting EventSource","controller":"routetable","controllerGroup":"ec2.services.k8s.aws","controllerKind":"RouteTable","source":"kind source: *v1alpha1.RouteTable"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting EventSource","controller":"vpcendpoint","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPCEndpoint","source":"kind source: *v1alpha1.VPCEndpoint"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"routetable","controllerGroup":"ec2.services.k8s.aws","controllerKind":"RouteTable"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"vpcendpoint","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPCEndpoint"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting EventSource","controller":"flowlog","controllerGroup":"ec2.services.k8s.aws","controllerKind":"FlowLog","source":"kind source: *v1alpha1.FlowLog"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"flowlog","controllerGroup":"ec2.services.k8s.aws","controllerKind":"FlowLog"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting EventSource","controller":"elasticipaddress","controllerGroup":"ec2.services.k8s.aws","controllerKind":"ElasticIPAddress","source":"kind source: *v1alpha1.ElasticIPAddress"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"elasticipaddress","controllerGroup":"ec2.services.k8s.aws","controllerKind":"ElasticIPAddress"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting EventSource","controller":"networkacl","controllerGroup":"ec2.services.k8s.aws","controllerKind":"NetworkACL","source":"kind source: *v1alpha1.NetworkACL"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting EventSource","controller":"vpcendpoint","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPCEndpoint","source":"kind source: *v1alpha1.VPCEndpoint"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"networkacl","controllerGroup":"ec2.services.k8s.aws","controllerKind":"NetworkACL"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"vpcendpoint","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPCEndpoint"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting EventSource","controller":"transitgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"TransitGateway","source":"kind source: *v1alpha1.TransitGateway"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"transitgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"TransitGateway"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting EventSource","controller":"networkacl","controllerGroup":"ec2.services.k8s.aws","controllerKind":"NetworkACL","source":"kind source: *v1alpha1.NetworkACL"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"networkacl","controllerGroup":"ec2.services.k8s.aws","controllerKind":"NetworkACL"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting EventSource","controller":"elasticipaddress","controllerGroup":"ec2.services.k8s.aws","controllerKind":"ElasticIPAddress","source":"kind source: *v1alpha1.ElasticIPAddress"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"elasticipaddress","controllerGroup":"ec2.services.k8s.aws","controllerKind":"ElasticIPAddress"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting EventSource","controller":"internetgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"InternetGateway","source":"kind source: *v1alpha1.InternetGateway"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"internetgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"InternetGateway"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting EventSource","controller":"internetgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"InternetGateway","source":"kind source: *v1alpha1.InternetGateway"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"internetgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"InternetGateway"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting EventSource","controller":"subnet","controllerGroup":"ec2.services.k8s.aws","controllerKind":"Subnet","source":"kind source: *v1alpha1.Subnet"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"subnet","controllerGroup":"ec2.services.k8s.aws","controllerKind":"Subnet"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting EventSource","controller":"dhcpoptions","controllerGroup":"ec2.services.k8s.aws","controllerKind":"DHCPOptions","source":"kind source: *v1alpha1.DHCPOptions"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"dhcpoptions","controllerGroup":"ec2.services.k8s.aws","controllerKind":"DHCPOptions"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting EventSource","controller":"natgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"NATGateway","source":"kind source: *v1alpha1.NATGateway"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"natgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"NATGateway"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting EventSource","controller":"dhcpoptions","controllerGroup":"ec2.services.k8s.aws","controllerKind":"DHCPOptions","source":"kind source: *v1alpha1.DHCPOptions"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"dhcpoptions","controllerGroup":"ec2.services.k8s.aws","controllerKind":"DHCPOptions"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting EventSource","controller":"securitygroup","controllerGroup":"ec2.services.k8s.aws","controllerKind":"SecurityGroup","source":"kind source: *v1alpha1.SecurityGroup"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"securitygroup","controllerGroup":"ec2.services.k8s.aws","controllerKind":"SecurityGroup"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting EventSource","controller":"securitygroup","controllerGroup":"ec2.services.k8s.aws","controllerKind":"SecurityGroup","source":"kind source: *v1alpha1.SecurityGroup"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"securitygroup","controllerGroup":"ec2.services.k8s.aws","controllerKind":"SecurityGroup"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting EventSource","controller":"natgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"NATGateway","source":"kind source: *v1alpha1.NATGateway"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"natgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"NATGateway"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting EventSource","controller":"transitgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"TransitGateway","source":"kind source: *v1alpha1.TransitGateway"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"transitgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"TransitGateway"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting EventSource","controller":"subnet","controllerGroup":"ec2.services.k8s.aws","controllerKind":"Subnet","source":"kind source: *v1alpha1.Subnet"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"subnet","controllerGroup":"ec2.services.k8s.aws","controllerKind":"Subnet"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting EventSource","controller":"instance","controllerGroup":"ec2.services.k8s.aws","controllerKind":"Instance","source":"kind source: *v1alpha1.Instance"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"instance","controllerGroup":"ec2.services.k8s.aws","controllerKind":"Instance"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting EventSource","controller":"instance","controllerGroup":"ec2.services.k8s.aws","controllerKind":"Instance","source":"kind source: *v1alpha1.Instance"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"instance","controllerGroup":"ec2.services.k8s.aws","controllerKind":"Instance"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting EventSource","controller":"vpcendpointserviceconfiguration","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPCEndpointServiceConfiguration","source":"kind source: *v1alpha1.VPCEndpointServiceConfiguration"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"vpcendpointserviceconfiguration","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPCEndpointServiceConfiguration"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting EventSource","controller":"vpcpeeringconnection","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPCPeeringConnection","source":"kind source: *v1alpha1.VPCPeeringConnection"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"vpcpeeringconnection","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPCPeeringConnection"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting EventSource","controller":"vpcendpointserviceconfiguration","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPCEndpointServiceConfiguration","source":"kind source: *v1alpha1.VPCEndpointServiceConfiguration"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"vpcendpointserviceconfiguration","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPCEndpointServiceConfiguration"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting EventSource","controller":"vpcpeeringconnection","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPCPeeringConnection","source":"kind source: *v1alpha1.VPCPeeringConnection"}
{"level":"info","ts":"2024-01-31T11:07:24.276Z","msg":"Starting Controller","controller":"vpcpeeringconnection","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPCPeeringConnection"}
{"level":"error","ts":"2024-01-31T11:07:24.340Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCPeeringConnection.ec2.services.k8s.aws","error":"no matches for kind \"VPCPeeringConnection\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:49\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:50\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:07:24.440Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCPeeringConnection.ec2.services.k8s.aws","error":"no matches for kind \"VPCPeeringConnection\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:07:24.443Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCEndpointServiceConfiguration.ec2.services.k8s.aws","error":"no matches for kind \"VPCEndpointServiceConfiguration\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:49\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:50\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"info","ts":"2024-01-31T11:07:24.540Z","msg":"Starting workers","controller":"natgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"NATGateway","worker count":1}
{"level":"info","ts":"2024-01-31T11:07:24.540Z","msg":"Starting workers","controller":"natgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"NATGateway","worker count":1}
{"level":"info","ts":"2024-01-31T11:07:24.540Z","msg":"Starting workers","controller":"dhcpoptions","controllerGroup":"ec2.services.k8s.aws","controllerKind":"DHCPOptions","worker count":1}
{"level":"info","ts":"2024-01-31T11:07:24.540Z","msg":"Starting workers","controller":"fieldexport","controllerGroup":"services.k8s.aws","controllerKind":"FieldExport","worker count":1}
{"level":"info","ts":"2024-01-31T11:07:24.540Z","msg":"Starting workers","controller":"dhcpoptions","controllerGroup":"ec2.services.k8s.aws","controllerKind":"DHCPOptions","worker count":1}
{"level":"info","ts":"2024-01-31T11:07:24.540Z","msg":"Starting workers","controller":"transitgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"TransitGateway","worker count":1}
{"level":"info","ts":"2024-01-31T11:07:24.540Z","msg":"Starting workers","controller":"routetable","controllerGroup":"ec2.services.k8s.aws","controllerKind":"RouteTable","worker count":1}
{"level":"info","ts":"2024-01-31T11:07:24.540Z","msg":"Starting workers","controller":"flowlog","controllerGroup":"ec2.services.k8s.aws","controllerKind":"FlowLog","worker count":1}
{"level":"info","ts":"2024-01-31T11:07:24.540Z","msg":"Starting workers","controller":"vpc","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPC","worker count":1}
{"level":"info","ts":"2024-01-31T11:07:24.540Z","msg":"Starting workers","controller":"vpcendpoint","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPCEndpoint","worker count":1}
{"level":"info","ts":"2024-01-31T11:07:24.543Z","msg":"Starting workers","controller":"elasticipaddress","controllerGroup":"ec2.services.k8s.aws","controllerKind":"ElasticIPAddress","worker count":1}
{"level":"error","ts":"2024-01-31T11:07:24.543Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCEndpointServiceConfiguration.ec2.services.k8s.aws","error":"no matches for kind \"VPCEndpointServiceConfiguration\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:49\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:50\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:07:24.639Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCPeeringConnection.ec2.services.k8s.aws","error":"no matches for kind \"VPCPeeringConnection\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:49\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:50\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:07:24.650Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCEndpointServiceConfiguration.ec2.services.k8s.aws","error":"no matches for kind \"VPCEndpointServiceConfiguration\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:07:24.739Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCEndpointServiceConfiguration.ec2.services.k8s.aws","error":"no matches for kind \"VPCEndpointServiceConfiguration\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"info","ts":"2024-01-31T11:07:24.739Z","msg":"Starting workers","controller":"vpcendpoint","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPCEndpoint","worker count":1}
{"level":"info","ts":"2024-01-31T11:07:24.739Z","msg":"Starting workers","controller":"securitygroup","controllerGroup":"ec2.services.k8s.aws","controllerKind":"SecurityGroup","worker count":1}
{"level":"info","ts":"2024-01-31T11:07:24.739Z","msg":"Starting workers","controller":"securitygroup","controllerGroup":"ec2.services.k8s.aws","controllerKind":"SecurityGroup","worker count":1}
{"level":"info","ts":"2024-01-31T11:07:24.743Z","msg":"Starting workers","controller":"internetgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"InternetGateway","worker count":1}
{"level":"error","ts":"2024-01-31T11:07:24.743Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCPeeringConnection.ec2.services.k8s.aws","error":"no matches for kind \"VPCPeeringConnection\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"info","ts":"2024-01-31T11:07:24.743Z","msg":"Starting workers","controller":"internetgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"InternetGateway","worker count":1}
{"level":"info","ts":"2024-01-31T11:07:24.743Z","msg":"Starting workers","controller":"subnet","controllerGroup":"ec2.services.k8s.aws","controllerKind":"Subnet","worker count":1}
{"level":"info","ts":"2024-01-31T11:07:24.743Z","msg":"Starting workers","controller":"adoptedresource","controllerGroup":"services.k8s.aws","controllerKind":"AdoptedResource","worker count":1}
{"level":"info","ts":"2024-01-31T11:07:25.039Z","msg":"Starting workers","controller":"networkacl","controllerGroup":"ec2.services.k8s.aws","controllerKind":"NetworkACL","worker count":1}
{"level":"info","ts":"2024-01-31T11:07:25.039Z","msg":"Starting workers","controller":"instance","controllerGroup":"ec2.services.k8s.aws","controllerKind":"Instance","worker count":1}
{"level":"info","ts":"2024-01-31T11:07:25.039Z","msg":"Starting workers","controller":"flowlog","controllerGroup":"ec2.services.k8s.aws","controllerKind":"FlowLog","worker count":1}
{"level":"info","ts":"2024-01-31T11:07:25.039Z","msg":"Starting workers","controller":"networkacl","controllerGroup":"ec2.services.k8s.aws","controllerKind":"NetworkACL","worker count":1}
{"level":"info","ts":"2024-01-31T11:07:25.039Z","msg":"Starting workers","controller":"instance","controllerGroup":"ec2.services.k8s.aws","controllerKind":"Instance","worker count":1}
{"level":"info","ts":"2024-01-31T11:07:25.039Z","msg":"Starting workers","controller":"routetable","controllerGroup":"ec2.services.k8s.aws","controllerKind":"RouteTable","worker count":1}
{"level":"info","ts":"2024-01-31T11:07:25.039Z","msg":"Starting workers","controller":"subnet","controllerGroup":"ec2.services.k8s.aws","controllerKind":"Subnet","worker count":1}
{"level":"info","ts":"2024-01-31T11:07:25.039Z","msg":"Starting workers","controller":"elasticipaddress","controllerGroup":"ec2.services.k8s.aws","controllerKind":"ElasticIPAddress","worker count":1}
{"level":"info","ts":"2024-01-31T11:07:25.039Z","msg":"Starting workers","controller":"transitgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"TransitGateway","worker count":1}
{"level":"info","ts":"2024-01-31T11:07:25.039Z","msg":"Starting workers","controller":"vpc","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPC","worker count":1}
{"level":"info","ts":"2024-01-31T11:07:26.039Z","logger":"ackrt","msg":"desired resource state has changed","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"FlowLog","namespace":"infra-production","name":"vpc-production-flow-log","is_adopted":false,"generation":2,"diff":[{"Path":{"Parts":["Spec","LogFormat"]},"A":null,"B":"${version} ${account-id} ${interface-id} ${srcaddr} ${dstaddr} ${srcport} ${dstport} ${protocol} ${packets} ${bytes} ${start} ${end} ${action} ${log-status}"},{"Path":{"Parts":["Spec","MaxAggregationInterval"]},"A":null,"B":600}]}
{"level":"info","ts":"2024-01-31T11:07:26.039Z","logger":"ackrt","msg":"updated resource","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"FlowLog","namespace":"infra-production","name":"vpc-production-flow-log","is_adopted":false,"generation":2}
{"level":"info","ts":"2024-01-31T11:07:26.040Z","logger":"ackrt","msg":"desired resource state has changed","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Instance","namespace":"infra-production","name":"ssh-jump-box","is_adopted":false,"generation":4,"diff":[{"Path":{"Parts":["Spec","BlockDeviceMappings"]},"A":[{"deviceName":"/dev/sda1","ebs":{"deleteOnTermination":false,"volumeSize":20}}],"B":[{"deviceName":"/dev/sda1","ebs":{"deleteOnTermination":false}}]}]}
{"level":"info","ts":"2024-01-31T11:07:26.040Z","logger":"ackrt","msg":"updated resource","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Instance","namespace":"infra-production","name":"ssh-jump-box","is_adopted":false,"generation":4}
{"level":"info","ts":"2024-01-31T11:07:26.239Z","logger":"ackrt","msg":"desired resource state has changed","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","kind":"Subnet","namespace":"infra-dev","name":"dev-public-eu-west-2c","is_adopted":false,"generation":2,"diff":[{"Path":{"Parts":["Spec","EnableResourceNameDNSAAAARecord"]},"A":null,"B":false},{"Path":{"Parts":["Spec","EnableResourceNameDNSARecord"]},"A":null,"B":false},{"Path":{"Parts":["Spec","HostnameType"]},"A":null,"B":"ip-name"}]}
{"level":"info","ts":"2024-01-31T11:07:26.239Z","logger":"ackrt","msg":"desired resource state has changed","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","kind":"RouteTable","namespace":"infra-dev","name":"dev-public-route-table","is_adopted":false,"generation":76149,"diff":[{"Path":{"Parts":["Spec","Routes"]},"A":[{"destinationCIDRBlock":"0.0.0.0/0","gatewayID":"igw-0c8674d7f71856bde","gatewayRef":{"from":{"name":"dev"}}},{"destinationCIDRBlock":"10.0.0.0/16","vpcPeeringConnectionID":"pcx-0a9e8e3ef90e7981b"}],"B":[{"destinationCIDRBlock":"10.0.0.0/16","vpcPeeringConnectionID":"pcx-0a9e8e3ef90e7981b"},{"destinationCIDRBlock":"0.0.0.0/0","gatewayID":"igw-0c8674d7f71856bde"}]}]}
{"level":"info","ts":"2024-01-31T11:07:26.240Z","logger":"ackrt","msg":"desired resource state has changed","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"SecurityGroup","namespace":"infra-production","name":"limit-amq-to-subnet","is_adopted":false,"generation":3,"diff":[{"Path":{"Parts":["Spec","IngressRules"]},"A":[{"fromPort":8162,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.0.0.0/8","description":"ingress"}],"toPort":8162},{"fromPort":5671,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.3.0.0/16","description":"ingress"}],"toPort":5671},{"fromPort":8883,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.3.0.0/16","description":"ingress"}],"toPort":8883},{"fromPort":61617,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.3.0.0/16","description":"ingress"}],"toPort":61617},{"fromPort":61614,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.3.0.0/16","description":"ingress"}],"toPort":61614},{"fromPort":61619,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.3.0.0/16","description":"ingress"}],"toPort":61619}],"B":[{"fromPort":8162,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.0.0.0/8","description":"ingress"}],"toPort":8162},{"fromPort":61617,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.3.0.0/16","description":"ingress"}],"toPort":61617},{"fromPort":61619,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.3.0.0/16","description":"ingress"}],"toPort":61619},{"fromPort":8883,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.3.0.0/16","description":"ingress"}],"toPort":8883},{"fromPort":61614,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.3.0.0/16","description":"ingress"}],"toPort":61614},{"fromPort":5671,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.3.0.0/16","description":"ingress"}],"toPort":5671}]}]}
{"level":"info","ts":"2024-01-31T11:07:26.540Z","logger":"ackrt","msg":"updated resource","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"SecurityGroup","namespace":"infra-production","name":"limit-amq-to-subnet","is_adopted":false,"generation":3}
{"level":"info","ts":"2024-01-31T11:07:26.863Z","logger":"ackrt","msg":"updated resource","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","kind":"Subnet","namespace":"infra-dev","name":"dev-public-eu-west-2c","is_adopted":false,"generation":2}
{"level":"info","ts":"2024-01-31T11:07:27.139Z","logger":"ackrt","msg":"updated resource","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","kind":"RouteTable","namespace":"infra-dev","name":"dev-public-route-table","is_adopted":false,"generation":76149}
{"level":"info","ts":"2024-01-31T11:07:27.142Z","logger":"ackrt","msg":"desired resource state has changed","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Instance","namespace":"infra-production","name":"ledger-b","is_adopted":false,"generation":9,"diff":[{"Path":{"Parts":["Spec","BlockDeviceMappings"]},"A":[{"deviceName":"/dev/sda1","ebs":{"deleteOnTermination":false,"volumeSize":500}}],"B":[{"deviceName":"/dev/sda1","ebs":{"deleteOnTermination":false}}]},{"Path":{"Parts":["Spec","MetadataOptions","InstanceMetadataTags"]},"A":"enabled","B":"disabled"}]}
{"level":"info","ts":"2024-01-31T11:07:27.142Z","logger":"ackrt","msg":"updated resource","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Instance","namespace":"infra-production","name":"ledger-b","is_adopted":false,"generation":9}
{"level":"info","ts":"2024-01-31T11:07:27.839Z","logger":"ackrt","msg":"desired resource state has changed","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"RouteTable","namespace":"infra-production","name":"production-private-route-table-eu-west-2a","is_adopted":false,"generation":102163,"diff":[{"Path":{"Parts":["Spec","Routes"]},"A":[{"destinationCIDRBlock":"0.0.0.0/0","natGatewayID":"nat-0c2a7ea919efd44c2","natGatewayRef":{"from":{"name":"natgateway-eu-west-2a"}}},{"destinationCIDRBlock":"10.0.0.0/16","vpcPeeringConnectionID":"pcx-0a7197b4f5ced6f01"}],"B":[{"destinationCIDRBlock":"10.0.0.0/16","vpcPeeringConnectionID":"pcx-0a7197b4f5ced6f01"},{"destinationCIDRBlock":"0.0.0.0/0","natGatewayID":"nat-0c2a7ea919efd44c2"}]}]}
{"level":"info","ts":"2024-01-31T11:07:27.939Z","logger":"ackrt","msg":"desired resource state has changed","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Instance","namespace":"infra-production","name":"mg-ui-b","is_adopted":false,"generation":4,"diff":[{"Path":{"Parts":["Spec","BlockDeviceMappings"]},"A":[{"deviceName":"/dev/sda1","ebs":{"deleteOnTermination":false,"volumeSize":500}}],"B":[{"deviceName":"/dev/sda1","ebs":{"deleteOnTermination":false}}]},{"Path":{"Parts":["Spec","MetadataOptions","HTTPTokens"]},"A":"optional","B":"required"}]}
{"level":"info","ts":"2024-01-31T11:07:27.939Z","logger":"ackrt","msg":"desired resource state has changed","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"SecurityGroup","namespace":"infra-production","name":"vpn","is_adopted":false,"generation":2,"diff":[{"Path":{"Parts":["Spec","EgressRules"]},"A":[{"fromPort":0,"ipProtocol":"all","ipRanges":[{"cidrIP":"0.0.0.0/0","description":"egress"}],"toPort":65535}],"B":[{"ipProtocol":"-1","ipRanges":[{"cidrIP":"0.0.0.0/0","description":"egress"}]}]},{"Path":{"Parts":["Spec","IngressRules"]},"A":[{"fromPort":0,"ipProtocol":"all","ipRanges":[{"cidrIP":"10.0.0.0/12","description":"ingress"}],"toPort":65535}],"B":[{"ipProtocol":"-1","ipRanges":[{"cidrIP":"10.0.0.0/12","description":"ingress"}]}]}]}
{"level":"info","ts":"2024-01-31T11:07:27.939Z","logger":"ackrt","msg":"updated resource","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Instance","namespace":"infra-production","name":"mg-ui-b","is_adopted":false,"generation":4}
{"level":"info","ts":"2024-01-31T11:07:28.140Z","logger":"ackrt","msg":"desired resource state has changed","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Subnet","namespace":"infra-production","name":"production-public-eu-west-2c","is_adopted":false,"generation":5,"diff":[{"Path":{"Parts":["Spec","EnableResourceNameDNSAAAARecord"]},"A":null,"B":false},{"Path":{"Parts":["Spec","EnableResourceNameDNSARecord"]},"A":null,"B":false},{"Path":{"Parts":["Spec","HostnameType"]},"A":null,"B":"ip-name"}]}
{"level":"info","ts":"2024-01-31T11:07:28.339Z","logger":"ackrt","msg":"updated resource","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"RouteTable","namespace":"infra-production","name":"production-private-route-table-eu-west-2a","is_adopted":false,"generation":102163}
{"level":"info","ts":"2024-01-31T11:07:28.640Z","logger":"ackrt","msg":"desired resource state has changed","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Instance","namespace":"infra-production","name":"mg-ui-a","is_adopted":false,"generation":4,"diff":[{"Path":{"Parts":["Spec","BlockDeviceMappings"]},"A":[{"deviceName":"/dev/sda1","ebs":{"deleteOnTermination":false,"volumeSize":500}}],"B":[{"deviceName":"/dev/sda1","ebs":{"deleteOnTermination":false}}]},{"Path":{"Parts":["Spec","MetadataOptions","HTTPTokens"]},"A":"optional","B":"required"}]}
{"level":"info","ts":"2024-01-31T11:07:28.640Z","logger":"ackrt","msg":"updated resource","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Instance","namespace":"infra-production","name":"mg-ui-a","is_adopted":false,"generation":4}
{"level":"info","ts":"2024-01-31T11:07:28.739Z","logger":"ackrt","msg":"desired resource state has changed","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"RouteTable","namespace":"infra-production","name":"production-private-route-table-eu-west-2b","is_adopted":false,"generation":102145,"diff":[{"Path":{"Parts":["Spec","Routes"]},"A":[{"destinationCIDRBlock":"0.0.0.0/0","natGatewayID":"nat-05a6a43f06ac1d749","natGatewayRef":{"from":{"name":"natgateway-eu-west-2b"}}},{"destinationCIDRBlock":"10.0.0.0/16","vpcPeeringConnectionID":"pcx-0a7197b4f5ced6f01"}],"B":[{"destinationCIDRBlock":"10.0.0.0/16","vpcPeeringConnectionID":"pcx-0a7197b4f5ced6f01"},{"destinationCIDRBlock":"0.0.0.0/0","natGatewayID":"nat-05a6a43f06ac1d749"}]}]}
{"level":"info","ts":"2024-01-31T11:07:28.763Z","logger":"ackrt","msg":"updated resource","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Subnet","namespace":"infra-production","name":"production-public-eu-west-2c","is_adopted":false,"generation":5}
{"level":"info","ts":"2024-01-31T11:07:28.939Z","logger":"ackrt","msg":"updated resource","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"SecurityGroup","namespace":"infra-production","name":"vpn","is_adopted":false,"generation":2}
{"level":"info","ts":"2024-01-31T11:07:28.953Z","logger":"ackrt","msg":"updated resource","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"RouteTable","namespace":"infra-production","name":"production-private-route-table-eu-west-2b","is_adopted":false,"generation":102145}
{"level":"info","ts":"2024-01-31T11:07:29.240Z","logger":"ackrt","msg":"desired resource state has changed","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","kind":"Instance","namespace":"infra-dev","name":"mg-main-a","is_adopted":false,"generation":5,"diff":[{"Path":{"Parts":["Spec","BlockDeviceMappings"]},"A":[{"deviceName":"/dev/sda1","ebs":{"deleteOnTermination":false,"volumeSize":500}}],"B":[{"deviceName":"/dev/sda1","ebs":{"deleteOnTermination":false}},{"deviceName":"/dev/sdf","ebs":{"deleteOnTermination":false}}]},{"Path":{"Parts":["Spec","SecurityGroups"]},"A":["server"],"B":["server","dev-sg"]}]}
{"level":"info","ts":"2024-01-31T11:07:29.241Z","logger":"ackrt","msg":"updated resource","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","kind":"Instance","namespace":"infra-dev","name":"mg-main-a","is_adopted":false,"generation":5}
{"level":"info","ts":"2024-01-31T11:07:29.339Z","logger":"ackrt","msg":"desired resource state has changed","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","kind":"Subnet","namespace":"infra-dev","name":"dev-private-eu-west-2a","is_adopted":false,"generation":2,"diff":[{"Path":{"Parts":["Spec","EnableResourceNameDNSAAAARecord"]},"A":null,"B":false},{"Path":{"Parts":["Spec","EnableResourceNameDNSARecord"]},"A":null,"B":false},{"Path":{"Parts":["Spec","HostnameType"]},"A":null,"B":"ip-name"}]}
{"level":"info","ts":"2024-01-31T11:07:29.439Z","logger":"ackrt","msg":"desired resource state has changed","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"RouteTable","namespace":"infra-production","name":"production-private-route-table-eu-west-2c","is_adopted":false,"generation":102159,"diff":[{"Path":{"Parts":["Spec","Routes"]},"A":[{"destinationCIDRBlock":"0.0.0.0/0","natGatewayID":"nat-07a301987eaa97785","natGatewayRef":{"from":{"name":"natgateway-eu-west-2c"}}},{"destinationCIDRBlock":"10.0.0.0/16","vpcPeeringConnectionID":"pcx-0a7197b4f5ced6f01"}],"B":[{"destinationCIDRBlock":"10.0.0.0/16","vpcPeeringConnectionID":"pcx-0a7197b4f5ced6f01"},{"destinationCIDRBlock":"0.0.0.0/0","natGatewayID":"nat-07a301987eaa97785"}]}]}
{"level":"info","ts":"2024-01-31T11:07:29.621Z","logger":"ackrt","msg":"updated resource","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"RouteTable","namespace":"infra-production","name":"production-private-route-table-eu-west-2c","is_adopted":false,"generation":102159}
{"level":"info","ts":"2024-01-31T11:07:29.642Z","logger":"ackrt","msg":"desired resource state has changed","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Instance","namespace":"infra-production","name":"mg-fix-a","is_adopted":false,"generation":8,"diff":[{"Path":{"Parts":["Spec","BlockDeviceMappings"]},"A":[{"deviceName":"/dev/sda1","ebs":{"deleteOnTermination":false,"volumeSize":500}}],"B":[{"deviceName":"/dev/sda1","ebs":{"deleteOnTermination":false}}]},{"Path":{"Parts":["Spec","MetadataOptions","HTTPTokens"]},"A":"optional","B":"required"},{"Path":{"Parts":["Spec","MetadataOptions","InstanceMetadataTags"]},"A":null,"B":"disabled"}]}
{"level":"info","ts":"2024-01-31T11:07:29.643Z","logger":"ackrt","msg":"updated resource","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Instance","namespace":"infra-production","name":"mg-fix-a","is_adopted":false,"generation":8}
{"level":"info","ts":"2024-01-31T11:07:29.732Z","logger":"ackrt","msg":"desired resource state has changed","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"RouteTable","namespace":"infra-production","name":"production-public-route-table","is_adopted":false,"generation":102179,"diff":[{"Path":{"Parts":["Spec","Routes"]},"A":[{"destinationCIDRBlock":"0.0.0.0/0","gatewayID":"igw-06ccbbf63dfe23366","gatewayRef":{"from":{"name":"production"}}},{"destinationCIDRBlock":"10.0.0.0/16","vpcPeeringConnectionID":"pcx-0a7197b4f5ced6f01"}],"B":[{"destinationCIDRBlock":"10.0.0.0/16","vpcPeeringConnectionID":"pcx-0a7197b4f5ced6f01"},{"destinationCIDRBlock":"0.0.0.0/0","gatewayID":"igw-06ccbbf63dfe23366"}]}]}
{"level":"info","ts":"2024-01-31T11:07:29.754Z","logger":"ackrt","msg":"desired resource state has changed","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","kind":"SecurityGroup","namespace":"infra-dev","name":"dev-security-group","is_adopted":false,"generation":2,"diff":[{"Path":{"Parts":["Spec","EgressRules"]},"A":[{"fromPort":0,"ipProtocol":"all","ipRanges":[{"cidrIP":"0.0.0.0/0","description":"egress"}],"toPort":65535}],"B":[{"ipProtocol":"-1","ipRanges":[{"cidrIP":"0.0.0.0/0","description":"egress"}]}]},{"Path":{"Parts":["Spec","IngressRules"]},"A":[{"fromPort":22,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.0.0.0/8","description":"ingress"}],"toPort":22},{"fromPort":443,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.0.0.0/8","description":"ingress"}],"toPort":443},{"fromPort":10080,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.0.0.0/8","description":"ingress"}],"toPort":10080},{"fromPort":10443,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.0.0.0/8","description":"ingress"}],"toPort":10443},{"fromPort":10000,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.0.0.0/8","description":"ingress"}],"toPort":30000},{"fromPort":8888,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.0.0.0/8","description":"ingress"}],"toPort":8888},{"fromPort":0,"ipProtocol":"all","ipRanges":[{"cidrIP":"10.10.0.0/16","description":"ingress"}],"toPort":65535}],"B":[{"fromPort":8888,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.0.0.0/8","description":"ingress"}],"toPort":8888},{"ipProtocol":"-1","ipRanges":[{"cidrIP":"10.10.0.0/16","description":"ingress"}]},{"fromPort":10000,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.0.0.0/8","description":"ingress"}],"toPort":30000},{"fromPort":22,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.0.0.0/8","description":"ingress"}],"toPort":22},{"fromPort":10443,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.0.0.0/8","description":"ingress"}],"toPort":10443},{"fromPort":443,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.0.0.0/8","description":"ingress"}],"toPort":443},{"fromPort":10080,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.0.0.0/8","description":"ingress"}],"toPort":10080}]}]}
{"level":"info","ts":"2024-01-31T11:07:29.939Z","logger":"ackrt","msg":"updated resource","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","kind":"Subnet","namespace":"infra-dev","name":"dev-private-eu-west-2a","is_adopted":false,"generation":2}
{"level":"info","ts":"2024-01-31T11:07:29.940Z","logger":"ackrt","msg":"desired resource state has changed","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Instance","namespace":"infra-production","name":"mg-main-a","is_adopted":false,"generation":9,"diff":[{"Path":{"Parts":["Spec","BlockDeviceMappings"]},"A":[{"deviceName":"/dev/sda1","ebs":{"deleteOnTermination":false,"volumeSize":500}}],"B":[{"deviceName":"/dev/sda1","ebs":{"deleteOnTermination":false}},{"deviceName":"/dev/sdf","ebs":{"deleteOnTermination":false}}]},{"Path":{"Parts":["Spec","MetadataOptions","InstanceMetadataTags"]},"A":null,"B":"disabled"}]}
{"level":"info","ts":"2024-01-31T11:07:29.940Z","logger":"ackrt","msg":"updated resource","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Instance","namespace":"infra-production","name":"mg-main-a","is_adopted":false,"generation":9}
{"level":"info","ts":"2024-01-31T11:07:30.066Z","logger":"ackrt","msg":"updated resource","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"RouteTable","namespace":"infra-production","name":"production-public-route-table","is_adopted":false,"generation":102179}
{"level":"info","ts":"2024-01-31T11:07:30.239Z","logger":"ackrt","msg":"desired resource state has changed","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","kind":"RouteTable","namespace":"infra-dev","name":"dev-private-route-table-eu-west-2a","is_adopted":false,"generation":72308,"diff":[{"Path":{"Parts":["Spec","Routes"]},"A":[{"destinationCIDRBlock":"0.0.0.0/0","natGatewayID":"nat-002c0709a0a96e0d0","natGatewayRef":{"from":{"name":"natgateway-eu-west-2a"}}},{"destinationCIDRBlock":"10.0.0.0/16","vpcPeeringConnectionID":"pcx-0a9e8e3ef90e7981b"}],"B":[{"destinationCIDRBlock":"10.0.0.0/16","vpcPeeringConnectionID":"pcx-0a9e8e3ef90e7981b"},{"destinationCIDRBlock":"0.0.0.0/0","natGatewayID":"nat-002c0709a0a96e0d0"}]}]}
{"level":"info","ts":"2024-01-31T11:07:30.254Z","logger":"ackrt","msg":"desired resource state has changed","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Instance","namespace":"infra-production","name":"mg-main-b","is_adopted":false,"generation":6,"diff":[{"Path":{"Parts":["Spec","BlockDeviceMappings"]},"A":[{"deviceName":"/dev/sda1","ebs":{"deleteOnTermination":false,"volumeSize":500}}],"B":[{"deviceName":"/dev/sda1","ebs":{"deleteOnTermination":false}}]},{"Path":{"Parts":["Spec","MetadataOptions","HTTPTokens"]},"A":"optional","B":"required"}]}
{"level":"info","ts":"2024-01-31T11:07:30.254Z","logger":"ackrt","msg":"updated resource","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Instance","namespace":"infra-production","name":"mg-main-b","is_adopted":false,"generation":6}
{"level":"info","ts":"2024-01-31T11:07:30.339Z","logger":"ackrt","msg":"desired resource state has changed","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Subnet","namespace":"infra-production","name":"production-private-eu-west-2b","is_adopted":false,"generation":5,"diff":[{"Path":{"Parts":["Spec","EnableResourceNameDNSAAAARecord"]},"A":null,"B":false},{"Path":{"Parts":["Spec","EnableResourceNameDNSARecord"]},"A":null,"B":false},{"Path":{"Parts":["Spec","HostnameType"]},"A":null,"B":"ip-name"}]}
{"level":"info","ts":"2024-01-31T11:07:30.439Z","logger":"ackrt","msg":"updated resource","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","kind":"RouteTable","namespace":"infra-dev","name":"dev-private-route-table-eu-west-2a","is_adopted":false,"generation":72308}
{"level":"info","ts":"2024-01-31T11:07:30.484Z","logger":"ackrt","msg":"desired resource state has changed","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","kind":"Instance","namespace":"infra-dev","name":"mg-fix-a","is_adopted":false,"generation":5,"diff":[{"Path":{"Parts":["Spec","BlockDeviceMappings"]},"A":[{"deviceName":"/dev/sda1","ebs":{"deleteOnTermination":false,"volumeSize":500}}],"B":[{"deviceName":"/dev/sda1","ebs":{"deleteOnTermination":false}}]},{"Path":{"Parts":["Spec","SecurityGroups"]},"A":["server"],"B":["server","dev-sg"]}]}
{"level":"info","ts":"2024-01-31T11:07:30.484Z","logger":"ackrt","msg":"updated resource","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","kind":"Instance","namespace":"infra-dev","name":"mg-fix-a","is_adopted":false,"generation":5}
{"level":"info","ts":"2024-01-31T11:07:30.534Z","logger":"ackrt","msg":"desired resource state has changed","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","kind":"RouteTable","namespace":"infra-dev","name":"dev-private-route-table-eu-west-2b","is_adopted":false,"generation":72286,"diff":[{"Path":{"Parts":["Spec","Routes"]},"A":[{"destinationCIDRBlock":"0.0.0.0/0","natGatewayID":"nat-05ce67c9ffaa32e27","natGatewayRef":{"from":{"name":"natgateway-eu-west-2b"}}},{"destinationCIDRBlock":"10.0.0.0/16","vpcPeeringConnectionID":"pcx-0a9e8e3ef90e7981b"}],"B":[{"destinationCIDRBlock":"10.0.0.0/16","vpcPeeringConnectionID":"pcx-0a9e8e3ef90e7981b"},{"destinationCIDRBlock":"0.0.0.0/0","natGatewayID":"nat-05ce67c9ffaa32e27"}]}]}
{"level":"info","ts":"2024-01-31T11:07:30.739Z","logger":"ackrt","msg":"updated resource","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Subnet","namespace":"infra-production","name":"production-private-eu-west-2b","is_adopted":false,"generation":5}
{"level":"info","ts":"2024-01-31T11:07:30.740Z","logger":"ackrt","msg":"updated resource","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","kind":"SecurityGroup","namespace":"infra-dev","name":"dev-security-group","is_adopted":false,"generation":2}
{"level":"info","ts":"2024-01-31T11:07:30.839Z","logger":"ackrt","msg":"updated resource","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","kind":"RouteTable","namespace":"infra-dev","name":"dev-private-route-table-eu-west-2b","is_adopted":false,"generation":72286}
{"level":"info","ts":"2024-01-31T11:07:30.844Z","logger":"ackrt","msg":"desired resource state has changed","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Instance","namespace":"infra-production","name":"ledger-a","is_adopted":false,"generation":8,"diff":[{"Path":{"Parts":["Spec","BlockDeviceMappings"]},"A":[{"deviceName":"/dev/sda1","ebs":{"deleteOnTermination":false,"volumeSize":500}}],"B":[{"deviceName":"/dev/sda1","ebs":{"deleteOnTermination":false}}]},{"Path":{"Parts":["Spec","MetadataOptions","HTTPTokens"]},"A":"optional","B":"required"},{"Path":{"Parts":["Spec","MetadataOptions","InstanceMetadataTags"]},"A":"enabled","B":"disabled"}]}
{"level":"info","ts":"2024-01-31T11:07:30.844Z","logger":"ackrt","msg":"updated resource","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Instance","namespace":"infra-production","name":"ledger-a","is_adopted":false,"generation":8}
{"level":"info","ts":"2024-01-31T11:07:31.101Z","logger":"ackrt","msg":"desired resource state has changed","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"SecurityGroup","namespace":"infra-production","name":"limit-kafka-to-subnet","is_adopted":false,"generation":3,"diff":[{"Path":{"Parts":["Spec","IngressRules"]},"A":[{"fromPort":9092,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.3.0.0/16","description":"ingress"}],"toPort":9092},{"fromPort":9094,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.3.0.0/16","description":"ingress"}],"toPort":9094}],"B":[{"fromPort":9094,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.3.0.0/16","description":"ingress"}],"toPort":9094},{"fromPort":9092,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.3.0.0/16","description":"ingress"}],"toPort":9092}]}]}
{"level":"info","ts":"2024-01-31T11:07:31.139Z","logger":"ackrt","msg":"desired resource state has changed","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","kind":"RouteTable","namespace":"infra-dev","name":"dev-private-route-table-eu-west-2c","is_adopted":false,"generation":72249,"diff":[{"Path":{"Parts":["Spec","Routes"]},"A":[{"destinationCIDRBlock":"0.0.0.0/0","natGatewayID":"nat-079f5b2a13a998706","natGatewayRef":{"from":{"name":"natgateway-eu-west-2c"}}}],"B":[{"destinationCIDRBlock":"0.0.0.0/0","natGatewayID":"nat-079f5b2a13a998706"}]}]}
{"level":"info","ts":"2024-01-31T11:07:31.139Z","logger":"ackrt","msg":"updated resource","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"SecurityGroup","namespace":"infra-production","name":"limit-kafka-to-subnet","is_adopted":false,"generation":3}
{"level":"info","ts":"2024-01-31T11:07:31.139Z","logger":"ackrt","msg":"desired resource state has changed","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Subnet","namespace":"infra-production","name":"production-public-eu-west-2b","is_adopted":false,"generation":5,"diff":[{"Path":{"Parts":["Spec","EnableResourceNameDNSAAAARecord"]},"A":null,"B":false},{"Path":{"Parts":["Spec","EnableResourceNameDNSARecord"]},"A":null,"B":false},{"Path":{"Parts":["Spec","HostnameType"]},"A":null,"B":"ip-name"}]}
{"level":"info","ts":"2024-01-31T11:07:31.239Z","logger":"ackrt","msg":"desired resource state has changed","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Instance","namespace":"infra-production","name":"mg-fix-b","is_adopted":false,"generation":5,"diff":[{"Path":{"Parts":["Spec","BlockDeviceMappings"]},"A":[{"deviceName":"/dev/sda1","ebs":{"deleteOnTermination":false,"volumeSize":500}}],"B":[{"deviceName":"/dev/sda1","ebs":{"deleteOnTermination":false}}]},{"Path":{"Parts":["Spec","MetadataOptions","HTTPTokens"]},"A":"optional","B":"required"}]}
{"level":"info","ts":"2024-01-31T11:07:31.239Z","logger":"ackrt","msg":"updated resource","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Instance","namespace":"infra-production","name":"mg-fix-b","is_adopted":false,"generation":5}
{"level":"info","ts":"2024-01-31T11:07:31.295Z","logger":"ackrt","msg":"updated resource","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","kind":"RouteTable","namespace":"infra-dev","name":"dev-private-route-table-eu-west-2c","is_adopted":false,"generation":72249}
{"level":"info","ts":"2024-01-31T11:07:31.539Z","logger":"ackrt","msg":"updated resource","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Subnet","namespace":"infra-production","name":"production-public-eu-west-2b","is_adopted":false,"generation":5}
{"level":"info","ts":"2024-01-31T11:07:31.539Z","logger":"ackrt","msg":"desired resource state has changed","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","kind":"SecurityGroup","namespace":"infra-dev","name":"server","is_adopted":false,"generation":1,"diff":[{"Path":{"Parts":["Spec","EgressRules"]},"A":[{"fromPort":0,"ipProtocol":"all","ipRanges":[{"cidrIP":"0.0.0.0/0","description":"egress"}],"toPort":65535}],"B":[{"ipProtocol":"-1","ipRanges":[{"cidrIP":"0.0.0.0/0","description":"egress"}]}]},{"Path":{"Parts":["Spec","IngressRules"]},"A":[{"fromPort":0,"ipProtocol":"all","ipRanges":[{"cidrIP":"10.10.0.0/16","description":"ingress"}],"toPort":65535}],"B":[{"ipProtocol":"-1","ipRanges":[{"cidrIP":"10.10.0.0/16","description":"ingress"}]}]}]}
{"level":"info","ts":"2024-01-31T11:07:31.647Z","logger":"ackrt","msg":"desired resource state has changed","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Instance","namespace":"infra-production","name":"mg-gov","is_adopted":false,"generation":4,"diff":[{"Path":{"Parts":["Spec","BlockDeviceMappings"]},"A":[{"deviceName":"/dev/sda1","ebs":{"deleteOnTermination":false,"volumeSize":500}}],"B":[{"deviceName":"/dev/sda1","ebs":{"deleteOnTermination":false}}]},{"Path":{"Parts":["Spec","CapacityReservationSpecification"]},"A":null,"B":{"capacityReservationPreference":"open"}},{"Path":{"Parts":["Spec","CPUOptions"]},"A":null,"B":{"coreCount":1,"threadsPerCore":2}},{"Path":{"Parts":["Spec","EBSOptimized"]},"A":null,"B":false},{"Path":{"Parts":["Spec","EnclaveOptions"]},"A":null,"B":{"enabled":false}},{"Path":{"Parts":["Spec","MaintenanceOptions"]},"A":null,"B":{"autoRecovery":"default"}},{"Path":{"Parts":["Spec","MetadataOptions"]},"A":null,"B":{"httpEndpoint":"enabled","httpProtocolIPv6":"disabled","httpPutResponseHopLimit":1,"httpTokens":"required","instanceMetadataTags":"disabled"}},{"Path":{"Parts":["Spec","Monitoring"]},"A":null,"B":{}},{"Path":{"Parts":["Spec","NetworkInterfaces"]},"A":null,"B":[{"description":"","interfaceType":"interface","networkInterfaceID":"eni-0debc311b02143fb1","privateIPAddress":"10.3.175.43","privateIPAddresses":[{"primary":true,"privateIPAddress":"10.3.175.43"}],"subnetID":"subnet-0eeac56411254cbc6"}]},{"Path":{"Parts":["Spec","Placement"]},"A":null,"B":{"availabilityZone":"eu-west-2c","groupName":"","tenancy":"default"}},{"Path":{"Parts":["Spec","PrivateDNSNameOptions"]},"A":null,"B":{"enableResourceNameDNSAAAARecord":false,"enableResourceNameDNSARecord":false,"hostnameType":"ip-name"}},{"Path":{"Parts":["Spec","PrivateIPAddress"]},"A":null,"B":"10.3.175.43"},{"Path":{"Parts":["Spec","SecurityGroups"]},"A":null,"B":["tutorial-sg"]}]}
{"level":"info","ts":"2024-01-31T11:07:31.647Z","logger":"ackrt","msg":"updated resource","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Instance","namespace":"infra-production","name":"mg-gov","is_adopted":false,"generation":4}
{"level":"info","ts":"2024-01-31T11:07:32.008Z","logger":"ackrt","msg":"desired resource state has changed","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","kind":"Subnet","namespace":"infra-dev","name":"dev-private-eu-west-2c","is_adopted":false,"generation":2,"diff":[{"Path":{"Parts":["Spec","EnableResourceNameDNSAAAARecord"]},"A":null,"B":false},{"Path":{"Parts":["Spec","EnableResourceNameDNSARecord"]},"A":null,"B":false},{"Path":{"Parts":["Spec","HostnameType"]},"A":null,"B":"ip-name"}]}
{"level":"info","ts":"2024-01-31T11:07:32.299Z","logger":"ackrt","msg":"updated resource","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","kind":"Subnet","namespace":"infra-dev","name":"dev-private-eu-west-2c","is_adopted":false,"generation":2}
{"level":"info","ts":"2024-01-31T11:07:32.399Z","logger":"ackrt","msg":"updated resource","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","kind":"SecurityGroup","namespace":"infra-dev","name":"server","is_adopted":false,"generation":1}
{"level":"info","ts":"2024-01-31T11:07:32.632Z","logger":"ackrt","msg":"desired resource state has changed","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","kind":"Subnet","namespace":"infra-dev","name":"dev-private-eu-west-2b","is_adopted":false,"generation":2,"diff":[{"Path":{"Parts":["Spec","EnableResourceNameDNSAAAARecord"]},"A":null,"B":false},{"Path":{"Parts":["Spec","EnableResourceNameDNSARecord"]},"A":null,"B":false},{"Path":{"Parts":["Spec","HostnameType"]},"A":null,"B":"ip-name"}]}
{"level":"info","ts":"2024-01-31T11:07:32.791Z","logger":"ackrt","msg":"desired resource state has changed","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"SecurityGroup","namespace":"infra-production","name":"server","is_adopted":false,"generation":1,"diff":[{"Path":{"Parts":["Spec","EgressRules"]},"A":[{"fromPort":0,"ipProtocol":"all","ipRanges":[{"cidrIP":"0.0.0.0/0","description":"egress"}],"toPort":65535}],"B":[{"ipProtocol":"-1","ipRanges":[{"cidrIP":"0.0.0.0/0","description":"egress"}]}]},{"Path":{"Parts":["Spec","IngressRules"]},"A":[{"fromPort":0,"ipProtocol":"all","ipRanges":[{"cidrIP":"10.3.0.0/16","description":"ingress"}],"toPort":65535}],"B":[{"ipProtocol":"-1","ipRanges":[{"cidrIP":"10.3.0.0/16","description":"ingress"}]}]}]}
{"level":"info","ts":"2024-01-31T11:07:32.922Z","logger":"ackrt","msg":"updated resource","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","kind":"Subnet","namespace":"infra-dev","name":"dev-private-eu-west-2b","is_adopted":false,"generation":2}
{"level":"info","ts":"2024-01-31T11:07:33.200Z","logger":"ackrt","msg":"desired resource state has changed","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","kind":"Subnet","namespace":"infra-dev","name":"dev-public-eu-west-2a","is_adopted":false,"generation":2,"diff":[{"Path":{"Parts":["Spec","EnableResourceNameDNSAAAARecord"]},"A":null,"B":false},{"Path":{"Parts":["Spec","EnableResourceNameDNSARecord"]},"A":null,"B":false},{"Path":{"Parts":["Spec","HostnameType"]},"A":null,"B":"ip-name"}]}
{"level":"info","ts":"2024-01-31T11:07:33.370Z","logger":"ackrt","msg":"updated resource","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"SecurityGroup","namespace":"infra-production","name":"server","is_adopted":false,"generation":1}
{"level":"info","ts":"2024-01-31T11:07:33.489Z","logger":"ackrt","msg":"updated resource","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","kind":"Subnet","namespace":"infra-dev","name":"dev-public-eu-west-2a","is_adopted":false,"generation":2}
{"level":"info","ts":"2024-01-31T11:07:33.535Z","logger":"ackrt","msg":"desired resource state has changed","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"SecurityGroup","namespace":"infra-production","name":"tutorial-security-group","is_adopted":false,"generation":8,"diff":[{"Path":{"Parts":["Spec","EgressRules"]},"A":[{"fromPort":0,"ipProtocol":"all","ipRanges":[{"cidrIP":"0.0.0.0/0","description":"egress"}],"toPort":65535}],"B":[{"ipProtocol":"-1","ipRanges":[{"cidrIP":"0.0.0.0/0","description":"egress"}]}]},{"Path":{"Parts":["Spec","IngressRules"]},"A":[{"fromPort":22,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.0.0.0/8","description":"ingress"}],"toPort":22},{"fromPort":10080,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.0.0.0/8","description":"ingress"}],"toPort":10080},{"fromPort":10443,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.0.0.0/8","description":"ingress"}],"toPort":10443},{"fromPort":10000,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.0.0.0/8","description":"ingress"}],"toPort":30000},{"fromPort":8888,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.0.0.0/8","description":"ingress"}],"toPort":8888},{"fromPort":0,"ipProtocol":"all","ipRanges":[{"cidrIP":"10.3.0.0/16","description":"ingress"}],"toPort":65535}],"B":[{"fromPort":8888,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.0.0.0/8","description":"ingress"}],"toPort":8888},{"ipProtocol":"-1","ipRanges":[{"cidrIP":"10.3.0.0/16","description":"ingress"}]},{"fromPort":10000,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.0.0.0/8","description":"ingress"}],"toPort":30000},{"fromPort":22,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.0.0.0/8","description":"ingress"}],"toPort":22},{"fromPort":10443,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.0.0.0/8","description":"ingress"}],"toPort":10443},{"fromPort":10080,"ipProtocol":"tcp","ipRanges":[{"cidrIP":"10.0.0.0/8","description":"ingress"}],"toPort":10080}]}]}
{"level":"info","ts":"2024-01-31T11:07:33.674Z","logger":"ackrt","msg":"desired resource state has changed","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Subnet","namespace":"infra-production","name":"production-private-eu-west-2a","is_adopted":false,"generation":5,"diff":[{"Path":{"Parts":["Spec","EnableResourceNameDNSAAAARecord"]},"A":null,"B":false},{"Path":{"Parts":["Spec","EnableResourceNameDNSARecord"]},"A":null,"B":false},{"Path":{"Parts":["Spec","HostnameType"]},"A":null,"B":"ip-name"}]}
{"level":"info","ts":"2024-01-31T11:07:33.957Z","logger":"ackrt","msg":"updated resource","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Subnet","namespace":"infra-production","name":"production-private-eu-west-2a","is_adopted":false,"generation":5}
{"level":"info","ts":"2024-01-31T11:07:34.085Z","logger":"ackrt","msg":"updated resource","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"SecurityGroup","namespace":"infra-production","name":"tutorial-security-group","is_adopted":false,"generation":8}
{"level":"info","ts":"2024-01-31T11:07:34.134Z","logger":"ackrt","msg":"desired resource state has changed","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Subnet","namespace":"infra-production","name":"production-private-eu-west-2c","is_adopted":false,"generation":5,"diff":[{"Path":{"Parts":["Spec","EnableResourceNameDNSAAAARecord"]},"A":null,"B":false},{"Path":{"Parts":["Spec","EnableResourceNameDNSARecord"]},"A":null,"B":false},{"Path":{"Parts":["Spec","HostnameType"]},"A":null,"B":"ip-name"}]}
{"level":"info","ts":"2024-01-31T11:07:34.410Z","logger":"ackrt","msg":"updated resource","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Subnet","namespace":"infra-production","name":"production-private-eu-west-2c","is_adopted":false,"generation":5}
{"level":"error","ts":"2024-01-31T11:07:34.443Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCPeeringConnection.ec2.services.k8s.aws","error":"no matches for kind \"VPCPeeringConnection\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"info","ts":"2024-01-31T11:07:34.610Z","logger":"ackrt","msg":"desired resource state has changed","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Subnet","namespace":"infra-production","name":"production-public-eu-west-2a","is_adopted":false,"generation":5,"diff":[{"Path":{"Parts":["Spec","EnableResourceNameDNSAAAARecord"]},"A":null,"B":false},{"Path":{"Parts":["Spec","EnableResourceNameDNSARecord"]},"A":null,"B":false},{"Path":{"Parts":["Spec","HostnameType"]},"A":null,"B":"ip-name"}]}
{"level":"error","ts":"2024-01-31T11:07:34.654Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCEndpointServiceConfiguration.ec2.services.k8s.aws","error":"no matches for kind \"VPCEndpointServiceConfiguration\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:07:34.743Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCEndpointServiceConfiguration.ec2.services.k8s.aws","error":"no matches for kind \"VPCEndpointServiceConfiguration\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:07:34.746Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCPeeringConnection.ec2.services.k8s.aws","error":"no matches for kind \"VPCPeeringConnection\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"info","ts":"2024-01-31T11:07:34.912Z","logger":"ackrt","msg":"updated resource","account":"966321756598","role":"arn:aws:iam::966321756598:role/ack-ec2-controller-tooling","region":"eu-west-2","kind":"Subnet","namespace":"infra-production","name":"production-public-eu-west-2a","is_adopted":false,"generation":5}
{"level":"info","ts":"2024-01-31T11:07:35.220Z","logger":"ackrt","msg":"desired resource state has changed","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","kind":"Subnet","namespace":"infra-dev","name":"dev-public-eu-west-2b","is_adopted":false,"generation":2,"diff":[{"Path":{"Parts":["Spec","EnableResourceNameDNSAAAARecord"]},"A":null,"B":false},{"Path":{"Parts":["Spec","EnableResourceNameDNSARecord"]},"A":null,"B":false},{"Path":{"Parts":["Spec","HostnameType"]},"A":null,"B":"ip-name"}]}
{"level":"info","ts":"2024-01-31T11:07:35.509Z","logger":"ackrt","msg":"updated resource","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","kind":"Subnet","namespace":"infra-dev","name":"dev-public-eu-west-2b","is_adopted":false,"generation":2}
{"level":"error","ts":"2024-01-31T11:07:44.446Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCPeeringConnection.ec2.services.k8s.aws","error":"no matches for kind \"VPCPeeringConnection\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:07:44.653Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCEndpointServiceConfiguration.ec2.services.k8s.aws","error":"no matches for kind \"VPCEndpointServiceConfiguration\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:07:44.742Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCEndpointServiceConfiguration.ec2.services.k8s.aws","error":"no matches for kind \"VPCEndpointServiceConfiguration\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:07:44.745Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCPeeringConnection.ec2.services.k8s.aws","error":"no matches for kind \"VPCPeeringConnection\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:07:54.453Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCPeeringConnection.ec2.services.k8s.aws","error":"no matches for kind \"VPCPeeringConnection\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:07:54.655Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCEndpointServiceConfiguration.ec2.services.k8s.aws","error":"no matches for kind \"VPCEndpointServiceConfiguration\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:07:54.742Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCEndpointServiceConfiguration.ec2.services.k8s.aws","error":"no matches for kind \"VPCEndpointServiceConfiguration\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:07:54.746Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCPeeringConnection.ec2.services.k8s.aws","error":"no matches for kind \"VPCPeeringConnection\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:08:04.442Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCPeeringConnection.ec2.services.k8s.aws","error":"no matches for kind \"VPCPeeringConnection\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:08:04.657Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCEndpointServiceConfiguration.ec2.services.k8s.aws","error":"no matches for kind \"VPCEndpointServiceConfiguration\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:08:04.742Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCEndpointServiceConfiguration.ec2.services.k8s.aws","error":"no matches for kind \"VPCEndpointServiceConfiguration\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:08:04.746Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCPeeringConnection.ec2.services.k8s.aws","error":"no matches for kind \"VPCPeeringConnection\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:08:14.444Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCPeeringConnection.ec2.services.k8s.aws","error":"no matches for kind \"VPCPeeringConnection\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:08:14.653Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCEndpointServiceConfiguration.ec2.services.k8s.aws","error":"no matches for kind \"VPCEndpointServiceConfiguration\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:08:14.745Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCEndpointServiceConfiguration.ec2.services.k8s.aws","error":"no matches for kind \"VPCEndpointServiceConfiguration\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:08:14.751Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCPeeringConnection.ec2.services.k8s.aws","error":"no matches for kind \"VPCPeeringConnection\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:08:24.446Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCPeeringConnection.ec2.services.k8s.aws","error":"no matches for kind \"VPCPeeringConnection\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:08:24.654Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCEndpointServiceConfiguration.ec2.services.k8s.aws","error":"no matches for kind \"VPCEndpointServiceConfiguration\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:08:24.742Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCEndpointServiceConfiguration.ec2.services.k8s.aws","error":"no matches for kind \"VPCEndpointServiceConfiguration\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:08:24.749Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCPeeringConnection.ec2.services.k8s.aws","error":"no matches for kind \"VPCPeeringConnection\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:08:34.443Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCPeeringConnection.ec2.services.k8s.aws","error":"no matches for kind \"VPCPeeringConnection\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:08:34.653Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCEndpointServiceConfiguration.ec2.services.k8s.aws","error":"no matches for kind \"VPCEndpointServiceConfiguration\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:08:34.743Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCEndpointServiceConfiguration.ec2.services.k8s.aws","error":"no matches for kind \"VPCEndpointServiceConfiguration\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:08:34.747Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCPeeringConnection.ec2.services.k8s.aws","error":"no matches for kind \"VPCPeeringConnection\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:08:44.448Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCPeeringConnection.ec2.services.k8s.aws","error":"no matches for kind \"VPCPeeringConnection\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:08:44.653Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCEndpointServiceConfiguration.ec2.services.k8s.aws","error":"no matches for kind \"VPCEndpointServiceConfiguration\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:08:44.743Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCEndpointServiceConfiguration.ec2.services.k8s.aws","error":"no matches for kind \"VPCEndpointServiceConfiguration\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:08:44.747Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCPeeringConnection.ec2.services.k8s.aws","error":"no matches for kind \"VPCPeeringConnection\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:08:54.445Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCPeeringConnection.ec2.services.k8s.aws","error":"no matches for kind \"VPCPeeringConnection\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:08:54.653Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCEndpointServiceConfiguration.ec2.services.k8s.aws","error":"no matches for kind \"VPCEndpointServiceConfiguration\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:08:54.743Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCEndpointServiceConfiguration.ec2.services.k8s.aws","error":"no matches for kind \"VPCEndpointServiceConfiguration\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:08:54.745Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCPeeringConnection.ec2.services.k8s.aws","error":"no matches for kind \"VPCPeeringConnection\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:09:04.443Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCPeeringConnection.ec2.services.k8s.aws","error":"no matches for kind \"VPCPeeringConnection\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:09:04.653Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCEndpointServiceConfiguration.ec2.services.k8s.aws","error":"no matches for kind \"VPCEndpointServiceConfiguration\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:09:04.744Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCEndpointServiceConfiguration.ec2.services.k8s.aws","error":"no matches for kind \"VPCEndpointServiceConfiguration\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:09:04.746Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCPeeringConnection.ec2.services.k8s.aws","error":"no matches for kind \"VPCPeeringConnection\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:09:14.443Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCPeeringConnection.ec2.services.k8s.aws","error":"no matches for kind \"VPCPeeringConnection\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:09:14.653Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCEndpointServiceConfiguration.ec2.services.k8s.aws","error":"no matches for kind \"VPCEndpointServiceConfiguration\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:09:14.742Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCEndpointServiceConfiguration.ec2.services.k8s.aws","error":"no matches for kind \"VPCEndpointServiceConfiguration\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:09:14.746Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VPCPeeringConnection.ec2.services.k8s.aws","error":"no matches for kind \"VPCPeeringConnection\" in version \"ec2.services.k8s.aws/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/source/kind.go:56"}
{"level":"error","ts":"2024-01-31T11:09:24.277Z","msg":"Could not wait for Cache to sync","controller":"vpcendpointserviceconfiguration","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPCEndpointServiceConfiguration","error":"failed to wait for vpcendpointserviceconfiguration caches to sync: timed out waiting for cache to be synced for Kind *v1alpha1.VPCEndpointServiceConfiguration","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:203\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:208\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:234\nsigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).reconcile.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/manager/runnable_group.go:223"}
{"level":"info","ts":"2024-01-31T11:09:24.277Z","msg":"Stopping and waiting for non leader election runnables"}
{"level":"info","ts":"2024-01-31T11:09:24.277Z","msg":"Stopping and waiting for leader election runnables"}
{"level":"info","ts":"2024-01-31T11:09:24.277Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"subnet","controllerGroup":"ec2.services.k8s.aws","controllerKind":"Subnet"}
{"level":"info","ts":"2024-01-31T11:09:24.277Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"natgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"NATGateway"}
{"level":"info","ts":"2024-01-31T11:09:24.277Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"internetgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"InternetGateway"}
{"level":"info","ts":"2024-01-31T11:09:24.277Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"internetgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"InternetGateway"}
{"level":"info","ts":"2024-01-31T11:09:24.277Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"networkacl","controllerGroup":"ec2.services.k8s.aws","controllerKind":"NetworkACL"}
{"level":"info","ts":"2024-01-31T11:09:24.277Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"routetable","controllerGroup":"ec2.services.k8s.aws","controllerKind":"RouteTable"}
{"level":"info","ts":"2024-01-31T11:09:24.277Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"flowlog","controllerGroup":"ec2.services.k8s.aws","controllerKind":"FlowLog"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"instance","controllerGroup":"ec2.services.k8s.aws","controllerKind":"Instance"}
{"level":"info","ts":"2024-01-31T11:09:24.277Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"instance","controllerGroup":"ec2.services.k8s.aws","controllerKind":"Instance"}
{"level":"error","ts":"2024-01-31T11:09:24.277Z","msg":"Could not wait for Cache to sync","controller":"vpcpeeringconnection","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPCPeeringConnection","error":"failed to wait for vpcpeeringconnection caches to sync: timed out waiting for cache to be synced for Kind *v1alpha1.VPCPeeringConnection","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:203\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:208\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:234\nsigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).reconcile.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/manager/runnable_group.go:223"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"networkacl","controllerGroup":"ec2.services.k8s.aws","controllerKind":"NetworkACL"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"vpcendpoint","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPCEndpoint"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"internetgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"InternetGateway"}
{"level":"info","ts":"2024-01-31T11:09:24.277Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"subnet","controllerGroup":"ec2.services.k8s.aws","controllerKind":"Subnet"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"networkacl","controllerGroup":"ec2.services.k8s.aws","controllerKind":"NetworkACL"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"vpcendpoint","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPCEndpoint"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"subnet","controllerGroup":"ec2.services.k8s.aws","controllerKind":"Subnet"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"transitgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"TransitGateway"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"dhcpoptions","controllerGroup":"ec2.services.k8s.aws","controllerKind":"DHCPOptions"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"securitygroup","controllerGroup":"ec2.services.k8s.aws","controllerKind":"SecurityGroup"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"dhcpoptions","controllerGroup":"ec2.services.k8s.aws","controllerKind":"DHCPOptions"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"securitygroup","controllerGroup":"ec2.services.k8s.aws","controllerKind":"SecurityGroup"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"transitgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"TransitGateway"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"vpc","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPC"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"flowlog","controllerGroup":"ec2.services.k8s.aws","controllerKind":"FlowLog"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"vpc","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPC"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"instance","controllerGroup":"ec2.services.k8s.aws","controllerKind":"Instance"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"vpc","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPC"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"elasticipaddress","controllerGroup":"ec2.services.k8s.aws","controllerKind":"ElasticIPAddress"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"elasticipaddress","controllerGroup":"ec2.services.k8s.aws","controllerKind":"ElasticIPAddress"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"dhcpoptions","controllerGroup":"ec2.services.k8s.aws","controllerKind":"DHCPOptions"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"dhcpoptions","controllerGroup":"ec2.services.k8s.aws","controllerKind":"DHCPOptions"}
{"level":"error","ts":"2024-01-31T11:09:24.277Z","msg":"Could not wait for Cache to sync","controller":"vpcendpointserviceconfiguration","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPCEndpointServiceConfiguration","error":"failed to wait for vpcendpointserviceconfiguration caches to sync: timed out waiting for cache to be synced for Kind *v1alpha1.VPCEndpointServiceConfiguration","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:203\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:208\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:234\nsigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).reconcile.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/manager/runnable_group.go:223"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"routetable","controllerGroup":"ec2.services.k8s.aws","controllerKind":"RouteTable"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"routetable","controllerGroup":"ec2.services.k8s.aws","controllerKind":"RouteTable"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"instance","controllerGroup":"ec2.services.k8s.aws","controllerKind":"Instance"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"vpcendpoint","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPCEndpoint"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"vpcendpoint","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPCEndpoint"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"securitygroup","controllerGroup":"ec2.services.k8s.aws","controllerKind":"SecurityGroup"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"securitygroup","controllerGroup":"ec2.services.k8s.aws","controllerKind":"SecurityGroup"}
{"level":"error","ts":"2024-01-31T11:09:24.278Z","msg":"error received after stop sequence was engaged","error":"failed to wait for vpcpeeringconnection caches to sync: timed out waiting for cache to be synced for Kind *v1alpha1.VPCPeeringConnection","stacktrace":"sigs.k8s.io/controller-runtime/pkg/manager.(*controllerManager).engageStopProcedure.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/manager/internal.go:490"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"elasticipaddress","controllerGroup":"ec2.services.k8s.aws","controllerKind":"ElasticIPAddress"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"natgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"NATGateway"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"elasticipaddress","controllerGroup":"ec2.services.k8s.aws","controllerKind":"ElasticIPAddress"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"transitgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"TransitGateway"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"natgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"NATGateway"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"adoptedresource","controllerGroup":"services.k8s.aws","controllerKind":"AdoptedResource"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"adoptedresource","controllerGroup":"services.k8s.aws","controllerKind":"AdoptedResource"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"subnet","controllerGroup":"ec2.services.k8s.aws","controllerKind":"Subnet"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"networkacl","controllerGroup":"ec2.services.k8s.aws","controllerKind":"NetworkACL"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"fieldexport","controllerGroup":"services.k8s.aws","controllerKind":"FieldExport"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"fieldexport","controllerGroup":"services.k8s.aws","controllerKind":"FieldExport"}
{"level":"error","ts":"2024-01-31T11:09:24.278Z","msg":"error received after stop sequence was engaged","error":"failed to wait for vpcendpointserviceconfiguration caches to sync: timed out waiting for cache to be synced for Kind *v1alpha1.VPCEndpointServiceConfiguration","stacktrace":"sigs.k8s.io/controller-runtime/pkg/manager.(*controllerManager).engageStopProcedure.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/manager/internal.go:490"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"transitgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"TransitGateway"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"flowlog","controllerGroup":"ec2.services.k8s.aws","controllerKind":"FlowLog"}
{"level":"info","ts":"2024-01-31T11:09:24.277Z","msg":"Starting workers","controller":"vpcpeeringconnection","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPCPeeringConnection","worker count":1}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"Shutdown signal received, waiting for all workers to finish","controller":"vpcpeeringconnection","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPCPeeringConnection"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"vpc","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPC"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"flowlog","controllerGroup":"ec2.services.k8s.aws","controllerKind":"FlowLog"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"routetable","controllerGroup":"ec2.services.k8s.aws","controllerKind":"RouteTable"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"internetgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"InternetGateway"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"natgateway","controllerGroup":"ec2.services.k8s.aws","controllerKind":"NATGateway"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"All workers finished","controller":"vpcpeeringconnection","controllerGroup":"ec2.services.k8s.aws","controllerKind":"VPCPeeringConnection"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"Stopping and waiting for caches"}
{"level":"info","ts":"2024-01-31T11:09:24.278Z","msg":"Stopping and waiting for webhooks"}
{"level":"info","ts":"2024-01-31T11:09:24.279Z","msg":"Stopping and waiting for HTTP servers"}
{"level":"info","ts":"2024-01-31T11:09:24.279Z","logger":"controller-runtime.metrics","msg":"Shutting down metrics server with timeout of 1 minute"}
{"level":"info","ts":"2024-01-31T11:09:24.279Z","msg":"Wait completed, proceeding to shutdown the manager"}
{"level":"error","ts":"2024-01-31T11:09:24.279Z","logger":"setup","msg":"unable to start controller manager","aws.service":"ec2","error":"failed to wait for vpcendpointserviceconfiguration caches to sync: timed out waiting for cache to be synced for Kind *v1alpha1.VPCEndpointServiceConfiguration","stacktrace":"main.main\n\t/github.com/aws-controllers-k8s/ec2-controller/cmd/controller/main.go:188\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:267"}

@gecube
Copy link
Author

gecube commented Jan 31, 2024

Fixed by changing HelmRelease description according to FluxCD documentation

https://fluxcd.io/flux/components/helm/helmreleases/#controlling-the-lifecycle-of-custom-resource-definitions

apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: ack-ec2-controller
  namespace: ack-system
spec:
  install:
    crds: CreateReplace # HERE
    remediation:
      retries: 3
  upgrade:
    crds: CreateReplace # HERE
    remediation:
      retries: 3
  chart:
    spec:
      chart: ec2-chart
      sourceRef:
        kind: HelmRepository
        name: aws-controllers-k8s
  interval: 1h0m0s
  values:
    metrics:
      service:
        create: true
    serviceAccount:
      annotations:
        eks.amazonaws.com/role-arn: arn:aws:iam::308712144460:role/ack-ec2-controller
    aws:
      region: eu-west-2
    leaderElection:
      enabled: true
Screenshot 2024-01-31 at 12 24 35

@gecube
Copy link
Author

gecube commented Jan 31, 2024

@a-hilaly Hi!

Just two observations

  1. should EC2 controller fail if there is no corresponding CRDs? Probably no, as known resources should be managed properly.
  2. Helm has a very crappy process of updating CRDs. It is not the first time I am observing it. FluxCD is nice as it has workaround (check the link above to the off doc)

@gecube gecube changed the title EC2 controller - crash loop back off EC2 controller - crash loop back off - when upgrading to 1.2.2 Jan 31, 2024
@gecube
Copy link
Author

gecube commented Jan 31, 2024 •
edited
Loading

The same with controllers:

eks-controller @1.2.1
iam-controller @1.3.3
lambda-controller @1.3.6

route53-controller - the same effect...

@a-hilaly a-hilaly added the service/ec2 Indicates issues or PRs that are related to ec2-controller. label Feb 1, 2024
@a-hilaly
Copy link
Member

a-hilaly commented Feb 1, 2024

Hi @gecube !

should EC2 controller fail if there is no corresponding CRDs? Probably no, as known resources should be managed properly.

This is a controller-runtime behaviour.. In each controller, there will be one reconciler per CRD - each one of them will complain if the CRD is not installed.

Helm has a very crappy process of updating CRDs. It is not the first time I am observing it. FluxCD is nice as it has workaround (check the link above to the off doc)

Yeah i completly agree here.. very likely we will build an ACK operator to handle ACK controllers deployment.. and it will not rely on helm.

@a-hilaly
Copy link
Member

a-hilaly commented Feb 6, 2024

@gecube Just double checking, this is not a crash loopback off right? just errors about missing CRDs?

@gecube
Copy link
Author

gecube commented Feb 6, 2024

@a-hilaly Hi! I am kindly asking also add tags for eks/iam/lambda/route53 controllers, as all of them are affected. Regarding your question - it is error about missing CRDs leading to crash loopback off. Please refer to the screenshots above with openshift console.

@a-hilaly
Copy link
Member

AFAIK eks and iam supports tags update. I just pinged the lambda service team to check with their controller.

@a-hilaly
Copy link
Member

@gecube we've made some updates to the helm charts and permission in the last releases. Maybe a reinstallation of the controller could help?

@gecube
Copy link
Author

gecube commented Feb 15, 2024

@a-hilaly yes, I have already fixed the issue - #2007 (comment)
Otherwise (in environment without FluxCD/Argo) I would just remove helm manually and install it back (and it is really interesting question if CRD will stay in cluster or all objects will be removed)...

@Vandita2020
Copy link
Member

Hey @gecube,
We have support for tags in Lambda to apply to the Lambda function. Currently Lambda supports tag for only Function resource.

@gecube gecube mentioned this issue Feb 21, 2024
Open
@gecube gecube mentioned this issue Mar 24, 2024
Closed
@a-hilaly
Copy link
Member

a-hilaly commented Jun 4, 2024

and it is really interesting question if CRD will stay in cluster or all objects will be removed

@gecube AFAIK helm won't update or delete the CRDs

Closing since the helm chart is fixed in recent versions
/close

@ack-prow ACK Prow
Copy link

ack-prow bot commented Jun 4, 2024

@a-hilaly: Closing this issue.

In response to this:

and it is really interesting question if CRD will stay in cluster or all objects will be removed

@gecube AFAIK helm won't update or delete the CRDs

Closing since the helm chart is fixed in recent versions
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@ack-prow ack-prow bot closed this as completed Jun 4, 2024
This was referenced Sep 4, 2024
Closed
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
service/ec2 Indicates issues or PRs that are related to ec2-controller.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@gecube @a-hilaly @Vandita2020