EC2 controller - no way of managing default security group in VPC #1896

gecube · 2023-09-14T07:41:40Z

Good day!

I am playing around with EC2 controller and found that basic creation of VPC with the manifest like

apiVersion: ec2.services.k8s.aws/v1alpha1
kind: VPC
metadata:
  name: dev
  namespace: infra-dev
spec:
  cidrBlocks:
    - 10.10.0.0/16
  enableDNSSupport: true
  enableDNSHostnames: true
  tags:
    - key: Name
      value: dev

leads to creation default security group.

Unfortunately, this security group has allow all rules for inbound and outbound connections and fails security check:

If I create additional security groups, they are created well.

I'd like to have nice and clean way of managing this "default" security group in terms of EC2 controller objects. Probably - the adoption pattern won't be very good here, as an operator of ACK I want to create all relevant objects and configure them in one go. I am kindly asking to give suggestion and options how to achieve the desired state.

gecube · 2023-09-14T08:01:07Z

Some more clues.

I can't create a default security group from code directly. It won't work as controller throws error:

2023-09-14T07:50:02.695Z	ERROR	Reconciler error	{"controller": "securitygroup", "controllerGroup": "ec2.services.k8s.aws", "controllerKind": "SecurityGroup", "SecurityGroup": {"name":"default","namespace":"infra-dev"}, "namespace": "infra-dev", "name": "default", "reconcileID": "138bd869-c9c9-457e-829f-a73fe0698afc", "error": "InvalidParameterValue: Cannot use reserved security group name: default\n\tstatus code: 400, request id: d85c0a9a-ff92-4b28-8acd-610732d1a698"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:329
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:274
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:235

Also there is no direct way to adopt this sg. If I try to apply manifest like:

apiVersion: services.k8s.aws/v1alpha1
kind: AdoptedResource
metadata:
  name: adopt-my-existing-bucket
  namespace: infra-dev
spec:
  aws:
    nameOrID: default
  kubernetes:
    group: ec2.services.k8s.aws
    kind: SecurityGroup
    metadata:
      name: default
      namespace: infra-dev

The EC2 controllers gives me in status:

  conditions:
    - message: "InvalidGroupId.Malformed: Invalid id: \"default\" (expecting \"sg-...\")\n\tstatus code: 400, request id: 56865034-285f-4356-ae9a-29efcd1ee9c2"
      status: 'False'
      type: ACK.Adopted

But I can't retrieve the Group ID directly from VPC description...
Everything I know about VPC is written directly into the status field of VPC:

status:
  ackResourceMetadata:
    ownerAccountID: '178394743802'
    region: eu-west-2
  cidrBlockAssociationSet:
    - associationID: vpc-cidr-assoc-0f97c3c42baf28acf
      cidrBlock: 10.10.0.0/16
      cidrBlockState:
        state: associated
  conditions:
    - lastTransitionTime: '2023-09-14T01:42:47Z'
      message: Resource synced successfully
      reason: ''
      status: 'True'
      type: ACK.ResourceSynced
  dhcpOptionsID: dopt-b0cbf6d8
  isDefault: false
  ownerID: '178394743802'
  state: available
  vpcID: vpc-0017152cc2d43a69a

and there is no security group id, but it could be logical to add it there.

gecube · 2023-09-14T08:10:54Z

O.K. so I can retrieve the security group name from Amazon Console and substitute it in YAML:

apiVersion: services.k8s.aws/v1alpha1
kind: AdoptedResource
metadata:
  name: default-security-group
  namespace: infra-dev
spec:
  aws:
    nameOrID: sg-0e87e0dd9f6d43f31
  kubernetes:
    group: ec2.services.k8s.aws
    kind: SecurityGroup
    metadata:
      name: default
      namespace: infra-dev

and then I am getting the cryptic error message like:

status:
  conditions:
    - message: >-
        SecurityGroup.ec2.services.k8s.aws "default" is invalid: spec.name:
        Required value
      status: 'False'
      type: ACK.Adopted

and

2023-09-14T08:08:46.656Z	ERROR	Reconciler error	{"controller": "adoptedresource", "controllerGroup": "services.k8s.aws", "controllerKind": "AdoptedResource", "AdoptedResource": {"name":"default-security-group","namespace":"infra-dev"}, "namespace": "infra-dev", "name": "default-security-group", "reconcileID": "d860b0a9-2f5b-48fe-afd7-324ab67a8394", "error": "SecurityGroup.ec2.services.k8s.aws \"default\" is invalid: spec.name: Required value"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:329
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:274
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:235

in EC2 controller logs

No idea what does it mean.

a-hilaly · 2023-09-14T20:13:12Z

@gecube Can you try settings metadata.name to something different than default?

Regarding the default security group created with the VPC, maybe we could consider deleting it right after a VPC creation. I wish there was a way to create a subnet-less-vpc in https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_CreateVpc.html

gecube · 2023-09-15T08:12:47Z

@a-hilaly Hi! The same:

spec:
  aws:
    nameOrID: sg-0e87e0dd9f6d43f31
  kubernetes:
    group: ec2.services.k8s.aws
    kind: SecurityGroup
    metadata:
      name: default-2
      namespace: infra-dev
status:
  conditions:
    - message: >-
        SecurityGroup.ec2.services.k8s.aws "default-2" is invalid: spec.name:
        Required value
      status: 'False'
      type: ACK.Adopted

logs:

{"level":"error","ts":"2023-09-15T08:11:00.409Z","msg":"Reconciler error","controller":"adoptedresource","controllerGroup":"services.k8s.aws","controllerKind":"AdoptedResource","AdoptedResource":{"name":"default-security-group-2","namespace":"infra-dev"},"namespace":"infra-dev","name":"default-security-group-2","reconcileID":"846a56de-f57b-4f9f-89a7-840ced5b3dc1","error":"SecurityGroup.ec2.services.k8s.aws \"default-2\" is invalid: spec.name: Required value","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:235"}
{"level":"info","ts":"2023-09-15T08:12:22.335Z","logger":"adoption.adopted-reconciler","msg":"starting adoption reconciliation","target_group":"ec2.services.k8s.aws","target_kind":"SecurityGroup","namespace":"infra-dev","name":"default-security-group-2","generation":1}
{"level":"error","ts":"2023-09-15T08:12:22.446Z","msg":"Reconciler error","controller":"adoptedresource","controllerGroup":"services.k8s.aws","controllerKind":"AdoptedResource","AdoptedResource":{"name":"default-security-group-2","namespace":"infra-dev"},"namespace":"infra-dev","name":"default-security-group-2","reconcileID":"1c7c782a-d3c4-40c5-b518-083ec65cdb04","error":"SecurityGroup.ec2.services.k8s.aws \"default-2\" is invalid: spec.name: Required value","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:235"}

mattzech · 2023-10-19T21:00:23Z

Any update on this? We are facing the same issue. Thanks!

ack-bot · 2024-04-17T01:02:40Z

Issues go stale after 180d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 60d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/aws-controllers-k8s/community.
/lifecycle stale

gecube · 2024-04-17T12:42:22Z

/remove-lifecycle stale

ack-bot · 2024-10-14T17:34:07Z

Issues go stale after 180d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 60d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/aws-controllers-k8s/community.
/lifecycle stale

a-hilaly · 2024-10-14T18:04:42Z

@gecube Good day! i think now you can request a deletion of the default securitygroup rules with aws-controllers-k8s/ec2-controller#212

ack-bot · 2025-04-12T19:55:21Z

Issues go stale after 180d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 60d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/aws-controllers-k8s/community.
/lifecycle stale

gecube · 2025-04-14T09:45:31Z

/remove-lifecycle stale

gecube changed the title ~~EC2 controller - VPC creation leads to creation of default security group~~ EC2 controller - VPC creation leads to creation of default security group, no way of managing it Sep 14, 2023

gecube changed the title ~~EC2 controller - VPC creation leads to creation of default security group, no way of managing it~~ EC2 controller - no way of managing default security group in VPC Sep 14, 2023

a-hilaly added service/ec2 Indicates issues or PRs that are related to ec2-controller. kind/bug Categorizes issue or PR as related to a bug. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Sep 14, 2023

gecube mentioned this issue Feb 5, 2024

Read-only Resources Feature Request #2008

Closed

ack-prow bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Apr 17, 2024

ack-prow bot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Apr 17, 2024

ack-prow bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Oct 14, 2024

a-hilaly removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Oct 14, 2024

ack-prow bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Apr 12, 2025

ack-prow bot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Apr 14, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

EC2 controller - no way of managing default security group in VPC #1896

EC2 controller - no way of managing default security group in VPC #1896

gecube commented Sep 14, 2023

gecube commented Sep 14, 2023

Uh oh!

gecube commented Sep 14, 2023

Uh oh!

a-hilaly commented Sep 14, 2023

Uh oh!

gecube commented Sep 15, 2023

Uh oh!

mattzech commented Oct 19, 2023

Uh oh!

ack-bot commented Apr 17, 2024

Uh oh!

gecube commented Apr 17, 2024

Uh oh!

ack-bot commented Oct 14, 2024

Uh oh!

a-hilaly commented Oct 14, 2024

Uh oh!

ack-bot commented Apr 12, 2025

Uh oh!

gecube commented Apr 14, 2025

Uh oh!

EC2 controller - no way of managing default security group in VPC #1896

EC2 controller - no way of managing default security group in VPC #1896

Comments

gecube commented Sep 14, 2023

gecube commented Sep 14, 2023

Uh oh!

gecube commented Sep 14, 2023

Uh oh!

a-hilaly commented Sep 14, 2023

Uh oh!

gecube commented Sep 15, 2023

Uh oh!

mattzech commented Oct 19, 2023

Uh oh!

ack-bot commented Apr 17, 2024

Uh oh!

gecube commented Apr 17, 2024

Uh oh!

ack-bot commented Oct 14, 2024

Uh oh!

a-hilaly commented Oct 14, 2024

Uh oh!

ack-bot commented Apr 12, 2025

Uh oh!

gecube commented Apr 14, 2025

Uh oh!