feat!: add custom egress rules to worker security groups

[ikarlashov]

Add custom egress rules to docker-autoscaler security group and remove condition to provision unused docker-machine security group.

Description

1. By default, the module provisions a security group for Docker Autoscaler workers with egress rules that allow ALL traffic. Unlike ingress rules, egress rules are not customizable, which poses a significant security concern. This PR introduces the ability to customize egress rules for the Docker Autoscaler workers' security group by declaring a separate variable for docker-autoscaler egress rules.
2. var.runner_worker_docker_autoscaler_asg becomes bulky and hard to read. Considering the complexity of the ingress rules structure, it makes sense to create a separate variable var.runner_worker_docker_autoscaler_ingress_rules for ingress rules. This way we will follow variable convention for existing security group rules variables, i.e. var.runner_worker_docker_machine_extra_egress_rules. In the result of the change: var.runner_worker_docker_autoscaler_asg.sg_ingresses is removed and its content should be moved to var.runner_worker_ingress_rules.
3. Adding var.runner_ingress_rules to manage Ingress rules for runner-manager security group.
4. Changed runner-manager egress rules' var name and its spec. var.runner_networking_egress_rules has migrated to var.runner_egress_rules with a new spec.
5. Additionally, PR removes the condition that provisions an unused security group intended solely for Docker Machine setup.

Migrations required

1. Move all docker-autoscaler ingress rules declaration from var.runner_worker_docker_autoscaler_asg.sg_ingresses to var.runner_worker_ingress_rules.
2. var.runner_networking_egress_rules migrated to var.runner_manager_egress_rules with a new spec.
3. Move var.runner_worker_docker_machine_extra_egress_rules to var.runner_worker_egress_rules. In case you used multiple cidr_blocks, ... you have to create multiple rules.

Attention: The default value for var.runner_worker_docker_machine_extra_egress_rules allowing all egress traffic has been replaced by a rule allowing traffic to port 22/443 only. Don't forget to add the rules you need to var.runner_worker_egress_rules.

Attention: Due to the resource replacement you might see inconsistencies and disappearing security group rules. In that case delete all rules from the Runner and Runner Workers and apply the module again.

Sample egress rule:

runner_worker_egress_rules = [
     {
      cidr_block    = "0.0.0.0/0"
      from_port     = 443
      protocol        = "tcp"
      to_port          = 443
      description   = "Allow HTTPS egress traffic."
    },
   {
    ipv6_cidr_block = "::/0"
    description         = "Allow HTTPS Egress everywhere"
    from_port           = 443
    protocol              = "tcp"
    to_port                =  443
   }
]

[github-actions]

Hey @ikarlashov! 👋

Thank you for your contribution to the project. Please refer to the contribution rules for a quick overview of the process.

Make sure that this PR clearly explains:

• the problem being solved
• the best way a reviewer and you can test your changes

With submitting this PR you confirm that you hold the rights of the code added and agree that it will published under this LICENSE.

The following ChatOps commands are supported:

• /help: notifies a maintainer to help you out

Simply add a comment with the command in the first line. If you need to pass more information, separate it with a blank line from the command.

This message was generated automatically. You are welcome to improve it.

[kayman-mk]

Thanks for your work to improve this module. I noticed the major change, but I think we can go on as it is easy to handle for the users.

[ikarlashov]

It would be nice to allow traffic within runner-manager ASG and Docker-autoscaler workers ASG. Basically add this to security_groups.tf:

resource "aws_vpc_security_group_egress_rule" "runner_manager_to_docker_autoscaler_egress" {
  count = var.runner_worker.type == "docker-autoscaler" ? 1 : 0

  security_group_id            = aws_security_group.runner.id
  from_port                    = 0
  to_port                      = 0
  ip_protocol                  = "-1"
  description                  = "Allow ALL Egress traffic between Runner Manager and Docker-autoscaler workers security group"
  referenced_security_group_id = aws_security_group.docker_autoscaler[0].id
}

But if I add it, terraform doesn't detect any changes for runner's SG.

[kayman-mk]

It would be nice to allow traffic within runner-manager ASG and Docker-autoscaler workers ASG. Basically add this to security_groups.tf:

resource "aws_vpc_security_group_egress_rule" "runner_manager_to_docker_autoscaler_egress" {
  count = var.runner_worker.type == "docker-autoscaler" ? 1 : 0

  security_group_id            = aws_security_group.runner.id
  from_port                    = 0
  to_port                      = 0
  ip_protocol                  = "-1"
  description                  = "Allow ALL Egress traffic between Runner Manager and Docker-autoscaler workers security group"
  referenced_security_group_id = aws_security_group.docker_autoscaler[0].id
}

But if I add it, terraform doesn't detect any changes for runner's SG.

Perhaps it is already in place?

[kayman-mk]

Did a refactoring and notice that we also have aws_security_group.docker_machine which has the same problems. I think we should have a runner_worker_ingress_rules/runner_worker_egress_rules without specifying the machine type as the meaning is the same.

[kayman-mk]

Verification

runner_worker_docker_autoscaler_ingress_rules =  [
    {
      cidr_block      = "10.0.0.0/8"  # Example CIDR block for a private network in Amazon
      from_port        = 22
      protocol         = "tcp"
      to_port          = 22
      description      = "Allow SSH Ingress traffic for private network."
    }
]

runner_worker_docker_autoscaler_egress_rules = [
     {
      cidr_block    = "0.0.0.0/0"
      from_port     = 443
      protocol        = "tcp"
      to_port          = 443
      description   = "Allow HTTPS egress traffic."
    },
   {
    ipv6_cidr_block = "::/0"
    description         = "Allow HTTPS Egress everywhere"
    from_port           = 443
    protocol              = "tcp"
    to_port                =  443
   }
]

[kayman-mk]

Looks good to me.

[kayman-mk]

I'd like to test this with my current setup before merging it.

[kayman-mk]

Checked with docker+machine.

1. agent was not able to contact the workers --> security group rule added.
2. workers were not able to clone via SSH --> default egress rule extended

@ikarlashov Could you please have a look at my commits?

[Alexandre-Orisha]

Did a refactoring and notice that we also have aws_security_group.docker_machine which has the same problems. I think we should have a runner_worker_ingress_rules/runner_worker_egress_rules without specifying the machine type as the meaning is the same.

Hello, my setup is using docker-autoscaler and I just upgraded from 8.1.0 to 9.0.0.

I think specifying the machine type would be better because now Terraform is trying to create egress rules for a docker+machine security group that doesn't exist.

The only way to make the module work is to explicitly give an empty runner_worker_egress_rules which is quite unhelpful because my workers won't be able to connect to the internet.

Please correct me if I'm wrong.

[kayman-mk]

@Alexandre-Orisha Perfectly right! Fixed in 9.0.1