Skip to content

Commit a290efa

Browse files
committed
Review updates
* Naming consistency and use the module's "environment" variable * Remove the term "workers" since it's not a documented term and replace it with "instances". This is still somewhat ambiguous when distinguishing between the parent runner 'agent' and the instances it spawns via docker-machine. * Remove useless 'create' variable on 'terminate-instances' submodule. The parent module already toggles 'create' for this module. * Update Lambda function to include 'pending' instances that should be terminated. * Housekeeping - remove unused variables, extraneous comments.
1 parent 0ac5b00 commit a290efa

File tree

14 files changed

+151
-221
lines changed

14 files changed

+151
-221
lines changed

README.md

+5-7
Original file line numberDiff line numberDiff line change
@@ -216,20 +216,20 @@ module "runner" {
216216

217217
### Auto Scaling Group Instance Termination
218218

219-
When the instance in the Auto Scaling Group is terminated, worker instances that were spawned may be left behind unattached to an active runner.
219+
When the runner in the Auto Scaling Group is terminated, runner instances that were spawned may be left behind unattached to an active runner.
220220

221221
This module provides an instance termination [lifecycle hook](https://docs.aws.amazon.com/autoscaling/ec2/userguide/lifecycle-hooks.html)
222-
that executes a provided Lambda function to terminate worker instances that were spawned.
222+
that executes a provided Lambda function to terminate instances that were spawned.
223223

224-
There is a caveat to this scenario in that a worker instance can potentially be running a job
224+
There is a caveat to this scenario in that a instance can potentially be running a job
225225
while it receives the termination signal.
226226

227227
The use of the termination lifecycle can be toggled using the [`asg_termination_lifecycle_hook_create`](#input_asg_terminate_lifecycle_hook_create) variable.
228228

229229
When using this feature, a `builds/` directory relative to the root module will
230230
persist that contains the packaged Lambda function.
231231

232-
Refer to the [terminate-workers module](https://github.com/npalm/terraform-aws-gitlab-runner/tree/develop/modules/terminate-workers) for more information.
232+
Refer to the [terminate-instances module](https://github.com/npalm/terraform-aws-gitlab-runner/tree/develop/modules/terminate-instances) for more information.
233233

234234
## Examples
235235

@@ -281,7 +281,7 @@ terraform destroy
281281
| Name | Source | Version |
282282
|------|--------|---------|
283283
| <a name="module_cache"></a> [cache](#module\_cache) | ./modules/cache | n/a |
284-
| <a name="module_terminate_workers_lifecycle_function"></a> [terminate\_workers\_lifecycle\_function](#module\_terminate\_workers\_lifecycle\_function) | ./modules/terminate-workers | n/a |
284+
| <a name="module_terminate_instances_lifecycle_function"></a> [terminate\_instances\_lifecycle\_function](#module\_terminate\_instances\_lifecycle\_function) | ./modules/terminate-instances | n/a |
285285

286286
## Resources
287287

@@ -348,9 +348,7 @@ terraform destroy
348348
| <a name="input_asg_delete_timeout"></a> [asg\_delete\_timeout](#input\_asg\_delete\_timeout) | Timeout when trying to delete the Runner ASG. | `string` | `"10m"` | no |
349349
| <a name="input_asg_max_instance_lifetime"></a> [asg\_max\_instance\_lifetime](#input\_asg\_max\_instance\_lifetime) | The seconds before an instance is refreshed in the ASG. | `number` | `null` | no |
350350
| <a name="input_asg_terminate_lifecycle_hook_create"></a> [asg\_terminate\_lifecycle\_hook\_create](#input\_asg\_terminate\_lifecycle\_hook\_create) | Boolean toggling the creation of the ASG instance terminate lifecycle hook. | `bool` | `false` | no |
351-
| <a name="input_asg_terminate_lifecycle_hook_create_package"></a> [asg\_terminate\_lifecycle\_hook\_create\_package](#input\_asg\_terminate\_lifecycle\_hook\_create\_package) | Boolean toggling the creation of the ASG instance terminate lifecycle hook Lambda package with the module. False implies it's packaged externally. | `bool` | `true` | no |
352351
| <a name="input_asg_terminate_lifecycle_hook_heartbeat_timeout"></a> [asg\_terminate\_lifecycle\_hook\_heartbeat\_timeout](#input\_asg\_terminate\_lifecycle\_hook\_heartbeat\_timeout) | The amount of time, in seconds, for the instances to remain in wait state. | `number` | `90` | no |
353-
| <a name="input_asg_terminate_lifecycle_hook_local_existing_package"></a> [asg\_terminate\_lifecycle\_hook\_local\_existing\_package](#input\_asg\_terminate\_lifecycle\_hook\_local\_existing\_package) | The path to the zip file containing the ASG instance terminate lifecycle hook Lambda package. Use this when the package is built external from the module. | `string` | `null` | no |
354352
| <a name="input_asg_terminate_lifecycle_hook_name"></a> [asg\_terminate\_lifecycle\_hook\_name](#input\_asg\_terminate\_lifecycle\_hook\_name) | Specifies a custom name for the ASG terminate lifecycle hook and related resources. | `string` | `null` | no |
355353
| <a name="input_aws_region"></a> [aws\_region](#input\_aws\_region) | AWS region. | `string` | n/a | yes |
356354
| <a name="input_cache_bucket"></a> [cache\_bucket](#input\_cache\_bucket) | Configuration to control the creation of the cache bucket. By default the bucket will be created and used as shared cache. To use the same cache across multiple runners disable the creation of the cache and provide a policy and bucket name. See the public runner example for more details. | `map(any)` | <pre>{<br> "bucket": "",<br> "create": true,<br> "policy": ""<br>}</pre> | no |

main.tf

+8-9
Original file line numberDiff line numberDiff line change
@@ -507,20 +507,19 @@ resource "aws_iam_role_policy_attachment" "eip" {
507507
################################################################################
508508
### Lambda function for ASG instance termination lifecycle hook
509509
################################################################################
510-
module "terminate_workers_lifecycle_function" {
511-
source = "./modules/terminate-workers"
510+
module "terminate_instances_lifecycle_function" {
511+
source = "./modules/terminate-instances"
512512

513513
count = var.asg_terminate_lifecycle_hook_create ? 1 : 0
514514

515-
name = local.name_asg_terminate_lifecycle_hook
516-
asg_name = aws_autoscaling_group.gitlab_runner_instance.name
515+
name = var.asg_terminate_lifecycle_hook_name == null ? "terminate-instances" : var.asg_terminate_lifecycle_hook_name
516+
environment = var.environment
517517
asg_arn = aws_autoscaling_group.gitlab_runner_instance.arn
518-
create = var.asg_terminate_lifecycle_hook_create
519-
create_package = var.asg_terminate_lifecycle_hook_create_package
520-
local_existing_package = var.asg_terminate_lifecycle_hook_local_existing_package
521-
role_permissions_boundary = var.permissions_boundary == "" ? null : "${var.arn_format}:iam::${data.aws_caller_identity.current.account_id}:policy/${var.permissions_boundary}"
518+
asg_name = aws_autoscaling_group.gitlab_runner_instance.name
522519
cloudwatch_logging_retention_in_days = var.cloudwatch_logging_retention_in_days
523-
name_iam_objects = "${local.name_iam_objects}-terminate-workers"
520+
lambda_memory_size = var.asg_terminate_lifecycle_lambda_memory_size
524521
lifecycle_heartbeat_timeout = var.asg_terminate_lifecycle_hook_heartbeat_timeout
522+
name_iam_objects = local.name_iam_objects
523+
role_permissions_boundary = var.permissions_boundary == "" ? null : "${var.arn_format}:iam::${data.aws_caller_identity.current.account_id}:policy/${var.permissions_boundary}"
525524
tags = local.tags
526525
}
+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
plugin "aws" {
2+
enabled = true
3+
version = "0.8.0"
4+
source = "github.com/terraform-linters/tflint-ruleset-aws"
5+
}

modules/terminate-workers/README.md modules/terminate-instances/README.md

+17-44
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# terminate-workers module
1+
# terminate-instances module
22

33
Module for Termination Lifecycle Hook Lambda Function
44

@@ -8,10 +8,10 @@ This module is used __internally__ by the parent [_terraform\-aws\-gitlab\-runne
88

99
The Lambda functions evaluates an EC2 instance tag called `gitlab-runner-parent-id`, set in the
1010
[runner config](../../template/runner-config.tpl) by the parent module's
11-
[user data](../../template/gitlab-runner.tpl). Worker instances created by the runner
12-
will have this tag applied with the parent worker's instance ID. When the runner
13-
instance in the ASG is terminated, the lifecycle hook triggers the Lambda to
14-
terminate worker instances with the matching parent tag and/or any "orphaned"
11+
[user data](../../template/gitlab-runner.tpl). Runner instances created by the runner
12+
will have this tag applied with the parent runner's instance ID. When the runner
13+
in the ASG is terminated, the lifecycle hook triggers the Lambda to
14+
terminate spawned runner instances with the matching parent tag and/or any "orphaned"
1515
instances with no running parent runner.
1616

1717
See [issue #214](https://github.com/npalm/terraform-aws-gitlab-runner/issues/214) for
@@ -35,29 +35,6 @@ module "runner" {
3535
source = "npalm/gitlab-runner/aws"
3636
3737
asg_terminate_lifecycle_hook_create = true
38-
asg_terminate_lifecycle_hook_create_package = true
39-
40-
...
41-
```
42-
43-
### Building the package externally
44-
45-
If desirable, the Lambda package may also be built externally.
46-
47-
Set `asg_terminate_lifecycle_hook_create_package` to `false` and
48-
specify the `.zip` file package with the `asg_terminate_lifecycle_hook_local_existing_package`
49-
variable.
50-
51-
This example shows interacting with this module via the parent module's
52-
input variables:
53-
54-
```terraform
55-
module "runner" {
56-
source = "npalm/gitlab-runner/aws"
57-
58-
asg_terminate_lifecycle_hook_create = true
59-
asg_terminate_lifecycle_hook_create_package = false
60-
asg_terminate_lifecycle_hook_local_existing_package = "${path.module}/custom-lambda/foo.zip"
6138
6239
...
6340
```
@@ -72,8 +49,8 @@ Note the `asg_terminate_lifecycle_hook_*` variables:
7249
module "runner" {
7350
source = "npalm/gitlab-runner/aws"
7451
75-
aws_region = "us-west-2"
76-
environment = "glrunners-foo"
52+
aws_region = "eu-west-1"
53+
environment = "glrunners-dev"
7754
runners_name = "glrunners-foo"
7855
runners_gitlab_url = "https://code.foo.org/"
7956
docker_machine_instance_type = "t3.large"
@@ -86,7 +63,6 @@ module "runner" {
8663
8764
asg_max_instance_lifetime = 604800
8865
asg_terminate_lifecycle_hook_create = true
89-
asg_terminate_lifecycle_hook_create_package = true
9066
9167
permissions_boundary = "FooOrg-Permissions-Boundary"
9268
runners_iam_instance_profile_name = "foo-gitlab-runner-profile"
@@ -124,14 +100,13 @@ module "runner" {
124100
|------|---------|
125101
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 0.13 |
126102
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 2.46 |
127-
| <a name="requirement_local"></a> [local](#requirement\_local) | >= 1.4 |
128103

129104
## Providers
130105

131106
| Name | Version |
132107
|------|---------|
133-
| <a name="provider_archive"></a> [archive](#provider\_archive) | n/a |
134-
| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 2.46 |
108+
| <a name="provider_archive"></a> [archive](#provider\_archive) | 2.2.0 |
109+
| <a name="provider_aws"></a> [aws](#provider\_aws) | 3.63.0 |
135110

136111
## Modules
137112

@@ -148,10 +123,10 @@ No modules.
148123
| [aws_iam_policy.lambda](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
149124
| [aws_iam_role.lambda](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource |
150125
| [aws_iam_role_policy_attachment.lambda](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
151-
| [aws_lambda_function.terminate_runner_workers](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_function) | resource |
126+
| [aws_lambda_function.terminate_runner_instances](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_function) | resource |
152127
| [aws_lambda_permission.current_version_triggers](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_permission) | resource |
153128
| [aws_lambda_permission.unqualified_alias_triggers](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_permission) | resource |
154-
| [archive_file.terminate_runner_workers_lambda](https://registry.terraform.io/providers/hashicorp/archive/latest/docs/data-sources/file) | data source |
129+
| [archive_file.terminate_runner_instances_lambda](https://registry.terraform.io/providers/hashicorp/archive/latest/docs/data-sources/file) | data source |
155130
| [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |
156131
| [aws_iam_policy_document.assume_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
157132
| [aws_iam_policy_document.lambda](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
@@ -161,23 +136,21 @@ No modules.
161136
| Name | Description | Type | Default | Required |
162137
|------|-------------|------|---------|:--------:|
163138
| <a name="input_asg_arn"></a> [asg\_arn](#input\_asg\_arn) | The ARN of the Auto Scaling Group to attach to. | `string` | n/a | yes |
164-
| <a name="input_asg_name"></a> [asg\_name](#input\_asg\_name) | The name of the Auto Scaling Group to attach to. | `string` | n/a | yes |
139+
| <a name="input_asg_name"></a> [asg\_name](#input\_asg\_name) | The name of the Auto Scaling Group to attach to. The 'environment' will be prefixed to this. | `string` | n/a | yes |
165140
| <a name="input_cloudwatch_logging_retention_in_days"></a> [cloudwatch\_logging\_retention\_in\_days](#input\_cloudwatch\_logging\_retention\_in\_days) | The number of days to retain logs in CloudWatch. | `number` | `30` | no |
166-
| <a name="input_create"></a> [create](#input\_create) | Boolean toggling whether the reources for the lifecycle hook should be created or not. | `bool` | `true` | no |
167-
| <a name="input_create_package"></a> [create\_package](#input\_create\_package) | Enable packaging the Lambda function by the module. Requires pip and zip. | `bool` | `true` | no |
141+
| <a name="input_environment"></a> [environment](#input\_environment) | A name that identifies the environment, used as a name prefix and for tagging. | `string` | n/a | yes |
142+
| <a name="input_lambda_memory_size"></a> [lambda\_memory\_size](#input\_lambda\_memory\_size) | The memory size in MB to allocate to the Lambda function. | `number` | `128` | no |
168143
| <a name="input_lifecycle_heartbeat_timeout"></a> [lifecycle\_heartbeat\_timeout](#input\_lifecycle\_heartbeat\_timeout) | The amount of time, in seconds, for the instances to remain in wait state. | `number` | `90` | no |
169-
| <a name="input_local_existing_package"></a> [local\_existing\_package](#input\_local\_existing\_package) | If a path is specified, this will use an existing ZIP file built externally for the Lambda function rather than attempt to build and package it in the module. | `string` | `null` | no |
170-
| <a name="input_name"></a> [name](#input\_name) | The name of the Lambda function to create. | `string` | n/a | yes |
171-
| <a name="input_name_iam_objects"></a> [name\_iam\_objects](#input\_name\_iam\_objects) | The name to use for IAM resources - roles and policies | `string` | `""` | no |
144+
| <a name="input_name"></a> [name](#input\_name) | The name of the Lambda function to create. The 'environment' will be prefixed to this. | `string` | n/a | yes |
145+
| <a name="input_name_iam_objects"></a> [name\_iam\_objects](#input\_name\_iam\_objects) | The name to use for IAM resources - roles and policies. | `string` | `""` | no |
172146
| <a name="input_role_permissions_boundary"></a> [role\_permissions\_boundary](#input\_role\_permissions\_boundary) | An optional IAM permissions boundary to use when creating IAM roles. | `string` | `null` | no |
173-
| <a name="input_source_path"></a> [source\_path](#input\_source\_path) | Used if 'local\_existing\_package' is not. This is the path to the Lambda function source to build and package. | `list(any)` | `null` | no |
174147
| <a name="input_tags"></a> [tags](#input\_tags) | Map of tags to apply to resources. | `map(any)` | `{}` | no |
175148

176149
## Outputs
177150

178151
| Name | Description |
179152
|------|-------------|
180-
| <a name="output_lambda_function_arn"></a> [lambda\_function\_arn](#output\_lambda\_function\_arn) | ---------------------------------------------------------------------------- Terminate Workers - Outputs ---------------------------------------------------------------------------- |
153+
| <a name="output_lambda_function_arn"></a> [lambda\_function\_arn](#output\_lambda\_function\_arn) | ---------------------------------------------------------------------------- Terminate Instances - Outputs ---------------------------------------------------------------------------- |
181154
| <a name="output_lambda_function_invoke_arn"></a> [lambda\_function\_invoke\_arn](#output\_lambda\_function\_invoke\_arn) | n/a |
182155
| <a name="output_lambda_function_name"></a> [lambda\_function\_name](#output\_lambda\_function\_name) | n/a |
183156
| <a name="output_lambda_function_source_code_hash"></a> [lambda\_function\_source\_code\_hash](#output\_lambda\_function\_source\_code\_hash) | n/a |

modules/terminate-workers/cloudwatch.tf modules/terminate-instances/cloudwatch.tf

+6-12
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,12 @@
11
# ----------------------------------------------------------------------------
2-
# Terminate Workers - CloudWatch/EventBridge Resources
2+
# Terminate Instances - CloudWatch/EventBridge Resources
33
#
44
# This deploys an event rule and target for triggering the provided Lambda
55
# function from the ASG lifecycle hook.
66
# ----------------------------------------------------------------------------
77
resource "aws_cloudwatch_event_rule" "terminate_instances" {
8-
count = var.create ? 1 : 0
9-
10-
name = var.name
11-
description = "Trigger GitLab runner worker lifecycle hook on termination."
8+
name = "${var.environment}-${var.name}"
9+
description = "Trigger GitLab runner instance lifecycle hook on termination."
1210

1311
event_pattern = <<EOF
1412
{
@@ -22,17 +20,13 @@ EOF
2220
}
2321

2422
resource "aws_cloudwatch_event_target" "terminate_instances" {
25-
count = var.create ? 1 : 0
26-
27-
rule = aws_cloudwatch_event_rule.terminate_instances[0].name
23+
rule = aws_cloudwatch_event_rule.terminate_instances.name
2824
target_id = "TriggerTerminateLambda"
29-
arn = aws_lambda_function.terminate_runner_workers[0].arn
25+
arn = aws_lambda_function.terminate_runner_instances.arn
3026
}
3127

3228
resource "aws_cloudwatch_log_group" "lambda" {
33-
count = var.create ? 1 : 0
34-
35-
name = "/aws/lambda/${var.name}"
29+
name = "/aws/lambda/${var.environment}-${var.name}"
3630
retention_in_days = var.cloudwatch_logging_retention_in_days
3731

3832
tags = var.tags

0 commit comments

Comments
 (0)