Skip to content

Commit bced356

Browse files
npalmkayman-mkmeMuszrSwsystem-ATuin
authored
feat: Terraform AWS Provider Version 4 Upgrade (cattle-ops#460)
* feat: Terraform AWS Provider Version 4 Upgrade BREAKING CHANGE: The module is upgraded to Terraform AWS provider 4.x. All new development will only support the new AWS Terraform provider. We keep a branch `terraform-aws-provider-3` to witch we welcome backports to AWS Terraform 3.x provider. Besides reviewing PR's we will do not any active checking on maintance on this branch. We strongly advise to update your deployment to the new provider version. For more details about upgrading see the [upgrade guide](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/guides/version-4-upgrade). * feat: Add metadata options for docker mahcine (cattle-ops#490) BREAKING CHANGE: By default, AWS metadata service ((IMDSv2)[https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service.html]) is enabled and required for both the agent instance and the docker machine instance. For docker machine this require the GitLab managed docker machines distribution is used. Which the module usages by default. Co-authored-by: Matthias Kay <[email protected]> Co-authored-by: Mustafa Abdul-Kader <[email protected]> Co-authored-by: Steve Wilson <[email protected]>
1 parent 560d9eb commit bced356

File tree

23 files changed

+189
-381
lines changed

23 files changed

+189
-381
lines changed

‎.github/workflows/ci.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ jobs:
2525
strategy:
2626
fail-fast: false
2727
matrix:
28-
terraform: [0.15.0, 1.0.11, latest]
28+
terraform: [1.0.11, latest]
2929
example:
3030
[
3131
"runner-default",

‎.pre-commit-config.yaml

+1
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ repos:
66
args:
77
- --args=-recursive
88
- id: terraform_tflint
9+
- id: terraform_docs
910
- repo: https://github.com/pre-commit/pre-commit-hooks
1011
rev: v4.2.0
1112
hooks:

‎README.md

+18-221
Large diffs are not rendered by default.

‎examples/runner-default/main.tf

-1
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,6 @@ module "runner" {
4343

4444
gitlab_runner_security_group_ids = [data.aws_security_group.default.id]
4545

46-
docker_machine_download_url = "https://gitlab-docker-machine-downloads.s3.amazonaws.com/v0.16.2-gitlab.2/docker-machine"
4746
docker_machine_spot_price_bid = "on-demand-price"
4847

4948
gitlab_runner_registration_config = {

‎examples/runner-default/versions.tf

+4-4
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,22 @@
11

22
terraform {
3-
required_version = ">= 0.15"
3+
required_version = ">= 1"
44
required_providers {
55
aws = {
66
source = "hashicorp/aws"
7-
version = "~> 3.74"
7+
version = "~> 4.7"
88
}
99
local = {
1010
source = "hashicorp/local"
11-
version = "~> 2.2"
11+
version = "~> 2"
1212
}
1313
null = {
1414
source = "hashicorp/null"
1515
version = "~> 3.0"
1616
}
1717
tls = {
1818
source = "hashicorp/tls"
19-
version = "~> 3.3"
19+
version = "~> 3"
2020
}
2121
random = {
2222
source = "hashicorp/random"

‎examples/runner-docker/versions.tf

+4-4
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,22 @@
11

22
terraform {
3-
required_version = ">= 0.15"
3+
required_version = ">= 1"
44
required_providers {
55
aws = {
66
source = "hashicorp/aws"
7-
version = "~> 3.74"
7+
version = "~> 4.7"
88
}
99
local = {
1010
source = "hashicorp/local"
11-
version = "~> 2.2"
11+
version = "~> 2"
1212
}
1313
null = {
1414
source = "hashicorp/null"
1515
version = "~> 3.0"
1616
}
1717
tls = {
1818
source = "hashicorp/tls"
19-
version = "~> 3.3"
19+
version = "~> 3"
2020
}
2121
random = {
2222
source = "hashicorp/random"

‎examples/runner-multi-region/README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -130,4 +130,4 @@ No output.
130130
## Outputs
131131

132132
No outputs.
133-
<!-- END_TF_DOCS -->
133+
<!-- END_TF_DOCS -->

‎examples/runner-multi-region/_docs/README.md

+18
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
# Example - Spot Runner - Public subnets
2+
3+
In this scenario the multiple runner agents can be created with different configuration by instantiating the module multiple times. Runners will scale automatically based on configuration. The S3 cache can be shared cross runners by managing the cache outside the module.
4+
5+
![runners-cache](https://github.com/npalm/assets/raw/master/images/terraform-aws-gitlab-runner/runner-cache.png)
6+
7+
This examples shows:
8+
- Usages of public subnets.
9+
- Usages of multiple runner instances sharing a common cache.
10+
- Overrides for tag naming.
11+
- Registration via GitLab token.
12+
- Auto scaling using `docker+machine` executor.
13+
- Register runner as [protected](https://docs.gitlab.com/ee/ci/runners/configure_runners.html#prevent-runners-from-revealing-sensitive-information).
14+
15+
16+
## Prerequisite
17+
18+
The Terraform version is managed using [tfenv](https://github.com/Zordrak/tfenv). If you are not using `tfenv` please check `.terraform-version` for the tested version.

‎examples/runner-multi-region/_docs/TF_MODULE.md

+25
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
## Providers
2+
3+
| Name | Version |
4+
|------|---------|
5+
| aws | 2.56 |
6+
| local | 1.4 |
7+
| null | 2.1.2 |
8+
| tls | 2.1.1 |
9+
10+
## Inputs
11+
12+
| Name | Description | Type | Default | Required |
13+
|------|-------------|------|---------|:-----:|
14+
| aws\_region | AWS region. | `string` | `"eu-west-1"` | no |
15+
| environment | A name that identifies the environment, will used as prefix and for tagging. | `string` | `"runner-public"` | no |
16+
| gitlab\_url | URL of the gitlab instance to connect to. | `string` | `"https://gitlab.com"` | no |
17+
| private\_ssh\_key\_filename | n/a | `string` | `"generated/id_rsa"` | no |
18+
| public\_ssh\_key\_filename | n/a | `string` | `"generated/id_rsa.pub"` | no |
19+
| registration\_token | n/a | `any` | n/a | yes |
20+
| runner\_name | Name of the runner, will be used in the runner config.toml | `string` | `"public-auto"` | no |
21+
22+
## Outputs
23+
24+
No output.
25+

‎examples/runner-multi-region/versions.tf

+4-4
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,22 @@
11

22
terraform {
3-
required_version = ">= 0.15"
3+
required_version = ">= 1"
44
required_providers {
55
aws = {
66
source = "hashicorp/aws"
7-
version = "~> 3.74"
7+
version = "~> 4.7"
88
}
99
local = {
1010
source = "hashicorp/local"
11-
version = "~> 2.2"
11+
version = "~> 2"
1212
}
1313
null = {
1414
source = "hashicorp/null"
1515
version = "~> 3.0"
1616
}
1717
tls = {
1818
source = "hashicorp/tls"
19-
version = "~> 3.3"
19+
version = "~> 3"
2020
}
2121
random = {
2222
source = "hashicorp/random"

‎examples/runner-pre-registered/versions.tf

+4-4
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,22 @@
11

22
terraform {
3-
required_version = ">= 0.15"
3+
required_version = ">= 1"
44
required_providers {
55
aws = {
66
source = "hashicorp/aws"
7-
version = "~> 3.74"
7+
version = "~> 4.7"
88
}
99
local = {
1010
source = "hashicorp/local"
11-
version = "~> 2.2"
11+
version = "~> 2"
1212
}
1313
null = {
1414
source = "hashicorp/null"
1515
version = "~> 3.0"
1616
}
1717
tls = {
1818
source = "hashicorp/tls"
19-
version = "~> 3.3"
19+
version = "~> 3"
2020
}
2121
random = {
2222
source = "hashicorp/random"

‎examples/runner-public/versions.tf

+4-4
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,22 @@
11

22
terraform {
3-
required_version = ">= 0.15"
3+
required_version = ">= 1"
44
required_providers {
55
aws = {
66
source = "hashicorp/aws"
7-
version = "~> 3.74"
7+
version = "~> 4.7"
88
}
99
local = {
1010
source = "hashicorp/local"
11-
version = "~> 2.2"
11+
version = "~> 2"
1212
}
1313
null = {
1414
source = "hashicorp/null"
1515
version = "~> 3.0"
1616
}
1717
tls = {
1818
source = "hashicorp/tls"
19-
version = "~> 3.3"
19+
version = "~> 3"
2020
}
2121
random = {
2222
source = "hashicorp/random"

‎locals.tf

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
locals {
22
// Convert list to a string separated and prepend by a comma
33
docker_machine_options_string = format(
4-
",%s",
4+
",\"amazonec2-metadata-token=${var.docker_machine_instance_metadata_options.http_tokens}\", \"amazonec2-metadata-token-response-hop-limit=${var.docker_machine_instance_metadata_options.http_put_response_hop_limit}\",%s",
55
join(",", formatlist("%q", concat(var.docker_machine_options, local.runners_docker_registry_mirror_option))),
66
)
77

‎main.tf

+14-3
Original file line numberDiff line numberDiff line change
@@ -160,7 +160,16 @@ resource "aws_autoscaling_group" "gitlab_runner_instance" {
160160
health_check_grace_period = 0
161161
max_instance_lifetime = var.asg_max_instance_lifetime
162162
enabled_metrics = var.metrics_autoscaling
163-
tags = local.agent_tags_propagated
163+
164+
dynamic "tag" {
165+
for_each = local.agent_tags
166+
167+
content {
168+
key = tag.key
169+
value = tag.value
170+
propagate_at_launch = true
171+
}
172+
}
164173

165174
launch_template {
166175
id = aws_launch_template.gitlab_runner_instance.id
@@ -277,8 +286,10 @@ resource "aws_launch_template" "gitlab_runner_instance" {
277286
tags = local.tags
278287

279288
metadata_options {
280-
http_endpoint = var.runner_instance_metadata_options_http_endpoint
281-
http_tokens = var.runner_instance_metadata_options_http_tokens
289+
http_endpoint = var.runner_instance_metadata_options.http_endpoint
290+
http_tokens = var.runner_instance_metadata_options.http_tokens
291+
http_put_response_hop_limit = var.runner_instance_metadata_options.http_put_response_hop_limit
292+
instance_metadata_tags = var.runner_instance_metadata_options.instance_metadata_tags
282293
}
283294

284295
lifecycle {

‎modules/cache/README.md

-106
Original file line numberDiff line numberDiff line change
@@ -25,109 +25,3 @@ module "runner" {
2525
}
2626
```
2727
<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
28-
## Requirements
29-
30-
| Name | Version |
31-
|------|---------|
32-
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 0.15 |
33-
34-
## Providers
35-
36-
| Name | Version |
37-
|------|---------|
38-
| <a name="provider_aws"></a> [aws](#provider\_aws) | n/a |
39-
| <a name="provider_random"></a> [random](#provider\_random) | n/a |
40-
41-
## Modules
42-
43-
No modules.
44-
45-
## Resources
46-
47-
| Name | Type |
48-
|------|------|
49-
| [aws_iam_policy.docker_machine_cache](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
50-
| [aws_s3_bucket.build_cache](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket) | resource |
51-
| [aws_s3_bucket_public_access_block.build_cache_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_public_access_block) | resource |
52-
| [random_string.s3_suffix](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/string) | resource |
53-
| [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |
54-
55-
## Inputs
56-
57-
| Name | Description | Type | Default | Required |
58-
|------|-------------|------|---------|:--------:|
59-
| <a name="input_arn_format"></a> [arn\_format](#input\_arn\_format) | ARN format to be used. May be changed to support deployment in GovCloud/China regions. | `string` | `"arn:aws"` | no |
60-
| <a name="input_cache_bucket_name_include_account_id"></a> [cache\_bucket\_name\_include\_account\_id](#input\_cache\_bucket\_name\_include\_account\_id) | Boolean to add current account ID to cache bucket name. | `bool` | `true` | no |
61-
| <a name="input_cache_bucket_prefix"></a> [cache\_bucket\_prefix](#input\_cache\_bucket\_prefix) | Prefix for s3 cache bucket name. | `string` | `""` | no |
62-
| <a name="input_cache_bucket_set_random_suffix"></a> [cache\_bucket\_set\_random\_suffix](#input\_cache\_bucket\_set\_random\_suffix) | Random string suffix for s3 cache bucket | `bool` | `false` | no |
63-
| <a name="input_cache_bucket_versioning"></a> [cache\_bucket\_versioning](#input\_cache\_bucket\_versioning) | Boolean used to enable versioning on the cache bucket, false by default. | `string` | `"false"` | no |
64-
| <a name="input_cache_expiration_days"></a> [cache\_expiration\_days](#input\_cache\_expiration\_days) | Number of days before cache objects expires. | `number` | `1` | no |
65-
| <a name="input_cache_lifecycle_clear"></a> [cache\_lifecycle\_clear](#input\_cache\_lifecycle\_clear) | Enable the rule to cleanup the cache for expired objects. | `bool` | `true` | no |
66-
| <a name="input_cache_lifecycle_prefix"></a> [cache\_lifecycle\_prefix](#input\_cache\_lifecycle\_prefix) | Object key prefix identifying one or more objects to which the clean up rule applies. | `string` | `"runner/"` | no |
67-
| <a name="input_create_cache_bucket"></a> [create\_cache\_bucket](#input\_create\_cache\_bucket) | This module is by default included in the runner module. To disable the creation of the bucket this parameter can be disabled. | `bool` | `true` | no |
68-
| <a name="input_environment"></a> [environment](#input\_environment) | A name that identifies the environment, used as prefix and for tagging. | `string` | n/a | yes |
69-
| <a name="input_name_iam_objects"></a> [name\_iam\_objects](#input\_name\_iam\_objects) | Set the name prefix of all AWS IAM resources created by this module | `string` | `""` | no |
70-
| <a name="input_tags"></a> [tags](#input\_tags) | Map of tags that will be added to created resources. By default resources will be tagged with name and environment. | `map(string)` | `{}` | no |
71-
72-
## Outputs
73-
74-
| Name | Description |
75-
|------|-------------|
76-
| <a name="output_arn"></a> [arn](#output\_arn) | The ARN of the created bucket. |
77-
| <a name="output_bucket"></a> [bucket](#output\_bucket) | Name of the created bucket. |
78-
| <a name="output_policy_arn"></a> [policy\_arn](#output\_policy\_arn) | Policy for users of the cache (bucket). |
79-
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
80-
81-
<!-- BEGIN_TF_DOCS -->
82-
## Requirements
83-
84-
| Name | Version |
85-
|------|---------|
86-
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 0.15 |
87-
88-
## Providers
89-
90-
| Name | Version |
91-
|------|---------|
92-
| <a name="provider_aws"></a> [aws](#provider\_aws) | n/a |
93-
| <a name="provider_random"></a> [random](#provider\_random) | n/a |
94-
95-
## Modules
96-
97-
No modules.
98-
99-
## Resources
100-
101-
| Name | Type |
102-
|------|------|
103-
| [aws_iam_policy.docker_machine_cache](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
104-
| [aws_s3_bucket.build_cache](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket) | resource |
105-
| [aws_s3_bucket_public_access_block.build_cache_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_public_access_block) | resource |
106-
| [random_string.s3_suffix](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/string) | resource |
107-
| [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |
108-
109-
## Inputs
110-
111-
| Name | Description | Type | Default | Required |
112-
|------|-------------|------|---------|:--------:|
113-
| <a name="input_arn_format"></a> [arn\_format](#input\_arn\_format) | ARN format to be used. May be changed to support deployment in GovCloud/China regions. | `string` | `"arn:aws"` | no |
114-
| <a name="input_cache_bucket_name_include_account_id"></a> [cache\_bucket\_name\_include\_account\_id](#input\_cache\_bucket\_name\_include\_account\_id) | Boolean to add current account ID to cache bucket name. | `bool` | `true` | no |
115-
| <a name="input_cache_bucket_prefix"></a> [cache\_bucket\_prefix](#input\_cache\_bucket\_prefix) | Prefix for s3 cache bucket name. | `string` | `""` | no |
116-
| <a name="input_cache_bucket_set_random_suffix"></a> [cache\_bucket\_set\_random\_suffix](#input\_cache\_bucket\_set\_random\_suffix) | Random string suffix for s3 cache bucket | `bool` | `false` | no |
117-
| <a name="input_cache_bucket_versioning"></a> [cache\_bucket\_versioning](#input\_cache\_bucket\_versioning) | Boolean used to enable versioning on the cache bucket, false by default. | `string` | `"false"` | no |
118-
| <a name="input_cache_expiration_days"></a> [cache\_expiration\_days](#input\_cache\_expiration\_days) | Number of days before cache objects expires. | `number` | `1` | no |
119-
| <a name="input_cache_lifecycle_clear"></a> [cache\_lifecycle\_clear](#input\_cache\_lifecycle\_clear) | Enable the rule to cleanup the cache for expired objects. | `bool` | `true` | no |
120-
| <a name="input_cache_lifecycle_prefix"></a> [cache\_lifecycle\_prefix](#input\_cache\_lifecycle\_prefix) | Object key prefix identifying one or more objects to which the clean up rule applies. | `string` | `"runner/"` | no |
121-
| <a name="input_create_cache_bucket"></a> [create\_cache\_bucket](#input\_create\_cache\_bucket) | This module is by default included in the runner module. To disable the creation of the bucket this parameter can be disabled. | `bool` | `true` | no |
122-
| <a name="input_environment"></a> [environment](#input\_environment) | A name that identifies the environment, used as prefix and for tagging. | `string` | n/a | yes |
123-
| <a name="input_name_iam_objects"></a> [name\_iam\_objects](#input\_name\_iam\_objects) | Set the name prefix of all AWS IAM resources created by this module | `string` | `""` | no |
124-
| <a name="input_tags"></a> [tags](#input\_tags) | Map of tags that will be added to created resources. By default resources will be tagged with name and environment. | `map(string)` | `{}` | no |
125-
126-
## Outputs
127-
128-
| Name | Description |
129-
|------|-------------|
130-
| <a name="output_arn"></a> [arn](#output\_arn) | The ARN of the created bucket. |
131-
| <a name="output_bucket"></a> [bucket](#output\_bucket) | Name of the created bucket. |
132-
| <a name="output_policy_arn"></a> [policy\_arn](#output\_policy\_arn) | Policy for users of the cache (bucket). |
133-
<!-- END_TF_DOCS -->

0 commit comments

Comments
 (0)