Skip to content

Commit 6c1180e

Browse files
baolsenkayman-mknpalm
authored
feat: support self-signed certificates (#584)
* feat: Support self-signed certificates * to make the PR check happy * fix merge --------- Co-authored-by: Matthias Kay <[email protected]> Co-authored-by: kayma <[email protected]> Co-authored-by: Niek Palm <[email protected]>
1 parent 323f535 commit 6c1180e

16 files changed

+343
-8
lines changed

‎.github/workflows/ci.yml

+1
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ jobs:
3232
"runner-multi-region",
3333
"runner-pre-registered",
3434
"runner-public",
35+
"runner-certificates"
3536
]
3637
defaults:
3738
run:

‎CHANGELOG.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -181,7 +181,7 @@
181181

182182
### ⚠ BREAKING CHANGES
183183

184-
* The module is upgraded to Terraform AWS provider 4.x. All new development will only support the new AWS Terraform provider. We keep a branch `terraform-aws-provider-3` to witch we welcome backports to AWS Terraform 3.x provider. Besides reviewing PR's we will do not any active checking on maintance on this branch. We strongly advise to update your deployment to the new provider version. For more details about upgrading see the [upgrade guide](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/guides/version-4-upgrade).
184+
* The module is upgraded to Terraform AWS provider 4.x. All new development will only support the new AWS Terraform provider. We keep a branch `terraform-aws-provider-3` to witch we welcome backports to AWS Terraform 3.x provider. Besides reviewing PR's we will do not any active checking on maintenance on this branch. We strongly advise to update your deployment to the new provider version. For more details about upgrading see the [upgrade guide](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/guides/version-4-upgrade).
185185
* By default, AWS metadata service ((IMDSv2)[https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service.html]) is enabled and required for both the agent instance and the docker machine instance. For docker machine this require the GitLab managed docker machines distribution is used. Which the module usages by default.
186186

187187

‎README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -304,7 +304,7 @@ module "runner" {
304304
runners_gitlab_url = "https://gitlab.com"
305305
306306
gitlab_runner_registration_config = {
307-
registration_token = "my-token
307+
registration_token = "my-token"
308308
tag_list = "docker"
309309
description = "runner default"
310310
locked_to_project = "true"

‎examples/runner-certificates/README.md

+120
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
# Example - Spot Runner - Private subnet
2+
3+
In this scenario the runner agent is running on a single EC2 node.
4+
5+
The example is intended to show how the runner can be configured for self-hosted Gitlab environments with certificates signed by a custom CA.
6+
7+
> This currently only works with the `docker` executor. Support for the `docker+machine` executor is not yet implemented. Contributions are welcome.
8+
9+
## Prerequisites
10+
11+
The terraform version is managed using [tfenv](https://github.com/Zordrak/tfenv). If you are not using `tfenv` please check `.terraform-version` for the tested version.
12+
13+
Before configuring certificates, it is important to review the [Gitlab documentation](https://docs.gitlab.com/runner/configuration/tls-self-signed.html).
14+
15+
In particular, note the following docker images are involved:
16+
17+
> - The **Runner helper image**, which is used to handle Git, artifacts, and cache operations. In this scenario, the user only needs to make a certificate file available at a specific location (for example, /etc/gitlab-runner/certs/ca.crt), and the Docker container will automatically install it for the user.
18+
19+
> - The **user image**, which is used to run the user script. In this scenario, the user must take ownership regarding how to install a certificate, since this is highly dependent on the image itself, and the Runner has no way of knowing how to install a certificate in each possible scenario.
20+
21+
### Certificates for the runner-helper image
22+
23+
The Gitlab **runner-helper image** needs to communicate with your Gitlab instance.
24+
25+
Create a PEM-encoded `.crt` file containing the public certificate of your Gitlab server instance.
26+
27+
```hcl
28+
module {
29+
...
30+
# Public cert of my companys gitlab instance
31+
runners_gitlab_certificate = file("${path.module}/my_gitlab_instance_cert.crt")
32+
...
33+
}
34+
```
35+
36+
Add your CA and intermediary certs to a second PEM-encoded `.crt` file.
37+
```hcl
38+
module {
39+
...
40+
# Other public certs relating to my company.
41+
runners_ca_certificate = file("${path.module}/my_company_ca_cert_bundle.crt")
42+
...
43+
}
44+
```
45+
46+
### Certificates for user images
47+
48+
For **user images**, you must:
49+
50+
1. Mount the certificates from the EC2 host into all user images.
51+
52+
The runner module can be configured to do this step. Configure the module like so:
53+
54+
```terraform
55+
module {
56+
# ...
57+
58+
# Mount EC2 host certs in docker so all user docker images can reference them.
59+
runners_additional_volumes = ["/etc/gitlab-runner/certs/:/etc/gitlab-runner/certs:ro"]
60+
61+
# ...
62+
}
63+
```
64+
65+
2. Trust the certificates from within the user image.
66+
67+
Each user image will need to execute commands to copy the certificates into the correct place and trust them.
68+
69+
The below examples some ways to do this, assuming user images with the Ubuntu OS or similar.
70+
For Alpine OS user images, the specific commands may differ.
71+
72+
**Option 1:** Build a custom user image and update your `Dockerfile`:
73+
```docker
74+
FROM python:3 # Some base image
75+
76+
RUN apt-get -y update
77+
RUN apt-get -y upgrade
78+
79+
RUN apt-get install -y ca-certificates
80+
RUN cp /etc/gitlab-runner/certs/* /usr/local/share/ca-certificates/
81+
RUN update-ca-certificates
82+
...
83+
```
84+
85+
**Option 2:** Add a section to each pipeline using `before_script`:
86+
87+
This change would need to be added to every pipeline file which requires certificates.
88+
It could be customised depending on the OS of the pipeline user image.
89+
90+
```yaml
91+
default:
92+
before_script:
93+
# Install certificates into user image
94+
- apt-get install -y ca-certificates
95+
- cp /etc/gitlab-runner/certs/* /usr/local/share/ca-certificates/
96+
- update-ca-certificates
97+
```
98+
99+
**Option 3:** Add the script from Option 2 into `runners_pre_build_script` variable:
100+
101+
This avoids maintaining the script in each pipeline file, but expects that all user images use the same OS.
102+
103+
```terraform
104+
module {
105+
# ...
106+
107+
runners_pre_build_script = <<EOT
108+
'''
109+
apt-get install -y ca-certificates
110+
cp /etc/gitlab-runner/certs/* /usr/local/share/ca-certificates/
111+
update-ca-certificates
112+
'''
113+
EOT
114+
115+
# ...
116+
}
117+
```
118+
119+
<!-- BEGIN_TF_DOCS -->
120+
<!-- END_TF_DOCS -->

‎examples/runner-certificates/main.tf

+73
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
data "aws_availability_zones" "available" {
2+
state = "available"
3+
}
4+
5+
module "vpc" {
6+
source = "terraform-aws-modules/vpc/aws"
7+
version = "2.70"
8+
9+
name = "vpc-${var.environment}"
10+
cidr = "10.1.0.0/16"
11+
12+
azs = [data.aws_availability_zones.available.names[0]]
13+
public_subnets = ["10.1.101.0/24"]
14+
enable_s3_endpoint = true
15+
map_public_ip_on_launch = false
16+
17+
tags = {
18+
Environment = var.environment
19+
}
20+
}
21+
22+
module "runner" {
23+
source = "../../"
24+
25+
###############################################
26+
# General
27+
###############################################
28+
29+
runners_name = var.runner_name
30+
runners_gitlab_url = var.gitlab_url
31+
32+
runners_executor = "docker"
33+
34+
aws_region = var.aws_region
35+
environment = var.environment
36+
37+
###############################################
38+
# Certificates
39+
###############################################
40+
41+
# Public cert of my companys gitlab instance
42+
runners_gitlab_certificate = file("${path.module}/my_gitlab_instance_cert.crt")
43+
44+
# Other public certs relating to my company.
45+
runners_ca_certificate = file("${path.module}/my_company_ca_cert_bundle.crt")
46+
47+
# Mount EC2 host certs in docker so all user docker images can reference them.
48+
# Each user image will need to do:
49+
# cp /etc/gitlab-runner/certs/* /usr/local/share/ca-certificates/
50+
# update-ca-certificates
51+
# Or similar OS-dependent commands. The above are an example for Ubuntu.
52+
runners_additional_volumes = ["/etc/gitlab-runner/certs/:/etc/gitlab-runner/certs:ro"]
53+
54+
###############################################
55+
# Registration
56+
###############################################
57+
58+
gitlab_runner_registration_config = {
59+
registration_token = var.registration_token
60+
tag_list = "docker_runner"
61+
description = "runner docker - auto"
62+
locked_to_project = "true"
63+
run_untagged = "false"
64+
maximum_timeout = "3600"
65+
}
66+
67+
###############################################
68+
# Network
69+
###############################################
70+
vpc_id = module.vpc.vpc_id
71+
subnet_id = element(module.vpc.public_subnets, 0)
72+
73+
}

‎examples/runner-certificates/my_company_ca_cert_bundle.crt

+15
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
-----BEGIN CERTIFICATE-----
2+
Put your various CA / intermediary public certificates and other general public certs in this file.
3+
-----END CERTIFICATE-----
4+
5+
-----BEGIN CERTIFICATE-----
6+
My certificate authority public cert 1
7+
-----END CERTIFICATE-----
8+
9+
-----BEGIN CERTIFICATE-----
10+
My intermediary public cert 1
11+
-----END CERTIFICATE-----
12+
13+
-----BEGIN CERTIFICATE-----
14+
Some other instance I need to get to public cert, eg my own docker registry server
15+
-----END CERTIFICATE-----

‎examples/runner-certificates/my_gitlab_instance_cert.crt

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
-----BEGIN CERTIFICATE-----
2+
Put your Gitlab instance public certificate in this file.
3+
Intermediary and CA public certificates are not required.
4+
-----END CERTIFICATE-----

‎examples/runner-certificates/providers.tf

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
provider "aws" {
2+
region = var.aws_region
3+
}
4+
5+
provider "local" {}
6+
7+
provider "null" {}
8+
9+
provider "tls" {}
10+
11+
provider "random" {}

‎examples/runner-certificates/variables.tf

+29
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
variable "aws_region" {
2+
description = "AWS region."
3+
type = string
4+
default = "eu-west-1"
5+
}
6+
7+
variable "environment" {
8+
description = "A name that identifies the environment, will used as prefix and for tagging."
9+
default = "runners-docker"
10+
type = string
11+
}
12+
13+
variable "runner_name" {
14+
description = "Name of the runner, will be used in the runner config.toml"
15+
type = string
16+
default = "docker"
17+
}
18+
19+
variable "gitlab_url" {
20+
description = "URL of the gitlab instance to connect to."
21+
type = string
22+
default = "https://gitlab.com"
23+
}
24+
25+
variable "registration_token" {
26+
description = "Gitlab runner registration token"
27+
type = string
28+
default = "something"
29+
}

‎examples/runner-certificates/versions.tf

+26
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
2+
terraform {
3+
required_version = ">= 1"
4+
required_providers {
5+
aws = {
6+
source = "hashicorp/aws"
7+
version = "~> 4.7"
8+
}
9+
local = {
10+
source = "hashicorp/local"
11+
version = "~> 2"
12+
}
13+
null = {
14+
source = "hashicorp/null"
15+
version = "~> 3.0"
16+
}
17+
tls = {
18+
source = "hashicorp/tls"
19+
version = "~> 3"
20+
}
21+
random = {
22+
source = "hashicorp/random"
23+
version = "~> 3.0"
24+
}
25+
}
26+
}

‎locals.tf

+38
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,42 @@
11
locals {
2+
# Manage certificates
3+
pre_install_gitlab_certificate = (
4+
length(var.runners_gitlab_certificate) > 0
5+
? <<-EOT
6+
mkdir -p /etc/gitlab-runner/certs/
7+
cat <<- EOF > /etc/gitlab-runner/certs/gitlab.crt
8+
${var.runners_gitlab_certificate}
9+
EOF
10+
EOT
11+
: ""
12+
)
13+
pre_install_ca_certificate = (
14+
length(var.runners_ca_certificate) > 0
15+
? <<-EOT
16+
mkdir -p /etc/gitlab-runner/certs/
17+
cat <<- EOF > /etc/gitlab-runner/certs/ca.crt
18+
${var.runners_ca_certificate}
19+
EOF
20+
EOT
21+
: ""
22+
)
23+
pre_install_certificates_end = <<-EOT
24+
chmod 600 /etc/gitlab-runner/certs/*.crt
25+
chmod -R a+r /etc/gitlab-runner
26+
cp /etc/gitlab-runner/certs/*.crt /etc/pki/ca-trust/source/anchors
27+
update-ca-trust extract
28+
EOT
29+
pre_install_certificates = (
30+
# If either (or both) _certificate variables are specified
31+
length(var.runners_gitlab_certificate) + length(var.runners_ca_certificate) > 0
32+
? join("\n", [
33+
local.pre_install_gitlab_certificate,
34+
local.pre_install_ca_certificate,
35+
local.pre_install_certificates_end
36+
])
37+
: ""
38+
)
39+
240
# Determine IAM role for runner instance
341
aws_iam_role_instance_name = coalesce(
442
var.runner_iam_role_name,

‎main.tf

+4-1
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,8 @@ locals {
6060
runners_userdata = var.runners_userdata
6161
runners_executor = var.runners_executor
6262
runners_install_amazon_ecr_credential_helper = var.runners_install_amazon_ecr_credential_helper
63+
curl_cacert = length(var.runners_gitlab_certificate) > 0 ? "--cacert /etc/gitlab-runner/certs/gitlab.crt" : ""
64+
pre_install_certificates = local.pre_install_certificates
6365
pre_install = var.userdata_pre_install
6466
post_install = var.userdata_post_install
6567
runners_gitlab_url = var.runners_gitlab_url
@@ -68,7 +70,7 @@ locals {
6870
secure_parameter_store_runner_sentry_dsn = local.secure_parameter_store_runner_sentry_dsn
6971
secure_parameter_store_region = var.aws_region
7072
gitlab_runner_registration_token = var.gitlab_runner_registration_config["registration_token"]
71-
giltab_runner_description = var.gitlab_runner_registration_config["description"]
73+
gitlab_runner_description = var.gitlab_runner_registration_config["description"]
7274
gitlab_runner_tag_list = var.gitlab_runner_registration_config["tag_list"]
7375
gitlab_runner_locked_to_project = var.gitlab_runner_registration_config["locked_to_project"]
7476
gitlab_runner_run_untagged = var.gitlab_runner_registration_config["run_untagged"]
@@ -82,6 +84,7 @@ locals {
8284
aws_region = var.aws_region
8385
gitlab_url = var.runners_gitlab_url
8486
gitlab_clone_url = var.runners_clone_url
87+
tls_ca_file = length(var.runners_gitlab_certificate) > 0 ? "tls-ca-file=\"/etc/gitlab-runner/certs/gitlab.crt\"" : ""
8588
runners_extra_hosts = var.runners_extra_hosts
8689
runners_vpc_id = var.vpc_id
8790
runners_subnet_id = length(var.subnet_id) > 0 ? var.subnet_id : var.subnet_id_runners

‎outputs.tf

+1-1
Original file line numberDiff line numberDiff line change
@@ -56,4 +56,4 @@ output "runner_launch_template_name" {
5656
output "runner_user_data" {
5757
description = "The user data of the Gitlab Runner Agent's launch template."
5858
value = local.template_user_data
59-
}
59+
}

0 commit comments

Comments
 (0)