Skip to content
This repository was archived by the owner on Jan 16, 2025. It is now read-only.

Commit 4c3ee20

Browse files
authored
feat!: replace tslog by awspowertools logging (#3037)
The module is using TSLog now for a sometime. But after breaking changes we are not able to update TSLog anymore. In the meantime, AWS have released [AWS Powertools for Lambda](https://awslabs.github.io/aws-lambda-powertools-typescript/latest/core/logger/), including logging capabilities. Since this solution is AWS native, we are replacing the outdated TSLog by AWS Powertools logging. ## Breaking change In this PR we replace TSLog by AWS Powertools logs with a minimal impact on code changes. The change can have be breaking in the following cases: - Processing the logging, we only will support JSON formatted log - Depending on the removed setting LOG_TYPE - Depending on the LOG_LEVEL silly, fatal or trace ## Log messages All log messages are JSON formatted and contain besides the message scope information like: `function-name`, `environment`, `module`. By enabling DEBUG log level the event received by the Lambda will be logged.
1 parent 9909d90 commit 4c3ee20

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

60 files changed

+1551
-1740
lines changed

README.md

+50-2
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@
44

55
This [Terraform](https://www.terraform.io/) module creates the required infrastructure needed to host [GitHub Actions](https://github.com/features/actions) self-hosted, auto-scaling runners on [AWS spot instances](https://aws.amazon.com/ec2/spot/). It provides the required logic to handle the life cycle for scaling up and down using a set of AWS Lambda functions. Runners are scaled down to zero to avoid costs when no workflows are active.
66

7+
> 📢 [`v3`](https://github.com/philips-labs/terraform-aws-github-runner/pull/3037) underlying loggin framework is replaces by [AWS Lambda Powertools](https://awslabs.github.io/aws-lambda-powertools-typescript/latest/). Depending on how you handle the logging of the module a migration could be required.
8+
79
> 📢 [`v2`](https://github.com/philips-labs/terraform-aws-github-runner/issues/2517) support via a Terraform [submodule](./modules/multi-runner/README.md) an option to create multiple runners at once. The webhook will deliver based on matching rules the events to a dedicated queue for the runners. Next per queue a scaling lambda will ensure the specific runner is created. For more details checkout the [examples](./examples/multi-runner/README.md).
810
911
> 📢 `v1` is available on a dedicated branch. The default branch `main` is related to `v2`, for fixes or backports you can submit a PR to the branch `v1`. For feature PR's we will ask you to at least submit a PR to `main`
@@ -31,6 +33,7 @@ This [Terraform](https://www.terraform.io/) module creates the required infrastr
3133
- [Experimental - Optional queue to publish GitHub workflow job events](#experimental---optional-queue-to-publish-github-workflow-job-events)
3234
- [Examples](#examples)
3335
- [Sub modules](#sub-modules)
36+
- [Logging](#logging)
3437
- [Debugging](#debugging)
3538
- [Security Consideration](#security-consideration)
3639
- [Requirements](#requirements)
@@ -41,6 +44,12 @@ This [Terraform](https://www.terraform.io/) module creates the required infrastr
4144
- [Outputs](#outputs)
4245
- [Contribution](#contribution)
4346
- [Philips Forest](#philips-forest)
47+
- [Requirements](#requirements-1)
48+
- [Providers](#providers-1)
49+
- [Modules](#modules-1)
50+
- [Resources](#resources-1)
51+
- [Inputs](#inputs-1)
52+
- [Outputs](#outputs-1)
4453

4554
## Motivation
4655

@@ -384,6 +393,45 @@ The following sub modules are optional and are provided as example or utility:
384393

385394
ARM64 configuration for submodules. When using the top level module configure `runner_architecture = "arm64"` and ensure the list of `instance_types` matches. When not using the top-level, ensure these properties are set on the submodules.
386395

396+
## Logging
397+
398+
The module uses [AWS Lambda Powertools](https://awslabs.github.io/aws-lambda-powertools-typescript/latest/) for logging. By default the log level is set to `info`, by setting the leg level to `debug` the incoming event of the Lambda is logged as well.
399+
400+
Log messages contains at least the following keys:
401+
402+
- `messages`: The logged messages
403+
- `environment`: The environment prefix provided via Terraform
404+
- `service`: The lambda
405+
- `module`: The TypeScript module writing the log message
406+
- `function-name`: The name of the lambda function (prefix + function name)
407+
- `github`: Depending on the lambda, contains GitHub context
408+
- `runner`: Depending on the lambda, specific context related to the runner
409+
410+
An example log messages of the scale-up function:
411+
412+
```json
413+
{
414+
"level": "INFO",
415+
"message": "Received event",
416+
"service": "runners-scale-up",
417+
"timestamp": "2023-03-20T08:15:27.448Z",
418+
"xray_trace_id": "1-6418161e-08825c2f575213ef760531bf",
419+
"module": "scale-up",
420+
"region": "eu-west-1",
421+
"environment": "my-linux-x64",
422+
"aws-request-id": "eef1efb7-4c07-555f-9a67-b3255448ee60",
423+
"function-name": "my-linux-x64-scale-up",
424+
"runner": {
425+
"type": "Repo",
426+
"owner": "test-runners/multi-runner"
427+
},
428+
"github": {
429+
"event": "workflow_job",
430+
"workflow_job_id": "1234"
431+
}
432+
}
433+
```
434+
387435
## Debugging
388436

389437
In case the setup does not work as intended follow the trace of events:
@@ -492,7 +540,7 @@ We welcome any improvement to the standard module to make the default as secure
492540
| <a name="input_lambda_security_group_ids"></a> [lambda\_security\_group\_ids](#input\_lambda\_security\_group\_ids) | List of security group IDs associated with the Lambda function. | `list(string)` | `[]` | no |
493541
| <a name="input_lambda_subnet_ids"></a> [lambda\_subnet\_ids](#input\_lambda\_subnet\_ids) | List of subnets in which the action runners will be launched, the subnets needs to be subnets in the `vpc_id`. | `list(string)` | `[]` | no |
494542
| <a name="input_log_level"></a> [log\_level](#input\_log\_level) | Logging level for lambda logging. Valid values are 'silly', 'trace', 'debug', 'info', 'warn', 'error', 'fatal'. | `string` | `"info"` | no |
495-
| <a name="input_log_type"></a> [log\_type](#input\_log\_type) | Logging format for lambda logging. Valid values are 'json', 'pretty', 'hidden'. | `string` | `"pretty"` | no |
543+
| <a name="input_log_type"></a> [log\_type](#input\_log\_type) | Logging format for lambda logging. Valid values are 'json', 'pretty', 'hidden'. | `string` | `null` | no |
496544
| <a name="input_logging_kms_key_id"></a> [logging\_kms\_key\_id](#input\_logging\_kms\_key\_id) | Specifies the kms key id to encrypt the logs with | `string` | `null` | no |
497545
| <a name="input_logging_retention_in_days"></a> [logging\_retention\_in\_days](#input\_logging\_retention\_in\_days) | Specifies the number of days you want to retain log events for the lambda log group. Possible values are: 0, 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, and 3653. | `number` | `180` | no |
498546
| <a name="input_minimum_running_time_in_minutes"></a> [minimum\_running\_time\_in\_minutes](#input\_minimum\_running\_time\_in\_minutes) | The time an ec2 action runner should be running at minimum before terminated if not busy. | `number` | `null` | no |
@@ -742,4 +790,4 @@ Talk to the forestkeepers in the `runners`-channel on Slack.
742790
| <a name="output_runners"></a> [runners](#output\_runners) | n/a |
743791
| <a name="output_ssm_parameters"></a> [ssm\_parameters](#output\_ssm\_parameters) | n/a |
744792
| <a name="output_webhook"></a> [webhook](#output\_webhook) | n/a |
745-
<!-- END_TF_DOCS -->
793+
<!-- END_TF_DOCS -->

examples/default/main.tf

+3
Original file line numberDiff line numberDiff line change
@@ -90,4 +90,7 @@ module "runners" {
9090

9191
# prefix GitHub runners with the environment name
9292
runner_name_prefix = "${local.environment}_"
93+
94+
# Enable debug logging for the lambda functions
95+
# log_level = "debug"
9396
}

examples/ephemeral/main.tf

+2-2
Original file line numberDiff line numberDiff line change
@@ -75,8 +75,8 @@ module "runners" {
7575
# data "aws_caller_identity" "current" {}
7676
# ami_owners = [data.aws_caller_identity.current.account_id]
7777

78-
# Enable logging
79-
log_level = "debug"
78+
# Enable debug logging for the lambda functions
79+
# log_level = "debug"
8080

8181
# Setup a dead letter queue, by default scale up lambda will kepp retrying to process event in case of scaling error.
8282
# redrive_policy_build_queue = {

examples/multi-runner/main.tf

+1
Original file line numberDiff line numberDiff line change
@@ -167,6 +167,7 @@ module "multi-runner" {
167167
# enable_workflow_job_events_queue = true
168168
# override delay of events in seconds
169169

170+
# Enable debug logging for the lambda functions
170171
# log_level = "debug"
171172

172173
}

main.tf

-3
Original file line numberDiff line numberDiff line change
@@ -165,7 +165,6 @@ module "webhook" {
165165
lambda_security_group_ids = var.lambda_security_group_ids
166166
aws_partition = var.aws_partition
167167

168-
log_type = var.log_type
169168
log_level = var.log_level
170169
}
171170

@@ -267,7 +266,6 @@ module "runners" {
267266

268267
kms_key_arn = var.kms_key_arn
269268

270-
log_type = var.log_type
271269
log_level = var.log_level
272270

273271
pool_config = var.pool_config
@@ -308,7 +306,6 @@ module "runner_binaries" {
308306
role_path = var.role_path
309307
role_permissions_boundary = var.role_permissions_boundary
310308

311-
log_type = var.log_type
312309
log_level = var.log_level
313310

314311
lambda_subnet_ids = var.lambda_subnet_ids

0 commit comments

Comments
 (0)