feat: run spot instances without specifying the spot price (#391)

* Determine the current "on demand" price and use as spot price (maximum)

* Fix the pricing resource name

* Determine the max spot price for the Gitlab runner as well

* Format code

* Simplify the spot price. "empty value" means up to on demand price.

* Use "on-demand-price"

* Updating the docs

* Update examples

Co-authored-by: kayma <[email protected]>

Author: kayman-mk

README.md

@@ -323,6 +323,7 @@ terraform destroy
 | <a name="input_ami_owners"></a> [ami\_owners](#input\_ami\_owners) | The list of owners used to select the AMI of Gitlab runner agent instances. | `list(string)` | <pre>[<br>  "amazon"<br>]</pre> | no |
 | <a name="input_arn_format"></a> [arn\_format](#input\_arn\_format) | ARN format to be used. May be changed to support deployment in GovCloud/China regions. | `string` | `"arn:aws"` | no |
 | <a name="input_asg_delete_timeout"></a> [asg\_delete\_timeout](#input\_asg\_delete\_timeout) | Timeout when trying to delete the Runner ASG. | `string` | `"10m"` | no |
+| <a name="input_asg_max_instance_lifetime"></a> [asg\_max\_instance\_lifetime](#input\_asg\_max\_instance\_lifetime) | The seconds before an instance is refreshed in the ASG. | `number` | `null` | no |
 | <a name="input_aws_region"></a> [aws\_region](#input\_aws\_region) | AWS region. | `string` | n/a | yes |
 | <a name="input_aws_zone"></a> [aws\_zone](#input\_aws\_zone) | Deprecated. Will be removed in the next major release. | `string` | `"a"` | no |
 | <a name="input_cache_bucket"></a> [cache\_bucket](#input\_cache\_bucket) | Configuration to control the creation of the cache bucket. By default the bucket will be created and used as shared cache. To use the same cache across multiple runners disable the creation of the cache and provide a policy and bucket name. See the public runner example for more details. | `map(any)` | <pre>{<br>  "bucket": "",<br>  "create": true,<br>  "policy": ""<br>}</pre> | no |
@@ -340,10 +341,9 @@ terraform destroy
 | <a name="input_docker_machine_options"></a> [docker\_machine\_options](#input\_docker\_machine\_options) | List of additional options for the docker machine config. Each element of this list must be a key=value pair. E.g. '["amazonec2-zone=a"]' | `list(string)` | `[]` | no |
 | <a name="input_docker_machine_role_json"></a> [docker\_machine\_role\_json](#input\_docker\_machine\_role\_json) | Docker machine runner instance override policy, expected to be in JSON format. | `string` | `""` | no |
 | <a name="input_docker_machine_security_group_description"></a> [docker\_machine\_security\_group\_description](#input\_docker\_machine\_security\_group\_description) | A description for the docker-machine security group | `string` | `"A security group containing docker-machine instances"` | no |
-| <a name="input_docker_machine_spot_price_bid"></a> [docker\_machine\_spot\_price\_bid](#input\_docker\_machine\_spot\_price\_bid) | Spot price bid. | `string` | `"0.06"` | no |
+| <a name="input_docker_machine_spot_price_bid"></a> [docker\_machine\_spot\_price\_bid](#input\_docker\_machine\_spot\_price\_bid) | Spot price bid. The maximum price willing to pay. By default the price is limited by the current on demand price for the instance type chosen. | `string` | `"on-demand-price"` | no |
 | <a name="input_docker_machine_version"></a> [docker\_machine\_version](#input\_docker\_machine\_version) | By default docker\_machine\_download\_url is used to set the docker machine version. Version of docker-machine. The version will be ingored once `docker_machine_download_url` is set. | `string` | `""` | no |
 | <a name="input_enable_asg_recreation"></a> [enable\_asg\_recreation](#input\_enable\_asg\_recreation) | Enable automatic redeployment of the Runner ASG when the Launch Configs change. | `bool` | `true` | no |
-| <a name="input_asg_max_instance_lifetime"></a> [asg\_max\_instance\_lifetime](#input\_asg\_max\_instance\_lifetime) | The seconds before an instance is refreshed in the ASG. | `number` | `null` | no |
 | <a name="input_enable_cloudwatch_logging"></a> [enable\_cloudwatch\_logging](#input\_enable\_cloudwatch\_logging) | Boolean used to enable or disable the CloudWatch logging. | `bool` | `true` | no |
 | <a name="input_enable_docker_machine_ssm_access"></a> [enable\_docker\_machine\_ssm\_access](#input\_enable\_docker\_machine\_ssm\_access) | Add IAM policies to the docker-machine instances to connect via the Session Manager. | `bool` | `false` | no |
 | <a name="input_enable_eip"></a> [enable\_eip](#input\_enable\_eip) | Enable the assignment of an EIP to the gitlab runner instance | `bool` | `false` | no |
@@ -381,7 +381,7 @@ terraform destroy
 | <a name="input_runner_instance_enable_monitoring"></a> [runner\_instance\_enable\_monitoring](#input\_runner\_instance\_enable\_monitoring) | Enable the GitLab runner instance to have detailed monitoring. | `bool` | `true` | no |
 | <a name="input_runner_instance_metadata_options_http_endpoint"></a> [runner\_instance\_metadata\_options\_http\_endpoint](#input\_runner\_instance\_metadata\_options\_http\_endpoint) | Enable the Gitlab runner agent instance metadata service. The allowed values are enabled, disabled. | `string` | `"enabled"` | no |
 | <a name="input_runner_instance_metadata_options_http_tokens"></a> [runner\_instance\_metadata\_options\_http\_tokens](#input\_runner\_instance\_metadata\_options\_http\_tokens) | Set if Gitlab runner agent instance metadata service session tokens are required. The allowed values are optional, required. | `string` | `"optional"` | no |
-| <a name="input_runner_instance_spot_price"></a> [runner\_instance\_spot\_price](#input\_runner\_instance\_spot\_price) | By setting a spot price bid price the runner agent will be created via a spot request. Be aware that spot instances can be stopped by AWS. | `string` | `null` | no |
+| <a name="input_runner_instance_spot_price"></a> [runner\_instance\_spot\_price](#input\_runner\_instance\_spot\_price) | By setting a spot price bid price the runner agent will be created via a spot request. Be aware that spot instances can be stopped by AWS. Choose "on-demand-price" to pay up to the current on demand price for the instance type chosen. | `string` | `null` | no |
 | <a name="input_runner_root_block_device"></a> [runner\_root\_block\_device](#input\_runner\_root\_block\_device) | The EC2 instance root block device configuration. Takes the following keys: `device_name`, `delete_on_termination`, `volume_type`, `volume_size`, `encrypted`, `iops`, `throughput`, `kms_key_id` | `map(string)` | `{}` | no |
 | <a name="input_runner_tags"></a> [runner\_tags](#input\_runner\_tags) | Map of tags that will be added to runner EC2 instances. | `map(string)` | `{}` | no |
 | <a name="input_runners_additional_volumes"></a> [runners\_additional\_volumes](#input\_runners\_additional\_volumes) | Additional volumes that will be used in the runner config.toml, e.g Docker socket | `list(any)` | `[]` | no |

examples/runner-default/main.tf

@@ -45,7 +45,7 @@ module "runner" {
   gitlab_runner_security_group_ids = [data.aws_security_group.default.id]
 
   docker_machine_download_url   = "https://gitlab-docker-machine-downloads.s3.amazonaws.com/v0.16.2-gitlab.2/docker-machine"
-  docker_machine_spot_price_bid = "0.06"
+  docker_machine_spot_price_bid = "on-demand-price"
 
   gitlab_runner_registration_config = {
     registration_token = var.registration_token

examples/runner-public/main.tf

@@ -44,7 +44,7 @@ module "runner" {
   subnet_ids_gitlab_runner = module.vpc.public_subnets
   subnet_id_runners        = element(module.vpc.public_subnets, 0)
 
-  docker_machine_spot_price_bid = "0.1"
+  docker_machine_spot_price_bid = "on-demand-price"
 
   runners_name             = var.runner_name
   runners_gitlab_url       = var.gitlab_url
@@ -91,7 +91,7 @@ module "runner2" {
   subnet_ids_gitlab_runner = module.vpc.public_subnets
   subnet_id_runners        = element(module.vpc.public_subnets, 0)
 
-  docker_machine_spot_price_bid = "0.1"
+  docker_machine_spot_price_bid = "on-demand-price"
 
   runners_name       = var.runner_name
   runners_gitlab_url = var.gitlab_url

main.tf

@@ -97,7 +97,7 @@ locals {
       runners_subnet_id           = var.subnet_id_runners
       runners_aws_zone            = data.aws_availability_zone.runners.name_suffix
       runners_instance_type       = var.docker_machine_instance_type
-      runners_spot_price_bid      = var.docker_machine_spot_price_bid
+      runners_spot_price_bid      = var.docker_machine_spot_price_bid == "on-demand-price" ? "" : var.docker_machine_spot_price_bid
       runners_ami                 = data.aws_ami.docker-machine.id
       runners_security_group_name = aws_security_group.docker_machine.name
       runners_monitoring          = var.runners_monitoring
@@ -249,7 +249,7 @@ resource "aws_launch_template" "gitlab_runner_instance" {
     content {
       market_type = instance_market_options.value
       spot_options {
-        max_price = var.runner_instance_spot_price
+        max_price = var.runner_instance_spot_price == "on-demand-price" ? "" : var.runner_instance_spot_price
       }
     }
   }

variables.tf

@@ -66,7 +66,7 @@ variable "runner_instance_enable_monitoring" {
 }
 
 variable "runner_instance_spot_price" {
-  description = "By setting a spot price bid price the runner agent will be created via a spot request. Be aware that spot instances can be stopped by AWS."
+  description = "By setting a spot price bid price the runner agent will be created via a spot request. Be aware that spot instances can be stopped by AWS. Choose \"on-demand-price\" to pay up to the current on demand price for the instance type chosen."
   type        = string
   default     = null
 }
@@ -96,9 +96,9 @@ variable "docker_machine_instance_type" {
 }
 
 variable "docker_machine_spot_price_bid" {
-  description = "Spot price bid."
+  description = "Spot price bid. The maximum price willing to pay. By default the price is limited by the current on demand price for the instance type chosen."
   type        = string
-  default     = "0.06"
+  default     = "on-demand-price"
 }
 
 variable "docker_machine_download_url" {