Skip to content

fix: correct the bugs of major version 7 (pre-release) #860

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

kayman-mk
Copy link
Collaborator

@kayman-mk kayman-mk commented Jun 1, 2023

Description

Corrects all bugs of the pre-release version 7 of the module before roll-out.

tmeijn and others added 6 commits June 1, 2023 20:08
## Description

Removes the earlier deprecated `runners_pull_policy` variable. Since were making a Major release I thought this one was nice to catch.

## Migrations required

YES. Replace the `runners_pull_policy` by `runners_pull_policies`.
This PR removes all variables which are marked as deprecated.

- `arn_format`
- `subnet_id_runners`
- `subnet_ids_gitlab_runner`
- `asg_terminate_lifecycle_hook_create`
- `asg_terminate_lifecycle_hook_heartbeat_timeout`
- `asg_terminate_lifecycle_lambda_memory_size`
- `asg_terminate_lifecycle_lambda_runtime`
- `asg_terminate_lifecycle_lambda_timeout`

Yes. Remove the variables from your configuration. This is done
automatically by the migration script.

None.

---------

Co-authored-by: Tyrone Meijn <[email protected]>

# Conflicts:
#	main.tf
…autoscaling options (cattle-ops#711)

## Description

Switches from hardcoded options to free-from scaling configuration. This
reduces the module complexity by allowing to get rid of a number of
variables while giving more control to the user to define their options
without us having to build support into it for.

Adds `idle_scale_factor` and `idle_count_min` Docker Machine options.
See
[documentation](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runnersmachine-section").

## Migrations required

YES - users will have to change the input name from
`runners_machine_autoscaling` to `runners_machine_autoscaling_options`.
No other changes should be needed, we just support _more_ options. A
migration script is available.

## Verification

No input given:

(end of rendered `config.toml`)


![image](https://user-images.githubusercontent.com/17970041/225890782-02fe4adc-4c6a-4237-9752-a64349464113.png)

Input:

```hcl

runners_machine_autoscaling_options = [
    {
      periods           = ["* * 9-17 * * mon-fri *", "* * 9-17 * * mon-fri *"]
      idle_count        = 50
      idle_count_min    = 10
      idle_time         = 3600
      timezone          = "UTC"
      idle_scale_factor = 1.5
    },
    {
      periods    = ["* * 9-17 * * mon-fri *", "* * 9-17 * * mon-fri *"]
      idle_count = 50
      idle_time  = 3600
      timezone   = "Europe/Amsterdam"
    }
  ]
```

Rendered `config.toml`:


![image](https://user-images.githubusercontent.com/17970041/225891085-add03ee8-3943-4c56-96a4-d1a8c252deb0.png)

Apply results:


![image](https://user-images.githubusercontent.com/17970041/225893020-a9850486-4aa6-4eb0-b996-558ec7bccfea.png)


Closes cattle-ops#556

---------

Co-authored-by: Matthias Kay <[email protected]>
## Description

Adds a new variable `runners_docker_options` which holds all values for
the `[runners.docker]` section and makes the single variables
- `runners_image`
- `runners_privileged`
- `runners_disable_cache`
- `runners_additional_volumes`
- `runners_shm_size`
- `runners_docker_runtime`
- `runners_helper_image`
- `runners_pull_policy`

obsolete.

## Migrations required

Yes, as the minimum Terraform version is 1.3.0 to support optional block
variables with defaults.

A migration script is provided to restructure the variables. See
`/migrations/migrate-to-7-0-0.sh`. Attention Mac users: The script will
not work out of the box as the `sed` implementation is different. Use a
Docker container with Alpine or Ubuntu to run the script.

```hcl
module "gitlab_ci_runner" {
  ...
  runners_docker_options {
    # set whatever is necessary
  }
```

## Verification

- [x] Use current configuration and ensure that the `config.toml`
remains unchanged
- [x] Set all new block variables and ensure that the `config.toml` is
valid (use `gitlab-runner verify)
- [x] Check that the default settings with Terraform 1.3 work as
expected
- [x] Verify all docker settings against the documentation to ensure
correct names

The runner starts in both cases and is available in Gitlab. No example
tested but used our active configuration at Hapag-Lloyd.

---------

Co-authored-by: Tyrone Meijn <[email protected]>
## Description

Groups variables into objects to
- reduce the number of variables (currently 118)
- to gain a better overview of all configuration settings

Creates new groups of variables:
- `runner_manager` in case it configures the "main" process which sets
the defaults for all runners
- `runner` in case it configures the runner created by the runner
manager
- `runner_worker` in case it configures the docker/docker+machine or
leave it as it is, if it is a global scope, e.g. common tags, the
environment, ...

## Migrations required

Yes and a script is provided to do that. It covers 98% of all migrations
(see migrations/migrate-to-7-0-0.sh)

## Verification

Please mention the examples you have verified.

---------

Co-authored-by: Tyrone Meijn <[email protected]>
@github-actions
Copy link
Contributor

github-actions bot commented Jun 1, 2023

Hey @kayman-mk! 👋

Thank you for your contribution to the project. Please refer to the contribution rules for a quick overview of the process.

Make sure that this PR clearly explains:

  • the problem being solved
  • the best way a reviewer and you can test your changes

With submitting this PR you confirm that you hold the rights of the code added and agree that it will published under this LICENSE.

The following ChatOps commands are supported:

  • /help: notifies a maintainer to help you out

Simply add a comment with the command in the first line. If you need to pass more information, separate it with a blank line from the command.

This message was generated automatically. You are welcome to improve it.

@kayman-mk kayman-mk changed the base branch from main to refactor-variables June 1, 2023 19:27
@kayman-mk
Copy link
Collaborator Author

kayman-mk commented Jun 1, 2023

First test with the Hapag-Lloyd test setup successfully done. Runner and Executors are up and running. Jobs were processed.

On-demand instances, no shared cache.

@kayman-mk kayman-mk self-assigned this Jun 1, 2023
@kayman-mk
Copy link
Collaborator Author

Second test with the live environment. Module conversion with script worked out of the box. Runner were up and running. All jobs were processed.

Spot instances, shared cache and fleet support.

@kayman-mk
Copy link
Collaborator Author

kayman-mk commented Jun 15, 2023

Another test with a new setup:

  • non-shared cache created by the module
  • on-demand instances
  • no fleets

Everything is ok

@kayman-mk
Copy link
Collaborator Author

Like before, but the Agent is running on an on-demand instance. Works also fine.

@kayman-mk
Copy link
Collaborator Author

Do not specify the runner_worker block: Works fine. I see default values in the config.toml

@kayman-mk
Copy link
Collaborator Author

Switched from docker+machine to docker. Wow, how easy that is. Everything up and running and all jobs were processed on the Runner.

@kayman-mk kayman-mk force-pushed the refactor-variables branch from 0f75011 to f8423df Compare June 15, 2023 19:41
@kayman-mk kayman-mk marked this pull request as ready for review June 15, 2023 19:52
@kayman-mk kayman-mk requested a review from npalm as a code owner June 15, 2023 19:52
@kayman-mk kayman-mk merged commit 81da9e6 into cattle-ops:refactor-variables Jun 15, 2023
@kayman-mk kayman-mk deleted the fix-bugs-version-7 branch June 15, 2023 20:08
kayman-mk added a commit that referenced this pull request Jul 6, 2023
## Description

Corrects all bugs of the pre-release version 7 of the module before
roll-out.

---------

Co-authored-by: Tyrone Meijn <[email protected]>
kayman-mk added a commit that referenced this pull request Jul 6, 2023
## Description

Corrects all bugs of the pre-release version 7 of the module before
roll-out.

---------

Co-authored-by: Tyrone Meijn <[email protected]>
kayman-mk added a commit that referenced this pull request Sep 7, 2023
## Description

Corrects all bugs of the pre-release version 7 of the module before
roll-out.

---------

Co-authored-by: Tyrone Meijn <[email protected]>
kayman-mk pushed a commit that referenced this pull request Sep 10, 2023
🤖 I have created a release *beep* *boop*
---


##
[7.0.0](6.5.2...7.0.0)
(2023-09-09)


### ⚠ BREAKING CHANGES

* group variables for better overview
([#810](#810))
* allow to set all docker options for the Executor
([#511](#511))
* add idle_count_min` and `idle_scale_factor` to Docker Machine
autoscaling options
([#711](#711))
* remove deprecated variables
([#738](#738))
* remove deprecated pull policy variable
([#710](#710))

### Features

* add idle_count_min` and `idle_scale_factor` to Docker Machine
autoscaling options
([#711](#711))
([1538d48](1538d48))
* allow to set all docker options for the Executor
([#511](#511))
([461561e](461561e))


### Bug Fixes

* add missing defaults
([#905](#905))
([eb44182](eb44182))
* correct the bugs of major version 7 (pre-release)
([#860](#860))
([f236b58](f236b58))
* remove deprecated pull policy variable
([#710](#710))
([8736ec7](8736ec7))


### Miscellaneous Chores

* remove deprecated variables
([#738](#738))
([676ed6a](676ed6a))


### Code Refactoring

* group variables for better overview
([#810](#810))
([c8a3b89](c8a3b89))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

---------

Signed-off-by: Niek Palm <[email protected]>
Co-authored-by: cattle-ops-releaser-2[bot] <134548870+cattle-ops-releaser-2[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants