Skip to content

cancel-spot-instances.sh fails in Terraform container #493

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Leonidimus opened this issue May 20, 2022 · 6 comments · Fixed by #653 or #727
Closed

cancel-spot-instances.sh fails in Terraform container #493

Leonidimus opened this issue May 20, 2022 · 6 comments · Fixed by #653 or #727

Comments

@Leonidimus
Copy link

This module "npalm/gitlab-runner/aws" is meant to be run in a Terraform container, however script bin/cancel-spot-instances.sh doesn't work because it requires bash which doesn't exist in Terraform images. Terraform images only have sh. Besides, the script also requires jq and awscli which don't exist either in Terraform alpine based images.

I get this error:

│ Error: local-exec provisioner error
│ 
│   with null_resource.cancel_spot_requests,
│   on main.tf line 142, in resource "null_resource" "cancel_spot_requests":
│  142:   provisioner "local-exec" {
│ 
│ Error running command
│ './.terraform/modules/gitlab-runner/bin/cancel-spot-instances.sh
│ voice-runners': exit status 127. Output: env: can't execute 'bash': No such
│ file or directory
@npalm
Copy link
Collaborator

npalm commented May 20, 2022

Thanks for reporting. I see not another way to cancel spot instances when destroy than using a local executor. We could explore the following options:

  • Run in /bin/sh
  • Add an op-out flag for the script

@kayman-mk
Copy link
Collaborator

kayman-mk commented Jun 19, 2022

As far as I remember this null_resource has been removed some time ago. Could you please check the module version you are using? Any chance to update to 5.0.2?

EDIT: Ah, I see. This null_resource is introduced in the examples.

@github-actions
Copy link
Contributor

github-actions bot commented Jan 1, 2023

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 15 days.

@github-actions github-actions bot added the stale Issue/PR is stale and closed automatically label Jan 1, 2023
@kayman-mk
Copy link
Collaborator

To be tackled in #623 as we need an option to remove resources created during runtime.

@github-actions github-actions bot removed the stale Issue/PR is stale and closed automatically label Jan 2, 2023
@kayman-mk
Copy link
Collaborator

The script will be removed. A Lambda function will handle the cancellation of spot instances when the agent is shutdown.

@npalm
Copy link
Collaborator

npalm commented Jan 13, 2023

The script was created to support the examples (for testing purpose).

kayman-mk added a commit that referenced this issue Mar 2, 2023
## Description

Whenever a spot executor is created a spot request is created. Occasionally there might be open spot requests which have not been fulfilled. These spot requests have to be deleted as soon as the agent terminates as no executors are needed.

Challenge: Make sure that spot requests are deleted only which belong to our module. Unfortunately the GitLab Runner does not tag them. We use the following logic for determinination:

- state is open or active
- they are associated to a SSH key which name starts with `runner`
- they have the `var.environment`/`var.overrides['name_docker_machine']` somewhere in the name

Solves part of #623 
Closes #493
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants