Skip to content

Occasionally EC2s are left in a Stopped state #398

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
cmavromichalis opened this issue Oct 25, 2021 · 6 comments
Closed

Occasionally EC2s are left in a Stopped state #398

cmavromichalis opened this issue Oct 25, 2021 · 6 comments

Comments

@cmavromichalis
Copy link

I've noticed that as I update the GitLab runner versions occasionally there are EC2's in a Stopped state. I have to manually delete these when running the terraform as the terraform won't destroy EC2's in a Stopped state. I haven't been able to root cause why they are put in a stopped state or any kind of pattern for why. It seems to just happen randomly and infrequently.

My scenario is the GitLab CI docker-machine runner - one runner agent as described in the README. We update GitLab and GitLab runner's versions once a month. Sometime during the month of use the EC2 is started then put in a Stopped state.

Are there logs I could look at or provide to help root cause this?

@kayman-mk
Copy link
Collaborator

Haven't seen this before. My machines are always terminated. But let me check my configuration.

@kayman-mk
Copy link
Collaborator

I didn't find any special settings in my configuration. Do you have log files available?

@cmavromichalis
Copy link
Author

I have looked through the EC2 logs but havent found anything that jumped out at me. Are there specific logs that would help?

@npalm
Copy link
Collaborator

npalm commented Nov 10, 2021

We also didn't see this behaviour

@kayman-mk
Copy link
Collaborator

This will be indirectly solved by #392

@kayman-mk
Copy link
Collaborator

@cmavromichalis Shouldn't be an issue anymore and can be closed right?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants