Skip to content

Report errors during user cleanup #2013

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Dec 4, 2020
Merged

Report errors during user cleanup #2013

Changes from 3 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
a427017
GC will shuffle data before emoving it
Dec 1, 2020
e749338
trying to unstuck CI
Dec 1, 2020
b4188e1
reverting
Dec 1, 2020
a2e1b73
Merge branch 'master' into enhancement-gc-shuffles-before-removing
GitHK Dec 2, 2020
b522873
Merge branch 'master' into enhancement-gc-shuffles-before-removing
GitHK Dec 3, 2020
a214a6d
instead of shuffling the error is trapped and logged
Dec 3, 2020
31414d7
removing unused import
Dec 3, 2020
5beda2f
Merge branch 'master' into enhancement-gc-shuffles-before-removing
GitHK Dec 4, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion services/web/server/src/simcore_service_webserver/resource_manager/garbage_collector.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import asyncio
import logging
import random
from contextlib import suppress
from itertools import chain
from typing import Dict, List, Tuple
Expand Down Expand Up @@ -195,7 +196,11 @@ async def remove_disconnected_user_resources(
dead_key_resources,
)

for resource_name, resource_value in dead_key_resources.items():
# shuffle order of guest users to allow for random errors to not stop
# the removal of "not erroring" services
dead_key_resources_items = dead_key_resources.items()
random.shuffle(dead_key_resources_items)
for resource_name, resource_value in dead_key_resources_items:

# Releasing a resource consists of two steps
# - (1) release actual resource (e.g. stop service, close project, deallocate memory, etc)
Expand Down Expand Up @@ -287,6 +292,9 @@ async def remove_users_manually_marked_as_guests(
guest_users: List[Tuple[int, str]] = await get_guest_user_ids_and_names(app)
logger.info("GUEST user candidates to clean %s", guest_users)

# shuffle order of guest users to allow for random errors to not stop
# the removal of "not erroring" services
random.shuffle(guest_users)
for guest_user_id, guest_user_name in guest_users:
if guest_user_id in user_ids_to_ignore:
logger.info(
Expand Down