Skip to content

✨ Enhance task cancellation #7565

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

giancarloromeo
Copy link
Contributor

@giancarloromeo giancarloromeo commented Apr 23, 2025

What do these changes do?

This PR revisits the concept of task cancellation (from user's point of view).

When a DELETE /tasks/{task_id} is sent, the system checks if the task is still not done and in that case it aborts it. It deletes ("untracks" it) in any case.

Cancel == Abort (if not done) + Delete

This fits also with the changes done in #7567.

Related issue/s

How to test

Dev-ops checklist

@giancarloromeo giancarloromeo added the a:storage issue related to storage service label Apr 23, 2025
@giancarloromeo giancarloromeo added this to the Pauwel Kwak milestone Apr 23, 2025
@giancarloromeo giancarloromeo self-assigned this Apr 23, 2025
Copy link

codecov bot commented Apr 23, 2025

Codecov Report

Attention: Patch coverage is 94.73684% with 1 line in your changes missing coverage. Please review.

Project coverage is 84.44%. Comparing base (8c0a500) to head (2c6d047).
Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7565      +/-   ##
==========================================
- Coverage   87.67%   84.44%   -3.24%     
==========================================
  Files        1774      758    -1016     
  Lines       68409    35266   -33143     
  Branches     1125      170     -955     
==========================================
- Hits        59979    29781   -30198     
+ Misses       8121     5427    -2694     
+ Partials      309       58     -251     
Flag Coverage Δ
integrationtests 65.12% <100.00%> (-0.06%) ⬇️
unittests 87.38% <94.73%> (+0.51%) ⬆️
Components Coverage Δ
api 76.84% <ø> (ø)
pkg_aws_library ∅ <ø> (∅)
pkg_dask_task_models_library ∅ <ø> (∅)
pkg_models_library ∅ <ø> (∅)
pkg_notifications_library ∅ <ø> (∅)
pkg_postgres_database ∅ <ø> (∅)
pkg_service_integration ∅ <ø> (∅)
pkg_service_library ∅ <ø> (∅)
pkg_settings_library ∅ <ø> (∅)
pkg_simcore_sdk 77.53% <ø> (-8.19%) ⬇️
agent ∅ <ø> (∅)
api_server ∅ <ø> (∅)
autoscaling ∅ <ø> (∅)
catalog ∅ <ø> (∅)
clusters_keeper ∅ <ø> (∅)
dask_sidecar ∅ <ø> (∅)
datcore_adapter ∅ <ø> (∅)
director ∅ <ø> (∅)
director_v2 78.32% <ø> (-13.04%) ⬇️
dynamic_scheduler ∅ <ø> (∅)
dynamic_sidecar 89.06% <ø> (-1.10%) ⬇️
efs_guardian ∅ <ø> (∅)
invitations ∅ <ø> (∅)
payments ∅ <ø> (∅)
resource_usage_tracker ∅ <ø> (∅)
storage 87.63% <94.44%> (-0.13%) ⬇️
webclient ∅ <ø> (∅)
webserver 86.04% <100.00%> (+0.01%) ⬆️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8c0a500...2c6d047. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

… github.com:giancarloromeo/osparc-simcore into is7563/clean-export-data-tasks-once-really-consumed
@giancarloromeo giancarloromeo marked this pull request as ready for review April 24, 2025 12:20
@giancarloromeo giancarloromeo requested a review from odeimaiz April 24, 2025 12:20
Copy link
Member

@odeimaiz odeimaiz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Contributor

@GitHK GitHK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe the tests should also reflect your change. I mean they should check that when cancelling and aborigine the task the task is removed from the system

Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR improves the task cancellation mechanism by renaming endpoints, functions, and tests from "abort" to "cancel" to better reflect their semantics.

  • Renames API endpoints and route names from “abort_async_job” to “cancel_async_job”.
  • Updates corresponding function names and test cases to use “cancel” terminology.
  • Revises internal client logic to ensure tasks are cancelled and untracked correctly.

Reviewed Changes

Copilot reviewed 9 out of 11 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
services/web/server/tests/unit/with_dbs/01/storage/test_storage.py Renamed the test function to match updated cancellation terminology.
services/web/server/src/simcore_service_webserver/tasks/_rest.py Updated route name and URL composition for task cancellation.
services/web/server/src/simcore_service_webserver/storage/_rest.py Changed endpoint reference from abort to cancel.
services/storage/tests/unit/test_modules_celery.py Updated test names and function calls for task cancellation.
services/storage/tests/unit/test_async_jobs.py Adjusted test for task cancellation and cleanup.
services/storage/src/simcore_service_storage/modules/celery/client.py Revised task cancellation logic and renamed internal store variable.
services/storage/src/simcore_service_storage/modules/celery/_task.py Added a call to forget the task result once cancellation is confirmed.
services/storage/src/simcore_service_storage/api/rpc/_async_jobs.py Updated RPC call to use cancel task.
api/specs/web-server/_long_running_tasks.py Renamed exposed function for task cancellation.
Files not reviewed (2)
  • .github/CODEOWNERS: Language not supported
  • services/web/server/src/simcore_service_webserver/api/v0/openapi.yaml: Language not supported
Comments suppressed due to low confidence (2)

services/web/server/src/simcore_service_webserver/tasks/_rest.py:90

  • Ensure that the endpoint renaming from 'abort_async_job' to 'cancel_async_job' is consistently reflected across all services and related documentation.
abort_href=f"{request.url.with_path(str(request.app.router['cancel_async_job'].url_for(task_id=str(job.job_id))))}"

services/storage/src/simcore_service_storage/modules/celery/client.py:72

  • [nitpick] Consider renaming '_abort_task' to '_cancel_task' to maintain naming consistency with the public 'cancel_task' method.
def _abort_task(self, task_id: TaskID) -> None:

@giancarloromeo giancarloromeo added the 🤖-automerge marks PR as ready to be merged for Mergify label Apr 24, 2025
Copy link
Contributor

@bisgaard-itis bisgaard-itis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you

@giancarloromeo giancarloromeo enabled auto-merge (squash) April 24, 2025 14:39
@giancarloromeo
Copy link
Contributor Author

@Mergifyio queue

Copy link
Contributor

mergify bot commented Apr 24, 2025

queue

🟠 Waiting for conditions to match

  • -closed [📌 queue requirement]
  • -conflict [📌 queue requirement]
  • -draft [📌 queue requirement]
  • any of: [📌 queue -> configuration change requirements]
    • -mergify-configuration-changed
    • check-success = Configuration changed
  • any of: [🔀 queue conditions]
    • all of: [📌 queue conditions of queue default]
      • #approved-reviews-by >= 2 [🛡 GitHub branch protection]
      • #approved-reviews-by>=2
      • #changes-requested-reviews-by = 0 [🛡 GitHub branch protection]
      • #changes-requested-reviews-by=0
      • #review-threads-unresolved = 0 [🛡 GitHub branch protection]
      • #review-threads-unresolved=0
      • -conflict
      • -draft
      • base=master
      • branch-protection-review-decision = APPROVED [🛡 GitHub branch protection]
      • label!=🤖-do-not-merge
      • label=🤖-automerge
      • any of: [🛡 GitHub branch protection]
        • check-skipped = deploy to dockerhub
        • check-neutral = deploy to dockerhub
        • check-success = deploy to dockerhub
      • any of: [🛡 GitHub branch protection]
        • check-success = system-tests
        • check-neutral = system-tests
        • check-skipped = system-tests
      • any of: [🛡 GitHub branch protection]
        • check-success = unit-tests
        • check-neutral = unit-tests
        • check-skipped = unit-tests
      • any of: [🛡 GitHub branch protection]
        • check-success = check OAS' are up to date
        • check-neutral = check OAS' are up to date
        • check-skipped = check OAS' are up to date
      • any of: [🛡 GitHub branch protection]
        • check-success = integration-tests
        • check-neutral = integration-tests
        • check-skipped = integration-tests
      • any of: [🛡 GitHub branch protection]
        • check-success = [build] docker images (excluding frontend) (3.11, ubuntu-24.04)
        • check-neutral = [build] docker images (excluding frontend) (3.11, ubuntu-24.04)
        • check-skipped = [build] docker images (excluding frontend) (3.11, ubuntu-24.04)

Copy link
Member

@sanderegg sanderegg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There seems to be quite a confusion between cancel and abort. Can you explain when you use one or the other please?
Now it's a whole mixture. Maybe choose one and keep it.

@giancarloromeo
Copy link
Contributor Author

There seems to be quite a confusion between cancel and abort. Can you explain when you use one or the other please?
Now it's a whole mixture. Maybe choose one and keep it.

We have both, see the description please.

Cancel == Abort (if not done) + Delete

Let's discuss offline tomorrow, if you still have doubts.

Copy link

@giancarloromeo giancarloromeo merged commit fb26261 into ITISFoundation:master Apr 24, 2025
93 of 95 checks passed
@giancarloromeo giancarloromeo deleted the is7563/clean-export-data-tasks-once-really-consumed branch April 24, 2025 18:48
@matusdrobuliak66 matusdrobuliak66 mentioned this pull request May 8, 2025
34 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🤖-automerge marks PR as ready to be merged for Mergify a:storage issue related to storage service
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Clean export_data tasks once their result is really consumed
5 participants