Skip to content

Revert #420 for aws #453

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 24, 2022
Merged

Revert #420 for aws #453

merged 1 commit into from
Mar 24, 2022

Conversation

0x2b3bfa0
Copy link
Member

Because of reasons yet unknown to me, rclone recursive deletion code introduced with #420 is flaky:

func Delete(ctx context.Context, destination string) error {
destinationFileSystem, err := fs.NewFs(ctx, destination)
if err != nil {
return err
}
actions := []func(context.Context) error{
func(ctx context.Context) error {
return operations.Delete(ctx, destinationFileSystem)
},
func(ctx context.Context) error {
return operations.Rmdirs(ctx, destinationFileSystem, "", true)
},
}
for _, action := range actions {
if err := action(ctx); err != nil {
if !errors.Is(err, fs.ErrorDirNotFound) && !strings.Contains(err.Error(), "no such host") {
return common.NotFoundError
}
return err
}
}
return nil
}

This pull request restores the old cloud–specific code that goes through all the objects and deletes them as part of the destroy operation.

Closes #452.

@0x2b3bfa0 0x2b3bfa0 temporarily deployed to automatic March 23, 2022 13:20 Inactive
@0x2b3bfa0 0x2b3bfa0 self-assigned this Mar 23, 2022
@0x2b3bfa0 0x2b3bfa0 requested a review from a team March 23, 2022 13:20
@0x2b3bfa0 0x2b3bfa0 added bug Something isn't working cloud-aws Amazon Web Services cloud-common Applies to every cloud vendor labels Mar 23, 2022
@0x2b3bfa0 0x2b3bfa0 temporarily deployed to automatic March 23, 2022 13:20 Inactive
@0x2b3bfa0 0x2b3bfa0 temporarily deployed to automatic March 23, 2022 13:20 Inactive
@0x2b3bfa0 0x2b3bfa0 temporarily deployed to automatic March 23, 2022 13:20 Inactive
@0x2b3bfa0
Copy link
Member Author

Merging this opens a follow–up technical debt issue to fix rclone flakiness for the common Delete() operation and revert this.

Copy link
Contributor

@casperdcl casperdcl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

interesting... could you reproduce #452 consistently and does this PR fix it consistently? Trying to understand how flaky is "flaky"

@0x2b3bfa0 0x2b3bfa0 temporarily deployed to automatic March 23, 2022 13:39 Inactive
@0x2b3bfa0 0x2b3bfa0 temporarily deployed to automatic March 23, 2022 13:40 Inactive
@0x2b3bfa0 0x2b3bfa0 temporarily deployed to automatic March 23, 2022 13:40 Inactive
@0x2b3bfa0 0x2b3bfa0 temporarily deployed to automatic March 23, 2022 13:40 Inactive
@0x2b3bfa0 0x2b3bfa0 temporarily deployed to automatic March 23, 2022 13:40 Inactive
@casperdcl casperdcl added p0-critical Max priority (ASAP) and removed bug Something isn't working labels Mar 24, 2022
Copy link
Contributor

@casperdcl casperdcl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

repeated testing seems to show this PR (partially reverting) works, but yes please do open a follow-up issue :)

@casperdcl casperdcl merged commit 3fa92f3 into master Mar 24, 2022
@casperdcl casperdcl deleted the revert-420-aws branch March 24, 2022 19:25
@0x2b3bfa0
Copy link
Member Author

0x2b3bfa0 commented Mar 24, 2022

@0x2b3bfa0
Copy link
Member Author

I've opened an I2 (irking issue) to investigate why rclone bucket emptying finishes way after the function has returned.

@0x2b3bfa0 0x2b3bfa0 linked an issue Mar 30, 2022 that may be closed by this pull request
17 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cloud-aws Amazon Web Services cloud-common Applies to every cloud vendor p0-critical Max priority (ASAP)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

task feedback 2 destroy: The bucket you tried to delete is not empty
2 participants