Skip to content

NC | lifecycle | add newer noncurrent versions rule #8949

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 21, 2025

Conversation

nadavMiz
Copy link
Contributor

@nadavMiz nadavMiz commented Apr 8, 2025

Describe the Problem

add lifecycle rule to delete noncurrent objects based on number of newer version. save progress in state struck to enable work in batches. change the state structure to enable working on both expire and noncurrent actions in the same rule

Explain the Changes

  1. implement function to filter based on newer noncurrent versions
  2. separate the state object to non-current and expire state to keep track of keyh_markers for both list_object and list-object-versions
  3. don't run action if is_finished is true. when both expire and noncurrent state is_finished flag is set to true, the rule is set to be finished
  4. move handling of the states and getting next batch to new functions: load_versions_list and load_object_list
  5. added tests
  6. since while using timeout we might not delete the batch objects before moving to the next batch causing the test to fail occasionally. reduce the timeout so no action will happen

Gaps

  1. move the accumulation of num_processed_objects to update_status()

Testing Instructions:

  1. run sudo npx jest test_nc_lifecycle_cli
  • Doc added/updated
  • Tests added

@nadavMiz nadavMiz requested a review from romayalon April 8, 2025 15:44
@nadavMiz nadavMiz force-pushed the lifecycle-noncurrent-versions branch from 80e457f to 4e8cce9 Compare April 8, 2025 16:23
@pull-request-size pull-request-size bot added size/L and removed size/XL labels Apr 8, 2025
@nadavMiz nadavMiz self-assigned this Apr 8, 2025
@nadavMiz nadavMiz force-pushed the lifecycle-noncurrent-versions branch from 4e8cce9 to d0812c8 Compare April 8, 2025 16:45
@nadavMiz nadavMiz force-pushed the lifecycle-noncurrent-versions branch 2 times, most recently from 91b22d3 to a7a2dd8 Compare April 10, 2025 06:49
@nadavMiz nadavMiz force-pushed the lifecycle-noncurrent-versions branch from a7a2dd8 to c4d2e56 Compare April 10, 2025 10:45
@nadavMiz nadavMiz force-pushed the lifecycle-noncurrent-versions branch 2 times, most recently from 42ee938 to 995cba6 Compare April 21, 2025 09:27
@nadavMiz nadavMiz requested a review from romayalon April 21, 2025 09:27
@nadavMiz nadavMiz force-pushed the lifecycle-noncurrent-versions branch from 995cba6 to 0cd81c4 Compare April 21, 2025 09:53
} else {
rule_state.is_finished = true;
}
const bucket_state = this.lifecycle_run_status.buckets_statuses[bucket_json.name].state;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not necessarily on this PR if you don't want, I think we can move the accumulation of num_processed_objects to update_status(), can you add it as a gap?

* @returns {Promise<Object[]>}
*/
async get_candidates_by_expiration_rule_posix(lifecycle_rule, bucket_json, object_sdk) {
const rule_state = this.lifecycle_run_status.buckets_statuses[bucket_json.name].rules_statuses[lifecycle_rule.id].state.expire;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems like my GPFS code is not aligned with the state.expire change,
let's talk on our meeting today what we need to do there

@nadavMiz nadavMiz force-pushed the lifecycle-noncurrent-versions branch from 0cd81c4 to 800a847 Compare April 21, 2025 12:10
@nadavMiz nadavMiz merged commit 1535c1e into noobaa:master Apr 21, 2025
11 checks passed
@romayalon romayalon mentioned this pull request May 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants