Skip to content

DataShard: clean readsets in DataCleanup #15438

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Mar 7, 2025
Merged

DataShard: clean readsets in DataCleanup #15438

merged 1 commit into from
Mar 7, 2025

Conversation

lex007in
Copy link
Collaborator

@lex007in lex007in commented Mar 6, 2025

Changelog entry

...

Changelog category

  • Not for changelog (changelog entry is not required)

Description for reviewers

OutReadSets are cleaned only in the next transaction, so we need to force a OutReadSets cleanup in DataCleanup.

@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 6, 2025

🟢 2025-03-06 20:09:57 UTC The validation of the Pull Request description is successful.

@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 6, 2025 •
edited
Loading

⚪ 2025-03-06 20:12:35 UTC Pre-commit check linux-x86_64-release-asan for b9a39d4 has started.
⚪ 2025-03-06 20:12:49 UTC Artifacts will be uploaded here
⚪ 2025-03-06 20:15:34 UTC ya make is running...
⚫ 2025-03-06 20:40:11 UTC Check cancelled

@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 6, 2025 •
edited
Loading

⚪ 2025-03-06 20:12:39 UTC Pre-commit check linux-x86_64-relwithdebinfo for b9a39d4 has started.
⚪ 2025-03-06 20:12:54 UTC Artifacts will be uploaded here
⚪ 2025-03-06 20:15:47 UTC ya make is running...
⚫ 2025-03-06 20:40:14 UTC Check cancelled

@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 6, 2025 •
edited
Loading

⚪ 2025-03-06 20:43:59 UTC Pre-commit check linux-x86_64-relwithdebinfo for 8edae69 has started.
⚪ 2025-03-06 20:44:13 UTC Artifacts will be uploaded here
⚪ 2025-03-06 20:46:54 UTC ya make is running...
🟡 2025-03-06 21:33:38 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
19216 17870 0 1 1228 117

⚪ 2025-03-06 21:35:15 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-03-06 21:45:36 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
176 (only retried tests) 70 0 0 0 106

🟢 2025-03-06 21:45:43 UTC Build successful.
🟢 2025-03-06 21:46:00 UTC ydbd size 2.1 GiB changed* by +16 Bytes, which is < 100.0 KiB vs main: OK

ydbd size dash main: 98be31c merge: 8edae69 diff diff %
ydbd size 2 293 348 056 Bytes 2 293 348 072 Bytes +16 Bytes +0.000%
ydbd stripped size 480 451 520 Bytes 480 451 520 Bytes 0 Bytes 0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 6, 2025 •
edited
Loading

⚪ 2025-03-06 20:44:04 UTC Pre-commit check linux-x86_64-release-asan for 8edae69 has started.
⚪ 2025-03-06 20:44:17 UTC Artifacts will be uploaded here
⚪ 2025-03-06 20:47:01 UTC ya make is running...
🟡 2025-03-06 21:51:53 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11752 11569 0 137 13 33

⚪ 2025-03-06 21:52:56 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-03-06 22:05:19 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
235 (only retried tests) 200 0 0 5 30

🟢 2025-03-06 22:05:26 UTC Build successful.
🟢 2025-03-06 22:05:51 UTC ydbd size 3.7 GiB changed* by -16 Bytes, which is <= 0 Bytes vs main: OK

ydbd size dash main: 98be31c merge: 8edae69 diff diff %
ydbd size 3 994 125 872 Bytes 3 994 125 856 Bytes -16 Bytes -0.000%
ydbd stripped size 1 388 741 992 Bytes 1 388 741 992 Bytes 0 Bytes 0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@lex007in lex007in requested a review from snaury March 7, 2025 02:05
@lex007in lex007in marked this pull request as ready for review March 7, 2025 02:05
@lex007in lex007in merged commit af2a9a9 into ydb-platform:main Mar 7, 2025
12 checks passed
@lex007in lex007in self-assigned this Mar 7, 2025
@serbel324 serbel324 mentioned this pull request Mar 10, 2025
Merged
@GrigoriyPA GrigoriyPA mentioned this pull request Mar 11, 2025
Merged
@lex007in lex007in deleted the readsets branch March 11, 2025 18:30
@serbel324 serbel324 mentioned this pull request Mar 13, 2025
Merged
@alexnick88 alexnick88 mentioned this pull request Mar 19, 2025
Closed
blinkov pushed a commit that referenced this pull request Mar 21, 2025
@lex007in @blinkov
DataShard: clean readsets in DataCleanup (#15438)
4e86a3d
lex007in added a commit to lex007in/ydb that referenced this pull request Apr 1, 2025
@lex007in
DataShard: clean readsets in DataCleanup (ydb-platform#15438)
1ac4434
@lex007in lex007in mentioned this pull request Apr 1, 2025
Merged
lex007in added a commit that referenced this pull request Apr 4, 2025
@lex007in
Data cleanup fixes (#16627)
1d89df8 
Fixes for data cleanup edge cases:
- DataShard: clean readsets in DataCleanup (#15438)
- DataShard and SchemeShard: handle borrowed parts in data erasure (#15451)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@snaury snaury snaury approved these changes

Assignees

@lex007in lex007in

Labels
not-for-changelog
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@lex007in @snaury