Fix balancing of large topics #7317

nshestakov · 2024-07-31T13:31:49Z

Changelog entry

...

Changelog category

Not for changelog (changelog entry is not required)

Additional information

Тут следующие исправления:

Доработка HtmlApp для упрощения диагностики
Не правильно инициализировалась партиция: была потеряна инициализаци PartitionConfig и PartitionGraph
Было потеряно отправка признака, что чтение дошло до конца партиции
Сделано более строгая обработка информации об коммитах в партицию, пришедшие со статистикой от партиции (важен порядок партиции, а он терялся)

github-actions · 2024-07-31T13:33:00Z

⚪ 2024-07-31 13:33:00 UTC Pre-commit check for 0cac21a has started.
⚫ 2024-07-31 13:35:22 UTC Check cancelled

github-actions · 2024-07-31T13:33:00Z

⚪ 2024-07-31 13:33:00 UTC Pre-commit check for 0cac21a has started.
⚫ 2024-07-31 13:34:47 UTC Check cancelled

github-actions · 2024-07-31T13:37:17Z

⚪ 2024-07-31 13:37:17 UTC Pre-commit check for b730df1 has started.
⚪ 2024-07-31 13:40:15 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-07-31 13:46:55 UTC Build successful.

github-actions · 2024-07-31T13:37:20Z

⚪ 2024-07-31 13:37:20 UTC Pre-commit check for b730df1 has started.
⚪ 2024-07-31 13:40:14 UTC Check linux-x86_64-relwithdebinfo is running...
⚫ 2024-07-31 14:29:26 UTC Check cancelled

github-actions · 2024-07-31T13:40:36Z

⚪ 2024-07-31 13:40:35 UTC Pre-commit check for b730df1 has started.
⚪ 2024-07-31 13:44:00 UTC Check linux-x86_64-release-asan is running...
⚫ 2024-07-31 14:29:22 UTC Check cancelled

github-actions · 2024-07-31T14:32:54Z

⚪ 2024-07-31 14:32:53 UTC Pre-commit check for 203ef1a has started.

github-actions · 2024-07-31T14:33:02Z

⚪ 2024-07-31 14:33:02 UTC Pre-commit check for 203ef1a has started.
⚪ 2024-07-31 14:36:14 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-07-31 16:11:24 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
13996	12652	0	3	1329	12

⚪ 2024-07-31 16:12:53 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟡 2024-07-31 16:21:11 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
15 (only retried tests)	5	0	1	1	8

⚪ 2024-07-31 16:22:21 UTC Failed tests rerun (try 3) linux-x86_64-relwithdebinfo is running...
🔴 2024-07-31 16:30:48 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
10 (only retried tests)	1	0	1	0	8

🟢 2024-07-31 16:32:06 UTC Build successful.
🟡 2024-07-31 16:32:44 UTC ydbd size 8.1 GiB changed* by +964.3 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `0c28fec`	merge: `203ef1a`	diff	diff %
ydbd size	8 679 207 296 Bytes	8 680 194 752 Bytes	+964.3 KiB	+0.011%
ydbd stripped size	471 941 440 Bytes	471 989 536 Bytes	+47.0 KiB	+0.010%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-07-31T14:33:45Z

⚪ 2024-07-31 14:33:45 UTC Pre-commit check for 203ef1a has started.
⚪ 2024-07-31 14:36:55 UTC Check linux-x86_64-release-asan is running...
🔴 2024-07-31 16:28:13 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
9662	9610	0	11	24	17

🟢 2024-07-31 16:29:17 UTC Build successful.
🟡 2024-07-31 16:29:44 UTC ydbd size 5.5 GiB changed* by +223.4 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `fd97335`	merge: `203ef1a`	diff	diff %
ydbd size	5 861 242 928 Bytes	5 861 471 672 Bytes	+223.4 KiB	+0.004%
ydbd stripped size	1 467 761 232 Bytes	1 467 806 864 Bytes	+44.6 KiB	+0.003%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-07-31T18:16:37Z

⚪ 2024-07-31 18:16:37 UTC Pre-commit check for 07fba25 has started.
⚪ 2024-07-31 18:19:36 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-07-31 20:10:04 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
14001	12653	0	3	1333	12

⚪ 2024-07-31 20:11:52 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟢 2024-07-31 20:19:36 UTC Tests successful.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
17 (only retried tests)	9	0	0	0	8

🟢 2024-07-31 20:20:49 UTC Build successful.
🟡 2024-07-31 20:21:19 UTC ydbd size 8.1 GiB changed* by +1.1 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `bd81a67`	merge: `07fba25`	diff	diff %
ydbd size	8 671 064 768 Bytes	8 672 175 280 Bytes	+1.1 MiB	+0.013%
ydbd stripped size	472 011 944 Bytes	472 059 624 Bytes	+46.6 KiB	+0.010%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-07-31T18:17:33Z

⚪ 2024-07-31 18:17:32 UTC Pre-commit check for 07fba25 has started.
⚪ 2024-07-31 18:20:29 UTC Check linux-x86_64-release-asan is running...
🔴 2024-07-31 20:40:26 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
9679	9636	0	7	22	14

🟢 2024-07-31 20:41:29 UTC Build successful.
🟡 2024-07-31 20:41:57 UTC ydbd size 5.4 GiB changed* by +225.4 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `0e835a2`	merge: `07fba25`	diff	diff %
ydbd size	5 841 185 032 Bytes	5 841 415 840 Bytes	+225.4 KiB	+0.004%
ydbd stripped size	1 463 889 520 Bytes	1 463 938 736 Bytes	+48.1 KiB	+0.003%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-07-31T18:17:53Z

⚪ 2024-07-31 18:17:53 UTC Pre-commit check for 07fba25 has started.
⚪ 2024-07-31 18:20:56 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-07-31 18:57:38 UTC Build successful.

nshestakov added 6 commits July 30, 2024 12:42

PQRB html app

1115eec

more html app improovements

0f451dc

Merge branch 'main' into httpApp

3cca39b

fix balancing topics with large partition number

21d2324

fix errors

15e6af0

possible npe

f217716

nshestakov requested a review from a team as a code owner July 31, 2024 13:31

nshestakov requested a review from niksaveliev July 31, 2024 13:31

github-actions bot added the not-for-changelog label Jul 31, 2024

revert

3cca94c

fix use after free

7259a2e

dimplify

1947255

github-actions bot added not-for-changelog and removed not-for-changelog labels Aug 1, 2024

Alek5andr-Kotov approved these changes Aug 1, 2024

View reviewed changes

nshestakov merged commit a8d3c99 into ydb-platform:main Aug 1, 2024
11 of 13 checks passed

nshestakov deleted the httpApp branch August 1, 2024 08:55

nshestakov added a commit to nshestakov/ydb that referenced this pull request Aug 1, 2024

Fix balancing of large topics (ydb-platform#7317)

25c9852

StekPerepolnen mentioned this pull request Aug 8, 2024

tenantinfo handler added DatabaseStorage #7554

Merged

GrigoriyPA mentioned this pull request Aug 8, 2024

YQ revert local grpc peer value #7555

Merged

nshestakov added a commit to nshestakov/ydb that referenced this pull request Aug 8, 2024

Fix balancing of large topics (ydb-platform#7317)

821ff9f

GrigoriyPA mentioned this pull request Aug 13, 2024

YQ kqprun pass actor system config #7693

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fix balancing of large topics #7317

Fix balancing of large topics #7317

Uh oh!

nshestakov commented Jul 31, 2024 •

edited

Loading

Uh oh!

github-actions bot commented Jul 31, 2024 •

edited

Loading

Uh oh!

github-actions bot commented Jul 31, 2024 •

edited

Loading

Uh oh!

github-actions bot commented Jul 31, 2024 •

edited

Loading

Uh oh!

github-actions bot commented Jul 31, 2024 •

edited

Loading

Uh oh!

github-actions bot commented Jul 31, 2024 •

edited

Loading

Uh oh!

github-actions bot commented Jul 31, 2024

Uh oh!

github-actions bot commented Jul 31, 2024 •

edited

Loading

Uh oh!

github-actions bot commented Jul 31, 2024 •

edited

Loading

Uh oh!

github-actions bot commented Jul 31, 2024 •

edited

Loading

Uh oh!

github-actions bot commented Jul 31, 2024 •

edited

Loading

Uh oh!

github-actions bot commented Jul 31, 2024 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Fix balancing of large topics #7317

Fix balancing of large topics #7317

Uh oh!

Conversation

nshestakov commented Jul 31, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changelog entry

Changelog category

Additional information

Uh oh!

github-actions bot commented Jul 31, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jul 31, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jul 31, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jul 31, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jul 31, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jul 31, 2024

Uh oh!

github-actions bot commented Jul 31, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jul 31, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jul 31, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jul 31, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jul 31, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nshestakov commented Jul 31, 2024 •

edited

Loading

github-actions bot commented Jul 31, 2024 •

edited

Loading

github-actions bot commented Jul 31, 2024 •

edited

Loading

github-actions bot commented Jul 31, 2024 •

edited

Loading

github-actions bot commented Jul 31, 2024 •

edited

Loading

github-actions bot commented Jul 31, 2024 •

edited

Loading

github-actions bot commented Jul 31, 2024 •

edited

Loading

github-actions bot commented Jul 31, 2024 •

edited

Loading

github-actions bot commented Jul 31, 2024 •

edited

Loading

github-actions bot commented Jul 31, 2024 •

edited

Loading

github-actions bot commented Jul 31, 2024 •

edited

Loading