Deadlock detection #16782

Hor911 · 2025-04-04T11:32:01Z

Changelog entry

...

Description for reviewers

...

github-actions · 2025-04-04T11:33:47Z

🟢 2025-04-04 11:33:47 UTC The validation of the Pull Request description is successful.

github-actions · 2025-04-04T11:35:31Z

⚪ 2025-04-04 11:35:30 UTC Pre-commit check linux-x86_64-relwithdebinfo for 0a18d50 has started.
⚪ 2025-04-04 11:35:45 UTC Artifacts will be uploaded here
⚪ 2025-04-04 11:39:01 UTC ya make is running...
🟡 2025-04-04 13:19:37 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
28897	26234	0	7	2540	116

⚪ 2025-04-04 13:22:00 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-04-04 13:30:35 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
172 (only retried tests)	69	0	2	0	101

⚪ 2025-04-04 13:30:44 UTC ya make is running... (failed tests rerun, try 3)
🔴 2025-04-04 13:38:06 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
142 (only retried tests)	41	0	2	0	99

🟢 2025-04-04 13:38:13 UTC Build successful.
🟢 2025-04-04 13:38:35 UTC ydbd size 2.2 GiB changed* by +36.2 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `c378eea`	merge: `0a18d50`	diff	diff %
ydbd size	2 351 269 256 Bytes	2 351 306 368 Bytes	+36.2 KiB	+0.002%
ydbd stripped size	491 735 744 Bytes	491 742 080 Bytes	+6.2 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-04-04T11:36:16Z

⚪ 2025-04-04 11:36:15 UTC Pre-commit check linux-x86_64-release-asan for 0a18d50 has started.
⚪ 2025-04-04 11:36:31 UTC Artifacts will be uploaded here
⚪ 2025-04-04 11:39:44 UTC ya make is running...
🟡 2025-04-04 13:36:55 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
14416	14246	0	100	33	37

⚪ 2025-04-04 13:38:10 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-04-04 13:51:56 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
224 (only retried tests)	130	0	39	22	33

⚪ 2025-04-04 13:52:05 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-04-04 14:04:21 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
126 (only retried tests)	53	0	35	6	32

🟢 2025-04-04 14:04:28 UTC Build successful.
🟢 2025-04-04 14:04:58 UTC ydbd size 3.8 GiB changed* by +75.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `ef19d60`	merge: `0a18d50`	diff	diff %
ydbd size	4 125 423 568 Bytes	4 125 500 328 Bytes	+75.0 KiB	+0.002%
ydbd stripped size	1 425 352 232 Bytes	1 425 378 856 Bytes	+26.0 KiB	+0.002%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

gridnevvvit · 2025-04-07T08:37:21Z

ydb/core/kqp/executer_actor/kqp_executer_stats.cpp

+
+ui64 ExportMaxStats(std::vector<ui64>& data) {
+
+    Y_DEBUG_ABORT_UNLESS((data.size() & 3) == 0);


AFL_ENSURE

в "дебажной верифайке" тут нет никакого смысла кроме желания проехаться.

Боюсь деградации по производительности. В теории все эти констрейнты (если никто ничего не пропустил в коде) соблюдаются "снаружи" с гораздо меньшей интенсивностью (resize и проверки выполняются 1 раз на обработку всей статистики а не для каждой метрики отдельно)

Надо бы замерить влияние

gridnevvvit · 2025-04-07T08:37:50Z

ydb/core/kqp/executer_actor/kqp_executer_stats.cpp

+}
+
+void TMaxStats::Set(ui32 index, ui64 value) {
+    Y_ASSERT(index < Values.size());


github-actions · 2025-04-07T15:56:44Z

⚪ 2025-04-07 15:56:43 UTC Pre-commit check linux-x86_64-relwithdebinfo for fbd8f5a has started.
⚪ 2025-04-07 15:56:58 UTC Artifacts will be uploaded here
⚪ 2025-04-07 16:00:07 UTC ya make is running...
🟡 2025-04-07 17:39:07 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
28894	26226	0	3	2556	109

⚪ 2025-04-07 17:41:42 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-04-07 17:49:33 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
158 (only retried tests)	57	0	0	0	101

🟢 2025-04-07 17:49:44 UTC Build successful.
🟢 2025-04-07 17:50:05 UTC ydbd size 2.2 GiB changed* by +78.9 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `b392a83`	merge: `fbd8f5a`	diff	diff %
ydbd size	2 355 526 624 Bytes	2 355 607 448 Bytes	+78.9 KiB	+0.003%
ydbd stripped size	492 514 656 Bytes	492 532 608 Bytes	+17.5 KiB	+0.004%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-04-07T15:58:00Z

⚪ 2025-04-07 15:58:00 UTC Pre-commit check linux-x86_64-release-asan for fbd8f5a has started.
⚪ 2025-04-07 15:58:16 UTC Artifacts will be uploaded here
⚪ 2025-04-07 16:01:32 UTC ya make is running...
🟡 2025-04-07 17:51:48 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
14395	14261	0	68	28	38

⚪ 2025-04-07 17:53:00 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-04-07 18:05:43 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
179 (only retried tests)	110	0	34	1	34

⚪ 2025-04-07 18:05:53 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-04-07 18:17:55 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
101 (only retried tests)	35	0	33	1	32

🟢 2025-04-07 18:18:02 UTC Build successful.
🟡 2025-04-07 18:18:31 UTC ydbd size 3.8 GiB changed* by +118.9 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `b392a83`	merge: `fbd8f5a`	diff	diff %
ydbd size	4 133 107 808 Bytes	4 133 229 592 Bytes	+118.9 KiB	+0.003%
ydbd stripped size	1 427 847 048 Bytes	1 427 882 184 Bytes	+34.3 KiB	+0.002%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

Hor911 added 3 commits March 30, 2025 00:05

Stats reporting refactoring

83500a1

Refactor TR stats + CurrentWaitInput/OutputTime

b6fd10e

Deadlock detection

f0f1aba

Hor911 requested review from a team as code owners April 4, 2025 11:32

github-actions bot added the new-feature label Apr 4, 2025

gridnevvvit reviewed Apr 7, 2025

View reviewed changes

Use complex StageId (TxId included)

9c8092d

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deadlock detection #16782

Deadlock detection #16782

Hor911 commented Apr 4, 2025

github-actions bot commented Apr 4, 2025

github-actions bot commented Apr 4, 2025 •

edited

Loading

github-actions bot commented Apr 4, 2025 •

edited

Loading

gridnevvvit Apr 7, 2025

Hor911 Apr 7, 2025

gridnevvvit Apr 7, 2025

github-actions bot commented Apr 7, 2025 •

edited

Loading

github-actions bot commented Apr 7, 2025 •

edited

Loading


		ui64 ExportMaxStats(std::vector<ui64>& data) {

		Y_DEBUG_ABORT_UNLESS((data.size() & 3) == 0);

Deadlock detection #16782

Are you sure you want to change the base?

Deadlock detection #16782

Conversation

Hor911 commented Apr 4, 2025

Changelog entry

Changelog category

Description for reviewers

github-actions bot commented Apr 4, 2025

github-actions bot commented Apr 4, 2025 • edited Loading

github-actions bot commented Apr 4, 2025 • edited Loading

gridnevvvit Apr 7, 2025

Choose a reason for hiding this comment

Hor911 Apr 7, 2025

Choose a reason for hiding this comment

gridnevvvit Apr 7, 2025

Choose a reason for hiding this comment

github-actions bot commented Apr 7, 2025 • edited Loading

github-actions bot commented Apr 7, 2025 • edited Loading

github-actions bot commented Apr 4, 2025 •

edited

Loading

github-actions bot commented Apr 4, 2025 •

edited

Loading

github-actions bot commented Apr 7, 2025 •

edited

Loading

github-actions bot commented Apr 7, 2025 •

edited

Loading