Fix stats inside plan for full stats mode #8553

abyss7 · 2024-08-30T16:40:04Z

Changelog entry

Fill final stats only once at the end of execution.

Changelog category

Bugfix

github-actions · 2024-08-30T16:43:16Z

⚪ 2024-08-30 16:43:15 UTC Pre-commit check linux-x86_64-release-clang14 for 825782a has started.
⚪ 2024-08-30 16:46:23 UTC ya make is running...
🔴 2024-08-30 16:46:38 UTC Build failed. see the logs.

github-actions · 2024-08-30T16:43:19Z

⚪ 2024-08-30 16:43:18 UTC Pre-commit check linux-x86_64-release-asan for 825782a has started.
⚪ 2024-08-30 16:46:27 UTC ya make is running...
🔴 2024-08-30 18:50:07 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
12123	12026	0	23	54	20

🟢 2024-08-30 18:51:09 UTC Build successful.
🟡 2024-08-30 18:51:39 UTC ydbd size 5.5 GiB changed* by +670.7 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `857ad2e`	merge: `825782a`	diff	diff %
ydbd size	5 946 329 248 Bytes	5 947 016 032 Bytes	+670.7 KiB	+0.012%
ydbd stripped size	1 488 222 384 Bytes	1 488 336 784 Bytes	+111.7 KiB	+0.008%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-08-30T16:43:22Z

⚪ 2024-08-30 16:43:22 UTC Pre-commit check linux-x86_64-relwithdebinfo for 825782a has started.
⚪ 2024-08-30 16:46:22 UTC ya make is running...
🔴 2024-08-30 16:46:36 UTC Build failed. see the logs.

github-actions · 2024-08-30T20:41:46Z

⚪ 2024-08-30 20:41:46 UTC Pre-commit check linux-x86_64-relwithdebinfo for e47cae0 has started.
⚪ 2024-08-30 20:44:53 UTC ya make is running...
🟡 2024-08-30 22:29:09 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
74045	60310	0	14	13699	22

⚪ 2024-08-30 22:36:01 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-08-30 22:44:58 UTC Tests successful.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
37 (only retried tests)	22	0	0	1	14

🟢 2024-08-30 22:45:05 UTC Build successful.
🟡 2024-08-30 22:45:42 UTC ydbd size 8.3 GiB changed* by +816.8 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `be50511`	merge: `e47cae0`	diff	diff %
ydbd size	8 907 663 160 Bytes	8 908 499 600 Bytes	+816.8 KiB	+0.009%
ydbd stripped size	481 313 512 Bytes	481 336 776 Bytes	+22.7 KiB	+0.005%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-08-30T20:41:47Z

⚪ 2024-08-30 20:41:47 UTC Pre-commit check linux-x86_64-release-asan for e47cae0 has started.
⚪ 2024-08-30 20:44:55 UTC ya make is running...
🔴 2024-08-30 22:45:01 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
12123	12054	0	15	34	20

🟢 2024-08-30 22:46:03 UTC Build successful.
🟡 2024-08-30 22:46:35 UTC ydbd size 5.5 GiB changed* by +678.7 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `658689d`	merge: `e47cae0`	diff	diff %
ydbd size	5 947 779 568 Bytes	5 948 474 592 Bytes	+678.7 KiB	+0.012%
ydbd stripped size	1 488 525 488 Bytes	1 488 648 144 Bytes	+119.8 KiB	+0.008%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-08-30T20:44:00Z

⚪ 2024-08-30 20:43:59 UTC Pre-commit check linux-x86_64-release-clang14 for e47cae0 has started.
⚪ 2024-08-30 20:47:13 UTC ya make is running...
🟢 2024-08-30 21:37:52 UTC Build successful.

gridnevvvit · 2024-09-02T07:48:47Z

ydb/core/kqp/executer_actor/kqp_executer_impl.h

        this->Send(Target, ResponseEv.release());

+        Request.Transactions.crop(0);


сначала crop, а потом Send. в request transaction AFAIK есть unboxed values и аллокатор, поэтому сначала нужно его зачистить а потом делать send, чтобы избежать обработки двумя акторами

Если в transaction есть UV, то почему нет баиндинга аллокатора?

Если в transaction есть UV, то почему нет баиндинга аллокатора?

потому биндинг есть в деструкторе.

github-actions · 2024-09-02T07:56:29Z

⚪ 2024-09-02 07:56:29 UTC Pre-commit check linux-x86_64-release-clang14 for 278b024 has started.
⚪ 2024-09-02 07:59:48 UTC ya make is running...
🟢 2024-09-02 08:49:12 UTC Build successful.

github-actions · 2024-09-02T07:58:15Z

⚪ 2024-09-02 07:58:15 UTC Pre-commit check linux-x86_64-release-asan for 278b024 has started.
⚪ 2024-09-02 08:01:22 UTC ya make is running...
🔴 2024-09-02 10:07:17 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
12128	12048	0	23	36	21

🟢 2024-09-02 10:08:23 UTC Build successful.
🟡 2024-09-02 10:08:53 UTC ydbd size 5.5 GiB changed* by +678.7 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `4cbf358`	merge: `278b024`	diff	diff %
ydbd size	5 947 599 336 Bytes	5 948 294 344 Bytes	+678.7 KiB	+0.012%
ydbd stripped size	1 488 307 984 Bytes	1 488 430 576 Bytes	+119.7 KiB	+0.008%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-09-02T07:58:41Z

⚪ 2024-09-02 07:58:41 UTC Pre-commit check linux-x86_64-relwithdebinfo for 278b024 has started.
⚪ 2024-09-02 08:01:56 UTC ya make is running...
🟡 2024-09-02 09:51:43 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
74046	60317	0	10	13700	19

⚪ 2024-09-02 09:58:55 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-09-02 10:06:24 UTC Tests successful.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
32 (only retried tests)	17	0	0	1	14

🟢 2024-09-02 10:06:30 UTC Build successful.
🟡 2024-09-02 10:07:04 UTC ydbd size 8.3 GiB changed* by +814.1 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `4cbf358`	merge: `278b024`	diff	diff %
ydbd size	8 907 349 176 Bytes	8 908 182 848 Bytes	+814.1 KiB	+0.009%
ydbd stripped size	481 310 568 Bytes	481 329 672 Bytes	+18.7 KiB	+0.004%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

rvu1024 · 2024-09-02T07:59:34Z

ydb/library/yql/dq/actors/protos/dq_stats.proto

@@ -390,7 +390,8 @@ message TDqExecutionStats {
    uint64 FirstRowTimeMs = 13;    // first result row timestamp, milliseconds

    repeated TDqStageStats Stages = 14;
-    repeated string TxPlansWithStats = 15;
+    reserved 15; // repeated string TxPlansWithStats = 15;
+    map<uint32, string> TxPlansWithStats = 16;


Зачем добавлять ключ, если он нигде не используется? Я увидел толко это место, и там txId игнорируется (https://github.com/ydb-platform/ydb/pull/8553/files#diff-6cefa5a3f132f45ba6bf9ed88bd0d2fecbc1c27c9e09cbd8d6f91284ddce0601R2820)

Tx type = uint32 только в kqp. В dq service он имеет тип string. Зачем прибивать конкретный тип в общем proto?

Тогда верну, как было - не подумал посмотреть, где ещё используется и с каким типом ключей. С т.з. нашего кода могло получиться, что мы не знаем для каких транзакций и сколько раз уже добавлены планы. Сделаю более аккуратно с нашей стороны.

rvu1024 · 2024-09-02T08:00:55Z

ydb/core/kqp/executer_actor/kqp_executer_impl.h

        this->Send(Target, ResponseEv.release());

+        Request.Transactions.crop(0);


Если в transaction есть UV, то почему нет баиндинга аллокатора?

github-actions · 2024-09-02T11:13:21Z

⚪ 2024-09-02 11:13:20 UTC Pre-commit check linux-x86_64-release-asan for 8fa477a has started.
⚪ 2024-09-02 11:16:26 UTC ya make is running...
🔴 2024-09-02 13:27:06 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
12157	12076	0	24	36	21

🟢 2024-09-02 13:28:08 UTC Build successful.
🟢 2024-09-02 13:28:41 UTC ydbd size 5.5 GiB changed* by +76.6 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `ebfe3cd`	merge: `8fa477a`	diff	diff %
ydbd size	5 953 849 840 Bytes	5 953 928 272 Bytes	+76.6 KiB	+0.001%
ydbd stripped size	1 489 337 360 Bytes	1 489 355 920 Bytes	+18.1 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-09-02T11:24:51Z

⚪ 2024-09-02 11:24:51 UTC Pre-commit check linux-x86_64-relwithdebinfo for 8fa477a has started.
⚪ 2024-09-02 11:28:04 UTC ya make is running...
🟡 2024-09-02 13:18:07 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
74142	60378	0	12	13734	18

⚪ 2024-09-02 13:25:25 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-09-02 13:32:51 UTC Tests successful.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
31 (only retried tests)	18	0	0	0	13

🟢 2024-09-02 13:32:58 UTC Build successful.
🟢 2024-09-02 13:33:32 UTC ydbd size 8.3 GiB changed* by +38.2 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `ebfe3cd`	merge: `8fa477a`	diff	diff %
ydbd size	8 916 405 512 Bytes	8 916 444 648 Bytes	+38.2 KiB	+0.000%
ydbd stripped size	481 603 144 Bytes	481 605 320 Bytes	+2.1 KiB	+0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-09-02T11:25:01Z

⚪ 2024-09-02 11:25:01 UTC Pre-commit check linux-x86_64-release-clang14 for 8fa477a has started.
⚪ 2024-09-02 11:28:21 UTC ya make is running...
🟢 2024-09-02 12:23:07 UTC Build successful.

(cherry picked from commit 35c8dfe)

This reverts commit 35c8dfe.

Fix stats inside plan for full stats mode

03e1072

abyss7 requested review from a team as code owners August 30, 2024 16:40

github-actions bot added the bugfix label Aug 30, 2024

maximyurchuk mentioned this pull request Aug 30, 2024

Revert "Don't run large_serializable in PR" #8560

Merged

maximyurchuk added the rebase-and-check Rebase PR with the current base branch and check label Aug 30, 2024

github-actions bot removed the rebase-and-check Rebase PR with the current base branch and check label Aug 30, 2024

abyss7 requested review from Darych, gridnevvvit and rvu1024 August 31, 2024 12:31

gridnevvvit reviewed Sep 2, 2024

View reviewed changes

Crop transactions before sending

785644e

rvu1024 reviewed Sep 2, 2024

View reviewed changes

Revert back some changes regarding proto

7ef3027

github-actions bot added bugfix and removed bugfix labels Sep 2, 2024

abyss7 removed the request for review from Darych September 2, 2024 11:09

gridnevvvit self-requested a review September 2, 2024 21:40

gridnevvvit approved these changes Sep 2, 2024

View reviewed changes

rvu1024 approved these changes Sep 3, 2024

View reviewed changes

abyss7 merged commit 35c8dfe into ydb-platform:main Sep 3, 2024
11 of 13 checks passed

KirillKurdyukov pushed a commit to KirillKurdyukov/ydb that referenced this pull request Sep 4, 2024

Fix stats inside plan for full stats mode (ydb-platform#8553)

fbf9e6c

siarheivesialou mentioned this pull request Sep 4, 2024

fix memory leak in ymq proxy #8706

Merged

This was referenced Sep 4, 2024

YQ-3151 added issues for timeout and cancelled #8718

Merged

YQ kqprun added cancel after for script requests #8768

Merged

Fixed unit test ManyColumnShards #8777

Merged

niksaveliev mentioned this pull request Sep 9, 2024

Fix CLI max partitions default #8944

Merged

abyss7 added a commit to abyss7/ydb that referenced this pull request Sep 9, 2024

Fix stats inside plan for full stats mode (ydb-platform#8553)

5cd777d

(cherry picked from commit 35c8dfe)

niksaveliev mentioned this pull request Sep 12, 2024

Fix reading without consumer from federation #9136

Merged

StekPerepolnen mentioned this pull request Sep 17, 2024

doc hc api images source #9376

Merged

abyss7 added a commit to abyss7/ydb that referenced this pull request Dec 16, 2024

Revert "Fix stats inside plan for full stats mode (ydb-platform#8553)"

3519a81

This reverts commit 35c8dfe.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix stats inside plan for full stats mode #8553

Fix stats inside plan for full stats mode #8553

abyss7 commented Aug 30, 2024 •

edited

Loading

github-actions bot commented Aug 30, 2024 •

edited

Loading

github-actions bot commented Aug 30, 2024 •

edited

Loading

github-actions bot commented Aug 30, 2024 •

edited

Loading

github-actions bot commented Aug 30, 2024 •

edited

Loading

github-actions bot commented Aug 30, 2024 •

edited

Loading

github-actions bot commented Aug 30, 2024 •

edited

Loading

gridnevvvit Sep 2, 2024

rvu1024 Sep 2, 2024

gridnevvvit Sep 2, 2024

github-actions bot commented Sep 2, 2024 •

edited

Loading

github-actions bot commented Sep 2, 2024 •

edited

Loading

github-actions bot commented Sep 2, 2024 •

edited

Loading

rvu1024 Sep 2, 2024

abyss7 Sep 2, 2024

rvu1024 Sep 2, 2024

github-actions bot commented Sep 2, 2024 •

edited

Loading

github-actions bot commented Sep 2, 2024 •

edited

Loading

github-actions bot commented Sep 2, 2024 •

edited

Loading

		this->Send(Target, ResponseEv.release());

		Request.Transactions.crop(0);

Fix stats inside plan for full stats mode #8553

Fix stats inside plan for full stats mode #8553

Conversation

abyss7 commented Aug 30, 2024 • edited Loading

Changelog entry

Changelog category

github-actions bot commented Aug 30, 2024 • edited Loading

github-actions bot commented Aug 30, 2024 • edited Loading

github-actions bot commented Aug 30, 2024 • edited Loading

github-actions bot commented Aug 30, 2024 • edited Loading

github-actions bot commented Aug 30, 2024 • edited Loading

github-actions bot commented Aug 30, 2024 • edited Loading

gridnevvvit Sep 2, 2024

Choose a reason for hiding this comment

rvu1024 Sep 2, 2024

Choose a reason for hiding this comment

gridnevvvit Sep 2, 2024

Choose a reason for hiding this comment

github-actions bot commented Sep 2, 2024 • edited Loading

github-actions bot commented Sep 2, 2024 • edited Loading

github-actions bot commented Sep 2, 2024 • edited Loading

rvu1024 Sep 2, 2024

Choose a reason for hiding this comment

abyss7 Sep 2, 2024

Choose a reason for hiding this comment

rvu1024 Sep 2, 2024

Choose a reason for hiding this comment

github-actions bot commented Sep 2, 2024 • edited Loading

github-actions bot commented Sep 2, 2024 • edited Loading

github-actions bot commented Sep 2, 2024 • edited Loading

abyss7 commented Aug 30, 2024 •

edited

Loading

github-actions bot commented Aug 30, 2024 •

edited

Loading

github-actions bot commented Aug 30, 2024 •

edited

Loading

github-actions bot commented Aug 30, 2024 •

edited

Loading

github-actions bot commented Aug 30, 2024 •

edited

Loading

github-actions bot commented Aug 30, 2024 •

edited

Loading

github-actions bot commented Aug 30, 2024 •

edited

Loading

github-actions bot commented Sep 2, 2024 •

edited

Loading

github-actions bot commented Sep 2, 2024 •

edited

Loading

github-actions bot commented Sep 2, 2024 •

edited

Loading

github-actions bot commented Sep 2, 2024 •

edited

Loading

github-actions bot commented Sep 2, 2024 •

edited

Loading

github-actions bot commented Sep 2, 2024 •

edited

Loading