Skip to content

Merging CBO into stable-24-3-8-analytics #8819

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 36 commits into from
Sep 6, 2024
Merged

Merging CBO into stable-24-3-8-analytics #8819

merged 36 commits into from
Sep 6, 2024

Conversation

pavelvelikhov
Copy link
Collaborator

Changelog entry

...

Changelog category

  • New feature

Additional information

...

pavelvelikhov and others added 27 commits September 5, 2024 15:04
@pavelvelikhov @pashandor789
Enabled CBO (ydb-platform#5780)
bd7d66a 
Co-authored-by: Pavel Ivanov <[email protected]>
@pavelvelikhov
Added table service flag for CBO (ydb-platform#6493)
b2ab2e5
@pavelvelikhov
Fixed bugs in CBO statistics calculation (ydb-platform#6537)
dc1eae0
@pavelvelikhov
Enabled constant folding by default (ydb-platform#6509)
37dfe2c
@pavelvelikhov
Fixed statistics calculation for index read (ydb-platform#6629)
3246373
@pashandor789 @pavelvelikhov
[CBO] Fix Selectivity (ydb-platform#6645)
bf51dd2
@pavelvelikhov
Cached overriden statistics for CBO (ydb-platform#6791)
9b7069e
@pavelvelikhov
Added parameter selectivity and boosted LookupJoin (ydb-platform#6874)
033c22d
@pashandor789 @pavelvelikhov
[CBO] Logging level fixed (ydb-platform#6911)
8551fdb
@pashandor789 @pavelvelikhov
[] Added query action check. (ydb-platform#7017)
b01b3b3
@pavelvelikhov
Fixed a problem in RBO in MapJoin implementation (ydb-platform#7044)
e509c5e
@pavelvelikhov
Reenabled grace join pragma (ydb-platform#7160)
7d434fb
@pavelvelikhov
Don't run CBO if statistics is not avaliable (ydb-platform#7089)
63105ce
@pavelvelikhov
Removed a rule that turned Shuffle into Map breaking some TPCDS queri…
6d7ab43 
…es (ydb-platform#6982)
@pavelvelikhov
Switched off GraceSelfJoinCore (ydb-platform#7308)
66a2d2d
@pavelvelikhov
Added Query Hints for the Optimizer (ydb-platform#7629)
0761181
@pavelvelikhov
Cleaned up code that removes aliases from attributes (ydb-platform#7745)
c0282dd
@pashandor789 @pavelvelikhov
[CBO] Make optimizer get only columns stats that are used in request (y…
95a0484 
…db-platform#7550)
@pashandor789 @pavelvelikhov
[CBO] Column shards tests added (ydb-platform#7972)
325affb
@pavelvelikhov
Fixed a subtle bug in removing aliases (ydb-platform#8003)
74698ad
@pavelvelikhov
Lowered threshold for MapJoin (ydb-platform#8147)
2a796d0
@pashandor789 @pavelvelikhov
[CBO] Join order hints added (ydb-platform#8106)
4eb4a9b
@pavelvelikhov
Added another cost based optimization level (ydb-platform#8348)
f91d492
@pavelvelikhov
Stopped pushing null as arguments to TCoIf in column shards (ydb-plat…
b621242 
…form#8536)
@pashandor789 @pavelvelikhov
[KQP] Make KqpOlapFilter be shown in a query plan (ydb-platform#8504)
140c037
@pashandor789 @pavelvelikhov
[CBO] Make hints process many trees. (ydb-platform#8554)
5b755d9
@pashandor789 @pavelvelikhov
[CBO] Join Order test improvement (ydb-platform#8602)
5cf466f
@pavelvelikhov pavelvelikhov requested a review from a team as a code owner September 5, 2024 15:39
@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 5, 2024 •
edited
Loading

⚪ 2024-09-05 16:58:30 UTC Pre-commit check for 414bb60 has started.
⚪ 2024-09-05 17:01:24 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-09-05 17:28:40 UTC Build failed. see the build logs.

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 5, 2024 •
edited
Loading

⚪ 2024-09-05 17:00:22 UTC Pre-commit check for 414bb60 has started.
⚪ 2024-09-05 17:03:49 UTC Build linux-x86_64-relwithdebinfo is running...
🔴 2024-09-05 17:37:09 UTC Build failed. see the build logs.
🔴 2024-09-05 17:37:11 UTC Tests run skipped.

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 6, 2024 •
edited
Loading

⚪ 2024-09-06 05:43:03 UTC Pre-commit check for 1c41581 has started.
⚪ 2024-09-06 05:45:54 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-09-06 06:18:28 UTC Build successful.

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 6, 2024 •
edited
Loading

⚪ 2024-09-06 05:43:51 UTC Pre-commit check for 1c41581 has started.
⚪ 2024-09-06 05:46:44 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-09-06 06:41:57 UTC Build successful.
⚪ 2024-09-06 06:42:13 UTC Tests are running...
🔴 2024-09-06 08:47:30 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
76582 63069 0 60 13423 30

🔴 2024-09-06 08:48:15 UTC ydbd size 8.2 GiB changed* by +4.8 MiB, which is >= 2.0 MiB vs stable-24-3-8-analytics: Alert

ydbd size dash stable-24-3-8-analytics: 1b9be9f merge: 1c41581 diff diff %
ydbd size 8 781 707 408 Bytes 8 786 759 488 Bytes +4.8 MiB +0.058%
ydbd stripped size 479 727 080 Bytes 479 891 752 Bytes +160.8 KiB +0.034%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 6, 2024 •
edited
Loading

⚪ 2024-09-06 05:44:27 UTC Pre-commit check for 1c41581 has started.
⚪ 2024-09-06 05:47:25 UTC Build linux-x86_64-release-asan is running...
🟢 2024-09-06 06:27:11 UTC Build successful.
⚪ 2024-09-06 06:27:24 UTC Tests are running...
🔴 2024-09-06 08:30:11 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
15493 15358 0 51 52 32

🔴 2024-09-06 08:30:52 UTC ydbd size 5.5 GiB changed* by +3.5 MiB, which is >= 2.0 MiB vs stable-24-3-8-analytics: Alert

ydbd size dash stable-24-3-8-analytics: 1b9be9f merge: 1c41581 diff diff %
ydbd size 5 948 410 576 Bytes 5 952 053 320 Bytes +3.5 MiB +0.061%
ydbd stripped size 1 490 567 712 Bytes 1 491 158 784 Bytes +577.2 KiB +0.040%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 6, 2024 •
edited
Loading

⚪ 2024-09-06 09:06:51 UTC Pre-commit check for b5d5d41 has started.
⚪ 2024-09-06 09:09:43 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-09-06 09:15:05 UTC Build successful.

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 6, 2024 •
edited
Loading

⚪ 2024-09-06 09:06:56 UTC Pre-commit check for b5d5d41 has started.
⚪ 2024-09-06 09:09:57 UTC Build linux-x86_64-relwithdebinfo is running...
⚫ 2024-09-06 09:31:16 UTC Check cancelled

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 6, 2024 •
edited
Loading

⚪ 2024-09-06 09:07:38 UTC Pre-commit check for b5d5d41 has started.
⚪ 2024-09-06 09:10:43 UTC Build linux-x86_64-release-asan is running...
⚫ 2024-09-06 09:31:22 UTC Check cancelled

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 6, 2024 •
edited
Loading

⚪ 2024-09-06 09:33:08 UTC Pre-commit check for 83ca09b has started.
⚪ 2024-09-06 09:36:10 UTC Build linux-x86_64-release-asan is running...
⚫ 2024-09-06 09:39:54 UTC Check cancelled

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 6, 2024 •
edited
Loading

⚪ 2024-09-06 09:33:31 UTC Pre-commit check for 83ca09b has started.
⚪ 2024-09-06 09:36:34 UTC Build linux-x86_64-relwithdebinfo is running...
⚫ 2024-09-06 09:39:25 UTC Check cancelled

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 6, 2024 •
edited
Loading

⚪ 2024-09-06 09:34:41 UTC Pre-commit check for 83ca09b has started.
⚪ 2024-09-06 09:37:34 UTC Build linux-x86_64-release-clang14 is running...
⚫ 2024-09-06 09:39:16 UTC Check cancelled

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 6, 2024 •
edited
Loading

⚪ 2024-09-06 09:42:04 UTC Pre-commit check for 2b2417f has started.
⚪ 2024-09-06 09:45:00 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-09-06 10:25:21 UTC Build successful.
⚪ 2024-09-06 10:25:35 UTC Tests are running...
🔴 2024-09-06 12:32:10 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
76579 63122 0 2 13423 32

🔴 2024-09-06 12:32:57 UTC ydbd size 8.2 GiB changed* by +4.8 MiB, which is >= 2.0 MiB vs stable-24-3-8-analytics: Alert

ydbd size dash stable-24-3-8-analytics: 1b9be9f merge: 2b2417f diff diff %
ydbd size 8 781 707 408 Bytes 8 786 759 488 Bytes +4.8 MiB +0.058%
ydbd stripped size 479 727 080 Bytes 479 891 752 Bytes +160.8 KiB +0.034%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 6, 2024 •
edited
Loading

⚪ 2024-09-06 09:43:10 UTC Pre-commit check for 2b2417f has started.
⚪ 2024-09-06 09:46:07 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-09-06 09:51:39 UTC Build successful.

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 6, 2024 •
edited
Loading

⚪ 2024-09-06 09:43:50 UTC Pre-commit check for 2b2417f has started.
⚪ 2024-09-06 09:46:47 UTC Build linux-x86_64-release-asan is running...
🟢 2024-09-06 10:11:19 UTC Build successful.
⚪ 2024-09-06 10:11:36 UTC Tests are running...
🔴 2024-09-06 12:14:52 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
15492 15371 0 29 56 36

🔴 2024-09-06 12:15:37 UTC ydbd size 5.5 GiB changed* by +3.5 MiB, which is >= 2.0 MiB vs stable-24-3-8-analytics: Alert

ydbd size dash stable-24-3-8-analytics: 1b9be9f merge: 2b2417f diff diff %
ydbd size 5 948 410 576 Bytes 5 952 053 320 Bytes +3.5 MiB +0.061%
ydbd stripped size 1 490 567 712 Bytes 1 491 158 784 Bytes +577.2 KiB +0.040%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@pavelvelikhov pavelvelikhov enabled auto-merge (squash) September 6, 2024 12:38
@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 6, 2024 •
edited
Loading

⚪ 2024-09-06 12:51:57 UTC Pre-commit check for 7cb68e6 has started.
⚪ 2024-09-06 12:55:24 UTC Build linux-x86_64-release-asan is running...
🟢 2024-09-06 13:19:56 UTC Build successful.
⚪ 2024-09-06 13:20:14 UTC Tests are running...
🔴 2024-09-06 15:28:29 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
15490 15377 0 28 52 33

🔴 2024-09-06 15:29:06 UTC ydbd size 5.5 GiB changed* by +3.5 MiB, which is >= 2.0 MiB vs stable-24-3-8-analytics: Alert

ydbd size dash stable-24-3-8-analytics: 1b9be9f merge: 7cb68e6 diff diff %
ydbd size 5 948 410 576 Bytes 5 952 053 320 Bytes +3.5 MiB +0.061%
ydbd stripped size 1 490 567 712 Bytes 1 491 158 784 Bytes +577.2 KiB +0.040%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 6, 2024 •
edited
Loading

⚪ 2024-09-06 13:02:48 UTC Pre-commit check for 7cb68e6 has started.
⚪ 2024-09-06 13:06:11 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-09-06 13:45:46 UTC Build successful.
⚪ 2024-09-06 13:46:00 UTC Tests are running...
🔴 2024-09-06 15:59:27 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
76580 63120 0 3 13425 32

🔴 2024-09-06 16:00:09 UTC ydbd size 8.2 GiB changed* by +4.8 MiB, which is >= 2.0 MiB vs stable-24-3-8-analytics: Alert

ydbd size dash stable-24-3-8-analytics: 1b9be9f merge: 7cb68e6 diff diff %
ydbd size 8 781 707 408 Bytes 8 786 759 488 Bytes +4.8 MiB +0.058%
ydbd stripped size 479 727 080 Bytes 479 891 752 Bytes +160.8 KiB +0.034%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 6, 2024 •
edited
Loading

⚪ 2024-09-06 13:06:02 UTC Pre-commit check for 7cb68e6 has started.
⚪ 2024-09-06 13:09:28 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-09-06 13:16:01 UTC Build successful.

@pavelvelikhov pavelvelikhov merged commit 8ffabfb into ydb-platform:stable-24-3-8-analytics Sep 6, 2024
6 of 11 checks passed
uzhastik pushed a commit to uzhastik/ydb that referenced this pull request Sep 17, 2024
@pavelvelikhov @pashandor789
@Tony-Romanov @uzhastik
Merging CBO into stable-24-3-8-analytics (ydb-platform#8819)
8b012a9 
Co-authored-by: Pavel Ivanov <[email protected]>
Co-authored-by: Tony-Romanov <[email protected]>
@StekPerepolnen StekPerepolnen mentioned this pull request Oct 29, 2024
Closed
@GrigoriyPA GrigoriyPA mentioned this pull request Feb 21, 2025
Closed
This was referenced Jun 3, 2025
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@fomichev3000 fomichev3000 fomichev3000 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@pavelvelikhov @fomichev3000 @pashandor789 @Tony-Romanov