Skip to content

Iterative B-Tree histograms builder #6047

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 29 commits into from
Jul 5, 2024

Conversation

kunga
Copy link
Member

@kunga kunga commented Jun 27, 2024

Changelog entry

...

Changelog category

  • Not for changelog (changelog entry is not required)

Additional information

...

Copy link

github-actions bot commented Jun 27, 2024

2024-06-27 17:36:05 UTC Pre-commit check for 992c202 has started.
2024-06-27 17:38:36 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-06-27 17:41:42 UTC Build failed. see the build logs.

Copy link

github-actions bot commented Jun 27, 2024

2024-06-27 17:38:56 UTC Pre-commit check for 992c202 has started.
2024-06-27 17:42:16 UTC Build linux-x86_64-release-asan is running...
🟢 2024-06-27 18:07:04 UTC Build successful.
2024-06-27 18:07:18 UTC Tests are running...
🔴 2024-06-27 20:10:50 UTC Test run completed, no test results found for commit 85564b3. Please check test log.
🟢 2024-06-27 20:11:22 UTC ydbd size 5.5 GiB changed* by -96 Bytes, which is <= 0 Bytes vs main: OK

ydbd size dash main: 521c1b0 merge: 992c202 diff diff %
ydbd size 5 916 526 256 Bytes 5 916 526 160 Bytes -96 Bytes -0.000%
ydbd stripped size 1 271 128 744 Bytes 1 271 128 680 Bytes -64 Bytes -0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation
2024-06-27 20:11:23 UTC Check cancelled

Copy link

github-actions bot commented Jun 27, 2024

2024-06-27 17:39:21 UTC Pre-commit check for 992c202 has started.
2024-06-27 17:42:25 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-06-27 18:20:43 UTC Build successful.
2024-06-27 18:21:05 UTC Tests are running...
🔴 2024-06-27 19:53:53 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
13435 12117 0 3 1294 21

🟢 2024-06-27 19:54:37 UTC ydbd size 8.4 GiB changed* by -15.9 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash main: 521c1b0 merge: 992c202 diff diff %
ydbd size 9 021 732 096 Bytes 9 021 715 832 Bytes -15.9 KiB -0.000%
ydbd stripped size 492 807 048 Bytes 492 807 048 Bytes 0 Bytes 0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jun 27, 2024

2024-06-27 20:15:04 UTC Pre-commit check for 6ec2387 has started.
2024-06-27 20:17:44 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-06-27 20:21:04 UTC Build failed. see the build logs.

Copy link

github-actions bot commented Jun 27, 2024

2024-06-27 20:15:06 UTC Pre-commit check for 6ec2387 has started.
2024-06-27 20:17:34 UTC Build linux-x86_64-release-asan is running...
🟢 2024-06-27 20:41:15 UTC Build successful.
2024-06-27 20:41:29 UTC Tests are running...
🔴 2024-06-27 22:40:27 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
9927 9341 0 90 320 176

🟢 2024-06-27 22:41:23 UTC ydbd size 5.5 GiB changed* by -96 Bytes, which is <= 0 Bytes vs main: OK

ydbd size dash main: b62f247 merge: 6ec2387 diff diff %
ydbd size 5 918 171 288 Bytes 5 918 171 192 Bytes -96 Bytes -0.000%
ydbd stripped size 1 271 363 592 Bytes 1 271 363 528 Bytes -64 Bytes -0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jun 27, 2024

2024-06-27 20:21:37 UTC Pre-commit check for 6ec2387 has started.
2024-06-27 20:24:43 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-06-27 21:02:34 UTC Build successful.
2024-06-27 21:02:52 UTC Tests are running...
🔴 2024-06-27 22:39:23 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
14212 12884 0 3 1303 22

🟢 2024-06-27 22:40:03 UTC ydbd size 8.4 GiB changed* by -15.9 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash main: b62f247 merge: 6ec2387 diff diff %
ydbd size 9 024 324 960 Bytes 9 024 308 688 Bytes -15.9 KiB -0.000%
ydbd stripped size 492 866 504 Bytes 492 866 504 Bytes 0 Bytes 0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jun 28, 2024

2024-06-28 06:38:26 UTC Pre-commit check for fdf96f9 has started.
2024-06-28 06:40:53 UTC Build linux-x86_64-release-asan is running...
🟢 2024-06-28 07:05:22 UTC Build successful.
2024-06-28 07:09:36 UTC Tests are running...
🔴 2024-06-28 09:14:19 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
9940 9354 0 83 325 178

🟡 2024-06-28 09:15:10 UTC ydbd size 5.5 GiB changed* by +321.2 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 4fde7f6 merge: fdf96f9 diff diff %
ydbd size 5 918 167 032 Bytes 5 918 495 976 Bytes +321.2 KiB +0.006%
ydbd stripped size 1 271 364 616 Bytes 1 271 465 288 Bytes +98.3 KiB +0.008%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jun 28, 2024

2024-06-28 06:38:27 UTC Pre-commit check for fdf96f9 has started.
2024-06-28 06:40:54 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-06-28 06:43:58 UTC Build failed. see the build logs.

Copy link

github-actions bot commented Jun 28, 2024

2024-06-28 06:44:15 UTC Pre-commit check for fdf96f9 has started.
2024-06-28 06:46:53 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-06-28 07:25:31 UTC Build successful.
2024-06-28 07:25:51 UTC Tests are running...
🔴 2024-06-28 09:00:24 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
14212 12884 0 7 1304 17

🟡 2024-06-28 09:01:09 UTC ydbd size 8.4 GiB changed* by +391.9 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: b62f247 merge: fdf96f9 diff diff %
ydbd size 9 024 324 960 Bytes 9 024 726 224 Bytes +391.9 KiB +0.004%
ydbd stripped size 492 866 504 Bytes 492 896 200 Bytes +29.0 KiB +0.006%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jun 28, 2024

2024-06-28 15:52:51 UTC Pre-commit check for 96ca95b has started.
2024-06-28 15:56:00 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-06-28 16:01:05 UTC Build successful.

Copy link

github-actions bot commented Jun 28, 2024

2024-06-28 16:01:40 UTC Pre-commit check for 96ca95b has started.
2024-06-28 16:04:45 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-06-28 16:44:06 UTC Build successful.
2024-06-28 16:44:20 UTC Tests are running...
🔴 2024-06-28 18:16:08 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
14210 12882 0 2 1312 14

🟡 2024-06-28 18:16:48 UTC ydbd size 8.4 GiB changed* by +414.4 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 2f7acc5 merge: 96ca95b diff diff %
ydbd size 8 998 139 688 Bytes 8 998 564 032 Bytes +414.4 KiB +0.005%
ydbd stripped size 492 075 400 Bytes 492 104 968 Bytes +28.9 KiB +0.006%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jun 28, 2024

2024-06-28 16:26:01 UTC Pre-commit check for 96ca95b has started.
2024-06-28 16:29:02 UTC Build linux-x86_64-release-asan is running...
🟢 2024-06-28 16:53:54 UTC Build successful.
2024-06-28 16:54:10 UTC Tests are running...
🔴 2024-06-28 19:00:02 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
9938 9353 0 88 326 171

🟡 2024-06-28 19:00:50 UTC ydbd size 5.5 GiB changed* by +322.3 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 2f7acc5 merge: 96ca95b diff diff %
ydbd size 5 901 830 528 Bytes 5 902 160 568 Bytes +322.3 KiB +0.006%
ydbd stripped size 1 268 873 480 Bytes 1 268 974 600 Bytes +98.8 KiB +0.008%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jun 28, 2024

2024-06-28 22:26:47 UTC Pre-commit check for 8278085 has started.
2024-06-28 22:29:17 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-06-28 22:34:54 UTC Build successful.

Copy link

github-actions bot commented Jun 28, 2024

2024-06-28 22:26:53 UTC Pre-commit check for 8278085 has started.
2024-06-28 22:29:28 UTC Build linux-x86_64-release-asan is running...
🟢 2024-06-28 22:54:41 UTC Build successful.
2024-06-28 22:54:58 UTC Tests are running...
🔴 2024-06-29 01:04:20 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
9946 9355 0 87 333 171

🟡 2024-06-29 01:05:35 UTC ydbd size 5.5 GiB changed* by +324.1 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 6cf4565 merge: 8278085 diff diff %
ydbd size 5 902 106 400 Bytes 5 902 438 296 Bytes +324.1 KiB +0.006%
ydbd stripped size 1 269 081 992 Bytes 1 269 183 496 Bytes +99.1 KiB +0.008%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jun 28, 2024

2024-06-28 22:36:27 UTC Pre-commit check for 8278085 has started.
2024-06-28 22:38:58 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-06-28 23:18:02 UTC Build successful.
2024-06-28 23:18:23 UTC Tests are running...
🔴 2024-06-29 00:55:13 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
14209 12888 0 4 1303 14

🟡 2024-06-29 00:55:46 UTC ydbd size 8.4 GiB changed* by +420.4 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 6cf4565 merge: 8278085 diff diff %
ydbd size 8 999 152 208 Bytes 8 999 582 728 Bytes +420.4 KiB +0.005%
ydbd stripped size 492 127 624 Bytes 492 161 288 Bytes +32.9 KiB +0.007%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@kunga kunga marked this pull request as ready for review July 1, 2024 08:44
@kunga kunga requested a review from snaury July 1, 2024 08:51
@kunga kunga self-assigned this Jul 1, 2024
@@ -1,525 +0,0 @@
#pragma once
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Перенес из хедера, можно не смотреть

Copy link

github-actions bot commented Jul 3, 2024

2024-07-03 15:09:27 UTC Pre-commit check for 7453fe0 has started.
2024-07-03 15:12:33 UTC Build linux-x86_64-release-asan is running...
🟢 2024-07-03 15:57:02 UTC Build successful.
2024-07-03 15:57:17 UTC Tests are running...
🔴 2024-07-03 17:51:20 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
10027 9453 0 80 322 172

🟢 2024-07-03 17:52:01 UTC ydbd size 5.3 GiB changed* by -97.6 MiB, which is <= 0 Bytes vs main: OK

ydbd size dash main: bd87563 merge: 7453fe0 diff diff %
ydbd size 5 814 908 560 Bytes 5 712 603 896 Bytes -97.6 MiB -1.759%
ydbd stripped size 1 247 931 016 Bytes 1 235 391 112 Bytes -12.0 MiB -1.005%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jul 3, 2024

2024-07-03 15:31:03 UTC Pre-commit check for 7453fe0 has started.
2024-07-03 15:33:50 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-07-03 16:31:40 UTC Build successful.
2024-07-03 16:31:53 UTC Tests are running...
🔴 2024-07-03 17:53:49 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
14302 12985 0 4 1303 10

🟢 2024-07-03 17:54:18 UTC ydbd size 8.2 GiB changed* by -144.0 MiB, which is <= 0 Bytes vs main: OK

ydbd size dash main: bd87563 merge: 7453fe0 diff diff %
ydbd size 8 926 807 928 Bytes 8 775 854 128 Bytes -144.0 MiB -1.691%
ydbd stripped size 486 814 568 Bytes 482 707 944 Bytes -3.9 MiB -0.844%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jul 3, 2024

2024-07-03 15:43:52 UTC Pre-commit check for 7453fe0 has started.
2024-07-03 15:47:06 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-07-03 16:26:38 UTC Build successful.

Copy link

github-actions bot commented Jul 3, 2024

2024-07-03 18:18:41 UTC Pre-commit check for fd0dcfa has started.
2024-07-03 18:21:18 UTC Build linux-x86_64-release-asan is running...
🟢 2024-07-03 18:50:32 UTC Build successful.
2024-07-03 18:50:47 UTC Tests are running...
🔴 2024-07-03 20:54:21 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
10124 9542 0 86 325 171

🟢 2024-07-03 20:55:12 UTC ydbd size 5.3 GiB changed* by -97.4 MiB, which is <= 0 Bytes vs main: OK

ydbd size dash main: bd87563 merge: fd0dcfa diff diff %
ydbd size 5 814 908 560 Bytes 5 712 815 784 Bytes -97.4 MiB -1.756%
ydbd stripped size 1 247 931 016 Bytes 1 235 447 336 Bytes -11.9 MiB -1.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jul 3, 2024

2024-07-03 18:44:50 UTC Pre-commit check for fd0dcfa has started.
2024-07-03 18:47:22 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-07-03 19:01:25 UTC Build successful.

Copy link

github-actions bot commented Jul 3, 2024

2024-07-03 18:49:25 UTC Pre-commit check for fd0dcfa has started.
2024-07-03 18:51:58 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-07-03 19:35:57 UTC Build successful.
2024-07-03 19:36:12 UTC Tests are running...
🔴 2024-07-03 21:31:26 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
38625 33390 0 4 5221 10

🟢 2024-07-03 21:32:16 UTC ydbd size 8.2 GiB changed* by -143.7 MiB, which is <= 0 Bytes vs main: OK

ydbd size dash main: 4313e61 merge: fd0dcfa diff diff %
ydbd size 8 926 807 512 Bytes 8 776 179 104 Bytes -143.7 MiB -1.687%
ydbd stripped size 486 814 568 Bytes 482 716 360 Bytes -3.9 MiB -0.842%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jul 4, 2024

2024-07-04 09:30:24 UTC Pre-commit check for 659b465 has started.
2024-07-04 09:32:55 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-07-04 09:40:57 UTC Build successful.

Copy link

github-actions bot commented Jul 4, 2024

2024-07-04 09:30:35 UTC Pre-commit check for 659b465 has started.
2024-07-04 09:33:04 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-07-04 10:12:21 UTC Build successful.
2024-07-04 10:12:36 UTC Tests are running...
🟢 2024-07-04 11:53:06 UTC Tests successful.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
38624 33390 0 0 5221 13

🟡 2024-07-04 11:53:49 UTC ydbd size 8.2 GiB changed* by +1.0 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 1aba0a7 merge: 659b465 diff diff %
ydbd size 8 775 503 896 Bytes 8 776 552 864 Bytes +1.0 MiB +0.012%
ydbd stripped size 482 710 312 Bytes 482 741 288 Bytes +30.2 KiB +0.006%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jul 4, 2024

2024-07-04 09:30:59 UTC Pre-commit check for 659b465 has started.
2024-07-04 09:33:29 UTC Build linux-x86_64-release-asan is running...
🟢 2024-07-04 09:59:08 UTC Build successful.
2024-07-04 09:59:24 UTC Tests are running...
🔴 2024-07-04 11:54:58 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
10125 9546 0 83 323 173

🟡 2024-07-04 11:55:38 UTC ydbd size 5.3 GiB changed* by +569.5 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 1aba0a7 merge: 659b465 diff diff %
ydbd size 5 712 433 376 Bytes 5 713 016 592 Bytes +569.5 KiB +0.010%
ydbd stripped size 1 235 350 184 Bytes 1 235 470 728 Bytes +117.7 KiB +0.010%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@kunga
Copy link
Member Author

kunga commented Jul 4, 2024

  • Перенёс код из хедеров
  • Написал много комментариев
  • Сделал чтобы SS учитывал значения в бакетах а не только индексы
    • Поставил границу для сплита что минимальный шард >= 25%, чтобы для 10 текущих бакетов мы могли брать 2 и 6 ключи
  • Добавил чтобы границы слайса использовались как граничные ключи вместо пустых
  • Исключил ноды которые начинаются в текущем ключе из значения ключа

@kunga kunga requested a review from snaury July 4, 2024 09:34

// category = -1 for Key = { }, IsBegin = true
// category = 0 for Key = {*}, IsBegin = *
// category = -1 for Key = { }, IsBegin = false
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Здесь должно быть +1?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants