YQ-3570 added s3 wildcards validations #8017

GrigoriyPA · 2024-08-19T15:42:07Z

Changelog entry

Added s3 wildcards validations

Changelog category

Improvement

Additional information

New validations:

Added wildcards validations in s3 bindings
Added wildcards validations in s3 io discovery

github-actions · 2024-08-19T15:43:43Z

⚪ 2024-08-19 15:43:43 UTC Pre-commit check for 66ee859 has started.
⚪ 2024-08-19 15:46:19 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-19 15:52:16 UTC Build successful.

github-actions · 2024-08-19T15:43:46Z

⚪ 2024-08-19 15:43:46 UTC Pre-commit check for 66ee859 has started.
⚪ 2024-08-19 15:46:21 UTC Check linux-x86_64-release-asan is running...
⚫ 2024-08-19 16:08:35 UTC Check cancelled

github-actions · 2024-08-19T15:43:46Z

⚪ 2024-08-19 15:43:46 UTC Pre-commit check for 66ee859 has started.
⚪ 2024-08-19 15:46:22 UTC Check linux-x86_64-relwithdebinfo is running...
⚫ 2024-08-19 16:08:35 UTC Check cancelled

github-actions · 2024-08-19T16:11:43Z

⚪ 2024-08-19 16:11:43 UTC Pre-commit check for a4ba584 has started.
⚪ 2024-08-19 16:15:32 UTC Check linux-x86_64-release-clang14 is running...
⚫ 2024-08-19 16:24:10 UTC Check cancelled

github-actions · 2024-08-19T16:11:47Z

⚪ 2024-08-19 16:11:47 UTC Pre-commit check for a4ba584 has started.
⚪ 2024-08-19 16:14:39 UTC Check linux-x86_64-relwithdebinfo is running...
⚫ 2024-08-19 16:24:15 UTC Check cancelled

github-actions · 2024-08-19T16:13:53Z

⚪ 2024-08-19 16:13:53 UTC Pre-commit check for a4ba584 has started.
⚪ 2024-08-19 16:16:41 UTC Check linux-x86_64-release-asan is running...
⚫ 2024-08-19 16:24:14 UTC Check cancelled

github-actions · 2024-08-19T16:25:37Z

⚪ 2024-08-19 16:25:37 UTC Pre-commit check for d5bcd8a has started.
⚪ 2024-08-19 16:28:30 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-19 16:33:34 UTC Build successful.

github-actions · 2024-08-19T16:25:38Z

⚪ 2024-08-19 16:25:37 UTC Pre-commit check for d5bcd8a has started.
⚪ 2024-08-19 16:28:37 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-19 18:54:28 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
40261	34810	0	21	5397	33

⚪ 2024-08-19 18:59:27 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟡 2024-08-19 19:10:58 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
150 (only retried tests)	88	0	12	23	27

⚪ 2024-08-19 19:11:09 UTC Failed tests rerun (try 3) linux-x86_64-relwithdebinfo is running...
🔴 2024-08-19 19:21:59 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
77 (only retried tests)	39	0	11	0	27

🟢 2024-08-19 19:22:07 UTC Build successful.
🟢 2024-08-19 19:22:42 UTC ydbd size 8.1 GiB changed* by +98.1 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `9b567af`	merge: `d5bcd8a`	diff	diff %
ydbd size	8 743 150 728 Bytes	8 743 251 200 Bytes	+98.1 KiB	+0.001%
ydbd stripped size	475 702 312 Bytes	475 723 560 Bytes	+20.8 KiB	+0.004%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-08-19T16:25:38Z

⚪ 2024-08-19 16:25:38 UTC Pre-commit check for d5bcd8a has started.
⚪ 2024-08-19 16:28:32 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-19 18:45:46 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
11399	11316	0	26	25	32

🟢 2024-08-19 18:46:44 UTC Build successful.
🟡 2024-08-19 18:47:16 UTC ydbd size 5.5 GiB changed* by +144.1 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `9b567af`	merge: `d5bcd8a`	diff	diff %
ydbd size	5 883 563 800 Bytes	5 883 711 360 Bytes	+144.1 KiB	+0.003%
ydbd stripped size	1 477 973 904 Bytes	1 478 007 600 Bytes	+32.9 KiB	+0.002%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-08-20T04:38:28Z

⚪ 2024-08-20 04:38:27 UTC Pre-commit check for ef4920f has started.
⚪ 2024-08-20 04:41:17 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-20 04:52:55 UTC Build successful.

github-actions · 2024-08-20T04:38:32Z

⚪ 2024-08-20 04:38:32 UTC Pre-commit check for ef4920f has started.
⚪ 2024-08-20 04:41:24 UTC Check linux-x86_64-relwithdebinfo is running...
⚫ 2024-08-20 04:56:59 UTC Check cancelled

github-actions · 2024-08-20T04:39:07Z

⚪ 2024-08-20 04:39:06 UTC Pre-commit check for ef4920f has started.
⚪ 2024-08-20 04:42:05 UTC Check linux-x86_64-release-asan is running...
⚫ 2024-08-20 04:56:59 UTC Check cancelled

github-actions · 2024-08-20T05:00:15Z

⚪ 2024-08-20 05:00:15 UTC Pre-commit check for dffd0dc has started.
⚪ 2024-08-20 05:03:03 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-20 05:08:19 UTC Build successful.

github-actions · 2024-08-20T05:00:24Z

⚪ 2024-08-20 05:00:24 UTC Pre-commit check for dffd0dc has started.
⚪ 2024-08-20 05:03:11 UTC Check linux-x86_64-release-asan is running...
⚫ 2024-08-20 06:27:47 UTC Check cancelled

github-actions · 2024-08-20T05:01:01Z

⚪ 2024-08-20 05:01:00 UTC Pre-commit check for dffd0dc has started.
⚪ 2024-08-20 05:03:58 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-20 06:25:24 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
40219	34850	0	13	5316	40

🟢 2024-08-20 06:26:21 UTC ydbd size 8.1 GiB changed* by +92.8 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `b0abac7`	merge: `dffd0dc`	diff	diff %
ydbd size	8 747 173 072 Bytes	8 747 268 088 Bytes	+92.8 KiB	+0.001%
ydbd stripped size	475 745 960 Bytes	475 759 016 Bytes	+12.8 KiB	+0.003%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

ydb/library/yql/providers/s3/object_listers/yql_s3_path.cpp

ydb/library/yql/providers/s3/object_listers/yql_s3_path_ut.cpp

ydb/library/yql/providers/s3/object_listers/yql_s3_path.cpp

github-actions · 2024-08-23T07:24:52Z

⚪ 2024-08-23 07:24:52 UTC Pre-commit check for a7d2803 has started.
⚪ 2024-08-23 07:27:47 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-23 07:40:36 UTC Build successful.

github-actions · 2024-08-23T07:25:26Z

⚪ 2024-08-23 07:25:25 UTC Pre-commit check for a7d2803 has started.
⚪ 2024-08-23 07:28:47 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-23 08:41:47 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
40328	34983	0	6	5321	18

⚪ 2024-08-23 08:45:29 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟡 2024-08-23 08:52:36 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
27 (only retried tests)	17	0	1	0	9

⚪ 2024-08-23 08:52:44 UTC Failed tests rerun (try 3) linux-x86_64-relwithdebinfo is running...
🟢 2024-08-23 08:59:44 UTC Tests successful.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
15 (only retried tests)	7	0	0	0	8

🟢 2024-08-23 08:59:51 UTC Build successful.
🟢 2024-08-23 09:00:33 UTC ydbd size 8.2 GiB changed* by +91.1 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `eb1276a`	merge: `a7d2803`	diff	diff %
ydbd size	8 753 305 232 Bytes	8 753 398 496 Bytes	+91.1 KiB	+0.001%
ydbd stripped size	476 022 792 Bytes	476 031 784 Bytes	+8.8 KiB	+0.002%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-08-23T07:25:26Z

⚪ 2024-08-23 07:25:26 UTC Pre-commit check for a7d2803 has started.
⚪ 2024-08-23 07:28:17 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-23 09:09:50 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
11222	11151	0	29	27	15

🟢 2024-08-23 09:10:48 UTC Build successful.
🟡 2024-08-23 09:11:20 UTC ydbd size 5.5 GiB changed* by +147.0 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `eb1276a`	merge: `a7d2803`	diff	diff %
ydbd size	5 890 857 720 Bytes	5 891 008 288 Bytes	+147.0 KiB	+0.003%
ydbd stripped size	1 479 182 192 Bytes	1 479 211 536 Bytes	+28.7 KiB	+0.002%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-08-23T11:30:25Z

⚪ 2024-08-23 11:30:24 UTC Pre-commit check for 5afce65 has started.
⚪ 2024-08-23 11:33:12 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-23 12:40:01 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
40337	34995	0	3	5319	20

⚪ 2024-08-23 12:43:28 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟢 2024-08-23 12:52:03 UTC Tests successful.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
26 (only retried tests)	15	0	0	0	11

🟢 2024-08-23 12:52:10 UTC Build successful.
🟢 2024-08-23 12:52:50 UTC ydbd size 8.2 GiB changed* by +87.1 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `c8af2fe`	merge: `5afce65`	diff	diff %
ydbd size	8 760 882 824 Bytes	8 760 972 032 Bytes	+87.1 KiB	+0.001%
ydbd stripped size	478 029 896 Bytes	478 038 504 Bytes	+8.4 KiB	+0.002%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-08-23T11:30:48Z

⚪ 2024-08-23 11:30:47 UTC Pre-commit check for 5afce65 has started.
⚪ 2024-08-23 11:33:43 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-23 11:40:43 UTC Build successful.

github-actions · 2024-08-23T11:31:09Z

⚪ 2024-08-23 11:31:08 UTC Pre-commit check for 5afce65 has started.
⚪ 2024-08-23 11:34:01 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-23 13:10:42 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
11241	11125	0	49	40	27

🟢 2024-08-23 13:11:40 UTC Build successful.
🟡 2024-08-23 13:12:13 UTC ydbd size 5.5 GiB changed* by +145.8 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `c8af2fe`	merge: `5afce65`	diff	diff %
ydbd size	5 896 665 712 Bytes	5 896 815 000 Bytes	+145.8 KiB	+0.003%
ydbd stripped size	1 481 809 616 Bytes	1 481 844 784 Bytes	+34.3 KiB	+0.002%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

GrigoriyPA requested a review from a team as a code owner August 19, 2024 15:42

github-actions bot added the new-feature label Aug 19, 2024

GrigoriyPA requested a review from uzhastik August 19, 2024 15:42

github-actions bot added improvement and removed new-feature labels Aug 19, 2024

github-actions bot added improvement and removed improvement labels Aug 19, 2024

GrigoriyPA added 5 commits August 20, 2024 04:25

Added wildcards validations

93ad20f

Fixed location passing

98cb9ca

Removed unused include

001b74b

Fixed unit test

fc7c07c

Fixed path pattern validation

178bb85

GrigoriyPA force-pushed the YQ-3570-fix-errror-invalid-regex branch from ce819a2 to 178bb85 Compare August 20, 2024 04:35

Fixed unit test

99e7132

GrigoriyPA added the rebase-and-check Rebase PR with the current base branch and check label Aug 20, 2024

uzhastik reviewed Aug 22, 2024

View reviewed changes

ydb/library/yql/providers/s3/object_listers/yql_s3_path.cpp Outdated Show resolved Hide resolved

uzhastik reviewed Aug 22, 2024

View reviewed changes

ydb/library/yql/providers/s3/object_listers/yql_s3_path_ut.cpp Outdated Show resolved Hide resolved

uzhastik reviewed Aug 22, 2024

View reviewed changes

ydb/library/yql/providers/s3/object_listers/yql_s3_path.cpp Outdated Show resolved Hide resolved

Fixed issues

d02fb48

GrigoriyPA requested a review from uzhastik August 23, 2024 07:24

Fixed issues 2

364c3e2

uzhastik approved these changes Aug 23, 2024

View reviewed changes

GrigoriyPA merged commit 4042963 into ydb-platform:main Aug 26, 2024
10 of 12 checks passed

GrigoriyPA deleted the YQ-3570-fix-errror-invalid-regex branch August 26, 2024 03:56

GrigoriyPA added a commit to GrigoriyPA/ydb that referenced this pull request Aug 26, 2024

YQ-3570 added s3 wildcards validations (ydb-platform#8017)

79d8dae

GrigoriyPA added a commit to GrigoriyPA/ydb that referenced this pull request Aug 26, 2024

YQ-3570 added s3 wildcards validations (ydb-platform#8017)

159c97b

This was referenced Aug 27, 2024

LOGBROKER 8891 add 4 json sqs requests #8328

Merged

LOGBROKER 8891 list dead letter source queues #8330

Merged

rekby pushed a commit to rekby-forks/ydb that referenced this pull request Aug 28, 2024

YQ-3570 added s3 wildcards validations (ydb-platform#8017)

4787227

niksaveliev mentioned this pull request Aug 29, 2024

Fix pqv1 max partitions #8485

Merged

This was referenced Aug 30, 2024

fix json map conversion in http proxy #8508

Merged

LOGBROKER 8891 fix not using optional fields in sqs json api properly #8523

Merged

stanislav-shchetinin pushed a commit to stanislav-shchetinin/ydb that referenced this pull request Aug 30, 2024

YQ-3570 added s3 wildcards validations (ydb-platform#8017)

82ea662

siarheivesialou mentioned this pull request Sep 4, 2024

fix memory leak in ymq proxy #8706

Merged

This was referenced Sep 9, 2024

Fix CLI max partitions default #8944

Merged

Fix reading without consumer from federation #9136

Merged

This was referenced Sep 17, 2024

doc hc api images source #9376

Merged

add viewer query script handlers #9423

Merged

jepett0 mentioned this pull request Feb 12, 2025

add external data source public API for description #14438

Merged

YQ-3570 added s3 wildcards validations #8017

YQ-3570 added s3 wildcards validations #8017

Uh oh!

Conversation

GrigoriyPA commented Aug 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changelog entry

Changelog category

Additional information

Uh oh!

github-actions bot commented Aug 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Aug 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

GrigoriyPA commented Aug 19, 2024 •

edited

Loading

github-actions bot commented Aug 19, 2024 •

edited

Loading

github-actions bot commented Aug 19, 2024 •

edited

Loading

github-actions bot commented Aug 19, 2024 •

edited

Loading

github-actions bot commented Aug 19, 2024 •

edited

Loading

github-actions bot commented Aug 19, 2024 •

edited

Loading

github-actions bot commented Aug 19, 2024 •

edited

Loading

github-actions bot commented Aug 19, 2024 •

edited

Loading

github-actions bot commented Aug 19, 2024 •

edited

Loading

github-actions bot commented Aug 19, 2024 •

edited

Loading

github-actions bot commented Aug 20, 2024 •

edited

Loading

github-actions bot commented Aug 20, 2024 •

edited

Loading

github-actions bot commented Aug 20, 2024 •

edited

Loading

github-actions bot commented Aug 20, 2024 •

edited

Loading

github-actions bot commented Aug 20, 2024 •

edited

Loading

github-actions bot commented Aug 20, 2024 •

edited

Loading

github-actions bot commented Aug 23, 2024 •

edited

Loading

github-actions bot commented Aug 23, 2024 •

edited

Loading

github-actions bot commented Aug 23, 2024 •

edited

Loading

github-actions bot commented Aug 23, 2024 •

edited

Loading

github-actions bot commented Aug 23, 2024 •

edited

Loading

github-actions bot commented Aug 23, 2024 •

edited

Loading