Skip to content

New serialization formats in Knn UDF #4445

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 64 commits into from
Jun 3, 2024
Merged

New serialization formats in Knn UDF #4445

merged 64 commits into from
Jun 3, 2024

Conversation

azevaykin
Copy link
Collaborator

@azevaykin azevaykin commented May 12, 2024 •
edited
Loading

Changelog entry

Serialization functions

Previously ToBinaryString serialized float vector as is. It has been removed.

New serialization functions:

  • ToBinaryStringFloat - save float as is;
  • ToBinaryStringUint8 - save as Uint8, all the elements are casted to integers in [0;255] range;
  • ToBinaryStringInt8 - save as Int8, all the elements are casted to integers in [-128;127] range;
  • ToBinaryStringBit - save as one bit per element, non positive 0 save as 0, positive as 1.

All serialization functions return tagged string.

Deserialization function

Deserialization function FromBinaryString renamed to FloatFromBinaryString.
It uses tagged string parameters to check type on compile time.

Distance functions

All distance functions use tagged string parameters to check type on compile time.

New distance functions:

  • ManhattanDistance
  • EuclideanDistance

Changelog category

  • Experimental feature

Additional information

...

@azevaykin azevaykin requested a review from a team as a code owner May 12, 2024 13:35
@github-actions GitHub Actions
Copy link

github-actions bot commented May 12, 2024 •
edited
Loading

⚪ 2024-05-12 13:38:21 UTC Pre-commit check for 215d7b2 has started.
⚪ 2024-05-12 13:38:24 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-12 13:39:40 UTC Build successful.

@github-actions GitHub Actions
Copy link

github-actions bot commented May 12, 2024 •
edited
Loading

⚪ 2024-05-12 13:38:35 UTC Pre-commit check for 215d7b2 has started.
⚪ 2024-05-12 13:38:38 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-12 13:40:03 UTC Build successful.
⚪ 2024-05-12 13:41:31 UTC Tests are running...
🟢 2024-05-12 14:18:44 UTC Tests successful.

Test history

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
2885 2854 0 0 29 2

@github-actions GitHub Actions
Copy link

github-actions bot commented May 12, 2024 •
edited
Loading

⚪ 2024-05-12 13:38:36 UTC Pre-commit check for 215d7b2 has started.
⚪ 2024-05-12 13:38:38 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-12 13:40:00 UTC Build successful.
⚪ 2024-05-12 13:41:31 UTC Tests are running...
🟢 2024-05-12 14:26:41 UTC Tests successful.

Test history

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
2011 1987 0 0 20 4

@github-actions GitHub Actions
Copy link

github-actions bot commented May 12, 2024 •
edited
Loading

⚪ 2024-05-12 19:27:14 UTC Pre-commit check for 92162cb has started.
⚪ 2024-05-12 19:27:16 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-12 19:28:37 UTC Build successful.

@github-actions GitHub Actions
Copy link

github-actions bot commented May 12, 2024 •
edited
Loading

⚪ 2024-05-12 19:27:23 UTC Pre-commit check for 92162cb has started.
⚪ 2024-05-12 19:27:26 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-12 19:28:46 UTC Build successful.
⚪ 2024-05-12 19:30:15 UTC Tests are running...
🟢 2024-05-12 20:16:51 UTC Tests successful.

Test history

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
2013 1986 0 0 20 7

@github-actions GitHub Actions
Copy link

github-actions bot commented May 12, 2024 •
edited
Loading

⚪ 2024-05-12 19:27:46 UTC Pre-commit check for 92162cb has started.
⚪ 2024-05-12 19:27:48 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-12 19:29:08 UTC Build successful.
⚪ 2024-05-12 19:30:34 UTC Tests are running...
🔴 2024-05-12 20:07:03 UTC Some tests failed, follow the links below.

Test history

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
2885 2853 0 1 29 2

@github-actions GitHub Actions
Copy link

github-actions bot commented May 14, 2024 •
edited
Loading

⚪ 2024-05-14 19:03:56 UTC Pre-commit check for 9411e2f has started.
⚪ 2024-05-14 19:03:58 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-14 19:04:58 UTC Build successful.
⚪ 2024-05-14 19:06:18 UTC Tests are running...
🔴 2024-05-14 19:42:39 UTC Some tests failed, follow the links below.

Test history

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
2887 2854 0 3 29 1

@github-actions GitHub Actions
Copy link

github-actions bot commented May 14, 2024 •
edited
Loading

⚪ 2024-05-14 19:03:57 UTC Pre-commit check for 9411e2f has started.
⚪ 2024-05-14 19:03:58 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-14 19:05:05 UTC Build successful.
⚪ 2024-05-14 19:06:24 UTC Tests are running...
🔴 2024-05-14 19:53:17 UTC Some tests failed, follow the links below.

Test history

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
2012 1987 0 1 20 4

@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 3, 2024 •
edited
Loading

⚪ 2024-06-03 14:05:28 UTC Pre-commit check for 050d601 has started.
⚪ 2024-06-03 14:07:29 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-06-03 14:08:33 UTC Build successful.

@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 3, 2024 •
edited
Loading

⚪ 2024-06-03 14:05:56 UTC Pre-commit check for 050d601 has started.
⚪ 2024-06-03 14:07:51 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-06-03 14:08:50 UTC Build successful.
⚪ 2024-06-03 14:08:57 UTC Tests are running...
🔴 2024-06-03 14:44:16 UTC Some tests failed, follow the links below.

Test history

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
3236 3203 0 3 29 1

@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 3, 2024 •
edited
Loading

⚪ 2024-06-03 14:06:47 UTC Pre-commit check for 050d601 has started.
⚪ 2024-06-03 14:08:56 UTC Build linux-x86_64-release-asan is running...
🟢 2024-06-03 14:10:05 UTC Build successful.
⚪ 2024-06-03 14:10:20 UTC Tests are running...
🔴 2024-06-03 14:57:04 UTC Some tests failed, follow the links below.

Test history

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
2334 2309 0 2 21 2

@azevaykin azevaykin requested a review from vitstn June 3, 2024 14:10
@azevaykin azevaykin merged commit 03ecb11 into ydb-platform:main Jun 3, 2024
8 of 12 checks passed
@azevaykin azevaykin deleted the BitIndex branch June 3, 2024 16:32
azevaykin added a commit to azevaykin/ydb that referenced this pull request Jun 3, 2024
@azevaykin @MBkkt
New serialization formats in Knn UDF (ydb-platform#4445)
950bff4 
Co-authored-by: Valerii Mironov <[email protected]>
This was referenced Jun 6, 2024
Closed
Closed
Merged
Merged
@serbel324 serbel324 mentioned this pull request Jun 7, 2024
Merged
@GrigoriyPA GrigoriyPA mentioned this pull request Jun 10, 2024
Merged
@niksaveliev niksaveliev mentioned this pull request Jun 10, 2024
Merged
This was referenced Jun 11, 2024
Merged
Merged
@GrigoriyPA GrigoriyPA mentioned this pull request Jun 13, 2024
Merged
This was referenced Jun 14, 2024
Merged
Merged
This was referenced Jun 15, 2024
Merged
Merged
@niksaveliev niksaveliev mentioned this pull request Jun 17, 2024
Closed
@GrigoriyPA GrigoriyPA mentioned this pull request Jun 18, 2024
Merged
@StekPerepolnen StekPerepolnen mentioned this pull request Jun 19, 2024
Merged
This was referenced Jun 19, 2024
Merged
Merged
This was referenced Jun 23, 2024
Merged
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@MBkkt MBkkt MBkkt left review comments

@vitstn vitstn vitstn approved these changes

Assignees
No one assigned
Labels
experimental-feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@azevaykin @vitstn @MBkkt