Skip to content

Some types problem in prefix vector index build #17274

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Tracked by #8967
kunga opened this issue Apr 16, 2025 · 1 comment
Closed
Tracked by #8967

Some types problem in prefix vector index build #17274

kunga opened this issue Apr 16, 2025 · 1 comment
Assignees

Comments

@kunga
Copy link
Member

kunga commented Apr 16, 2025

From #16813 (comment)

An attempt to build a prefix index

ALTER TABLE alice
ADD INDEX idx_vector
GLOBAL USING vector_kmeans_tree
ON (intent, embedding)
WITH (distance=cosine, vector_type="float", vector_dimension=1024, levels=1, clusters=200);
The attempt run gave other error. SchemeShard Schemeshard is restarting constantly.
2025-04-16T14:18:39.1744802319641209Z :BUILD_INDEX INFO: TIndexBuilder::TXTYPE_PROGRESS_INDEX_BUILD: TTxBuildProgress: Execute: 281474977495657 Done TBuildInfo{ IndexBuildId: 281474977495657, Uid: , DomainPathId: [OwnerId: 72075186224037897, LocalPathId: 1], TablePathId: [OwnerId: 72075186224037897, LocalPathId: 39], IndexType: EIndexTypeGlobalVectorKmeansTree, IndexName: idx_vector, IndexColumn: embedding, State: Done, IsCancellationRequested: 0, Issue: , SubscribersCount: 0, CreateSender: [0:0:0], AlterMainTableTxId: 0, AlterMainTableTxStatus: StatusSuccess, AlterMainTableTxDone: 0, LockTxId: 281474977520757, LockTxStatus: StatusAccepted, LockTxDone: 1, InitiateTxId: 281474977520758, InitiateTxStatus: StatusAccepted, InitiateTxDone: 1, SnapshotStepId: 0, ApplyTxId: 281474977783562, ApplyTxStatus: StatusAccepted, ApplyTxDone: 1, UnlockTxId: 281474977783563, UnlockTxStatus: StatusAccepted, UnlockTxDone: 1, ToUploadShards: 0, DoneShards: 0, Processed: { upload rows: 757527998, upload bytes: 2871356539834, read rows: 781311204, read bytes: 2993359438798 }, Billed: { upload rows: 0, upload bytes: 0, read rows: 0, read bytes: 0 }}
2025-04-16T14:18:39.1744802319641221Z :BUILD_INDEX INFO: TIndexBuilder::TXTYPE_PROGRESS_INDEX_BUILD: TTxBuildProgress: Execute: 281474977795672 Filling TBuildInfo{ IndexBuildId: 281474977795672, Uid: , DomainPathId: [OwnerId: 72075186224037897, LocalPathId: 1], TablePathId: [OwnerId: 72075186224037897, LocalPathId: 39], IndexType: EIndexTypeGlobalVectorKmeansTree, IndexName: idx_vector, IndexColumn: intent, IndexColumn: embedding, State: Filling, IsCancellationRequested: 0, Issue: , SubscribersCount: 0, CreateSender: [0:0:0], AlterMainTableTxId: 0, AlterMainTableTxStatus: StatusSuccess, AlterMainTableTxDone: 0, LockTxId: 281474977840757, LockTxStatus: StatusAccepted, LockTxDone: 1, InitiateTxId: 281474977840758, InitiateTxStatus: StatusAccepted, InitiateTxDone: 1, SnapshotStepId: 0, ApplyTxId: 0, ApplyTxStatus: StatusSuccess, ApplyTxDone: 0, UnlockTxId: 0, UnlockTxStatus: StatusSuccess, UnlockTxDone: 0, ToUploadShards: 0, DoneShards: 0, Processed: { upload rows: 288624165, upload bytes: 1198069787672, read rows: 288624165, read bytes: 1198069787672 }, Billed: { upload rows: 0, upload bytes: 0, read rows: 0, read bytes: 0 }}
2025-04-16T14:18:39.1744802319641227Z :BUILD_INDEX DEBUG: TIndexBuilder::TXTYPE_PROGRESS_INDEX_BUILD: FillIndex::InitiateShards { K = 200, Level = 2 / 2, Parent = [1..5580836854440..5580836854440], Child = [5580836854441..5580836854441..1121748207742440], State = MultiLocal }, { Rows = 0, Sample = Collect }, { Done = 0, ToUpload = 0, InProgress = 0 }
2025-04-16T14:18:39.1744802319641238Z :BUILD_INDEX DEBUG: infinite range { From: -inf, To: inf }
2025-04-16T14:18:39.1744802319641243Z :BUILD_INDEX DEBUG: shard 72075186224037897:46789 range { From: -inf, To: { count: 2 } }
2025-04-16T14:18:39.1744802319641414Z :TABLET_EXECUTOR CRITICAL_INFO: Tablet 72075186224037897 unhandled exception yexception: ydb/core/scheme/scheme_tablecell.h:171: AsValue<T>() type size8 doesn't match TCell size 43
0. /-S/contrib/libs/cxxsupp/libcxxrt/exception.cc:839: throw_exception(__cxxabiv1::__cxa_exception*) @ 0x9618BF3
1. /-S/contrib/libs/cxxsupp/libcxxrt/exception.cc:882: __cxa_throw @ 0x9618BF3
2. /-S/ydb/core/scheme/scheme_tablecell.h:171: unsigned long NKikimr::TCell::AsValue<unsigned long, unsigned long>() const @ 0x90BCF6E
3. /home/kungasc/ydb/ydb/core/tx/schemeshard/schemeshard_info_types.h:3273: NKikimr::NSchemeShard::TIndexBuildInfo::TKMeans::RangeToBorders(NKikimr::TSerializedTableRange const&) const::'lambda0'()::operator()() const @ 0x106DCFB1
4. /home/kungasc/ydb/ydb/core/tx/schemeshard/schemeshard_info_types.h:3270: NKikimr::NSchemeShard::TIndexBuildInfo::TKMeans::RangeToBorders(NKikimr::TSerializedTableRange const&) const @ 0x106DCFB1
5. /home/kungasc/ydb/ydb/core/tx/schemeshard/schemeshard_info_types.cpp:2256: NKikimr::NSchemeShard::TIndexBuildInfo::AddParent(NKikimr::TSerializedTableRange const&, NKikimr::NSchemeShard::TShardIdx) @ 0x106DCFB1
6. /home/kungasc/ydb/ydb/core/tx/schemeshard/schemeshard_build_index__progress.cpp:1345: NKikimr::NSchemeShard::TSchemeShard::TIndexBuilder::TTxProgress::InitiateShards(NKikimr::NIceDb::TNiceDb&, NKikimr::NSchemeShard::TIndexBuildInfo&) @ 0xCC98A9A
7. /home/kungasc/ydb/ydb/core/tx/schemeshard/schemeshard_build_index__progress.cpp:1049: NKikimr::NSchemeShard::TSchemeShard::TIndexBuilder::TTxProgress::FillIndex(NKikimr::NTabletFlatExecutor::TTransactionContext&, NKikimr::NSchemeShard::TIndexBuildInfo&) @ 0xCC973FF
8. /home/kungasc/ydb/ydb/core/tx/schemeshard/schemeshard_build_index__progress.cpp:1126: NKikimr::NSchemeShard::TSchemeShard::TIndexBuilder::TTxProgress::DoExecute(NKikimr::NTabletFlatExecutor::TTransactionContext&, NActors::TActorContext const&) @ 0xCC95F09
9. /home/kungasc/ydb/ydb/core/tx/schemeshard/schemeshard_build_index_tx_base.cpp:417: NKikimr::NSchemeShard::TSchemeShard::TIndexBuilder::TTxBase::Execute(NKikimr::NTabletFlatExecutor::TTransactionContext&, NActors::TActorContext const&) @ 0xCCBCC99
10. /-S/ydb/core/tablet_flat/flat_executor.cpp:1910: NKikimr::NTabletFlatExecutor::TExecutor::ExecuteTransaction(NKikimr::NTabletFlatExecutor::TSeat*) @ 0xBA41103
11. /-S/ydb/core/tablet_flat/flat_executor.cpp:4143: NKikimr::NTabletFlatExecutor::TExecutor::StateWork(TAutoPtr<NActors::IEventHandle, TDelete>&) @ 0xBA2ED91
12. /place/sandbox-data/tasks/7/0/2989721707/ydb/ydb/library/actors/core/actor.cpp:280: NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&) @ 0xA1424B1

@kunga kunga mentioned this issue Apr 16, 2025
37 tasks
@kunga kunga self-assigned this Apr 16, 2025
@kunga
Copy link
Member Author

kunga commented Apr 23, 2025

Resolved in #17321

@kunga kunga closed this as completed Apr 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant