Skip to content

Fix small reads of multi-part blobs #54573

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

DaveCTurner
Copy link
Contributor

No description provided.

@DaveCTurner DaveCTurner added >bug :Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs labels Apr 1, 2020
@DaveCTurner DaveCTurner requested a review from tlrx April 1, 2020 11:23
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (:Distributed/Snapshot/Restore)

Copy link
Member

@tlrx tlrx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Thanks David for catching and fixing this and sorry for this bug. I left a suggestion, feel free to use it or not.

(currentPart == startPart) ? getRelativePositionInPart(position) : 0L,
(currentPart == endPart) ? getRelativePositionInPart(length) : getLengthOfPart(currentPart)
startInPart,
endInPart - startInPart
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I was able to reproduce and came with a similar fix.

For better readability I'd suggest to isolate single part case (and be paranoid about the length to read) from the multiple parts case, something like:

Suggested change
endInPart - startInPart
if (fileInfo.numberOfParts() == 1L) {
return blobContainer.readBlob(fileInfo.partName(0L), position, Math.min(length, fileInfo.partBytes(0) - position));
}
final long startPart = getPartNumberForPosition(position);
final long endPart = getPartNumberForPosition(position + length);
return new SlicedInputStream(endPart - startPart + 1L) {
@Override
protected InputStream openSlice(long slice) throws IOException {
final long currentPart = startPart + slice;
final long startInPart = (currentPart == startPart) ? getRelativePositionInPart(position) : 0L;
final long endInPart
= (currentPart == endPart) ? getRelativePositionInPart(position + length) : getLengthOfPart(currentPart);
return blobContainer.readBlob(fileInfo.partName(currentPart), startInPart, endInPart - startInPart
);
}
};

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, sounds good, see 558e68a.

@DaveCTurner DaveCTurner merged commit 6e7a5e0 into elastic:feature/searchable-snapshots Apr 2, 2020
@DaveCTurner DaveCTurner deleted the 2020-04-01-fix-small-reads-of-multi-part-blobs branch April 2, 2020 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants