Skip to content

add stream test to csv test case 497 #541

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Feb 26, 2025

Conversation

pjfanning
Copy link
Member

@pjfanning pjfanning commented Feb 26, 2025

#497 issue only seems to happen with byte array input - work ok with InputStream input.

When byte array input to UTF8Reader, the _inputSource is always null and the _ioContext is null which is what is checked in canModifyBuffer(). Is there any way that this code can be adjusted to work for byte array input - or should we take the byte array and wrap as ByteArrayInputStream? - which would also fix the broken test.

The code that causes the issue.

if (!canModifyBuffer()) {
// 15-Aug-2022, tatu: Occurs (only) if we have half-decoded UTF-8
// characters; uncovered by:
//
// https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=50036
//
// and need to be reported as IOException
if (_inputSource == null) {
throw new IOException(String.format(
"End-of-input after first %d byte(s) of a UTF-8 character: needed at least one more",
available));
}
}

Needed for a fuzz test to work but also breaks the 497 test.

@cowtowncoder
Copy link
Member

If I remember correctly, it's not really so much the case of InputStream actually working, but it not hitting AAOBE -- since there's empty input buffer extending beyond content end.
Or maybe put another way: it working by reading out of actual content buffer.

One way to test this would be to pass byte[] where content doesn't end at buffer end but couple of bytes before, having empty slots at end.

I'll merge this: I understand the bug itself but don't have time to fix it. Adding more tests may not help with that part significantly.

@cowtowncoder cowtowncoder merged commit e306fa6 into FasterXML:2.19 Feb 26, 2025
4 checks passed
@pjfanning pjfanning deleted the csv-issue branch February 26, 2025 09:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants