-
-
Notifications
You must be signed in to change notification settings - Fork 32k
GH-128131: Completely support random read access of uncompressed unencrypted files in ZipFile #128143
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GH-128131: Completely support random read access of uncompressed unencrypted files in ZipFile #128143
Changes from 7 commits
f18239d
f90340f
5d23be6
1a64610
57cb51c
4d9cea4
3d37f31
67f05de
25f0a7e
f2f2374
a2c9037
6a55aad
1239005
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
@@ -3447,6 +3447,47 @@ def test_too_short(self): | |||||||||||||
self.assertEqual( | ||||||||||||||
b"zzz", zipfile._Extra.strip(b"zzz", (self.ZIP64_EXTRA,))) | ||||||||||||||
|
||||||||||||||
class StoredZipExtFileRandomReadTest(unittest.TestCase): | ||||||||||||||
def test_random_read(self): | ||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Please add a docstring describing what this function is testing. Give an overview of what the problem is and what is being guaranteed by the test. Also, link to the reported issue, where the problem should be described in exquisite detail. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The name of this function seems too broad. It's not simply testing a random read, but it's also testing the case where it's following an optimized path for an uncompressed file. Consider |
||||||||||||||
from _pyio import BytesIO | ||||||||||||||
class StatIO(BytesIO): | ||||||||||||||
def __init__(self): | ||||||||||||||
super().__init__() | ||||||||||||||
self.bytes_read = 0 | ||||||||||||||
|
||||||||||||||
def read(self, size=-1): | ||||||||||||||
bs = super().read(size) | ||||||||||||||
self.bytes_read += len(bs) | ||||||||||||||
return bs | ||||||||||||||
5ec1cff marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||||
|
||||||||||||||
sio = StatIO() | ||||||||||||||
# 20000 bytes | ||||||||||||||
txt = b'0123456789' * 2000 | ||||||||||||||
|
||||||||||||||
with zipfile.ZipFile(sio, "w", compression=zipfile.ZIP_STORED) as zipf: | ||||||||||||||
zipf.writestr("foo.txt", txt) | ||||||||||||||
|
||||||||||||||
# check random seek and read on a file | ||||||||||||||
with zipfile.ZipFile(sio, "r") as zipf: | ||||||||||||||
with zipf.open("foo.txt", "r") as fp: | ||||||||||||||
# seek length must be greater than ZipExtFile.MIN_READ_SIZE (4096) | ||||||||||||||
5ec1cff marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||||
# forward seek | ||||||||||||||
old_count = sio.bytes_read | ||||||||||||||
fp.seek(10002, os.SEEK_CUR) | ||||||||||||||
self.assertEqual(fp.tell(), 10002) | ||||||||||||||
arr = fp.read(100) | ||||||||||||||
self.assertEqual(fp.tell(), 10102) | ||||||||||||||
self.assertEqual(arr, txt[10002:10102]) | ||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Re-use the variables defined above.
Suggested change
|
||||||||||||||
self.assertLessEqual(sio.bytes_read - old_count, 4096, 'Redundant bytes were read during forward seek and read!') | ||||||||||||||
5ec1cff marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||||
|
||||||||||||||
# backward seek | ||||||||||||||
old_count = sio.bytes_read | ||||||||||||||
fp.seek(-5003, os.SEEK_CUR) | ||||||||||||||
picnixz marked this conversation as resolved.
Show resolved
Hide resolved
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||
self.assertEqual(fp.tell(), 5099) | ||||||||||||||
5ec1cff marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||||
arr = fp.read(100) | ||||||||||||||
self.assertEqual(fp.tell(), 5199) | ||||||||||||||
self.assertEqual(arr, txt[5099:5199]) | ||||||||||||||
self.assertLessEqual(sio.bytes_read - old_count, 4096, 'Redundant bytes were read during backward seek and read!') | ||||||||||||||
5ec1cff marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||||
|
||||||||||||||
5ec1cff marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||||
if __name__ == "__main__": | ||||||||||||||
unittest.main() |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
Completely support random access of uncompressed unencrypted read-only | ||
zip files obtained by :meth:`ZipFile.open <zipfile.ZipFile.open>`. |
Uh oh!
There was an error while loading. Please reload this page.