Skip to content

gh-80050: Update BufferedReader.read docs around non-blocking #130653

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
May 21, 2025
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 41 additions & 44 deletions Doc/library/io.rst
Original file line number Diff line number Diff line change
Expand Up @@ -528,14 +528,13 @@ I/O Base Classes
It inherits from :class:`IOBase`.

The main difference with :class:`RawIOBase` is that methods :meth:`read`,
:meth:`readinto` and :meth:`write` will try (respectively) to read as much
input as requested or to consume all given output, at the expense of
making perhaps more than one system call.
:meth:`readinto` and :meth:`write` will try (respectively) to read
as much input as requested or to emit all provided data.

In addition, those methods can raise :exc:`BlockingIOError` if the
underlying raw stream is in non-blocking mode and cannot take or give
enough data; unlike their :class:`RawIOBase` counterparts, they will
never return ``None``.
In addition, if the underlying raw stream is in non-blocking mode, when the
system returns would block :meth:`write` will raise :exc:`BlockingIOError`
with :attr:`BlockingIOError.characters_written` and :meth:`read` will return
data read so far or ``None`` if no data is available.

Besides, the :meth:`read` method does not have a default
implementation that defers to :meth:`readinto`.
Expand Down Expand Up @@ -568,29 +567,40 @@ I/O Base Classes

.. method:: read(size=-1, /)

Read and return up to *size* bytes. If the argument is omitted, ``None``,
or negative, data is read and returned until EOF is reached. An empty
:class:`bytes` object is returned if the stream is already at EOF.
Read and return up to *size* bytes. If the argument is omitted, ``None``,
or negative read as much as possible.

If the argument is positive, and the underlying raw stream is not
interactive, multiple raw reads may be issued to satisfy the byte count
(unless EOF is reached first). But for interactive raw streams, at most
one raw read will be issued, and a short result does not imply that EOF is
imminent.
Fewer bytes may be returned than requested. An empty :class:`bytes` object
is returned if the stream is already at EOF. More than one read may be
made and calls may be retried if specific errors are encountered, see
:meth:`os.read` and :pep:`475` for more details. Less than size bytes
being returned does not imply that EOF is imminent.

A :exc:`BlockingIOError` is raised if the underlying raw stream is in
non blocking-mode, and has no data available at the moment.
When reading as much as possible the default implementation will use
``raw.readall`` if available (which should implement
:meth:`RawIOBase.readall`), otherwise will read in a loop until read
returns ``None``, an empty :class:`bytes`, or a non-retryable error. For
most streams this is to EOF, but for non-blocking streams more data may
become available.

.. note::

When the underlying raw stream is non-blocking, implementations may
either raise :exc:`BlockingIOError` or return ``None`` if no data is
available. :mod:`io` implementations return ``None``.

.. method:: read1(size=-1, /)

Read and return up to *size* bytes, with at most one call to the
underlying raw stream's :meth:`~RawIOBase.read` (or
:meth:`~RawIOBase.readinto`) method. This can be useful if you are
implementing your own buffering on top of a :class:`BufferedIOBase`
object.
Read and return up to *size* bytes, calling :meth:`~RawIOBase.readinto`
which may retry if :py:const:`~errno.EINTR` is encountered per
:pep:`475`. If *size* is ``-1`` or not provided, the implementation will
choose an arbitrary value for *size*.

If *size* is ``-1`` (the default), an arbitrary number of bytes are
returned (more than zero unless EOF is reached).
.. note::

When the underlying raw stream is non-blocking, implementations may
either raise :exc:`BlockingIOError` or return ``None`` if no data is
available. :mod:`io` implementations return ``None``.

.. method:: readinto(b, /)

Expand Down Expand Up @@ -767,34 +777,21 @@ than raw I/O does.

.. method:: peek(size=0, /)

Return bytes from the stream without advancing the position. At most one
single read on the raw stream is done to satisfy the call. The number of
bytes returned may be less or more than requested.
Return bytes from the stream without advancing the position. The number of
bytes returned may be less or more than requested. If the underlying raw
stream is non-blocking and the operation would block, returns empty bytes.

.. method:: read(size=-1, /)

Read and return *size* bytes, or if *size* is not given or negative, until
EOF or if the read call would block in non-blocking mode.

.. note::

When the underlying raw stream is non-blocking, a :exc:`BlockingIOError`
may be raised if a read operation cannot be completed immediately.
In :class:`BufferedReader` this is the same as :meth:`io.BufferedIOBase.read`

.. method:: read1(size=-1, /)

Read and return up to *size* bytes with only one call on the raw stream.
If at least one byte is buffered, only buffered bytes are returned.
Otherwise, one raw stream read call is made.
In :class:`BufferedReader` this is the same as :meth:`io.BufferedIOBase.read1`

.. versionchanged:: 3.7
The *size* argument is now optional.

.. note::

When the underlying raw stream is non-blocking, a :exc:`BlockingIOError`
may be raised if a read operation cannot be completed immediately.

.. class:: BufferedWriter(raw, buffer_size=DEFAULT_BUFFER_SIZE)

A buffered binary stream providing higher-level access to a writeable, non
Expand Down Expand Up @@ -826,8 +823,8 @@ than raw I/O does.

Write the :term:`bytes-like object`, *b*, and return the
number of bytes written. When in non-blocking mode, a
:exc:`BlockingIOError` is raised if the buffer needs to be written out but
the raw stream blocks.
:exc:`BlockingIOError` with :attr:`BlockingIOError.characters_written` set
Copy link
Contributor Author

@cmaloney cmaloney May 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

characters_written is fairly well tested in this case

cpython/Lib/test/test_io.py

Lines 1918 to 1943 in a7f317d

def test_write_non_blocking(self):
raw = self.MockNonBlockWriterIO()
bufio = self.tp(raw, 8)
self.assertEqual(bufio.write(b"abcd"), 4)
self.assertEqual(bufio.write(b"efghi"), 5)
# 1 byte will be written, the rest will be buffered
raw.block_on(b"k")
self.assertEqual(bufio.write(b"jklmn"), 5)
# 8 bytes will be written, 8 will be buffered and the rest will be lost
raw.block_on(b"0")
try:
bufio.write(b"opqrwxyz0123456789")
except self.BlockingIOError as e:
written = e.characters_written
else:
self.fail("BlockingIOError should have been raised")
self.assertEqual(written, 16)
self.assertEqual(raw.pop_written(),
b"abcdefghijklmnopqrwxyz")
self.assertEqual(bufio.write(b"ABCDEFGHI"), 9)
s = raw.pop_written()
# Previously buffered bytes were flushed
self.assertTrue(s.startswith(b"01234567A"), s)

is raised if the buffer needs to be written out but the raw stream blocks.


.. class:: BufferedRandom(raw, buffer_size=DEFAULT_BUFFER_SIZE)
Expand Down
Loading