Skip to content

Commit 951320e

Browse files
[3.11] gh-98154: Clarify Usage of "Reference Count" In the Docs (gh-107753)
PEP 683 (immortal objects) revealed some ways in which the Python documentation has been unnecessarily coupled to the implementation details of reference counts. In the end users should focus on reference ownership, including taking references and releasing them, rather than on how many reference counts an object has. This change updates the documentation to reflect that perspective.
1 parent 22b39d1 commit 951320e

14 files changed

+123
-83
lines changed

Doc/c-api/allocation.rst

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -29,12 +29,13 @@ Allocating Objects on the Heap
2929
3030
.. c:macro:: PyObject_New(TYPE, typeobj)
3131
32-
Allocate a new Python object using the C structure type *TYPE* and the
33-
Python type object *typeobj* (``PyTypeObject*``).
34-
Fields not defined by the Python object header
35-
are not initialized; the object's reference count will be one. The size of
36-
the memory allocation is determined from the :c:member:`~PyTypeObject.tp_basicsize` field of
37-
the type object.
32+
Allocate a new Python object using the C structure type *TYPE*
33+
and the Python type object *typeobj* (``PyTypeObject*``).
34+
Fields not defined by the Python object header are not initialized.
35+
The caller will own the only reference to the object
36+
(i.e. its reference count will be one).
37+
The size of the memory allocation is determined from the
38+
:c:member:`~PyTypeObject.tp_basicsize` field of the type object.
3839
3940
4041
.. c:macro:: PyObject_NewVar(TYPE, typeobj, size)

Doc/c-api/arg.rst

Lines changed: 11 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -330,8 +330,10 @@ Other objects
330330

331331
``O`` (object) [PyObject \*]
332332
Store a Python object (without any conversion) in a C object pointer. The C
333-
program thus receives the actual object that was passed. The object's reference
334-
count is not increased. The pointer stored is not ``NULL``.
333+
program thus receives the actual object that was passed. A new
334+
:term:`strong reference` to the object is not created
335+
(i.e. its reference count is not increased).
336+
The pointer stored is not ``NULL``.
335337

336338
``O!`` (object) [*typeobject*, PyObject \*]
337339
Store a Python object in a C object pointer. This is similar to ``O``, but
@@ -415,7 +417,8 @@ inside nested parentheses. They are:
415417
mutually exclude each other.
416418

417419
Note that any Python object references which are provided to the caller are
418-
*borrowed* references; do not decrement their reference count!
420+
*borrowed* references; do not release them
421+
(i.e. do not decrement their reference count)!
419422

420423
Additional arguments passed to these functions must be addresses of variables
421424
whose type is determined by the format string; these are used to store values
@@ -650,8 +653,10 @@ Building values
650653
Convert a C :c:type:`Py_complex` structure to a Python complex number.
651654
652655
``O`` (object) [PyObject \*]
653-
Pass a Python object untouched (except for its reference count, which is
654-
incremented by one). If the object passed in is a ``NULL`` pointer, it is assumed
656+
Pass a Python object untouched but create a new
657+
:term:`strong reference` to it
658+
(i.e. its reference count is incremented by one).
659+
If the object passed in is a ``NULL`` pointer, it is assumed
655660
that this was caused because the call producing the argument found an error and
656661
set an exception. Therefore, :c:func:`Py_BuildValue` will return ``NULL`` but won't
657662
raise an exception. If no exception has been raised yet, :exc:`SystemError` is
@@ -661,7 +666,7 @@ Building values
661666
Same as ``O``.
662667
663668
``N`` (object) [PyObject \*]
664-
Same as ``O``, except it doesn't increment the reference count on the object.
669+
Same as ``O``, except it doesn't create a new :term:`strong reference`.
665670
Useful when the object is created by a call to an object constructor in the
666671
argument list.
667672

Doc/c-api/buffer.rst

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -102,7 +102,9 @@ a buffer, see :c:func:`PyObject_GetBuffer`.
102102
.. c:member:: PyObject *obj
103103
104104
A new reference to the exporting object. The reference is owned by
105-
the consumer and automatically decremented and set to ``NULL`` by
105+
the consumer and automatically released
106+
(i.e. reference count decremented)
107+
and set to ``NULL`` by
106108
:c:func:`PyBuffer_Release`. The field is the equivalent of the return
107109
value of any standard C-API function.
108110

@@ -454,7 +456,8 @@ Buffer-related functions
454456
455457
.. c:function:: void PyBuffer_Release(Py_buffer *view)
456458
457-
Release the buffer *view* and decrement the reference count for
459+
Release the buffer *view* and release the :term:`strong reference`
460+
(i.e. decrement the reference count) to the view's supporting object,
458461
``view->obj``. This function MUST be called when the buffer
459462
is no longer being used, otherwise reference leaks may occur.
460463

Doc/c-api/bytes.rst

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -187,8 +187,8 @@ called with a non-bytes parameter.
187187
.. c:function:: void PyBytes_ConcatAndDel(PyObject **bytes, PyObject *newpart)
188188
189189
Create a new bytes object in *\*bytes* containing the contents of *newpart*
190-
appended to *bytes*. This version decrements the reference count of
191-
*newpart*.
190+
appended to *bytes*. This version releases the :term:`strong reference`
191+
to *newpart* (i.e. decrements its reference count).
192192
193193
194194
.. c:function:: int _PyBytes_Resize(PyObject **bytes, Py_ssize_t newsize)

Doc/c-api/exceptions.rst

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -99,7 +99,8 @@ For convenience, some of these functions will always return a
9999
100100
This is the most common way to set the error indicator. The first argument
101101
specifies the exception type; it is normally one of the standard exceptions,
102-
e.g. :c:data:`PyExc_RuntimeError`. You need not increment its reference count.
102+
e.g. :c:data:`PyExc_RuntimeError`. You need not create a new
103+
:term:`strong reference` to it (e.g. with :c:func:`Py_INCREF`).
103104
The second argument is an error message; it is decoded from ``'utf-8'``.
104105
105106

Doc/c-api/intro.rst

Lines changed: 33 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -287,60 +287,67 @@ true if (and only if) the object pointed to by *a* is a Python list.
287287
Reference Counts
288288
----------------
289289

290-
The reference count is important because today's computers have a finite (and
291-
often severely limited) memory size; it counts how many different places there
292-
are that have a reference to an object. Such a place could be another object,
293-
or a global (or static) C variable, or a local variable in some C function.
294-
When an object's reference count becomes zero, the object is deallocated. If
295-
it contains references to other objects, their reference count is decremented.
296-
Those other objects may be deallocated in turn, if this decrement makes their
297-
reference count become zero, and so on. (There's an obvious problem with
298-
objects that reference each other here; for now, the solution is "don't do
299-
that.")
290+
The reference count is important because today's computers have a finite
291+
(and often severely limited) memory size; it counts how many different
292+
places there are that have a :term:`strong reference` to an object.
293+
Such a place could be another object, or a global (or static) C variable,
294+
or a local variable in some C function.
295+
When the last :term:`strong reference` to an object is released
296+
(i.e. its reference count becomes zero), the object is deallocated.
297+
If it contains references to other objects, those references are released.
298+
Those other objects may be deallocated in turn, if there are no more
299+
references to them, and so on. (There's an obvious problem with
300+
objects that reference each other here; for now, the solution
301+
is "don't do that.")
300302

301303
.. index::
302304
single: Py_INCREF()
303305
single: Py_DECREF()
304306

305-
Reference counts are always manipulated explicitly. The normal way is to use
306-
the macro :c:func:`Py_INCREF` to increment an object's reference count by one,
307-
and :c:func:`Py_DECREF` to decrement it by one. The :c:func:`Py_DECREF` macro
307+
Reference counts are always manipulated explicitly. The normal way is
308+
to use the macro :c:func:`Py_INCREF` to take a new reference to an
309+
object (i.e. increment its reference count by one),
310+
and :c:func:`Py_DECREF` to release that reference (i.e. decrement the
311+
reference count by one). The :c:func:`Py_DECREF` macro
308312
is considerably more complex than the incref one, since it must check whether
309313
the reference count becomes zero and then cause the object's deallocator to be
310-
called. The deallocator is a function pointer contained in the object's type
311-
structure. The type-specific deallocator takes care of decrementing the
312-
reference counts for other objects contained in the object if this is a compound
314+
called. The deallocator is a function pointer contained in the object's type
315+
structure. The type-specific deallocator takes care of releasing references
316+
for other objects contained in the object if this is a compound
313317
object type, such as a list, as well as performing any additional finalization
314318
that's needed. There's no chance that the reference count can overflow; at
315319
least as many bits are used to hold the reference count as there are distinct
316320
memory locations in virtual memory (assuming ``sizeof(Py_ssize_t) >= sizeof(void*)``).
317321
Thus, the reference count increment is a simple operation.
318322

319-
It is not necessary to increment an object's reference count for every local
320-
variable that contains a pointer to an object. In theory, the object's
323+
It is not necessary to hold a :term:`strong reference` (i.e. increment
324+
the reference count) for every local variable that contains a pointer
325+
to an object. In theory, the object's
321326
reference count goes up by one when the variable is made to point to it and it
322327
goes down by one when the variable goes out of scope. However, these two
323328
cancel each other out, so at the end the reference count hasn't changed. The
324329
only real reason to use the reference count is to prevent the object from being
325330
deallocated as long as our variable is pointing to it. If we know that there
326331
is at least one other reference to the object that lives at least as long as
327-
our variable, there is no need to increment the reference count temporarily.
332+
our variable, there is no need to take a new :term:`strong reference`
333+
(i.e. increment the reference count) temporarily.
328334
An important situation where this arises is in objects that are passed as
329335
arguments to C functions in an extension module that are called from Python;
330336
the call mechanism guarantees to hold a reference to every argument for the
331337
duration of the call.
332338

333339
However, a common pitfall is to extract an object from a list and hold on to it
334-
for a while without incrementing its reference count. Some other operation might
335-
conceivably remove the object from the list, decrementing its reference count
340+
for a while without taking a new reference. Some other operation might
341+
conceivably remove the object from the list, releasing that reference,
336342
and possibly deallocating it. The real danger is that innocent-looking
337343
operations may invoke arbitrary Python code which could do this; there is a code
338344
path which allows control to flow back to the user from a :c:func:`Py_DECREF`, so
339345
almost any operation is potentially dangerous.
340346

341347
A safe approach is to always use the generic operations (functions whose name
342348
begins with ``PyObject_``, ``PyNumber_``, ``PySequence_`` or ``PyMapping_``).
343-
These operations always increment the reference count of the object they return.
349+
These operations always create a new :term:`strong reference`
350+
(i.e. increment the reference count) of the object they return.
344351
This leaves the caller with the responsibility to call :c:func:`Py_DECREF` when
345352
they are done with the result; this soon becomes second nature.
346353

@@ -356,7 +363,7 @@ to objects (objects are not owned: they are always shared). "Owning a
356363
reference" means being responsible for calling Py_DECREF on it when the
357364
reference is no longer needed. Ownership can also be transferred, meaning that
358365
the code that receives ownership of the reference then becomes responsible for
359-
eventually decref'ing it by calling :c:func:`Py_DECREF` or :c:func:`Py_XDECREF`
366+
eventually releasing it by calling :c:func:`Py_DECREF` or :c:func:`Py_XDECREF`
360367
when it's no longer needed---or passing on this responsibility (usually to its
361368
caller). When a function passes ownership of a reference on to its caller, the
362369
caller is said to receive a *new* reference. When no ownership is transferred,
@@ -414,9 +421,9 @@ For example, the above two blocks of code could be replaced by the following
414421

415422
It is much more common to use :c:func:`PyObject_SetItem` and friends with items
416423
whose references you are only borrowing, like arguments that were passed in to
417-
the function you are writing. In that case, their behaviour regarding reference
418-
counts is much saner, since you don't have to increment a reference count so you
419-
can give a reference away ("have it be stolen"). For example, this function
424+
the function you are writing. In that case, their behaviour regarding references
425+
is much saner, since you don't have to take a new reference just so you
426+
can give that reference away ("have it be stolen"). For example, this function
420427
sets all items of a list (actually, any mutable sequence) to a given item::
421428

422429
int

Doc/c-api/module.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -498,7 +498,7 @@ state:
498498
.. note::
499499
500500
Unlike other functions that steal references, ``PyModule_AddObject()``
501-
only decrements the reference count of *value* **on success**.
501+
only releases the reference to *value* **on success**.
502502
503503
This means that its return value must be checked, and calling code must
504504
:c:func:`Py_DECREF` *value* manually on error.

Doc/c-api/object.rst

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,8 @@ Object Protocol
1515
.. c:macro:: Py_RETURN_NOTIMPLEMENTED
1616
1717
Properly handle returning :c:data:`Py_NotImplemented` from within a C
18-
function (that is, increment the reference count of NotImplemented and
19-
return it).
18+
function (that is, create a new :term:`strong reference`
19+
to NotImplemented and return it).
2020

2121

2222
.. c:function:: int PyObject_Print(PyObject *o, FILE *fp, int flags)
@@ -320,11 +320,12 @@ Object Protocol
320320
321321
When *o* is non-``NULL``, returns a type object corresponding to the object type
322322
of object *o*. On failure, raises :exc:`SystemError` and returns ``NULL``. This
323-
is equivalent to the Python expression ``type(o)``. This function increments the
324-
reference count of the return value. There's really no reason to use this
323+
is equivalent to the Python expression ``type(o)``.
324+
This function creates a new :term:`strong reference` to the return value.
325+
There's really no reason to use this
325326
function instead of the :c:func:`Py_TYPE()` function, which returns a
326-
pointer of type :c:expr:`PyTypeObject*`, except when the incremented reference
327-
count is needed.
327+
pointer of type :c:expr:`PyTypeObject*`, except when a new
328+
:term:`strong reference` is needed.
328329
329330
330331
.. c:function:: int PyObject_TypeCheck(PyObject *o, PyTypeObject *type)

Doc/c-api/refcounting.rst

Lines changed: 30 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -13,31 +13,36 @@ objects.
1313

1414
.. c:function:: void Py_INCREF(PyObject *o)
1515
16-
Increment the reference count for object *o*.
16+
Indicate taking a new :term:`strong reference` to object *o*,
17+
indicating it is in use and should not be destroyed.
1718
1819
This function is usually used to convert a :term:`borrowed reference` to a
1920
:term:`strong reference` in-place. The :c:func:`Py_NewRef` function can be
2021
used to create a new :term:`strong reference`.
2122
23+
When done using the object, release it by calling :c:func:`Py_DECREF`.
24+
2225
The object must not be ``NULL``; if you aren't sure that it isn't
2326
``NULL``, use :c:func:`Py_XINCREF`.
2427
28+
Do not expect this function to actually modify *o* in any way.
29+
2530
2631
.. c:function:: void Py_XINCREF(PyObject *o)
2732
28-
Increment the reference count for object *o*. The object may be ``NULL``, in
29-
which case the macro has no effect.
33+
Similar to :c:func:`Py_INCREF`, but the object *o* can be ``NULL``,
34+
in which case this has no effect.
3035
3136
See also :c:func:`Py_XNewRef`.
3237
3338
3439
.. c:function:: PyObject* Py_NewRef(PyObject *o)
3540
36-
Create a new :term:`strong reference` to an object: increment the reference
37-
count of the object *o* and return the object *o*.
41+
Create a new :term:`strong reference` to an object:
42+
call :c:func:`Py_INCREF` on *o* and return the object *o*.
3843
3944
When the :term:`strong reference` is no longer needed, :c:func:`Py_DECREF`
40-
should be called on it to decrement the object reference count.
45+
should be called on it to release the reference.
4146
4247
The object *o* must not be ``NULL``; use :c:func:`Py_XNewRef` if *o* can be
4348
``NULL``.
@@ -67,9 +72,12 @@ objects.
6772
6873
.. c:function:: void Py_DECREF(PyObject *o)
6974
70-
Decrement the reference count for object *o*.
75+
Release a :term:`strong reference` to object *o*, indicating the
76+
reference is no longer used.
7177
72-
If the reference count reaches zero, the object's type's deallocation
78+
Once the last :term:`strong reference` is released
79+
(i.e. the object's reference count reaches 0),
80+
the object's type's deallocation
7381
function (which must not be ``NULL``) is invoked.
7482
7583
This function is usually used to delete a :term:`strong reference` before
@@ -78,6 +86,8 @@ objects.
7886
The object must not be ``NULL``; if you aren't sure that it isn't ``NULL``,
7987
use :c:func:`Py_XDECREF`.
8088
89+
Do not expect this function to actually modify *o* in any way.
90+
8191
.. warning::
8292
8393
The deallocation function can cause arbitrary Python code to be invoked (e.g.
@@ -92,32 +102,35 @@ objects.
92102
93103
.. c:function:: void Py_XDECREF(PyObject *o)
94104
95-
Decrement the reference count for object *o*. The object may be ``NULL``, in
96-
which case the macro has no effect; otherwise the effect is the same as for
97-
:c:func:`Py_DECREF`, and the same warning applies.
105+
Similar to :c:func:`Py_DECREF`, but the object *o* can be ``NULL``,
106+
in which case this has no effect.
107+
The same warning from :c:func:`Py_DECREF` applies here as well.
98108
99109
100110
.. c:function:: void Py_CLEAR(PyObject *o)
101111
102-
Decrement the reference count for object *o*. The object may be ``NULL``, in
112+
Release a :term:`strong reference` for object *o*.
113+
The object may be ``NULL``, in
103114
which case the macro has no effect; otherwise the effect is the same as for
104115
:c:func:`Py_DECREF`, except that the argument is also set to ``NULL``. The warning
105116
for :c:func:`Py_DECREF` does not apply with respect to the object passed because
106117
the macro carefully uses a temporary variable and sets the argument to ``NULL``
107-
before decrementing its reference count.
118+
before releasing the reference.
108119
109-
It is a good idea to use this macro whenever decrementing the reference
110-
count of an object that might be traversed during garbage collection.
120+
It is a good idea to use this macro whenever releasing a reference
121+
to an object that might be traversed during garbage collection.
111122
112123
.. c:function:: void Py_IncRef(PyObject *o)
113124
114-
Increment the reference count for object *o*. A function version of :c:func:`Py_XINCREF`.
125+
Indicate taking a new :term:`strong reference` to object *o*.
126+
A function version of :c:func:`Py_XINCREF`.
115127
It can be used for runtime dynamic embedding of Python.
116128
117129
118130
.. c:function:: void Py_DecRef(PyObject *o)
119131
120-
Decrement the reference count for object *o*. A function version of :c:func:`Py_XDECREF`.
132+
Release a :term:`strong reference` to object *o*.
133+
A function version of :c:func:`Py_XDECREF`.
121134
It can be used for runtime dynamic embedding of Python.
122135
123136

Doc/c-api/sys.rst

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,9 @@ Operating System Utilities
88
.. c:function:: PyObject* PyOS_FSPath(PyObject *path)
99
1010
Return the file system representation for *path*. If the object is a
11-
:class:`str` or :class:`bytes` object, then its reference count is
12-
incremented. If the object implements the :class:`os.PathLike` interface,
11+
:class:`str` or :class:`bytes` object, then a new
12+
:term:`strong reference` is returned.
13+
If the object implements the :class:`os.PathLike` interface,
1314
then :meth:`~os.PathLike.__fspath__` is returned as long as it is a
1415
:class:`str` or :class:`bytes` object. Otherwise :exc:`TypeError` is raised
1516
and ``NULL`` is returned.

0 commit comments

Comments
 (0)