Skip to content

Commit 784fb45

Browse files
committed
bpo-36974: expand call protocol documentation
1 parent fde9cf7 commit 784fb45

File tree

6 files changed

+206
-99
lines changed

6 files changed

+206
-99
lines changed

Doc/c-api/call.rst

Lines changed: 144 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,109 @@
55
Call Protocol
66
=============
77

8+
CPython supports two different calling protocols:
9+
*tp_call* and vectorcall.
10+
11+
The *tp_call* Protocol
12+
----------------------
13+
14+
Classes can implement callables by setting :c:member:`~PyTypeObject.tp_call`.
15+
16+
A call is made using a tuple for the positional arguments
17+
and a dict for the keyword arguments, just like ``f(*args, **kwargs)``
18+
in the Python language.
19+
*args* must be non-NULL
20+
(use an empty tuple if there are no arguments)
21+
but *kwargs* may be *NULL* if there are no keyword arguments.
22+
23+
This convention is not only used by *tp_call*:
24+
also :c:member:`~PyTypeObject.tp_new` and :c:member:`~PyTypeObject.tp_init`
25+
pass arguments this way
26+
(apart from the special handling of *subtype* and *self*).
27+
28+
For making a call this way, use :c:func:`PyObject_Call`.
29+
30+
.. _vectorcall:
31+
32+
The Vectorcall Protocol
33+
-----------------------
34+
35+
.. versionadded:: 3.8
36+
37+
The vectorcall protocol was introduced in :pep:`590`
38+
for making calls more efficient.
39+
40+
Classes can implement the vectorcall protocol by enabling the
41+
:const:`_Py_TPFLAGS_HAVE_VECTORCALL` flag and setting
42+
:c:member:`~PyTypeObject.tp_vectorcall_offset` to the offset inside the
43+
object structure where a *vectorcallfunc* appears.
44+
This is a function with the following signature:
45+
46+
.. c:type:: PyObject *(*vectorcallfunc)(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwnames)
47+
48+
- *callable* is the object being called.
49+
- *args* is a C array consisting of the positional arguments followed by the
50+
values of the keyword arguments.
51+
This can be *NULL* if there are no arguments.
52+
- *nargsf* is the number of positional arguments plus possibly the
53+
:const:`PY_VECTORCALL_ARGUMENTS_OFFSET` flag.
54+
To get the actual number of positional arguments from *nargsf*,
55+
use :c:func:`PyVectorcall_NARGS`.
56+
- *kwnames* is a tuple containing the names of the keyword arguments,
57+
in other words the keys of the kwargs dict.
58+
These names must be strings (instances of ``str`` or a subclass)
59+
and they must be unique.
60+
If there are no keyword arguments, then *kwnames* can instead be *NULL*.
61+
62+
.. c:var:: PY_VECTORCALL_ARGUMENTS_OFFSET
63+
64+
If this flag is set in a vectorcall *nargsf* argument, the callee is allowed
65+
to temporarily change ``args[-1]``. In other words, *args* points to
66+
argument 1 (not 0) in the allocated vector.
67+
The callee must restore the value of ``args[-1]`` before returning.
68+
69+
For :c:func:`_PyObject_VectorcallMethod`, this flag means instead that
70+
``args[0]`` may be changed.
71+
72+
Whenever they can do so cheaply (without additional allocation), callers
73+
are encouraged to use :const:`PY_VECTORCALL_ARGUMENTS_OFFSET`.
74+
Doing so will allow callables such as bound methods to make their onward
75+
calls (which include a prepended *self* argument) very efficiently.
76+
77+
.. warning::
78+
79+
As rule of thumb, CPython will internally use the vectorcall
80+
protocol if the callable supports it. However, this is not a hard rule,
81+
*tp_call* may be used instead.
82+
Therefore, a class supporting vectorcall must also set
83+
:c:member:`~PyTypeObject.tp_call`.
84+
Moreover, the callable must behave the same
85+
regardless of which protocol is used.
86+
The recommended way to achieve this is by setting
87+
:c:member:`~PyTypeObject.tp_call` to :c:func:`PyVectorcall_Call`.
88+
89+
An object should not implement vectorcall if that would be slower
90+
than *tp_call*. For example, if the callee needs to convert
91+
the arguments to an args tuple and kwargs dict anyway, then there is no point
92+
in implementing vectorcall.
93+
94+
For making a vectorcall, use :c:func:`_PyObject_Vectorcall`.
95+
96+
Recursion Control
97+
-----------------
98+
99+
When using *tp_call*, callees do not need to worry about
100+
:ref:`recursion <recursion>`: CPython uses
101+
:c:func:`Py_EnterRecursiveCall` and :c:func:`Py_LeaveRecursiveCall`
102+
for calls made using *tp_call*.
103+
104+
For efficiency, this is not the case for calls done using vectorcall:
105+
the callee should use *Py_EnterRecursiveCall* and *Py_LeaveRecursiveCall*
106+
if needed.
107+
108+
C API Functions
109+
---------------
110+
8111
.. c:function:: int PyCallable_Check(PyObject *o)
9112
10113
Determine if the object *o* is callable. Return ``1`` if the object is callable
@@ -147,31 +250,14 @@ Call Protocol
147250
148251
.. c:function:: PyObject* _PyObject_Vectorcall(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwnames)
149252
150-
Call a callable Python object *callable*, using
151-
:c:data:`vectorcall <PyTypeObject.tp_vectorcall_offset>` if possible.
152-
153-
*args* is a C array with the positional arguments.
154-
155-
*nargsf* is the number of positional arguments plus optionally the flag
156-
:const:`PY_VECTORCALL_ARGUMENTS_OFFSET` (see below).
157-
To get actual number of arguments, use
158-
:c:func:`PyVectorcall_NARGS(nargsf) <PyVectorcall_NARGS>`.
159-
160-
*kwnames* can be either NULL (no keyword arguments) or a tuple of keyword
161-
names, which must be strings. In the latter case, the values of the keyword
162-
arguments are stored in *args* after the positional arguments.
163-
The number of keyword arguments does not influence *nargsf*.
164-
165-
*kwnames* must contain only objects of type ``str`` (not a subclass),
166-
and all keys must be unique.
253+
Call a callable Python object *callable*.
254+
The arguments are the same as for :c:type:`vectorcallfunc`.
255+
If *callable* supports vectorcall_, this directly calls
256+
the vectorcall function stored in *callable*.
167257
168258
Return the result of the call on success, or raise an exception and return
169259
*NULL* on failure.
170260
171-
This uses the vectorcall protocol if the callable supports it;
172-
otherwise, the arguments are converted to use
173-
:c:member:`~PyTypeObject.tp_call`.
174-
175261
.. note::
176262
177263
This function is provisional and expected to become public in Python 3.9,
@@ -180,48 +266,60 @@ Call Protocol
180266
181267
.. versionadded:: 3.8
182268
183-
.. c:var:: PY_VECTORCALL_ARGUMENTS_OFFSET
269+
.. c:function:: PyObject* _PyObject_FastCallDict(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwdict)
184270
185-
If set in a vectorcall *nargsf* argument, the callee is allowed to
186-
temporarily change ``args[-1]``. In other words, *args* points to
187-
argument 1 (not 0) in the allocated vector.
188-
The callee must restore the value of ``args[-1]`` before returning.
271+
Call *callable* with positional arguments passed exactly as in the vectorcall_ protocol
272+
but with keyword arguments passed as a dictionary *kwdict*.
273+
The *args* array contains only the positional arguments.
189274
190-
For :c:func:`_PyObject_VectorcallMethod`, this flag means instead that
191-
``args[0]`` may be changed.
275+
Regardless of which protocol is used internally,
276+
a conversion of arguments needs to be done.
277+
Therefore, this function should only be used if the caller
278+
already has a dictionary ready to use for the keyword arguments,
279+
but not a tuple for the positional arguments.
192280
193-
Whenever they can do so cheaply (without additional allocation), callers
194-
are encouraged to use :const:`PY_VECTORCALL_ARGUMENTS_OFFSET`.
195-
Doing so will allow callables such as bound methods to make their onward
196-
calls (which include a prepended *self* argument) cheaply.
281+
.. note::
282+
283+
This function is provisional and expected to become public in Python 3.9,
284+
with a different name and, possibly, changed semantics.
285+
If you use the function, plan for updating your code for Python 3.9.
197286
198287
.. versionadded:: 3.8
199288
200289
.. c:function:: Py_ssize_t PyVectorcall_NARGS(size_t nargsf)
201290
202291
Given a vectorcall *nargsf* argument, return the actual number of
203292
arguments.
204-
Currently equivalent to ``nargsf & ~PY_VECTORCALL_ARGUMENTS_OFFSET``.
293+
Currently equivalent to::
294+
295+
(Py_ssize_t)(nargsf & ~PY_VECTORCALL_ARGUMENTS_OFFSET)
296+
297+
However, the function ``PyVectorcall_NARGS`` should be used to allow
298+
for future extensions.
205299
206300
.. versionadded:: 3.8
207301
208-
.. c:function:: PyObject* _PyObject_FastCallDict(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwdict)
302+
.. c:function:: vectorcallfunc _PyVectorcall_Function(PyObject *op)
209303
210-
Same as :c:func:`_PyObject_Vectorcall` except that the keyword arguments
211-
are passed as a dictionary in *kwdict*. This may be *NULL* if there
212-
are no keyword arguments.
304+
If *op* does not support the vectorcall protocol (either because the type
305+
does not or because the specific instance does not), return *NULL*.
306+
Otherwise, return the vectorcall function pointer stored in *op*.
307+
This function never sets an exception.
213308
214-
For callables supporting :c:data:`vectorcall <PyTypeObject.tp_vectorcall_offset>`,
215-
the arguments are internally converted to the vectorcall convention.
216-
Therefore, this function adds some overhead compared to
217-
:c:func:`_PyObject_Vectorcall`.
218-
It should only be used if the caller already has a dictionary ready to use.
309+
This is mostly useful to check whether or not *op* supports vectorcall,
310+
which can be done by checking ``_PyVectorcall_Function(op) != NULL``.
219311
220-
.. note::
312+
.. versionadded:: 3.8
221313
222-
This function is provisional and expected to become public in Python 3.9,
223-
with a different name and, possibly, changed semantics.
224-
If you use the function, plan for updating your code for Python 3.9.
314+
.. c:function:: PyObject* PyVectorcall_Call(PyObject *callable, PyObject *tuple, PyObject *dict)
315+
316+
Call *callable*'s :c:type:`vectorcallfunc` with positional and keyword
317+
arguments given in a tuple and dict, respectively.
318+
319+
This is a specialized function, intended to be put in the
320+
:c:member:`~PyTypeObject.tp_call` slot or be used in an implementation of ``tp_call``.
321+
It does not check the :const:`_Py_TPFLAGS_HAVE_VECTORCALL` flag
322+
and it does not fall back to ``tp_call``.
225323
226324
.. versionadded:: 3.8
227325

Doc/c-api/exceptions.rst

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -695,13 +695,17 @@ The following functions are used to create and modify Unicode exceptions from C.
695695
``0`` on success, ``-1`` on failure.
696696
697697
698+
.. _recursion:
699+
698700
Recursion Control
699701
=================
700702
701703
These two functions provide a way to perform safe recursive calls at the C
702704
level, both in the core and in extension modules. They are needed if the
703705
recursive code does not necessarily invoke Python code (which tracks its
704706
recursion depth automatically).
707+
It's also not needed for *tp_call* implementations
708+
because the :ref:`call protocol <call>` takes care of recursion handling.
705709
706710
.. c:function:: int Py_EnterRecursiveCall(const char *where)
707711

Doc/c-api/structures.rst

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -201,7 +201,8 @@ also keyword arguments. So there are a total of 6 calling conventions:
201201

202202
Extension of :const:`METH_FASTCALL` supporting also keyword arguments,
203203
with methods of type :c:type:`_PyCFunctionFastWithKeywords`.
204-
Keyword arguments are passed the same way as in the vectorcall protocol:
204+
Keyword arguments are passed the same way as in the
205+
:ref:`vectorcall protocol <vectorcall>`:
205206
there is an additional fourth :c:type:`PyObject\*` parameter
206207
which is a tuple representing the names of the keyword arguments
207208
(which are guaranteed to be strings)

Doc/c-api/type.rst

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -191,9 +191,10 @@ The following functions and structs are used to create
191191
* :c:member:`~PyTypeObject.tp_cache`
192192
* :c:member:`~PyTypeObject.tp_subclasses`
193193
* :c:member:`~PyTypeObject.tp_weaklist`
194-
* :c:member:`~PyTypeObject.tp_print`
195194
* :c:member:`~PyTypeObject.tp_weaklistoffset`
196195
* :c:member:`~PyTypeObject.tp_dictoffset`
196+
* :c:member:`~PyTypeObject.tp_vectorcall_offset`
197+
* :c:member:`~PyTypeObject.tp_vectorcall`
197198
* :c:member:`~PyBufferProcs.bf_getbuffer`
198199
* :c:member:`~PyBufferProcs.bf_releasebuffer`
199200

0 commit comments

Comments
 (0)