Skip to content

gh-95913: Edit, expand & format Bytecode sect in 3.11 WhatsNew #98559

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Oct 24, 2022
Merged

gh-95913: Edit, expand & format Bytecode sect in 3.11 WhatsNew #98559

Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
48a18d5
Fix Sphinx syntax in bytecode section and tweak phrasing
CAM-Gerlach Oct 20, 2022
4a3d830
Use clear, consistant phrasing on all Bytecode entries
CAM-Gerlach Oct 20, 2022
611a628
Re-order opcode changes to Added, Replaced, Removed, Changed
CAM-Gerlach Oct 20, 2022
68bef2a
Add missing added and removed opcodes to section
CAM-Gerlach Oct 21, 2022
f34698c
Add table with all added/replace/removed opcodes
CAM-Gerlach Oct 22, 2022
650b4e4
Merge opcode information from list into tables
CAM-Gerlach Oct 22, 2022
87e06bb
Add exception reprisentation change to Changed Opcodes
CAM-Gerlach Oct 23, 2022
9c165b8
Unify & simplify added and replaced tables & entries
CAM-Gerlach Oct 23, 2022
1e0d7cb
Move added opcodes out of table into a bulleted list instead
CAM-Gerlach Oct 23, 2022
a0b90d8
Re-order Replaced Opcodes table by replaced opcode
CAM-Gerlach Oct 23, 2022
4ed6cfb
Consolidate changed and removed sections to be more concise
CAM-Gerlach Oct 23, 2022
7c94ab0
Consolidate jump opcode changes further
CAM-Gerlach Oct 23, 2022
c1a7743
CHECK_EG_MATCH is new
iritkatriel Oct 24, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
120 changes: 81 additions & 39 deletions Doc/whatsnew/3.11.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1407,58 +1407,100 @@ contributors are volunteers from the community.
CPython bytecode changes
========================

* The bytecode now contains inline cache entries, which take the form of
:opcode:`CACHE` instructions. Many opcodes expect to be followed by an exact
number of caches, and instruct the interpreter to skip over them at runtime.
Populated caches can look like arbitrary instructions, so great care should be
taken when reading or modifying raw, adaptive bytecode containing quickened
data.
The bytecode now contains inline cache entries,
which take the form of the newly-added :opcode:`CACHE` instructions.
Many opcodes expect to be followed by an exact number of caches,
and instruct the interpreter to skip over them at runtime.
Populated caches can look like arbitrary instructions,
so great care should be taken when reading or modifying
raw, adaptive bytecode containing quickened data.

* Replaced all numeric ``BINARY_*`` and ``INPLACE_*`` instructions with a single
:opcode:`BINARY_OP` implementation.

* Replaced the three call instructions: :opcode:`CALL_FUNCTION`,
:opcode:`CALL_FUNCTION_KW` and :opcode:`CALL_METHOD` with
:opcode:`PUSH_NULL`, :opcode:`PRECALL`, :opcode:`CALL`,
and :opcode:`KW_NAMES`.
This decouples the argument shifting for methods from the handling of
keyword arguments and allows better specialization of calls.
.. _whatsnew311-added-opcodes:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The list we have now has all the jump related changes together, but if you split it into a "added" table and a "replaced" table then they are no longer together. I don't know if that's an improvement.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created an alternative PR -#98566. It tries to consolidate related items into a high level descriptions, rather than enumerate all the changes. Maybe this is more useful for whatsnew. The full details, if you need them, as in the dis doc.

I added an item about the exc-info related changes. Didn't do anything about these:

GET_AWAITABLE

WITH_EXCEPT_START
If the low bit of namei is set, then a NULL is pushed to the stack before the global variable.

LOAD_GLOBAL
i is no longer offset by the length of co_varnames

LOAD_CLOSURE
LOAD_DEREF
LOAD_CLASSDEREF
STORE_DEREF
DELETE_DEREF

Copy link
Member Author

@CAM-Gerlach CAM-Gerlach Oct 23, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great points, thanks. I've incorporated the changes suggested via your PR, specifically:

  • Consolidated the jump opcode additions, replacements and changes (except for the NO_INTERRUPT opcode addition, which didn't appear to be closely related to the others) into a single Replaced table entry with a linked note
  • Just linked the high-level jump opcode name patterns, instead of every opcode.
  • Added a "Changed" note about the exception stack representation
  • (EXC/EG changes were already consolidated)

Beyond that, I've given this PR a major revamp based on your feedback above:

  • Converted the "Added" section back to (much shorter) bullets
  • Inlined the previous separate notes into the Replaced table
  • Combined the previous "Removed" and "Changed" sections into one, condensing the Removed items into a single bullet to take up much less space
  • Made additional textual refinements, particularly in the Changed entries, to be both more clear and more concise

In total, this retains nearly the same space consumption of the previous dense wall of flat bullets format (once allowing for the missing substantive additions, replacements and removals added here), while being much easier for the reader to quickly navigate, find what they're looking for and jump to more information in the dis docs.

Flat Bullets Initial PR Revised PR
image image image


* Removed ``COPY_DICT_WITHOUT_KEYS`` and ``GEN_START``.
New opcodes
-----------

* :opcode:`ASYNC_GEN_WRAP`, :opcode:`RETURN_GENERATOR` and :opcode:`SEND`,
used in generators and co-routines.

* :opcode:`MATCH_CLASS` and :opcode:`MATCH_KEYS` no longer push an additional
boolean value indicating whether the match succeeded or failed. Instead, they
indicate failure with :const:`None` (where a tuple of extracted values would
otherwise be).
* :opcode:`COPY_FREE_VARS`,
which avoids needing special caller-side code for closures.

* Replace several stack manipulation instructions (``DUP_TOP``, ``DUP_TOP_TWO``,
``ROT_TWO``, ``ROT_THREE``, ``ROT_FOUR``, and ``ROT_N``) with new
:opcode:`COPY` and :opcode:`SWAP` instructions.
* :opcode:`JUMP_BACKWARD_NO_INTERRUPT`,
for use in certain loops where handling interrupts is undesirable.

* Replaced :opcode:`JUMP_IF_NOT_EXC_MATCH` by :opcode:`CHECK_EXC_MATCH` which
performs the check but does not jump.
* :opcode:`MAKE_CELL`, to create :ref:`cell-objects`.

* Replaced :opcode:`JUMP_IF_NOT_EG_MATCH` by :opcode:`CHECK_EG_MATCH` which
performs the check but does not jump.
* :opcode:`CHECK_EG_MATCH` and :opcode:`PREP_RERAISE_STAR`,
to handle the :ref:`new exception groups and except* <whatsnew311-pep654>`
added in :pep:`654`.

* Replaced :opcode:`JUMP_ABSOLUTE` by the relative :opcode:`JUMP_BACKWARD`.
* :opcode:`PUSH_EXC_INFO`, for use in exception handlers.

* Added :opcode:`JUMP_BACKWARD_NO_INTERRUPT`, which is used in certain loops where it
is undesirable to handle interrupts.
* :opcode:`RESUME`, a no-op,
for internal tracing, debugging and optimization checks.

* Replaced :opcode:`POP_JUMP_IF_TRUE` and :opcode:`POP_JUMP_IF_FALSE` by
the relative :opcode:`POP_JUMP_FORWARD_IF_TRUE`, :opcode:`POP_JUMP_BACKWARD_IF_TRUE`,
:opcode:`POP_JUMP_FORWARD_IF_FALSE` and :opcode:`POP_JUMP_BACKWARD_IF_FALSE`.

* Added :opcode:`POP_JUMP_FORWARD_IF_NOT_NONE`, :opcode:`POP_JUMP_BACKWARD_IF_NOT_NONE`,
:opcode:`POP_JUMP_FORWARD_IF_NONE` and :opcode:`POP_JUMP_BACKWARD_IF_NONE`
opcodes to speed up conditional jumps.
.. _whatsnew311-replaced-opcodes:

* :opcode:`JUMP_IF_TRUE_OR_POP` and :opcode:`JUMP_IF_FALSE_OR_POP` are now
relative rather than absolute.
Replaced opcodes
----------------

* :opcode:`RESUME` has been added. It is a no-op. Performs internal tracing,
debugging and optimization checks.
+------------------------------------+-----------------------------------+-----------------------------------------+
| Replaced Opcode(s) | New Opcode(s) | Notes |
+====================================+===================================+=========================================+
| | :opcode:`!BINARY_*` | :opcode:`BINARY_OP` | Replaced all numeric binary/in-place |
| | :opcode:`!INPLACE_*` | | opcodes with a single opcode |
+------------------------------------+-----------------------------------+-----------------------------------------+
| | :opcode:`!CALL_FUNCTION` | | :opcode:`CALL` | Decouples argument shifting for methods |
| | :opcode:`!CALL_FUNCTION_KW` | | :opcode:`KW_NAMES` | from handling of keyword arguments; |
| | :opcode:`!CALL_METHOD` | | :opcode:`PRECALL` | allows better specialization of calls |
| | | :opcode:`PUSH_NULL` | |
+------------------------------------+-----------------------------------+-----------------------------------------+
| | :opcode:`!DUP_TOP` | | :opcode:`COPY` | Stack manipulation instructions |
| | :opcode:`!DUP_TOP_TWO` | | :opcode:`SWAP` | |
| | :opcode:`!ROT_TWO` | | |
| | :opcode:`!ROT_THREE` | | |
| | :opcode:`!ROT_FOUR` | | |
| | :opcode:`!ROT_N` | | |
+------------------------------------+-----------------------------------+-----------------------------------------+
| | :opcode:`!JUMP_IF_NOT_EXC_MATCH` | | :opcode:`CHECK_EXC_MATCH` | Now performs check but doesn't jump |
+------------------------------------+-----------------------------------+-----------------------------------------+
| | :opcode:`!JUMP_ABSOLUTE` | | :opcode:`JUMP_BACKWARD` | See [#bytecode-jump]_; |
| | :opcode:`!POP_JUMP_IF_FALSE` | | :opcode:`POP_JUMP_BACKWARD_IF_* | ``TRUE``, ``FALSE``, |
| | :opcode:`!POP_JUMP_IF_TRUE` | <POP_JUMP_BACKWARD_IF_TRUE>` | ``NONE`` and ``NOT_NONE`` variants |
| | | :opcode:`POP_JUMP_FORWARD_IF_* | for each direction |
| | <POP_JUMP_FORWARD_IF_TRUE>` | |
+------------------------------------+-----------------------------------+-----------------------------------------+
| | :opcode:`!SETUP_WITH` | :opcode:`BEFORE_WITH` | :keyword:`with` block setup |
| | :opcode:`!SETUP_ASYNC_WITH` | | |
+------------------------------------+-----------------------------------+-----------------------------------------+

.. [#bytecode-jump] All jump opcodes are now relative, including the
existing :opcode:`JUMP_IF_TRUE_OR_POP` and :opcode:`JUMP_IF_FALSE_OR_POP`.
The argument is now an offset from the current instruction
rather than an absolute location.


.. _whatsnew311-changed-opcodes:
.. _whatsnew311-removed-opcodes:
.. _whatsnew311-changed-removed-opcodes:

Changed/removed opcodes
-----------------------

* Changed :opcode:`MATCH_CLASS` and :opcode:`MATCH_KEYS`
to no longer push an additional boolean value to indicate success/failure.
Instead, ``None`` is pushed on failure
in place of the tuple of extracted values.

* Changed opcodes that work with exceptions to reflect them
now being represented as one item on the stack instead of three
(see :gh:`89874`).

* Removed :opcode:`!COPY_DICT_WITHOUT_KEYS`, :opcode:`!GEN_START`,
:opcode:`!POP_BLOCK`, :opcode:`!SETUP_FINALLY` and :opcode:`!YIELD_FROM`.


.. _whatsnew311-deprecated:
Expand Down