gh-132732: Automatically constant evaluate pure operations

[Fidget-Spinner]

• Issue: Constant evaluate/propagate pure ops automatically #132732

[python-cla-bot]

All commit authors signed the Contributor License Agreement.

[brandtbucher]

This is really neat!

Other than two opcodes I found that shouldn't be marked pure, I just have one thought:

Rather than rewriting the bodies like this to use the symbols-manipulating functions (which seems error-prone), would we be able to just use stackrefs to do this?

For example, _BINARY_OP_ADD_INT is defined like this:

PyObject *left_o = PyStackRef_AsPyObjectBorrow(left);
PyObject *right_o = PyStackRef_AsPyObjectBorrow(right);
// ...
res = PyStackRef_FromPyObjectSteal(res_o);

Rather than rewriting uses of these functions, could it be easier to just do something like this, since we're guranteed not to escape?

if (sym_is_const(ctx, stack_pointer[-2]) && sym_is_const(ctx, stack_pointer[-1])) {
    // Generated code to turn constant symbols into stackrefs:
    _PyStackRef left = PyStackRef_FromPyObjectBorrow(sym_get_const(ctx, stack_pointer[-2]));
    _PyStackRef right = PyStackRef_FromPyObjectBorrow(sym_get_const(ctx, stack_pointer[-1]));
    _PyStackRef res;
    // Now the actual body, same as it appears in executor_cases.c.h:
    PyObject *left_o = PyStackRef_AsPyObjectBorrow(left);
    PyObject *right_o = PyStackRef_AsPyObjectBorrow(right);
    // ...
    res = PyStackRef_FromPyObjectSteal(res_o);
    // Generated code to turn stackrefs into constant symbols:
    stack_pointer[-1] = sym_new_const(ctx, PyStackRef_AsPyObjectSteal(res));
}

I'm not too familiar with the design of the cases generator though, so maybe this is way harder or something. Either way, I'm excited to see this get in!

[Fidget-Spinner]

Rather than rewriting uses of these functions, could it be easier to just do something like this, since we're guranteed not to escape?

Seems feasible. I could try to rewrite all occurences of the variable with a stackref-producing const one. Let me try that.

[Fidget-Spinner]

I've verified no refleak on test_capi.test_opt locally apart from #132731 which is pre-existing.

[markshannon]

There's a lot going on in this PR, probably too much for one PR.

Could we start with a PR to fix up the pure annotations so that they are on the correct instructions and maybe add the pure_guard annotation that Brandt suggested?

[markshannon]

Could we have the default code generator generate a function for the body of the pure instruction and then call that from the three interpreters?

[brandtbucher]

Could we have the default code generator generate a function for the body of the pure instruction and then call that from the three interpreters?

Hm, I think I’d prefer not to. Sounds like it could hurt performance, especially for the JIT (where things can’t inline).

[brandtbucher]

I think a good progression would be:

• Implement the pure attribute, and the optimizer changes. Remove the pure attributes where they don’t belong (so nothing breaks) and leave the existing ones as proof that the implementation works. (This PR)
• Audit the existing non-pure bytecodes and add pure where it makes sense. (Follow-up PR)
• Implement the pure_guard attribute, and annotate any bytecodes that can use it. (Follow-up PR)

[Fidget-Spinner]

Could we have the default code generator generate a function for the body of the pure instruction and then call that from the three interpreters?

Hm, I think I’d prefer not to. Sounds like it could hurt performance, especially for the JIT (where things can’t inline).

I thought about this and I think we can inline if we autogenerate a header file and include that directly. But then we're at the mercy of the compiler in both the normal interpreter and the JIT deciding to inline or not to inline the body again. Which I truly do not want.

[Fidget-Spinner]

@brandtbucher @markshannon what can I do to get this PR moving?

@tomasr8 if youd like to review, here's a summary of the PR:

1. If a bytecode operation is pure (no side effects) we can mark it as pure in bytecodes.c.
2. In the optimizer, we automatically generate the body that does evaluation of the symbolic constants by copy pasting the bytecodes.c definition into the optimizer's C code. Of course we check that the inputs are constants first.
3. All changes to the cases generator is for the second point.

[tomasr8]

Thanks for the ping! I actually wanted to try/review this PR, I was just very busy this week with work :/ I'll have a look this weekend :)

[tomasr8]

Only had time to skim the PR, I'll do a more thorough review this weekend :)

[markshannon]

Unfortunately this approach has a critical flaw. It is possible for the optimizer to see values that the executing code never would. For example, through a combination of statistical branch profiling and global to constant conversion.

class Disaster:
    def __add__(self, other):
        halt_and_catch_fire()

We don't want to be evaluating Disaster() +1 when optimizing BINARY_OP_ADD_INT.

Maybe consider the approach used for TO_BOOL where we call optimize_to_bool for each family member, thus reducing the code duplication.

In addition, we could then optimize BINARY_OP. After all, 1 + 1 is always 2, not just for BINARY_OP_ADD_INT.

[bedevere-app]

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

And if you don't make the requested changes, you will be poked with soft cushions!

[Fidget-Spinner]

We don't want to be evaluating Disaster() +1 when optimizing BINARY_OP_ADD_INT

In the first place, that's not possible. We only optimize what we specialize in the interpreter. The interpreter will never specialize that to BINARY_OP_ADD_INT. Furthermore, even if it did specialize Disaster() to that by some oddity, we check _GUARD_TOS_INT and _GUARD_NOS_INT first in the specializer. Disaster() would never pass these checks.

Unfortunately this approach has a critical flaw. It is possible for the optimizer to see values that the executing code never would. For example, through a combination of statistical branch profiling and global to constant conversion.

If you want to be more assured, how about I merge #132968 to add type assertions to our optimizer? That should make things safer.

[Fidget-Spinner]

We don't want to be evaluating Disaster() +1 when optimizing BINARY_OP_ADD_INT

In the first place, that's not possible. We only optimize what we specialize in the interpreter. The interpreter will never specialize that to BINARY_OP_ADD_INT. Furthermore, even if it did specialize Disaster() to that by some oddity, we check _GUARD_TOS_INT and _GUARD_NOS_INT first in the specializer. Disaster() would never pass these checks.

I forgot the guards don't actually guard at the optimization time. My bad. Yeah it seems we need some sort of check.

[Fidget-Spinner]

@tomasr8 sorry this is going to make your life harder with the removing sym_is_const and sym_matches_type PR. I think you can still remove sym_matches_type, but sym_is_const is now a little harder.

[Fidget-Spinner]

I have made the requested changes; please review again

[bedevere-app]

Thanks for making the requested changes!

@markshannon: please review the changes made to this pull request.

Addressed review.

[Fidget-Spinner]

Note: I mark the object as stackref immortal (but not real immortal!) to simplify the reference management. This means we have no refcounting in the optimizer when constant evaluating stuff. Which makes things easier to reason about, as constants during the lifetime of the optimizer are effectively immortal anyways (the optimizer holds a single reference to all constants).

[Fidget-Spinner]

Waiting for #134284 to be merged first, then I can use PyStackRef_FromPyObjectBorrow

[brandtbucher]

It didn't really review the cases generator too closely (since I'm still not very familiar with it) but based on the code it generates, everything at least seems correct.

[brandtbucher]

Suggested change

[brandtbucher]

We shouldn't constant-evaluate anything involving mutable containers. (Even tuple scares me a tiny bit, since it can contain arbitrary objects, but I'm pretty sure it's okay.)

Suggested change

	return (typ == &PyLong_Type) ||
	(typ == &PyUnicode_Type) ||
	(typ == &PyFloat_Type) ||
	(typ == &PyDict_Type) ||
	(typ == &PyTuple_Type) ||
	(typ == &PyList_Type);
	return (typ == &_PyNone_Type) ||
	(typ == &PyBool_Type) ||
	(typ == &PyLong_Type) ||
	(typ == &PyFloat_Type) ||
	(typ == &PyUnicode_Type) ||
	(typ == &PyTuple_Type);

[brandtbucher]

Add this back?

[brandtbucher]

Minor: maybe use // instead of /*/*/ for these comments, since they're not multi-line?

[brandtbucher]

It may just be a week of conference sleep schedule, but my brain hurts trying to reason about the refcounting here. Why are we stealing a borrow? Shouldn't we be stealing a steal, or borrowing a borrow? Currently:

• If the tag bit is unset on the stackref, stealing a borrow will leave the refcount on the object unchanged and the tag bit unset. When the symbol is cleared after optimizing, the refcount on the object will be one less, which is correct.
• If the tag bit is set on the stackref, stealing a borrow will leave the refcount on the object itself unchanged and the tag bit still set. When the symbol is cleared after optimizing, the refcount on the object will be one less, which seems incorrect.

(I haven't looked at the peek code yet.)

Another option, that I might like better, is making all of our constant symbols use stackrefs under-the-hood. Then we could avoid refcounting entirely. But that's a bigger change that could happen later if needed.

[brandtbucher]

This is also making me realize that we really should make it possible to detect refleaks/memory leaks on JIT builds soon. The problem is that new executors are allocated all over the place, leading to things like #120501.