Skip to content

gh-107557: Setup abstract interpretation #107847

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 54 commits into from
Aug 15, 2023
Merged
Show file tree
Hide file tree
Changes from 44 commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
d20fbb8
gh-107557: Tier 2 abstract interpreter barebones
Fidget-Spinner Aug 2, 2023
2aeea51
📜🤖 Added by blurb_it.
blurb-it[bot] Aug 2, 2023
1a728ab
Copy Guido's input and output code, and fix build
Fidget-Spinner Aug 2, 2023
17fccbc
fix separator
Fidget-Spinner Aug 2, 2023
a1da69d
credit Jules
Fidget-Spinner Aug 2, 2023
b458e17
add jules to co-authors
Fidget-Spinner Aug 2, 2023
f81f888
add pycore_optimizer.h to headers in makefile
Fidget-Spinner Aug 2, 2023
0020320
fix: remove whitespace
Fidget-Spinner Aug 2, 2023
1f93072
fix make smelly
Fidget-Spinner Aug 2, 2023
dac63e3
fix: build
Fidget-Spinner Aug 2, 2023
e62e015
fix wrong symbol
Fidget-Spinner Aug 2, 2023
a7f654c
ignore static globals check for abstract interpreter
Fidget-Spinner Aug 2, 2023
f4040b8
Merge remote-tracking branch 'upstream/main' into abstract_interpreter
Fidget-Spinner Aug 4, 2023
ec58145
merge Guido's changes
Fidget-Spinner Aug 4, 2023
4292767
remove unused stuff
Fidget-Spinner Aug 4, 2023
fdcca90
Turn on the abstract interpreter
Fidget-Spinner Aug 4, 2023
5110fb9
Merge remote-tracking branch 'upstream/main' into abstract_interpreter
Fidget-Spinner Aug 5, 2023
9f443a2
Merge remote-tracking branch 'origin/abstract_interpreter' into parti…
Fidget-Spinner Aug 5, 2023
7632ed1
(leaky) data structures for constant propagation
Fidget-Spinner Aug 5, 2023
0d0c4c4
(with cycles) try to fix the type prop
Fidget-Spinner Aug 6, 2023
4c8953e
fix: cycles
Fidget-Spinner Aug 6, 2023
3bd36fa
cleanup
Fidget-Spinner Aug 6, 2023
229097f
Fix+Refactor: Handling of root nodes in special-cased type prop (#40)
JuliaPoo Aug 7, 2023
ca0fab7
partially partially evaluate
Fidget-Spinner Aug 8, 2023
68c684f
rename vars
Fidget-Spinner Aug 8, 2023
46c5777
fixx off by one
Fidget-Spinner Aug 8, 2023
b839ee4
partial eval working for real this time
Fidget-Spinner Aug 9, 2023
6ecf3d2
Fix: Inconsistent `AbstractInterpContext` used in `PARTITIONNODE_OVER…
JuliaPoo Aug 9, 2023
b6eeb25
fix test, refactor, bugfix
Fidget-Spinner Aug 9, 2023
d5cceb9
re-compute jump offsets and targets
Fidget-Spinner Aug 10, 2023
8c0d65f
Fix+Refactor: Extra EXIT_TRACE emitted (#42)
JuliaPoo Aug 10, 2023
95db909
fix: overallocate buffer and virtual/real stack offset calculation
Fidget-Spinner Aug 10, 2023
1e05ef8
more bugfix
Fidget-Spinner Aug 10, 2023
d7d8b52
Merge remote-tracking branch 'upstream/main' into partition_algo
Fidget-Spinner Aug 10, 2023
4d7abc7
Perf+Cleanup: Removed temporary allocation in `remove_duplicate_save_…
JuliaPoo Aug 11, 2023
3d76f9a
clean up code
Fidget-Spinner Aug 11, 2023
e81def2
Merge branch 'partition_algo' of https://github.com/Fidget-Spinner/cp…
Fidget-Spinner Aug 11, 2023
9a5a3f7
make static
Fidget-Spinner Aug 11, 2023
df490d0
make types static
Fidget-Spinner Aug 11, 2023
1e4fc94
make const and ignore in c analyzer
Fidget-Spinner Aug 11, 2023
6de77a7
fix c-analyzer ignored list
Fidget-Spinner Aug 11, 2023
a11fc80
more cleanup
Fidget-Spinner Aug 13, 2023
c61015b
Merge remote-tracking branch 'upstream/main' into partition_algo
Fidget-Spinner Aug 13, 2023
56c62eb
regen files
Fidget-Spinner Aug 13, 2023
3c08ebe
address review
Fidget-Spinner Aug 14, 2023
d5f16be
regen
Fidget-Spinner Aug 14, 2023
1e61c49
and env var to block tests
Fidget-Spinner Aug 14, 2023
6c24b49
regen again
Fidget-Spinner Aug 14, 2023
2be404d
fix generated files
Fidget-Spinner Aug 14, 2023
29e255d
Address review
Fidget-Spinner Aug 15, 2023
3c44117
fix up INSERT
Fidget-Spinner Aug 15, 2023
b758b47
remove experimental parts
Fidget-Spinner Aug 15, 2023
80c7f18
revert more changes
Fidget-Spinner Aug 15, 2023
6a2b204
use memmove
Fidget-Spinner Aug 15, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@ Programs/test_frozenmain.h generated
Python/Python-ast.c generated
Python/executor_cases.c.h generated
Python/generated_cases.c.h generated
Python/abstract_interp_cases.c.h generated
Python/opcode_targets.h generated
Python/stdlib_module_names.h generated
Tools/peg_generator/pegen/grammar_parser.py generated
Expand Down
2 changes: 1 addition & 1 deletion Include/cpython/optimizer.h
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ typedef struct _PyExecutorObject {
typedef struct _PyOptimizerObject _PyOptimizerObject;

/* Should return > 0 if a new executor is created. O if no executor is produced and < 0 if an error occurred. */
typedef int (*optimize_func)(_PyOptimizerObject* self, PyCodeObject *code, _Py_CODEUNIT *instr, _PyExecutorObject **);
typedef int (*optimize_func)(_PyOptimizerObject* self, PyCodeObject *code, _Py_CODEUNIT *instr, _PyExecutorObject **, int curr_stackentries);

typedef struct _PyOptimizerObject {
PyObject_HEAD
Expand Down
142 changes: 142 additions & 0 deletions Include/internal/pycore_opcode_metadata.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 20 additions & 0 deletions Include/internal/pycore_optimizer.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
#ifndef Py_INTERNAL_OPTIMIZER_H
#define Py_INTERNAL_OPTIMIZER_H
#ifdef __cplusplus
extern "C" {
#endif

#ifndef Py_BUILD_CORE
# error "this header requires Py_BUILD_CORE define"
#endif

#include "pycore_uops.h"

int _Py_uop_analyze_and_optimize(PyCodeObject *code,
_PyUOpInstruction *trace, int trace_len, int curr_stackentries);


#ifdef __cplusplus
}
#endif
#endif /* !Py_INTERNAL_OPTIMIZER_H */
6 changes: 3 additions & 3 deletions Include/internal/pycore_uops.h
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,11 @@ extern "C" {
# error "this header requires Py_BUILD_CORE define"
#endif

#define _Py_UOP_MAX_TRACE_LENGTH 32
#define _Py_UOP_MAX_TRACE_LENGTH 256

typedef struct {
uint32_t opcode;
uint32_t oparg;
int32_t opcode;
int32_t oparg;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious about this change. Are you using negative opcodes or opargs? IIUC oparg really is treated as a 32-bit unsigned int in ceval.c and bytecodes.c.

Copy link
Member Author

@Fidget-Spinner Fidget-Spinner Aug 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm using both negative ints and opargs, but only during the analysis phase. They are converted back to unsigned after the last pass.

For all intents and purposes outside of the optimizer pass, they can be treated as unsigned.

uint64_t operand; // A cache entry
} _PyUOpInstruction;

Expand Down
24 changes: 24 additions & 0 deletions Lib/test/test_capi/test_misc.py
Original file line number Diff line number Diff line change
Expand Up @@ -2619,5 +2619,29 @@ def testfunc(it):
next(it)


class TestUopsOptimization(unittest.TestCase):

def test_int_constant_propagation(self):
def testfunc(loops):
num = 0
while num < loops:
x = 0
y = 1
z = 2
a = x + y + z + x + y + z + x + y + z
num += 1
return a

opt = _testinternalcapi.get_uop_optimizer()
res = None
with temporary_optimizer(opt):
res = testfunc(3)

ex = get_first_executor(testfunc)
self.assertIsNotNone(ex)
self.assertEqual(res, 9)
binop_count = [opname for opname, _, _ in ex if opname == "_BINARY_OP_ADD_INT"]
self.assertEqual(len(binop_count), 1)

if __name__ == "__main__":
unittest.main()
Loading