Are we really needed a smart statement timeout?

Intro

If we make 'JOB' benchmark results with specific preferences (script):

fsync = off
max_parallel_workers_per_gather = 0
statement_timeout = 600000
from_collapse_limit = 20
join_collapse_limit = 20

we will see the result on the graph below (gnuplot template, raw data):

As you can see, we have 15 queries, that can't be executed in reasonable time: 14a.sql, 14c.sql, 21a.sql, 21b.sql, 22a.sql, 22b.sql, 22c.sql, 22d.sql, 25a.sql, 25c.sql, 30a.sql, 30c.sql, 31a.sql, 31b.sql, 31c.sql. The list slightly changes at another computer (Why?).

This situation looks better with the parallel workers machinery usage, but for clarity we turned it off. What if we had a machinery to learn on a partial query plan, interrupted by a statement_timeout? We should have a chance to recognize situations, when we were interrupted under AQO control and have an option to make one more attempt with, possible, different query plan.

Use learning on partial plans

After enabling the partial learning feature (aqo.learn_query_statement = 'on') we got such a picture for the timed out queries:

Test Number	Query	Execution time, s	Interrupts
15	14a.sql	8	1
17	14c.sql	8	1
32	18a.sql	13	0+1
37	19c.sql	18	0+1
46	21a.sql	7	3+1
47	21b.sql	5	1
49	22a.sql	7	3+1+1
51	22c.sql	418	7
52	22d.sql	10	3+2
58	25a.sql	12	1+1
60	25c.sql	16	4
67	28a.sql	306	0+1
70	29a.sql	14	0+1+1
72	29c.sql	14	0+2+2+2+1
77	30a.sql	13	1+1
78	30b.sql	13	1
79	30c.sql	15	1+3+1
80	31a.sql	14	1+5+2
81	31b.sql	86	1+1+1
82	31c.sql	15	3+5+1
103	7a.sql	9	1+1

A set of timed out queries slightly changed (why?). All queries fit into 10min limit now. Interruptions in the table - number of learnings on timed out queries. Learning process on some queries increased execution time. So, after successful execution we observe interruptions too. In the table we use '+' to show the fact of timed out query after successful execution. Also, for the most part of queries optimizer has found a good solution.

Speedup factor as a result of AQO learning procedure shown at the picture below.