Skip to content

"Guest mode", for cohabitation with Qt etc. #1551

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 39 commits into from
Jun 2, 2020
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
11d7db2
First pass attempt at "guest mode"
njsmith May 25, 2020
5b75987
fix stupid missing arg in _io_windows.py
njsmith May 25, 2020
9018066
Use thread cache to vroom vroom faster
njsmith May 27, 2020
06e7523
Add a big comment explaining the trickiest change
njsmith May 27, 2020
513d744
reduce indentation
njsmith May 28, 2020
5631de4
guest mode: Add basic signal handling, and send TrioInternalError to …
njsmith May 28, 2020
b131958
Add a basic test of guest mode
njsmith May 28, 2020
91e3554
Try to adapt to py36 asyncio limitations
njsmith May 28, 2020
4340d3a
Add missing return
njsmith May 28, 2020
fd81370
3.6 asyncio is super annoying
njsmith May 28, 2020
1469d44
missing import
njsmith May 28, 2020
cc4d144
avoid some set_wakeup_fd warnings
njsmith May 28, 2020
b7a21f2
Add missing method that was causing guest mode test to hang on macOS
njsmith May 28, 2020
50322d9
Get better debug info from test_guest_mode_basic
njsmith May 28, 2020
5b7f4dd
cffi does not implicitly coerce 0 to NULL
njsmith May 28, 2020
3ec60fc
add some pragma: no covers
njsmith May 29, 2020
09783ec
add missing import
njsmith May 29, 2020
f023eea
TESTS
njsmith May 29, 2020
1bd0249
Make "assert no RuntimeWarnings" tests more reliable
njsmith May 29, 2020
ebea122
Fix indentation on some asserts
njsmith May 29, 2020
ef8a85a
Add test that guest mode properly routes TrioInternalErrors
njsmith May 29, 2020
cd86726
Replace some 'type(x) is y' with 'isinstance(x, y)'
njsmith May 29, 2020
b85176d
Small coverage tweaks
njsmith May 29, 2020
3559500
Fix test so it exercises what it's supposed to be exercising
njsmith May 29, 2020
d4343dc
Add run_sync_soon_not_threadsafe= kwarg on start_guest_run
njsmith May 30, 2020
a827cfb
First pass a comprehensive docs
njsmith Jun 1, 2020
8018cef
newsfragment
njsmith Jun 1, 2020
eea8010
black
njsmith Jun 1, 2020
7c0744b
Doc updates
njsmith Jun 1, 2020
be88548
Add note about custom clocks in guest mode
njsmith Jun 1, 2020
b84a715
Fix comment
njsmith Jun 1, 2020
9317072
Merge branch 'master' of github.com:python-trio/trio into guest-loop
njsmith Jun 1, 2020
c120632
Enable KI handling in guest mode
njsmith Jun 1, 2020
7c9ce28
Add note about guest mode KI handling to docs
njsmith Jun 1, 2020
19882c7
Clarify that you can't magically share async code between host and guest
njsmith Jun 1, 2020
7207cde
Remove defunct draft docs
njsmith Jun 2, 2020
c6697db
Remove obsolete placeholder
njsmith Jun 2, 2020
66967ce
Make kqueue code more similar to epoll code
njsmith Jun 2, 2020
3083de5
Improve wording in docs
njsmith Jun 2, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions docs/source/design.rst
Original file line number Diff line number Diff line change
Expand Up @@ -534,3 +534,77 @@ there is no provision for "pluggable" backends. The intuition here is
that we'd rather focus our energy on making one set of solid, official
backends that provide a high-quality experience out-of-the-box on all
supported systems.


Guest mode
----------

One of Trio's more unusual features is that it supports being run in
"guest mode" on top of some other event loop (the "host").

XX TODO: document this properly

the basic idea of pushing ``get_events`` into a thread

actual core logic is identical whether running in regular mode or
guest mode; alternate between waiting for I/O+timeout vs running trio tasks

one extra wrinkle is: normally tasks can only become runnable, and
deadline can only change, if trio task is running. In guest mode,
that's no longer true. So reschedule() and deadline changes need to
potentially trigger the scheduler, or at least update the I/O
deadline. Do this in the simplest possible way: force the I/O thread
to return immediately via the normal path.

subtlety around wait_all_tasks_blocked and 'events' semantics

diagram::


Normal mode

Main thread executing trio.run:

+---------------------------+
| wait for I/O+timeout |
+---------------------------+
| run trio tasks |
+---------------------------+
| wait for I/O+timeout |
+---------------------------+
| run trio tasks |
+---------------------------+
| wait for I/O+timeout |
+---------------------------+
| run trio tasks |
+---------------------------+
.
.
.


Guest mode

Main thread executing host loop: Trio I/O thread:

+---------------------------+
| host loop does its thing | +---------------------------+
| | | wait for trio I/O+timeout |
+---------------------------+ +---------------------------+
/
+---------------------------+ <---------------/
| run trio tasks |
+---------------------------+ ----------------\
\
+---------------------------+ v
| host loop does its thing | +---------------------------+
| | | wait for trio I/O+timeout |
+---------------------------+ +---------------------------+
/
+---------------------------+ <---------------/
| run trio tasks |
+---------------------------+ ----------------\
\
.
.
.
Loading