Runner errors for Python 3.7 when done

[basnijholt]

Exception in thread QueueManagerThread:
Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/opt/conda/lib/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/conda/lib/python3.7/concurrent/futures/process.py", line 368, in _queue_management_worker
    thread_wakeup.clear()
  File "/opt/conda/lib/python3.7/concurrent/futures/process.py", line 92, in clear
    while self._reader.poll():
  File "/opt/conda/lib/python3.7/multiprocessing/connection.py", line 255, in poll
    self._check_closed()
  File "/opt/conda/lib/python3.7/multiprocessing/connection.py", line 136, in _check_closed
    raise OSError("handle is closed")
OSError: handle is closed

[basnijholt]

Might actually be a bug in Python 3.7: https://bugs.python.org/issue34073
Related: https://bugs.python.org/issue34075

However, it's not exactly the same

# Bug occurs in this way
import asyncio
from concurrent.futures import ProcessPoolExecutor
import os
loop = asyncio.get_event_loop_policy().new_event_loop()
loop.set_default_executor(ProcessPoolExecutor())
future = loop.run_in_executor(None, os.getpid)
print(loop.run_until_complete(future))
loop.close()

# We're calling it like this with no error printed
import asyncio
from concurrent.futures import ProcessPoolExecutor
import os
loop = asyncio.get_event_loop_policy().new_event_loop()
ex = ProcessPoolExecutor()
future = loop.run_in_executor(ex, os.getpid)
print(loop.run_until_complete(future))
loop.close()

[basnijholt]

The following code is causing the bug:

import random
import concurrent.futures
import asyncio

executor = concurrent.futures.ProcessPoolExecutor()
ioloop = asyncio.get_event_loop()

async def _run():
    result = await ioloop.run_in_executor(executor, random.random)
    executor.shutdown(wait=False)  # if `wait=True` the problem is gone

task = ioloop.create_task(_run())

[basnijholt]

I've opened an issue in the Python bug tracker: https://bugs.python.org/issue36281

[basnijholt]

Closed by #164.

This should be changed back in the future, see #173.