event.headers returns array of bytearray since 0.8.0

[DaanDeMeyer]

In the process of upgrading wsproto to h11 0.8.0 I noticed that the _normed_header_dict method https://github.com/python-hyper/wsproto/blob/cbb93a34808cf4894914e91df81e99f957f2753d/wsproto/connection.py#L49 (which receives headers directly from h11 without any transformations as far as I'm aware) receives an array of strings when using h11 0.7.0 and an array of bytearrays when using h11 0.8.0 which breaks wsproto when using h11 0.8.0.

[njsmith]

So like I noted over in the wsproto PR, it's probably some kind of bug in h11, but I can't seem to reproduce this behavior myself:

In [1]: import h11

In [2]: h11.__version__
Out[2]: '0.8.0'

In [3]: c = h11.Connection(our_role=h11.SERVER)

In [4]: c.receive_data(b"GET / HTTP/1.1\r\nHost: example.com\r\n\r\n")

In [5]: c.next_event()
Out[5]: Request(method=b'GET', target=b'/', headers=[(b'host', b'example.com')], http_version=b'1.1')

Can you give more details on what version of Python you're using? Do you have an example that demonstrates this?

[DaanDeMeyer]

System information:

• OS: Linux
• Distro: Arch
• Python: python-2.7.14

When I execute the same code I do get bytearrays:

[pquentin]

I can confirm that this issue only occurs on Python 2.

[DaanDeMeyer]

@njsmith I found the cause. The validate function in _util.py causes the issue. I've gathered the related code here (run with python2 and python3 to see the differences in output):

import re
request_line_re = re.compile("(?P<method>[-!#$%&'*+.^_`|~0-9a-zA-Z]+) (?P<target>[^ ]+) HTTP/(?P<http_version>[0-9]\\.[0-9])".encode("ascii"))

try:
    _fullmatch = type(re.compile('')).fullmatch
except AttributeError:
    def _fullmatch(regex, data):  # version specific: Python < 3.4
        match = regex.match(data)
        if match and match.end() != len(data):
            match = None
        return match

def validate(regex, data, msg="malformed data", *format_args):
    match = _fullmatch(regex, data)
    if not match:
        if format_args:
            msg = msg.format(*format_args)
        raise LocalProtocolError(msg)
    test = match.groupdict()
    return test

result = validate(request_line_re, bytearray(b'GET / HTTP/1.1'))
print(result)

validate is called from maybe_read_from_IDLE_client and in the EventBundle constructor the headers are never converted back to bytestrings since _parsed is set to true when calling the Request constructor in maybe_read_from_IDLE_client.

[njsmith]

#64