Change unittest's _SubTest to not sort its params when printing test failures

[ericvsmith]

BPO	30664
Nosy	@ericvsmith, @serhiy-storchaka, @mlouielu, @nitishch
PRs	bpo-30664: The description of a unittest subtest now preserves the #2265

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2017-06-23.18:52:10.362>
created_at = <Date 2017-06-14.12:00:09.821>
labels = ['3.7', 'easy', 'type-feature', 'library']
title = "Change unittest's _SubTest to not sort its params when printing test failures"
updated_at = <Date 2017-06-23.18:52:10.361>
user = 'https://github.com/ericvsmith'

bugs.python.org fields:

activity = <Date 2017-06-23.18:52:10.361>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2017-06-23.18:52:10.362>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2017-06-14.12:00:09.821>
creator = 'eric.smith'
dependencies = []
files = []
hgrepos = []
issue_num = 30664
keywords = ['easy']
message_count = 9.0
messages = ['295998', '296001', '296002', '296003', '296004', '296005', '296261', '296263', '296736']
nosy_count = 4.0
nosy_names = ['eric.smith', 'serhiy.storchaka', 'louielu', 'nitishch']
pr_nums = ['2265']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue30664'
versions = ['Python 3.7']

[ericvsmith]

Now that **kwargs are sorted, it would be better if the error given by a unittest subTest (as implemented in uniitest._case._SubTest) didn't sort the parameters when they're printed, but instead printed them out in order.

This might be complicated by the ChainMap that's used as part of the implementation, but it should still be doable.

For example, I have code that has:

    with self.subTest(hash=hash, cmp=cmp, frozen=frozen):

But when it errors out, it produces:

FAIL: test_hash_rules (tst.TestCase) (cmp=True, frozen=True, hash=None)

It would be easier to check my code if the order the values was printed was the same as the order in the self.subTest() call.

[ericvsmith]

Correction: it's implemented in unittest.case._SubTest, specifically in _subDescription().

[mlouielu]

I think the question will be, when using multiple subTest (this is why using ChainMap I think), how to determine their order?:

    with self.subTest(c=i, b=i, a=i):
        with self.subTest(b=i, c=50, a=60):
	    self.assertEqual(i, 2)

>> FAIL: test_foo (main.Test) (a=60, b=4, c=50)

[ericvsmith]

Good question.

It looks like ChainMap does something I wouldn't expect:

>>> for k, v in ChainMap({'a': 0, 'b': 1, 'c': 2}, {'b': 3, 'a': 4}).items():
...  print(k, v)
... 
b 1
a 0
c 2

Once we define what we'd like the output to look like, I'm sure it would be easy enough to get the order right.

[mlouielu]

Additional note, the order will changed it random way in ChainMap, e.g.:

>>> for k, v in ChainMap({'a': 0, 'b': 1, 'c': 2}, {'b': 3, 'a': 4}).items(): print(k, v)
...
a 0
c 2
b 1

-----restart----

>>> for k, v in ChainMap({'a': 0, 'b': 1, 'c': 2}, {'b': 3, 'a': 4}).items(): print(k, v)
...
b 1
c 2
a 0

[ericvsmith]

Correct on the order changed with regular dicts. That's why I'm targeting this specifically for Python 3.7 and with **kwargs, where order is guaranteed. We might have to use a structure other than a ChainMap of dicts, like a ChainMap of OrderDicts.

[serhiy-storchaka]

PR 2265 implements a private subclass of ChainMap that preserves ordering.

[serhiy-storchaka]

Note that the order of parameters of nested subtests is from inner to outer.

[serhiy-storchaka]

New changeset 48fbe52 by Serhiy Storchaka in branch 'master':
bpo-30664: The description of a unittest subtest now preserves the (bpo-2265)
48fbe52