Question: How to conditionally skip a generated test case

[wonboyn]

Hi there,

I have a test case function that's using the @parametrize decorator:
`
@parametrize(name=(HelperUtil.get_all_widgets()))

def case_widget(self, name):
return name
`

Then on the test function I have @parametrize_with_cases:
`
@parametrize_with_cases('val', cases=MyClassName)

def test_function(val):
`

This is working perfectly & processing the dozen odd values that get returned from @parametrize.
However, I'd like the test function that uses this to skip certain "name" values (eg skip if name in ['fred', 'harry', 'mick']

Based on your doco, I think the @parametrize_with_cases(filter=) option using regex can probably do this.
Can this be achieved with filters & if so what's the format of id's that get returned from @parametrize?

Thanks in advance.

Tim

[smarie]

Thanks for this question @wonboyn

We'll need a complete reproducible example to discuss. Let's use this one:

from pytest_cases import parametrize, parametrize_with_cases

class MyClassName:
    @parametrize(name=("joe", "bob", "alice"))
    def case_widget(self, name):
        return name

    def case_foo(self):
        return "foo"

@parametrize_with_cases('val', cases=MyClassName)
def test_function(val):
    print(val)

This generates 4 test nodes:

test_issue_211.py::test_function[widget-name=joe] 
test_issue_211.py::test_function[widget-name=bob] 
test_issue_211.py::test_function[widget-name=alice] 
test_issue_211.py::test_function[foo] 

The first 3 test nodes use the case widget while the last one uses case foo.

Your question is about selecting both foo and some parametrized versions of widget, for example not widget "bob".

Filtering cases: KO

Unfortunately, filtering cases (using filters, tags, globs as described in here) applies solely on the cases directly collected by @parametrize_with_cases. So you can use it to select widget + foo, or only widget, or only foo, or none of them ; however if widget is selected, all of its versions (all of joe, bob and alice) will be selected. So it does not provide a solution to your need.

Inspecting current_cases: half of the solution

We can use the new current_cases fixture to see which cases are used by the current test node:

@parametrize_with_cases('val', cases=MyClassName)
def test_function(val, current_cases):
    print(val)
    case_id, case_func = current_cases['val']
    print((case_id, case_func))

Yields

joe
('widget', <function MyClassName.case_widget at 0x000001FE3878CEE0>)
bob
('widget', <function MyClassName.case_widget at 0x000001FE3878CEE0>)
alice
('widget', <function MyClassName.case_widget at 0x000001FE3878CEE0>)
foo
('foo', <function MyClassName.case_foo at 0x000001DD74756E50>)

(note: while writing this I found issue #212 so if you do not have the exact same output you'll need to update to version 3.5.2)

Unfortunately this does not fully answer the question, since this fixture only tells you if widget or foo is used, not the parameter of widget. At least it provides half of the response by helping you to detect that widget is active.

Inspecting request: the solution

request is the pytest fixture providing all information about the current test node.
There are two ways that you can use it to solve your problem:

• request.node.name provides you with the current node name as a string. You can therefore write:

@parametrize_with_cases('val', cases=MyClassName)
def test_function(val, request):
    print(val)
    if request.node.name == 'test_function[widget-name=joe]':
        pytest.skip("joe is skipped !")
    ...

• request.node.callspec.params is a dictionary that contains all parameter values set for the test, including parametrized fixtures. You can use it to get the parameter value of the widget case fixture that is generated behind the scenes:

@parametrize_with_cases('val', cases=MyClassName)
def test_function(val, request, current_cases):
    print(val)

    # get the current case
    case_id, case_func = current_cases['val']

    # skip if the case is widget and its param is "joe"
    if case_func is MyClassName.case_widget:
        if request.node.callspec.params['widget'].argvalues[0] == "joe":
            pytest.skip("joe is skipped !")
    ...

As you can see, this is a bit dirty but at least that's two workarounds for you.

In the future we may wish to modify the current_cases fixture so that each entry is a tuple containing 3 things instead of two: (case_id, case_func, case_param or None)

What do you think ?

[wonboyn]

Thanks for getting back to me.

In the short term I've gone with the option of evaluating each case in the test function & skipping if not appropriate.
It works but is a little cludgy.

I guess the better longer term option is a custom function to filter the results currently being parametrized.

[smarie]

Thanks @wonboyn for the feedback!

[smarie]

Hi @wonboyn , just for information: #214 has now been fixed in 3.6.0 so you can now solve your problem easily with the current_cases fixture alone:

@parametrize_with_cases('val', cases=MyClassName)
def test_function(val, current_cases):
    print(val)

    # get the current case and its parameters
    val_id, val_func, val_params = current_cases['val']

    # skip if the case is widget and its param is "joe"
    if val_func is MyClassName.case_widget and val_params["name"] == "joe":
        pytest.skip("joe is skipped !")
    ...

Let me know if this works ok for you